Sssshhhhh - my show is on
This weekend, the first episode of Mosaic will be shown in the UK. If you’ve not heard of it, Mosaic is a six-part murder mystery series from Steven Soderbergh, who you probably know as the director of Magic Mike, Erin Brockovich, and Oceans Eleven (plus its numerically progressive sequels). So far, so standard.
But if you live in the US, Mosaic has already been available since November as an interactive app for smartphones, tablets and Apple TV, allowing viewers to choose which characters and narrative threads they follow, rather than the director and editor assembling the pieces for them.
In its more traditional telly form Mosaic is shorn of its interactivity, but it’s one of many signs that media companies are starting to experiment with the way they tell stories – and the smart home will be at the centre of it.
Choose your own adventure
Last summer Netflix released Puss in Book: Trapped in an Epic Tale and Buddy Thunderstruck: The Maybe Pile, two one-off shows that periodically ask the viewer to choose between pairs of on-screen options to advance the story. As the names suggest, these titles are clearly for kids but the streaming giant has stated its intentions to make something similarly interactive that’s aimed at adults.
Read next: The best visual skills for Amazon Echo Show and Echo Spot
These ‘choose your own adventure stories’ (or branching narrative if you want to use the technical term) have been around for yonks, usually in the form of a book that would instruct you to turn to different pages depending on what choice you made, or basic CD-ROM games that you might’ve played back in the ‘90s, but touchscreen devices and on-demand streaming have given them a new lease of life.
Netflix’s Puss in Book: Trapped in an Epic Tale
As you might expect from stories about talking animals (at least those not directed by Wes Anderson), neither of Netflix’s interactive stories are particularly complex. The choices you make don’t affect the subsequent options given to you, so possible permutations are limited, but they’re well implemented on a phone or tablet. Watch on a smart TV and you have to use menus a bit like the pop-up ones you’d get on a Blu-ray disc.
“The children’s programming space was a natural place for us to start since kids are eager to ‘play’ with their favorite characters and already inclined to tap, touch and swipe at screens,” Netflix’s director of product innovation, Carla Engelbrecht Fisher, wrote in a blog post announcing the shows’ availability. “They also talk to their screens, as though the characters can hear them. Now, that conversation can be two-way.”
Engelbrecht Fisher isn’t referring to voice control but TV is changing and so are remotes. Since Amazon added Alexa to its Fire TV Stick in 2016 it has been possible to search for stuff to watch simply by asking.
Google has voice controls built into the remotes and homescreens of Android TVs from Sony, Philips and Sharp. The cream of Sony and Hisense’s 2018 TV ranges will also come with Alexa support, while LG’s will have Google Assistant as well. These will allow you to alter settings, chance sources or adjust the volume using your voice, but the possibilities for it in entertainment are much more exciting than simple tweaking.
Even Sky’s budget Now TV Smart Stick allows you to search for shows with your voice, rather than painstakingly tapping out each letter on the remote. We’re becoming much more comfortable with ‘voice first’ devices, particularly in our homes – and your TV is next.
Branch management
Late last year the BBC’s R&D department released The Inspection Chamber – a radio play for the PlayStation generation that owes debts to sci-fi classics such as Kubrick’s 2001: A Space Odyssey, The Hitchhiker’s Guide to the Galaxy and Valve’s Portal games.
Made in collaboration with audio experts Rosina Sound, the 20-minute ‘interactive voice drama’ is currently only available for Amazon’s Echo devices because the listener has to take on the role of one of its characters, an alien being of unknown origin that’s under interrogation from two humans and an AI called Dave – how very meta. As the story unfolds you’re asked questions that must be answered before the story can progress.
“Branching paths are never really very satisfying as pieces of entertainment because they often put you in the position of having to be the director rather than a participant,” Henry Cooke, a senior producer from BBC R&D, explains to The Ambient. “That can be complex, overwhelming and not much fun to work through, so the interaction model we picked was less branching path and more immersive theatre.”
The BBC’s The Inspection Chamber
That means a fairly linear timeline, with a universal start point and a small number of branches that all converge back in the middle before one of a few endings. “The details of the experience change but the overall narrative doesn’t,” says Cooke. “That means the user gets to have agency, contribute to the story, and the characters respond to what you’re saying, but we can tell a better story.”
The Inspection Chamber does this in a number of ways but it starts with the fact that it’s controlled entirely by voice, which is a more natural, visceral form of interaction than typing on a keyboard or touching a screen. Having a display with a list of options might allow you to present more choices but it also feels more finite.
Of course, there are only a limited number of things you can say in response to The Inspection Chamber’s questions and it never feels like you’re having a monumental effect on the direction things are going in but it’s not entirely restricted to simple binary choices, which gives things a sense of possibility that doesn’t come across in the Netflix shows.
With these questions, though, things are not all as they seem. Any open-ended question it asks you has been specifically written to be answerable in any way; all it’s looking for is an audible response. “We don’t actually care what you say, we just give the impression that we do,” reveals Cooke. “It’s an old stage trick.” And unless you’re deliberately trying to fool it, chances are you’ll play along in blissful ignorance.
Within its fairly tight narrative framework it also uses call-backs to your previous answers, which only serve to enhance that sense of personal influence. Compared to Netflix’s A or B branches it feels like a much more immersive technique, even if you’re not really affecting how things turn out as much as you think.
“If you’re making decisions when being told a story it has to mean something, otherwise you’re just making arbitrary choices in order to digest another chunk of media,” Cooke says. So while he doesn’t see branching narratives as the future of TV, he is excited by the possibilities offered by voice-activated devices and smart home products.
He references Google Home’s Voice Match tech as a potential opening for multi-person experiences and refers to the regular habits that exist around smart speakers as a way of structuring short, episodic stories, a bit like the app he worked on before joining the BBC that was based around the idea that you’d found a phone belonging to The Thick of It’s spectacularly foul-mouthed Malcolm Tucker.
Combine that with the movement sensors or security cameras that probably exist somewhere in your smart home network and a device could even detect who was home and make sure each individual picks up the story from the right place.
“It’s not just about whether you interact with voice or via a screen, it’s about using the medium that makes the most sense,” Cooke says. “Something like the Echo Show points towards that.”
Passive resistance
The Inspection Chamber isn’t the only interactive drama available for the Echo.
Earplay is an Alexa skill that gives you access to a small library of voice-controlled stories, including a thriller that puts you in the shoes (or should that be vocal chords?) of a secret agent. You answer the occasional direct question but spend most of the time making decisions on behalf of your character, putting it somewhere between The Inspection Chamber’s conversational narrative and the traditional A/B branches in the Netflix shows.
Earplay’s chief creative officer is Dave Grossman, a veteran of point ‘n’ click games such as The Secret of Monkey Island and Telltale’s The Walking Dead series. “We’re trying to provide an experience you can engage with from across the room, or while your hands and eyes are engaged with a chore”, he tells The Ambient. “We want it to fit into the parts of your day that a podcast or an audiobook might.”
That doesn’t mean he thinks interactive storytelling doesn’t have a future on screens, though. “We like to provide something extra for people who do have an Echo Show,” Grossman says, “but we have to treat it as an optional part of the package, like doing illustrations for a book.
Like Cooke, he’s not convinced that interactivity is the future of TV either but feels it could establish its own niche – with a few changes. “Netflix’s Puss in Book is an overly simple and stilted example but if they were to double down on the design and frequency of interaction, it could become a compelling medium rather than a novelty,” he says. “And as it becomes a ubiquitous method for easy, intuitive control of all sorts of experiences, voice control is certainly likely to play a part.”
Steven Soderbergh’s Mosaic
Whether Netflix will extend the voice control offered via Alexa to its own interactive storytelling is yet to be seen, but if it does, the secret will be in finding a balance between too much and too little interaction.
“Because it’s a new medium people bring their expectations from various places,” explains Cooke. “Dyed-in-the-wool radio listeners disliked being asked questions so often and found it intrusive, but people who came to The Inspection Chamber from a gaming background didn’t think they were being asked questions often enough.”
For anyone who just wants to veg out in front of the TV, suddenly asking them to become an active participant in their evening’s entertainment might have little appeal, even if they can do it with nothing more than their voice. But for those looking for something a little more stimulating than an audiobook or podcast to take their mind off tackling the washing up, interactive stories could be the solution.
For everyone else, the smart home can still enhance storytelling with Philips’ Hue Entertainment experience, which syncs your smart lighting with what’s happening on screen. But why stop there? How about getting Nest to turn up the heating when you’re watching something set in the desert or making your Ring doorbell chime when the killer’s at the door during a slasher film?
Too far? Probably. But Earplay’s Grossman can imagine almost any controllable object in your home becoming part of a story, albeit with certain limitations. Firstly, not all homes will have the same arsenal of smart devices, meaning they could only ever augment the story rather than play an essential role in it, and secondly there are issues of privacy.
“You’d really have to trust me before you’d give me a list of all of your smart home devices and let me take them over as part of some cockamamie story I’m trying to involve you in,” he says. “Although I have always wanted to engineer some kind of deep philosophical discussion that you’d have with your microwave oven…”
