The future of TV? How feely-vision could tickle all our senses

February 8, 2016 by Marianna Obrist, University Of Sussex, The Conversation
Credit: Shutterstock

Imagine a party on a warm summer's evening. You can see the beautiful greenery and the dipping sun, you can smell the freshly cut grass and taste the cool drinks on offer. You hear someone walk up behind you and feel them tap you on the shoulder. Now imagine you're not really at the party – but sat at home and the scene and all these sensations are coming from your TV.

Working out how television programmes could one day stimulate all our senses is an interesting question for researchers like myself, who are exploring the future of TV. But the bigger, more exciting challenge is how we can not only imitate what is happening on the screen, but also use smell, taste and touch in a way that's not a novelty and enhances the emotional experience of a show, just as a soundtrack does.

There's good reason to think about how the TV industry can innovate in this way. Despite the rise of online video, millions if not billions of people still watch traditional broadcast media through television sets. TV remains a powerful format for programme making and watching that follows specific restrictions and guidelines.

But more people are watching TV programmes online after their original broadcast, on other devices such as tablets and phones, are even using multiple screens to engage with more than one piece of content at once. Broadcasters need to create new ways of experiencing TV that capture the audience's full attention and immerse them in a multisensory world.

Experimenting with the senses

The video will load shortly

Creating truly compelling TV that stimulates all our senses is not an easy task. Programme makers and technology manufacturers know how to design their products so you can see depth and distance on the screen. But sound and vision aren't always enough. Being able to smell the odours that a character on screen would smell, or feel the objects or atmosphere they would feel, can create anticipation and build suspense in the same way as sound currently does.

Cinema is already experimenting with these extra senses. Films with touch and smell sensations can be experienced in newly equipped 4DX cinemas, such as the one in Milton Keynes. The sense of taste seems a final frontier for technology development, but the interest in taste experiences has started to take off. For example, audience members at Edible Cinema each receive a package of food and drinks to match what characters on screen experience.

The question for the TV industry is what multisensory experiences should it design for – and how. My Sussex Computer Human Interaction lab is trying to better understand how we use our senses so that designers and developers can help us interact with their technology in the most compelling way possible.

Our latest work focuses on cutting edge technology such as the mid-air touch feedback or "haptic" device developed by Ultrahaptics, a start-up in Bristol. We're looking at how this technology could evoke emotions in the audience by allowing them to feel physical sensations without touching actual objects.

For example, projecting a pattern of ultrasound beams onto your hand can create different tactile sensations, such as a feeling of raindrops on your palm (without the water), or a flow of air as if you were holding your hand out of the window of a moving car. When carefully designed, this haptic feedback can produce even more specific patterns that allow you to feel different shapes, that change in size or that quickly move around.

Emotional feedback

By experimenting with different shapes, we've studied how this kind of haptic feedback can produce different emotions. We found that short, sharp bursts of air to the area around the thumb, index finger and middle part of the palm generate excitement. Slow and moderate stimulation of the outer palm and the area around the little finger create sad feelings.

This gives us a starting point to find out how mid-air touch sensations could be meaningfully integrated into other experiences, such as watching a movie. One challenge will be to make enhance the viewing experience without seeming intrusive or creepy, as suggested by "the feelies" cinema experience portrayed in the dystopian novel Brave New World.

We've recently began a five-year project to expand the research into taste and smell, as well as touch. The SenseX project will aim to provide guidelines and tools on how to design and integrate sensory stimuli for inventors and innovators to create richer interactive experiences. Relatively soon, we may be able to realise truly compelling and multi-faceted media experiences, such as 9-dimensional TV (adding tastes on top of 4DX), that evoke emotions through all our senses.

Explore further: Technology can transfer human emotions to your palm through air, say scientists

Related Stories

UltraHaptics—it's magic in the air (w/ Video)

October 7, 2013

A system that allows users to experience multi-point haptic feedback above an interactive surface without having to touch or hold any device will be unveiled this week [Friday 11 October] at one of the world's most important ...

Recommended for you

Computer learns to recognize sounds by watching video

December 1, 2016

In recent years, computers have gotten remarkably good at recognizing speech and images: Think of the dictation software on most cellphones, or the algorithms that automatically identify people in photos posted to Facebook.

4 comments

Adjust slider to filter visible comments by rank

Display comments: newest first

mike74
not rated yet Feb 08, 2016
That's all very good, and exciting. But when a character in a film is being tortured, or suffering from a painful illness, I'd rather not experience that. So the question is, how would future filmmakers handle giving people these new non-visual experiences through most of their movie, but when it comes to certain moments, have to rely on just visual effects to tell that part of the story. Won't those parts feel less 'real', because they lack the bits that we wouldn't necessarily want to experience?
That said, this is something I'm looking forward to. And hopefully it will be useful for other platforms, too, like VR.
antialias_physorg
1 / 5 (1) Feb 08, 2016
From a tech stanpoint I can see it. But when you check on reality (HD-TV, 3D, high def audi, ... ) none of these have made a difference in the actual enjoyment of TV (or movies).
In the end it's the content that matters - and content has been very much the same over the past decades.

But when a character in a film is being tortured, or suffering from a painful illness, I'd rather not experience that.

Why not? I could think of no more powerful teaching tool to get people to reject torture (or war, or racism, or bullying, or any other kind of psychological trauma one human experiences at the hand of another). Currently waging war and torturing people is very abstract. It's a long way off. It's something we can ignore or pretend that "it really isn't that bad" or only happens to a few people.

Get kids to live through a full emotional version of Schindler's List and see how many grow up to be racists (might plan some therapy, though)
pntaylor
not rated yet Feb 08, 2016
I can see this working for VR sex but that's it. Anything else, some one would actually want to experience, would be too boring to watch. This is only good for novelty entertainment, like flying suit sequences and the like. Maybe some hardcore gamers, too.
antialias_physorg
1 / 5 (1) Feb 08, 2016
Anything else, some one would actually want to experience, would be too boring to watch.

Might be good for training soldiers. Show them images of scenes where killings happen (or just plain obeying of orders) and give them an emotional high.
Yeah. I can see any number of military organisations being all over this.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.