Social robotics: Beyond the uncanny valley

December 29, 2011 by Stuart Mason Dambrot, feature

The Uncanny Valley. Courtesy: Masahiro Mori, Karl F. MacDorman, Takashi Minato
( -- From science fiction and academia through assembly lines and telemedicine, robots have become both conceptually and physically ubiquitous. Technologically, robotics technology has advanced dramatically since the time of their namesake introduction in R.U.R. (Rossum's Universal Robots), a 1920 Czech-language science fiction (which nonetheless was conceptually quite visionary, since the robots it depicted were biological, and therefore essentially synthetic humans) in which robot was the English version of robota, meaning forced labor, in turn derived from rab, or slave. Today’s virtual and physical robots, however – imbued with artificial intelligence, artificial muscles, vision and pattern recognition, speech recognition and synthesis, sensors and actuators, and increasingly sophisticated interactivity – seem to be approaching those envisioned in Isaac Asimov’s seminal work I, Robot (but still from their human-level-and-beyond artificial intelligence, and certainly nowhere near the living robots envisioned in R.U.R.) That said, however, something’s still glaringly missing – namely, the ability to seamlessly interact with humans and other robots in a spontaneous, natural way that does not rely exclusively on specific preprogrammed behaviors. This is far more difficult than it seems, owing largely to the challenge of computationally emulating evolutionarily-determined perceptually-and emotionally-mediated contextual engagement. Enter Social Robotics: the effort to make robots more…well, sociable.

Social Robotics has its roots in the mid-20th century work of William Grey Walter, a neurophysiologist and roboticist who constructed autonomous electronic robots to demonstrate that complex behavior could arise from robust connectivity between just a few neurons. As robots became more sophisticated and animations more realistic, it was found that our empathy for these human analogues grew with their similarity to ourselves. But there’s a catch: As robots become increasingly humanoid in appearance and behavior past a certain point, a phenomenon known as the uncanny valley emerges.

A phrase introduced in 1970 by robotics professor Masahiro Mori, the uncanny valley is best described as the reaction we have to robotic appearance or behavior when it is perceived as almost human. The gap between barely human and fully human leaves us feeling uneasy as a result of the way evolution has shaped our brains when perceiving familiarity – especially that of anthropomorphic forms. As Mori wrote in his original paper about a prosthetic hand that is lifelike o the eye but not to the touch (and as translated by Karl F. MacDorman and Takashi Minato), “In mathematical terms, strangeness can be represented by negative familiarity, so the prosthetic hand is at the bottom of the valley. So in this case, the appearance is quite human like, but the familiarity is negative. This is the uncanny valley.

This experience of strangeness or even revulsion in turn prevents us from experiencing empathy for the robot or animation – and empathy is essential for optimal human/robot social interactions. In terms of robot behavior, this means that social robots must successfully operate with the complex web of societal rules that humans learn primarily through implicit experience rather than explicit programming.

At a recent New York Academy of Sciences event, Familiar but Strange: Exploring our Relationships with Robots, two exceptional speakers – roboticist extraordinaire Heather Knight and motion capture, computer vision and animation wizard Christoph Bregler – explored this mysterious space in considerable depth.

Geminoid Research. Copyright © Hiroshi Ishiguro Laboratory, ATR

Heather Knight – currently conducting her doctoral research at Carnegie Mellon's Robotics Institute – approaches social robotics in innovative ways. Founder of Marilyn Monrobot Labs in New York City, which creates socially intelligent robot performances and sensor-based electronic art, one of her more engaging projects is the integration of robot design and theater – as demonstrated by her creation and production of Robot Film Festival, which took place in New York City on July 16-17, 2010. (The next Robot Film Festival is scheduled for 2012; submissions will be accepted for consideration starting in January.)

“As we go about designing our future,” Knight says, “some of the things I think about are creating intelligent machines, building relatable robotic characters, and creating companions that can exist and help us in our everyday lives.” Knight sees the integration of robotics and theater and a key pathway to that future.

”Robots and theater have a long history together,” Knight points out – a history beyond the legacy of R.U.R. “Artists tend to use the medium of their times – and right now, that medium is technology.” Moreover, she observes, making robots responsive to their audience, as well as applying physical theater techniques through gestural communications, allows emotion to infuse the entire robotic form. Knight illustrates this point with the example of look-step-reach-grab behavior: Even without a word being spoken, every such sequence and its associated timing profile embodies and communicates a robust amount of specific but distinct intentional, emotional and cognitive meaning.

However, Knight – whose previous work includes robotics and instrumentation at NASA's Jet Propulsion Laboratory, interactive installations with Syyn Labs, field applications and sensor design at Aldebaran Robotics, as well as being an alumnus of MIT Media Lab’s Personal Robots Group – cautions that it’s not a matter of simply taking existing templates already successful in acting and embedding them in robots. “I think actual collaboration – the procedural knowledge of working with performance, performers and directors – is really important, because not everything can be codified in words.”

When discussing the uncanny valley, Knight cites how culture influences not only how we react to a perceived as being almost human, but also the way we represent robots and other technology in the world, as well as how we select the types of robotic research we fund and pursue. “I think that if we create modern narratives, perhaps we can reshape some of the current emphasis of where we’re going with technology.”

Chris Bregler, Associate Professor of Computer Science at NYU's Courant Institute and director of the NYU Movement Lab, inhabits the world of animation and entertainment, focusing on Hollywood robots – virtual characters and actors without physical embodiment. Conducting interdisciplinary research in the virtual world of motion capture, animation, computer vision, graphics, statistical learning, gaming, biomedical applications, human-computer interaction, and artificial intelligence – work that has resulted in numerous publications, patents, and awards from the National Science Foundation, Sloan Foundation, Packard Foundation, Electronic Arts, Microsoft, Google, U.S. Navy, U.S. Air Force, and other sources - he has a visually demonstrable take on the uncanny valley.

Social Robotics: Beyond the Uncanny Valley
Final Fantasy image. Copyright © 2001 FFFP All rights reserved.
“Hollywood robots were previously played by human actors,” Bregler notes, commenting on the film AI: Artificial Intelligence. “Over the last 10 years, however, there’s been a huge revolution in Hollywood.” Today those same Hollywood robots would be played by virtual robots due to advances in animation, special effects, and live action – and, he adds, “Everything changes every year.”

For Bregler, the uncanny valley is also evident in motion capture as applied to virtual actors, such as the animated characters in Final Fantasy which have achieved astounding realism as stationary images – but all familiarity is lost once there’s facial movement. Due to technological limitations, body and face motion capture cannot reproduce subtle small-scale movements that communicate essential real-world information. Without these present, viewers immediately enter the uncanny valley.

Bregler points out another problem with virtual characters: There’s no weight or force in their virtual body or captured movements. “When you do animation, there are reverse kinematics – a bit like a puppet.” In other words, the force is externally applied rather than being an intrinsic component of the character’s physicality. This was perfectly fine with the aliens in Avatar, Bregler illustrates, but if the same performance was applied to people, the illusion would have dissolved – and the same holds true for more ephemeral issues, such as the way evolution has enabled us to tell, for example, if someone may be lying from facial, gestural and tonal cues we may not be explicitly aware of (familiarity) until they’re absent (uncanny valley). Surprisingly, even higher primates have an uncanny valley regarding others of their species.

Moreover, he adds that “The uncanny valley is now a common problem everywhere – In the game industry, in the motion capture industry – and while there are movies that dance around the , others fall right into it.” Bregler sees The Incredibles, which was hand-animated, as being in the former category – but the precisely motion captured Polar Express in the latter.

“We’re not there yet with primary human motion and behavior,” he explains. “We’re also very far away from being able to simulate the human brain – even the spinal cord. We need a shortcut.”

Explore further: Scientists striving to put a human face on the robot generation

More information: Masahiro Mori, The Uncanny Valley. Energy, 7(4), pp. 33-35, 1970, Translated by Karl F. MacDorman and Takashi Minato

Toward Social Mechanisms of Android Science, Vancouver, Canada, 26 July 2006

Heather Knight, Eight Lessons Learned about Non-verbal Interactions through Robot Theater. SOCIAL ROBOTICS: Lecture Notes in Computer Science, 2011, Volume 7072/2011, 42-51, DOI: 10.1007/978-3-642-25504-5_5

Heather Knight: Silicon-based comedy. TED Initiatives, TED Women, December 2010

Christoph Bregler, Next Generation Motion Capture: From the Silver Screen to the Stadium. MIT Sloan Sports Analytics Conference, March 2-3, 2011

Christoph Bregler et al., Squidball: An Experiment in Large-Scale Motion Capture and Game Design. Intelligent Technologies for Interactive Entertainment, Lecture Notes in Computer Science, 2005, Volume 3814/2005, 23-33, DOI: 10.1007/11590323_3


Related Stories

When robots learn from our mistakes

May 26, 2011

( -- Robots typically acquire new capacities by imitation. Now, EPFL scientists are doing the inverse -- developing machines that can learn more rapidly and outperform humans by starting from failed or inaccurate ...

Your brain on androids

July 14, 2011

Ever get the heebie-jeebies at a wax museum? Feel uneasy with an anthropomorphic robot? What about playing a video game or watching an animated movie, where the human characters are pretty realistic but just not quite right ...

High-precision robots available in kit form

June 17, 2011

( -- A doctoral student from EPFL's Laboratory of Robotics Systems has developed a concept for modular industrial robots, based on the technology of parallel robots, whose precision is expressed in nanometers.

Robotic dragon, an unlikely teacher

October 21, 2011

David DeSteno, an associate professor of psychology at Northeastern, and researchers from the Massachusetts Institute of Technology (MIT) and Harvard University, are examining how social robots can aid preschoolers in language ...

Driving a robot from Space Station

June 30, 2011

( -- Meet Justin, an android who will soon be controlled remotely by the astronauts in ESA’s Columbus laboratory on the International Space Station. With this and other intriguing experiments like the Eurobot ...

Recommended for you

Female golden snub-nosed monkeys share nursing of young

February 21, 2019

An international team of researchers including The University of Western Australia and China's Central South University of Forestry and Technology has discovered that female golden snub-nosed monkeys in China are happy to ...

When does one of the central ideas in economics work?

February 20, 2019

The concept of equilibrium is one of the most central ideas in economics. It is one of the core assumptions in the vast majority of economic models, including models used by policymakers on issues ranging from monetary policy ...

In colliding galaxies, a pipsqueak shines bright

February 20, 2019

In the nearby Whirlpool galaxy and its companion galaxy, M51b, two supermassive black holes heat up and devour surrounding material. These two monsters should be the most luminous X-ray sources in sight, but a new study using ...

Research reveals why the zebra got its stripes

February 20, 2019

Why do zebras have stripes? A study published in PLOS ONE today takes us another step closer to answering this puzzling question and to understanding how stripes actually work.


Adjust slider to filter visible comments by rank

Display comments: newest first

not rated yet Dec 29, 2011
Enough with "Polar Express" as example. It's from 2004.
3 / 5 (2) Dec 29, 2011
from the article:
Technologically, of course, robots have advanced dramatically since their namesake introduction in R.U.R. (Rossum's Universal Robots)
Mr. Dambrot has not read the play, evidently.
Rossum's robots were, in their own way, *more* technologically advanced than today's specimens, since they were biologic constructs, not merely mechanical. Also, they had human-level intelligence (being artificial humans, after all), unlike even the best of today's robots.
not rated yet Dec 29, 2011
Rossum's robots were, in their own way, *more* technologically advanced than today's specimens....

R.U.R. was fiction, Silverhill. Science fiction imagines technology that scientists and engineers then aspire to instantiate. Yes, in concept the robots depicted in R.U.R. were more advanced than today's, but were merely symbolic. If you want to equate those fictional robots with today's actual devices, however, do so at your own metaphorical peril. For example: Could a Star Trek Communicator - which Motorola engineers acknowledged provided a template for their first folding cellular telephones - be described as more advanced than those early cell phones? Again, only in concept. The key point to notice is that any real robot of sufficient sophistication would be more advanced by virtue of its physical existence. Too subtle a point, perhaps....
not rated yet Dec 30, 2011
R.U.R. was fiction, Silverhill.
This had not escaped me, be assured.
Phrase it this way, if you insist:
"Rossum's robots, _were they possible with today's technology, which they are not, would be more advanced than what we have now._ Therefore to say that today's tech is more advanced than what Capek's (hypothetic, not-yet-realizable) tech would be, is careless at least."

(Addressed at the *conceptual* level, various things in fiction are more advanced than the concept behind some real thing.)

Too subtle a point, perhaps....
Or too capable of non-congruent interpretations, perhaps....
1 / 5 (1) Dec 30, 2011
...were they possible with today's technology, which they are not, would be more advanced than what we have now..

Exactly. Technology is the instantiation of concept. I take your point (how could I not?) but find your conflation of idea and reality, coupled with your arrogance, more revealing than your insight.I therefore dismiss your inevitable dismissive condescension and wish you well.
not rated yet Dec 30, 2011
Technology is the instantiation of concept.
Already well known.

I take your point (how could I not?)
Thank you.

your conflation of idea and reality
I do not conflate them, sir/madam, I *compare* them. Note my use of "if..." statements.

your arrogance
WTF? I amplified my thought, that you and others might take my meaning more easily, since I did not wish to be misinterpreted. And the "thanks" I get, for this effort, is your contumely. Be ashamed. (Be also not so quick to be dismissive; be open to clarificative communication, as am I.)
not rated yet Dec 30, 2011
from the article:
Technologically, of course, robots have advanced dramatically since their namesake introduction in R.U.R. (Rossum's Universal Robots)
Mr. Dambrot has not read the play, evidently.
Rossum's robots were, in their own way, *more* technologically advanced than today's specimens, since they were biologic constructs, not merely mechanical. Also, they had human-level intelligence (being artificial humans, after all), unlike even the best of today's robots.

It was just the introduction of the word "robot" but today we call biologic constructs "androids" which are diferent from "robots", so Rossum's robots weren't robots but androids ;)
3 / 5 (2) Dec 31, 2011
Well, they were android robots, to use the fullest description. An android is just a construct that resembles a human, whereas "robot" can be applied to any servitor construct, regardless of its shape. :-)
1 / 5 (1) Jan 04, 2012
Enough with "Polar Express" as example. It's from 2004.

Or Final Fantasy from 2001 :)

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.