Robot as social interface (w/ Video)

Human interaction with robots is about to get a little more personal. Meet "Furhat," the face of tomorrow's interactive technology.

An increasingly important – and sometimes frustrating – part of daily life is dealing with so-called "user interfaces". Whether it's a smartphone or an airport check-in system, the user's ability to get what they want out of the machine relies on their own adaptability to unfamiliar interfaces.

But what if you could simply talk to a machine the way you talk to a human being? And what if the machine could also ask you questions, or even address two different people at once?

These kinds of interactive abilities are being developed at KTH Royal Institute of Technology with the help of an award-winning robotic head that takes its name from the fur hat it wears.

With a computer-generated, animated face that is rear-projected on a 3D mask, Furhat is actually a platform for testing various interactive technologies, such as speech synthesis, speech recognition and eye-tracking. The robot can conduct conversations with multiple people, turning its head and looking each person straight in the eye, while moving its animated lips in synch with its words.

Samer Al Moubayed, one of the KTH researchers behind the development of Furhat, says the project represents the third generation of spoken dialogue systems that has been in development at KTH's Department for Speech, Music and Hearing during the last 15 years.

Interest in the robot extends far beyond its home at KTH's Stockholm campus. Furhat won an outstanding demonstration award at the ICMI conference in Los Angeles in 2012, and has been featured at robotics conventions in the U.K., France and Portugal. In addition, Al Moubayed received the Christian Benoit Award, a cash prize of EUR 7,500 awarded to promote promising research in the field of speech communication. Furhat also was named runner up for the international Robotdalen Innovation Award in 2013.

The Furhat team aims to develop its technology for commercial use, with the help of funding from Sweden's Vinnova, a government agency that supports innovation projects.The team behind the development of Furhat also includes: Jonas Beskow, Joakim Gustafson, and Gabriel Skantze.

"Furhat is becoming a popular research platform for scientists around the world who study human interaction with machines," Al Moubayed says. "It's very simple, it's potentially very cheap to make, and people want to use it in their own research areas."

Furhat also has attracted attention from researchers at Microsoft and Disney. "They have been following the development of Furhat for a long time," says Al Moubayed, who returned last week from a two-week visit to the U.S., where he presented the hardware and software behind Furhat at Microsoft's and Disney's research labs.

Al Moubayed says that Furhat has potential as an interactive user interface for a variety of applications. In schools, it could be used to conduct knowledge games with children. In assisted living centres, it could share information and chat with people.

Perched on a stand atop a bureau in Al Moubayed's office, Furhat greets a visitor and asks their name. It then addresses Al Moubayed, and after a couple more questions, asks if the visitor has any questions for Furhat. During a pause in the exchange, the robot offers to tell a joke.

Part of what sets Furhat apart from other interactive robots is its ability to make not only conversation, but eye contact – an important element of communication.

"You want an interface that fulfills or reaches a critical quality that people can interact with in a natural way, otherwise the interaction you get is not natural anymore, and does not resemble how people interact with each other."

Furhat's ability to turn its face to multiple people in a conversation is enabled by face-tracking software. But its ability to make eye contact is achieved through projection. Unlike a 2D image, which can appear to be looking at everybody in the room at once – a phenomenon known as the "Mona Lisa effect" – Furhat appears to shift its gaze because the face is projected onto 3D-printed model of a human face.

"When we first experimented with this, the effect was strong immediately," Al Moubayed says. "You could bond with, or relate to, the face.

"It is an avatar that can really be present in the physical environment."

Such technologies are being explored as a potential therapeutic tool for children with autism and other disorders that affect social interaction, he says. The technology can also be used for telepresence applications in which 3D replicas of people's faces become the screens that we look at when conducting a video conference call.

"There are many different types of interest that drive this work," he says. "And we're just starting to explore its potential."