Mask-bot: A robot with a human face

Nov 07, 2011
Dr. Takaaki Kuratate and his robot communication interface Mask-Bot Credit: Uli Benz / Technical University of Munich

Robotics researchers in Munich, Germany, have joined forces with Japanese scientists to develop an ingenious technical solution that gives robots a human face. By using a projector to beam the 3D image of a face onto the back of a plastic mask, and a computer to control voice and facial expressions, the researchers have succeeded in creating Mask-bot, a startlingly human-like plastic head.

Mask-bot can already reproduce simple dialog. When Dr. Takaaki Kuratate says "rainbow", for example, Mask-bot flutters its and responds with an astoundingly elaborate sentence on the subject: "When the strikes in the air, they act like a prism and form a rainbow". And when it talks, Mask-bot also moves its head a little and raises its to create a knowledgeable impression.

What at first looks deceptively like a real talking person is actually the prototype of a new face that a team at the Institute for Cognitive Systems (ICS) at TU München has developed in collaboration with a group in Japan. "Mask-bot will influence the way in which we humans communicate with robots in the future," predicts Prof. Gordon Cheng, head of the ICS team. The researchers developed several innovations in the course of creating Mask-bot.

The projection of any number of realistic 3D is one of these. Although other groups have also developed three-dimensional heads, these display a more cartoon-like style. Mask-bot, however, can display realistic three-dimensional heads on a transparent plastic mask, and can change the face on-demand. A projector positioned behind the mask accurately beams a onto the back of the mask, creating very realistic features that can be seen from various angles, including the side.

This video is not supported by your browser at this time.

Many comparable systems project faces onto the front of a mask – following the same concept as cinema projection. "Walt Disney was a pioneer in this field back in the 1960s," explains Kuratate. "He made the installations in his Haunted Mansion by projecting the faces of grimacing actors onto busts." Whereas Walt Disney projected images from the front, the makers of Mask-bot use on-board rear projection to ensure a seamless face-to-face interaction.

This means that there is only a twelve centimeter gap between the high-compression, x0.25 fish-eye lens with a macro adapter and the face mask. The CoTeSys team therefore had to ensure that an entire face could actually be beamed onto the mask at this short distance. Mask-bot is also bright enough to function in daylight thanks to a particularly strong and small projector and a coating of luminous paint sprayed on the inside of the plastic mask. "You don't have to keep Mask-bot behind closed curtains," laughs Kuratate.

This part of the new system could soon be deployed in video conferences. "Usually, participants are shown on screen. With Mask-bot, however, you can create a realistic replica of a person that actually sits and speaks with you at the conference table. You can use a generic mask for male and female, or you can provide a custom-made mask for each person," explains Takaaki Kuratate.

In order to be used as a face, Mask-bot must be able to function without requiring a video image of the person speaking. A new program already enables the system to convert a normal two-dimensional photograph into a correctly proportioned projection for a three-dimensional mask. Further algorithms provide the and voice.

To replicate facial expressions, Takaaki Kuratate developed a talking head animation engine – a system in which a computer filters an extensive series of face motion data from people collected by a motion capture system and selects the facial expressions that best match a specific sound, called a phoneme, when it is being spoken. The computer extracts a set of facial coordinates from each of these expressions, which it can then assign to any new face, thus bringing it to life. Emotion synthesis software delivers the visible emotional nuances that indicate, for example, when someone is happy, sad or angry.

Mask-bot can realistically reproduce content typed via a keyboard – in English, Japanese and soon German. A powerful text-to-speech system converts text to audio signals, producing a female or male voice, which can then set to quiet or loud, happy or sad, all at the touch of a button.

But Mask-bot is not yet able to understand much of the spoken word. It can currently only listen and make appropriate responses as part of a fixed programming sequence. However, this is not something that is restricted to Mask-bot alone. A huge amount of research worldwide is still being channeled into the development of programs that enable robots to listen and understand humans in real time and make appropriate responses. "Our established international collaboration is therefore more important than ever," explains Kuratate.

The CoTeSys group has chosen a different route to researchers who use dozens of small motors to manipulate individual facial parts and reproduce facial expressions. Mask-bot will be able to combine expression and voice much faster than slower mechanical faces. The Munich researchers are already working on the next generation. Mask-bot 2 will see the mask, and computer control system all contained inside a mobile robot. Whereas the first prototype cost just under EUR 3,000, costs for the successor model should come in at around EUR 400. Besides video conferencing, "These systems could soon be used as companions for older people who spend a lot of time on their own," continues Kuratate.

Explore further: Robot scouts rooms people can't enter

Provided by Technische Universitaet Muenchen

4.1 /5 (7 votes)

Related Stories

3,000-year-old mask found in Aleutians

Jul 29, 2007

A whalebone mask discovered at an Alaskan archaeological site was probably broken during an Aleut funeral 3,000 years ago, scientists said.

The moaning mouth bot learns to sing (w/ video)

Jul 13, 2011

(PhysOrg.com) -- Some of you may recall the moaning mouth bot. This bot, which is basically a disembodied mouth that first came to the public attention for making a series of moaning noises that were truly ...

Recommended for you

Robot scouts rooms people can't enter

Apr 23, 2014

(Phys.org) —Firefighters, police officers and military personnel are often required to enter rooms with little information about what dangers might lie behind the door. A group of engineering students at ...

A robot dives into search for Malaysian Airlines flight

Apr 18, 2014

In the hunt for signs of Malaysian Airlines flight MH370—which disappeared on March 8 after deviating for unknown reasons from its scheduled flight path—all eyes today turn to a company that got its start ...

User comments : 5

Adjust slider to filter visible comments by rank

Display comments: newest first

Isaacsname
not rated yet Nov 07, 2011
I could see reconstructive surgeons or plastic surgery practioners finding this handy, otherwise, it seems to be somewhat lacking in application ? Idk, maybe it's just me..
GrinningGhost
5 / 5 (2) Nov 07, 2011
While Disney did invent the original front-projection version for the Haunted Mansion attraction in 1969, the writer failed to mention how it has been upgraded over the decades. In the 80s, the Madame Leota character was projected internally via fiber optics, which allowed the table her crystal ball is mounted on to float about some. The image was inferior, and the fiber optics kept breaking, so in the 90s, she was back to a static table and front projection. However, in 2005, an entirely new internal projection system was created - within the head form - which now allows the character to float around the room inside of her crystal ball. Six years before this!
Deesky
5 / 5 (1) Nov 07, 2011
But Mask-bot is not yet able to understand much of the spoken word. It can currently only listen and make appropriate responses as part of a fixed programming sequence

Well, those are two different fields - visual aesthetics vs AI. AI can proceed on its own track and need not be linked with this display tech. I think its best immediate application would be as a tele-operated avatar, as mentioned in the article (though it might appear a bit creepy).
BenjaminButton
not rated yet Nov 08, 2011
This is totally behind the times and unworthy of a news spot. There's no AI involved, it's just a rear projected face, you could do that at home.
antialias_physorg
not rated yet Nov 08, 2011
There's no AI involved,

And the article does not claim that there is. So what's the problem?

This is about making robots more interaction friendly. Instead of metal eyes and tubes this gives you a much more 'human' interaction which is very important for people who don't understand the limitations of technology (e.g. if you want to use robots in health care or in support of the elderly).

Unfortunately the way it looks now it falls squarely into the nucanny valley.

More news stories

SK Hynix posts Q1 surge in net profit

South Korea's SK Hynix Inc said Thursday its first-quarter net profit surged nearly 350 percent from the previous year on a spike in sales of PC memory chips.

FCC to propose pay-for-priority Internet standards

The Federal Communications Commission is set to propose new open Internet rules that would allow content companies to pay for faster delivery over the so-called "last mile" connection to people's homes.

Brazil enacts Internet 'Bill of Rights'

Brazil's president signed into law on Wednesday a "Bill of Rights" for the digital age that aims to protect online privacy and promote the Internet as a public utility by barring telecommunications companies ...

Phase transiting to a new quantum universe

(Phys.org) —Recent insight and discovery of a new class of quantum transition opens the way for a whole new subfield of materials physics and quantum technologies.

Imaging turns a corner

(Phys.org) —Scientists have developed a new microscope which enables a dramatically improved view of biological cells.

Fresh hope for preventing pneumonia in the elderly

There are calls for the frail and elderly not be be overlooked for vaccines against pneumonia this winter, with UNSW research challenging conventional wisdom on immunisation effectiveness in older patients.