Robot knows who wants one for the road

Sep 24, 2013 by Nancy Owano report

(Phys.org) —JAMES has a head that is actually a tablet. JAMES is an efficient waiter yet only has one arm. JAMES can read your body language to know you want a drink without your saying a word. Perhaps the biggest surprise of all, however, is that the EU is funding this project. JAMES stands for The Joint Action in Multimodal Embodied Systems. The project started in 2011 and continues until January. Actually, its research and results will be helpful for scientists to know how robots can assist people in meaningful ways. Beyond rescue missions, intelligence-gathering and warfare, robots may be useful partners in daily life. JAMES offers interesting cues as to what can be accomplished in robot-human interactions.

The stated goal of the project is to develop an artificial embodied agent that "supports socially appropriate, multi-party, multimodal interaction." Its researchers want to enhance so that the can interact with humans in a way that is socially appropriate. A bartending scenario can truly put such capabilities to the test.

Professor Dr. Jan de Ruiter of the Psycholinguistics Research Group at Germany's Bielefeld University along with a partner team in Crete, Munich, and Edinburgh set out to solve the problem of how to use robots as bartenders. A discussion of the work was published August 30 in Frontiers in Psychology, in a paper titled, "Automatic detection of service initiation signals used in bars," by Sebastian Loth, Kerstin Huth and Jan P. De Ruiter.

While attempts have been made to design robots as efficient drink-mixers, the thornier challenge is to design robots that can respond to drink orders. One limited way would be to design a robot capable of responding to orders from a human entering an order via or smartphone, but the JAMES team has envisioned a robot able to recognize the intention of the bystander, whether wanting or not wanting to order, and respond accordingly.

This video is not supported by your browser at this time.
This video shows the new robot bartender of the JAMES project.

One might easily assume that the way most people let the bartender know they want a drink in a crowded, noisy bar is to shout and wave their hand.

The researchers found out, however, that it was more important for the robot to understand nonverbal cues and another type of . The team took a video camera to establishments in Germany and in Scotland, and recorded people ordering drinks at the bar.

Only 1 in 25 waved. Most people were nonverbal in their drink ordering behavior, standing perpendicular to the bar and looking at the bartender. The videos revealed that they turned away from the bar or spoke with a person next to them if they did not want to order.

The robot's hardware is maintained and programmed by Fortiss GmbH in Munich and findings from the surveillance footage were a factor in programming JAMES. The robot responds only if certain the person wants to order a drink.

A variety of software components have been developed by JAMES project partners and used on the JAMES robot. Many of these components will be released at the end of the project.

The project page lists components that include Robotics Library, an open source C++ library for robot kinematics, motion planning and control; Planning with Knowledge and Sensing (PKS), a C++ library for symbolic planning, used on JAMES for high-level social, dialogue, and physical action selection. Third-party software that has been involved includes Kinect for Windows, used on JAMES for automated speech recognition; Open CV, an open source computer vision and machine learning software library, used on JAMES for computer vision, and several others.

Explore further: Q&A: Drones might help explain how tornadoes form

More information: homepages.inf.ed.ac.uk/rpetrick/projects/james/

Related Stories

Dogs' behavior could help to design social robots

Sep 12, 2013

Designers of social robots, take note. Bring your dog to the lab next time you test a prototype, and watch how your pet interacts with it. You might just learn a thing or two that could help you fine-tune ...

TUM robots 'Kinect' to sandwiches and popcorn

Dec 12, 2011

(PhysOrg.com) -- A robotics team from the Technical University of Munich are now able to show an audience how their cuisine robots James and Rosie have graduated from a previously famous repertoire of sausages ...

Charlotte robot tells the world where it's not going

Jun 03, 2013

(Phys.org) —A resourceful thinker who likes to learn as he goes, Kevin Ochs started out on a project with the intention of brushing up skills in C++ programming. He has come up with something quite interesting ...

Recommended for you

Why the Sony hack isn't big news in Japan

11 hours ago

Japan's biggest newspaper, Yomiuri Shimbun, featured a story about Sony Corp. on its website Friday. It wasn't about hacking. It was about the company's struggling tablet business.

Off-world manufacturing is a go with space printer

14 hours ago

On Friday, the BBC reported on a NASA email exchange with a space station which involved astronauts on the International Space Station using their 3-D printer to make a wrench from instructions sent up in ...

Cadillac CT6 will get streaming video mirror

15 hours ago

Cadillac said Thursday it will add high resolution streaming video to the function of a rearview mirror, so that the driver's vision and safety can be enhanced. The technology will debut on the 2016 Cadillac ...

Sony faces 4th ex-employee lawsuit over hack

15 hours ago

A former director of technology for Sony Pictures Entertainment has sued the company over the data breach that resulted in the online posting of his private financial and personal information.

User comments : 1

Adjust slider to filter visible comments by rank

Display comments: newest first

antialias_physorg
not rated yet Sep 24, 2013
The team took a video camera to establishments in Germany and in Scotland, and recorded people ordering drinks at the bar.

Damn. Some guys get all the fun locations for doing research. Beats visiting operating rooms and studying surgeons or stomping around the arctic trying to watch polar bears any day.

But I think what they should try next is detect when someone has had enough. The lone drunk that is sittnig at the bar might get served non-stop otherwise.
(Although I'm not sure how to get testers for that scenario)

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.