Robot knows who wants one for the road

Sep 24, 2013 by Nancy Owano report

(Phys.org) —JAMES has a head that is actually a tablet. JAMES is an efficient waiter yet only has one arm. JAMES can read your body language to know you want a drink without your saying a word. Perhaps the biggest surprise of all, however, is that the EU is funding this project. JAMES stands for The Joint Action in Multimodal Embodied Systems. The project started in 2011 and continues until January. Actually, its research and results will be helpful for scientists to know how robots can assist people in meaningful ways. Beyond rescue missions, intelligence-gathering and warfare, robots may be useful partners in daily life. JAMES offers interesting cues as to what can be accomplished in robot-human interactions.

The stated goal of the project is to develop an artificial embodied agent that "supports socially appropriate, multi-party, multimodal interaction." Its researchers want to enhance so that the can interact with humans in a way that is socially appropriate. A bartending scenario can truly put such capabilities to the test.

Professor Dr. Jan de Ruiter of the Psycholinguistics Research Group at Germany's Bielefeld University along with a partner team in Crete, Munich, and Edinburgh set out to solve the problem of how to use robots as bartenders. A discussion of the work was published August 30 in Frontiers in Psychology, in a paper titled, "Automatic detection of service initiation signals used in bars," by Sebastian Loth, Kerstin Huth and Jan P. De Ruiter.

While attempts have been made to design robots as efficient drink-mixers, the thornier challenge is to design robots that can respond to drink orders. One limited way would be to design a robot capable of responding to orders from a human entering an order via or smartphone, but the JAMES team has envisioned a robot able to recognize the intention of the bystander, whether wanting or not wanting to order, and respond accordingly.

This video is not supported by your browser at this time.
This video shows the new robot bartender of the JAMES project.

One might easily assume that the way most people let the bartender know they want a drink in a crowded, noisy bar is to shout and wave their hand.

The researchers found out, however, that it was more important for the robot to understand nonverbal cues and another type of . The team took a video camera to establishments in Germany and in Scotland, and recorded people ordering drinks at the bar.

Only 1 in 25 waved. Most people were nonverbal in their drink ordering behavior, standing perpendicular to the bar and looking at the bartender. The videos revealed that they turned away from the bar or spoke with a person next to them if they did not want to order.

The robot's hardware is maintained and programmed by Fortiss GmbH in Munich and findings from the surveillance footage were a factor in programming JAMES. The robot responds only if certain the person wants to order a drink.

A variety of software components have been developed by JAMES project partners and used on the JAMES robot. Many of these components will be released at the end of the project.

The project page lists components that include Robotics Library, an open source C++ library for robot kinematics, motion planning and control; Planning with Knowledge and Sensing (PKS), a C++ library for symbolic planning, used on JAMES for high-level social, dialogue, and physical action selection. Third-party software that has been involved includes Kinect for Windows, used on JAMES for automated speech recognition; Open CV, an open source computer vision and machine learning software library, used on JAMES for computer vision, and several others.

Explore further: Doing research in the pub: Researchers study how orders are placed at the bar

More information: homepages.inf.ed.ac.uk/rpetrick/projects/james/

Related Stories

Dogs' behavior could help to design social robots

Sep 12, 2013

Designers of social robots, take note. Bring your dog to the lab next time you test a prototype, and watch how your pet interacts with it. You might just learn a thing or two that could help you fine-tune ...

TUM robots 'Kinect' to sandwiches and popcorn

Dec 12, 2011

(PhysOrg.com) -- A robotics team from the Technical University of Munich are now able to show an audience how their cuisine robots James and Rosie have graduated from a previously famous repertoire of sausages ...

Charlotte robot tells the world where it's not going

Jun 03, 2013

(Phys.org) —A resourceful thinker who likes to learn as he goes, Kevin Ochs started out on a project with the intention of brushing up skills in C++ programming. He has come up with something quite interesting ...

Recommended for you

Social Security spent $300M on 'IT boondoggle'

7 hours ago

(AP)—Six years ago the Social Security Administration embarked on an aggressive plan to replace outdated computer systems overwhelmed by a growing flood of disability claims.

Cheaper wireless plans cut into AT&T 2Q profit

7 hours ago

(AP)—AT&T posted lower net income for the latest quarter due to cheaper cellphone plans it introduced as a response to aggressive pricing from smaller competitor T-Mobile US.

Six charged in global e-ticket hacking scheme

8 hours ago

Criminal charges were filed Wednesday against six people in what authorities said was a global cyber-crime ring that created fraudulent e-tickets for major concerts and sporting events.

User comments : 1

Adjust slider to filter visible comments by rank

Display comments: newest first

antialias_physorg
not rated yet Sep 24, 2013
The team took a video camera to establishments in Germany and in Scotland, and recorded people ordering drinks at the bar.

Damn. Some guys get all the fun locations for doing research. Beats visiting operating rooms and studying surgeons or stomping around the arctic trying to watch polar bears any day.

But I think what they should try next is detect when someone has had enough. The lone drunk that is sittnig at the bar might get served non-stop otherwise.
(Although I'm not sure how to get testers for that scenario)