Robot knows who wants one for the road

Sep 24, 2013 by Nancy Owano report

(Phys.org) —JAMES has a head that is actually a tablet. JAMES is an efficient waiter yet only has one arm. JAMES can read your body language to know you want a drink without your saying a word. Perhaps the biggest surprise of all, however, is that the EU is funding this project. JAMES stands for The Joint Action in Multimodal Embodied Systems. The project started in 2011 and continues until January. Actually, its research and results will be helpful for scientists to know how robots can assist people in meaningful ways. Beyond rescue missions, intelligence-gathering and warfare, robots may be useful partners in daily life. JAMES offers interesting cues as to what can be accomplished in robot-human interactions.

The stated goal of the project is to develop an artificial embodied agent that "supports socially appropriate, multi-party, multimodal interaction." Its researchers want to enhance so that the can interact with humans in a way that is socially appropriate. A bartending scenario can truly put such capabilities to the test.

Professor Dr. Jan de Ruiter of the Psycholinguistics Research Group at Germany's Bielefeld University along with a partner team in Crete, Munich, and Edinburgh set out to solve the problem of how to use robots as bartenders. A discussion of the work was published August 30 in Frontiers in Psychology, in a paper titled, "Automatic detection of service initiation signals used in bars," by Sebastian Loth, Kerstin Huth and Jan P. De Ruiter.

While attempts have been made to design robots as efficient drink-mixers, the thornier challenge is to design robots that can respond to drink orders. One limited way would be to design a robot capable of responding to orders from a human entering an order via or smartphone, but the JAMES team has envisioned a robot able to recognize the intention of the bystander, whether wanting or not wanting to order, and respond accordingly.

This video is not supported by your browser at this time.
This video shows the new robot bartender of the JAMES project.

One might easily assume that the way most people let the bartender know they want a drink in a crowded, noisy bar is to shout and wave their hand.

The researchers found out, however, that it was more important for the robot to understand nonverbal cues and another type of . The team took a video camera to establishments in Germany and in Scotland, and recorded people ordering drinks at the bar.

Only 1 in 25 waved. Most people were nonverbal in their drink ordering behavior, standing perpendicular to the bar and looking at the bartender. The videos revealed that they turned away from the bar or spoke with a person next to them if they did not want to order.

The robot's hardware is maintained and programmed by Fortiss GmbH in Munich and findings from the surveillance footage were a factor in programming JAMES. The robot responds only if certain the person wants to order a drink.

A variety of software components have been developed by JAMES project partners and used on the JAMES robot. Many of these components will be released at the end of the project.

The project page lists components that include Robotics Library, an open source C++ library for robot kinematics, motion planning and control; Planning with Knowledge and Sensing (PKS), a C++ library for symbolic planning, used on JAMES for high-level social, dialogue, and physical action selection. Third-party software that has been involved includes Kinect for Windows, used on JAMES for automated speech recognition; Open CV, an open source computer vision and machine learning software library, used on JAMES for computer vision, and several others.

Explore further: Doing research in the pub: Researchers study how orders are placed at the bar

More information: homepages.inf.ed.ac.uk/rpetrick/projects/james/

Related Stories

Dogs' behavior could help to design social robots

Sep 12, 2013

Designers of social robots, take note. Bring your dog to the lab next time you test a prototype, and watch how your pet interacts with it. You might just learn a thing or two that could help you fine-tune ...

TUM robots 'Kinect' to sandwiches and popcorn

Dec 12, 2011

(PhysOrg.com) -- A robotics team from the Technical University of Munich are now able to show an audience how their cuisine robots James and Rosie have graduated from a previously famous repertoire of sausages ...

Charlotte robot tells the world where it's not going

Jun 03, 2013

(Phys.org) —A resourceful thinker who likes to learn as he goes, Kevin Ochs started out on a project with the intention of brushing up skills in C++ programming. He has come up with something quite interesting ...

Recommended for you

Students turn $250 wheelchair into geo-positioning robot

7 hours ago

Talk about your Craigslist finds! A team of student employees at The University of Alabama in Huntsville's Systems Management and Production Center (SMAP) combined inspiration with innovation to make a $250 ...

Using robots to study evolution

Apr 14, 2014

A new paper by OIST's Neural Computation Unit has demonstrated the usefulness of robots in studying evolution. Published in PLOS ONE, Stefan Elfwing, a researcher in Professor Kenji Doya's Unit, has succes ...

User comments : 1

Adjust slider to filter visible comments by rank

Display comments: newest first

antialias_physorg
not rated yet Sep 24, 2013
The team took a video camera to establishments in Germany and in Scotland, and recorded people ordering drinks at the bar.

Damn. Some guys get all the fun locations for doing research. Beats visiting operating rooms and studying surgeons or stomping around the arctic trying to watch polar bears any day.

But I think what they should try next is detect when someone has had enough. The lone drunk that is sittnig at the bar might get served non-stop otherwise.
(Although I'm not sure how to get testers for that scenario)

More news stories

Quantenna promises 10-gigabit Wi-Fi by next year

(Phys.org) —Quantenna Communications has announced that it has plans for releasing a chipset that will be capable of delivering 10Gbps WiFi to/from routers, bridges and computers by sometime next year. ...

Floating nuclear plants could ride out tsunamis

When an earthquake and tsunami struck the Fukushima Daiichi nuclear plant complex in 2011, neither the quake nor the inundation caused the ensuing contamination. Rather, it was the aftereffects—specifically, ...

Unlocking secrets of new solar material

(Phys.org) —A new solar material that has the same crystal structure as a mineral first found in the Ural Mountains in 1839 is shooting up the efficiency charts faster than almost anything researchers have ...

Patent talk: Google sharpens contact lens vision

(Phys.org) —A report from Patent Bolt brings us one step closer to what Google may have in mind in developing smart contact lenses. According to the discussion Google is interested in the concept of contact ...

How kids' brain structures grow as memory develops

Our ability to store memories improves during childhood, associated with structural changes in the hippocampus and its connections with prefrontal and parietal cortices. New research from UC Davis is exploring ...