Body language to be read by computers one of new innovative solutions

Oct 01, 2010
Body language to be read by computers one of new innovative solutions

Can a computer read your body language? A consortium of European researchers thinks so, and has developed a range of innovative solutions from escalator safety to online marketing.

The keyboard and mouse are no longer the only means of communicating with computers. Modern consumer devices will respond to the touch of a finger and even the spoken word, but can we go further still? Can a computer learn to make sense of how we walk and stand, to understand our gestures and even to read our ?

The EU-funded MIAUCE project set out to do just that. "The motivation of the project is to put humans in the loop of interaction between the computer and their environment,” explains project coordinator Chaabane Djeraba, of CNRS in Lille.

“We would like to have a form of ambient intelligence where computers are completely hidden,” he says. “This means a multimodal interface so people can interact with their environment. The computer sees their behavior and then extracts information useful for the user."

It is hard to imagine a world where hidden computers try to anticipate our needs, so the MIAUCE project has developed concrete prototypes of three kinds of applications.

Escalator accidents

The first is to monitor the safety of crowds at busy places such as airports and shopping centres. Surveillance cameras are used to detect situations such as accidents on escalators.

“The background technology of this research is based on ,” says Djeraba. “We extract information from videos. This is the basic technology and technical method we use.”

It’s quite a challenge. First the video stream must be analysed in real time to extract a hierarchy of three levels of features. At its lowest, this is a of shapes, movements and flows. At the next level this basic description is interpreted in terms of crowd density, speed and direction. At the highest level the computer is able to decide when the activity becomes ‘abnormal’ perhaps because someone has fallen on an escalator and caused a pile-up that needs urgent intervention.

It is at the second level and the third “semantic” level of interpretation that MIAUCE has been most concerned.

One of the MIAUCE partners is already working with a manufacturer of escalators to augment existing video monitoring systems at international airports where there may be hundreds of escalators. If a collapse can be detected automatically then the seconds saved in responding could save lives as well.

But safety is only one possible kind of application where computers could read our .

Face swapping

A second could be in marketing, specifically to monitor how customers behave in shops. “We would like to analyse how people walk around in a shop,” Djeraba says, “and the behaviour of people in the shop, where they look, for example.”

The same partner is developing two products. One will be a ‘people counter’ to monitor pedestrian flows in the street outside a shop. It is expected to be particularly attractive to fashion stores who wish to attract passers-by. Another is a ‘heat map generator’ to watch the movements of people inside the store, so that the manager can see which parts of the displays are attracting the most attention.

The third application addressed by MIAUCE is interactive web television, a technology of increasing interest where viewers can select what they want to see. As part of the project, the viewer’s webcam is used to monitor their face to see which part of the screen they are looking at.

It could be used to feed the user further information based on the evidence of what they have shown an interest in. Project partner Tilde, a software company in Latvia, is commercialising this application.

MIAUCE has also developed a related technology of ‘face swapping’ in which the viewer’s face can replace that of a model. This could be used for trying out hairstyles and clothing.

Ethics and anonymity

These are all ingenious applications but are there not ethical and legal worries about reading people’s behavior in this way?

Djeraba acknowledges that the project team took such issues very seriously and several possible applications of their technology were ruled out on such grounds.

They worked to some basic rules, such as placing cameras only on private premises and always with a warning notice, but the fundamental principle was anonymity. “We have to anonymise people,” he says. “What we are doing here is analysing user behavior without any identification, this is a fundamental requirement for such systems.”

They also took account of whether the applications would be acceptable to society as a whole. No one would reasonably object to the monitoring of escalators, for example, if the aim was to improve public safety. But the technology must not identify individuals or even such characteristics as skin color.

“Generally speaking, anonymity is the critical point. If we anonymise it’s OK, if we don’t anonymise it’s not OK,” Djeraba says.

Explore further: For Google's self-driving cars, learning to deal with the bizarre is essential

add to favorites email to friend print save as pdf

Related Stories

Software: running commentary for smarter surveillance?

Mar 16, 2010

(PhysOrg.com) -- Cutting-edge surveillance software that automatically detects human motion, behaviour and facial expressions, generates a running commentary of what?s happening and re-enacts events virtually ...

Emotional machines

Apr 03, 2008

Emotions are an intrinsic part of communications. But machines don’t have, perceive or react to them, which makes us – their handlers – hot under the collar. But thanks to building blocks developed by ...

Gesture-driven computers

Feb 29, 2008

It isn’t always easy to communicate with a computer. Two Fraunhofer Institutes will be presenting new possibilities of man-machine interaction at CeBIT in Hanover (Germany) on March 4 through 9. They will demonstrate how ...

Rate of escalator injuries to older adults has doubled

Mar 13, 2008

In the first large scale national study of escalator-related injuries to older adults, researchers led by Joseph O’Neil, M.D., MPH, and Greg Steele, Dr.PH., MPH, of the Indiana University School of Medicine, report that ...

Enabling the blind to find their way

Oct 24, 2008

(PhysOrg.com) -- “Eyes on the future” is the mantra of the ‘World Sight Day’ held this month to raise awareness of blindness and vision impairment. New technologies, developed by European researchers offering the ...

Visual system interprets sign languages

Jun 02, 2010

Spanish sign language is used by over 100,000 people with hearing impairments and is made up of hundreds of signs. CVC-UAB researchers Sergio Escalera, Petia Radeva and Jordi Vitria selected over twenty of these signs to ...

Recommended for you

Audi to develop Tesla Model S all-electric rival

3 hours ago

The Tesla Model S has a rival. Audi is to develop all-electric family car. This is to be a family car that will offer an all-electric range of 280 miles (450 kilometers), according to Auto Express, which ...

A green data center with an autonomous power supply

8 hours ago

A new data center in the United States is generating electricity for its servers entirely from renewable sources, converting biogas from a sewage treatment plant into electricity and water. Siemens implemented ...

After a data breach, it's consumers left holding the bag

9 hours ago

Shoppers have launched into the holiday buying season and retailers are looking forward to year-end sales that make up almost 20% of their annual receipts. But as you check out at a store or click "purchase" on your online shopping cart ...

Can we create an energy efficient Internet?

9 hours ago

With the number of Internet connected devices rapidly increasing, researchers from Melbourne are starting a new research program to reduce energy consumption of such devices.

Brain inspired data engineering

10 hours ago

What if next-generation ICT systems could be based on the brain's structure and its cognitive and adaptive processes? A groundbreaking paradigm of brain-inspired intelligent ICT architectures is being born.

User comments : 0

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.