A computer system allows a machine to recognize a person's emotional state

November 21, 2011
Scientists at the Universidad Carlos III de Madrid and the Universidad de Granada have developed a new computer system that allows a machine to automatically recognize the emotional state of a person who is orally communicating with it. Credit: UC3M

The system created by these researchers can be used to automatically adapt the dialogue to the user's situation, so that the machine's response is adequate to the person's emotional state. "Thanks to this new development, the machine will be able to determine how the user feels (emotions) and how s/he intends to continue the dialogue (intentions)", explains one of its creators, David Grill, a professor in UC3M's Computer Science Department.

To detect the user's emotional state, the scientists focused on that can make talking with an automatic system frustrating. Specifically, their work considered anger, and doubt. To automatically detect these feelings, information regarding the tone of voice, the speed of speech, the duration of pauses, the energy of the voice signal and so on, up to a total of sixty different acoustic parameters, was used.

In addition, information regarding how the dialogue developed was used to adjust for the probability that the user was in one or another. For example, if the system did not correctly recognize what the interlocutor wanted to say several times, or if it asked the user to repeat information that s/he had already given, these factors could anger or bore the user when he was interacting with the system. Moreover, the authors of the study, which has been published in the Journal on Advances in , point out that it is important that the machine be able to predict how the rest of the dialogue is going to continue. "To that end, we have developed a that uses earlier dialogues to learn what actions the user is most likely to take at any given moment", the researchers highlight.

Once both emotion and intention have been detected, the scientists propose automatically adapting the dialogue to the situation the user is experiencing. For example, if he has , more detailed help can be offered, whereas if s/he is bored, such an offer could be counterproductive. The authors defined the guidelines for obtaining this adaptation by carrying out an empirical evaluation with actual users; in this way they were able to demonstrate that an adaptable system works better in objective terms (for example, it produces shorter and more successful dialogues) and it was perceived as being more useful by the users.

This study was carried out by Professor David Grill Barres, of the Applied Artificial Intelligence Group of UC3M's Computer Science Department, together with Professors Zoraida Callejas Carrión and Ramón López-Cózar Delgado, of the Spoken and Multimodal Group of the Computer Languages and Systems Department of the UGR. This achievement falls within the area of affective computation (computer systems that are capable of processing and/or responding to the user's emotions).

Explore further: Multilingual speech-based technology to talk about!

More information: Predicting user mental states in spoken dialogue systems, Z. Callejas, D. Grill, R. López-Cózar, EURASIP Journal on Advances in Signal Processing, 2011:6, pp. 1-23

Related Stories

Multilingual speech-based technology to talk about!

January 19, 2005

Paving the way for much more intuitive, interactive, and user-friendly ‘spoken dialogue technology’, DUMAS developed a multilingual speech-based system that creates new ways to communicate. DUMAS, a three-year IST-funded ...

Taking computer chat to a whole new level

May 30, 2008

Natural spoken dialogue technology has long been a dream for many. Advances by European researchers are making this a reality. The results of their work could soon be used to allow us to verbally interact with technology ...

Avatars as communicators of emotions

July 9, 2008

Current interactive systems enable users to communicate with computers in many ways, but not taking into account emotional communication. A PhD thesis presented at the University of the Basque Country puts forward the use ...

Your Next Computer May Know How You Feel

April 14, 2010

(PhysOrg.com) -- Friends, loved ones and pets can sense your mood almost instantly - and one day your computer may be able to do so pretty quickly as well.

'Emotionsense' determines emotions by phone

September 29, 2010

 A system which enables psychologists to track people’s emotional behavior through their mobile phones has been successfully road-tested by researchers.

Recommended for you

World is embracing clean energy, professor says

February 1, 2016

Renewable, energy efficient and flexible electricity sources are being adopted by policy makers and investors across the globe and this is sign of optimism in the battle against climate change, a University of Exeter energy ...

Battery technology could charge up water desalination

February 4, 2016

The technology that charges batteries for electronic devices could provide fresh water from salty seas, says a new study by University of Illinois engineers. Electricity running through a salt water-filled battery draws the ...

Researchers find vulnerability in two-factor authentication

February 3, 2016

Two-factor authentication is a computer security measure used by major online service providers to protect the identify of users in the event of a password loss. The process is familiar: When a password is forgotten, the ...

World's first 'robot run' farm to open in Japan

February 1, 2016

A Japanese firm said Monday it would open the world's first fully automated farm with robots handling almost every step of the process, from watering seedlings to harvesting crops.

5 comments

Adjust slider to filter visible comments by rank

Display comments: newest first

HurfDurf
not rated yet Nov 21, 2011
Look Dave, I can see you're really upset about this. I honestly think you ought to sit down calmly, take a stress pill, and think things over.
Isaacsname
not rated yet Nov 21, 2011
...It's the Voight-Kampf test ..o,O
bredmond
not rated yet Nov 21, 2011
...It's the Voight-Kampf test ..o,O


Is this to test whether I am a replicant or a lesbian?
powerup1
1 / 5 (1) Nov 22, 2011
In the future UI will use this and other biometrics, eventually our brain waves. It will allow systems to emotionally adjust to the user. I can imagine the day when we will carry a psychotherapist in our pockets. I know that some people are afraid of this type of technology, but I believe it will give us a better quality of life.
bredmond
not rated yet Nov 23, 2011
In the future UI will use this and other biometrics, eventually our brain waves. It will allow systems to emotionally adjust to the user. I can imagine the day when we will carry a psychotherapist in our pockets. I know that some people are afraid of this type of technology, but I believe it will give us a better quality of life.


Cellphone application. Maybe connected to other biometric sensors which we can wear. I still am anxious for the nanite cloud in symbiance inside our cells. Something like mitochondria, but with different function.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.