A computer system allows a machine to recognize a person's emotional state

Nov 21, 2011
Scientists at the Universidad Carlos III de Madrid and the Universidad de Granada have developed a new computer system that allows a machine to automatically recognize the emotional state of a person who is orally communicating with it. Credit: UC3M

The system created by these researchers can be used to automatically adapt the dialogue to the user's situation, so that the machine's response is adequate to the person's emotional state. "Thanks to this new development, the machine will be able to determine how the user feels (emotions) and how s/he intends to continue the dialogue (intentions)", explains one of its creators, David Grill, a professor in UC3M's Computer Science Department.

To detect the user's emotional state, the scientists focused on that can make talking with an automatic system frustrating. Specifically, their work considered anger, and doubt. To automatically detect these feelings, information regarding the tone of voice, the speed of speech, the duration of pauses, the energy of the voice signal and so on, up to a total of sixty different acoustic parameters, was used.

In addition, information regarding how the dialogue developed was used to adjust for the probability that the user was in one or another. For example, if the system did not correctly recognize what the interlocutor wanted to say several times, or if it asked the user to repeat information that s/he had already given, these factors could anger or bore the user when he was interacting with the system. Moreover, the authors of the study, which has been published in the Journal on Advances in , point out that it is important that the machine be able to predict how the rest of the dialogue is going to continue. "To that end, we have developed a that uses earlier dialogues to learn what actions the user is most likely to take at any given moment", the researchers highlight.

Once both emotion and intention have been detected, the scientists propose automatically adapting the dialogue to the situation the user is experiencing. For example, if he has , more detailed help can be offered, whereas if s/he is bored, such an offer could be counterproductive. The authors defined the guidelines for obtaining this adaptation by carrying out an empirical evaluation with actual users; in this way they were able to demonstrate that an adaptable system works better in objective terms (for example, it produces shorter and more successful dialogues) and it was perceived as being more useful by the users.

This study was carried out by Professor David Grill Barres, of the Applied Artificial Intelligence Group of UC3M's Computer Science Department, together with Professors Zoraida Callejas Carrión and Ramón López-Cózar Delgado, of the Spoken and Multimodal Group of the Computer Languages and Systems Department of the UGR. This achievement falls within the area of affective computation (computer systems that are capable of processing and/or responding to the user's emotions).

Explore further: MU researchers develop more accurate Twitter analysis tools

More information: Predicting user mental states in spoken dialogue systems, Z. Callejas, D. Grill, R. López-Cózar, EURASIP Journal on Advances in Signal Processing, 2011:6, pp. 1-23

Provided by Carlos III University of Madrid

5 /5 (1 vote)
add to favorites email to friend print save as pdf

Related Stories

Taking computer chat to a whole new level

May 30, 2008

Natural spoken dialogue technology has long been a dream for many. Advances by European researchers are making this a reality. The results of their work could soon be used to allow us to verbally interact with technology ...

Your Next Computer May Know How You Feel

Apr 14, 2010

(PhysOrg.com) -- Friends, loved ones and pets can sense your mood almost instantly - and one day your computer may be able to do so pretty quickly as well.

Avatars as communicators of emotions

Jul 09, 2008

Current interactive systems enable users to communicate with computers in many ways, but not taking into account emotional communication. A PhD thesis presented at the University of the Basque Country puts ...

Multilingual speech-based technology to talk about!

Jan 19, 2005

Paving the way for much more intuitive, interactive, and user-friendly ‘spoken dialogue technology’, DUMAS developed a multilingual speech-based system that creates new ways to communicate. DUMAS, a three-year IST-funded pro ...

Recommended for you

Chameleon: Cloud computing for computer science

Aug 26, 2014

Cloud computing has changed the way we work, the way we communicate online, even the way we relax at night with a movie. But even as "the cloud" starts to cross over into popular parlance, the full potential ...

User comments : 5

Adjust slider to filter visible comments by rank

Display comments: newest first

HurfDurf
not rated yet Nov 21, 2011
Look Dave, I can see you're really upset about this. I honestly think you ought to sit down calmly, take a stress pill, and think things over.
Isaacsname
not rated yet Nov 21, 2011
...It's the Voight-Kampf test ..o,O
bredmond
not rated yet Nov 21, 2011
...It's the Voight-Kampf test ..o,O


Is this to test whether I am a replicant or a lesbian?
powerup1
1 / 5 (1) Nov 22, 2011
In the future UI will use this and other biometrics, eventually our brain waves. It will allow systems to emotionally adjust to the user. I can imagine the day when we will carry a psychotherapist in our pockets. I know that some people are afraid of this type of technology, but I believe it will give us a better quality of life.
bredmond
not rated yet Nov 23, 2011
In the future UI will use this and other biometrics, eventually our brain waves. It will allow systems to emotionally adjust to the user. I can imagine the day when we will carry a psychotherapist in our pockets. I know that some people are afraid of this type of technology, but I believe it will give us a better quality of life.


Cellphone application. Maybe connected to other biometric sensors which we can wear. I still am anxious for the nanite cloud in symbiance inside our cells. Something like mitochondria, but with different function.