Computers excel at identifying smiles of frustration (w/ Video)

May 25, 2012 by Lin Edwards report

(Phys.org) -- Researchers at the Massachusetts Institute of Technology (MIT) in the US have trained computers to recognize smiles, and they have turned out to be more adept at recognizing smiles of frustration than humans.

The scientists, led by M. Ehsan Hoque, used a to film subjects as they filled out a long form which was automatically deleted when the subjects pressed Submit. Most (90%) of the subjects smiled with the frustration, but when asked to assume a frustrated expression the same percentage did not smile.

The researchers also studied “happiness” smiles, produced when subjects were shown cute videos of babies, and they were also asked to adopt a happy smile without an emotional prompt. The volunteers were also asked to interpret images of people either genuinely smiling or faking a smile of delight or frustration.

This video is not supported by your browser at this time.
Video: Melanie Gonick

The researchers, from the MIT Media Lab, developed an algorithm to track the movements made by muscle groups producing various and quantified the facial features using the Facial Action Coding System (FACS).

FACS was developed in the 1970s and has become the most widely used system in facial recognition applications and for people needing to understand facial behaviours. To use FACS, researchers decompose the expression into “Action Units,” and they may also record any asymmetry, as well as the intensity and duration of the expression.

Smiles that are easily identified as insincere or “fake” are made using only the zygomatic major muscles, which are voluntary muscles that lift the corners of the mouth. In genuine smiles (known as Duchenne smiles) the corners of the mouth rise, but involuntary muscles also raise the cheeks and produce small wrinkles around the eyes. The study found that genuine frustrated and happy smiles differ primarily in their timing, with frustrated smiles appearing more quickly than smiles of delight.

Can you tell which of these smiles is showing happiness? Or which one is the result of frustration? A computer system developed at MIT can. The answer: The smile on the right is the sign of frustration. Images courtesy of Hoque et al.

The computer algorithm was able to identify smiles of frustration over 90 percent of the time, while human subjects performed below chance. Hoque said the computer’s better performance is probably because it focuses solely on the mechanical features of the smile and breaks them down into action units, while humans rely on other cues such as how quickly the smile is produced to determine the emotion behind it. The study also showed that most do not expect a smile in response to .

The findings, published in IEEE Transactions on Affective Computing, might be useful in training children with autism to identify the different kinds of smiles, and they might also find application in programming computers to respond to the mood of their users.

Explore further: IBM Watson Group to transform the consumer shopping experience

Related Stories

Older adults spot phoney smiles better, study shows

Apr 12, 2011

(PhysOrg.com) -- Your great aunt may be slowing down as she grows older, but a study created in a Brandeis laboratory reveals that she’s probably better than you are at perceiving a genuine smile.

Researchers measure the value of a smile

May 10, 2011

(PhysOrg.com) -- Researchers at Bangor University have placed an economic value on a smile, demonstrating scientifically the effect that a genuine smile can have on our decision-making. The psychologists call ...

Patients, dentists differ on smile ratings, JADA study

Dec 10, 2007

People rate their smiles higher than dentists do, according to a new study. Teeth and eyes rated as the most important features of an attractive face, the study also found, and people younger than age 50 were most satisfied ...

Broader smile, longer life: study

Mar 24, 2010

The broader your smile and the deeper the creases around your eyes when you grin, the longer you are likely to live, according to a study published in Psychological Science this week.

Recommended for you

Computer-assisted accelerator design

Apr 22, 2014

Stephen Brooks uses his own custom software tool to fire electron beams into a virtual model of proposed accelerator designs for eRHIC. The goal: Keep the cost down and be sure the beams will circulate in ...

First steps towards "Experimental Literature 2.0"

Apr 21, 2014

As part of a student's thesis, the Laboratory of Digital Humanities at EPFL has developed an application that aims at rearranging literary works by changing their chapter order. "The human simulation" a saga ...

User comments : 1

Adjust slider to filter visible comments by rank

Display comments: newest first

nayTall
not rated yet May 25, 2012
the above example was a cinch to spot, imo. i feel like in most cases there's an utterance or released/ inhaled breath involved in a smile of frustration. i'd like to see more examples, myself.. and who was studied, anyway? considering something like face blindness (prosopagnosia) was only recently discovered as far more common than previously thought, how do we know to rely on the consistency of the subjects' abilities to discern these things?

More news stories

Amazon Prime wins streaming deal with HBO

Amazon scored a deal Wednesday to distribute old shows from premium cable TV channel HBO to its monthly Prime subscribers, landing a blow on rival Netflix in the streaming video battle.

Is nuclear power the only way to avoid geoengineering?

"I think one can argue that if we were to follow a strong nuclear energy pathway—as well as doing everything else that we can—then we can solve the climate problem without doing geoengineering." So says Tom Wigley, one ...