Computers excel at identifying smiles of frustration (w/ Video)

May 25, 2012 by Lin Edwards, report

( -- Researchers at the Massachusetts Institute of Technology (MIT) in the US have trained computers to recognize smiles, and they have turned out to be more adept at recognizing smiles of frustration than humans.

The scientists, led by M. Ehsan Hoque, used a to film subjects as they filled out a long form which was automatically deleted when the subjects pressed Submit. Most (90%) of the subjects smiled with the frustration, but when asked to assume a frustrated expression the same percentage did not smile.

The researchers also studied “happiness” smiles, produced when subjects were shown cute videos of babies, and they were also asked to adopt a happy smile without an emotional prompt. The volunteers were also asked to interpret images of people either genuinely smiling or faking a smile of delight or frustration.

Video: Melanie Gonick

The researchers, from the MIT Media Lab, developed an algorithm to track the movements made by muscle groups producing various and quantified the facial features using the Facial Action Coding System (FACS).

FACS was developed in the 1970s and has become the most widely used system in facial recognition applications and for people needing to understand facial behaviours. To use FACS, researchers decompose the expression into “Action Units,” and they may also record any asymmetry, as well as the intensity and duration of the expression.

Smiles that are easily identified as insincere or “fake” are made using only the zygomatic major muscles, which are voluntary muscles that lift the corners of the mouth. In genuine smiles (known as Duchenne smiles) the corners of the mouth rise, but involuntary muscles also raise the cheeks and produce small wrinkles around the eyes. The study found that genuine frustrated and happy smiles differ primarily in their timing, with frustrated smiles appearing more quickly than smiles of delight.

Can you tell which of these smiles is showing happiness? Or which one is the result of frustration? A computer system developed at MIT can. The answer: The smile on the right is the sign of frustration. Images courtesy of Hoque et al.

The computer algorithm was able to identify smiles of frustration over 90 percent of the time, while human subjects performed below chance. Hoque said the computer’s better performance is probably because it focuses solely on the mechanical features of the smile and breaks them down into action units, while humans rely on other cues such as how quickly the smile is produced to determine the emotion behind it. The study also showed that most do not expect a smile in response to .

The findings, published in IEEE Transactions on Affective Computing, might be useful in training children with autism to identify the different kinds of smiles, and they might also find application in programming computers to respond to the mood of their users.

Explore further: Phony friends? Rejected people better able to spot fake smiles

Related Stories

Older adults spot phoney smiles better, study shows

April 12, 2011

( -- Your great aunt may be slowing down as she grows older, but a study created in a Brandeis laboratory reveals that she’s probably better than you are at perceiving a genuine smile.

Researchers measure the value of a smile

May 10, 2011

( -- Researchers at Bangor University have placed an economic value on a smile, demonstrating scientifically the effect that a genuine smile can have on our decision-making. The psychologists call this ‘social ...

Study: Facial expressions of emotion are innate, not learned

December 29, 2008

Facial expressions of emotion are hardwired into our genes, according to a study published today in the Journal of Personality and Social Psychology. The research suggests that facial expressions of emotion are innate rather ...

Patients, dentists differ on smile ratings, JADA study

December 10, 2007

People rate their smiles higher than dentists do, according to a new study. Teeth and eyes rated as the most important features of an attractive face, the study also found, and people younger than age 50 were most satisfied ...

Broader smile, longer life: study

March 24, 2010

The broader your smile and the deeper the creases around your eyes when you grin, the longer you are likely to live, according to a study published in Psychological Science this week.

Recommended for you

Team breaks world record for fast, accurate AI training

November 7, 2018

Researchers at Hong Kong Baptist University (HKBU) have partnered with a team from Tencent Machine Learning to create a new technique for training artificial intelligence (AI) machines faster than ever before while maintaining ...

1 comment

Adjust slider to filter visible comments by rank

Display comments: newest first

not rated yet May 25, 2012
the above example was a cinch to spot, imo. i feel like in most cases there's an utterance or released/ inhaled breath involved in a smile of frustration. i'd like to see more examples, myself.. and who was studied, anyway? considering something like face blindness (prosopagnosia) was only recently discovered as far more common than previously thought, how do we know to rely on the consistency of the subjects' abilities to discern these things?

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.