Computers excel at identifying smiles of frustration (w/ Video)

May 25, 2012 by Lin Edwards report

(Phys.org) -- Researchers at the Massachusetts Institute of Technology (MIT) in the US have trained computers to recognize smiles, and they have turned out to be more adept at recognizing smiles of frustration than humans.

The scientists, led by M. Ehsan Hoque, used a to film subjects as they filled out a long form which was automatically deleted when the subjects pressed Submit. Most (90%) of the subjects smiled with the frustration, but when asked to assume a frustrated expression the same percentage did not smile.

The researchers also studied “happiness” smiles, produced when subjects were shown cute videos of babies, and they were also asked to adopt a happy smile without an emotional prompt. The volunteers were also asked to interpret images of people either genuinely smiling or faking a smile of delight or frustration.

This video is not supported by your browser at this time.
Video: Melanie Gonick

The researchers, from the MIT Media Lab, developed an algorithm to track the movements made by muscle groups producing various and quantified the facial features using the Facial Action Coding System (FACS).

FACS was developed in the 1970s and has become the most widely used system in facial recognition applications and for people needing to understand facial behaviours. To use FACS, researchers decompose the expression into “Action Units,” and they may also record any asymmetry, as well as the intensity and duration of the expression.

Smiles that are easily identified as insincere or “fake” are made using only the zygomatic major muscles, which are voluntary muscles that lift the corners of the mouth. In genuine smiles (known as Duchenne smiles) the corners of the mouth rise, but involuntary muscles also raise the cheeks and produce small wrinkles around the eyes. The study found that genuine frustrated and happy smiles differ primarily in their timing, with frustrated smiles appearing more quickly than smiles of delight.

Can you tell which of these smiles is showing happiness? Or which one is the result of frustration? A computer system developed at MIT can. The answer: The smile on the right is the sign of frustration. Images courtesy of Hoque et al.

The computer algorithm was able to identify smiles of frustration over 90 percent of the time, while human subjects performed below chance. Hoque said the computer’s better performance is probably because it focuses solely on the mechanical features of the smile and breaks them down into action units, while humans rely on other cues such as how quickly the smile is produced to determine the emotion behind it. The study also showed that most do not expect a smile in response to .

The findings, published in IEEE Transactions on Affective Computing, might be useful in training children with autism to identify the different kinds of smiles, and they might also find application in programming computers to respond to the mood of their users.

Explore further: Oculus unveils new prototype VR headset

Related Stories

Older adults spot phoney smiles better, study shows

Apr 12, 2011

(PhysOrg.com) -- Your great aunt may be slowing down as she grows older, but a study created in a Brandeis laboratory reveals that she’s probably better than you are at perceiving a genuine smile.

Researchers measure the value of a smile

May 10, 2011

(PhysOrg.com) -- Researchers at Bangor University have placed an economic value on a smile, demonstrating scientifically the effect that a genuine smile can have on our decision-making. The psychologists call ...

Patients, dentists differ on smile ratings, JADA study

Dec 10, 2007

People rate their smiles higher than dentists do, according to a new study. Teeth and eyes rated as the most important features of an attractive face, the study also found, and people younger than age 50 were most satisfied ...

Broader smile, longer life: study

Mar 24, 2010

The broader your smile and the deeper the creases around your eyes when you grin, the longer you are likely to live, according to a study published in Psychological Science this week.

Recommended for you

Oculus unveils new prototype VR headset

22 hours ago

Oculus has unveiled a new prototype of its virtual reality headset. However, the VR company still isn't ready to release a consumer edition.

Who drives Alibaba's Taobao traffic—buyers or sellers?

Sep 18, 2014

As Chinese e-commerce firm Alibaba prepares for what could be the biggest IPO in history, University of Michigan professor Puneet Manchanda dug into its Taobao website data to help solve a lingering chicken-and-egg question.

Computerized emotion detector

Sep 16, 2014

Face recognition software measures various parameters in a mug shot, such as the distance between the person's eyes, the height from lip to top of their nose and various other metrics and then compares it with photos of people ...

User comments : 1

Adjust slider to filter visible comments by rank

Display comments: newest first

nayTall
not rated yet May 25, 2012
the above example was a cinch to spot, imo. i feel like in most cases there's an utterance or released/ inhaled breath involved in a smile of frustration. i'd like to see more examples, myself.. and who was studied, anyway? considering something like face blindness (prosopagnosia) was only recently discovered as far more common than previously thought, how do we know to rely on the consistency of the subjects' abilities to discern these things?