Computer spots micro clue to lies

November 23, 2011 by Pete Wilton
Computer spots micro clue to lies
An example of a facial micro-expression (top-left) being interpolated through graph embedding (top-right); the result from which spatiotemporal local texture descriptors are extracted (bottom-right), enabling recognition with multiple kernel learning.

( -- Whether you are playing poker or haggling over a deal you might think that you can hide your true emotions.

But telltale signs can reveal that you are concealing something, and now researchers at Oxford University and Oulu University are developing software that can recognise these ‘micro-expressions’ - which could be bad news for liars.

‘Micro-expressions are very rapid facial expressions, lasting between a twenty-fifth and a third of a second, that reveal emotions people try to hide,’ Tomas Pfister of Oxford University’s Department of Engineering Science tells me.

‘They can be used for lie detection and are actively used by trained officials at US airports to detect suspicious behaviour.

‘For example, a terrorist trying to conceal a plan to commit suicide would very likely show a very short expression of intense anguish. Similarly, a business negotiator who has been proposed a suitable price for a big deal would likely show a happy micro-expression.’

Tomas is leading efforts to create software that can automatically detect these micro-expressions - something he says is particularly attractive because humans are not very good at accurately spotting them.

He explains that two characteristics of micro-expressions make them particularly challenging for a computer to recognise:

Firstly, they are involuntary: ‘How can we get human training data for our algorithm when the expressions are involuntary?’ he comments. ‘We cannot rely on actors as they cannot act out involuntary expressions.’

The second big problem is that they occur for only a fraction of a second: this means that, with normal speed cameras, they will only appear in a very limited number of frames, leaving only a small amount of data for a computer to go on.

The researchers looked to tackle the first problem by an experiment in which those taking part were induced to suppress their emotions.

Computer spots micro clue to lies
Left: The lower figure shows a temporal cross-section during the 6 frames long facial micro-expression depicted in the upper figure. The cross-section is positioned at a given x-coordinate on the upper lip of the subject. Right: An illustration of the temporal interpolation method: the video is mapped onto a curve along which a new video is sampled.

‘Subjects were recorded watching 16 emotion-eliciting film clips while asked to attempt to suppress their facial expressions,’ Tomas explains.

‘They were told that experimenters are watching their face and that if their facial expression leaks and the experimenter guesses the clip they are  watching correctly, they will be asked to fill in a dull 500-question survey. This induced 77 micro-expressions in 6 (now 21) subjects.’

To overcome the problem of the limited number of frames the researchers used a temporal interpolation method where each micro-expression is interpolated - essentially ‘gaps’ in the data are filled in with existing data - across a larger number of frames. This makes it possible to detect micro-expressions even with a standard camera.

Early results from the work are promising, with the automated method able to detect micro-expressions better than a human, Tomas comments:

‘The human detection accuracies reported in literature are significantly lower than our 79% accuracy. We are currently running human micro-expression recognition experiments on our data to get a directly comparable human accuracy.’

But the writing may not be on the wall for liars and con-artists just yet.

Automated recognition of micro-expressions is one thing, Tomas says, but detecting deception, and uncovering the truth, is considerably harder:

Micro-expressions should be treated only as clues that a person is hiding something, not as conclusive evidence for deception. They cannot indicate what that person is hiding or why they are attempting to conceal it.

Tomas adds: ‘That said, our initial experiments do indicate that our approach can distinguish deceptive from truthful micro-expressions, but we will need to conduct further experiments to confirm this.’

Explore further: How could Santa know if you've been good or bad?

Related Stories

How could Santa know if you've been good or bad?

December 21, 2009

( -- By using technology to detect guilty expressions, of course. CSIRO is using automated expression recognition technology to tell whether someone is in pain and, according to computer scientist, CSIRO’s Dr ...

Older people not as good at lying or detecting lies: study

May 27, 2011

(Medical Xpress) -- Older people cannot lie as convincingly as younger people, are worse at detecting when others are lying, and the latter is linked to age-related decline in emotion recognition, new University of Otago ...

Expressionless faces provide clues on how we read emotions

April 15, 2010

( -- With smiles, grimaces or raised eyebrows, most of us show our feelings on our faces, but people with Moebius syndrome, a rare condition that causes facial paralysis, can't make any facial expressions at all. ...

Researcher develops image processing system that detects moods

December 2, 2008

Who knows what evil lurks in the hearts of men? Dr. Prabir Bhattacharya and his computers might. He and Concordia graduate student Abu Sayeed Sohail are developing a computer image processing system that detects and classifies ...

Recommended for you

Startup Pi out to slice the charging cord

September 19, 2017

Silicon Valley youngster Pi on Monday claimed it had developed the world's first wireless charger that does away with cords or mats to charge devices.


Adjust slider to filter visible comments by rank

Display comments: newest first

not rated yet Nov 23, 2011
I would assume the use of Botox will increase among terrorists, when this thing becomes more common.
not rated yet Nov 23, 2011
Those of us with Asperger's Syndrome have great difficulty reading others' emotions, even when they are NOT trying to conceal them. This could become a very useful tool for us.
not rated yet Nov 23, 2011
In addition to using Botox, you could simply use people with anhedonia, as this skews the emotional responses away from the norm.
not rated yet Nov 23, 2011
So basically, if I were trying to get on a plane for my mom's funeral, and I told them I was just going for a family visit (I wouldn't want to discuss the funeral with a retarded TSA official) - the software would tell them I'm concealing something, and I'll get interrogated???

This comes perilously close to violating the sanctity of a person's mind... People should be able to conceal something if they want to. The government has no right to probe our innermost thoughts.
not rated yet Nov 23, 2011
This comes perilously close to violating the sanctity of a person's mind... People should be able to conceal something if they want to. The government has no right to probe our innermost thoughts.

I always just go with the aluminum-foil brain-shielding helmet. So far it has worked, mostly. The men in suits still follow me though.
1 / 5 (7) Nov 23, 2011
Lol It'll be great the first time someone gets busted for perjury based on the fact that "we definitely know so and so told a lie, but we aren't actually sure what the truth is."

Or even invoking one's fifth amendment right against self incrimination.

Judge: "Please answer the question."
Witness: "I'd like to invoke my fifth amendment rights."
Judge: "Would you be incriminating yourself?"
Witness: "Yes, of course."
Bailiff: "Your Honor, the witness is lying."
Judge: "Please answer the question."

Interesting times.

1 / 5 (7) Nov 23, 2011
On second thought this could actually be a good thing. Coupled with sufficiently advanced AI the judicial system could be replaced with objective machinery. Isn't that sort of the goal in the first place?
not rated yet Nov 24, 2011
Finally, the researchers turned their camera on a sampling of American news media. Unfortunately, technical issues prevented completion of the test when the computer's CPU reached dangerous levels of overheating.
1 / 5 (1) Nov 27, 2011
Coupled with sufficiently advanced AI the judicial system could be replaced with objective machinery.
The robots won't make it past the metal detectors. Case adjourned.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.