The trouble with facial recognition technology (in the real world)

December 14, 2016 by Robin Kramer And Kay Ritchie, The Conversation
A face in a crowd in Mission: Impossible Rogue Nation.

If you've watched any sort of spy thriller or action film over the last few years – think Jason Bourne or Mission: Impossible – the chances are you've seen facial recognition software in action. These movie scenes often involve an artist's sketch compared to mug shots, or sometimes even a live CCTV stream, and with the clock ticking, a match is usually found for the culprit in the nick of time.

It seems natural then to assume that what happens in the film world is similar to what happens (most of the time) in the real world. We might think that our are constantly being tracked and recognised as we walk past security cameras in city centres – but this is not actually the case.

Not only would such a system require millions of cameras capable of producing high-quality footage, but it would also require the integration of photo-ID databases such as mugshots from every police force, previous passport images, and driving license images for everyone in the country.

And yet even if this high level of integration was possible, a far more basic problem still exists – facial recognition systems are still not 100% accurate.

Humans on top

In relatively easy situations, where faces are front-on and reasonably clear, computer algorithms can now outperform humans. But in much harder conditions when faces are pictured from different angles or in poor lighting, humans still come out on top.

A lot of this is because of how these types of systems work in a real-world setting. Generally a computer face recognition system will provide a "candidate list" of best guesses, and from this list a human operator makes the final decision. But because these lists tend to produce several very suitable lookalikes as the target, the operator's choice is a tough job – with error rates as high as 50%.

The trouble with facial recognition technology (in the real world)
Does the middle ID image belong to the person on the left or the right? (Answer: all three are the same person).

For similar security systems, such as the eGates used in several airports, information on their accuracy is unavailable. But there have been several cases of people being successfully "cleared" while using their partner's passport – which could well be a cause for concern.

Face in a crowd

When it comes to recognising a face, research shows people are near perfect when shown familiar faces but fare much worse when the faces are unfamiliar.

It seems, the difficulty arises from the enormous amount of variability we see across different images of the same person. So even minor changes – like the addition of a pair of glasses – can make us significantly worse. And most people seem to be unaware of their drop in performance.

In general, passport officers are no better than university students when matching unfamiliar people – and research shows that training in this area doesn't seem to make a difference – you've either got it or you haven't.

But there are also large individual differences in abilities over the population – from those who are clinically "face blind" to the highly accurate "super recognisers" – who are able to do just that, be super at recognising.

The trouble with facial recognition technology (in the real world)
Catching criminals in the new Jason Bourne film is still more fiction than fact. Screenshot, Author provided
Faces of the future

Looking to the future, for automatic systems there is increasing evidence that using "face averages" may improve identification rates. These averages are computer-generated images, made by combining several photographs of the same person.

Computer algorithms show an advantage when using these averages to compare to a target image. A simple smartphone system is also better at recognising a user if an average image is stored in its memory.

But it may well be that long-term, we choose to leave behind our belief that facial comparison is the solution – and instead embrace other means of identification.

This could include iris and retinal scans, fingerprints, and even signature matching – all of which are considered to be more accurate than using faces.

So as things currently stand, there's no need to be concerned about being recognised by cameras as you walk down the high street just yet. Because for the time being, you're still just a face in a crowd.

Explore further: Scientists show that a 'Superman' disguise could actually work

Related Stories

Forensic examiners pass the face matching test

September 1, 2015

The first study to test the skills of FBI agents and other law enforcers who have been trained in facial recognition has provided a reassuring result - they perform better than the average person or even computers on this ...

Recommended for you

Archaeologists discover Incan tomb in Peru

February 16, 2019

Peruvian archaeologists discovered an Incan tomb in the north of the country where an elite member of the pre-Columbian empire was buried, one of the investigators announced Friday.

Where is the universe hiding its missing mass?

February 15, 2019

Astronomers have spent decades looking for something that sounds like it would be hard to miss: about a third of the "normal" matter in the Universe. New results from NASA's Chandra X-ray Observatory may have helped them ...

What rising seas mean for local economies

February 15, 2019

Impacts from climate change are not always easy to see. But for many local businesses in coastal communities across the United States, the evidence is right outside their doors—or in their parking lots.

The friendly extortioner takes it all

February 15, 2019

Cooperating with other people makes many things easier. However, competition is also a characteristic aspect of our society. In their struggle for contracts and positions, people have to be more successful than their competitors ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.