New computer program first to recognise sketches more accurately than a human

July 21, 2015
The program could successfully identify a seagull, pigeon, flying bird and standing bird better than humans. Credit: Mathias Eitz, James Hays and Marc Alexa

Known as Sketch-a-Net, the program is capable of correctly identifying the subject of sketches 74.9 per cent of the time compared to humans that only managed a success rate of 73.1 per cent.

As sketching becomes more relevant with the increase in the use of touchscreens, the development could provide a foundation for new ways to interact with computers.

Touchscreens could understand what you are drawing enabling you to retrieve a specific image by drawing it with your fingers, which is more natural than keyword searches for finding items such as furniture or fashion accessories. The improvement could also aid police forensics when an artist's impression of a criminal needs to be matched to a mugshot or CCTV database.

The research, which was accepted at the British Machine Vision Conference, also showed that the program performed better at determining finer details in . For example, it was able to successfully distinguish the specific bird variants 'seagull', 'flying-bird', 'standing-bird' and 'pigeon' with 42.5 per cent accuracy compared to humans that only achieved 24.8 per cent.

Sketches are very intuitive to humans and have been used as a communication tool for thousands of years but recognising free-hand sketches is challenging because they are abstract, varied and consist of black and white lines rather than coloured pixels like a photo. Solving sketch recognition will lead to a greater scientific understanding of visual perception.

Sketch-a-Net is a 'deep neural network' – a type of computer program designed to emulate the processing of the human brain. It is particularly successful because it accommodates the unique characteristics of sketches, particularly the order the strokes were drawn. This was information that was previously ignored but is especially important for understanding drawings on touchscreens.

Timothy Hospedales, co-author of the study and Lecturer in the School of Electronic Engineering and Computer Science, QMUL, said: "It's exciting that our can solve the task even better than humans can. Sketches are an interesting area to study because they have been used since pre-historic times for communication and now, with the increase in use of touchscreens, they are becoming a much more common communication tool again. This could really have a huge impact for areas such as police forensics, touchscreen use and image retrieval, and ultimately will help us get to the bottom of visual understanding."

Explore further: Computer program can identify sketches

More information: "Sketch-a-Net that Beats Humans." arxiv.org/abs/1501.07873

Related Stories

Computer program can identify sketches

September 13, 2012

(Phys.org)—Computers are good at speed, numbers, and massive amounts of data, but understanding the content of a simple drawing is more difficult. Researchers at Brown and the Technical University of Berlin have produced ...

Method developed to match police sketch, mug shot

March 3, 2011

(PhysOrg.com) -- The long-time practice of using police facial sketches to nab criminals has been, at best, an inexact art. But the process may soon be a little more exact thanks to the work of some Michigan State University ...

New website can identify birds using photos

June 4, 2015

In a breakthrough for computer vision and for bird watching, researchers and bird enthusiasts have enabled computers to achieve a task that stumps most humans—identifying hundreds of bird species pictured in photos.

Recommended for you

A not-quite-random walk demystifies the algorithm

December 15, 2017

The algorithm is having a cultural moment. Originally a math and computer science term, algorithms are now used to account for everything from military drone strikes and financial market forecasts to Google search results.

US faces moment of truth on 'net neutrality'

December 14, 2017

The acrimonious battle over "net neutrality" in America comes to a head Thursday with a US agency set to vote to roll back rules enacted two years earlier aimed at preventing a "two-speed" internet.

FCC votes along party lines to end 'net neutrality' (Update)

December 14, 2017

The Federal Communications Commission repealed the Obama-era "net neutrality" rules Thursday, giving internet service providers like Verizon, Comcast and AT&T a free hand to slow or block websites and apps as they see fit ...

The wet road to fast and stable batteries

December 14, 2017

An international team of scientists—including several researchers from the U.S. Department of Energy's (DOE) Argonne National Laboratory—has discovered an anode battery material with superfast charging and stable operation ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.