Computer vision may not be as good as thought

Jan 25, 2008
Computer vision may not be as good as thought
The human brain easily recognizes that these cars are all the same object, but the variations in the car's size, orientation and position are a challenge for computer-vision algorithms. Image / Nicolas Pinto

For years, scientists have been trying to teach computers how to see like humans, and recent research has seemed to show computers making progress in recognizing visual objects. A new MIT study, however, cautions that this apparent success may be misleading because the tests being used are inadvertently stacked in favor of computers.

Computer vision is important for applications ranging from “intelligent” cars to visual prosthetics for the blind. Recent computational models show apparently impressive progress, boasting 60-percent success rates in classifying natural photographic image sets. These include the widely used Caltech101 database, intended to test computer vision algorithms against the variety of images seen in the real world.

However, James DiCarlo, a neuroscientist in the McGovern Institute for Brain Research at MIT, graduate student Nicolas Pinto and David Cox of the Rowland Harvard Institute argue that these image sets have design flaws that enable computers to succeed where they would fail with more authentically varied images. For example, photographers tend to center objects in a frame and to prefer certain views and contexts. The visual system, by contrast, encounters objects in a much broader range of conditions.

“The ease with which we recognize visual objects belies the computational difficulty of this feat,” explains DiCarlo, senior author of the study in the online Jan. 25 PLoS Computational Biology. “The core challenge is image variation. Any given object can cast innumerable images onto the retina depending on its position, distance, orientation, lighting and background.”

The team exposed the flaws in current tests of computer object recognition by using a simple “toy” computer model inspired by the earliest steps in the brain's visual pathway. Artificial neurons with properties resembling those in the brain's primary visual cortex analyze each point in the image and capture low-level information about the position and orientation of line boundaries. The model lacks the more sophisticated analysis that happens in later stages of visual processing to extract information about higher-level features of the visual scene such as shapes, surfaces or spaces between objects.

The researchers intended this model as a straw man, expecting it to fail as a way to establish a baseline. When they tested it on the Caltech101 images, however, the model did surprisingly well, with performance similar or better than five state-of-the-art object-recognition systems.

How could that be? “We suspected that the supposedly natural images in current computer vision tests do not really engage the central problem of variability, and that our intuitions about what makes objects hard or easy to recognize are incorrect,” Pinto explains.

To test this idea, the authors designed a more carefully controlled test. Using just two categories-planes and cars-they introduced variations in position, size and orientation that better reflect the range of variation in the real world.

“With only two types of objects to distinguish, this test should have been easier for the 'toy' computer model, but it proved harder,” Cox says. The team's conclusion: “Our model did well on the Caltech101 image set not because it is a good model but because the 'natural' images fail to adequately capture real-world variability.”

As a result, the researchers argue for revamping the current standards and images used by the computer-vision community to compare models and measure progress. Before computers can approach the performance of the human brain, they say, scientists must better understand why the task of object recognition is so difficult and the brain's abilities are so impressive.

Source: Massachusetts Institute of Technology

Explore further: Napster co-founder to invest in allergy research

add to favorites email to friend print save as pdf

Related Stories

Dragonflies on the hunt display complex choreography

Dec 10, 2014

The dragonfly is a swift and efficient hunter. Once it spots its prey, it takes about half a second to swoop beneath an unsuspecting insect and snatch it from the air. Scientists at the Howard Hughes Medical ...

Image descriptions from computers show gains

Nov 18, 2014

"Man in black shirt is playing guitar." "Man in blue wetsuit is surfing on wave." "Black and white dog jumps over bar." The picture captions were not written by humans but through software capable of accurately ...

Recommended for you

Study reveals mature motorists worse at texting and driving

3 hours ago

A Wayne State University interdisciplinary research team in the Eugene Applebaum College of Pharmacy and Health Sciences has made a surprising discovery: older, more mature motorists—who typically are better drivers in ...

Napster co-founder to invest in allergy research

Dec 17, 2014

(AP)—Napster co-founder Sean Parker missed most of his final year in high school and has ended up in the emergency room countless times because of his deadly allergy to nuts, shellfish and other foods.

LA mayor plans 7,000 police body cameras in 2015

Dec 16, 2014

Mayor Eric Garcetti announced a plan Tuesday to equip 7,000 Los Angeles police officers with on-body cameras by next summer, making LA's police department the nation's largest law enforcement agency to move ...

Merriam-Webster names 'culture' word of the year

Dec 15, 2014

A nation, a workplace, an ethnicity, a passion, an outsized personality. The people who comprise these things, who fawn or rail against them, are behind Merriam-Webster's 2014 word of the year: culture.

In Curiosity Hacked, children learn to make, not buy

Dec 14, 2014

With her right hand, my 8-year-old daughter, Kalian, presses the red-hot soldering iron against the circuit board. With her left hand, she guides a thin, tin wire until it's pressing against both the circuit board and the ...

User comments : 0

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.