Seeing things: Researchers teach computers to recognize objects

Oct 13, 2009 by Larry Hardesty
A new object recognition algorithm treats unrelated images as if they were consecutive frames of video. Since it assumes that the objects in both images are the same, it tries to deform the objects in the first image until they map onto the objects in the second. If the objects in one of the images have already been outlined and labeled, the algorithm can simply transfer the labels to the other image. Courtesy of Ce Liu

( -- If computers could recognize objects, they could automatically search through hours of video footage for a particular two-minute scene. A tourist strolling down a street in a strange city could take a cell-phone photo of an unmarked monument and immediately find out what it was. And an Internet image search on, say, "Shakespeare" would pull up pictures of Shakespeare, not pictures of Gwyneth Paltrow in the movie Shakespeare in Love. Though object recognition is one of the major research topics in computer vision, MIT researchers may have found a way to make it much more practical.

Typically, object recognition algorithms need to be "trained" using in which objects have been outlined and labeled by hand. By looking at a million pictures of cars labeled "car," an can learn to recognize features shared by images of cars. The problem is that for every new class of objects — trees, buildings, telephone poles — the algorithm has to be trained all over again.

But Esther and Harold E. Edgerton Associate Professor of Electrical Engineering and Computer Science Antonio Torralba and Computer Science and Lab graduate students Ce Liu, PhD '09, and Jenny Yuen have developed an object recognition system that doesn't require any training. Nonetheless, it still identifies objects with 50 percent greater accuracy than the best prior algorithm.

Credits - Images courtesy of Ce Liu

The system uses a modified version of a so-called motion estimation algorithm, a type of algorithm common in video processing. Since consecutive frames of video usually change very little, data compression schemes often store the unchanging aspects of a scene once, updating only the positions of moving objects. The motion estimation algorithm determines which objects have moved from one frame to the next. In a video, that's usually fairly easy to do: most objects don't move very far in one-30th of a second. Nor does the algorithm need to know what the object is; it just has to recognize, say, corners and edges, and how their appearance typically changes under different perspectives.

The MIT researchers' new system essentially treats unrelated images as if they were consecutive frames in a video sequence. When the modified motion estimation algorithm tries to determine which objects have "moved" between one image and the next, it usually picks out objects of the same type: it will guess, for instance, that the 2006 Infiniti in image two is the same object as the 1965 Chevy in image one.

If the first image comes from the type of database used to train computer vision systems, the Infiniti will already be labeled "car." The new system simply transfers the label to the Chevy.

Credits - Courtesy of Ce Liu

Credits - Courtesy of Ce Liu

The greater the resemblance of the labeled and unlabeled images, the better the algorithm works. Fortunately, Torralba's earlier work was largely directed toward amassing a huge database of labeled images. Torralba and his colleagues have developed a simple web-based system called LabelMe that lets online volunteers tag objects in digital images, and they also created a web site called 80 Million Tiny Images that sorts the images according to subject matter. When confronted with an unlabeled image, the new object recognition algorithm is likely to find something similar in Torralba's database. And as the database grows larger, that likelihood will only increase.

"It's a real commonsense solution to a fundamental problem in computer vision," says Marshall Tappen, a researcher at the University of Central Florida. "The results are great and better than you can get with much more complicated methods." Tappen adds that "a large database makes it possible to do lots of really interesting thing that no one's even envisioned. There are lots of interesting things it can do beyond just standard object recognition, so I think it's really going to enable a lot of innovation." Tappen points in particular to recent work on image editing and image completion done by Alyosha Efros at Carnegie Mellon University. "If you look at his last few Siggraph papers" — that is, papers presented at Siggraph, the major conference in the field of computer graphics — "they're all using LabelMe," Tappen says.

Provided by Massachusetts Institute of Technology (news : web)

Explore further: See what a child will look like using automated age-progression software (w/ video)

add to favorites email to friend print save as pdf

Related Stories

Researchers develop new image-recognition software

May 21, 2008

It takes surprisingly few pixels of information to be able to identify the subject of an image, a team led by an MIT researcher has found. The discovery could lead to great advances in the automated identification ...

New system estimates geographic location of photos

Jun 18, 2008

Researchers at Carnegie Mellon University have devised the first computerized method that can analyze a single photograph and determine where in the world the image likely was taken. It's a feat made possible ...

Computer vision may not be as good as thought

Jan 25, 2008

For years, scientists have been trying to teach computers how to see like humans, and recent research has seemed to show computers making progress in recognizing visual objects. A new MIT study, however, cautions ...

New algorithm improves robot vision

Dec 07, 2005

Except in fanciful movies like 2003's The Matrix Revolutions, where fearsome squid-like robots maneuvered with incredible ease, most robots are too clumsy to move around obstacles at high speeds. This is tru ...

Recommended for you

Dish Network denies wrongdoing in $2M settlement

8 hours ago

The state attorney general's office says Dish Network Corp. will reimburse Washington state customers about $2 million for what it calls a deceptive surcharge, but the satellite TV provider denies any wrongdoing.

Yahoo sees signs of growth in 'core' (Update)

8 hours ago

Yahoo reported a stronger-than-expected first-quarter profit Tuesday, results hailed by chief executive Marissa Mayer as showing growth in the Web giant's "core" business.

Intel reports lower 1Q net income, higher revenue

8 hours ago

Intel's earnings fell in the first three months of the year amid a continued slump in the worldwide PC market, but revenue grew slightly because of solid demand for tablet processors and its data center services.

Earthquake simulation tops one quadrillion flops

10 hours ago

A team of computer scientists, mathematicians and geophysicists at Technische Universitaet Muenchen (TUM) and Ludwig-Maximillians Universitaet Muenchen (LMU) have – with the support of the Leibniz Supercomputing ...

Twitter buys data analytics partner Gnip

11 hours ago

Twitter says it has bought its data partner Gnip, which provides analysis of the more than 500 million tweets its users share each day—to advertisers, academic institutions, politicians and other customers.

User comments : 1

Adjust slider to filter visible comments by rank

Display comments: newest first

not rated yet Oct 14, 2009
this seems to work very similar to how the eye process of sacades works to integrate multiple images for identifying descrete objects. good work.

More news stories

Intel reports lower 1Q net income, higher revenue

Intel's earnings fell in the first three months of the year amid a continued slump in the worldwide PC market, but revenue grew slightly because of solid demand for tablet processors and its data center services.

Low Vitamin D may not be a culprit in menopause symptoms

A new study from the Women's Health Initiative (WHI) shows no significant connection between vitamin D levels and menopause symptoms. The study was published online today in Menopause, the journal of The North American Menopa ...

Astronomers: 'Tilt-a-worlds' could harbor life

A fluctuating tilt in a planet's orbit does not preclude the possibility of life, according to new research by astronomers at the University of Washington, Utah's Weber State University and NASA. In fact, ...