Teaching a computer to perceive the world without human input

Sep 23, 2013 by Marlene Cimons

Humans can see an object—a chair, for example—and understand what they are seeing, even when something about it changes, such as its position. A computer, on the other hand, can't do that. It can learn to recognize a chair, but can't necessarily identify a different chair, or even the same chair if its angle changes.

"If I show a kid a chair, he will know it's a chair, and if I show him a different chair, he can still figure out that it's a chair," says Ming-Hsuan Yang, an assistant professor of and at the University of California, Merced. "If I change the angle of the chair 45 degrees, the will be different, but the kid will still be able to recognize it. But teaching a computer to see things is very difficult. They are very good at processing numbers, but not good at generalizing things."

Yang's goal is to change this. He is developing that he hopes will give computers, using a single camera, the ability to detect, track and recognize objects, including scenarios where the items drift, disappear, reappear or when other objects obscure them. The goal is to simulate without human input.

Most humans effortlessly can locate in a wide range of environments, since they are continually about the things they see, but it is a challenge for computers. Yang hopes the algorithms he's developing will enable computers to do the same thing, that is, continually amass information about the objects they are tracking.

"While it is not possible to enumerate all possible appearance variation of objects, it is possible to teach computers to interpolate from a wide range of training samples, thereby enabling machines to perceive the world," he says.

Currently, "for a computer, an image is composed of a long string of numbers," Yang says. "If the chair moves, the numbers for those two will be very different. What we want to do is generalize all the examples from a large amount of data, so the computer will still be able to recognize it, even when it changes. How do we know when we have enough data? We cannot encompass all the possibilities, so we are trying to define 'chair' in terms of its functionalities."

Potentially, computers that can "see" and track moving objects could improve assistive technology for the visually impaired, and also could have applications in medicine, such as locating and following cells; in tracking insect and animal motion; in traffic modeling for "smart" buildings, and improved navigation and surveillance in robots.

"For the visually impaired, the most important things are depth and obstacles," Yang says. "This could help them see the world around them. They don't need to see very far away, just to see whether there are obstacles near them, two or three feet away. The computer program, for example, could be in a cane. The camera would be able to create a 3-D world and give them feedback. The computer can tell them that the surface is uneven, so they will know, or sense a human or a car in front of them."

Yang is conducting his research under a National Science Foundation Faculty Early Career Development (CAREER) award, which he received in 2012. The award supports junior faculty who exemplify the role of teacher-scholars through outstanding research, excellent education and the integration of education, and research within the context of the mission of their organization. He is receiving $473,797 over five years.

Yang's project also includes developing a code library of tracking algorithms and a large data set, which will become publicly available. The grant also provides for an educational component that will involve both undergraduate and graduate students, with an emphasis on encouraging underrepresented minority groups from California's Central Valley to study computer sciences and related fields. The goal is to integrate computer vision material in undergraduate courses so that students will want to continue studying in the field.

Additionally, Yang is helping several undergraduate students design vision applications for mobile phones, and trying to write programs that will enable computers to infer depth and distance, as well as to interpret the images it "sees."

"It is not clear exactly how human vision works, but one way to explain visual perception of depth is based on people's two eyes and trigonometry," he says. "By figuring out the geometry of the points, we can figure out depth. We do it all the time, without thinking. But for computers, it's still very difficult to do that.

"The Holy Grail of computer vision is to tell a story using an image or video, and have the computer understand on some level what it is seeing," he adds. "If you give an image to a kid, and ask the kid to tell a story, the kid can do it. But if you ask a computer program to do it, now it can only do a few primitive things. A kid already has the cognitive knowledge to tell a story based on the image, but the computer just sees things as is, but doesn't have any background information. We hope to give the some interpretation, but we aren't there yet."

Explore further: Computer scientists can predict the price of Bitcoin

add to favorites email to friend print save as pdf

Related Stories

Safe navigation for visually impaired persons

Sep 09, 2013

Blind and visually impaired persons find it very difficult to move about in unknown environments. The white canes used by them are still the preferred tools for orientation. The project "A Mobility and Navigational ...

Crowdsourcing creates a database of surfaces

Aug 28, 2013

(Phys.org) —Computer graphics are moving off the movie screen and into everyday life. Home remodeling specialists, for example, may soon be able to to show you how your kitchen would look with marble countertops ...

Micro cameras flex their way into the future of imaging

Sep 20, 2013

Imagine sticking a thin sheet of microscopic cameras to the surface of a car to provide a rear-view image, or wrapping that sheet around a pole to provide 360-degree surveillance of an intersection under construction.

Recommended for you

Tablets, cars drive AT&T wireless gains—not phones

6 hours ago

AT&T says it gained 2 million wireless subscribers in the latest quarter, but most were from non-phone services such as tablets and Internet-connected cars. The company is facing pricing pressure from smaller rivals T-Mobile ...

Twitter looks to weave into more mobile apps

6 hours ago

Twitter on Wednesday set out to weave itself into mobile applications with a free "Fabric" platform to help developers build better programs and make more money.

Blink, point, solve an equation: Introducing PhotoMath

7 hours ago

"Ma, can I go now? My phone did my homework." PhotoMath, from the software development company MicroBlink, will make the student's phone do math homework. Just point the camera towards the mathematical expression, ...

Google unveils app for managing Gmail inboxes

7 hours ago

Google is introducing an application designed to make it easier for its Gmail users to find and manage important information that can often become buried in their inboxes.

User comments : 1

Adjust slider to filter visible comments by rank

Display comments: newest first

1 / 5 (1) Sep 25, 2013
Perhaps it really takes a genius to understand what in terms of innovation threaten's national security when it come's to intellectual property concerning cognition?

Isn't it obvious that AI has advanced far more then is public knowledge- that proprietary/government neural evolving synthetic cognizer's are not just arguably feasible but rather absurd to pretend to be not yet available? As a retiree from the field of real competitive intelligence I'm certain of this. Chess and 'driving' are 80's challenge's- well, prior millenia let's say as by the 60's chess was just about making it portable and pretense that we need a 3d model of the world many seconds down the road within millimeter's, essentially what archeologists do for tall stone chappel's in the past be done for goobot's we use to take trot's is silly- that's useful, but not just for driving. Driving doesn't need that at all in fact. Democracy is endangered by government having infinite intelligence in it's closet's.