Software that recognizes behavior patterns developed to improve computer tracking of human activity

October 11, 2012, University of Michigan
Bourne pursuit: Improving computer tracking of human activity

(—Contrary to what you might see in police dramas, you don't have to be Jason Bourne to shake off a computer tracking you through a video feed. Cross paths with someone who vaguely resembles you, and the computer is likely to swap your labels.

But researchers at the University of Michigan have found a way to improve a computer's human-tracking accuracy by more than 30 percent by looking not only at where the targets are going, but also at what they're doing.

"By creating that understands which activity a person or a group of people is performing, we can obtain much more robust and stable tracking results," says Silvio Savarese, professor of electrical engineering and computer sciences. "This is a new way of solving the tracking problem and can potentially revolutionize the way researchers look at the tracking problem in general."

Savarese and doctoral student Wongun Choi conducted the research.

In order to say anything useful about what its video feeds sees, a computer needs to be able to track people and things moving through the scene. Already, computers can identify and track people who are standing or walking, but camera movements and obstacles that temporarily hide targets can throw them off. In order to make computer tracking more reliable, Savarese's team taught the software to recognize interactions such as people walking together, standing in a line or crossing a street.

The of an individual give information about his or her interactions, and the interactions can predict future behavior of an individual. For instance, when two people appear to be walking and talking together, the computer can connect their tracks. If the pair then pass behind an obstacle with a third person, the computer now has the to predict that when the individuals reappear, the two in conversation will probably still be together.

If such advanced software tries to follow the behaviors of targets with "brute force"—that is, by considering every possible interaction that could occur and deciding which is most likely—Savarese says that it could take years to give its tracking solution. Ideally, for applications like vehicle collision prevention, such software should run in real time.

To speed things up, the team taught their software to think more like a human. They fed it example videos with targets and behaviors labeled. That way, the computer could capitalize on the analytical strength of our brains: recognizing patterns based on previous experience.

"Our method reduces the computational complexity and makes it possible to solve the problem of inferring what a person will do based on their activities as an individual, their interactions with other individuals and their behavior in larger groups," Savarese says.

His research team will continue to speed up the process, and he reports that a simplified version of the tracking software is approaching real-time operation.

The problem of tracking targets is significant in many fields, from robot vision to observing animal herds in the wild and picking out suspicious activity in a crowd. The application Savarese has in mind would aid drivers, keeping an eye on the pedestrians and sounding an alarm or braking if one of them is about to take an unexpected step into the street.

Explore further: Training eye movement may reduce driver distraction

More information: Silvio Savarese:

Related Stories

Training eye movement may reduce driver distraction

June 8, 2010

More than 16 people are killed and more than 1,300 people are injured each day in crashes involving a distracted driver, a phenomenon that could be reduced with the right application of motion information and appropriate ...

App helps San Fran police track stolen iPhone fast

July 22, 2010

(AP) -- A man accused of swiping an Apple iPhone out of a woman's hand in San Francisco may have been shocked when police found him only nine minutes later. It turns out the phone had been tracking his every move.

'Smart' surveillance system may tag suspicious or lost people

December 17, 2008

Engineers here are developing a computerized surveillance system that, when completed, will attempt to recognize whether a person on the street is acting suspiciously or appears to be lost. Intelligent video cameras, large ...

Watching me, watching you

October 21, 2009

( -- Software that tracks shoppers' eye movements as they browse supermarket shelves may seem a bit Big Brother, but the latest technology in 'eye-tracking', which monitors what grabs a person's attention, could ...

Scientists crack code on tracking zebras

May 25, 2011

Field biologists following thousands of wild zebras in Africa used to joke about how nice it would be to have a bar code reader to help them identify and catalogue individual animals.

Recommended for you

Researchers find tweeting in cities lower than expected

February 20, 2018

Studying data from Twitter, University of Illinois researchers found that less people tweet per capita from larger cities than in smaller ones, indicating an unexpected trend that has implications in understanding urban pace ...

Augmented reality takes 3-D printing to next level

February 20, 2018

Cornell researchers are taking 3-D printing and 3-D modeling to a new level by using augmented reality (AR) to allow designers to design in physical space while a robotic arm rapidly prints the work.


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.