Software that recognizes behavior patterns developed to improve computer tracking of human activity

October 11, 2012
Bourne pursuit: Improving computer tracking of human activity

(—Contrary to what you might see in police dramas, you don't have to be Jason Bourne to shake off a computer tracking you through a video feed. Cross paths with someone who vaguely resembles you, and the computer is likely to swap your labels.

But researchers at the University of Michigan have found a way to improve a computer's human-tracking accuracy by more than 30 percent by looking not only at where the targets are going, but also at what they're doing.

"By creating that understands which activity a person or a group of people is performing, we can obtain much more robust and stable tracking results," says Silvio Savarese, professor of electrical engineering and computer sciences. "This is a new way of solving the tracking problem and can potentially revolutionize the way researchers look at the tracking problem in general."

Savarese and doctoral student Wongun Choi conducted the research.

In order to say anything useful about what its video feeds sees, a computer needs to be able to track people and things moving through the scene. Already, computers can identify and track people who are standing or walking, but camera movements and obstacles that temporarily hide targets can throw them off. In order to make computer tracking more reliable, Savarese's team taught the software to recognize interactions such as people walking together, standing in a line or crossing a street.

The of an individual give information about his or her interactions, and the interactions can predict future behavior of an individual. For instance, when two people appear to be walking and talking together, the computer can connect their tracks. If the pair then pass behind an obstacle with a third person, the computer now has the to predict that when the individuals reappear, the two in conversation will probably still be together.

If such advanced software tries to follow the behaviors of targets with "brute force"—that is, by considering every possible interaction that could occur and deciding which is most likely—Savarese says that it could take years to give its tracking solution. Ideally, for applications like vehicle collision prevention, such software should run in real time.

To speed things up, the team taught their software to think more like a human. They fed it example videos with targets and behaviors labeled. That way, the computer could capitalize on the analytical strength of our brains: recognizing patterns based on previous experience.

"Our method reduces the computational complexity and makes it possible to solve the problem of inferring what a person will do based on their activities as an individual, their interactions with other individuals and their behavior in larger groups," Savarese says.

His research team will continue to speed up the process, and he reports that a simplified version of the tracking software is approaching real-time operation.

The problem of tracking targets is significant in many fields, from robot vision to observing animal herds in the wild and picking out suspicious activity in a crowd. The application Savarese has in mind would aid drivers, keeping an eye on the pedestrians and sounding an alarm or braking if one of them is about to take an unexpected step into the street.

Explore further: Training eye movement may reduce driver distraction

More information: Silvio Savarese:

Related Stories

Training eye movement may reduce driver distraction

June 8, 2010

More than 16 people are killed and more than 1,300 people are injured each day in crashes involving a distracted driver, a phenomenon that could be reduced with the right application of motion information and appropriate ...

App helps San Fran police track stolen iPhone fast

July 22, 2010

(AP) -- A man accused of swiping an Apple iPhone out of a woman's hand in San Francisco may have been shocked when police found him only nine minutes later. It turns out the phone had been tracking his every move.

'Smart' surveillance system may tag suspicious or lost people

December 17, 2008

Engineers here are developing a computerized surveillance system that, when completed, will attempt to recognize whether a person on the street is acting suspiciously or appears to be lost. Intelligent video cameras, large ...

Watching me, watching you

October 21, 2009

( -- Software that tracks shoppers' eye movements as they browse supermarket shelves may seem a bit Big Brother, but the latest technology in 'eye-tracking', which monitors what grabs a person's attention, could ...

Scientists crack code on tracking zebras

May 25, 2011

Field biologists following thousands of wild zebras in Africa used to joke about how nice it would be to have a bar code reader to help them identify and catalogue individual animals.

Recommended for you

US faces moment of truth on 'net neutrality'

December 14, 2017

The acrimonious battle over "net neutrality" in America comes to a head Thursday with a US agency set to vote to roll back rules enacted two years earlier aimed at preventing a "two-speed" internet.

FCC votes along party lines to end 'net neutrality' (Update)

December 14, 2017

The Federal Communications Commission repealed the Obama-era "net neutrality" rules Thursday, giving internet service providers like Verizon, Comcast and AT&T a free hand to slow or block websites and apps as they see fit ...

The wet road to fast and stable batteries

December 14, 2017

An international team of scientists—including several researchers from the U.S. Department of Energy's (DOE) Argonne National Laboratory—has discovered an anode battery material with superfast charging and stable operation ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.