Carnegie Mellon method uses network of cameras to track people in complex indoor settings

June 12th, 2013
Researchers at Carnegie Mellon University were able to track and map the location of multiple people in a nursing home using a network of 15 cameras. Here, the people tracked by five of those cameras are plotted on a map of the nursing facility Credit: Carnegie Mellon University
Researchers at Carnegie Mellon University have developed a method for tracking the locations of multiple individuals in complex, indoor settings using a network of video cameras, creating something similar to the fictional Marauder's Map used by Harry Potter to track comings and goings at the Hogwarts School.

The method was able to automatically follow the movements of 13 people within a nursing home, even though individuals sometimes slipped out of view of the cameras. None of Potter's magic was needed to track them for prolonged periods; rather, the researchers made use of multiple cues from the video feed: apparel color, person detection, trajectory and, perhaps most significantly, facial recognition.

Multi-camera, multi-object tracking has been an active field of research for a decade, but automated techniques have only focused on well-controlled lab environments. The Carnegie Mellon team, by contrast, proved their technique with actual residents and employees in a nursing facility—with camera views compromised by long hallways, doorways, people mingling in the hallways, variations in lighting and too few cameras to provide comprehensive, overlapping views.

The performance of the Carnegie Mellon algorithm significantly improved on two of the leading algorithms in multi-camera, multi-object tracking. It located individuals within one meter of their actual position 88 percent of the time, compared with 35 percent and 56 percent for the other algorithms. The researchers— Alexander Hauptmann, principal systems scientist in the Computer Science Department (CSD); Shoou-I Yu, a Ph.D. student in the Language Technologies Institute; and Yi Yang, a CSD post-doctoral researcher—will present their findings June 27 at the Computer Vision and Pattern Recognition Conference in Portland, Ore.

Though Harry Potter could activate the Marauder's Map only by first solemnly swearing "I am up to no good," the Carnegie Mellon researchers developed their tracking technique as part of an effort to monitor the health of nursing home residents. "The goal is not to be Big Brother, but to alert the caregivers of subtle changes in activity levels or behaviors that indicate a change of health status," Hauptmann said. All of the people in this study consented to being tracked.

These automated tracking techniques also would be useful in airports, public facilities and other areas where security is a concern. Despite the importance of cameras in identifying perpetrators following this spring's Boston Marathon bombing and the 2005 London bombings, much of the video analysis necessary for tracking people continues to be done manually, Hauptmann noted.

The CMU work on monitoring nursing home residents began in 2005 as part of a National Institutes of Health-sponsored project called CareMedia, which is now associated with the Quality of Life Technology Center, a National Science Foundation engineering research center at CMU and the University of Pittsburgh.

"We thought it would be easy," Hauptmann said of multi-camera tracking, "but it turned out to be incredibly challenging." Something as simple as tracking based on color of clothing proved difficult, for instance, because the same color apparel can appear different to cameras in different locations, depending on variations in lighting. Likewise, a camera's view of an individual can often be blocked by other people passing in hallways, by furniture and when an individual enters a room or other area not covered by cameras, so individuals must be regularly re-identified by the system.

Face detection helps immensely in re-identifying individuals on different cameras. But Yang noted that faces can be recognized in less than 10 percent of the video frames. So the researchers developed mathematical models that enabled them to combine information, such as appearance, facial recognition and motion trajectories.

Using all of the information is key to the tracking process, but Yu said facial recognition proved to be the greatest help. When the researchers removed facial recognition information from the mix, their on-track performance in the nursing home data dropped from 88 percent to 58 percent, not much better than one of the existing tracking algorithms.

The nursing home video analyzed by the researchers was recorded in 2005 using 15 cameras; the recordings are just more than six minutes long. Further work will be necessary to extend the technique during longer periods of time and enable real-time monitoring. The researchers also are looking at additional ways to use video to monitor resident activity while preserving privacy, such as by only recording the outlines of people together with distance information from depth cameras similar to the Microsoft Kinect.

More information:

Provided by Carnegie Mellon University

This Science News Wire page contains a press release issued by an organization mentioned above and is provided to you “as is” with little or no review from Phys.Org staff.

More news stories

Gaia spies two temporarily magnified stars

While scanning the sky to measure the position of over one billion stars in our Galaxy, ESA's Gaia satellite has detected two rare instances of stars whose light was temporarily boosted by other celestial objects passing ...

Changing semiconductor properties at room temperature

It's a small change that makes a big difference. Researchers have developed a method that uses a one-degree change in temperature to alter the color of light that a semiconductor emits. The method, which uses a thin-film ...

Computer model is 'crystal ball' for E. coli bacteria

It's difficult to make predictions, especially about the future, and even more so when they involve the reactions of living cells—huge numbers of genes, proteins and enzymes, embedded in complex pathways and feedback loops. ...

Novel light sources made of 2-D materials

Physicists from the University of Würzburg have designed a light source that emits photon pairs, which are particularly well suited for tap-proof data encryption. The experiment's key ingredients: a semiconductor crystal ...

How planets like Jupiter form

Young giant planets are born from gas and dust. Researchers of ETH Zürich and the Universities of Zürich and Bern simulated different scenarios relying on the computing power of the Swiss National Supercomputing Centre ...

Science: Public interest high, literacy stable

While public interest in science continues to grow, the level of U.S. scientific literacy remains largely unchanged, according to a survey by the University of Michigan Institute for Social Research.