Surveillance tech from Carnegie Mellon can watch and predict

October 28, 2012 by Nancy Owano report
Surveillance cameras
Credit: Wikipedia

(—Yet another flavor of surveillance technology belongs to devices that can not only eyeball but perform "activity forecasting." At Carnegie Mellon, the Mind's Eye program is creating intelligent software that will recognize human activities in video and predict what might happen next. Earlier this week, new research from Carnegie Mellon presented a security camera system that can watch and predict what a person will do next with specially programmed software. The system would eventually sound an alarm if it recognized that an action was not permitted, detecting anomalous behaviors. One example of such a scenario would be the cameras at an airport or bus station, flagging a bag abandoned for more than a few minutes.

The Carnegie Mellon pair disclosed details about their Army-funded research in a paper earlier this week, at the Semantic Technology for Intelligence, Defense, and Security conference at George Mason University. Their paper, "Using Ontologies in a Cognitive-Grounded System: Automatic Action Recognition in Video Surveillance," presents the knowledge infrastructure of a high-level artificial visual . Their research focused on a cognitive engine for to automatically detect and interpret a person's actions through a surveillance feed.

Their automated approach could one day tempt security decision makers to reconsider using humans to monitor cameras, as human operators are not only expensive to maintain but present risks if distracted or drowsy. Scientists in the past have discussed the fact that technology has progressed to a point where mounting cameras to capture video imagery is economical; the cost factor comes in the form of hiring humans to watch the imagery. Also, cameras that do nothing but watch can only provide information after a crime has occurred. Investigators may get to see how a car was stolen or a store clerk robbed but after the fact, whereas the researchers' approach is designed to help prevent crimes or dangerous events from happening.

The researchers said that to approximate human visual intelligence In order to achieve the desired level of complexity, computer vision algorithms need to be complemented with higher-level tools of analysis involving knowledge representation and reasoning, often under conditions of uncertainty.

Alessandro Oltramari, a postdoctoral researcher and Christian Lebiere, both from the Department of Psychology at Carnegie Mellon, suggest that this automated approach could find applications both in military and civil environments.

Their technology leverages the advances that have been made so far, over the years, in recognizing objects and their properties. Their cognitive engine also incorporates research on activity forecasting conducted by a team led by postdoctoral fellow Kris Kitani. That team has tried to understand what humans will do by calculating which physical trajectories are most likely. "We address the task of inferring the future actions of people from noisy visual input. We denote this task activity forecasting," said Kitani and his colleagues in a research paper on the subject.

Explore further: New Technologies Improve Video Surveillance

More information:

Related Stories

New Technologies Improve Video Surveillance

December 14, 2006

Surveillance cameras are sprouting up in more and more places, forming an ever more powerful tool for solving crimes after they happen. But what about using them to prevent or stop criminal and terrorist acts? This requires ...

System improves automated monitoring of security cameras

June 4, 2012

Police and security teams guarding airports, docks and border crossings from terrorist attack or illegal entry need to know immediately when someone enters a prohibited area, and who they are. A network of surveillance cameras ...

Software: running commentary for smarter surveillance?

March 16, 2010

( -- Cutting-edge surveillance software that automatically detects human motion, behaviour and facial expressions, generates a running commentary of what?s happening and re-enacts events virtually could soon be ...

Startup lets webcams detect people

February 8, 2010

Vitamin D Video on Monday released a finished version of software that detects people in surveillance footage recorded by common Web cameras.

Recommended for you

Windows update will bring 3-D, game tools and doodling

March 29, 2017

A major update to Microsoft's Windows 10 system will start reaching consumers and businesses on April 11, offering 3-D drawing tools, game-broadcasting capabilities and better ways to manage your web browsing.


Adjust slider to filter visible comments by rank

Display comments: newest first

2.3 / 5 (3) Oct 28, 2012
First it was politically correct "zero tolerance" as euphemism for mindlessness. Now the mind is removed in favor of Future Crime mindless precognition.
1 / 5 (1) Oct 28, 2012
Whatever. This is cool!
1 / 5 (1) Oct 29, 2012
Activity forecasting = preempting activity.

As effective as preempting weather or climate and predicting their forecasts.


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.