Surveillance tech from Carnegie Mellon can watch and predict

October 28, 2012 by Nancy Owano report
Surveillance cameras
Credit: Wikipedia

(—Yet another flavor of surveillance technology belongs to devices that can not only eyeball but perform "activity forecasting." At Carnegie Mellon, the Mind's Eye program is creating intelligent software that will recognize human activities in video and predict what might happen next. Earlier this week, new research from Carnegie Mellon presented a security camera system that can watch and predict what a person will do next with specially programmed software. The system would eventually sound an alarm if it recognized that an action was not permitted, detecting anomalous behaviors. One example of such a scenario would be the cameras at an airport or bus station, flagging a bag abandoned for more than a few minutes.

The Carnegie Mellon pair disclosed details about their Army-funded research in a paper earlier this week, at the Semantic Technology for Intelligence, Defense, and Security conference at George Mason University. Their paper, "Using Ontologies in a Cognitive-Grounded System: Automatic Action Recognition in Video Surveillance," presents the knowledge infrastructure of a high-level artificial visual . Their research focused on a cognitive engine for to automatically detect and interpret a person's actions through a surveillance feed.

Their automated approach could one day tempt security decision makers to reconsider using humans to monitor cameras, as human operators are not only expensive to maintain but present risks if distracted or drowsy. Scientists in the past have discussed the fact that technology has progressed to a point where mounting cameras to capture video imagery is economical; the cost factor comes in the form of hiring humans to watch the imagery. Also, cameras that do nothing but watch can only provide information after a crime has occurred. Investigators may get to see how a car was stolen or a store clerk robbed but after the fact, whereas the researchers' approach is designed to help prevent crimes or dangerous events from happening.

The researchers said that to approximate human visual intelligence In order to achieve the desired level of complexity, computer vision algorithms need to be complemented with higher-level tools of analysis involving knowledge representation and reasoning, often under conditions of uncertainty.

Alessandro Oltramari, a postdoctoral researcher and Christian Lebiere, both from the Department of Psychology at Carnegie Mellon, suggest that this automated approach could find applications both in military and civil environments.

Their technology leverages the advances that have been made so far, over the years, in recognizing objects and their properties. Their cognitive engine also incorporates research on activity forecasting conducted by a team led by postdoctoral fellow Kris Kitani. That team has tried to understand what humans will do by calculating which physical trajectories are most likely. "We address the task of inferring the future actions of people from noisy visual input. We denote this task activity forecasting," said Kitani and his colleagues in a research paper on the subject.

Explore further: New Technologies Improve Video Surveillance

More information: … veGroundedSystem.pdf

Related Stories

New Technologies Improve Video Surveillance

December 14, 2006

Surveillance cameras are sprouting up in more and more places, forming an ever more powerful tool for solving crimes after they happen. But what about using them to prevent or stop criminal and terrorist acts? This requires ...

System improves automated monitoring of security cameras

June 4, 2012

Police and security teams guarding airports, docks and border crossings from terrorist attack or illegal entry need to know immediately when someone enters a prohibited area, and who they are. A network of surveillance cameras ...

Software: running commentary for smarter surveillance?

March 16, 2010

( -- Cutting-edge surveillance software that automatically detects human motion, behaviour and facial expressions, generates a running commentary of what?s happening and re-enacts events virtually could soon be ...

Startup lets webcams detect people

February 8, 2010

Vitamin D Video on Monday released a finished version of software that detects people in surveillance footage recorded by common Web cameras.

Recommended for you

Automated safety systems are preventing car crashes

August 23, 2017

Safety systems to prevent cars from drifting into another lane or that warn drivers of vehicles in their blind spots are beginning to live up to their potential to significantly reduce crashes, according to two studies released ...

Newest solar cells underperform in cloudy countries

August 22, 2017

To determine how efficient new solar cells convert sunlight into electricity, small sample cells are tested under ideal conditions. However, the reported efficiency is not very representative of the actual annual yield when ...

Google to serve next version of Android as 'Oreo"

August 22, 2017

An upcoming update to Google's Android software finally has a delectable name. The next version will be known as Oreo, extending Google's tradition of naming each version after a sweet treat.

Forget oil, Russia goes crazy for cryptocurrency

August 16, 2017

Standing in a warehouse in a Moscow suburb, Dmitry Marinichev tries to speak over the deafening hum of hundreds of computers stacked on shelves hard at work mining for crypto money.

Researchers clarify mystery about proposed battery material

August 15, 2017

Battery researchers agree that one of the most promising possibilities for future battery technology is the lithium-air (or lithium-oxygen) battery, which could provide three times as much power for a given weight as today's ...


Adjust slider to filter visible comments by rank

Display comments: newest first

2.3 / 5 (3) Oct 28, 2012
First it was politically correct "zero tolerance" as euphemism for mindlessness. Now the mind is removed in favor of Future Crime mindless precognition.
1 / 5 (1) Oct 28, 2012
Whatever. This is cool!
1 / 5 (1) Oct 29, 2012
Activity forecasting = preempting activity.

As effective as preempting weather or climate and predicting their forecasts.


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.