New algorithm improves speed and accuracy of pedestrian detection

February 8, 2016
Pedestrian detection system was developed in the Statistical Visual Computing Lab at UC San Diego. Credit: Statistical Visual Computing Lab, UC San Diego

What if computers could recognize objects as well as the human brain could? Electrical engineers at the University of California, San Diego have taken an important step toward that goal by developing a pedestrian detection system that performs in near real-time (2-4 frames per second) and with higher accuracy (close to half the error) compared to existing systems. The technology, which incorporates deep learning models, could be used in "smart" vehicles, robotics and image and video search systems.

"We're aiming to build computer vision systems that will help computers better understand the world around them," said Nuno Vasconcelos, electrical engineering professor at the UC San Diego Jacobs School of Engineering who directed the research. A big goal is real-time vision, he says, especially for pedestrian detection systems in self-driving cars. Vasconcelos is a faculty affiliate of the Center for Visual Computing and the Contextual Robotics Institute, both at UC San Diego.

The new pedestrian detection developed by Vasconcelos and his team combines a traditional computer vision classification architecture, known as cascade detection, with models.

Pedestrian detection systems typically break down an image into small windows that are processed by a classifier that signals the presence or absence of a pedestrian. This approach is challenging because pedestrians appear in different sizes—depending on distance to the camera—and locations within an image. Typically, millions of windows must be inspected by video frame at speeds ranging from 5-30 frames per second.

In cascade detection, the detector operates throughout a series of stages. In the first stages, the algorithm quickly identifies and discards windows that it can easily recognize as not containing a person (such as the sky). The next stages process the windows that are harder for the algorithm to classify, such as those containing a tree, which the algorithm could recognize as having person-like features (shape, color, contours, etc.). In the final stages, the algorithm must distinguish between a pedestrian and very similar objects. However, because the final stages only process a few windows, the overall complexity is low.

Traditional cascade detection relies on "weak learners," which are simple classifiers, to do the job at each stage. The first stages use a small number of weak learners to reject the easy windows, while the later stages rely on larger numbers of weak learners to process the harder windows. While this method is fast, it isn't powerful enough when it reaches the final stages. That's because the weak learners used in all stages of the cascade are identical. So even though there are more classifiers in the last stages, they're not necessarily capable of performing highly complex classification.

The video will load shortly

Deep learning models

To address this problem, Vasconcelos and his team developed a novel algorithm that incorporates deep learning models in the final stages of a cascaded detector. Deep learning models are better suited for complex pattern recognition, which they can perform after being trained with hundreds or thousands of examples—in this case, images that either have or don't have a person. However, deep learning models are too complex for real-time implementation. While they work well for the final cascade stages, they are too complex to be used in the early ones.

The solution is a new cascade architecture that combines classifiers from different families: simple classifiers (weak learners) in the early stages complex classifiers (deep learning models) in the later stages. This is not trivial to accomplish, noted Vasconcelos, since the algorithm used to learn the cascade has to find the combination of weak learners that achieves the optimal trade-off between detection accuracy and complexity for each cascade stage. Accordingly, Vasconcelos and his team introduced a new mathematical formulation for this problem, which resulted in a new algorithm for cascade design.

"No previous algorithms have been capable of optimizing the trade-off between detection accuracy and speed for cascades with stages of such different complexities. In fact, these are the first cascades to include stages of deep learning. The results we're obtaining with this new algorithm are substantially better for real-time, accurate pedestrian detection," said Vasconcelos.

The algorithm currently only works for binary detection tasks, such as pedestrian detection, but the researchers are aiming to extend the cascade technology to detect many objects simultaneously.

"One approach to this problem is to train, for example, five different detectors to recognize five different objects. But we want to train just one detector to do this. Developing that algorithm is the next challenge," said Vasconcelos.

The work, titled "Learning Complexity-Aware Cascades for Deep Pedestrian Detection," was presented Dec. 15, 2015 at the International Conference on Computer Vision in Santiago, Chile. 

Explore further: Google team rises to 2014 visual recognition challenge

Related Stories

Google team rises to 2014 visual recognition challenge

September 8, 2014

Google's Christian Szegedy, software engineer, blogged Friday about GoogleNet's entry into a visual recognition challenge, the results of which indicate improvements in the state of machine vision technology. The Large Scale ...

Improving machine learning with an old approach

December 22, 2015

Computer scientist Rong Ge has an interesting approach to machine learning. While most machine learning specialists will build an algorithm which molds to a specific dataset, Ge builds an algorithm which he can guarantee ...

Teaching robots to see

December 15, 2014

Syed Saud Naqvi, a PhD student from Pakistan, is working on an algorithm to help computer programmes and robots to view static images in a way that is closer to how humans see.

Scientists teach machines to learn like humans

December 10, 2015

A team of scientists has developed an algorithm that captures our learning abilities, enabling computers to recognize and draw simple visual concepts that are mostly indistinguishable from those created by humans. The work, ...

Recommended for you

Wireless power could enable ingestible electronics

April 27, 2017

Researchers at MIT, Brigham and Women's Hospital, and the Charles Stark Draper Laboratory have devised a way to wirelessly power small electronic devices that can linger in the digestive tract indefinitely after being swallowed. ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.