Technique to see objects hidden around corners

March 5, 2018, Stanford University
Illustration of the non-line-of-sight imaging system. Credit: Stanford Computational Imaging Lab

A driverless car is making its way through a winding neighborhood street, about to make a sharp turn onto a road where a child's ball has just rolled. Although no person in the car can see that ball, the car stops to avoid it. This is because the car is outfitted with extremely sensitive laser technology that reflects off nearby objects to see around corners.

This scenario is one of many that researchers at Stanford University are imagining for a system that can produce images of objects hidden from view. They are focused on applications for autonomous vehicles, some of which already have similar laser-based systems for detecting objects around the car, but other uses could include seeing through foliage from aerial vehicles or giving rescue teams the ability to find people blocked from view by walls and rubble.

"It sounds like magic but the idea of non-line-of-sight imaging is actually feasible," said Gordon Wetzstein, assistant professor of electrical engineering and senior author of the paper describing this work, published March 5 in Nature.

Seeing the unseen

The Stanford group isn't alone in developing methods for bouncing lasers around corners to capture images of objects. Where this research advances the field is in the extremely efficient and effective algorithm the researchers developed to process the final image.

"A substantial challenge in non-line-of-sight imaging is figuring out an efficient way to recover the 3-D structure of the hidden from the noisy measurements," said David Lindell, graduate student in the Stanford Computational Imaging Lab and co-author of the paper. "I think the big impact of this method is how computationally efficient it is."

Artistic interpretation of paper topic. Credit: Stefani Billings

For their system, the researchers set a laser next to a highly sensitive photon detector, which can record even a single particle of light. They shoot pulses of light at a wall and, invisible to the human eye, those pulses bounce off objects around the corner and bounce back to the wall and to the detector. Currently, this scan can take from two minutes to an hour, depending on conditions such as lighting and the reflectivity of the hidden object.

Once the scan is finished, the algorithm untangles the paths of the captured photons and, like the mythical image enhancement technology of television crime shows, the blurry blob takes much sharper form. It does all this in less than a second and is so efficient it can run on a regular laptop. Based on how well the algorithm currently works, the researchers think they could speed it up so that it is nearly instantaneous once the scan is complete.

Into the 'wild'

The team is continuing to work on this system, so it can better handle the variability of the real world and complete the scan more quickly. For example, the distance to the object and amount of ambient light can make it difficult for their technology to see the light particles it needs to resolve out-of-sight objects. This technique also depends on analyzing scattered light particles that are intentionally ignored by guidance systems currently in cars - known as LIDAR systems.

Graduate student David Lindell and Matt O'Toole, a post-doctoral scholar, work in the lab. Credit: L.A. Cicero

"We believe the computation algorithm is already ready for LIDAR systems," said Matthew O'Toole, a postdoctoral scholar in the Stanford Computational Imaging Lab and co-lead author of the paper. "The key question is if the current hardware of LIDAR systems supports this type of imaging."

Before this system is road ready, it will also have to work better in daylight and with objects in motion, like a bouncing ball or running child. The researchers did test their technique successfully outside but they worked only with indirect light. Their technology did perform particularly well picking out retroreflective objects, such as safety apparel or traffic signs. The researchers say that if the technology were placed on a car today, that car could easily detect things like road signs, safety vests or road markers, although it might struggle with a person wearing non-reflective clothing.

"This is a big step forward for our field that will hopefully benefit all of us," said Wetzstein. "In the future, we want to make it even more practical in the 'wild.'"

Wetzstein is also an assistant professor, by courtesy, of computer science and a member of Stanford Bio-X and the Stanford Neurosciences Institute.

Explore further: System for seeing around corners could improve self-driving cars and search-and-rescue

More information: Confocal Non-Line-of-Sight Imaging Based on the Light Cone Transform, Nature (2018).

Related Stories

The amazing camera that can see around corners (w/ video)

December 8, 2015

How can a person see around a blind corner? One answer is to develop X-ray vision. A more mundane approach is to use a mirror. But if neither are an option, a group of scientists led by Genevieve Gariepy have developed a ...

Recommended for you

Weird superconductor leads double life

March 21, 2018

Until about 50 years ago, all known superconductors were metals. This made sense, because metals have the largest number of loosely bound "carrier" electrons that are free to pair up and flow as electrical current with no ...

Taming chaos: Calculating probability in complex systems

March 20, 2018

Daily weather patterns, brain activity on an EEG (electroencephalogram) and heartbeats on an EKG (electrocardiogram) each generate lines of complex data. To analyze this data, perhaps to predict a storm, seizure or heart ...

Shedding light on the mystery of the superconducting dome

March 20, 2018

University of Groningen physicists, and colleagues from Nijmegen and Hong Kong, have induced superconductivity in a monolayer of tungsten disulfide. By using an increasing electric field, they were able to show how the material ...


Adjust slider to filter visible comments by rank

Display comments: newest first

1 / 5 (1) Mar 06, 2018
This can be used as a road radar even in cars with drivers.
1 / 5 (1) Mar 06, 2018
That lead graphic makes me think someone is trying to 'kill da wabbit'.
1 / 5 (1) Mar 06, 2018
According to some theories, social insects with compound eyes are capable of multidimensional vision and they're also able recognize objects (predators?) behind corner by their shadows or specular reflections.
3 / 5 (2) Mar 06, 2018
1 / 5 (2) Mar 06, 2018
Unless it can be adapted to public transportation, I don't see how it's going to benefit most of the people who comment in these comment forums.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.