Plumbing the possibilities of a camera that 'sees around corners'

July 28, 2016 by Brian Mattmiller, Morgridge Institute for Research
Seeing around corners' technology could help emergency personnel identify people in danger during fires or natural disasters. Credit: Morgridge Institute for Research

The Morgridge Institute for Research and University of Wisconsin-Madison researchers are working to optimize a camera capable of a slick optical trick: Snapping pictures around corners.

The imaging project, supported by a new $4.4 million grant from the U.S. Defense Department's Advanced Research Projects Agency (DARPA), will work over four years to explore the limitations and potential applications of scattered-light technology that can recreate scenes outside human line of sight.

The technology, pioneered by Morgridge imaging specialist Andreas Velten, uses pulses of scattered light photons that bounce through a scene and are recaptured by finely tuned sensors connected to the camera. Information from this scattered light helps the researchers digitally rebuild a 3D environment that is either hidden or obstructed from view.

While in its infancy, the technology has generated excitement about potential applications in medical imaging, disaster relief, navigation, robotic surgery and even space exploration. UW-Madison is one of eight university teams receiving 2016 DARPA grants to probe different forms of non-line-of-sight imaging.

Velten, also a scientist with the UW-Madison Laboratory for Optical and Computational Instrumentation (LOCI), first demonstrated non-line-of-sight imaging in 2012 with the Massachusetts Institute of Technology. The technique was able to recreate human figures and other shapes that were positioned around corners.

Another application could provide law enforcement with better threat assessment tools in hostile situations. Credit: Morgridge Institute for Research

He now is partnering with Mohit Gupta, UW-Madison assistant professor of computer science, to see how far they can push the quality and complexity of these pictures—a fundamental step before devices can become reality.

Questions include: Can they recapture movement? Can they determine what material an object is made of? Can they differentiate between very similar shapes? Can they push this method further to see around two, three, or more corners?

Velten and Gupta have developed a theoretical framework to determine how complex a scene they can recapture. They are creating models where they bounce light a half-dozen or more times through a space to capture objects that are either hidden or outside the field of view.

"The more times you can bounce this light within a scene, the more possible data you can collect," Velten says. "Since the first light is the strongest, and each proceeding bounce gets weaker and weaker, the sensor has to be sensitive enough to capture even a few photons of light."

Gupta brings expertise in computer vision to the project and will be developing methods to make better sense of the limited data coming back from these techniques. Gupta will work to develop algorithms that can better decode the data that helps recreate scenes.

The scattered light technology could have applications for robotic surgery, enhancing the ability to see obstructed spaces less invasively. Credit: Morgridge Institute for Research
"The information we will get is going to be noisy and the shapes will be blob-like, not much to the naked eye, so the visualization part of this will be huge," Gupta says. "Because this problem is so new, we don't even know what's possible."

The first two years of the grant will be devoted to the pushing the limitations of this technique. The second two years will focus on hardware development to make field applications possible.

Some of the possibilities are tantalizing. For example, Gupta says it could be used for safety tests of jet engines, examining performance while the engines are running. It could also be used to probe impossible-to-see spaces in shipwrecks such as the Titanic. Velten currently has a NASA project examining whether the technology can be used to probe the dimensions of moon caves.

Interestingly, the light used in this project would be pretty useless for standard photography. Regular cameras rely on the opening burst of light on the subject, while this project focuses on the indirect that comes later and scatters and bounces through the scene.

"We are interested in capturing exactly what a conventional camera doesn't capture," he says.

Explore further: New technology may illuminate mystery moon caves

Related Stories

New technology may illuminate mystery moon caves

August 17, 2015

It's widely believed that the moon features networks of caves created when violent lava flows tore under the surface from ancient volcanoes. Some craters may actually be "skylights" where cave ceilings have crumbled.

Creative Cameras exhibit explores light-in-flight imaging

July 6, 2014

The Creative Cameras exhibition team from Heriot-Watt University and the University of Glasgow are asking some interesting questions. How do you take images so fast that you can see light travelling through air? How do you ...

A camera that peers around corners (w/ video)

March 20, 2012

In December, MIT Media Lab researchers caused a stir by releasing a slow-motion video of a burst of light traveling the length of a plastic bottle. But the experimental setup that enabled that video was designed for a much ...

Recommended for you

Researchers engineer a tougher fiber

February 22, 2019

North Carolina State University researchers have developed a fiber that combines the elasticity of rubber with the strength of a metal, resulting in a tougher material that could be incorporated into soft robotics, packaging ...

A quantum magnet with a topological twist

February 22, 2019

Taking their name from an intricate Japanese basket pattern, kagome magnets are thought to have electronic properties that could be valuable for future quantum devices and applications. Theories predict that some electrons ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.