Researchers generate 3D images using just one photon per pixel (w/ video)

single photon correlation
Single-photon camera imaging set-up. Credit: Shin et al.

(—Every time you take a photograph, your camera detects more than a billion photons. For a basic one-megapixel camera, that's more than 1,000 photons per pixel. Now in a new study, researchers have developed an algorithm that is so efficient that it can generate high-quality 3D images using a single-photon camera that detects just one signal photon per pixel.

The researchers, led by Jeffrey Shapiro, a professor of electrical engineering and computer science at the Massachusetts Institute of Technology (MIT), along with coauthors at MIT, Politecnico di Milano, and Boston University, have published a paper on the new photon-efficient approach to imaging with a single-photon camera in a recent issue of Nature Communications.

"Our work shows that we can use these new photon-counting cameras at much lower fluxes and much higher efficiencies than previously thought," Shapiro told

Compared to other camera-based 3D imaging techniques that have recently been developed, the new framework has the highest photon efficiency to date, resulting in a visibly better reconstruction accuracy and an order of magnitude better depth resolution.

To demonstrate how the new single-photon imaging algorithm works in low- environments, the researchers illuminated a scene of interest (such as a mannequin and sunflower) with a pulsed laser that emits low-light pulses every 50 nanoseconds. A light diffuser spatially spreads out the pulses so that they flood the entire scene.

Video showing the three stages of the reconstruction algorithm. Credit: Shin et al.

A single-photon camera then captures the light reflected by the illuminated objects, along with some background light that the researchers added to simulate a realistic environment. The key element of the camera is a single-photon avalanche diode (SPAD) array, which detects incoming photons and records their precise times of arrival.

In order to reconstruct a scene from a very small number of detected photons, the accounts for the fact that a scene's features are statistically correlated. Whereas most camera systems treat each pixel independently, the new algorithm recognizes that neighboring pixels often have strong spatial correlations, which are punctuated by sharp boundaries that separate two different features (such as the mannequin's shirt and the sunflower in the test scene). By solving a mathematical problem of convex optimization, the new algorithm exploits these correlations to generate high-resolution 3D images using very few photons.

The images in this experiment have a final resolution of 384 x 384 pixels, and an average of one signal photon and one background light photon are detected per pixel.

Another advantage of the new single-photon is that images can be captured very quickly, making it useful for applications that require fast and accurate imaging using extremely small amounts of light.

Video comparing the new algorithm with a conventional filtered-histogram method. As the number of detected photons decreases, the new method substantially outperforms the conventional method in both reflectivity and depth reconstructions. Credit: Shin et al.

By requiring vastly fewer photons than conventional cameras, single-photon cameras have potential applications in low-light conditions, such as biological imaging, astronomy, and providing 3D vision for self-navigating advanced robotic systems, such as unmanned aerial vehicles and exploration rovers.

In the future, the researchers plan to pursue various applications by collaborating with researchers with different specialties. Working with MIT's chemistry department, for example, they have begun to apply their photon-efficient approach to fluorescence imaging, an imaging modality that has many applications in biology.

They are also pushing the technology to its limits under DARPA sponsorship as part of DARPA's Revolutionary Enhancement of Visibility by Exploiting Active Light-fields (REVEAL) program. REVEAL's goal is to use all the information available in the light field to see into occluded spaces without using mirrors.

"We're going to use laser light and single-photon detection to see around a corner," Shapiro explained. "Our approach will rely on diffuse reflections from a wall. Because the light returned from the hidden objects will be very weak, photon efficiency is essential."

More information: Dongeek Shin et al. "Photon-efficient imaging with a single-photon camera." Nature Communications. DOI: 10.1038/ncomms12046

See more details at the researchers' project page.

Journal information: Nature Communications

© 2016

Citation: Researchers generate 3D images using just one photon per pixel (w/ video) (2016, July 13) retrieved 14 June 2024 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

Creative Cameras exhibit explores light-in-flight imaging


Feedback to editors