# Seeing depth through a single lens

##### August 5, 2013

Researchers at the Harvard School of Engineering and Applied Sciences (SEAS) have developed a way for photographers and microscopists to create a 3D image through a single lens, without moving the camera.

Published in the journal Optics Letters, this improbable-sounding technology relies only on computation and mathematics—no unusual hardware or fancy lenses. The effect is the equivalent of seeing a stereo image with one eye closed.

That's easier said than done, as principal investigator Kenneth B. Crozier, John L. Loeb Associate Professor of the Natural Sciences, explains.

"If you close one eye, depth perception becomes difficult. Your eye can focus on one thing or another, but unless you also move your head from side to side, it's difficult to gain much sense of objects' relative distances," Crozier says. "If your viewpoint is fixed in one position, as a microscope would be, it's a challenging problem."

Offering a workaround, Crozier and graduate student Antony Orth essentially compute how the image would look if it were taken from a different angle. To do this, they rely on the clues encoded within the rays of light entering the camera.

"Arriving at each pixel, the light's coming at a certain angle, and that contains important information," explains Crozier. "Cameras have been developed with all kinds of new hardware—microlens arrays and absorbing masks—that can record the direction of the light, and that allows you to do some very interesting things, such as take a picture and focus it later, or change the perspective view. That's great, but the question we asked was, can we get some of that functionality with a regular camera, without adding any extra hardware?"

The key, they found, is to infer the angle of the light at each pixel, rather than directly measuring it (which standard image sensors and film would not be able to do). The team's solution is to take two images from the same camera position but focused at different depths. The slight differences between these two images provide enough information for a computer to mathematically create a brand-new image as if the camera had been moved to one side.

By stitching these two images together into an animation, Crozier and Orth provide a way for amateur photographers and microscopists alike to create the impression of a stereo image without the need for expensive hardware. They are calling their computational method "light-field moment imaging"—not to be confused with "light field cameras" (like the Lytro), which achieve similar effects using high-end hardware rather than computational processing.

Importantly, the technique offers a new and very accessible way to create 3D images of translucent materials, such as biological tissues.

Biologists can use a variety of tools to create 3D optical images, including light-field microscopes, which are limited in terms of spatial resolution and are not yet commercially available; confocal microscopes, which are expensive; and a computational method called "shape from focus," which uses a stack of images focused at different depths to identify at which layer each object is most in focus. That's less sophisticated than Crozier and Orth's new technique because it makes no allowance for overlapping materials, such as a nucleus that might be visible through a cell membrane, or a sheet of tissue that's folded over on itself. Stereo microscopes may be the most flexible and affordable option right now, but they are still not as common in laboratories as traditional, monocular microscopes.

"This method devised by Orth and Crozier is an elegant solution to extract depth information with only a minimum of information from a sample," says Conor L. Evans, an assistant professor at Harvard Medical School and an expert in biomedical imaging, who was not involved in the research. "Depth measurements in microscopy are usually made by taking many sequential images over a range of depths; the ability to glean depth information from only two images has the potential to accelerate the acquisition of digital microscopy data."

"As the method can be applied to any image pair, microscopists can readily add this approach to our toolkit," Evans adds. "Moreover, as the computational method is relatively straightforward on modern computer hardware, the potential exists for real-time rendering of depth-resolved information, which will be a boon to microscopists who currently have to comb through large data sets to generate similar 3D renders. I look forward to using their method in the future."

The new technology also suggests an alternative way to create 3D movies for the big screen.

"When you go to a 3D movie, you can't help but move your head to try to see around the 3D image, but of course it's not going to do anything because the stereo image depends on the glasses," explains Orth, a Ph.D. student in applied physics. "Using light-field moment imaging, though, we're creating the perspective-shifted images that you'd fundamentally need to make that work—and just from a regular camera. So maybe one day this will be a way to just use all of the existing cinematography hardware, and get rid of the glasses. With the right screen, you could play that back to the audience, and they could move their heads and feel like they're actually there."

For the 3D effect to be noticeable, the camera aperture must be wide enough to let in light from a wide range of angles so that the differences between the two images focused at different depths are distinct. However, while a cellphone camera proves too small (Orth tried it on his iPhone), a standard 50 mm lens on a single-lens reflex camera is more than adequate.

Explore further: Single-pixel power: Scientists make 3-D images without a camera

## Related Stories

#### Single-pixel power: Scientists make 3-D images without a camera

May 17, 2013

Their system uses simple, cheap detectors which have just a single pixel to sense light instead of the millions of pixels used in the imaging sensors of digital cameras.

#### Technology for editing 3-D photos developed

May 17, 2013

Taking pictures with 3D cameras may start catching on thanks to an innovation by Brigham Young University computer scientists and developers at Adobe.

#### Team creates techniques for high quality, high resolution stereo panoramas

June 21, 2013

Stereoscopic panoramas promise an inviting, immersive experience for viewers but, at high resolutions, distortions can develop that make viewing unpleasant or even intolerable. A team at Disney Research Zurich has found methods ...

#### Disney researchers reconstruct detailed 3D scenes from hundreds of high-resolution 2D images

July 19, 2013

Investigators at Disney Research, Zürich have developed a method for using hundreds of photographic images to build 3D computer models of complex, real-life scenes that meet the increasing demands of today's movie, TV and ...

#### Pushing microscopy beyond standard limits

July 29, 2013

Engineers at the California Institute of Technology (Caltech) have devised a method to convert a relatively inexpensive conventional microscope into a billion-pixel imaging system that significantly outperforms the best available ...

#### Crowd sourcing project to allow 3D scan-to-print web app

August 5, 2013

Technology to allow for printing three dimensional objects is evolving rapidly, making it difficult for some to keep up. It's also still relatively expensive. Currently, people who wish to print such an object have but two ...

## Recommended for you

#### 'Material universe' yields surprising new particle

November 25, 2015

An international team of researchers has predicted the existence of a new type of particle called the type-II Weyl fermion in metallic materials. When subjected to a magnetic field, the materials containing the particle act ...

#### Physicists set quantum record by using photons to carry messages from electrons almost 2 kilometers apart

November 25, 2015

Researchers from Stanford have advanced a long-standing problem in quantum physics – how to send "entangled" particles over long distances.

#### CERN collides heavy nuclei at new record high energy

November 25, 2015

The world's most powerful accelerator, the 27 km long Large Hadron Collider (LHC) operating at CERN in Geneva established collisions between lead nuclei, this morning, at the highest energies ever. The LHC has been colliding ...

#### Researchers discover why E. coli move faster in syrup-like fluids than in water

November 25, 2015

Swimming in a pool of syrup would be difficult for most people, but for bacteria like E. coli, it's easier than swimming in water. Scientists have known for decades that these cells move faster and farther in viscoelastic ...

#### Ground-breaking research could challenge underlying principles of physics

November 20, 2015

An international team of physicists, including a Plymouth University academic, has published ground-breaking research on the decay of subatomic particles called kaons – which could change how scientists understand the formation ...

#### Exploring the physics of a chocolate fountain

November 24, 2015

A mathematics student has worked out the secrets of how chocolate behaves in a chocolate fountain, answering the age-old question of why the falling 'curtain' of chocolate surprisingly pulls inwards rather than going straight ...