Your next phone camera might be able to sense depth

April 12, 2013 by Troy Wolverton

The camera in your smartphone may soon have a new trick: depth perception. Toshiba, Samsung and Silicon Valley startup Pelican Imaging are developing image sensors and software that would allow cameras to detect the distance of objects within the scenes they photograph.

The new could start showing up in smartphones as soon as this year. Initially, it will allow consumers to do things like refocus their pictures after they take them, much like they can with Lytro's "light-field" camera today.

But because of their small size and, in some cases, high resolution, the new cameras could also be used in a wide range of other applications. In the future, they could be employed in new, more precise versions of Microsoft's Kinect, the -sensing ; in cars as collision-preventing backup cameras; as identification systems that can precisely distinguish individual faces; and in a kind of three-dimensional scanner for 3-D printing.

"This type of technology is the next big thing for imaging," said Chris Chute, an imaging analyst at research firm IDC.

I recently met with Pelican and got a demonstration of its technology. The company has designed a chip that, instead of containing a solitary , has an array of 16 or even 20 of them.

To get depth information, the chip essentially employs the principle of parallax, the same basic method astronomers use to measure the distance to nearby stars. Pelican's software is able to determine the distance to a particular point in an image by using the known distance between its multiple sensors and the sensors' viewing angles to that point.

Pelican's software takes the images recorded by each of the individual sensors in its array and combines them, yielding not only an 8 megapixel image, but one with depth information for each point within it.

The company's technology appears to be an improvement on what's come before. Pelican's system yields both a high-resolution image, which Lytro's camera doesn't do, and a high-resolution depth map, which the Kinect can't match.

Unlike Kinect and similar systems, Pelican's technology doesn't involve bouncing light or lasers off an object. That should make it perform much better outdoors in bright light situations, which tend to trouble the Kinect. And unlike other depth-cameras, Pelican's system is being designed from the ground up to be small enough to fit into a .

At our meeting, Pelican CEO Chris Pickett and marketing Vice President Paul Gallagher demonstrated how they could take a picture with the company's camera module and change the focus of the image after the fact. They could focus on a person in the foreground or something in the background, or have everything be in focus.

While it will be cool to have that capability in a smartphone, you already can do the same things with a Lytro camera.

But Pelican's camera has other potential uses. For example, it could allow users to easily select an object in an image to adjust its exposure or copy it to another picture. It could allow users to interact with their phones with 3-D gestures. And it could be used in for face-detection systems that determine whether someone is authorized to use a device.

Pelican says that it's already working with some of the existing smartphone camera system makers and hopes to have its system on a smartphone by early next year.

But cameras like Pelican's could also have some important uses outside of phones. Many cars now have backup cameras and some have proximity sensors that warn drivers when they are about to hit something. A single Pelican camera could serve both functions.

They could also be used inside cars. Gallagher noted that cars include sensors that determine when an air bag should be deployed. Currently, those sensors detect only weight. They can't detect whether a rider's feet are propped up on the dash or whether the person is leaning forward to look in the glove compartment, both of which could be dangerous positions if an air bag were to deploy. But a car's safety system could use information from a depth camera to determine riders' positions and whether it would be safe to deploy an air bag.

I don't know whether Pelican's technology - or that of its rivals - will take off in the market. But it's exciting to picture the possibilities.

A new kind of camera for your phone

Local startup Pelican Imaging has developed a camera system for smartphones that senses depth as well as light.

-What it does: When taking a photograph, it records the distance from the to each point in the picture.

-How it works: The image chip includes an array of 16 or 20 low-resolution image sensors. Pelican's software combines the low-resolution images to form a high-resolution composite and gleans depth information through the principle of parallax, the mathematical method astronomers use to find the distance to .

-What you can do with it: Smartphone users could refocus images after they shoot them or easily select objects within a photo and copy them to another. The system could potentially be used in face-detection and car safety systems.

-When it will be available: Pelican hopes to have its system in smartphones early next year. Similar sensors from Toshiba or Samsung could be in smartphones sooner.

Explore further: Toshiba smartphone camera sensor has eye on future


Related Stories

Toshiba smartphone camera sensor has eye on future

December 28, 2012

(—Toshiba may be targeting the end of 2013 for the launch of a new camera sensor for smartphones and tablets, where the picture taker can choose a specific area of focus in the photo after having taken the photo. ...

Cameras offer advantages even to smartphone owners

January 28, 2013

If you have a smartphone, you likely take lots of pictures with it. So why would you ever need a stand-alone digital camera? That's a question many consumers are asking. Digital camera sales are falling, while smartphone ...

Retailers to add radical 'focus later' camera

September 25, 2012

A radical camera that lets users adjust the focus after taking pictures will be available in October at shops in Australia, Canada, Singapore, Hong Kong and the United States.

Gadget Watch: Control a PC with body motions

January 13, 2012

(AP) -- Don't trash your keyboard and mouse just yet. But three companies at the International Consumer Electronics Show demonstrated depth-sensing cameras that let you to control your computer by moving your hands or body.

Recommended for you

Researchers find tweeting in cities lower than expected

February 20, 2018

Studying data from Twitter, University of Illinois researchers found that less people tweet per capita from larger cities than in smaller ones, indicating an unexpected trend that has implications in understanding urban pace ...

Augmented reality takes 3-D printing to next level

February 20, 2018

Cornell researchers are taking 3-D printing and 3-D modeling to a new level by using augmented reality (AR) to allow designers to design in physical space while a robotic arm rapidly prints the work.

What do you get when you cross an airplane with a submarine?

February 15, 2018

Researchers from North Carolina State University have developed the first unmanned, fixed-wing aircraft that is capable of traveling both through the air and under the water – transitioning repeatedly between sky and sea. ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.