Microsoft Research turns 2D camera into depth sensor

August 14, 2014 by Nancy Owano weblog

Microsoft Research at SIGGRAPH 14 made news this week with its presentation of how to turn a regular video camera into a depth camera. "Learning to be a depth camera for close-range human capture and interaction" was presented by the research team, which represented Microsoft Research and the iCub facility at the Italian Institute of Technology.(Istituto Italiano di Tecnologia). In brief, as described in MIT Technology Review, "Microsoft researchers say simple hardware changes and machine learning techniques let a regular smartphone camera act as a depth sensor." The team focused on hands and faces; they modified the camera on a smartphone to capture hands and faces depth.

"We present a machine learning technique for estimating absolute, per-pixel depth using any conventional monocular 2D camera, with minor hardware modifications. Our approach targets close-range human capture and interaction where dense 3D estimation of hands and faces is desired," said the authors. They used hybrid classification-regression forests to learn how to map from near infrared intensity images to absolute, metric depth in realtime. As their video shows, they illustrated the results with varied human-computer interactions. One of the contributions their paper makes, they stated, is that "We demonstrate a new technique for turning a cheap color or monochrome camera into a depth sensor, for close-range human capture and interaction." After all, they said, "Whilst depth cameras are becoming more of a commodity, they have yet to (and arguably will never) surpass the ubiquity of regular 2D cameras, which are now used in the majority of our mobile devices and desktop computers."

The researchers also said, "Our hope is to allow practitioners to more rapidly prototype depth-based applications in a variety of new contexts." Two hardware designs for depth sensing were presented: a modified web camera for desktop depth sensing, in addition to a modified cellphone camera for mobile applications. They demonstrated efficient and accurate hand and face tracking in both scenarios.

This video is not supported by your browser at this time.

Describing how their approach worked, they said they devised an algorithm that learns the correlation between pixel intensities and absolute depth measurements. The algorithm is implemented on conventional color or monochrome cameras. The hardware modifications required were removal of any near infrared (NIR) cut filter (typically used in regular RGB sensors), and the addition of a bandpass filter and low-cost LEDs, both operating in a narrow NIR range.

Nonetheless, the authors stated that the method they described in the technical paper is not for a general-purpose depth . "Whilst this method cannot replace commodity depth sensors for general use, our hope is that it will enable 3D face and hand sensing and interactive systems in novel contexts,"

Explore further: Gadget Watch: Control a PC with body motions

More information:research.microsoft.com/en-us/projects/depth4free/default.aspx

www.technologyreview.com/news/529986/turning-a-regular-smartphone-camera-into-a-3-d-one/

Related Stories

Gadget Watch: Control a PC with body motions

January 13, 2012

(AP) -- Don't trash your keyboard and mouse just yet. But three companies at the International Consumer Electronics Show demonstrated depth-sensing cameras that let you to control your computer by moving your hands or body.

Crowd sourcing project to allow 3D scan-to-print web app

August 5, 2013

Technology to allow for printing three dimensional objects is evolving rapidly, making it difficult for some to keep up. It's also still relatively expensive. Currently, people who wish to print such an object have but two ...

New MIT technology allows 3D image interaction

January 2, 2014

(AP)—Researchers at the Massachusetts Institute of Technology have found a way to allow people in one place to interact with three-dimensional versions of people or objects in a different location.

'Smart glass' micro-iris for smartphone cameras

June 18, 2014

A small, low-powered camera component made from a "smart glass" material has been created by a group of researchers in Germany with the hope of inspiring the next generation of smartphone cameras.

Recommended for you

Software turns smartphones into tools for medical research

July 27, 2015

Jody Kearns doesn't like to spend time obsessing about her Parkinson's disease. The 56-year-old dietitian from Syracuse, New York, had to give up bicycling because the disorder affected her balance. But she still works, drives ...

Where is solar power headed?

July 22, 2015

Most experts agree that to have a shot at curbing the worst impacts of climate change, we need to extricate our society from fossil fuels and ramp up our use of renewable energy.

1 comment

Adjust slider to filter visible comments by rank

Display comments: newest first

EnricM
not rated yet Aug 15, 2014
Nice! Imagine this added to web pages, it could be a mayor revolution for web apps, mobile or conventional

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.