Microsoft Research turns 2D camera into depth sensor

Aug 14, 2014 by Nancy Owano weblog

Microsoft Research at SIGGRAPH 14 made news this week with its presentation of how to turn a regular video camera into a depth camera. "Learning to be a depth camera for close-range human capture and interaction" was presented by the research team, which represented Microsoft Research and the iCub facility at the Italian Institute of Technology.(Istituto Italiano di Tecnologia). In brief, as described in MIT Technology Review, "Microsoft researchers say simple hardware changes and machine learning techniques let a regular smartphone camera act as a depth sensor." The team focused on hands and faces; they modified the camera on a smartphone to capture hands and faces depth.

"We present a machine learning technique for estimating absolute, per-pixel depth using any conventional monocular 2D camera, with minor hardware modifications. Our approach targets close-range human capture and interaction where dense 3D estimation of hands and faces is desired," said the authors. They used hybrid classification-regression forests to learn how to map from near infrared intensity images to absolute, metric depth in realtime. As their video shows, they illustrated the results with varied human-computer interactions. One of the contributions their paper makes, they stated, is that "We demonstrate a new technique for turning a cheap color or monochrome camera into a depth sensor, for close-range human capture and interaction." After all, they said, "Whilst depth cameras are becoming more of a commodity, they have yet to (and arguably will never) surpass the ubiquity of regular 2D cameras, which are now used in the majority of our mobile devices and desktop computers."

The researchers also said, "Our hope is to allow practitioners to more rapidly prototype depth-based applications in a variety of new contexts." Two hardware designs for depth sensing were presented: a modified web camera for desktop depth sensing, in addition to a modified cellphone camera for mobile applications. They demonstrated efficient and accurate hand and face tracking in both scenarios.

This video is not supported by your browser at this time.

Describing how their approach worked, they said they devised an algorithm that learns the correlation between pixel intensities and absolute depth measurements. The algorithm is implemented on conventional color or monochrome cameras. The hardware modifications required were removal of any near infrared (NIR) cut filter (typically used in regular RGB sensors), and the addition of a bandpass filter and low-cost LEDs, both operating in a narrow NIR range.

Nonetheless, the authors stated that the method they described in the technical paper is not for a general-purpose depth . "Whilst this method cannot replace commodity depth sensors for general use, our hope is that it will enable 3D face and hand sensing and interactive systems in novel contexts,"

Explore further: Accurate information on people's behaviour helps to improve retail design


Related Stories

New MIT technology allows 3D image interaction

Jan 02, 2014

(AP)—Researchers at the Massachusetts Institute of Technology have found a way to allow people in one place to interact with three-dimensional versions of people or objects in a different location.

Gadget Watch: Control a PC with body motions

Jan 13, 2012

(AP) -- Don't trash your keyboard and mouse just yet. But three companies at the International Consumer Electronics Show demonstrated depth-sensing cameras that let you to control your computer by moving ...

'Smart glass' micro-iris for smartphone cameras

Jun 18, 2014

A small, low-powered camera component made from a "smart glass" material has been created by a group of researchers in Germany with the hope of inspiring the next generation of smartphone cameras.

Crowd sourcing project to allow 3D scan-to-print web app

Aug 05, 2013

Technology to allow for printing three dimensional objects is evolving rapidly, making it difficult for some to keep up. It's also still relatively expensive. Currently, people who wish to print such an object ...

Recommended for you

Making houses in Sweden with additive manufacturing

14 hours ago

In a collaborative project worth SEK 35 million, researchers and external partners are together developing technology to make full-scale 3D prints of cellulos- based material. It is not a matter of small ...

3-D printer churns out handheld underground water model

15 hours ago

Cutting-edge three dimensional printing technology has produced a plastic model representing the land surface and underground water systems across 3.5 million hectares of the La Grange groundwater allocation ...

Major step for implantable drug-delivery device

17 hours ago

An implantable, microchip-based device may soon replace the injections and pills now needed to treat chronic diseases: Earlier this month, MIT spinout Microchips Biotech partnered with a pharmaceutical giant ...

User comments : 1

Adjust slider to filter visible comments by rank

Display comments: newest first

not rated yet Aug 15, 2014
Nice! Imagine this added to web pages, it could be a mayor revolution for web apps, mobile or conventional

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.