Microsoft Research turns 2D camera into depth sensor

Aug 14, 2014 by Nancy Owano weblog

Microsoft Research at SIGGRAPH 14 made news this week with its presentation of how to turn a regular video camera into a depth camera. "Learning to be a depth camera for close-range human capture and interaction" was presented by the research team, which represented Microsoft Research and the iCub facility at the Italian Institute of Technology.(Istituto Italiano di Tecnologia). In brief, as described in MIT Technology Review, "Microsoft researchers say simple hardware changes and machine learning techniques let a regular smartphone camera act as a depth sensor." The team focused on hands and faces; they modified the camera on a smartphone to capture hands and faces depth.

"We present a machine learning technique for estimating absolute, per-pixel depth using any conventional monocular 2D camera, with minor hardware modifications. Our approach targets close-range human capture and interaction where dense 3D estimation of hands and faces is desired," said the authors. They used hybrid classification-regression forests to learn how to map from near infrared intensity images to absolute, metric depth in realtime. As their video shows, they illustrated the results with varied human-computer interactions. One of the contributions their paper makes, they stated, is that "We demonstrate a new technique for turning a cheap color or monochrome camera into a depth sensor, for close-range human capture and interaction." After all, they said, "Whilst depth cameras are becoming more of a commodity, they have yet to (and arguably will never) surpass the ubiquity of regular 2D cameras, which are now used in the majority of our mobile devices and desktop computers."

The researchers also said, "Our hope is to allow practitioners to more rapidly prototype depth-based applications in a variety of new contexts." Two hardware designs for depth sensing were presented: a modified web camera for desktop depth sensing, in addition to a modified cellphone camera for mobile applications. They demonstrated efficient and accurate hand and face tracking in both scenarios.

This video is not supported by your browser at this time.

Describing how their approach worked, they said they devised an algorithm that learns the correlation between pixel intensities and absolute depth measurements. The algorithm is implemented on conventional color or monochrome cameras. The hardware modifications required were removal of any near infrared (NIR) cut filter (typically used in regular RGB sensors), and the addition of a bandpass filter and low-cost LEDs, both operating in a narrow NIR range.

Nonetheless, the authors stated that the method they described in the technical paper is not for a general-purpose depth . "Whilst this method cannot replace commodity depth sensors for general use, our hope is that it will enable 3D face and hand sensing and interactive systems in novel contexts,"

Explore further: Accurate information on people's behaviour helps to improve retail design

More information:research.microsoft.com/en-us/p… th4free/default.aspx

www.technologyreview.com/news/… mera-into-a-3-d-one/

add to favorites email to friend print save as pdf

Related Stories

New MIT technology allows 3D image interaction

Jan 02, 2014

(AP)—Researchers at the Massachusetts Institute of Technology have found a way to allow people in one place to interact with three-dimensional versions of people or objects in a different location.

Gadget Watch: Control a PC with body motions

Jan 13, 2012

(AP) -- Don't trash your keyboard and mouse just yet. But three companies at the International Consumer Electronics Show demonstrated depth-sensing cameras that let you to control your computer by moving ...

'Smart glass' micro-iris for smartphone cameras

Jun 18, 2014

A small, low-powered camera component made from a "smart glass" material has been created by a group of researchers in Germany with the hope of inspiring the next generation of smartphone cameras.

Crowd sourcing project to allow 3D scan-to-print web app

Aug 05, 2013

Technology to allow for printing three dimensional objects is evolving rapidly, making it difficult for some to keep up. It's also still relatively expensive. Currently, people who wish to print such an object ...

Recommended for you

Using sound to picture the world in a new way

20 hours ago

Have you ever thought about using acoustics to collect data? The EAR-IT project has explored this possibility with various pioneering applications that impact on our daily lives. Monitoring traffic density ...

Sweeping air devices for greener planes

Oct 21, 2014

The large amount of jet fuel required to fly an airplane from point A to point B can have negative impacts on the environment and—as higher fuel costs contribute to rising ticket prices—a traveler's wallet. ...

User comments : 1

Adjust slider to filter visible comments by rank

Display comments: newest first

EnricM
not rated yet Aug 15, 2014
Nice! Imagine this added to web pages, it could be a mayor revolution for web apps, mobile or conventional