Human-computer interactions could be improved by a new efficient and accurate hand-gesture-recognition model

Jan 22, 2014
Gesture recognition: Gaining the upper hand
Using raw data from single depth images, a three-step modelling process provides quick, accurate estimations of hand poses for use in numerous applications such as robotics and ergonomics. Credit: C. Xu and L. Chang

Virtual reality, gaming and robotics rely on computers using depth images from cameras to recognize, understand and replicate actual body movements. Programming a computer to accurately identify hand gestures is particularly challenging because of the speed, complexity and dexterity of human hand movements.

Devices such as the popular Microsoft Kinect provide both video images and depth information. Processing software must combine these data to build up a three-dimensional (3D) representation of , but this can be computationally expensive and is not always accurate.

Now, Li Cheng and Chi Xu at the A*STAR Bioinformatics Institute in Singapore have successfully trialed a capable of estimating hand movements and poses using only raw depth data, thus allowing for faster processing. The method appears robust and at times more accurate than high-end programs.

"Hand pose estimation is a difficult problem," states Cheng. "For example, fingers are often clustered together and tend to obstruct each other. Recent programs are still not sufficiently reliable—3Gear, for example, only holds six gestures in its database and ignores all other possibilities. Our new method provides quick results for a large number of different poses."

The researchers trained their model using synthetic images of hand gestures before applying the model to realistic low-resolution, noisy depth images, as typically found in Kinect-type systems. They applied a feature classification technique called a Hough transform to the data, which uses a forest of points to locate the hand and verify the likely orientation of the palm and fingers.

"As the hand can easily roll sideways—a movement not seen in other parts of the body—we used two further steps in the process to improve 3D accuracy," explains Cheng. Rotating a whole image is computationally costly; instead, the researchers applied a second Hough 'forest' with specially modified depth features to account for possible hand rotation. This step produced a list of probable hand poses from which the best fit was chosen (see image).

Unlike previous programs where the palm of the hand is assumed to be flat, the new model is capable of simulating the arch of the palm. This significantly helps in accurately representing gestures. Tests using real hand movements demonstrated that the model could deliver fast and precise results.

The researchers hope their model will provide more efficient and accurate hand pose estimations in computer software. In the future, they also aim to include scenarios that involve hands interacting with physical objects.

Explore further: Researchers using Kinect to allow deaf people to communicate via computer (w/ Video)

More information: Xu, C. & Cheng, L. Efficient hand pose estimation from a single depth image. International Conference on Computer Vision, 3–6 December 2013. www.cv-foundation.org/openaccess/content_iccv_2013/papers/Xu_Efficient_Hand_Pose_2013_ICCV_paper.pdf

add to favorites email to friend print save as pdf

Related Stories

Microsoft hand research ripens Kinect for work (w/ video)

Mar 07, 2013

(Phys.org) —Beyond reading body motions, Kinect is getting a workup by researchers at Microsoft, now showing substantial control additions. Microsoft Research this week showed how Microsoft Kinect for Windows ...

The hand as a joystick

May 26, 2011

You need a mouse and a keyboard, a touch-screen or a joystick to control a computer system. In the future, a new gesture command system will make it possible to use just the fingers of a hand.

Recommended for you

Large streams of data warn cars, banks and oil drillers

21 hours ago

Better warning systems that alert motorists to a collision, make banks aware of the risk of losses on bad customers, and tell oil companies about potential problems with new drilling. This is the aim of AMIDST, the EU project ...

User comments : 0

More news stories

Sony's PlayStation 4 sales top seven million

Sony says it has sold seven million PlayStation 4 worldwide since its launch last year and admitted it can't make them fast enough, in a welcome change of fortune for the Japanese consumer electronics giant.

Robotics goes micro-scale

(Phys.org) —The development of light-driven 'micro-robots' that can autonomously investigate and manipulate the nano-scale environment in a microscope comes a step closer, thanks to new research from the ...

Biologists help solve fungi mysteries

(Phys.org) —A new genetic analysis revealing the previously unknown biodiversity and distribution of thousands of fungi in North America might also reveal a previously underappreciated contributor to climate ...