Gesture-driven computers

Feb 29, 2008

It isn’t always easy to communicate with a computer. Two Fraunhofer Institutes will be presenting new possibilities of man-machine interaction at CeBIT in Hanover (Germany) on March 4 through 9. They will demonstrate how computers can be operated simply by gesturing or pointing a finger.

A man stands in front of a large screen gesticulating in a seemingly hectic manner. As if by magic, images suddenly appear on the display. Their movements follow the actor’s gestures, rotate at the slightest turn of a finger, and become larger or smaller as desired. This scene will look familiar to anyone who has watched the science fiction film ‘Minority Report’. Paul Chojecki, scientist and project manager at the Fraunhofer Institute for Telecommunications, Heinrich-Hertz-Institut, HHI in Berlin, explains the iPoint Presenter in a manner reminiscent of the John Anderton character played by Tom Cruise.

At the heart of the system is a set of cameras which enable the computer to observe the person standing in front of the projection screen. The moment this person moves their hands, the computer reacts without being touched at all. “It begins by determining the position of the user’s index finger, then follows its movements,” Chojecki explains. The user can point to buttons or use gestures to move virtual objects. Through ‘multipointing interaction’, i.e. commands using multiple fingers, he can rotate, enlarge or minimize objects. This requires neither special gloves nor any particular markings. Anyone can intuitively operate the device with their bare hands without any preparation whatsoever.

The iPoint Presenter will be demonstrated for the first time at CeBIT 2008, using the example of an interactive game and a photo viewer. But these are just two of the many possible applications it can be used for. It could replace touch screens at info terminals, for example, or help to edit and organize photos. “What is special about it is that the human-computer cummunication is entirely contact-free. The system is therefore ideal for scenarios in which contact between the user and the computer is not allowed or not possible, for example in an operating theater,” says Chojecki.

The system would also be ideal for presentations in large auditoriums. The speaker would no longer need a mouse or a laser pointer, and could click through the presentation and highlight important information simply by pointing. A particularly useful feature for situations like these is that the system can be extended to as many as nine cameras. This immensely increases the user’s operating range and enables them to interact with very large screens, for example at trade fairs or advertising events.

Identifying gestures

Gestures enable people of different nationalities to communicate without the need for spoken words. How useful would it be if this type of communication were also possible between humans and technical devices? This form of giving commands would make many situations safer and more pleasant than they are today. Drivers, for example, could operate their car radios and navigators more easily, and TV viewers at home in their armchairs would no longer need a remote control to flick through the channels.

A whole new generation of video games could be created if the technology involved were able to identify and interpret human gestures. Even machines, household appliances or video conference systems could be controlled by mere hand signals. The system could also be of help to physically disabled people, enabling them to interact with a computer without the need for a mouse and keyboard.

To translate these scenarios into reality as soon as possible, researchers at the Fraunhofer Institute for Digital Media Technology IDMT in Ilmenau are now teaching computers to understand human gestures, and are developing a method of automatically recognizing different hand signals. “Our work is based on optical pattern recognition,” explains IDMT project manager Valiantsin Hardzeyeu. “This technique mimics the way in which humans see things. To this end, we modeled the processes taking place in the human visual apparatus – from the point where the photons hit the retina to the stage in which they are processed in the visual cortex – in a computer simulation.”

A first prototype, which comprises an ‘intelligent’ camera connected to a computer with this new type of pattern recognition software, will be presented at the Fraunhofer stand. The camera will record visitors’ gestures, and the software behind it will analyze them and convert the hand signals into machine commands.

Source: Fraunhofer-Gesellschaft

Explore further: Earthquake simulation tops one quadrillion flops

add to favorites email to friend print save as pdf

Related Stories

New avatars capable of laughing

Apr 04, 2014

Today's computer-based avatars lack one of our most deeply rooted human characteristics: laughter. Computer scientists have now teamed up with psychologists to give avatars the ability to laugh.

Researchers launch new FaceFries app

Mar 19, 2014

Want fries with your social media? That's a question Simon Fraser University researcher Steve DiPaola, co-creator of a new app, is asking with the launch of FaceFries.

BoomRoom's sound by design uses array of loudspeakers

Jan 31, 2014

(Phys.org) —Jörg Müller, a researcher at the Technical University of Berlin, thinks of ways to make effective use of sounds in human-computer interactions. He and his team explore how to direct sounds ...

Recommended for you

Dish Network denies wrongdoing in $2M settlement

8 hours ago

The state attorney general's office says Dish Network Corp. will reimburse Washington state customers about $2 million for what it calls a deceptive surcharge, but the satellite TV provider denies any wrongdoing.

Yahoo sees signs of growth in 'core' (Update)

8 hours ago

Yahoo reported a stronger-than-expected first-quarter profit Tuesday, results hailed by chief executive Marissa Mayer as showing growth in the Web giant's "core" business.

Intel reports lower 1Q net income, higher revenue

8 hours ago

Intel's earnings fell in the first three months of the year amid a continued slump in the worldwide PC market, but revenue grew slightly because of solid demand for tablet processors and its data center services.

Earthquake simulation tops one quadrillion flops

10 hours ago

A team of computer scientists, mathematicians and geophysicists at Technische Universitaet Muenchen (TUM) and Ludwig-Maximillians Universitaet Muenchen (LMU) have – with the support of the Leibniz Supercomputing ...

Twitter buys data analytics partner Gnip

11 hours ago

Twitter says it has bought its data partner Gnip, which provides analysis of the more than 500 million tweets its users share each day—to advertisers, academic institutions, politicians and other customers.

User comments : 0

More news stories

Intel reports lower 1Q net income, higher revenue

Intel's earnings fell in the first three months of the year amid a continued slump in the worldwide PC market, but revenue grew slightly because of solid demand for tablet processors and its data center services.

Low Vitamin D may not be a culprit in menopause symptoms

A new study from the Women's Health Initiative (WHI) shows no significant connection between vitamin D levels and menopause symptoms. The study was published online today in Menopause, the journal of The North American Menopa ...

Astronomers: 'Tilt-a-worlds' could harbor life

A fluctuating tilt in a planet's orbit does not preclude the possibility of life, according to new research by astronomers at the University of Washington, Utah's Weber State University and NASA. In fact, ...