Operating a computer by gesture only

Operating computers without touching them, using only hand and arm gestures: it sounds futuristic, but it's already possible. Researcher Wim Fikkert of the Centre for Telematics and Information Technology of the University of Twente, The Netherlands, investigated the gestures that people would naturally use to operate a computer in this way. Remarkably, most test subjects chose the same gestures of their own accord.

We are accustomed to operating computers with a keyboard and mouse, and touchscreens have been with us for several years now. The next step is that we will be able to operate computers without touching anything.

Wim Fikkert of the University of Twente investigated how people could operate computers using just a few hand and arm gestures. The starting point of his research was not the technological aspect, but the gestures that people would use naturally.

Remarkably, most test subjects chose the same gestures of their own accord. The users chose a separate gesture for each command, and also stuck to their choices. Another striking finding was that the users explicitly changed the shape of their hands at the beginning of a gesture and relaxed at the end of it.

Fikkert used various different test designs for his experiments. In the simplest test the subjects thought they were controlling a computer with their gestures, when in fact someone else was doing so. In the most advanced test the subjects themselves interacted with a 4 x 1.5 metre screen at the University of Twente's Smart Experience Laboratory. They operated the system using wireless lasers on the backs of their hands for pointing and small buttons on their fingers for giving commands.