G-Speak will make the keyboard and mouse obsolete (w/ Video)

March 1, 2010 by Lin Edwards report

(PhysOrg.com) -- The mouse may soon become obsolete, with interfaces that interpret gestures rapidly approaching a stage at which they can be released for general consumers.

A new system being developed by Oblong Industries harnesses gesture technology that uses special surfaces and displays that can track hand movements, providing the operator is wearing the special conducting . The system works with images and videos, and has been dubbed the “G-Speak” spatial operating environment (SOE).

In a G-Speak environment everything on screen can be directly manipulated by gestures such as pointing, and the system simplifies the control of real-world objects such as robots or vehicles, and allows physical tools and interfaces to be used as input devices.

John Underkoffler, co-founder of Oblong Industries, says G-Speak should be available for consumers within five years, and its combination of interpretation of , real-world pixels and recombinant networking represents the biggest step forward in computer interfaces since 1984.

Underkoffler says applications for the new interface technology include the operation of 3-D interfaces, processing of large data sets, integration of multiple computers and large screens into building-scale work environments. G-speak networking’s collection of library components allows applications to scale dynamically and transparently across multiple computers, and in a LAN network this makes more effective use of CPU power and provides support for applications enabling collaborative work over the network. It will also be possible to add new people and computers to the network, and to add new code at run-time.

G-speak controls applications through hand poses, pointing, and hand movements, with input from several hands simultaneously being fully supported. Hand and finger motions are tracked to an accuracy of 0.1 mm at 100 Hz.

Using gestural I/Os is much more efficient at complex navigation tasks and selection/sorting, than the mouse and keyboard, but both will still be available for use when appropriate.

G-Speak is already in use by Fortune 50 companies, and by some universities and government agencies, and a software development kit running on Linux and Mac is already available.

Explore further: Hand movements challenge computer mouse

More information:
-- Oblong.com - oblong.com/article/086E19gPvDcktAf9.html
-- More videos: vimeo.com/user922585/videos
-- Another video on YouTube: www.youtube.com/watch?v=KqyHM29VNqM

Related Stories

Hand movements challenge computer mouse

June 1, 2006

The computer mouse could become a thing of the past and hand gestures a thing of the future if Los Angeles-based G-speak has its way, a report said.

Gesture recognition

December 18, 2008

A system that can recognize human gestures could provide a new way for people with physical disabilities to interact with computers. A related system for the able bodied could also be used to make virtual worlds more realistic. ...

Gesture-driven computers

February 29, 2008

It isn’t always easy to communicate with a computer. Two Fraunhofer Institutes will be presenting new possibilities of man-machine interaction at CeBIT in Hanover (Germany) on March 4 through 9. They will demonstrate how ...

A 'hands-on' approach to computers

April 6, 2009

(PhysOrg.com) -- At a time when ever more aspects of our lives are moving toward the virtual, online world -- stores, newspapers, games and even social interactions -- Hiroshi Ishii seems to be swimming against that current: ...

Recommended for you

Europe's first 'eTree' puts down roots in central France

May 29, 2017

A solar tree with giant square leaves that convert sunlight into electricity was unveiled in the central French town of Nevers on Monday, allowing passers-by to charge their phones, surf the internet... or just enjoy the ...

Chinese fans trash blackout as Google AI wins again

May 25, 2017

Chinese netizens fumed Thursday over a government ban on live coverage of Google algorithm AlphaGo's battle with the world's top Go player, as the programme clinched their three-match series in the ancient board game.


Adjust slider to filter visible comments by rank

Display comments: newest first

4.4 / 5 (8) Mar 01, 2010
So, they replaced the simple, inexpensive 3-button mouse with expensive gloves and complex gestures?
4.9 / 5 (8) Mar 01, 2010
I've been trying out some of these techniques and really, after a few minutes, your hands get really tired. I don't think this tech will go very far without addressing that.
4.3 / 5 (6) Mar 01, 2010
I think the whole touch screen / gesture thing is an absolute gimmick, totally unusable in the real world.
Can you imagine using this kind of thing for 3-4 hours straight? Your arms would be dying!
A mouse is extremely pinpoint accurate, fast, and requires very tiny movements. You cannot improve on a good mouse.
4.9 / 5 (7) Mar 01, 2010
I agree. The mouse is here to stay until a direct brain interface comes about. This is obviously inspired by Minority Report.
2 / 5 (4) Mar 01, 2010
The whole triple screen is powered by three genetic mutants floating in a tank out of sight.
5 / 5 (1) Mar 01, 2010
Just preaching to the choir here - a few years ago i bought a P5 glove interface. It was supposed to be mostly for games, but i was more interested in developing new ways for interacting with the main OS. I wrote some new software for it, but like a few here already mentioned, your arms get tired after a relatively short period of time. The only way i could see this system would be useful is if it did NOT require a glove interface, and could be tethered to various computers around your house. You could walk by and interact with it quickly without sitting down with a keyboard and mouse.
3 / 5 (2) Mar 01, 2010
Why not 2 cameras reading your hands in 3d? I would like to be able to manipulate a model in 3d, spinning it, moving faces, adding elements until my arms got tired. This with a mouse-like touchpad and keyboard would be good. Mechanics, cooks, etc use their arms all the time- maybe you'd get used to it, as in exercise.
2.3 / 5 (3) Mar 01, 2010
I've used systems silimar to this one a year ago. They were excellent then and they'll be even more excellent going forward.

You'd think it would make you tired but it really doesn't after the first few uses. When it comes to sorting through large data stacks technology like this is invaluable.
1 / 5 (2) Mar 01, 2010
Well at least this would help eliminate carpal tunnel syndrome, since your movements are more natural. You might just develop a case of arm-al tunnel syndrome though.
5 / 5 (2) Mar 01, 2010
I thinkt he problem with this system is you have to hold your arms out in front of you.... very tiring.

If you think of a the way a craftsman manipulates small objects - hands on his lap, or wrists resting on the table in front of him, and small movements with hands and fingers - not the huge gestures of 'Minority Report'
Mar 01, 2010
This comment has been removed by a moderator.
4.7 / 5 (3) Mar 01, 2010
What would happen if someone approached you as you are at your computer and your hands grab a stack of papers, or you swat at a fly, scratch your nose, or anything else causes hand gestures not intended for computer interactions?
1 / 5 (1) Mar 01, 2010
What would happen if someone approached you as you are at your computer and your hands grab a stack of papers, or you swat at a fly, scratch your nose, or anything else causes hand gestures not intended for computer interactions?

Depends on the software and hardware. The ones I used had a "trigger" button on the palm. You had to put pressure on it to have your movements register.
5 / 5 (1) Mar 01, 2010
Which will take longer to arrive, g-speak or affordable giant flat screen monitors?
1 / 5 (2) Mar 01, 2010
The best use for this would be to emulate mouse without a mouse. Imagine you are somewhere out with a notebook and dont have a mouse with you. Touchpad is impractical. Just use your hand like you would with a mouse, easy and not exhausting.
not rated yet Mar 01, 2010
Which will take longer to arrive, g-speak or affordable giant flat screen monitors?

3 / 5 (2) Mar 01, 2010
Gee, I broke my clavicle (collar bone) now I can't use that computer for many weeks - waving an arm connected to the shoulder of the broken clavicle is extremely painful.

Gee, I came back from Iraq with just one arm, now I can't use that computer.

And so on and so on.

Requiring arms to do what one wrist and a finger does is overkill.
1 / 5 (1) Mar 01, 2010
Requiring arms to do what one wrist and a finger does is overkill.

How do you feel about that whole "hands at 10 and 2" thing when you drive?
Mar 01, 2010
This comment has been removed by a moderator.
3 / 5 (2) Mar 01, 2010
There are 100s of muscles/fibers in our faces and forearms ready to be mapped directly to input.

Before we get to fully neural interfaces, I expect we will see a massive improvement over the keyboard/mouse paradigm by taking advantage of the many subtle twitches & contractions that our muscles can perform for a very small energy cost.
5 / 5 (1) Mar 01, 2010
This is all for people who never learned how to type. I don't see how anything short of a neural interface can be more efficient than twitching your fingers and your wrists occasionally.
3 / 5 (2) Mar 01, 2010
amen cmn, yeah lets get rid of the precision of the mouse, where you move your hands very small distances to get large changes... no, that's so... old school. Let's make ourselves look like complete dorks waving our hands all over the place like an idiot.

I just don't think these people get it. It's not a brand new way of controlling your computer, it's just trading gloves for a mouse and you STILL MOVE THEM IN SPECIFIC WAYS to get a result.

When these guys come up with something that lets the OS literally react and respond to what I am thinking I want to do, THEN I'll be impressed.
3.7 / 5 (3) Mar 01, 2010
How are you supposed to watch porn with this?

Very carefully... as they showed near the end of the Family Guy Empire Strike Back episode, where "Luke" gets a new cybernetic hand and asks the doc if he's good to go, the doc answers "Practice with the new hand, or you'll rip your ******* right off."
1 / 5 (2) Mar 02, 2010
One idea is to have a set area where your hands (without gloves) are tracked. A virtual keyboard is displayed on the bottom of that area for simple data entry. Then when your hands are above and not touching the virtual keyboard below you can manipulate any object on the screen.
- Arms can rest on the desk (elbows on the table)
- Small movements for simple data entry (virtual keyboard).
- Any display can be used.
- Small tracking area makes for small movements overall.

On the other hand touch screens on a larger scale would be just as affective too. And those are current marketable technologies.
5 / 5 (1) Mar 02, 2010
I prefer brain-computer interfaces
This is no practical in my opinion, especially for the masses, maybe for some specifical nisches and it could be used only for short period of time....
1 / 5 (3) Mar 03, 2010
I enjoy see all the lazy people who sit at there computer complain about needing to work to accomplish a task on a computer. maybe... just maybe, an interface like this will turn people off of sitting in front of their computer for hours at a time, and actually get out and get fresh air and excersize.
5 / 5 (4) Mar 03, 2010
maybe... just maybe, an interface like this will turn people off of sitting in front of their computer for hours at a time, and actually get out and get fresh air and excersize

Perhaps you need to spend a little less time outside exercising and a little more time on dictionary.com and grammar.net.
3 / 5 (2) Mar 06, 2010
I think we're holding on to a dinosaur here. It is like watching the original Star Trek and seeing toggle switches and reels on computers. The better solution is no physical interface. Use brain interface technology which has been featured on this site.
Mar 06, 2010
This comment has been removed by a moderator.
not rated yet Mar 07, 2010
Geez, what was his last invention?.. a pen the size of a welding tank on a 12x12ft notepad? I agree with everybody. Id rather move a mouse a couple inches than have to bone up on airplane tarmac marshalling techniques. I also agree that that the mouse is so efficient, that its most effective replacement will be a neuro interface.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.