Mouseless, the 'invisible' computer mouse (w/ Video)

Jul 08, 2010 by Lin Edwards report

(PhysOrg.com) -- Mouseless is a computer mouse that allows you to interact with a computer with a mouse in the same way as usual - except that there is no mouse hardware. The researchers call it an "invisible mouse."

A group of scientists working with the Fluid Interfaces Group at the Massachusetts Institute of Technology (MIT) Media Lab in Cambridge Massachusetts, were interested in removing the requirement for a physical , while still allowing computer users to interact with their computers in a way with which they were familiar.

The researchers, Pranav Mistry, Liyan Chang, and Pattie Maes, developed an infrared (IR) laser beam and associated camera that could be incorporated into the computer so that a plane of IR laser would be created just above the surface on which the is resting. The user acts as though a physical mouse were present and the is intersected by the hand, and parts of the hand are shown up as bright spots of light that change position as the hand moves. The built-in camera then interprets the changes in position of the hand and fingers and translates them as moves of the mouse and clicks on the two buttons, and the on the screen moves as if the user was operating a physical mouse.

This video is not supported by your browser at this time.
Mouseless is an invisible computer mouse.

There are no plans for commercializing the "invisible mouse," but the prototype Mouseless was built for around $20 USD. The MIT group of researchers are now working on improvements to the recognition and tracking algorithms with the aim of building up an expanded command library. This may in the future lead to more complex than is possible at present, and could ultimately give the Mouseless a number of advantages over a physical mouse, since the number of functions handled by a physical mouse is limited.

Mouseless working prototype system

The MIT Media Lab’s Fluid Interfaces group works on ways of integrating the world of digital information and services more naturally into our normal lives, by designing intuitive and intelligent interfaces.

Pranav Mistry is a PhD student and research assistant at MIT Media Lab. His project “SixthSense” won the Popular Science Invention of the Year award in 2009.

• Learn about becoming PhysOrg.com sponsor

Explore further: Tailored 'activity coaching' by smartphone

More information: www.pranavmistry.com/projects/mouseless/

Related Stories

Movea's Gyration Air Mouse

Sep 24, 2008

Movea´s Gyration Air Mouse works both in the air and on the desk. Wave your Air Mouse in the air and navigate your mouse pointer on your desktop. Movea's MotionSense technology that enables it to work both ...

Hand movements challenge computer mouse

Jun 01, 2006

The computer mouse could become a thing of the past and hand gestures a thing of the future if Los Angeles-based G-speak has its way, a report said.

Mouse adapter for hand tremor sufferers

Mar 15, 2005

IBM researchers invent a new assitivecomputer mouse adapter to help hand tremor sufferers use personal computers. It eliminates excessive cursor movement, thereby allowing more normal use of a personal computer. ...

Gesture recognition

Dec 18, 2008

A system that can recognize human gestures could provide a new way for people with physical disabilities to interact with computers. A related system for the able bodied could also be used to make virtual worlds more realistic. ...

Recommended for you

Tailored 'activity coaching' by smartphone

Oct 17, 2014

Today's smartphone user can obtain a lot of data about his or her health, thanks to built-in or separate sensors. Researcher Harm op den Akker of the University of Twente (CTIT Institute) now takes this health ...

WASP has printer, will travel, to make houses

Oct 16, 2014

At Maker Faire Rome, an Italian 3D printer company is demonstrating a tall, portable machine that will bring 3D-printed dwellings to impoverished countries. WASP has been exploring low-cost solutions to ...

User comments : 26

Adjust slider to filter visible comments by rank

Display comments: newest first

visual
4.5 / 5 (2) Jul 08, 2010
WTF, how can they not plan commercializing this? They can be selling one with every laptop, or even every desktop keyboard that is ever made from next year onwards...
CreepyD
2 / 5 (1) Jul 08, 2010
I can't see this ever being accurate enough.
For most people maybe, but for someone who is fast on a PC, you will NEVER replace a good mouse and keyboard.
Only a direct brain interface has that potential.
I do think people would buy this though if it was commercialized.
ArchV
2 / 5 (1) Jul 08, 2010
How do you click and drag? ... Gotcha.
stanfrax
1 / 5 (1) Jul 08, 2010
its a grat idea but it will put corporations at a loss having loosing a little profit margin - they want to keep the slave wheel turning - just another peice of held back tech like the everlasting battery and light bulb
Justavian
3.5 / 5 (2) Jul 08, 2010
I have a seven button mouse, and use six of those buttons constantly (the regular three buttons, plus copy, cut, and paste). Are they going to reproduce that? Also, there were non-physical laser keyboards at one point, and those obviously didn't catch on. I don't think this would be any different.
akademy
4 / 5 (1) Jul 08, 2010
How do you click and drag? ... Gotcha.

It looks like every "click" is a double tap of the finger. So I'd imagine a drag would be a single tap of the finger, until there's another single tap.
ArchV
1 / 5 (1) Jul 08, 2010
Because changing our normal single click we've been using for years to a double click wouldn't be confusing at all ;) I'm glad this won't be commercialized. Not that I care really, I wouldn't buy it :)
El_Nose
5 / 5 (1) Jul 08, 2010
@ArchV

You know before the web - almost everyone used double clicking - and outside of a browser you have to double click to do everything except changing context.

it would seem the click drag is safe -- this is implemented in the OS. the click on the border gives the border context, written into the program that describes border windows ( in windows everything on the screen except text is technically a window and they all have slightly different rules -- this is hidden from the user but programmers are aware of the difference) -- anyway clicking on a border to drag, once the initial click down is detected you enter a new part of the program that allows you to resize until a release of the mouse button is detected.

This is just a working prototype I wouldn;t expect that to be addressed at this stage but looks highly feasible.

this will probably not cut into anyones bottom line as hardware that has to move tends to be expensive to engineer. This is a sensor with software.
danlgarmstrong
not rated yet Jul 08, 2010
Wonder when someone will have a computer that reads sign language? Do they exist now...I'll google it and let you know...
Yep - of course
http://en.wikiped...ognition
AngryMoose
3 / 5 (2) Jul 08, 2010
Pretty sure the ends of your fingers would feel pretty strange after rubbing the desk for a few hours while gaming. Think I'll stick with my trusty mouse until the neuro interface makes it so I don't have to do anything other than breath
JJC
3 / 5 (1) Jul 08, 2010
It looks a bit laggy in the video, though maybe it is just the recording.

Also, the camera & laser would need to be MUCH smaller for this to be marketable.

And unlike a wireless mouse that can go anywhere (including switching to other side for lefties), this can only be used in a fairly small section to the right of the laptop.
denfire
4 / 5 (1) Jul 08, 2010
there is something there though.... if they can package it better and improve the software so that it becomes more accurate... i think they got a winner... ont surpised if apple gets to it in the near future!
Javinator
4 / 5 (1) Jul 08, 2010
" This may in the future lead to more complex gesture recognition than is possible at present, and could ultimately give the Mouseless a number of advantages over a physical mouse, since the number of functions handled by a physical mouse is limited."

I think once the research hits this point it will be much more commercially viable. Cool concept though.
ClickHere
4 / 5 (2) Jul 08, 2010
Mouseless mouse? Last three laptops I've owned have had mouseless mouses, they call it a touchpad, and it's already multi-gesture capable.

For the type of work I do on a computer I'd still rather have a multi-gesture capable 24" LCD, or better still, as others have mentioned, neural input.
newsreader
not rated yet Jul 08, 2010
I'd like to see an interface that allows you to position the mouse pointer with your eyes and then you hit a key on the keyboard to select.
SMMAssociates
not rated yet Jul 08, 2010
I hate touchpads, and rodents, but like trackballs....

But this might be a useful device for the more mobile folks who still hate touchpads :)....

(A built-in trackball on one notebook, some years ago, worked well enough to install the software for an external trackball.... Otherwise, it was a waste....)

GaryB
not rated yet Jul 08, 2010
I can no longer live without the scroll wheel. I suppose you could try to approximate that with middle finger gestures. Responsiveness and accuracy would be the key here. I think the camera better be faster than 30fps. More like 100fps.
Anthony_Casey
not rated yet Jul 08, 2010
1. Humans need a certain amount of tactile feed-back. I guess there is some from the desktop surface, but I'm not sure if that is enough to make it feel natural enough.
2. Once they get to the point where is can detect more sophisticated hand gestures, then it might work.
3. Move the camera around in front of the monitor, and turn the laser into a 3-d grid and you could have the old-school VR-type interface without the gloves.

But as-is, I don't see people buying into it.
trekgeek1
not rated yet Jul 08, 2010
Seems like the cursor was floating about even when the hand changed direction. Touch screen is the only alternative until we just have a neurological interface.
dirk_bruere
not rated yet Jul 08, 2010
OK - so why didn't the video camera shooting the YouTube video pick up the IR light?
trekgeek1
not rated yet Jul 08, 2010
OK - so why didn't the video camera shooting the YouTube video pick up the IR light?


Because that camera isn't designed to pick up that part of the spectrum. Some cameras will be able to detect infrared in black and white mode.
Neurons_At_Work
not rated yet Jul 08, 2010
I'd like to see an interface that allows you to position the mouse pointer with your eyes and then you hit a key on the keyboard to select.

They already have this, and I use it from time to time with the webcam built into my laptop. You don't even have to touch the keyboard; just pause on an area and it double-clicks for you. It's called 'Camera Mouse 2010'. Freeware, and fairly fun to use. It's at www.cameramouse.org.
Neurons_At_Work
not rated yet Jul 08, 2010
Also, if you check out the picture on the website next to 'More Information--', it does show the ability to left click, right click, move, scroll, and drag--all of the basic mouse functions seem to be addressed.

They REALLY ought to release the code and put the design on www.instructables.com or something--I've got $20 just waiting for something like this.
MorituriMax
not rated yet Jul 09, 2010
Yawn.

Yawn.

I really have to laugh when they try and take an input device and SHAZAM make it better! All they are doing is substituting a table surface for the mouse.

The table does not move like the mouse so you get that wonderful sensation, there is nothing revolutionary about how you tell it what you want to do, you do exactly the same basic thing you do with.. gasp, existing mice.

Yawn.

Yawn.
jt81ma
not rated yet Jul 10, 2010
Now lets take a look at this five years from now, when you aren't moving your hand, just minor finger movements. Think bio-metric scanners of today. Each fingernail is different- the laser and camera can focus on each.... now you have five times the capability of your mouse or other HID! By reducing the movements, you increase the focusing ability and accuracy, not to mention reducing your energy expenditure and chance of RMIs.

Just think about it, if I need to move the mouse across the screen, I simply move my index finger and the system gauges speed and direction and reacts accordingly until it recognizes the finger return to approximated original locale. Your middle finger can still be scroll, an that leaves your thumb and other two fingers for other directional, preset or macro commands..... or better yet, multiple commands at the same time!

I cannot wait to see where they take this and how quickly they increase accuracy!
Jimbaloid
not rated yet Jul 13, 2010
My physical mouse is set so sensitive I do not move my hand for some operations, merely flex my fingers. I too am skeptical for the accuracy and lack of anything tactile. Then there is one quality of the 'traditional' mouse that many attempts to 'better' it fail on, I can lift my hand away from the physical mouse and leave the pointer to be still in one place. I can also lift the mouse to 'reset' the relative location on the table - distinguishing it from a tablet device.