Using gesture and voice to control 3D molecular graphics (w/ Video)

Oct 17, 2013
This is Dr. Kenny Sabir, one of the key developers, using the Molecular Control Toolkit. Credit: Dr Sean O'Donoghue's Lab, CSIRO and Garvan Institute of Medical Research

Scientists from Sydney's Garvan Institute of Medical Research and CSIRO announced their 'Molecular Control Toolkit' at the 3rd IEEE Symposium on Biological Data Visualization in Atlanta, Georgia, USA on Sunday.

The is an innovative piece of software designed to connect commercially available gesture and voice recognition devices with any molecular graphics system, allowing people to control 3D representations of with their hands.

3D is now a standard feature of many computer games, and may become as pervasive as using a mouse or touchpad.

Researchers in the life sciences need to be able to visualise the structures of complex molecules in order to understand their function, model their behaviour and design better drugs.

By connecting the Molecular Control Toolkit with devices such as LeapMotion or Kinect, scientists can use hand movements to rotate a molecule, change the centre of rotation, zoom into and through a molecule, or select a structure of interest.

Previously, to carry out these functions, a researcher first had to learn complex keyboard and mouse combinations. The toolkit now allows intuitive manipulation of a molecule after just a few minutes of instruction.

Earlier this year, 18 life scientists working at Garvan, all with high computer literacy but little experience with 3D gesture control, were invited to test the toolkit. Each person was shown what to do, and then given a few simple tasks to perform. Most participants grasped the basics within half an hour.

This video is not supported by your browser at this time.

Professor Mike Rogers, biochemist and bone researcher, took part in this pilot study, and says he found the experience "fun, tactile and intuitive". "It would be a great tool for giving lectures or presentations, allowing you to manipulate molecules easily with your hands," he said.

"For example, it's a very visually powerful way of highlighting the parts of a protein that might be important for drug binding."

"To our knowledge, this is the first gesture toolkit for molecular graphics that supports multiple devices. It also has a set of controls sufficiently rich to be useful in the day-to-day work of a broad range of life scientists," said team leader Dr Seán O'Donoghue, who works at both Garvan and CSIRO.

A paper describing this work will appear shortly in the IEEE Digital Library.

Explore further: Elliptic Labs develops ultrasonic gesture control for hand-held devices

More information: aquaria.ws/

add to favorites email to friend print save as pdf

Related Stories

Gesture recognition

Dec 18, 2008

A system that can recognize human gestures could provide a new way for people with physical disabilities to interact with computers. A related system for the able bodied could also be used to make virtual worlds more realistic. ...

Microsoft hand research ripens Kinect for work (w/ video)

Mar 07, 2013

(Phys.org) —Beyond reading body motions, Kinect is getting a workup by researchers at Microsoft, now showing substantial control additions. Microsoft Research this week showed how Microsoft Kinect for Windows ...

Pioneering touchless surgical technology

Jun 01, 2012

Researchers and surgeons from King’s College London and Guy’s and St Thomas’ NHS Foundation Trust are piloting ‘touchless’ technology in the operating theatre.

Recommended for you

Oculus unveils new prototype VR headset

14 hours ago

Oculus has unveiled a new prototype of its virtual reality headset. However, the VR company still isn't ready to release a consumer edition.

Wireless sensor transmits tumor pressure

20 hours ago

The interstitial pressure inside a tumor is often remarkably high compared to normal tissues and is thought to impede the delivery of chemotherapeutic agents as well as decrease the effectiveness of radiation ...

Tim Cook puts personal touch on iPhone 6 launch

21 hours ago

Apple chief Tim Cook personally kicked off sales of the iPhone 6, joining in "selfies" and shaking hands with customers Friday outside the company's store near his Silicon Valley home.

Team improves solar-cell efficiency

Sep 19, 2014

New light has been shed on solar power generation using devices made with polymers, thanks to a collaboration between scientists in the University of Chicago's chemistry department, the Institute for Molecular ...

Calif. teachers fund to boost clean energy bets

Sep 19, 2014

The California State Teachers' Retirement System says it plans to increase its investments in clean energy and technology to $3.7 billion, from $1.4 billion, over the next five years.

User comments : 0