Multimedia system provides new view of musical performance

Feb 03, 2009

Musicians can now use 3D computer analysis to radically improve their technique, thanks to the latest research in multimedia technology from the University of Leeds.

Dr Kia Ng of the University's Faculty of Engineering and School of Music has devised a way to use motion capture to record a musician's posture and movement as they play and then map the results against ideal performance settings. The system is known as the i-Maestro 3D Augmented Mirror (AMIR) and is a powerful tool for music teachers, students and experienced or professional musicians to improve their technique.

"Learning to play an instrument is a physical activity," said Dr Ng. "If a student develops a bad posture early on, this can be potentially very damaging to their career and our system can help teachers to easily identify problems. Similarly, the system enables experienced musicians to make small changes in gesture and posture that can improve the sound they make.

"Many musicians already use video recordings of their performance to analyse technique, but this only provides a 2D image. The 3D image and analysis provided by AMIR will be of immense value to musicians and teachers alike."

The prototype has been designed for stringed instruments such as violin and cello but could be adapted for other instruments. Small markers are attached to key points on the instrument, the musician's body and the bow. As the musician plays, 12 cameras record the movement at very fast speed - 200 frames per second - and map the instrument in 3D onto the screen. Bow speed, angle and position are all measured for real-time analysis and feedback, as is - for violinists - the pressure by which the instrument is held on the shoulder. Dr Ng has even incorporated a Wii Balance Board to include data on the musician's balance as they play.

The musician or teacher can then hear and see a video of the performance alongside an on-screen analysis of posture and bow technique, which if necessary they can work through frame by frame or bow stroke by bow stroke.

Dr Ng, himself a violinist, explains: "What makes a great sound is difficult to analyse, but with technique, some things come down to basic physics. If the bow is held perpendicular to the string and parallel to the bridge, the minimum effort will produce the maximum result. Our system can measure this and show musicians exactly when their technique becomes less effective."

A video of the system in action with cello and violin can be seen on the i-Maestro website at www.i-maestro.org where the prototype software can also be downloaded free of charge.

Dr Ng hopes that AMIR will in the future be used widely by teachers and music colleges as a useful tool alongside more traditional teaching methods. However, at present the motion caption hardware needed to work the system can cost anywhere between £5,000 and - for the sophisticated set-up used at the University of Leeds - £100,000. Until hardware costs reduce to make such systems more widely affordable, he plans to offer musicians the opportunity to use the system at the University of Leeds Interdisciplinary Centre for Scientific Research in Music (www.icsrim.org.uk) laboratories on a consultancy basis.

Source: University of Leeds

Explore further: MIT groups develop smartphone system THAW that allows for direct interaction between devices

add to favorites email to friend print save as pdf

Related Stories

Passports for 'endangered' musical instruments

Mar 13, 2013

From pianos with ivory keys to violin bows crafted with tortoise shell, musical instruments made from protected species will soon be able to roam the globe more easily—with their own passports.

Can you hear the shape of a graph?

Jul 05, 2011

The Bristol exhibit focuses on quantum graphs, which can be thought of as a network of guitar strings tied to each other.  When a system like this vibrates it produces a series of sounds with different ...

Recommended for you

Who drives Alibaba's Taobao traffic—buyers or sellers?

Sep 18, 2014

As Chinese e-commerce firm Alibaba prepares for what could be the biggest IPO in history, University of Michigan professor Puneet Manchanda dug into its Taobao website data to help solve a lingering chicken-and-egg question.

Computerized emotion detector

Sep 16, 2014

Face recognition software measures various parameters in a mug shot, such as the distance between the person's eyes, the height from lip to top of their nose and various other metrics and then compares it with photos of people ...

Cutting the cloud computing carbon cost

Sep 12, 2014

Cloud computing involves displacing data storage and processing from the user's computer on to remote servers. It can provide users with more storage space and computing power that they can then access from anywhere in the ...

Teaching computers the nuances of human conversation

Sep 12, 2014

Computer scientists have successfully developed programs to recognize spoken language, as in automated phone systems that respond to voice prompts and voice-activated assistants like Apple's Siri.

User comments : 0