Multimedia system provides new view of musical performance

Feb 03, 2009

Musicians can now use 3D computer analysis to radically improve their technique, thanks to the latest research in multimedia technology from the University of Leeds.

Dr Kia Ng of the University's Faculty of Engineering and School of Music has devised a way to use motion capture to record a musician's posture and movement as they play and then map the results against ideal performance settings. The system is known as the i-Maestro 3D Augmented Mirror (AMIR) and is a powerful tool for music teachers, students and experienced or professional musicians to improve their technique.

"Learning to play an instrument is a physical activity," said Dr Ng. "If a student develops a bad posture early on, this can be potentially very damaging to their career and our system can help teachers to easily identify problems. Similarly, the system enables experienced musicians to make small changes in gesture and posture that can improve the sound they make.

"Many musicians already use video recordings of their performance to analyse technique, but this only provides a 2D image. The 3D image and analysis provided by AMIR will be of immense value to musicians and teachers alike."

The prototype has been designed for stringed instruments such as violin and cello but could be adapted for other instruments. Small markers are attached to key points on the instrument, the musician's body and the bow. As the musician plays, 12 cameras record the movement at very fast speed - 200 frames per second - and map the instrument in 3D onto the screen. Bow speed, angle and position are all measured for real-time analysis and feedback, as is - for violinists - the pressure by which the instrument is held on the shoulder. Dr Ng has even incorporated a Wii Balance Board to include data on the musician's balance as they play.

The musician or teacher can then hear and see a video of the performance alongside an on-screen analysis of posture and bow technique, which if necessary they can work through frame by frame or bow stroke by bow stroke.

Dr Ng, himself a violinist, explains: "What makes a great sound is difficult to analyse, but with technique, some things come down to basic physics. If the bow is held perpendicular to the string and parallel to the bridge, the minimum effort will produce the maximum result. Our system can measure this and show musicians exactly when their technique becomes less effective."

A video of the system in action with cello and violin can be seen on the i-Maestro website at www.i-maestro.org where the prototype software can also be downloaded free of charge.

Dr Ng hopes that AMIR will in the future be used widely by teachers and music colleges as a useful tool alongside more traditional teaching methods. However, at present the motion caption hardware needed to work the system can cost anywhere between £5,000 and - for the sophisticated set-up used at the University of Leeds - £100,000. Until hardware costs reduce to make such systems more widely affordable, he plans to offer musicians the opportunity to use the system at the University of Leeds Interdisciplinary Centre for Scientific Research in Music (www.icsrim.org.uk) laboratories on a consultancy basis.

Source: University of Leeds

Explore further: Avatars make the Internet sign to deaf people

add to favorites email to friend print save as pdf

Related Stories

Passports for 'endangered' musical instruments

Mar 13, 2013

From pianos with ivory keys to violin bows crafted with tortoise shell, musical instruments made from protected species will soon be able to roam the globe more easily—with their own passports.

Can you hear the shape of a graph?

Jul 05, 2011

The Bristol exhibit focuses on quantum graphs, which can be thought of as a network of guitar strings tied to each other.  When a system like this vibrates it produces a series of sounds with different ...

Recommended for you

Avatars make the Internet sign to deaf people

Aug 29, 2014

It is challenging for deaf people to learn a sound-based language, since they are physically not able to hear those sounds. Hence, most of them struggle with written language as well as with text reading ...

Chameleon: Cloud computing for computer science

Aug 26, 2014

Cloud computing has changed the way we work, the way we communicate online, even the way we relax at night with a movie. But even as "the cloud" starts to cross over into popular parlance, the full potential ...

User comments : 0