Multimedia system provides new view of musical performance

Feb 03, 2009

Musicians can now use 3D computer analysis to radically improve their technique, thanks to the latest research in multimedia technology from the University of Leeds.

Dr Kia Ng of the University's Faculty of Engineering and School of Music has devised a way to use motion capture to record a musician's posture and movement as they play and then map the results against ideal performance settings. The system is known as the i-Maestro 3D Augmented Mirror (AMIR) and is a powerful tool for music teachers, students and experienced or professional musicians to improve their technique.

"Learning to play an instrument is a physical activity," said Dr Ng. "If a student develops a bad posture early on, this can be potentially very damaging to their career and our system can help teachers to easily identify problems. Similarly, the system enables experienced musicians to make small changes in gesture and posture that can improve the sound they make.

"Many musicians already use video recordings of their performance to analyse technique, but this only provides a 2D image. The 3D image and analysis provided by AMIR will be of immense value to musicians and teachers alike."

The prototype has been designed for stringed instruments such as violin and cello but could be adapted for other instruments. Small markers are attached to key points on the instrument, the musician's body and the bow. As the musician plays, 12 cameras record the movement at very fast speed - 200 frames per second - and map the instrument in 3D onto the screen. Bow speed, angle and position are all measured for real-time analysis and feedback, as is - for violinists - the pressure by which the instrument is held on the shoulder. Dr Ng has even incorporated a Wii Balance Board to include data on the musician's balance as they play.

The musician or teacher can then hear and see a video of the performance alongside an on-screen analysis of posture and bow technique, which if necessary they can work through frame by frame or bow stroke by bow stroke.

Dr Ng, himself a violinist, explains: "What makes a great sound is difficult to analyse, but with technique, some things come down to basic physics. If the bow is held perpendicular to the string and parallel to the bridge, the minimum effort will produce the maximum result. Our system can measure this and show musicians exactly when their technique becomes less effective."

A video of the system in action with cello and violin can be seen on the i-Maestro website at www.i-maestro.org where the prototype software can also be downloaded free of charge.

Dr Ng hopes that AMIR will in the future be used widely by teachers and music colleges as a useful tool alongside more traditional teaching methods. However, at present the motion caption hardware needed to work the system can cost anywhere between £5,000 and - for the sophisticated set-up used at the University of Leeds - £100,000. Until hardware costs reduce to make such systems more widely affordable, he plans to offer musicians the opportunity to use the system at the University of Leeds Interdisciplinary Centre for Scientific Research in Music (www.icsrim.org.uk) laboratories on a consultancy basis.

Source: University of Leeds

Explore further: Coping with floods—of water and data

add to favorites email to friend print save as pdf

Related Stories

Passports for 'endangered' musical instruments

Mar 13, 2013

From pianos with ivory keys to violin bows crafted with tortoise shell, musical instruments made from protected species will soon be able to roam the globe more easily—with their own passports.

Can you hear the shape of a graph?

Jul 05, 2011

The Bristol exhibit focuses on quantum graphs, which can be thought of as a network of guitar strings tied to each other.  When a system like this vibrates it produces a series of sounds with different ...

Recommended for you

Coping with floods—of water and data

Dec 19, 2014

Halloween 2013 brought real terror to an Austin, Texas, neighborhood, when a flash flood killed four residents and damaged roughly 1,200 homes. Following torrential rains, Onion Creek swept over its banks and inundated the ...

Cloud computing helps make sense of cloud forests

Dec 17, 2014

The forests that surround Campos do Jordao are among the foggiest places on Earth. With a canopy shrouded in mist much of time, these are the renowned cloud forests of the Brazilian state of São Paulo. It is here that researchers ...

User comments : 0

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.