Engineering and Music: A Powerful Duet for Art and Science

An engineer with a love of music, and a musician who likes technology, Mark Bocko and Dave Headlam are both professors at the University of Rochester. For more than ten years their collaboration has been moving both fields forward.

"We very quickly realized that the things he was interested in and the things I was interested in, in music theory, were actually very similar," says Headlam, who teaches music theory at Rochester's Eastman School of Music.

Both are part of the university's Music Research Lab (MRL). Its goal is "to perform musically-informed research and to develop technologies that reflect the expertise of musicians as well as scientists and engineers."

Bocko, a professor of electrical and computer engineering, uses a computer to figure out with extraordinary precision what a musician is doing to create the sound.

"And so the whole idea is you want to capture the essence of the physics of how the instrument works," he says.

For instance, Bocko can study every aspect of how a clarinet player interacts with an instrument.

"So, what the computer learns is how hard they were blowing, the blowing pressure at every instant in time, what their mouth clamping force was on the reed, and the fingering they used," continues Bocko. "But, it's really how the more subtle inputs and the changes of the blowing pressure over time, and how things are connected together. It is learning those parameters from a performance that is the essential part of this."

On the engineering side, this kind of information has led to improvements in music compression.

Most of us are familiar with the musical compression of an iPod, or MP3 player. The basic idea with music compression is just to reduce the size of the file that you have to transmit or store so hundreds of songs can fit on a small device.

With help from the National Science Foundation, Bocko's team has tackled a different type of music compression. They have done it by analyzing both instruments and musicians to better understand what’s critical, and what's not, in a music file.

"If you look at how much data there is in an audio file, 1.5 million bits per second, and then you look at a musician playing an instrument, then you ask, well, how much information can a human actually transmit in a second? A musician can change their blowing pressure, the force of their lips on the reed, their fingerings, and all of that. But there’s no way that there's one and a half million bits of information being imparted by the musician to the instrument," explains Bocko.

So he has "taught" a computer how to play a clarinet, using precise acoustical measurements of the real instrument.

This "virtual instrument" enabled Bocko to compress a clarinet solo into a very tiny file; about 1000 times smaller than an MP3 file.

This knowledge won't be used to build a better iPod. What it is likely to do is improve other things that involve audio transmission. For instance, it could improve videoconferencing to get rid of that annoying lag when TV anchors are talking to reporters a half a world away.

"In the news business, if they are talking to someone in Afghanistan, and the anchor asks a question, the person is standing there for half a second like they are asleep or something. So reducing a lot of that latency is a good media application," notes Bocko.

Other possibilities using this improved compression include playing music over the Internet, with musicians performing the same piece of music in different cities. This telepresence could also link other professionals: from musicians to dancers to surgeons.

The engineering findings are also helping bring new tools to music teachers. By "seeing" what a musical note looks like on a computer, the research is adding a 21st century tweak to music education.

"In some respects, what goes on at Eastman could have gone on 150 years ago. You have a room with a teacher and an instrument like a violin; the technology was perfected in the 1600s," explains Headlam. "What we're trying to do is to find sort of the next stage of the practice room and the lesson, where students can use technology to try to get to that point at which they can play their instrument, and where that sort of kinetic and cognitive and musical aspects all come together. So they can just deal with musical content, rather than having to worry about technique."

It's a matter of adding another sense to music training. "Music students spend a lot of time with their ears, but you can also use your eyes and coordinate the two," continues Headlam. "So, you can imagine if you are playing something, you are looking at a screen, at some sort of oscilloscope display, and you see your line as you're playing, and then you see your teacher's line, and then those lines gradually come together as you are hearing what you want to do. Then at a certain point they come together, and in that way you have accomplished your goal by combining your sight and your hearing."

Bocko, who plays the bassoon, says his engineering research is often a combination of work and play.

"I am a musician of sorts, and I'm interested in really understanding what makes good music good. And so it's been a way for me to indulge my interest in music, by working with my colleagues at the Eastman School and immersing myself in this," he says.