Sound With Space and Motion; Real Life Reflection

Jun 29, 2004
Sound With Space and Motion (UC Davis graduate student Bob Dalton tries out the sound studio)

A conversation snaps around you. A string trio plays in front of you as you turn your attention from cello to violin. A soprano walks across the stage, and you turn your head to follow her voice. Conventional sound recording cannot capture those experiences -- but an invention by audio engineers at the University of California, Davis is changing that.

Motion-tracked binaural sound ( MTB ) captures cues for direction, distance and movement and the subtleties of natural, ambient sound that other systems don't. Developed by Ralph Algazi, Richard Duda and Dennis Thompson at the Interface Laboratory in the UC Davis Center for Image Processing and Integrated Computing (CIPIC), the patent-pending technique uses off-the-shelf equipment that won't break the bank.

For the listener, it's an eerie experience of listening to invisible people talking, playing music or singing around you while you can turn your attention from one to another.

"Conventional audio playback doesn't reflect how you hear in real life," Algazi said. "Your body, the shape of your head and the room acoustics all affect how you hear."

Conventional binaural recording uses microphones embedded in a dummy head to record sound, so that playback through headphones mimics the recording. That method does a fair job of reproducing sounds to the left and right but not to the front, where sounds appear to come from immediately in front of or behind your head. Furthermore, it doesn't allow for movement, Duda said.

"The problem is that people do not keep still," Duda said.

The new method records through multiple microphones (eight for voice, 16 for music) spaced around a head-sized ball or cylinder. The sound is played back through headphones with a small tracking device attached to the top to follow head movements. As you turn your head while listening, the system mixes sound from different microphones, reproducing what you would hear if you were in the room.

Listeners can move their heads to locate a sound source, turn to "face" a person speaking and tell when sound sources are nearby or far away. In addition, MTB captures the ambient sounds of the location, so you recognize the echoes in a church or the confines of a conference room.

The engineers have made sample recordings with musicians from the UC Davis music department and visiting classical and bluegrass musicians.

"The system can capture the sound of instruments much more fully than a conventional single microphone. It captures changes in sound and the effects of a room in a way that is much closer to reality," Algazi said.

"I think they have something really wonderful," said Pablo Ortiz, chair of the Department of Music at UC Davis.

"The thing that's interesting to me is the way that it records space," Ortiz said.

William Beck, a composer of electronic music and lecturer in the music department, said that live recordings could be a major application.

"The 'being there' feel is something people would really like," Beck said.

The technique could also be used for teleconferencing, computer games and virtual- or augmented-reality systems. The work, which is supported by grants from the National Science Foundation, was presented at the 116th Convention of the Audio Engineering Society held in May 2004 in Berlin, Germany.

The CIPIC Interface Laboratory is a multidisciplinary research center that studies human perception and its role in the interface between humans and machines.

The original release can be found here.

Explore further: Hackers force message on websites via US firm

add to favorites email to friend print save as pdf

Related Stories

Rosetta's comet sings strange, seductive song

Nov 12, 2014

Scientists can't figure exactly why yet, but Comet 67P/Churyumov-Gerasimenko has been singing since at least August. Listen to the video – what do you think? I hear a patter that sounds like frogs, purring ...

Scientist creates automatic birdsong recognition app

Oct 30, 2014

Dr Dan Stowell, an EPSRC Research Fellow in QMUL's School of Electrical Engineering and Computer Science, has used a grant from Queen Mary Innovation to develop a prototype for an app that turns his research ...

Recommended for you

After a data breach, it's consumers left holding the bag

5 minutes ago

Shoppers have launched into the holiday buying season and retailers are looking forward to year-end sales that make up almost 20% of their annual receipts. But as you check out at a store or click "purchase" on your online shopping cart ...

Can we create an energy efficient Internet?

15 minutes ago

With the number of Internet connected devices rapidly increasing, researchers from Melbourne are starting a new research program to reduce energy consumption of such devices.

Brain inspired data engineering

1 hour ago

What if next-generation ICT systems could be based on the brain's structure and its cognitive and adaptive processes? A groundbreaking paradigm of brain-inspired intelligent ICT architectures is being born.

E-Voting: Risky technology or great improvement?

1 hour ago

On this forthcoming weekend the Australian state election takes place, and in Victoria State they will be using a new e-voting system to improve secrecy, reliability and user-friendliness. But how secure are such systems? ...

Faradair team determined to make hybrid BEHA fly

2 hours ago

Aiming to transform their concept into a real success, the Faradair team behind a six-seat Bio-Electric-Hybrid-Aircraft (BEHA) have taken this hybrid aircraft project into a crowdfunding campaign on Kickstarter. ...

User comments : 0

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.