Computers synthesize sounds to go with graphics

August 9, 2011 By Bill Steele, Cornell University
Rapid movement of heated gases generates high-frequency sounds that are an important part of the sound of fire, but computer-generated images don't simulate those details. Cornell researchers synthesize low-frequency sounds to match the graphics, then map in the highs based on the sounds of real fire. Credit: Doug James

Computer-generated imagery usually relies on recorded sound to complete the illusion. Recordings can, however, limit the range of sounds you can produce, especially in future virtual reality environments where you can't always know ahead of time what the action will be.

Doug James, associate professor of , develops to synthesize sound "on the fly" based on simulated physics models. His research group has already invented sound synthesis methods for splashing fluids, fracturing glass and other solids, and noisy thin shells such as sheet metal.

Now they have devised methods for synthesizing more realistic sounds of hard objects colliding and the roar of fire. They report their latest work at the 2011 ACM SIGGRAPH conference in Vancouver, Canada, Aug. 7-11.

To synthesize collision sounds, the computer calculates the forces computer-generated objects would exert if they were real, how those forces would make the objects vibrate and how those vibrations transfer to the air to make sound.

Previous efforts often assumed that the contacting objects were rigid, but "in reality, there is no such thing as a rigid object," the researchers say. Objects vibrate when they collide, which can produce further chattering and squeaking sounds. Resolving all the frictional contact events between rapidly vibrating objects is computationally expensive. To speed things up, their algorithm simulates only the fraction of contacts and vibrations needed to synthesize the sound.

"If you set a box on a there are many contacts," James explained. "We identify a small set of physically faithful contacts, the simplest set that will get the job done." Similarly, the algorithm selects from many different ways the objects can vibrate, ignoring those that produce inaudible sounds.

Demonstrations include the sound of a ruler overhanging the edge of table and "buzzing" when plucked, pounding on a table to make dishes clatter and ring and the varied sounds of a Rube Goldberg machine that rolls marbles into a cup that moves a lever that pushes a bunny into a shopping cart that rolls downhill.

Fire is animated by mimicking the chemical reactions and fluid-like flow of burning gases. But flame sounds come from things that happen very rapidly in the expanding gases, and computer animators do not need to model those costly details to get good-looking flames.

"We can simulate the low-frequency sound for flame animations, but not all the fiery details," James explained, "so we rely on models based on real fire sounds to paint the fiery details onto the low-frequency sound." This keeps the cost down and has the added benefit of providing "style controls" based on what type of real flame sound is used as input data.

The researchers demonstrated with a fire-breathing dragon statue, a candle in the wind, a torch swinging through the air, a jet of flame injected into a small chamber and a burning brick. The last simulation was run with several variations of the method, and the results compared with a high-speed video and sound recording of a real burning brick.

The flame sounds still lack some of the "whoosh" in the middle frequencies, the researchers noted, and the process is not yet ready for real-time applications, needing several hours to synthesize fire sound for a short video clip, but they say more powerful parallel hardware will help.

Explore further: Researchers bring noise to virtual worlds

Related Stories

Researchers bring noise to virtual worlds

October 27, 2009

( -- Computer scientists have developed a method to synthesize the sounds of cymbals, falling garbage cans and lids, and plastic water-cooler bottles and recycling bins.

In Brief: The cocktail party problem

January 4, 2011

People can identify a repeating sound in a noisy room, but only when the noise includes mixtures of distinct distracting sounds, according to a study published this week in the Proceedings of the National Academy of Sciences.

Modeling the Sound of Music

October 3, 2005

If a musical instrument has never been built before, how can you know what it will sound like? That's the question UC Berkeley graduate student Cynthia Bruyns is answering with Vibration Lab, software she's designing to simulate ...

Recommended for you

Fish-inspired material changes color using nanocolumns

March 20, 2019

Inspired by the flashing colors of the neon tetra fish, researchers have developed a technique for changing the color of a material by manipulating the orientation of nanostructured columns in the material.


Adjust slider to filter visible comments by rank

Display comments: newest first

5 / 5 (1) Aug 09, 2011
That's wonderful hush, but doesn't have any relevance to the physics of generating sounds.
not rated yet Aug 10, 2011
games! games! games!

5 / 5 (2) Aug 10, 2011
At least explain why you feel human hearing does not have "any relevance to the physics of generating sounds."

Because sound is generated by an object and propagates through a medium which then vibrates your eardrum. Your hearing is a byproduct not the cause of the sound. The processing of sound would seem to be a lot easier than the generation because there are only two contact points in hearing a sound when there can be hundreds to millions of creations points for sound.
5 / 5 (1) Aug 10, 2011
I'd like to hear what some cosmological images sound like.
5 / 5 (1) Aug 12, 2011
Thanks Hush1, I understand that.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.