Computers synthesize sounds to go with graphics

Aug 09, 2011 By Bill Steele
Rapid movement of heated gases generates high-frequency sounds that are an important part of the sound of fire, but computer-generated images don't simulate those details. Cornell researchers synthesize low-frequency sounds to match the graphics, then map in the highs based on the sounds of real fire. Credit: Doug James

Computer-generated imagery usually relies on recorded sound to complete the illusion. Recordings can, however, limit the range of sounds you can produce, especially in future virtual reality environments where you can't always know ahead of time what the action will be.

Doug James, associate professor of , develops to synthesize sound "on the fly" based on simulated physics models. His research group has already invented sound synthesis methods for splashing fluids, fracturing glass and other solids, and noisy thin shells such as sheet metal.

Now they have devised methods for synthesizing more realistic sounds of hard objects colliding and the roar of fire. They report their latest work at the 2011 ACM SIGGRAPH conference in Vancouver, Canada, Aug. 7-11.

To synthesize collision sounds, the computer calculates the forces computer-generated objects would exert if they were real, how those forces would make the objects vibrate and how those vibrations transfer to the air to make sound.

Previous efforts often assumed that the contacting objects were rigid, but "in reality, there is no such thing as a rigid object," the researchers say. Objects vibrate when they collide, which can produce further chattering and squeaking sounds. Resolving all the frictional contact events between rapidly vibrating objects is computationally expensive. To speed things up, their algorithm simulates only the fraction of contacts and vibrations needed to synthesize the sound.

"If you set a box on a there are many contacts," James explained. "We identify a small set of physically faithful contacts, the simplest set that will get the job done." Similarly, the algorithm selects from many different ways the objects can vibrate, ignoring those that produce inaudible sounds.

Demonstrations include the sound of a ruler overhanging the edge of table and "buzzing" when plucked, pounding on a table to make dishes clatter and ring and the varied sounds of a Rube Goldberg machine that rolls marbles into a cup that moves a lever that pushes a bunny into a shopping cart that rolls downhill.

Fire is animated by mimicking the chemical reactions and fluid-like flow of burning gases. But flame sounds come from things that happen very rapidly in the expanding gases, and computer animators do not need to model those costly details to get good-looking flames.

"We can simulate the low-frequency sound for flame animations, but not all the fiery details," James explained, "so we rely on models based on real fire sounds to paint the fiery details onto the low-frequency sound." This keeps the cost down and has the added benefit of providing "style controls" based on what type of real flame sound is used as input data.

The researchers demonstrated with a fire-breathing dragon statue, a candle in the wind, a torch swinging through the air, a jet of flame injected into a small chamber and a burning brick. The last simulation was run with several variations of the method, and the results compared with a high-speed video and sound recording of a real burning brick.

The flame sounds still lack some of the "whoosh" in the middle frequencies, the researchers noted, and the process is not yet ready for real-time applications, needing several hours to synthesize fire sound for a short video clip, but they say more powerful parallel hardware will help.

Explore further: Gamers' funding fuels meteoric rise of 'Star Citizen'

Related Stories

Researchers bring noise to virtual worlds

Oct 27, 2009

(PhysOrg.com) -- Computer scientists have developed a method to synthesize the sounds of cymbals, falling garbage cans and lids, and plastic water-cooler bottles and recycling bins.

In Brief: The cocktail party problem

Jan 04, 2011

People can identify a repeating sound in a noisy room, but only when the noise includes mixtures of distinct distracting sounds, according to a study published this week in the Proceedings of the National Academy of Sciences.

Modeling the Sound of Music

Oct 03, 2005

If a musical instrument has never been built before, how can you know what it will sound like? That's the question UC Berkeley graduate student Cynthia Bruyns is answering with Vibration Lab, software she's designing to simulate ...

Recommended for you

Gamers' funding fuels meteoric rise of 'Star Citizen'

16 hours ago

Chris Roberts' brain spun out a grand vision: a rich, immersive galaxy; exquisite spaceships traversing between infinite star systems with thousands of computer gamers manning the cockpits, racing, dogfighting and defending ...

User comments : 9

Adjust slider to filter visible comments by rank

Display comments: newest first

hush1
3 / 5 (2) Aug 09, 2011
Interesting that human hearing has no cost overhead. Always ready for real-time application, needing milliseconds to paint the details of the physical acoustical phenomena occurring. "On the fly" sound pattern recognition.
that_guy
5 / 5 (1) Aug 09, 2011
That's wonderful hush, but doesn't have any relevance to the physics of generating sounds.
hush1
not rated yet Aug 09, 2011
What physics to sound generation has meaning to life forms evolving without this range and spectrum or perception to one of Nature's physical properties?

At least explain why you feel human hearing does not have "any relevance to the physics of generating sounds."
iiibogdan
not rated yet Aug 10, 2011
games! games! games!

http://www.youtub...7CV6giII
asherdanner
5 / 5 (2) Aug 10, 2011
At least explain why you feel human hearing does not have "any relevance to the physics of generating sounds."

Because sound is generated by an object and propagates through a medium which then vibrates your eardrum. Your hearing is a byproduct not the cause of the sound. The processing of sound would seem to be a lot easier than the generation because there are only two contact points in hearing a sound when there can be hundreds to millions of creations points for sound.
Isaacsname
5 / 5 (1) Aug 10, 2011
I'd like to hear what some cosmological images sound like.
hush1
not rated yet Aug 10, 2011
@asherdanner
"The processing of sound would seem to be a lot easier than the generation..." - asherdanner
The mechanics of sound processing is related to sound generation. Both related mathematically by Fourier analysis.

Hearing is not a one way process. Brain signals send to the stereocilia of the ear will cause the stereocilia to vibrate.

The phenomenon is audibly measurable. The conjecture: That is a possible partial explanation for tinnitus.

@Isaacname
"I'd like to hear what some cosmological images sound like." - Isaacname

The audible equivalents for us to the sun (or any star) can be found octavely stepping up or down the fundamentals of the waves produced by the motion of a star's gases.

The step down or up octave equivalent (of gas motion) is the 'music'
(or if you will, Pythagoras' "harmony of the spheres" - somewhat removed from the original motivation behind the statement)
of the cosmos.
hush1
5 / 5 (1) Aug 10, 2011
@iiibogdan
A wonderful link. Thks. Worth repeating here:
http://www.youtub...7CV6giII
Isaacsname
5 / 5 (1) Aug 12, 2011
Thanks Hush1, I understand that.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.