Computers synthesize sounds to go with graphics

Computers synthesize sounds to go with graphics
Rapid movement of heated gases generates high-frequency sounds that are an important part of the sound of fire, but computer-generated images don't simulate those details. Cornell researchers synthesize low-frequency sounds to match the graphics, then map in the highs based on the sounds of real fire. Credit: Doug James

Computer-generated imagery usually relies on recorded sound to complete the illusion. Recordings can, however, limit the range of sounds you can produce, especially in future virtual reality environments where you can't always know ahead of time what the action will be.

Doug James, associate professor of , develops to synthesize sound "on the fly" based on simulated physics models. His research group has already invented sound synthesis methods for splashing fluids, fracturing glass and other solids, and noisy thin shells such as sheet metal.

Now they have devised methods for synthesizing more realistic sounds of hard objects colliding and the roar of fire. They report their latest work at the 2011 ACM SIGGRAPH conference in Vancouver, Canada, Aug. 7-11.

To synthesize collision sounds, the computer calculates the forces computer-generated objects would exert if they were real, how those forces would make the objects vibrate and how those vibrations transfer to the air to make sound.

Previous efforts often assumed that the contacting objects were rigid, but "in reality, there is no such thing as a rigid object," the researchers say. Objects vibrate when they collide, which can produce further chattering and squeaking sounds. Resolving all the frictional contact events between rapidly vibrating objects is computationally expensive. To speed things up, their algorithm simulates only the fraction of contacts and vibrations needed to synthesize the sound.

"If you set a box on a there are many contacts," James explained. "We identify a small set of physically faithful contacts, the simplest set that will get the job done." Similarly, the algorithm selects from many different ways the objects can vibrate, ignoring those that produce inaudible sounds.

Demonstrations include the sound of a ruler overhanging the edge of table and "buzzing" when plucked, pounding on a table to make dishes clatter and ring and the varied sounds of a Rube Goldberg machine that rolls marbles into a cup that moves a lever that pushes a bunny into a shopping cart that rolls downhill.

Fire is animated by mimicking the chemical reactions and fluid-like flow of burning gases. But flame sounds come from things that happen very rapidly in the expanding gases, and computer animators do not need to model those costly details to get good-looking flames.

"We can simulate the low-frequency sound for flame animations, but not all the fiery details," James explained, "so we rely on models based on real fire sounds to paint the fiery details onto the low-frequency sound." This keeps the cost down and has the added benefit of providing "style controls" based on what type of real flame sound is used as input data.

The researchers demonstrated with a fire-breathing dragon statue, a candle in the wind, a torch swinging through the air, a jet of flame injected into a small chamber and a burning brick. The last simulation was run with several variations of the method, and the results compared with a high-speed video and sound recording of a real burning brick.

The flame sounds still lack some of the "whoosh" in the middle frequencies, the researchers noted, and the process is not yet ready for real-time applications, needing several hours to synthesize fire sound for a short video clip, but they say more powerful parallel hardware will help.

Explore further

Researchers bring noise to virtual worlds

Provided by Cornell University
Citation: Computers synthesize sounds to go with graphics (2011, August 9) retrieved 29 September 2020 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Feedback to editors

User comments