Researchers create sounds of animated things breaking (w/ Video)

Jul 14, 2010 By Bill Steele
The sounds of a piggy bank being smashed include the initial fracture and the clattering of ceramic shards and coins hitting the floor. Cornell computer scientists are synthesizing these sounds to match computer-animated images.

(PhysOrg.com) -- A delicate wine glass shatters on the floor. A rock is thrown through a window. A child smashes his piggy bank. Dramatic moments like these in an animated movie or video game or some future virtual reality won't seem realistic unless the sound matches the action.

Cornell computer scientists are developing technology to synthesize the sounds that go with computer-animated images of brittle materials being smashed. Their methods look at the computer graphic model that underlies the animation, figure out how a corresponding real object would vibrate when fractured, and how that vibration would create sound.

For years, filmmakers have dubbed in recorded sound, but it is difficult to get it to match the action. And in a game or virtual reality, programmers can't know in advance just how hard or far a wine glass will fall.

"We'll compute motion and appearance and sound in an integrated way," explained Doug James, associate professor of . "We won't just compute motion and appearance and have the sound as something you bolt on afterward."

James and graduate student Changxi Zheng will present their work at the SIGGRAPH 2010 conference in Los Angeles July 25-29. It follows previous work in which James and his students created sounds of dripping and splashing water and the clattering falls of such thin-walled objects as garbage cans and plastic bottles.

This video is not supported by your browser at this time.

When a rigid object is struck or hits the floor, it can be deformed until the stress exceeds its strength, and then it shatters, releasing the energy stored by the deformation. Research shows that the sound comes mainly from the way all the little pieces vibrate just after the break, rather than from the whole object in the instant of fracture.

The computer calculates how each shard will vibrate when given the amount of energy stored by the deformation. The calculation takes into account how far the object was dropped or how hard it was thrown to determine the amount of energy available. It assumes that more fracture will occur in areas that have been strained the most, creating more, smaller pieces.

In most cases, the initial smash is followed by the scattering of debris on the floor. To accelerate computation of those sounds, the sound synthesis program can treat each irregularly shaped shard as an ellipsoid of similar size and shape. Then it draws on preloaded "soundbanks" -- not recorded sounds, but computer routines for calculating the vibration of ellipsoids of various sizes and materials.

To refine their procedures, the researchers smashed real objects and photographed them with high-speed, slow-motion cameras and recorded the sounds, then compared the actual sounds with their computed simulations. To demonstrate the results, they created videos of the smashing of a wine glass, a dinner plate, a glass table filled with dinnerware and a piggy bank full of coins.

All of this is still an approximation of the real thing, James admitted, but it's a start, he said, and one that needs to be done now to make ready for the coming of new, more powerful systems.

"This is the first time anybody's ever built computer-synthesized models of these events with ," he said. "Everything after this will be better. The future's going to be very different. Computers will be a thousand times as fast. It will be insane."

Explore further: Computerized emotion detector

Related Stories

Researchers bring noise to virtual worlds

Oct 27, 2009

(PhysOrg.com) -- Computer scientists have developed a method to synthesize the sounds of cymbals, falling garbage cans and lids, and plastic water-cooler bottles and recycling bins.

Modeling the Sound of Music

Oct 03, 2005

If a musical instrument has never been built before, how can you know what it will sound like? That's the question UC Berkeley graduate student Cynthia Bruyns is answering with Vibration Lab, software she's designing to simulate ...

DISSCO makes 'music' for Argonne, UIUC researchers

Jun 21, 2005

A mathematician and a musician have teamed up to create a new computer program that both composes music and creates the instrumentation to play it. The software is available for free from SourceForge.net.

Swedish research can make Super Mario more realistic

Mar 06, 2009

Computer games are being developed at an ever more rapid pace, and the technical demands are rising, not least regarding graphics boards.  At Mälardalen University in Sweden, researchers have now found a solution to ...

Study: Female crickets steered by sound

Oct 11, 2005

Cambridge University scientists in England say female crickets react to, and make steering corrections toward, the sound pattern in a male cricket's song.

Recommended for you

Computerized emotion detector

Sep 16, 2014

Face recognition software measures various parameters in a mug shot, such as the distance between the person's eyes, the height from lip to top of their nose and various other metrics and then compares it with photos of people ...

Cutting the cloud computing carbon cost

Sep 12, 2014

Cloud computing involves displacing data storage and processing from the user's computer on to remote servers. It can provide users with more storage space and computing power that they can then access from anywhere in the ...

Teaching computers the nuances of human conversation

Sep 12, 2014

Computer scientists have successfully developed programs to recognize spoken language, as in automated phone systems that respond to voice prompts and voice-activated assistants like Apple's Siri.

Mapping the connections between diverse sets of data

Sep 12, 2014

What is a map? Most often, it's a visual tool used to demonstrate the relationship between multiple places in geographic space. They're useful because you can look at one and very quickly pick up on the general ...

User comments : 3

Adjust slider to filter visible comments by rank

Display comments: newest first

plasticpower
5 / 5 (2) Jul 15, 2010
Must require a ton of processing power, but this could be just as big for video games as the introduction of crazy powerful GPUs! I suspect this might require it's own SPU (Sound Processing Unit) that's very similar to how the graphics card is put together and it will probably have it's own "language" like OpenGL (OpenAL maybe?). Very cool!
Javinator
not rated yet Jul 15, 2010
I don't really think this technology would be required for filmmakers since they could just keep doing trial an error with sound effects (ie. breaking glasses) that would likely be much cheaper than using this technology.

As plastic said though, this could be extrememly cool for video games at some point in the future when we can feasibly incorporate the technology it into video game consoles/computers.
PinkElephant
not rated yet Jul 15, 2010
The key thing about this, is that it uses real physics simulation.

I've long expected, and still do, that the GPUs will split up into graphics-only vs. physics-only subspecies.

For pure graphics, you can't beat the quality and versatility of raytracing, with real-time adaptive NURBS tessellation, geometry texturing, radiocity and caustics: for that, we already have prototypes that can produce almost a frame per second on relatively simple scenes with top-flight graphics cards, so we're probably about 5 to 10 years away from being able to handle very complex scenes in real-time on a dedicated card. Then it's on to volumetric visualization...

But to concurrently perform complex physics simulations (kinematics, shattering, fluid dynamics, reverse-kinematics on articulated skeletons, sound generation, etc.) one would need a whole new class and type of dedicated hardware. The "language" is already emerging, embodied in projects like PhysX, ODE, and Havoc.

The Matrix is coming...