Researchers create sounds of animated things breaking (w/ Video)

Jul 14, 2010 By Bill Steele
The sounds of a piggy bank being smashed include the initial fracture and the clattering of ceramic shards and coins hitting the floor. Cornell computer scientists are synthesizing these sounds to match computer-animated images.

(PhysOrg.com) -- A delicate wine glass shatters on the floor. A rock is thrown through a window. A child smashes his piggy bank. Dramatic moments like these in an animated movie or video game or some future virtual reality won't seem realistic unless the sound matches the action.

Cornell computer scientists are developing technology to synthesize the sounds that go with computer-animated images of brittle materials being smashed. Their methods look at the computer graphic model that underlies the animation, figure out how a corresponding real object would vibrate when fractured, and how that vibration would create sound.

For years, filmmakers have dubbed in recorded sound, but it is difficult to get it to match the action. And in a game or virtual reality, programmers can't know in advance just how hard or far a wine glass will fall.

"We'll compute motion and appearance and sound in an integrated way," explained Doug James, associate professor of . "We won't just compute motion and appearance and have the sound as something you bolt on afterward."

James and graduate student Changxi Zheng will present their work at the SIGGRAPH 2010 conference in Los Angeles July 25-29. It follows previous work in which James and his students created sounds of dripping and splashing water and the clattering falls of such thin-walled objects as garbage cans and plastic bottles.

This video is not supported by your browser at this time.

When a rigid object is struck or hits the floor, it can be deformed until the stress exceeds its strength, and then it shatters, releasing the energy stored by the deformation. Research shows that the sound comes mainly from the way all the little pieces vibrate just after the break, rather than from the whole object in the instant of fracture.

The computer calculates how each shard will vibrate when given the amount of energy stored by the deformation. The calculation takes into account how far the object was dropped or how hard it was thrown to determine the amount of energy available. It assumes that more fracture will occur in areas that have been strained the most, creating more, smaller pieces.

In most cases, the initial smash is followed by the scattering of debris on the floor. To accelerate computation of those sounds, the sound synthesis program can treat each irregularly shaped shard as an ellipsoid of similar size and shape. Then it draws on preloaded "soundbanks" -- not recorded sounds, but computer routines for calculating the vibration of ellipsoids of various sizes and materials.

To refine their procedures, the researchers smashed real objects and photographed them with high-speed, slow-motion cameras and recorded the sounds, then compared the actual sounds with their computed simulations. To demonstrate the results, they created videos of the smashing of a wine glass, a dinner plate, a glass table filled with dinnerware and a piggy bank full of coins.

All of this is still an approximation of the real thing, James admitted, but it's a start, he said, and one that needs to be done now to make ready for the coming of new, more powerful systems.

"This is the first time anybody's ever built computer-synthesized models of these events with ," he said. "Everything after this will be better. The future's going to be very different. Computers will be a thousand times as fast. It will be insane."

Explore further: IBM Watson Group to transform the consumer shopping experience

Related Stories

Researchers bring noise to virtual worlds

Oct 27, 2009

(PhysOrg.com) -- Computer scientists have developed a method to synthesize the sounds of cymbals, falling garbage cans and lids, and plastic water-cooler bottles and recycling bins.

Modeling the Sound of Music

Oct 03, 2005

If a musical instrument has never been built before, how can you know what it will sound like? That's the question UC Berkeley graduate student Cynthia Bruyns is answering with Vibration Lab, software she's designing to simulate ...

DISSCO makes 'music' for Argonne, UIUC researchers

Jun 21, 2005

A mathematician and a musician have teamed up to create a new computer program that both composes music and creates the instrumentation to play it. The software is available for free from SourceForge.net.

Swedish research can make Super Mario more realistic

Mar 06, 2009

Computer games are being developed at an ever more rapid pace, and the technical demands are rising, not least regarding graphics boards.  At Mälardalen University in Sweden, researchers have now found a solution to ...

Study: Female crickets steered by sound

Oct 11, 2005

Cambridge University scientists in England say female crickets react to, and make steering corrections toward, the sound pattern in a male cricket's song.

Recommended for you

Computer-assisted accelerator design

Apr 22, 2014

Stephen Brooks uses his own custom software tool to fire electron beams into a virtual model of proposed accelerator designs for eRHIC. The goal: Keep the cost down and be sure the beams will circulate in ...

First steps towards "Experimental Literature 2.0"

Apr 21, 2014

As part of a student's thesis, the Laboratory of Digital Humanities at EPFL has developed an application that aims at rearranging literary works by changing their chapter order. "The human simulation" a saga ...

User comments : 3

Adjust slider to filter visible comments by rank

Display comments: newest first

plasticpower
5 / 5 (2) Jul 15, 2010
Must require a ton of processing power, but this could be just as big for video games as the introduction of crazy powerful GPUs! I suspect this might require it's own SPU (Sound Processing Unit) that's very similar to how the graphics card is put together and it will probably have it's own "language" like OpenGL (OpenAL maybe?). Very cool!
Javinator
not rated yet Jul 15, 2010
I don't really think this technology would be required for filmmakers since they could just keep doing trial an error with sound effects (ie. breaking glasses) that would likely be much cheaper than using this technology.

As plastic said though, this could be extrememly cool for video games at some point in the future when we can feasibly incorporate the technology it into video game consoles/computers.
PinkElephant
not rated yet Jul 15, 2010
The key thing about this, is that it uses real physics simulation.

I've long expected, and still do, that the GPUs will split up into graphics-only vs. physics-only subspecies.

For pure graphics, you can't beat the quality and versatility of raytracing, with real-time adaptive NURBS tessellation, geometry texturing, radiocity and caustics: for that, we already have prototypes that can produce almost a frame per second on relatively simple scenes with top-flight graphics cards, so we're probably about 5 to 10 years away from being able to handle very complex scenes in real-time on a dedicated card. Then it's on to volumetric visualization...

But to concurrently perform complex physics simulations (kinematics, shattering, fluid dynamics, reverse-kinematics on articulated skeletons, sound generation, etc.) one would need a whole new class and type of dedicated hardware. The "language" is already emerging, embodied in projects like PhysX, ODE, and Havoc.

The Matrix is coming...

More news stories

SK Hynix posts Q1 surge in net profit

South Korea's SK Hynix Inc said Thursday its first-quarter net profit surged nearly 350 percent from the previous year on a spike in sales of PC memory chips.

FCC to propose pay-for-priority Internet standards

The Federal Communications Commission is set to propose new open Internet rules that would allow content companies to pay for faster delivery over the so-called "last mile" connection to people's homes.

Brazil enacts Internet 'Bill of Rights'

Brazil's president signed into law on Wednesday a "Bill of Rights" for the digital age that aims to protect online privacy and promote the Internet as a public utility by barring telecommunications companies ...

Is nuclear power the only way to avoid geoengineering?

"I think one can argue that if we were to follow a strong nuclear energy pathway—as well as doing everything else that we can—then we can solve the climate problem without doing geoengineering." So says Tom Wigley, one ...

When things get glassy, molecules go fractal

Colorful church windows, beads on a necklace and many of our favorite plastics share something in common—they all belong to a state of matter known as glasses. School children learn the difference between ...

FDA proposes first regulations for e-cigarettes

The federal government wants to prohibit sales of electronic cigarettes to minors and require approval for new products and health warning labels under regulations being proposed by the Food and Drug Administration.