Computer scientists synthesize cloth sounds for animation

September 26, 2012 by Bill Steele, Cornell University
Computer scientists synthesize cloth sounds for animation
To match sound to action, a computer looks at frames of computer animation of a piece of fabric and measures how much parts of the image are raised or lowered as a basis for synthesizing "crumpling" sound.

(—Someday, virtual reality may be so well done that you won't be able to tell it's just computer animation. To make that happen we'll have to hear it as well as see it, from dramatic noises like the sound of a breaking glass to subtle details like the rustling of clothing as characters move.

Cornell researchers who specialize in creating sound for are adding the sounds of cloth to their repertoire. Doug James and Steve Marschner, associate professors of , and graduate student Steven An reported their method of synthesizing cloth sounds at the SIGGRAPH conference Aug. 7 in Los Angeles.

Soundtracks for animated movies are still mostly constructed by hand, manually matching recorded sounds to the action. But it's not always possible to find appropriate sounds, and in interactive virtual environments you won't always know ahead of time what the action will be. One solution is to synthesize sound based on the physics of what's happening in a scene.

First person view: Using binaural recordings, realistic sounds of a person's clothing can be added to enhance virtual reality.

The Cornell researchers built a system that uses digitally synthesized sound as a sort of to insert real recorded sounds. They synthesize an approximate sound based on two characteristics of the action: , as the cloth moves against itself or other objects, and crumpling, as the cloth deforms. To teach their system to synthesize these sounds, the researchers made controlled recordings of friction sounds by spinning a cloth-covered roller while holding a piece of cloth against it, and crumpling sounds by manipulating a piece of cloth while avoiding any sliding contact.

But the computer-synthesized sounds still aren't very realistic, so the "target" sound is broken up into microsecond chunks, which the computer matches against a database of similar chunks of recorded real cloth sounds which are then reassembled smoothly to produce a convincing soundtrack. Binaural recordings were included to produce "first-person" sounds for a viewer wearing stereo headphones.

A human sound designer oversees the process, choosing the type of fabric and tweaking the rules by which sounds are matched.

Computer scientists synthesize cloth sounds for animation
To create a model of friction sounds, researchers hold a piece of cloth against a motorized cloth roller. Ultimately, rotor speed will be matched to the speed of movement in an animation.

In part, James said, the approach was inspired by speech synthesis. "People thought they could synthesize speech using oscillator models," James explained, "but it didn't sound realistic, sort of like 'Speak & Spell,' so they ended up concatenating bits of recorded human speech for realism."

As demonstrations, the researchers created the sounds of cotton and polyester sheets being spread over a couch, a person exercising in a nylon windbreaker and corduroy pants, and the same person boxing from a first-person viewpoint.

The computation is still time-consuming, so it's not ready for real-time virtual environments yet. Getting the ideal sound still involves trial and error, the researchers reported. "This is a proof of concept," James said. "One next step would be optimizing the method to make it run in real time."

Practical applications may require a more elaborate database of cloth sounds, and still to be added are impact and tension sounds like those of a flag whipping in the wind.

Explore further: Computers synthesize sounds to go with graphics

Related Stories

Computers synthesize sounds to go with graphics

August 9, 2011

Computer-generated imagery usually relies on recorded sound to complete the illusion. Recordings can, however, limit the range of sounds you can produce, especially in future virtual reality environments where you can't always ...

Scientists make animated collisions sounds realistic

August 21, 2012

( -- Bang. Clatter. Tinkle. Jingle. When solid objects collide in the real world we hear a sharp impact sound, sometimes followed by a ringing aftershock. Creating sounds like that to accompany computer animation ...

Researchers bring noise to virtual worlds

October 27, 2009

( -- Computer scientists have developed a method to synthesize the sounds of cymbals, falling garbage cans and lids, and plastic water-cooler bottles and recycling bins.

Sounds of northern lights are born close to ground

July 9, 2012

For the first time, researchers at Aalto University in Finland have located where the sounds associated with the northern lights are created. The auroral sounds that have been described in folktales and by wilderness wanderers ...

Recommended for you

Top takeaways from Consumers Electronics Show

January 13, 2018

The 2018 Consumer Electronics Show, which concluded Friday in Las Vegas, drew some 4,000 exhibitors from dozens of countries and more than 170,000 attendees, showcased some of the latest from the technology world.

Finnish firm detects new Intel security flaw

January 12, 2018

A new security flaw has been found in Intel hardware which could enable hackers to access corporate laptops remotely, Finnish cybersecurity specialist F-Secure said on Friday.


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.