Computer scientists synthesize cloth sounds for animation

Sep 26, 2012 by Bill Steele
Computer scientists synthesize cloth sounds for animation
To match sound to action, a computer looks at frames of computer animation of a piece of fabric and measures how much parts of the image are raised or lowered as a basis for synthesizing "crumpling" sound.

(Phys.org)—Someday, virtual reality may be so well done that you won't be able to tell it's just computer animation. To make that happen we'll have to hear it as well as see it, from dramatic noises like the sound of a breaking glass to subtle details like the rustling of clothing as characters move.

Cornell researchers who specialize in creating sound for are adding the sounds of cloth to their repertoire. Doug James and Steve Marschner, associate professors of , and graduate student Steven An reported their method of synthesizing cloth sounds at the SIGGRAPH conference Aug. 7 in Los Angeles.

Soundtracks for animated movies are still mostly constructed by hand, manually matching recorded sounds to the action. But it's not always possible to find appropriate sounds, and in interactive virtual environments you won't always know ahead of time what the action will be. One solution is to synthesize sound based on the physics of what's happening in a scene.

This video is not supported by your browser at this time.
First person view: Using binaural recordings, realistic sounds of a person's clothing can be added to enhance virtual reality.

The Cornell researchers built a system that uses digitally synthesized sound as a sort of to insert real recorded sounds. They synthesize an approximate sound based on two characteristics of the action: , as the cloth moves against itself or other objects, and crumpling, as the cloth deforms. To teach their system to synthesize these sounds, the researchers made controlled recordings of friction sounds by spinning a cloth-covered roller while holding a piece of cloth against it, and crumpling sounds by manipulating a piece of cloth while avoiding any sliding contact.

But the computer-synthesized sounds still aren't very realistic, so the "target" sound is broken up into microsecond chunks, which the computer matches against a database of similar chunks of recorded real cloth sounds which are then reassembled smoothly to produce a convincing soundtrack. Binaural recordings were included to produce "first-person" sounds for a viewer wearing stereo headphones.

A human sound designer oversees the process, choosing the type of fabric and tweaking the rules by which sounds are matched.

Computer scientists synthesize cloth sounds for animation
To create a model of friction sounds, researchers hold a piece of cloth against a motorized cloth roller. Ultimately, rotor speed will be matched to the speed of movement in an animation.

In part, James said, the approach was inspired by speech synthesis. "People thought they could synthesize speech using oscillator models," James explained, "but it didn't sound realistic, sort of like 'Speak & Spell,' so they ended up concatenating bits of recorded human speech for realism."

As demonstrations, the researchers created the sounds of cotton and polyester sheets being spread over a couch, a person exercising in a nylon windbreaker and corduroy pants, and the same person boxing from a first-person viewpoint.

The computation is still time-consuming, so it's not ready for real-time virtual environments yet. Getting the ideal sound still involves trial and error, the researchers reported. "This is a proof of concept," James said. "One next step would be optimizing the method to make it run in real time."

Practical applications may require a more elaborate database of cloth sounds, and still to be added are impact and tension sounds like those of a flag whipping in the wind.

Explore further: Ant colonies help evacuees in disaster zones

Related Stories

Computers synthesize sounds to go with graphics

Aug 09, 2011

Computer-generated imagery usually relies on recorded sound to complete the illusion. Recordings can, however, limit the range of sounds you can produce, especially in future virtual reality environments where ...

Scientists make animated collisions sounds realistic

Aug 21, 2012

(Phys.org) -- Bang. Clatter. Tinkle. Jingle. When solid objects collide in the real world we hear a sharp impact sound, sometimes followed by a ringing aftershock. Creating sounds like that to accompany computer ...

Researchers bring noise to virtual worlds

Oct 27, 2009

(PhysOrg.com) -- Computer scientists have developed a method to synthesize the sounds of cymbals, falling garbage cans and lids, and plastic water-cooler bottles and recycling bins.

Sounds of northern lights are born close to ground

Jul 09, 2012

For the first time, researchers at Aalto University in Finland have located where the sounds associated with the northern lights are created. The auroral sounds that have been described in folktales and by wilderness wanderers ...

Recommended for you

'Chief Yahoo' David Filo returns to board

32 minutes ago

Yahoo announced the nomination of three new board members, including company co-founder David Filo, who earned the nickname and formal job title of "Chief Yahoo."

Fired Yahoo exec gets $58M for 15 months of work

42 minutes ago

Yahoo's recently fired chief operating officer, Henrique de Castro, left the Internet company with a severance package of $58 million even though he lasted just 15 months on the job.

Simplicity is key to co-operative robots

8 hours ago

A way of making hundreds—or even thousands—of tiny robots cluster to carry out tasks without using any memory or processing power has been developed by engineers at the University of Sheffield, UK.

Freight train industry to miss safety deadline

9 hours ago

The U.S. freight railroad industry says only one-fifth of its track will be equipped with mandatory safety technology to prevent most collisions and derailments by the deadline set by Congress.

User comments : 0

More news stories

'Chief Yahoo' David Filo returns to board

Yahoo announced the nomination of three new board members, including company co-founder David Filo, who earned the nickname and formal job title of "Chief Yahoo."

Simplicity is key to co-operative robots

A way of making hundreds—or even thousands—of tiny robots cluster to carry out tasks without using any memory or processing power has been developed by engineers at the University of Sheffield, UK.

Floating nuclear plants could ride out tsunamis

When an earthquake and tsunami struck the Fukushima Daiichi nuclear plant complex in 2011, neither the quake nor the inundation caused the ensuing contamination. Rather, it was the aftereffects—specifically, ...

New clinical trial launched for advance lung cancer

Cancer Research UK is partnering with pharmaceutical companies AstraZeneca and Pfizer to create a pioneering clinical trial for patients with advanced lung cancer – marking a new era of research into personalised medicines ...

More vets turn to prosthetics to help legless pets

A 9-month-old boxer pup named Duncan barreled down a beach in Oregon, running full tilt on soft sand into YouTube history and showing more than 4 million viewers that he can revel in a good romp despite lacking ...