Cut-and-paste simplicity for computer animation

Jul 27, 2010
Cut-and-paste simplicity for computer animation

Tools developed by European researchers bring cut-and-paste simplicity to gaming and animation. Users will be able to cut-and-paste complex elements like emotion, tone of voice and facial expression, making compelling new content, cheaply and quickly.

There is an explosion. Victims lie maimed or dead around the street. You are part of the emergency response and you have to quickly decide who needs urgent medical treatment. Breathing rate, skin tone, the wound severity and mental acuity are key indicators, and you quickly perform brief confirming tests as you move through the scene.

This is an animation, but it is much richer and more detailed than the current triage simulations available on the healthcare training market. What’s more, the simulation was developed more quickly and cheaply than the current state of the art.

It is also a clear demonstration of 21st century audiovisual animation, where recently developed European technology has brought the cut-and-paste simplicity of the internet to animation, audio-processing and for audiovisual elements.

Cut-and-paste simplicity

It is the work of the Salero project, a large continent-wide effort to streamline the production process and maximise content reuse in computer gaming and animation for online and broadcast entertainment.

Thirteen partners spent over €13 million (€8 million provided by the EU) to create two dozen applications, tools and showcases in an all-points efforts to dramatically improve audiovisual animation workflow. Audio and video transmission, processing and transformation are all affected by the broad range of tools and techniques developed by the project over the last four years.

The fundamental concept behind Salero was to create systems that would allow audio and to be easily and quickly repurposed for many different scenes within one project, and for transferring and adapting content between different projects.

In the long term, film and TV producers, animators, video game developers and performing arts groups will be able to mix and match digital elements simply. And they will be able to quickly and convincingly modify and adapt the content to suit their specific needs.

Synthetic emotions

The project focused on three main areas; audio processing, computer animation and semantic search. In each area, they produced tools that will enable existing media to be found and adapted to new content, either automatically or semi-automatically.

With the synthetic emotional model, for example, artists can animate a large emotional range by simply changing two variables, called activation and evaluation. Activation controls the strength of the emotion while evaluation controls the positive and negative balance between, say, a smile and a frown. Using just these two variables an animator is able to create dozens of expressions in a fraction of the time it takes currently.

Another tool, the Maskle, provides an easy way to transfer already defined emotions to a new face. In a first step, the animator selects ‘handles’ on the characters face, at the top, bottom and sides of the lips, for instance, and so defines basic expressions simply by moving these handles. The Maskle is then applied to a newly designed face and can help to specify a spectrum of emotions based on the defined basic expressions using the synthetic emotional model.

But Salero’s animation tools go way beyond facial expression. They work on body mechanics, too. The user can set a range of variables for a character, including his or her gender, ethnicity, age and weight, with each element modifying the appearance, according to Georg Thallinger, a researcher with Joanneum Research and coordinator of the Salero project.

“But these variables also impact the character’s gait and body movement. Using other tools we have developed, the animator can then simply point to an area where the character must go, and the software chooses the path and animates the character variables, and the quality of the surface along the route,” he tells ICT Results.

The required gait is determined automatically, so if a character must move across sand, or tarmac, body movement adapts appropriately.

Sophisticated semi-automation

This is an enormously sophisticated, semi-automated process, and it will save thousands of production man-hours. Tools like these, and others developed by Salero, will ultimately mean that state-of-the-art animation effects reach beyond big budget movie-making.

Audio processing, too, received a big boost in Salero. Here the team developed audio processing and synthesising tools to make voice recording cheaper and faster. Audio transformation, for example, can take a voice recording and change the gender, age, speed, timbre and pitch of the voice, easily adapting it from a strong, young woman to a weak old man.

Even the emotional stress of the voice can be adapted, from joyful to angry, by adjusting certain values. Computer-generated voices can be created and then transformed, too.

“It means you do not need as many voice actors, which is expensive, and it does not take as long to create a wide variety of characters expressing a broad range of emotions,” notes Thallinger.

Intelligent content

Work within the project even developed a system to identify emotional stress, so that the character animation can be perfectly synched with tone of voice. “It leads to a much more realistic and convincing animation when movement is synched across voice and video in this way,” Thallinger stresses.

Underlying all these efforts is a broad thrust to create intelligent content, which advertises its elements or component structure. For this task, Salero borrowed ideas from the ‘semantic web’, developing an ontology, or dictionary of specific terms, to describe audiovisual content.

It also developed archiving and retrieval tools and, particularly, a very clever annotating tool. “We knew that there is no way an animator is going to spend much time tagging content in a real work environment, there is just no time, so we used a statement-based annotating tool that quickly populates the content with relevant metadata by getting the animator to make specific statements about it.”

Excited studios

The work of the Salero project has generated a lot of interest. “In particular, studios are very excited about the audio and animating tools,” reveals Thallinger. “For the annotating work, they were less enthusiastic, mainly because I think they are not familiar with it. But towards the end of the project, audiovisual companies started expressing a lot more interest about semantic technologies, because I think the concepts around that field start to get known also in industry.”

Indeed, over time the annotating tools promise to deliver the greatest impact to the industry, allowing producers to rapidly create content from an ever-expanding library of earlier work. It could lead to stock animation galleries, in the same manner that stock photo galleries exist now.

In the meantime, Salero has permitted studios to create compelling, Hollywood-quality content quickly, efficiently and cost-effectively.

This is the first of a two-part special feature on Salero.

Explore further: Avatars make the Internet sign to deaf people

More information: Salero project - www.salero.eu/en/index.html

Related Stories

Listen, watch, read -- computers search for meaning

Oct 30, 2009

(PhysOrg.com) -- European researchers have created the first integrated semantic search platform that integrates text, video and audio. The system can 'watch' films, 'listen' to audio and 'read' text to find relevant responses ...

The search -- computers dig deeper for meaning (w/ Video)

Nov 02, 2009

(PhysOrg.com) -- Search engine technology is in a state of flux as it digs ever deeper for new meaning. Europe is poised to reap the benefits of the new age of semantic search thanks to the work of European researchers.

Video fingerprinting offers search solution

Nov 09, 2009

(PhysOrg.com) -- The explosive growth of video on the internet calls for new ways of sorting and searching audiovisual content. A team of European researchers has developed a groundbreaking solution that is ...

Avatars as communicators of emotions

Jul 09, 2008

Current interactive systems enable users to communicate with computers in many ways, but not taking into account emotional communication. A PhD thesis presented at the University of the Basque Country puts ...

Rich musical pickings with easier access to archives

Apr 22, 2009

(PhysOrg.com) -- Digital sound archives offer enormously rich resources but accessing them is currently difficult, and often arbitrary. European researchers believe they have developed a solution, one that offers compelling ...

Recommended for you

Avatars make the Internet sign to deaf people

Aug 29, 2014

It is challenging for deaf people to learn a sound-based language, since they are physically not able to hear those sounds. Hence, most of them struggle with written language as well as with text reading ...

Chameleon: Cloud computing for computer science

Aug 26, 2014

Cloud computing has changed the way we work, the way we communicate online, even the way we relax at night with a movie. But even as "the cloud" starts to cross over into popular parlance, the full potential ...

User comments : 1

Adjust slider to filter visible comments by rank

Display comments: newest first

MikeMike
4 / 5 (1) Jul 27, 2010
"maximise content reuse" = cookie cutter production
= BORING!