Intelligent animation—engineers collaborate to incorporate AI into a computer-based rendering system

July 26, 2017 by James Badham, University of California - Santa Barbara
A before-and-after image of the denoising capability of the AI neural network model Credit: Steve Bako

Modern films and TV shows are filled with spectacular computer-generated sequences computed by rendering systems that simulate the flow of light in a three-dimensional scene and convert the information into a two-dimensional image. But computing the thousands of light rays (per frame) to achieve accurate color, shadows, reflectivity and other light-based characteristics is a labor-intensive, time-consuming and expensive undertaking. An alternative is to render the images using only a few light rays. That saves time and labor but results in inaccuracies that show up as objectionable "noise" in the final image.

UC Santa Barbara electrical and computer engineering Ph.D. student Steve Bako and his advisor, Pradeep Sen, are advancing on a solution. Over the past couple of years, the two have worked with researchers at Disney Research and Pixar Animation Studios to develop a new technology based on artificial intelligence (AI) and deep learning to eliminate that and enable production-quality rendering at much higher speeds. Bako will present the findings at the SIGGRAPH conference, the premier venue for technical research in computer graphics, to be held July 31-August 3 in Los Angeles.

Bako spent a year working at Pixar. The team tested the software by using millions of examples from the film "Finding Dory" to train a deep-learning model known as a convolutional neural network. Through this process, the system learned to transform noisy images into noise-free versions that resemble those computed with significantly more rays. Once trained, the system successfully removed the noise on test images from entirely different films, such as Pixar's latest release, "Cars 3," and their upcoming feature "Coco," even though they had completely disparate styles and color palettes.

"Noise is a really big problem for production rendering," said Tony DeRose, head of research at Pixar. "This new technology allows us to automatically remove the noise while preserving the detail in our scenes."

The work presents a significant step forward over previous state-of-the-art denoising methods, which often left artifacts or residual noise that required artists to either render more light rays or to tweak the denoising filter to improve the quality of a specific image. Disney and Pixar plan to incorporate the technology in their production pipelines to accelerate the movie-making process.

"Other approaches for removing image noise have grown increasingly complex, with diminishing returns," said Markus Gross, vice president for research at Disney Research. "By leveraging deep learning, this work presents an important step forward for removing undesirable artifacts from animated films."

To facilitate further exploration of this exciting area, the team will make their code and the deep learning model's trained weights available to the research community.

Explore further: Team accelerates rendering with AI

Related Stories

Team accelerates rendering with AI

June 30, 2017

Modern films and TV shows are filled with spectacular, computer-generated sequences which are computed by rendering systems that simulate the flow of light in a 3D scene. However, computing many light rays is an immensely ...

Animation pioneer Pixar marks 25 years

July 2, 2011

Pixar, a pioneer of computer animation that has made a dozen profitable feature films and become one of the most successful studios on the planet, is celebrating its 25th birthday.

Recommended for you

Researchers find tweeting in cities lower than expected

February 20, 2018

Studying data from Twitter, University of Illinois researchers found that less people tweet per capita from larger cities than in smaller ones, indicating an unexpected trend that has implications in understanding urban pace ...

Augmented reality takes 3-D printing to next level

February 20, 2018

Cornell researchers are taking 3-D printing and 3-D modeling to a new level by using augmented reality (AR) to allow designers to design in physical space while a robotic arm rapidly prints the work.

What do you get when you cross an airplane with a submarine?

February 15, 2018

Researchers from North Carolina State University have developed the first unmanned, fixed-wing aircraft that is capable of traveling both through the air and under the water – transitioning repeatedly between sky and sea. ...

1 comment

Adjust slider to filter visible comments by rank

Display comments: newest first

EmceeSquared
not rated yet Jul 26, 2017
The noisy image looks more real.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.