Disney Research transforms movie-quality animations for interactive viewing

Cinema-quality animations and virtual reality graphics that need to be rendered in real-time are often mutually exclusive categories, but Disney Research has developed a new process that transforms high-resolution animated content into a novel video format to enable immersive viewing.

The end-to-end solution the researchers devised will make it easier to repurpose animated film assets for use in video games and location-based VR venues. Viewers wearing head-mounted displays can interact with movie animations in a new way, based on the position and orientation of their heads.

"This new solution promises huge savings on the most costly aspects of interactive media production," said Professor Kenny Mitchell, senior research scientist.

The researchers will present their real-time rendering method May 16 at the Graphics Interface 2017 conference in Edmonton, Alberta.

"We've seen a resurgence in interest in virtual reality in recent years as companies have released a number of head-mounted displays for consumers," said Professor Markus Gross, vice president at Disney Research. "The subsequent demand for VR and other immersive content is driving innovations such as this ground-breaking set of methods for reusing rendered animated films."

Virtual reality scenes must be rendered in real-time and that performance requirement means using animations that are less complex than the highly detailed animations typical of feature films. That means when artists produce an interactive experience tied-in to the film or a related video game, they have to convert the film animations into a lower-quality form compatible with real-time rendering or game engines. That process is both laborious and expensive.

Mitchell and his colleagues opt instead for an approach that relies on automated pre-processing: The 3D scenes are rendered from the perspective of a number of camera positions calculated to provide the best viewpoints for all of the surfaces in the scene with as few cameras as possible and encode it all in a modular video format. This content can then be rendered in real-time from an arbitrary point of view, allowing for motion parallax, head tilting and rotations.

"This process enables consumption of immersive pre-rendered video in six degrees of freedom using a head-mounted display," said Babis Koniaris, a post-doctoral associate on the team. "It can also be used for rendering film-quality visuals for video games."

In addition to Mitchell and Koniaris, the research team included Maggie Kosek and David Sinclair. The research was partly supported by the Innovate UK project #102684, titled OSCIR.

Combining creativity and innovation, this research continues Disney's rich legacy of leveraging technology to enhance the tools and systems of tomorrow.