Green-screen keying method cuts time, boosts quality in film compositing

November 14, 2016, Disney Research
Credit: Disney Research

Filming an actor in front of a green screen and then superimposing the actor over another background is commonplace in feature film production, but getting rid of all traces of the green screen remains a chore. A new "keying" method devised by Disney Research and ETH Zurich, however, improves the results substantially.

What's more, these superior results can be achieved in about one-tenth the time it takes a professional compositor to perform this task manually.

"Keying is a crucial intermediate step in adding synthetic backgrounds and objects seamlessly into live-action shots," said Markus Gross, vice president at Disney Research. "It's also a major bottleneck in postproduction, so a keying method that is both fast and produces high-quality results will have a major impact on the production of feature films."

A report on the method, which is based on a novel algorithm that computes underlying and their mixing ratios, will be presented at the Association for Computing Machinery's (ACM) Conference on Computer Graphics and Interactive Techniques (SIGGRAPH 2017) next summer in Los Angeles and has been published in the August issue of ACM Transactions on Graphics journal (ACM TOG).

A number of commercial software tools are available to help a compositing artist perform keying, said Tunç Aydin, research scientist at Disney Research. But those tools have difficulty discerning the green screen background from the foreground in regions of the image where the colors of objects mix with each other, he noted. This mixing can occur because of motion blur, intricate object boundaries, such as hair, or colors cast onto an object from the green screen itself. Whatever the cause, these imperfections usually force the artist to painstakingly paint those regions manually.

The research team opted to create a color unmixing algorithm, which can be used to extract multiple color layers using a color model of the image, said Yagiz Aksoy, a Ph.D. student at Disney Research and ETH Zurich. A subset of these colors is then combined to produce the final keying result.

The team created a user interface that enables the artist to create a color model of the image simply by scribbling over each distinct-colored area of the image. The shape of the scribble doesn't matter; the program simply needs to identify each area of color so it can create a representation of all the colors in the scene. The algorithm computes the underlying colors and their mixing ratios for each area.

The researchers showed that the method would be applicable for keying scenes with non-green-screen backgrounds. Also, the ability to represent the image with multiple color layers, rather than just foreground and background layers as in conventional methods, creates opportunities for new applications, Aydin said. Colors of objects in a scene, for instance, could be changed.

In comprehensive comparisons with commercial keying tools and with other published methods, the researchers showed that the results for their method were superior in quality as it required only one-tenth of the manual editing time it would take a professional artist to process the same content.

Combining creativity and innovation, this research continues Disney's rich legacy of inventing new ways to tell great stories and leveraging technology required to build the future of entertainment.

Explore further: Researchers develop fast, economical method for high-definition video compositing

More information: "Interactive High-Quality Green-Screen Keying via Color Unmixing-Paper" [PDF, 1.97 MB]

Related Stories

New device improves full-color image projection

October 21, 2015

A team of researchers at Disney Research and Dartmouth College has developed a new way to display full-color images using only two black patterns printed on transparencies affixed to two sides of a prism.

Recommended for you

Fish-inspired material changes color using nanocolumns

March 20, 2019

Inspired by the flashing colors of the neon tetra fish, researchers have developed a technique for changing the color of a material by manipulating the orientation of nanostructured columns in the material.

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.