Green-screen keying method cuts time, boosts quality in film compositing

November 14, 2016, Disney Research
Credit: Disney Research

Filming an actor in front of a green screen and then superimposing the actor over another background is commonplace in feature film production, but getting rid of all traces of the green screen remains a chore. A new "keying" method devised by Disney Research and ETH Zurich, however, improves the results substantially.

What's more, these superior results can be achieved in about one-tenth the time it takes a professional compositor to perform this task manually.

"Keying is a crucial intermediate step in adding synthetic backgrounds and objects seamlessly into live-action shots," said Markus Gross, vice president at Disney Research. "It's also a major bottleneck in postproduction, so a keying method that is both fast and produces high-quality results will have a major impact on the production of feature films."

A report on the method, which is based on a novel algorithm that computes underlying and their mixing ratios, will be presented at the Association for Computing Machinery's (ACM) Conference on Computer Graphics and Interactive Techniques (SIGGRAPH 2017) next summer in Los Angeles and has been published in the August issue of ACM Transactions on Graphics journal (ACM TOG).

A number of commercial software tools are available to help a compositing artist perform keying, said Tunç Aydin, research scientist at Disney Research. But those tools have difficulty discerning the green screen background from the foreground in regions of the image where the colors of objects mix with each other, he noted. This mixing can occur because of motion blur, intricate object boundaries, such as hair, or colors cast onto an object from the green screen itself. Whatever the cause, these imperfections usually force the artist to painstakingly paint those regions manually.

The research team opted to create a color unmixing algorithm, which can be used to extract multiple color layers using a color model of the image, said Yagiz Aksoy, a Ph.D. student at Disney Research and ETH Zurich. A subset of these colors is then combined to produce the final keying result.

The team created a user interface that enables the artist to create a color model of the image simply by scribbling over each distinct-colored area of the image. The shape of the scribble doesn't matter; the program simply needs to identify each area of color so it can create a representation of all the colors in the scene. The algorithm computes the underlying colors and their mixing ratios for each area.

The researchers showed that the method would be applicable for keying scenes with non-green-screen backgrounds. Also, the ability to represent the image with multiple color layers, rather than just foreground and background layers as in conventional methods, creates opportunities for new applications, Aydin said. Colors of objects in a scene, for instance, could be changed.

In comprehensive comparisons with commercial keying tools and with other published methods, the researchers showed that the results for their method were superior in quality as it required only one-tenth of the manual editing time it would take a professional artist to process the same content.

Combining creativity and innovation, this research continues Disney's rich legacy of inventing new ways to tell great stories and leveraging technology required to build the future of entertainment.

Explore further: Researchers develop fast, economical method for high-definition video compositing

More information: "Interactive High-Quality Green-Screen Keying via Color Unmixing-Paper" [PDF, 1.97 MB]

Related Stories

New device improves full-color image projection

October 21, 2015

A team of researchers at Disney Research and Dartmouth College has developed a new way to display full-color images using only two black patterns printed on transparencies affixed to two sides of a prism.

Recommended for you

Technology near for real-time TV political fact checks

January 18, 2019

A Duke University team expects to have a product available for election year that will allow television networks to offer real-time fact checks onscreen when a politician makes a questionable claim during a speech or debate.

Privacy becomes a selling point at tech show

January 7, 2019

Apple is not among the exhibitors at the 2019 Consumer Electronics Show, but that didn't prevent the iPhone maker from sending a message to attendees on a large billboard.

China's Huawei unveils chip for global big data market

January 7, 2019

Huawei Technologies Ltd. showed off a new processor chip for data centers and cloud computing Monday, expanding into new and growing markets despite Western warnings the company might be a security risk.


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.