New technology for animation film experts: Movie heroes to be transferred to virtual worlds more easily, realistically

New technology for animation film experts: Movie heroes to be transferred to virtual worlds more easily, realistically
Actors in their normal clothing are filmed with ordinary cameras. The movements are then analysed with special computer software and transferred to a virtual character in the form of a skeleton. Credit: MPI for Informatics

Hollywood devotes great effort to chasing monsters through realistic-looking environments. Researchers at the Max Planck Institute for Informatics in Saarbrücken have now developed a technology that greatly simplifies the production of such scenes. Actors' movements are captured with a few cameras in a real scene and then transferred extremely realistically to virtual characters. This will not only simplify the work of cartoon makers, but also assist doctors and sportsmen with motion analysis.

The new technology will soon be marketed by a newly-established business and presented at the computer trade show CeBIT in Hanover from March 5 to March 9 in Hall 9, Booth F34.

Whenever computer-animated characters roam through wild landscapes, such a Gollum in Lord of the Rings, there were real actors at work. Film studios usually use a procedure called 'motion capture'. The actors wear skin-tight suits with markers attached to them reflecting beams of that are sent out and received by a special . In this way, the movements of a real actor are recorded and can later be transferred to a virtual character, using animation software. "However, the suits are very uncomfortable for the actors, and the markers interfere with their movements", says Nils Hasler from the Max Planck Institute for Informatics in Saarbrücken. For this reason, the Computer Graphics researchers there have developed a method that eliminates the need for markers but captures the movements quickly and realistically.

This method allows actors in their normal clothing to be filmed with ordinary cameras. The movements are then analysed with special computer software and transferred to a virtual character in the form of a skeleton. "We require only a few cameras instead of the several dozen cameras needed for the special effects in Hollywood. The movements are computed so quickly that we can transfer them directly to the without time delay," Hasler explains. The meanwhile patented computation method has been further refined in the past months. It can now deal with scenes in which several participants are simultaneously active and body parts overlap. "The system even detects a person's movements when they are covered up by other objects or when there are disturbances in the background. This will allow us to shoot visual effects outside of the studio in the future, for example, out in open nature," the Saarbrücken-based researcher reckons.

The computer scientists in Christian Theobalt's "Graphics, Vision & Video" team were able to solve yet another problem in the past few months, as Hasler explains proudly: "It was difficult for our software to reconstruct the body movements of actors wearing big coats or women entering a scene in long ballroom dresses. Our new computation method enables us to capture surfaces in such precise detail that, e.g., the draping folds of clothing can be reproduced realistically." The new technology is also useful in areas outside the film and game industry. Athletes could use it to analyse specific, individual body movements without bothersome markers. Sports journalists would be able to comment on motion sequences, like in pole vault and discus competitions, in live television broadcasts.

"The field of medicine would also profit. It would be easier for doctors to depict and track the degree of recovery after operations on joints," Hasler explains. The researcher from the Max Planck Institute for Informatics wants to establish a company together with Professor Christian Theobalt and his research colleague, Carsten Stoll, in order to offer their software as a commercial product. "We have already had quite a few inquiries from companies in the film and sports marketing industries," Hasler reveals.

Technical background

The technology used in this method is quite affordable. Anywhere from five to twelve ordinary video cameras are needed. The computer scientists use their software to produce a 3-D model of the depicted person from a skeleton with 58 joints. In order to capture the movements, the computation method continuously works on overlaying the two-dimensional image from the video camera and the 3-D model as exactly as possible. The researchers can solve the necessary equations for this task efficiently and quickly. With this method, they capture filmed movements and visualize them in the virtual characters within just a few milliseconds.

Explore further

Capturing movements of actors and athletes in real time with conventional video cameras

More information: Elhayek, A. et al. Spatio-temporal Motion Tracking with Unsynchronized Cameras, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, USA, 2012
Provided by Max Planck Society
Citation: New technology for animation film experts: Movie heroes to be transferred to virtual worlds more easily, realistically (2013, February 28) retrieved 22 August 2019 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Feedback to editors

User comments

Feb 28, 2013
It is quite unfortunate the the misconception that all computer-animated characters are created with motion capture from the movements of "real actors" in being repeated in this article. In fact, many computer-animated characters' movement is created solely by talented animators (who may refer to live action footage as reference, but not to copy frame by frame). Animators are also needed to clean up and tweak the results of motion capture in most cases, or the results would not look very good.

Mar 01, 2013
At Phasespace, we make both the 3,600 x 3,600 resolution at 960 frames per second Motion Capture and Four Megapixel 100 FPS Video cameras used by Max Planck for this research. It allows researchers and animators to capture lifelike motions very quickly, and depending on the application, with better fidelity and resolution with the LED tracking technology, or more ease of use and freedom from wearing markers or sensors with the Camera Technology. Each has their place. In many applications in entertainment, it is impossible to get a dragon to suit up, or video capture, so the animation must be done by hand or with a procedural program. Many times the artist wants to use squash and stretch techniques that would be impossible with human actors. Each technique has it's place and the focus should be the story, not the technology. We are making an animated movie you can see on kickstarter, Tower of the dragon, that uses all three techniques.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more