New technology for animation film experts: Movie heroes to be transferred to virtual worlds more easily, realistically

February 28, 2013
New technology for animation film experts: Movie heroes to be transferred to virtual worlds more easily, realistically
Actors in their normal clothing are filmed with ordinary cameras. The movements are then analysed with special computer software and transferred to a virtual character in the form of a skeleton. Credit: MPI for Informatics

Hollywood devotes great effort to chasing monsters through realistic-looking environments. Researchers at the Max Planck Institute for Informatics in Saarbrücken have now developed a technology that greatly simplifies the production of such scenes. Actors' movements are captured with a few cameras in a real scene and then transferred extremely realistically to virtual characters. This will not only simplify the work of cartoon makers, but also assist doctors and sportsmen with motion analysis.

The new technology will soon be marketed by a newly-established business and presented at the computer trade show CeBIT in Hanover from March 5 to March 9 in Hall 9, Booth F34.

Whenever computer-animated characters roam through wild landscapes, such a Gollum in Lord of the Rings, there were real actors at work. Film studios usually use a procedure called 'motion capture'. The actors wear skin-tight suits with markers attached to them reflecting beams of that are sent out and received by a special . In this way, the movements of a real actor are recorded and can later be transferred to a virtual character, using animation software. "However, the suits are very uncomfortable for the actors, and the markers interfere with their movements", says Nils Hasler from the Max Planck Institute for Informatics in Saarbrücken. For this reason, the Computer Graphics researchers there have developed a method that eliminates the need for markers but captures the movements quickly and realistically.

This method allows actors in their normal clothing to be filmed with ordinary cameras. The movements are then analysed with special computer software and transferred to a virtual character in the form of a skeleton. "We require only a few cameras instead of the several dozen cameras needed for the special effects in Hollywood. The movements are computed so quickly that we can transfer them directly to the without time delay," Hasler explains. The meanwhile patented computation method has been further refined in the past months. It can now deal with scenes in which several participants are simultaneously active and body parts overlap. "The system even detects a person's movements when they are covered up by other objects or when there are disturbances in the background. This will allow us to shoot visual effects outside of the studio in the future, for example, out in open nature," the Saarbrücken-based researcher reckons.

The video will load shortly.

The computer scientists in Christian Theobalt's "Graphics, Vision & Video" team were able to solve yet another problem in the past few months, as Hasler explains proudly: "It was difficult for our software to reconstruct the body movements of actors wearing big coats or women entering a scene in long ballroom dresses. Our new computation method enables us to capture surfaces in such precise detail that, e.g., the draping folds of clothing can be reproduced realistically." The new technology is also useful in areas outside the film and game industry. Athletes could use it to analyse specific, individual body movements without bothersome markers. Sports journalists would be able to comment on motion sequences, like in pole vault and discus competitions, in live television broadcasts.

"The field of medicine would also profit. It would be easier for doctors to depict and track the degree of recovery after operations on joints," Hasler explains. The researcher from the Max Planck Institute for Informatics wants to establish a company together with Professor Christian Theobalt and his research colleague, Carsten Stoll, in order to offer their software as a commercial product. "We have already had quite a few inquiries from companies in the film and sports marketing industries," Hasler reveals.

Technical background

The technology used in this method is quite affordable. Anywhere from five to twelve ordinary video cameras are needed. The computer scientists use their software to produce a 3-D model of the depicted person from a skeleton with 58 joints. In order to capture the movements, the computation method continuously works on overlaying the two-dimensional image from the video camera and the 3-D model as exactly as possible. The researchers can solve the necessary equations for this task efficiently and quickly. With this method, they capture filmed movements and visualize them in the virtual characters within just a few milliseconds.

Explore further: Capturing movements of actors and athletes in real time with conventional video cameras

More information: Elhayek, A. et al. Spatio-temporal Motion Tracking with Unsynchronized Cameras, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, USA, 2012

Related Stories

Body-mounted cameras turn motion capture inside out

August 8, 2011

Traditional motion capture techniques use cameras to meticulously record the movements of actors inside studios, enabling those movements to be translated into digital models. But by turning the cameras around — mounting ...

Researchers demonstrate markerless motion capture

August 6, 2012

Conventional motion capture for film and game production involves multiple cameras and actors festooned with markers. A new technique developed by Disney Research, Pittsburgh, has demonstrated how three-dimensional motion ...

Scientists create program to calculate body shape

October 27, 2008

Imagine you are a police detective trying to identify a suspect wearing a trench coat, baggy pants and a baseball cap pulled low. Or imagine you are a fashion industry executive who wants to market virtual clothing that customers ...

Cebit 2012: 3-D animations for everyone

March 6, 2012

3D movies like "Toy Story" or "Transformers" are based on everyday objects that are able to move like humans. Such 3D characters are created by skilled experts in time-consuming manual work. Computer scientists at the Max ...

Recommended for you

Newest solar cells underperform in cloudy countries

August 22, 2017

To determine how efficient new solar cells convert sunlight into electricity, small sample cells are tested under ideal conditions. However, the reported efficiency is not very representative of the actual annual yield when ...

Google to serve next version of Android as 'Oreo"

August 22, 2017

An upcoming update to Google's Android software finally has a delectable name. The next version will be known as Oreo, extending Google's tradition of naming each version after a sweet treat.

Forget oil, Russia goes crazy for cryptocurrency

August 16, 2017

Standing in a warehouse in a Moscow suburb, Dmitry Marinichev tries to speak over the deafening hum of hundreds of computers stacked on shelves hard at work mining for crypto money.

Researchers clarify mystery about proposed battery material

August 15, 2017

Battery researchers agree that one of the most promising possibilities for future battery technology is the lithium-air (or lithium-oxygen) battery, which could provide three times as much power for a given weight as today's ...


Adjust slider to filter visible comments by rank

Display comments: newest first

not rated yet Feb 28, 2013
It is quite unfortunate the the misconception that all computer-animated characters are created with motion capture from the movements of "real actors" in being repeated in this article. In fact, many computer-animated characters' movement is created solely by talented animators (who may refer to live action footage as reference, but not to copy frame by frame). Animators are also needed to clean up and tweak the results of motion capture in most cases, or the results would not look very good.
not rated yet Mar 01, 2013
At Phasespace, we make both the 3,600 x 3,600 resolution at 960 frames per second Motion Capture and Four Megapixel 100 FPS Video cameras used by Max Planck for this research. It allows researchers and animators to capture lifelike motions very quickly, and depending on the application, with better fidelity and resolution with the LED tracking technology, or more ease of use and freedom from wearing markers or sensors with the Camera Technology. Each has their place. In many applications in entertainment, it is impossible to get a dragon to suit up, or video capture, so the animation must be done by hand or with a procedural program. Many times the artist wants to use squash and stretch techniques that would be impossible with human actors. Each technique has it's place and the focus should be the story, not the technology. We are making an animated movie you can see on kickstarter, Tower of the dragon, that uses all three techniques.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.