New technology for animation film experts: Movie heroes to be transferred to virtual worlds more easily, realistically

Feb 28, 2013
New technology for animation film experts: Movie heroes to be transferred to virtual worlds more easily, realistically
Actors in their normal clothing are filmed with ordinary cameras. The movements are then analysed with special computer software and transferred to a virtual character in the form of a skeleton. Credit: MPI for Informatics

Hollywood devotes great effort to chasing monsters through realistic-looking environments. Researchers at the Max Planck Institute for Informatics in Saarbrücken have now developed a technology that greatly simplifies the production of such scenes. Actors' movements are captured with a few cameras in a real scene and then transferred extremely realistically to virtual characters. This will not only simplify the work of cartoon makers, but also assist doctors and sportsmen with motion analysis.

The new technology will soon be marketed by a newly-established business and presented at the computer trade show CeBIT in Hanover from March 5 to March 9 in Hall 9, Booth F34.

Whenever computer-animated characters roam through wild landscapes, such a Gollum in Lord of the Rings, there were real actors at work. Film studios usually use a procedure called 'motion capture'. The actors wear skin-tight suits with markers attached to them reflecting beams of that are sent out and received by a special . In this way, the movements of a real actor are recorded and can later be transferred to a virtual character, using animation software. "However, the suits are very uncomfortable for the actors, and the markers interfere with their movements", says Nils Hasler from the Max Planck Institute for Informatics in Saarbrücken. For this reason, the Computer Graphics researchers there have developed a method that eliminates the need for markers but captures the movements quickly and realistically.

This method allows actors in their normal clothing to be filmed with ordinary cameras. The movements are then analysed with special computer software and transferred to a virtual character in the form of a skeleton. "We require only a few cameras instead of the several dozen cameras needed for the special effects in Hollywood. The movements are computed so quickly that we can transfer them directly to the without time delay," Hasler explains. The meanwhile patented computation method has been further refined in the past months. It can now deal with scenes in which several participants are simultaneously active and body parts overlap. "The system even detects a person's movements when they are covered up by other objects or when there are disturbances in the background. This will allow us to shoot visual effects outside of the studio in the future, for example, out in open nature," the Saarbrücken-based researcher reckons.

This video is not supported by your browser at this time.

The computer scientists in Christian Theobalt's "Graphics, Vision & Video" team were able to solve yet another problem in the past few months, as Hasler explains proudly: "It was difficult for our software to reconstruct the body movements of actors wearing big coats or women entering a scene in long ballroom dresses. Our new computation method enables us to capture surfaces in such precise detail that, e.g., the draping folds of clothing can be reproduced realistically." The new technology is also useful in areas outside the film and game industry. Athletes could use it to analyse specific, individual body movements without bothersome markers. Sports journalists would be able to comment on motion sequences, like in pole vault and discus competitions, in live television broadcasts.

"The field of medicine would also profit. It would be easier for doctors to depict and track the degree of recovery after operations on joints," Hasler explains. The researcher from the Max Planck Institute for Informatics wants to establish a company together with Professor Christian Theobalt and his research colleague, Carsten Stoll, in order to offer their software as a commercial product. "We have already had quite a few inquiries from companies in the film and sports marketing industries," Hasler reveals.

Technical background

The technology used in this method is quite affordable. Anywhere from five to twelve ordinary video cameras are needed. The computer scientists use their software to produce a 3-D model of the depicted person from a skeleton with 58 joints. In order to capture the movements, the computation method continuously works on overlaying the two-dimensional image from the video camera and the 3-D model as exactly as possible. The researchers can solve the necessary equations for this task efficiently and quickly. With this method, they capture filmed movements and visualize them in the virtual characters within just a few milliseconds.

Explore further: Oculus unveils new prototype VR headset

More information: Elhayek, A. et al. Spatio-temporal Motion Tracking with Unsynchronized Cameras, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, USA, 2012

Related Stories

Body-mounted cameras turn motion capture inside out

Aug 08, 2011

Traditional motion capture techniques use cameras to meticulously record the movements of actors inside studios, enabling those movements to be translated into digital models. But by turning the cameras around — mounting ...

Researchers demonstrate markerless motion capture

Aug 06, 2012

Conventional motion capture for film and game production involves multiple cameras and actors festooned with markers. A new technique developed by Disney Research, Pittsburgh, has demonstrated how three-dimensional motion ...

Scientists create program to calculate body shape

Oct 27, 2008

Imagine you are a police detective trying to identify a suspect wearing a trench coat, baggy pants and a baseball cap pulled low. Or imagine you are a fashion industry executive who wants to market virtual ...

Cebit 2012: 3-D animations for everyone

Mar 06, 2012

3D movies like "Toy Story" or "Transformers" are based on everyday objects that are able to move like humans. Such 3D characters are created by skilled experts in time-consuming manual work. Computer scientists ...

Recommended for you

Oculus unveils new prototype VR headset

Sep 20, 2014

Oculus has unveiled a new prototype of its virtual reality headset. However, the VR company still isn't ready to release a consumer edition.

Who drives Alibaba's Taobao traffic—buyers or sellers?

Sep 18, 2014

As Chinese e-commerce firm Alibaba prepares for what could be the biggest IPO in history, University of Michigan professor Puneet Manchanda dug into its Taobao website data to help solve a lingering chicken-and-egg question.

Computerized emotion detector

Sep 16, 2014

Face recognition software measures various parameters in a mug shot, such as the distance between the person's eyes, the height from lip to top of their nose and various other metrics and then compares it with photos of people ...

User comments : 2

Adjust slider to filter visible comments by rank

Display comments: newest first

williamansley
not rated yet Feb 28, 2013
It is quite unfortunate the the misconception that all computer-animated characters are created with motion capture from the movements of "real actors" in being repeated in this article. In fact, many computer-animated characters' movement is created solely by talented animators (who may refer to live action footage as reference, but not to copy frame by frame). Animators are also needed to clean up and tweak the results of motion capture in most cases, or the results would not look very good.
TracyMc
not rated yet Mar 01, 2013
At Phasespace, we make both the 3,600 x 3,600 resolution at 960 frames per second Motion Capture and Four Megapixel 100 FPS Video cameras used by Max Planck for this research. It allows researchers and animators to capture lifelike motions very quickly, and depending on the application, with better fidelity and resolution with the LED tracking technology, or more ease of use and freedom from wearing markers or sensors with the Camera Technology. Each has their place. In many applications in entertainment, it is impossible to get a dragon to suit up, or video capture, so the animation must be done by hand or with a procedural program. Many times the artist wants to use squash and stretch techniques that would be impossible with human actors. Each technique has it's place and the focus should be the story, not the technology. We are making an animated movie you can see on kickstarter, Tower of the dragon, that uses all three techniques.