Researchers demonstrate markerless motion capture

August 6, 2012

Conventional motion capture for film and game production involves multiple cameras and actors festooned with markers. A new technique developed by Disney Research, Pittsburgh, has demonstrated how three-dimensional motion capture can be accomplished with a single camera and without aid of markers.

The technique, developed in with Brown University, not only captures the 3D poses of , as is done with traditional motion capture systems, but derives "biped controllers" — programs that incorporate the underlying physics of the motion. Bipedal controllers generate the poses by computing the forces acting on the body and integrating them over time.

Leonid Sigal, research scientist at Disney Research, Pittsburgh, said this understanding of the process that generates those poses will enable computer graphic artists to not only reproduce those motions in animations, but to change the environments in which the action takes place. The motion captured from an actor walking across a room, for instance, could be applied to an animated character walking down a slope, crossing a slippery surface or carrying a heavy object.

Though Sigal emphasized that the technique requires more refinement, recovering 3D motion capture from a single camera opens up a number of new possibilities for motion capture, including the ability to capture motions or expressions from old movies or existing video. It could aid in capturing motion from outdoor scenes or types of motion that might be difficult to achieve within a traditional motion capture studio. It would be particularly suitable for use in developing games and other applications that involve interaction between an animated character and a player or a changing environment, he added.

Outside of the realm of filmmaking and game creation, coaches might use video of athletes recorded during practice or during competition to analyze body mechanics.

In addition to Sigal, the research team included Jessica Hodgins, director of Disney Research, Pittsburgh; Marek Vondrak, a Disney Research intern and a PhD student in computer science at Brown, and Odest Jenkins, associate professor of computer science at Brown. The researchers will present their findings Aug. 6 at SIGGRAPH 2012, the International Conference on Computer Graphics and Interactive Techniques, at the Los Angeles Convention Center.

Solving the problem of markerless motion capture and bipedal controller estimation simultaneously is critical, said Vondrak, whose PhD thesis focuses on the role of physics-based models in video-based motion capture. When recovering poses, the estimated biped controllers help to determine which of the possible sequences of 3D poses would reasonably fit the video images. On the other hand, recovered controllers allow direct application and natural adaptation of the motions to characters in new and dynamically changing environments.

"We didn't want to break these problems up, but to do them simultaneously," Sigal said. "Given the current technology, I don't think you can do them separately."

The technique is computationally intensive and does not yet match the quality possible with traditional motion capture, Sigal said. But given the difficulty of markerless motion capture with a single , the preliminary results are very promising, he added. Moreover, many real-world applications of markerless likely will employ multiple cameras, which reduces the technical challenge. The Disney researchers also attempted to make the process totally automated; adding a human to the loop for tasks such as identifying joint locations could also improve results.

Explore further: Body-mounted cameras turn motion capture inside out

Related Stories

Body-mounted cameras turn motion capture inside out

August 8, 2011

Traditional motion capture techniques use cameras to meticulously record the movements of actors inside studios, enabling those movements to be translated into digital models. But by turning the cameras around — mounting ...

Ski Faster with Camera-less Fusion Motion Capture

July 1, 2008

Professional skiers can now learn how to ski faster with the aid of a new system used to capture 3D motion of athletic movements – Fusion Motion Capture (FMC). Featured in Wiley-Blackwell’s journal, Sports Technology, ...

A new window to the face

August 9, 2011

The human face is a complicated thing—powered by 52 muscles; contoured by the nose, eyebrows, and other features; and capable of an almost infinite range of expressions, from joy to anger to sorrow to puzzlement.

Taking movies beyond Avatar -- for under $150 (w/ Video)

November 12, 2010

A new development in virtual cameras at the University of Abertay Dundee, UK, is developing the pioneering work of James Cameron's blockbuster Avatar using a Nintendo Wii-like motion controller – all for less than $150.

Recommended for you

Swiss unveil stratospheric solar plane

December 7, 2016

Just months after two Swiss pilots completed a historic round-the-world trip in a Sun-powered plane, another Swiss adventurer on Wednesday unveiled a solar plane aimed at reaching the stratosphere.

Solar panels repay their energy 'debt': study

December 6, 2016

The climate-friendly electricity generated by solar panels in the past 40 years has all but cancelled out the polluting energy used to produce them, a study said Tuesday.


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.