Researchers dress virtual avatars with digitally captured clothing

August 2, 2017, Max Planck Institute for Intelligent Systems Tübingen
Figure 1: ClothCap enables automatic transfer of 3D clothing to new bodies. Captured subject on the left. Synthetic animation on the right for a new body. Credit: MPI-IS, Perceiving Systems Department

Researchers at the Max Planck Institute for Intelligent Systems (MPI-IS) have developed technology to digitally capture clothing on moving people, turn it into a 3-D digital form, and dress virtual avatars with it. This new technology makes virtual clothing try-on practical.

Ever wonder how you might look in a beautiful dress worn by Michelle Obama? Imagine trying it on and seeing how the fabric moves and fits before buying. This technology could make it possible.

Traditional virtual try-on involves getting the 2-D clothing pattern from the manufacturer, sizing it to an individual, and simulating how the clothing drapes on the . The new technique replaces garment simulation with garment capture. Capturing and transferring existing garments greatly simplifies the process of virtual try-on.

"Our approach is to scan a person wearing the garment, separate the clothing from the person, and then rendering it on top of a new person," says Dr. Gerard Pons-Moll, research scientist at MPI-IS and principal investigator of the project. "This process captures all the detail present in real clothing, including how it moves, which is hard to replicate with simulation," says Pons-Moll. The ClothCap (ClothCapture) results exceed the realism of existing approaches (Figure 1).

ClothCap uses 4-D movies of people recorded with a unique 4-D high-resolution scanner. The system uses 66 cameras and projectors to illuminate the person being scanned. "This scanner captures every wrinkle of clothing at high resolution. It is like having 66 eyes looking at a person from every possible angle," says Michael Black, director at MPI-IS. "This allows us to study humans in like never before."

ClothCap computes the body shape and motion under clothing while separating and tracking the garments on the body as it moves. "The software turns the captured scans into separate meshes corresponding to the clothing and the body," says Dr. Sergi Pujades, postdoctoral researcher at MPI and one of the main authors of this work.

Credit: Michael Black et. al.

Traditional marker-based records only skeletal motion; placing hundreds of those markers on the clothing is unpractical and doesn't lend to mapping the captured clothing to new figures. ClothCap is simpler because the clothing is captured in correspondence with the body. "The algorithm literally subtracts the clothing from the recorded subject and adds it to new body to produce a realistic result," says Gerard Pons-Moll. "It's like doing arithmetic with people and their clothing—it's cool!"

"Motion capture revolutionized the fields of animation, gaming and biomechanics. Much in the same way that motion capture replaced a lot of manual editing of motion from scratch, techniques like ClothCap could replace the way clothed characters are edited and animated digitally," explains Pons-Moll. This is the first method to capture clothing in motion and animate new characters with it (Figure 2).

Figure 2: Dressing crowds. With ClothCap it is possible to scan a person in clothing once and then visualize such clothing on different bodies. Credit: MPI-IS, Perceiving Systems Department
"Other methods capture people in clothing, but nobody could realistically animate new subjects with the captured clothing," says Sergi Pujades.

ClothCap provides a foundational technology for virtual clothing try-on. "First, a retailer needs to scan a professional model in a variety of poses and clothing to create a digital wardrobe of clothing items. Then a user can select an item and visualize how it looks on their virtual avatar," says Michael Black.

The work has several limitations. For example, cloth wrinkles do not change with body shape and ClothCap does not allow the synthesis of novel motions. The team plans to address such limitations in future work.

Explore further: Tiny motions bring digital doubles to life (w/ Video)

More information: ClothCap: seamless 4D clothing capture and retargeting, ACM Transactions on Graphics (TOG) TOG Homepage archive, Volume 36, Issue 4, July 2017, Article No. 73 , DOI: 10.1145/3072959.3073711 ,


Related Stories

Tiny motions bring digital doubles to life (w/ Video)

December 7, 2014

Researchers at the Max Planck Institute for Intelligent Systems unveil new technology for motion and shape capture (MoSh) that helps animators jump the "Uncanny Valley" by turning a few moving dots into detailed body shapes ...

Scientists create program to calculate body shape

October 27, 2008

Imagine you are a police detective trying to identify a suspect wearing a trench coat, baggy pants and a baseball cap pulled low. Or imagine you are a fashion industry executive who wants to market virtual clothing that customers ...

Recommended for you

Nanoscale Lamb wave-driven motors in nonliquid environments

March 19, 2019

Light driven movement is challenging in nonliquid environments as micro-sized objects can experience strong dry adhesion to contact surfaces and resist movement. In a recent study, Jinsheng Lu and co-workers at the College ...

OSIRIS-REx reveals asteroid Bennu has big surprises

March 19, 2019

A NASA spacecraft that will return a sample of a near-Earth asteroid named Bennu to Earth in 2023 made the first-ever close-up observations of particle plumes erupting from an asteroid's surface. Bennu also revealed itself ...

The powerful meteor that no one saw (except satellites)

March 19, 2019

At precisely 11:48 am on December 18, 2018, a large space rock heading straight for Earth at a speed of 19 miles per second exploded into a vast ball of fire as it entered the atmosphere, 15.9 miles above the Bering Sea.


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.