Body Talk: A new crowdshaping technology uses words to create accurate 3-D body models

Researchers from the Max Planck Institute for Intelligent Systems and the University of Texas at Dallas, revealed new crowdshaping technology at SIGGRAPH 2016 that creates accurate 3-D body models from 2D photos using crowdsourced linguistic descriptions of body shape. The Body Talk system takes a single photo and produces 3-D body shapes that look like the person and are accurate enough to size clothing. It does this using the help of 15 volunteers who rate the body shape in the photo using 30 words or fewer. The researchers believe this technology has applications in online shopping, gaming, virtual reality and healthcare.

"High-end scanners and their lower-cost alternatives have yet to become ubiquitous in consumer technology," said Dr. Michael J. Black, Director of the Perceiving Systems Department at the Max Planck Institute for Intelligent Systems. "Capturing 3-D body shape remains a challenge, and our goal is to make it easy and fun. Our research shows that people have a shared understanding of body shape that is expressed in our use of language".

The system uses machine learning to discover this relationship between our verbal descriptions of bodies and their actual 3-D shape. "Language provides rich descriptions of body features related to gender, physical strength, attractiveness, and health" said Dr. Alice J. O'Toole, Professor in the School of Behavioral and Brain Sciences at the University of Texas at Dallas.

"What is surprising is that language captures 3-D shape so accurately and that the collective judgment of the "crowd" can be used to create realistic avatars. This process of crowdshaping opens up a practical way to scan bodies without a scanner" said Dr. Stephan Streuber, the lead author of the study. In the paper, Body talk: crowdshaping realistic 3-D avatars with words, the researchers found that realistic 3-D bodies could be created using as few as ten words. They demonstrated a "3-D paparazzi" application that takes a single photo of a celebrity, has it rated by 15 people, and then creates a 3-D avatar of the celebrity that can be animated. They further showed that they can extract measurements from the crowdshaped bodies with an accuracy sufficient for many clothing sizing applications. They even create 3-D avatars of characters from books using just the written description.

"When scanning technology is unavailable or inapplicable, Body Talk can be used to visualize mental representations of human body shape," said Dr. Black. "This system can be used in many areas of science and medicine, for example in studies of body perception disorders, obesity, or cross-cultural attitudes about body shape."

Learn more about the Body Talk system and try it yourself.