Creating an avatar from a 3-D selfie

August 5, 2015, Ecole Polytechnique Federale de Lausanne
Credit: EPFL

Generating a 3D duplicate of someone without the aid of a Hollywood studio: this is the challenge taken up by EPFL researchers, who have successfully condensed an expensive and complex process to use only a smartphone camera.

"We wanted the process to be fast and easy: all you have to do is take a video of yourself and then snap a few more shots to get , and our algorithm does the rest," said researcher Alexandru Ichim from the Computer Graphics and Geometry Laboratory at EPFL. The user's digital double can be displayed on a screen and animated in real time if so desired with a that follows the user's movements. "The goal was to make the process accessible to anyone with a , even an old model, as long as it can take video," said the researcher.

The program's designers see a number of possible uses in the near future: gaming, virtual reality, online discussions with other avatars, embedding in films, video conferences, and even avatar therapy for people suffering from schizophrenia – something that's already done at University College London.

Smiles are unique

Creating an avatar is simple for the user, but the underlying algorithms are not. Using a smartphone to replace studio conditions – which include proper lighting and numerous cameras – was a real challenge. "We begin by assuming that people will take pictures of themselves in conditions that are impossible to control," said Alexandru. The main difficulties: changes in the light, blurry shots without a tripod, and limited picture quality depending on the smartphone's camera.

All these factors had to be juggled to achieve a good final result. Because to be credible, an avatar has to be nearly perfect. "A small detail will turn people off immediately," said Alexandru. "The avatar has to have the right facial geometry and reproduce the texture, color and details like face wrinkles." Then there's the animation: everyone has their own way of smiling, yawning and frowning. Faithfully recreating such expressions required new algorithms for face animation to bring the reconstructed 3D avatars alive, even on mobile devices.

In the future, the EPFL researchers hope to further refine their process, including getting past the limits they currently face. Precisely duplicating hair, for example, is very complicated. And they have not yet customized the inside of the mouth – with the teeth and tongue – or the ears. Modeling the entire body is also in the works, although the researchers wanted to focus on the face first, since it is the most expressive part of human communication.

Credit: EPFL
Credit: EPFL
Credit: EPFL

Explore further: Software enables avatar to reproduce our emotions in real time

More information: "Dynamic 3D Avatar Creation from Hand-held Video Input." ACM Transactions on Graphics (Proceedings of SIGGRAPH), 2015.

Related Stories

Intel brings out the avatar in you with new app

June 20, 2014

Intel on Thursday announced Pocket Avatars, a messaging app that will allow you to send messages through video avatars. The app is available for free download in Google Play and App store in the United States and Canada. ...

A new window to the face

August 9, 2011

The human face is a complicated thing—powered by 52 muscles; contoured by the nose, eyebrows, and other features; and capable of an almost infinite range of expressions, from joy to anger to sorrow to puzzlement.

Recommended for you

Solving the jet/cocoon riddle of a gravitational wave event

February 22, 2019

An international research team including astronomers from the Max Planck Institute for Radio Astronomy in Bonn, Germany, has combined radio telescopes from five continents to prove the existence of a narrow stream of material, ...

1 comment

Adjust slider to filter visible comments by rank

Display comments: newest first

not rated yet Aug 05, 2015
You can do the same with your laptop webcam and free downloadable 3d software. http://www.fastco...ts-video

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.