Realistic avatars for the virtual zoo

June 21, 2018, Max Planck Institute for Intelligent Systems Tübingen
The researchers generate 3D textured articulated models of animals from images. Starting from an initial coarse shape obtained with the SMAL model, we refine the animal shape from multiple uncalibrated images and varying poses and capture a detailed texture map from multiple frames. Credit: Silvia Zuffi, Angjoo Kanazawa, Michael J. Black

Filmmakers and developers of computer games will have a new way of animating animals in the future. A team led by researchers at the Max Planck Institute for Intelligent Systems in Tübingen, Germany, has developed a technique that uses photographs alone to create lifelike 3-D models of almost all quadrupeds. Animations of these avatars can realistically imitate the movements of animals. But the simple method of bringing animals to life on the computer is not only interesting for the entertainment industry. Many people have lost a beloved family pet. Now this technology can bring them back to "life" as a virtual 3-D avatar. It could also benefit biologists in species protection and help to make children in particular aware of the importance of biodiversity.

"Populating the virtual world with animals used to be a lot of manual work," says Michael J. Black, director of the Perceiving Systems department at the MPI for Intelligent Systems. Today, animators laboriously design 3-D animals using graphics software. While there are technologies to create realistic 3-D avatars of humans, no such technology can automatically capture the shape of wild animals. "You can't bring a tiger into the lab to be scanned and you can't take expensive scanners out into nature," says Black.

The team, whose technique creates avatars of animals including their fur or skin with reasonable effort, includes scientists from the CNR Institute for Applied Mathematics and Information Technology (IMATI) in Milan and the University of California in Berkeley as well as Max Planck researchers. "Ultimately, 3-D animal capture is about reconstructing the 3-D structure from 2-D images of moving objects captured by various unknown and moving cameras," explains Silvia Zuffi, the lead author on the project from IMATI.

In their new method, the researchers only need a few photos of an animal in different views to calculate realistic models that can then be animated naturally. Unlike other approaches, the images from which the avatars are generated can also come from different cameras. The computer program delivers such convincing results because it doesn't start from scratch when it creates a model. "The key is to start with a rough 3-D shape that can explain the shapes of many animals and then refine this based on the observed images," says Michael J. Black.

Helping endangered animals in the wild

In their current work, the researchers demonstrate that their method works for tigers, bears, horses and rhinos. "Our algorithm generates realistic models of animals for which no previous 3-D models exist," says Silvia Zuffi. For example, the researchers were able to reconstruct the 3-D shape of the extinct Tasmanian tiger from low-quality film footage shot in the 1930s. The researchers now want to make the method fully automated so that it can operate on thousands of images of animals in the wild.

Fully automated 3-D animal shape capture would not only facilitate the work of game developers and animators, but also supports animal conservation. Given photographs of animals captured by camera traps, the method could determine whether their shape, i.e. their weight, has changed. Species conservationists would thus receive quick and reliable statements as to whether endangered are doing well.

Explore further: Researchers dress virtual avatars with digitally captured clothing

More information: Silvia Zuffi, Angjoo Kanazawa, Michael J. Black, "Lions and Tigers and Bears: Capturing Non-Rigid, 3D, Articulated Shape from Images," Proc. CVPR, 2018.

The work appears in the annual IEEE Conference on Computer Vision and Pattern Recognition (CVPR) held in Salt Lake City. 

For more about the research see: smalr

The 3D models are freely available for download from: sketchfab

Related Stories

Tiny motions bring digital doubles to life (w/ Video)

December 7, 2014

Researchers at the Max Planck Institute for Intelligent Systems unveil new technology for motion and shape capture (MoSh) that helps animators jump the "Uncanny Valley" by turning a few moving dots into detailed body shapes ...

Romania to ban wild animals in circuses

June 13, 2017

Tigers, lions, bears and other wild animals will be banned from circuses in Romania after the country's parliament passed a bill on Tuesday in a move welcomed by animal rights groups.

Recommended for you

Paleontologists report world's biggest Tyrannosaurus rex

March 22, 2019

University of Alberta paleontologists have just reported the world's biggest Tyrannosaurus rex and the largest dinosaur skeleton ever found in Canada. The 13-metre-long T. rex, nicknamed "Scotty," lived in prehistoric Saskatchewan ...

NASA instruments image fireball over Bering Sea

March 22, 2019

On Dec. 18, 2018, a large "fireball—the term used for exceptionally bright meteors that are visible over a wide area—exploded about 16 miles (26 kilometers) above the Bering Sea. The explosion unleashed an estimated 173 ...

Coffee-based colloids for direct solar absorption

March 22, 2019

Solar energy is one of the most promising resources to help reduce fossil fuel consumption and mitigate greenhouse gas emissions to power a sustainable future. Devices presently in use to convert solar energy into thermal ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.