Using mobile devices to augment reality can enhance creative play and exploration

November 9, 2015, Disney Research

A child need not choose between the immersive, but often passive world of digital media or the physical interaction of real-world games and activities. Scientists at Disney Research say augmented reality with mobile devices serves as the perfect bridge between the two and can enhance creativity in the process.

To demonstrate their concept, called Augmented Creativity, Disney researchers have developed several prototype apps that add computer-generated animation, sound and visuals to such activities as coloring books, scavenger hunts and team games that require physical interaction.

"Exploring the world and new activities is what childhood is all about," said Markus Gross, vice president at Disney Research. "Our research brings the seamless fusion of the real and virtual world together with an intelligent and creative gameplay. We believe that these concepts offer exciting virtual enhancements over real-world interactions."

The researchers presented their Augmented Creativity concept and related apps at the Symposium on Mobile Graphics and Interactive Applications.

"We have been able to use the power and popularity of to direct renewed emphasis on traditional activities of creative play," said Robert W. Sumner, principal research scientist. "One of our most compelling prototypes is an augmented-reality musical arrangement app that provides a tangible way for children to explore different music styles and arrangements by adding, removing, and re-arranging physical cards that represent different instruments and styles."

Another app, based on coloring, allows children to customize 3-D animated characters simply by coloring them as they normally would in a coloring book."

In a prototype for a multi-player game, players each use a mobile tablet to track a virtual object as they move around and talk with each other to cooperatively frustrate an invading alien force. They also developed a city-wide gaming framework, enabling the development of games such as scavenger hunts that get players outdoors, searching for interactive elements superimposed on buildings, parks and roads.

But Augmented Creativity isn't just about play. The researchers created a framework to help users write interactive narratives - a trickier task than standard storytelling because it requires the user to define and implement all possible interactions that someone might experience within the narrative. Another app helps children grasp the concept of programming robots; the child can program the robot to perform certain behaviors and then use to see not just how the robot behaves, but exactly how the program is being executed.

"Augmented reality can help children develop a deeper understanding of programming by making the dynamics of program execution visible," Sumner said. "And it opens up new opportunities in gaming, by making it possible to create new classes of games that include user-programmed characters."

In addition to Gross and Sumner, the research team included Fabio Zünd of ETH Zurich, Mattia Ryffel, Stéphane Magnenat, Alessia Marra, Maurizio Nitti, Gioacchino Noris and Kenny Mitchell of Disney Research, and Mubbasir Kapadia of Rutgers University.

Explore further: Researchers use augmented reality to turn coloring books into 3-D experience

More information: "Augmented Creativity- Bridging the Real and Virtual Worlds to Enhance Creative Play-Paper" [PDF, 14.60 MB]

Related Stories

New robot takes augmented reality to a new level

February 11, 2013

(—Researchers and engineers at a Japanese company called Different Dimensions have taken the concept of augmented reality to new heights by adding the touch of an avatar to the experience. That touch comes courtesy ...

Recommended for you

Cellular microRNA detection with miRacles

March 26, 2019

MicroRNAs (miRNAs) are short noncoding regulatory RNAs that can repress gene expression post-transcriptionally and are therefore increasingly used as biomarkers of disease. Detecting miRNAs can be arduous and expensive as ...

What happened before the Big Bang?

March 26, 2019

A team of scientists has proposed a powerful new test for inflation, the theory that the universe dramatically expanded in size in a fleeting fraction of a second right after the Big Bang. Their goal is to give insight into ...

Probiotic bacteria evolve inside mice's GI tracts

March 26, 2019

Probiotics—which are living bacteria taken to promote digestive health—can evolve once inside the body and have the potential to become less effective and sometimes even harmful, according to a new study from Washington ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.