Drawing in the third dimension

Imagine you could reach inside your old Batman comic, grab the Caped Crusader by the shoulder, and spin the whole scene around to get a new 3-D view.

A new software platform from small business Mental Canvas may soon let you do just that.

The technology allows you to draw like you would with pen and paper, except when you put the pen down, the sketch is viewable from multiple directions—like having access to every camera angle.

"I think of it as a spatial drawing," says Julie Dorsey, computer scientist at Yale and founder of Mental Canvas, a tech startup funded by the National Science Foundation (NSF). "Fundamentally, the technology expands on what we think of as a conventional drawing or sketch."

The tool could someday enable scientists to create 3-D versions of their back-of-the-envelope molecular structures. Architecture students can gain new perspectives on their building designs. Filmmakers can find new ways to visualize storyboards.

Leonardo wishes he had this

The technology started as passion project by Dorsey nearly 10 years ago.

There is a disconnect, she thought, between sketching in 2-D, which is fast and fluid, and modeling in 3-D, which is slow and requires precise geometry.

Dorsey studied architecture as an undergraduate student. She was drawn to computer graphics and graduate work in computer science as a way to expand her interests in designing, rendering and exploring human-made spaces, also known as "built" scenes.

"The built world poses an amazing range of difficult problems: from modeling stone weathering on a Gothic cathedral to simulating lighting effects in a city at night," she said.

But while we're surrounded by amazing devices—from smart watches to smart glasses—we're still drawing like it's the Renaissance.

"Sketching today isn't that different than what Leonardo [da Vinci] was doing, even with a tablet," Dorsey says. "Today's digital illustration packages merely simulate drawing on paper. They don't accelerate the sketching process or enhance a sketch's value as an ideation or communication tool."

Dorsey began to develop her technology in 2007 with support from a small NSF exploratory research grant. Prior to that, she had explored novel 3-D digital environments for years, with NSF's support.

Building the platform meant developing a new media type, as well as a set of tools to design and interact with this media form. Dorsey's challenge was to find a way to explore complex geometric forms through drawing. Her technology incorporates elements of computer-aided design, 2-D and 3-D graphics, software engineering, and human-computer interaction.

"Most people think IT innovations happen in a garage overnight," says Peter Atherton, NSF program officer who oversees Mental Canvas' Small Business Innovation Research grant. "But it actually most often takes years of hard work and support."

Honing the product

Consumer technologies—even really cool ones—are only successful if consumers actually use them.

In 2012, Dorsey participated in the NSF Innovation Corps program, which trains scientists and engineers to think more like businesspeople in order to get their inventions out of the lab and into the marketplace. She and her team learned how to hone their pitch and product for prospective customers.

Including, for instance, how to make her software stand out among other products already on the market.

"In other systems, you can create drawings or paintings on individual layers and those layers slide around in a single plane," she says. "Our technology involves drawing in space, so the underlying representation is 3-D rather than 2-D, and it's really fast and fluid."

As a first demonstration of the capabilities of the software and new media type, Mental Canvas has applied the technology to an illustrated book called "The Other Side" by Istvan Banyai.

The book is a modern-day graphic novel, taking the reader on a complex, visual journey. With Dorsey's technology rendering the story in 3-D, the effect is immersive. The company launched it today.

Doodling in 3-D

The company plans to make the drawing software itself available to consumers in the coming year, and to follow shortly thereafter with versions for specific markets, such as storyboarding and industrial design, that require more computing power and functionality.

The NSF SBIR grant is intended to provide seed funding as the company gains commercial footing, and feeds back into the nation's innovation ecosystem.

"Imagine picture books and graphic novels—and illustrations in general—in the future aren't flat but have active 3-D qualities to move around the story," Dorsey says.