Virtual Projection team puts iPhone writing on the wall (w/ video)

January 26, 2012 by Nancy Owano, report
Virtual Projection walkthrough: (a) Shaking the device to create a view. (b) Interacting with a non-projected view. (c-e) Creating a projection by aiming at the secondary display, long pressing, and releasing. (f) Synchronized interaction. (g) Projection frustum can be used for filtering or navigating. (h) Projections can be moved or deleted by aiming, long pressing, and dragging out of the display. Image credit: Dominikus Baur, et al. Proceedings of CHI 2012 (preprint)

( -- A collaborative team from the University of Calgary, University of Munich, and Columbia, have figured out a way to use a smartphone to project the phone’s display on to external displays nearby. The team thinks of its technology approach, Virtual Projection, as "borrowing available display space in the environment." Dominikus Baur, Sebastian Boring, and Steven Feiner are behind Virtual Projection. Feiner is Professor of Computer Science at Columbia University where he directs the Computer Graphics and User Interfaces Lab.

According to a formal description of their technique, Virtual Projection “is based on tracking a handheld device without an optical projector and allows selecting a target display on which to position, scale, and orient an item in a single gesture.”

The phone user holds the phone up to the target computer screen, the phone camera captures and compares images from the screen to work out location -- the system relies on tracking where the phone is being pointed--and passes information back to the computer screen via WiFi to place the projection on the screen. Multiple users can place images on the same screen, when the users want the images to work together.

Baur, currently a postdoctoral fellow at the University of Calgary, took to his blog this week to elaborate on what the team set out to accomplish: “When we started with Virtual Projection the initial idea was just to artificially replicate the workings of an optical projector."

The team wanted to come up with an easy solution and took up the concept. “While Virtual Projection has the clear downside that it requires a suitable display and does not work on any regular surface, we can at least fix some of the downsides of its real-world mode," he said. "One of the first things was getting rid of distortion. When projectors are aimed towards a wall at an angle, keystone distortion can arise, warping the resulting image. Virtual projections can be freely adjusted when it comes to distortion and transformations of the resulting image.”

Components used for the team’s no-cables prototype included regular, unmodified iPhones, a server that runs on a Windows-PC, and Wifi .

Their official paper on this project, “Virtual Projection: Exploring Optical Projection as a Metaphor for Multi-Device Interactions,” is to appear at the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI 2012) that starts May 6 in Austin. CHI is an international conference on human-computer interaction.

The team is upbeat that they are on to something which, sooner than later, could be a pervasive, everyday opportunity for communication. "Imagine having a VP server running on every display that you encounter in your daily life and being able to 'borrow' the space for a while (e.g., to look something up on a map)," said Baur. "Give it a few more years (and a friendly industry consortium) and this could become reality.”

Explore further: Invoked computing: Pizza box is too loud! I can't hear the banana

More information: … al-projection_pp.pdf

Related Stories

Reach out and touch 3D characters with RePro3D (w/ video)

September 17, 2011

( -- Lonely gamers who have felt the pain of being separated by a screen from their favorite personalities now have a way to reach out and touch their game characters, and that new way is RePro3D. A group of researchers ...

Holodesk prototype puts life in computers (w/ video)

October 20, 2011

( -- A research project at Microsoft Research Cambridge has brought forth a prototype called Holodesk, which lets you manipulate virtual objects with your hand. You literally "get your hands on" the virtual display. ...

Recommended for you

Google Assistant adds more languages in global push

February 23, 2018

Google said Friday its digital assistant software would be available in more than 30 languages by the end of the years as it steps up its artificial intelligence efforts against Amazon and others.

Researchers find tweeting in cities lower than expected

February 20, 2018

Studying data from Twitter, University of Illinois researchers found that less people tweet per capita from larger cities than in smaller ones, indicating an unexpected trend that has implications in understanding urban pace ...


Adjust slider to filter visible comments by rank

Display comments: newest first

5 / 5 (1) Jan 26, 2012
This is pretty cool.
Looking forward to see advances on this technology, although I think their vision of having VP on every mobile phone or VP servers on every display is a fairly hard thing to happen.

Still, nicely done and I'd like to see it happen.
not rated yet Jan 26, 2012
This article restored my faith in the diversity and immediacy of entirely new applications for the mobile phone. Everyone knows that the possibilities are endless, but corporate politics, big business interests, and other things tend to jeopardize creativity.

Most of us still consider the mobile thing in our pocket a kind of wireless telephone, with some features tacked on. But it is becoming increasingly clear to many of us, that soon phone calls will be an incidental item on the feature list.

For example, what would I lose if I went for a data-only plan, with no telephone connection? I think the only thing would be emergency calls. Regular talking I'd do via Skype, which comes with my phone.

Lately I have also toyed with the idea of using my ancient real-buttons phone for talking and the smartphone for everything else. -- The real-butons one is so much easier to use while driving, sporting, or outdoors.
not rated yet Jan 26, 2012
will this work for ipad or playbook or galaxy pads also?
not rated yet Jan 26, 2012
looks laggy :/

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.