Rebuilding the world one pixel at a time

"The idea behind "Rome in a Day"' is that we wanted to see how big of a city or model we could build from photos on the internet," says Steve Seitz from the university's graphics and imaging laboratory. With support from the National Science Foundation (NSF), they're rebuilding Rome pixel by pixel rather than brick by brick.

This video is not supported by your browser at this time.

The project starts with a trip to the photo-sharing site Flickr to search for images of the real thing. Once pictures are identified, the computer starts the process of making 3-D objects from 2-D stills. Sameer Agarwal, a former postdoctoral scholar, at the university is mostly responsible for creating the algorithm that makes 3-D objects in virtual space from thousands of 2-D images.

"If I am a sculpture and there were three photographs of me, we would try to find three points in each photograph that point to my nose. From that we know that there are three points in these images that correspond to a single point in the 3-D world," explains Agarwal. "We would be able to say where in a particular image corresponding to that camera, the image of my nose should show up. This statement can be written as an equation involving the position and orientation of the camera, the position of my nose and where in the image my nose shows up. And you can connect all of these equations together and solve them to, in one shot, obtain both the positions of the cameras as well as the position of my nose in the 3-D world relative to those cameras."

Computers map huge clusters of points in 3-D space creating ghost-like images called "Point Clouds."

Seitz says the imaging is very accurate. "For the buildings, I think we can get accuracy to within a few centimeters. We've measured this. For individual objects that are photographed closer, we can potentially do a lot better, like millimeter accuracy."

Finally, color and texture are added. What Seitz and his colleagues have gotten are virtual 3-D tours of cities like Dubrovnik, Croatia or Venice, Italy.

"What excites me is the ability to capture the real world; to be able to reconstruct the experience of being somewhere without actually being there," says Seitz.

In the future this "next generation" technology may show up in places online like mapping sites, video games or real estate sites--it's a virtual guarantee.

Provided by National Science Foundation