New system estimates geographic location of photos

Jun 18, 2008
Determining Location Based on a Single Image
A method developed by Carnegie Mellon University researchers can estimate where a photo was taken by matching it to similar, GPS-tagged photos in the Flickr online photo collection. Credit: James Hays/Carnegie Mellon University

Researchers at Carnegie Mellon University have devised the first computerized method that can analyze a single photograph and determine where in the world the image likely was taken. It's a feat made possible by searching through millions of GPS-tagged images in the Flickr online photo collection.

The IM2GPS algorithm developed by computer science graduate student James Hays and Alexei A. Efros, assistant professor of computer science and robotics, doesn't attempt to scan a photo for location clues, such as types of clothing, the language on street signs, or specific types of vegetation, as a person might do. Rather, it analyzes the composition of the photo, notes how textures and colors are distributed and records the number and orientation of lines in the photo. It then searches Flickr for photos that are similar in appearance.

"We're not asking the computer to tell us what is depicted in the photo but to find other photos that look like it," Efros said. "It was surprising to us how effective this approach proved to be. Who would have guessed that similarity in overall image appearance would correlate to geographic proximity so well?"

Hays and Efros found they could accurately geolocate the images within 200 kilometers for 16 percent of more than 200 photos in their test set — up to 30 times better than chance. And even if their algorithm failed to identify the specific location, they often found that it could narrow the possibilities, such as by identifying the locale as a beach or a desert.

"It seems there's not as much ambiguity in the visual world as you might guess," said Hays, who will present the research at the IEEE Computer Society Conference on Computer Vision and Pattern Recognition June 24-26 in Anchorage, Alaska. "Estimating geographic information from images is a difficult, but very much a doable, computer vision problem."

Identifying the locale of a photo could enhance image search techniques, making them less dependent on captions or associated text. A computer system for geolocating photos could be useful in finding family photos from a specific trip and in some forensic applications. Determining the location of photos also makes it possible to combine them with geographic data bases related to climate, population density, vegetation, topography and land use.

Knowing the locale also can aid in such computer vision tasks as object identification, Hays said. If a computer recognizes that a photo likely was taken in Japan, for instance, the computer will have a better idea of what a taxicab should look like.

Hays said many online photos have some sort of geographic label, but these human descriptions can often be incorrect, or overly broad, such as a photo of the Grand Canyon labeled "U.S." The growing number of online photos that have GPS tags, by contrast, are unambiguous regarding their location, even though many are photos of rooms, people or events such as birthday parties that are useless for geolocation tasks. By using photos with both geographic keywords and GPS coordinates, Hays and Efros were able to find more than six million photos that were useful and accurately geolocated.

The IM2GPS algorithm readily located photographs of such landmarks as the Cathedral of Notre Dame in Paris. More surprisingly, it was able to recognize that a narrow street in Barcelona was typical of Mediterranean villages, rather than an American alleyway.

But some odd matches also occurred. The architecturally unique Sydney Opera House seemed to the computer to be similar to a hotel in Mississippi as well as a bridge in London. A shot of the Eiffel Tower at dusk was matched to other Eiffel Tower shots, but also to San Francisco's Coit Tower and New York's Statue of Liberty, both shot at dusk.

One reason for this confusion, Hays explained, is that the algorithm is not designed to recognize specific objects so much as it is to recognize geographic areas. For instance, an image of Utah's Monument Valley caused the IM2GPS algorithm to successfully retrieve a number of other images from Monument Valley and the American Southwest, rather than images of a specific rock formation.

For more information, see the IM2GPS project Web site: graphics.cs.cmu.edu/projects/im2gps/

Source: Carnegie Mellon University

Explore further: Professor analyzes online data to predict future fashion trends

Related Stories

Paperspace computer aims to resolve hardware distractions

Mar 05, 2015

A promotional video for Paperspace made its point quickly. A woman sitting at a computer sighed as she pressed down on the keys impatiently, as the straight-faced announcer noted computers can be such a pain. ...

US turns to rewards in hunt for overseas cyber criminals

Mar 19, 2015

The FBI considers Evgeniy Bogachev one of the world's most prolific and brilliant cyber criminals, slapping his photos—bald, beefy-faced and smiling faintly—on "Wanted" fliers posted online. The Russian ...

Researchers study fluctuations in solar radiation

Mar 18, 2015

The Sun is the Earth's principal source of energy and climate driver. Yet sometimes it sends more light to the Earth than other times. Astronomers working with Natalie Krivova at the Max Planck Institute ...

High-tech boost for beef producers

Mar 17, 2015

The farmers of the future will soon have a new tool at their disposal. Within two years, an artificial intelligence system trained to recognise indicators of animal condition will take its place in the yards ...

Scientists fly kites on Earth to study Mars

Mar 16, 2015

An unconventional research method allows for a new look at geologic features on Earth, revealing that some of the things we see on Mars and other planets may not be what they seem.

Recommended for you

Cost-effective production of magnetic sensors

1 hour ago

They are found wherever other measurement methods fail: magnetic sensors. They defy harsh environmental conditions and also function in fluids. A new procedure is now revolutionizing the production of two-dimensional ...

Measurement of components in 3D under water

1 hour ago

Conveying systems for oil and gas, operated in the sea have many important underwater components. The maintenance of such components is elaborate and expensive, as measuring them is complicated. Fraunhofer ...

Wear your social network

1 hour ago

What if your likes and interests on social media were broadcast to the world offline? Would that make it easier for you to make real-world connections with people? That's the idea behind Social Textiles, ...

Reliable systems for recharging electric vehicles

2 hours ago

The success of electric vehicle networks depends on economical vehicles – and efficient power grids. Existing power lines were not designed for the loads generated by electric vehicles. Fraunhofer researchers ...

Plastic parts for internal combustion engines

2 hours ago

Efforts to produce lighter vehicles necessarily include engine parts, such as the cylinder casing, which could shed up to 20 percent of its weight if it were made of fiber-reinforced plastic rather than aluminum ...

User comments : 2

Adjust slider to filter visible comments by rank

Display comments: newest first

Damon
3 / 5 (2) Jun 18, 2008
And we still can't find Bin Laden!
Sean_W
3.2 / 5 (5) Jun 18, 2008
He doesn't pose for outdoor shots much. The natural lighting exposes how gray his beard has gotten. Just For Men (or Just For Muslims) hair and beard colour is hard to get in Central Asian caves.

But the technology might be useful for tracking down the scenes of child porn manufacturing if a window scene is partially visible. And lots of other crimes could be solved by linking someone to a location via a photo that may have otherwise seemed meaningless. Not to mention resolving other less criminal mysteries like the history behind old photographs in someone's attic.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.