What makes Paris look like Paris? Software finds stylistic core

Aug 07, 2012
What details in an ordinary street scene can help identify the city it is in? Software developed by Carnegie Mellon University researchers have developed software that can automatically identify elements, such as the emblematic street signs, balustrade window and balcony supports indicative of Paris in the upper photos, and the neoclassical entryway, Victorian window and cast iron railings indicative of London in the lower photos. Credit: Carnegie Mellon University

Paris is one of those cities that has a look all its own, something that goes beyond landmarks such as the Eiffel Tower or Notre Dame. Researchers at Carnegie Mellon University and INRIA/Ecole Normale Supérieure in Paris have developed visual data mining software that can automatically detect these sometimes subtle features, such as street signs, streetlamps and balcony railings, that give Paris and other cities a distinctive look.

The software analyzed more than 250 million visual elements gleaned from 40,000 Google Street View images of , London, New York, Barcelona and eight other cities to find those that were both frequent and could be used to discriminate one city from the others. This yielded sets of geo-informative visual elements unique to each city, such as cast-iron balconies in Paris, fire escapes in New York City and bay windows in San Francisco.

The discovered visual elements can be useful for a variety of computational geography tasks. Examples include mapping architectural correspondences and influences within and across cities, or finding representative elements at different geo-spatial scales such as a continent, a city, or a specific neighborhood.

This video is not supported by your browser at this time.

Researchers will present their findings Aug. 9 at SIGGRAPH 2012, the International Conference on Computer Graphics and Interactive Techniques, at the Los Angeles Convention Center.

Alexei Efros, associate professor of robotics and computer science at CMU, noted that although finding patterns in very large databases — so-called Big Data mining —is widely used, it has so far been limited to text or numerical data. "Visual Data is much more difficult, so the field of visual data mining is still in its infancy, but I believe it holds a lot of promise. Our data mining technique was able to go through millions of image patches automatically — something that no human would be patient enough to do," said Efros, who collaborated with colleagues including Abhinav Gupta, assistant research professor of robotics, and Carl Doersch, a Ph.D. student in CMU's Machine Learning Department. "In the long run, we wish to automatically build a digital visual atlas of not only architectural but also natural geo-informative features for the entire planet."

For this study, the researchers started with 25,000 randomly selected visual elements from city images gathered from Google Street View. A machine learning program then analyzed these visual elements to determine which details made them different from similar visual elements in other cities. After several iterations, the software identified the top-scoring patches for identifying a city. For Paris, those patches corresponded to doors, balconies, windows with railings, street signs (the shape and color of the signs, not the street names on the signs), and special Parisian lampposts. It had more trouble identifying geo-informative elements for U.S. cities, which the researches attributed to the relative lack of stylistic coherence in American cities with their melting pot of styles and influences.

"We let the data speak for itself," said Gupta, noting the entire process is automated, yet produces a set of images that convey a better stylistic feel for a city than a set of random images.

Doersch said this process requires a significant amount of computing time, keeping 150 processors working overnight. By comparison, art directors for the 2007 Pixar movie "Ratatouille" spent a week running around Paris taking photos so they could capture the look and feel of Paris in their computer model of the .

Explore further: Coping with floods—of water and data

More information: Project web site: graphics.cs.cmu.edu/projects/whatMakesParis/.

Related Stories

Israel gives green light to Google Street View

Aug 22, 2011

Israel's ministry of justice said on Monday that it has given the go-ahead for Google Street View to start photographing streets in Israel to put on its 3D-mapping website.

Recommended for you

Coping with floods—of water and data

Dec 19, 2014

Halloween 2013 brought real terror to an Austin, Texas, neighborhood, when a flash flood killed four residents and damaged roughly 1,200 homes. Following torrential rains, Onion Creek swept over its banks and inundated the ...

Cloud computing helps make sense of cloud forests

Dec 17, 2014

The forests that surround Campos do Jordao are among the foggiest places on Earth. With a canopy shrouded in mist much of time, these are the renowned cloud forests of the Brazilian state of São Paulo. It is here that researchers ...

User comments : 11

Adjust slider to filter visible comments by rank

Display comments: newest first

Scottingham
not rated yet Aug 07, 2012
"the researches attributed to the relative lack of stylistic coherence in American cities with their melting pot of styles and influences."

Or American cities are just fugly

antialias_physorg
not rated yet Aug 07, 2012
U.S. cities, which the researches attributed to the relative lack of stylistic coherence in American cities with their melting pot of styles and influences.

Interesting. This is something I've always tried to put my finger on why american cities just feel so 'wrong' wherever I went.
There's superficial style but the whole thing doesn't seem integrated (and if you look at anything in detail it's mostly shoddy).

It just seems like someone put up a building, chose some style at random and didn't care how it fit in with the rest of it.
The impression one gets is just a disjointed jumble of indvidually pretentious buildings that fuse into one really awful amalgam.
tadchem
5 / 5 (1) Aug 07, 2012
Many of these stylistic elements have been developed over centuries of tradition. All cities have a variety of styles. Passers-by remember the ones that appeal to them and imitate them in other buildings. Forgettable ones get forgotten, and are not copied. American cities are simply too young to have developed these traditions. San Francisco (despite its oriels) is only 164 years old, and was almost completely rebuilt only 106 years ago.
antialias_physorg
not rated yet Aug 07, 2012
Many of these stylistic elements have been developed over centuries of tradition.

That's probably the reason. Style develops and shifts ever so slightly over the decades. This gives older and newer buildings a sort of continuity in style. This is something that is sorely missing from US cities. There are indivdiual pieces - but no style.
Claudius
3 / 5 (3) Aug 07, 2012
Interesting how much the photos look like New Orleans' French Quarter.
Bowler_4007
1 / 5 (1) Aug 07, 2012
Many of these stylistic elements have been developed over centuries of tradition.

That's probably the reason. Style develops and shifts ever so slightly over the decades. This gives older and newer buildings a sort of continuity in style. This is something that is sorely missing from US cities. There are indivdiual pieces - but no style.
i disagree i would say american cities have a "mish-mash" style, but just imagine how boring the world would be if everywhere was pretty much the same
SatanLover
1 / 5 (1) Aug 07, 2012
sigh the french fascist jews are at it again, dictating people what is stylish.
Milou
not rated yet Aug 07, 2012
And I thought Disney and Las Vegas were giving us the perfect French style/experience???
antialias_physorg
not rated yet Aug 08, 2012
i disagree i would say american cities have a "mish-mash" style, but just imagine how boring the world would be if everywhere was pretty much the same

If it had its own style it would be OK. There were some attempts in New york to get an american style going.
But these superficial copies just lok...well..it feels like when you see a japanese guy dressed as a rocker. Even though he really tries to get close to the original it just looks pathetic.
Shootist
1 / 5 (1) Aug 08, 2012
What makes Paris look like Paris?


The holes in the sidewalk that serve as toilets?
SatanLover
1 / 5 (1) Aug 08, 2012
BTW some tourist areas in spain look exactly like paris.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.