Computer 'anthropologists' study global fashion

August 10, 2017 by Leslie Morris, Cornell University
Cornell researchers using AI to Develop Cultural Anthropology Tools. Credit: Cornell University

Each day billions of photographs are uploaded to photo-sharing services and social media platforms, and Cornell computer science researchers are figuring out ways to analyze this visual treasure trove through deep-learning methods.

Kavita Bala, professor of computer science; Noah Snavely, associate professor computer science at Cornell Tech; and Kevin Matzen, M.S. '15, Ph.D. '16, have released their results in a new paper, "StreetStyle: Exploring world-wide clothing styles from millions of photos."

"We present a framework for visual discovery at scale, analyzing clothing and fashion across millions of images of people around the world and spanning several years," Snavely said.

Bala said the group used deep learning to detect various attributes – the color or sleeve length of shirts, whether a person is wearing glasses or a hat, and so on – in millions of images.

"Using these detected attributes, we can then derive visual insight," Bala said. "For example, where in the world is wearing hats more common? At what time of the year? Which colors are more popular in summer versus winter? Our approach produces a first-of-its-kind analysis of global and per-city fashion choices and spatio-temporal trends."

The researchers looked specifically for fashion trends based on time and location. To eliminate all unrelated photos, the group first used facial-recognition technology to exclude photos that did not include people in them. Then, the group filtered the results to include photos that included the upper half of the body.

That left about 15 million photos. With the narrower selection of photos, the team developed an object-recognition program that recognized items of clothing. The program also learned a number of descriptors like sleeve length, color and pattern.

Once the images were tagged, the group put the data through another program, this one to recognize patterns not in the clothes but in the data. That churned out information on what clothing items were being paired with what, which trends were popular in which areas and how the trends changed over the three-year time period.

This research provides a look into cultural, social and economics factors that shape societies and provides insights into civilization.

"The combination of big data, machine learning, computer vision and automated analysis algorithms makes for a very powerful analysis tool in visual discovery of fashion and other areas," Matzen said.

Explore further: Cornell CIS and Adobe collaboration creates artificial intelligence photo tool

More information: StreetStyle: Exploring world-wide clothing styles from millions of photos. arXiv.

Related Stories

Creature-cataloging contest for computers

August 3, 2017

At a glance, would you be able to tell the difference between a donkey and a mule? A jaguar and a leopard? Most computers can't, at least not yet, but a contest hosted by Caltech and Cornell Tech, the engineering campus of ...

Where can I buy a chair like that? This app will tell you

August 23, 2016

If you think you have a knack for interior design, or just want to spruce up your own home, new technology developed by Cornell researchers may help you choose furnishings the way professionals do. And professionals may find ...

A 'Flickr-ing' view of the world, in 4-D

February 13, 2015

Imagine a version of Google Street View where you could hit the rewind button and see any point in time over the last five years. Cornell researchers are building something like that, at least for a few much-visited places.

Recommended for you

Robots as tools and partners in rehabilitation

August 17, 2018

In future decades, the need for effective strategies for medical rehabilitation will increase significantly, because patients' rate of survival after diseases with severe functional deficits, such as a stroke, will increase. ...

Security gaps identified in internet protocol IPsec

August 15, 2018

In collaboration with colleagues from Opole University in Poland, researchers at Horst Görtz Institute for IT Security (HGI) at Ruhr-Universität Bochum (RUB) have demonstrated that the internet protocol IPsec is vulnerable ...

Researchers find flaw in WhatsApp

August 8, 2018

Researchers at Israeli cybersecurity firm said Wednesday they had found a flaw in WhatsApp that could allow hackers to modify and send fake messages in the popular social messaging app.


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.