New algorithm finds you, even in untagged photos

Dec 02, 2013
The location of tags in a images tells us a story. From them, we can extract a tag relativity graph. The graph enables social search by understanding tag relationships. Credit: University of Toronto

A new algorithm designed at the University of Toronto has the power to profoundly change the way we find photos among the billions on social media sites such as Facebook and Flickr. This month, the United States Patent and Trademark Office will issue a patent on this technology.

Developed by Parham Aarabi, a professor in The Edward S. Rogers Sr. Department of Electrical & Computer Engineering, and his former Master's student Ron Appel, the search tool uses tag locations to quantify relationships between individuals, even those not tagged in any given photo.

Imagine you and your mother are pictured together, building a sandcastle at the beach. You're both tagged in the photo quite close together. In the next photo, you and your father are eating watermelon. You're both tagged. Because of your close 'tagging' relationship with both your mother in the first picture and your father in the second, the can determine that a relationship exists between those two and quantify how strong it may be.

In a third photo, you fly a kite with both parents, but only your mother is tagged. Given the strength of your 'tagging' relationship with your parents, when you search for photos of your father the algorithm can return the untagged photo because of the very high likelihood he's pictured.

"Two things are happening: we understand relationships, and we can search images better," says Professor Aarabi.

The nimble algorithm, called relational social image search, achieves high reliability without using computationally intensive object- or facial-recognition software.

"If you want to search a trillion photos, normally that takes at least a trillion operations. It's based on the number of photos you have," says Aarabi. "Facebook has almost half a trillion photos, but a billion users—it's almost a 500 order of magnitude difference. Our algorithm is simply based on the number of tags, not on the number of photos, which makes it more efficient to search than standard approaches."

Work on this project began in 2005 in Professor Aarabi's Mobile Applications Lab, Canada's first lab space for mobile application development.

Currently the algorithm's interface is primarily for research, but Aarabi aims to see it incorporated on the back-end of large image databases or social networks. "I envision the interface would be exactly like you use Facebook —for users, nothing would change. They would just get better results," says Aarabi.

While testing the algorithm, Aarabi and Appel discovered an unforeseen application: a new way to generate maps. They tagged a few photographs of buildings around the University of Toronto and ran them through the system with a bunch of untagged campus photos. "The result we got was of almost a pseudo-map of the campus from all these we had taken, which was very interesting," says Aarabi.

Explore further: Google+ adds photo, video tools to ramp up challenge (Update)

More information: This work will be presented at the IEEE International Symposium on Multimedia Dec. 10, 2013.

Related Stories

The photo tag is back on Facebook

Feb 27, 2013

Tagging photos is hardly new on Facebook. In 2012, Facebook temporarily suspended the feature to make what it called technical improvements. This may or may not have had something to do with the outcry from privacy advocates ...

Recommended for you

User comments : 1

Adjust slider to filter visible comments by rank

Display comments: newest first

totten
not rated yet Dec 03, 2013
I believe Professor Aarabi meant a factor of 500, not 500 order of magnitude. half a trillion photos and 1 billion users is between 2 and 3 orders of magnitude.

More news stories

Ex-Apple chief plans mobile phone for India

Former Apple chief executive John Sculley, whose marketing skills helped bring the personal computer to desktops worldwide, says he plans to launch a mobile phone in India to exploit its still largely untapped ...

Airbnb rental site raises $450 mn

Online lodging listings website Airbnb inked a $450 million funding deal with investors led by TPG, a source close to the matter said Friday.

Health care site flagged in Heartbleed review

People with accounts on the enrollment website for President Barack Obama's signature health care law are being told to change their passwords following an administration-wide review of the government's vulnerability to the ...

A homemade solar lamp for developing countries

(Phys.org) —The solar lamp developed by the start-up LEDsafari is a more effective, safer, and less expensive form of illumination than the traditional oil lamp currently used by more than one billion people ...

NASA's space station Robonaut finally getting legs

Robonaut, the first out-of-this-world humanoid, is finally getting its space legs. For three years, Robonaut has had to manage from the waist up. This new pair of legs means the experimental robot—now stuck ...

Filipino tests negative for Middle East virus

A Filipino nurse who tested positive for the Middle East virus has been found free of infection in a subsequent examination after he returned home, Philippine health officials said Saturday.

Egypt archaeologists find ancient writer's tomb

Egypt's minister of antiquities says a team of Spanish archaeologists has discovered two tombs in the southern part of the country, one of them belonging to a writer and containing a trove of artifacts including reed pens ...