Researchers Study Digg.com to Investigate Collective Attention

Nov 16, 2007 By Lisa Zyga feature
Researchers Study Digg.com to Investigate Collective Attention
One result of the study: The distribution of 29,864 popular stories on digg.com by the number of diggs each story received. The log-normal distribution shows that more than half of the stories received 200-800 diggs, with the number of stories with more than 800 diggs gradually decreasing up to about 4000 diggs. Credit: Fang Wu and Bernardo Huberman. ©2007 PNAS.

In a world where millions of people are bombarded with thousands of messages daily, understanding how some messages become popular among large populations is vital for successful advertising, marketing and journalism strategies.

Researchers Fang Wu and Bernardo Huberman from Hewlett-Packard Laboratories in Palo Alto, California, thought that the popularity of an idea or item in a large population should depend on its novelty, which tends to decrease over time due to a variety of factors. In order to test this idea, the researchers analyzed how a news story propagates on the interactive Web site digg.com.

Their results, which are published in PNAS, analyze 29,864 popular digg stories during the year 2006, with more than 1 million users determining the popularity of the submissions.

“The greatest significance of our results is that we were able to establish a connection between novelty and collective attention in large groups of people,” Huberman told PhysOrg.com. “Psychologists have studied the interplay between novelty and attention with human subjects for many years, but never at the group level on such a scale.”

Among Wu and Huberman’s results is the existence of a natural time scale over which attention grows and fades. Since the researchers only considered popular stories (those which made it to the front page, based on a complex algorithm), the initial growth must be rapid. However, once a story makes it to the front page, the data indicate a straight correlation between the average log of diggs and the time variance. In other words, the more popular a story becomes, the faster it spreads.

“Basically, at any given time, a group of n people will read the story, digg it and tell it to others, but only some random fraction of those told will actually read and digg the story,” Huberman explained. “This process, which naively would lead to multiplicative growth in the number of diggs, is counteracted by the fact that the novelty of the story is decaying in time. When put together, this process leads to a log-normal distribution of the number of diggs and also to a fast increase at the beginning that tapers off with a rate predicted by our equations, and verified to high precision with the data.”

As for the decline of interest in a story, a story’s digg rate decays, on average, following a stretched exponential law. This is because many factors contribute to the decreasing novelty of a successful story (such as topic category, time of day when it was made popular, etc.). Since individual stories tend to fade exponentially, the combined result looks like a stretched exponential. Interestingly, stretched exponentials have been used to explain other decays, such as the discharge of a capacitor and luminescence decays.

The researchers also analyzed the probability distribution of stories based on the total number of diggs received, which is approximately log-normal. Specifically, most popular stories quickly climb to 200 diggs, but the largest section (20%) tops off at between 400 and 600 diggs. After that, the distribution of diggs drops off more gradually, with a handful reaching up past the 3000-digg mark.

This study is one of the first to analyze data on collective attention from very large groups of people in a non-laboratory, yet isolated, setting. As such, the researchers hope that the results will help experts in choosing which items to display in different types of news media, advertising, or any other medium that relies on social networking to spread its content.

“Our theory points to a methodology for displaying content on a portal,” Huberman said. “If one is interested in maximizing the number of views of the portal, then given our theory and a few measurements, they can decide on posting the newest stories first (like digg does now) or the more favorite ones.”

More information:

HP’s Information Dynamics Lab: www.hpl.hp.com/research/idl/

Wu, Fang, and Huberman, Bernardo A. “Novelty and collective attention.” Proceedings of the National Academy of Sciences. November 6, 2007, vol. 104, no. 45, 17599-17601.

Copyright 2007 PhysOrg.com.
All rights reserved. This material may not be published, broadcast, rewritten or redistributed in whole or part without the express written permission of PhysOrg.com.

Explore further: Best of Last Week – Evidence of quark-gluon interactions, new portable device hack and why we may never live forever

add to favorites email to friend print save as pdf

Related Stories

Google Reader death brings online outcry

Mar 14, 2013

Outrage over Google's decision to pull the plug on Reader flooded the Internet on Thursday as lovers of the RSS service for tracking website updates lobbied to keep it alive. ...

Google invests in social with Kevin Rose hire

Mar 16, 2012

Google confirmed Friday that it has hired Digg co-founder Kevin Rose and other members of the team at his latest project, a Milk incubator for startups making "apps" for mobile gadgets.

Once hot Digg slashes staff by over one-third

Oct 25, 2010

Digg, the news-sharing site once one of the hottest destinations on the Web, announced Monday it was cutting over one-third of its staff in a bid to try to be profitable next year.

HP researchers try to tell you who your friends are

Dec 17, 2009

Most people have scores of contacts, scattered around their mobile phone, e-mail address book and multiple social networking sites. Scientists at Hewlett-Packard can tell you which of those contacts are your closest friends.

Recommended for you

Computer games give a boost to English

9 hours ago

If you want to make a mark in the world of computer games you had better have a good English vocabulary. It has now also been scientifically proven that someone who is good at computer games has a larger ...

Saddam Hussein—a sincere dictator?

14 hours ago

Are political speeches manipulative and strategic? They could be – when politicians say one thing in public, and privately believe something else, political scientists say. Saddam Hussein's legacy of recording private discussions ...

Oldest representative of a weird arthropod group

14 hours ago

Biologists at Ludwig-Maximilians-Universitaet (LMU) in Munich have assigned a number of 435-million-year-old fossils to a new genus of predatory arthropods. These animals lived in shallow marine habitats ...

User comments : 1

Adjust slider to filter visible comments by rank

Display comments: newest first

maynard
3 / 5 (3) Nov 16, 2007
More like 'collective attention deficit'.