Scientists Construct Model of the World Wide Web

Apr 08, 2008 By Lisa Zyga feature
Traffic statistics for the Web page, where the long periods of low activity and short bursts of high activity are similar to other sites that the researchers analyzed. Credit: Simkin and Roychowdhury.

Although the Internet contains well over 100 million Web sites, two electrical engineers think they know what the traffic patterns of the entire Web look like.

Mikhail Simkin and Vwani Roychowdhury, electrical engineers from both the University of California Los Angeles and, have constructed a model of the Web using the traffic statistics of just three Web pages: , , . (Traffic patterns from a dozen other Web pages the researchers studied were very similar.) Using several years of data from these three pages, the researchers show how the Internet overall reaches a self-organized critical (SOC) state with long-lasting traffic.

“One of the main implications of our findings is that traffic and [the corresponding] fame is a prolonged phenomenon instead of a one-time fling, and recurs in a spasmodic fashion,” Roychowdhury told

Most of the time, traffic to any single Web page is relatively low and steady, where visitors come from search engines, Web directories, online encyclopedias, and other constant sources. But these long periods of low traffic are interrupted by bursts of heavy traffic that follow a power law, usually the effect of numerous blog entries linking to those pages.

The researchers use a branching model to describe the probability and extent of these bursts. Basically, there’s a certain probability that a viewer will post a blog entry with a link to that Web page, and then a certain number of viewers who will visit the Web page via the blogger’s link. The product of these two variables determines whether or not a Web page will reach the critical value of 1, which determines if the branch keeps growing or dead-ends.

“A system is in a critical state if a single movement in an individual constituent element leads, on the average, to the movement of precisely one other element in the system,” Roychowdhury explained.

If a system is in a super- or sub-critical state, movement of one element leads, on average, to the movement of either more or less than one other element, respectively. That means that a signal generated in a super-critical system should increase forever, while a signal in a sub-critical system eventually dies out.

“But in a critical system, something very interesting happens,” Roychowdhury said. “Almost all signal cascades will die out, but some of them can last for a long time and can cover a large area. Clearly, sub- and super-critical systems are not that interesting unless we want a system that is either not that responsive or a system that explodes at the slightest provocation. Critical systems, on the other hand, allow for a responsive system to exist without it being blown apart. Many physical systems naturally gravitate towards a critical state, and this phenomenon is termed SOC.”

As the researchers explain, competition for viewers and links is a driving force of the Web, and this competition pushes the entire Web into an SOC state. Based on their data, the researchers determined the values for the two variables above for the “true art or fake art” site that closely produce its traffic patterns: they found that its link probability of 0.01 and referral number of 95 visitors per link results in a slightly sub-critical value of 0.95 for that particular Web page.

But since some Web pages are more interesting than others, some pages will achieve the critical value of 1 or even surpass it.

“To explain how the Web evolves into the SOC state, we need to use the concept of Darwinian fitness, which is a scientific measure of digital fangs and claws that help the Web page to fight for links with its competitors,” Simkin said. “The success in this competition depends not only on the Web page's own fitness, but also on the average fitness of other pages currently discussed in the blogosphere, with which our Web page competes.”

If this average is low, Simkin explained, then the fittest papers are super-critical. This means that, with time, they increase their share of the blogosphere. But in turn, this leads to the increase of the average fitness. The process continues until the fittest pages become exactly critical.

“One finding that is important for Webmasters is that our work disproves the so-called fifteen minutes of fame paradigm, according to which things can get popular soon after release and quickly become forgotten,” Simkin said. “One, of course, knows that this paradigm is manifestly wrong for immortal classics. However, our work shows it to be wrong not only for great creations, but for anything which is of any intrinsic (not created by advertisement) merit.”

The researchers found that the traffic to a Web page with fixed content can persist for at least several years.

“So one should not hurry to delete old Web pages,” Simkin said. “When there is enough – say a year – of access statistics, our model can be used to infer a page's fitness and predict the average volume and fluctuations of future traffic.”

Roychowdhury is a cofounder and Simkin a consultant for a start-up company called that focuses on next-generation Internet advertising. The company utilizes similar physics-based modeling of the Web, though not the direct results of the present study.

The researchers add that the Web is just one of many complex systems that exhibit self-organized criticality, with other examples including evolutionary patterns, earthquakes, and citations in research papers. They suggest that their model could also be used to explain the spreading of cultural elements, like movies, books, and fashion styles.

More information: Simkin, M. V. and Roychowdhury, V. P. “A theory of web traffic.” Europhysics Letters, 82 (2008) 28006.

Copyright 2008
All rights reserved. This material may not be published, broadcast, rewritten or redistributed in whole or part without the express written permission of

Explore further: Mist-collecting plants may bioinspire technology to help alleviate global water shortages

Related Stories

Beijing behind Internet security violation: group

Mar 25, 2015

China's cyberspace administration is "complicit" in attacks on major Internet companies including Google, an anti-censorship group said Wednesday, calling on firms worldwide to strengthen their defences.

Fitness app connects exercisers to experts

Mar 24, 2015

Can advanced networking and next-generation applications help solve some of our nation's most pressing health problems? Can mobile devices and high-speed Internet be used to improve our health and well-being? ...

China web freedom group faces online disruption

Mar 20, 2015

A U.S.-subsidized advocacy group that helps Internet users inside China bypass blockages on censored content says it is suffering a mysterious denial-of-service attack disrupting its operations.

US lawmakers weigh in on 'net neutrality'

Mar 17, 2015

A decision to impose tough new regulations on cable and wireless companies that provide Internet service to Americans wasn't influenced by politics, a top U.S. regulator told House lawmakers on Tuesday.

Wireless show highlights: Eyes on Apple, Blackberry

Mar 03, 2015

As a major wireless show continues this week in Barcelona, Spain, Apple is highlighting the capabilities of its iPhone cameras with a gallery of photos taken by its users around the world.

Recommended for you

Super sensitive measurement of magnetic fields

21 hours ago

There are electrical signals in the nervous system, the brain and throughout the human body and there are tiny magnetic fields associated with these signals that could be important for medical science. Researchers ...

New idea for Dyson sphere proposed

22 hours ago

(—A pair of Turkish space scientists with Bogazici University has proposed that researchers looking for the existence of Dyson spheres might be looking at the wrong objects. İbrahim Semiz and ...

Turning back time by controlling magnetic interactions

23 hours ago

In many materials, macroscopic magnetic properties emerge when microscopically small magnets align in a fixed pattern throughout the whole solid. In a publication in Nature Communications, Johan Mentink, Karste ...

User comments : 1

Adjust slider to filter visible comments by rank

Display comments: newest first

5 / 5 (1) Apr 13, 2008
@nilbud I think your missing the point. Blogs are just one example where this power law could be applied. More broadly I see this applying to any form of linking, and that includes those novel concepts with no links in sight whatsoever. As Steve Gillmor states; "Links are dead". In other words; novel concepts embedded in language semantics that lead to people searching for more on a subject...

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.