How natural disasters and political unrest affect the Internet

In the wake of the recent earthquake and tsunami in Japan, as well as political unrest in Egypt and Libya, researchers at Northwestern University are analyzing data that provides unique insight into the effects of these crises on the Internet.

Using data from approximately 1.4 million users of Ono and NEWS, software programs developed by the AquaLab research group of Fabián E. Bustamante, associate professor of electrical engineering and computer science at the McCormick School of Engineering, researchers can get a glimpse of Internet traffic from the user perspective.

"Often people look at Internet traffic from either the network itself, at network routers, or from large Internet services, to see how a crisis might have affected Internet activity," Bustamante explains. "By using data from users of our software and collaborators, we're able to see beyond firewalls and NATs (Network Address Translators) and capture the view from the very edge of the network – the users themselves."

By analyzing the data, the team found how users in Japan only showed reduced activity within the 24 hours after the earthquake. In the case of both Libya and Egypt, which have been subject to Internet shutdowns, the events can be clearly seen in sudden drops in the number of online users.

The software of Bustamante's group can also identify effects of network throttling, a practice of limiting the rate of data transfer to control congestion. Virgin Media, a large Internet service provider in the United Kingdom, recently completed a test of its new throttling policy, targeting peer-to-peer users. The user data showed that the policy had a dramatic effect on its users, at times cutting their upload capacity in half.

About Ono and NEWS

Ono (Hawaiian for "delicious") and NEWS (Network Early Warning System), are two of the software pieces developed in the AquaLab group, led by Bustamante.

Both are extensions to the popular BitTorrent peer-to-peer system. Ono addresses the tension between Internet Service Providers and some of their customers who use P2P services. The growing network traffic credited to P2P has driven service providers to forcefully reduce user services traffic at the expense of unhappy subscribers and the risk of government investigations.

Ono helps BitTorrent peers find other peers that are nearby, in a networking sense, by reusing the network "view" of other large-scale services such as content distribution networks (CDN) like Akamai and Limelight. CDN services redistribute content from popular websites on their servers around the world and redirect clients to nearby copies based on network conditions. Using the key assumption that two computers sent to the same CDN server are likely close to each other, Ono allows P2P users to quickly identify nearby users. Bustamante and his group have shown that Ono can provide over 30 percent average download rate improvement and, in environments with large available bandwidth, a 200 percent increase in download rates.

NEWS relies on peer-to-peer users to help detect and report Internet network performance problems. Because choppy playback on streaming video may be irritating to a user, such issues can drive thousands of potential customers away from the site providing the feed. As the Internet continues to grow, these network problems, or anomalies, become all the more frequent and frustrating. Determining the existence, let alone the impact, of network anomalies is important because the Internet has no overall monitoring system. Current monitoring systems try to identify network anomalies and can look for issues that could lead to performance problems but cannot tell whether individual users are actually experiencing problems.

Every day, millions of Internet users worldwide naturally generate data traffic that inherently provides information about whether the network is working or not. (Think of the millions of peer-to-peer users in systems like BitTorrent or Skype.). By sharing high-level information about their experience, these users could very efficiently and accurately detect where problems occur in real time. "You can think of it as crowd-sourcing network monitoring," says Bustamante.