Data Travels Six Times Faster in the Clouds

February 26, 2009,
Sector is cloud computing system designed for data intensive computing. Sector is designed to run on racks of comodity computers (such as pictured). The racks may be located within a single data center or across several geographically distributed data centers. Credit: Michal Sabala, NCDM, University of Illinois at Chicago

(PhysOrg.com) -- The National Center for Data Mining (NCDM) at the University of Chicago at Illinois established a cloud computing system that can quickly compile data from widely geographically distributed data centers across high performance networks. NCDM used the Open Cloud Testbed, managed by the Open Cloud Consortium, to demonstrate the "Sector System" at the annual meeting of the American Association for the Advancement of Science conference earlier this month in Chicago.

"We demonstrated that our system is six times faster than competing technology," said Robert Grossman, NCDM director and Open Data Group managing partner. "Without the requirement of costly and combersome data transfer from various locations to one central location, this opens the way to exciting collaborative scientific discovery."

This is a diagram of Phase 1 of the Open Cloud Testbed. Phase 1 of the Open Cloud Testbed consists of 4 racks located at the University of Illinois at Chicago, the StarLight Facility in Chicago, Johns Hopkins University in Baltimore, Maryland, and Calit2 at the University of California at San Diego. The 4 racks are connected by a 10 Gb/s network, provided by the Cisco C-Wave and regional high performance networks at each of the locations. The Open Cloud Testbed is managed by the Open Cloud Consortium. Credit: Open Cloud Consortium

Grossman and his team demonstrated using a common benchmark called Terasort. They found there was less than a 5 percent performance penalty when Terasort was run across the four data centers distributed across the country compared to running the entire computation within one data center. Prior to the Sector System, such computations were rarely done, as performance penalties were as high as 30 percent.

"With the Sector System, data intensive computing can scale not only to a data center, but for the first time, across data centers," said Grossman." This enables locating data centers in areas in which power and cooling is cost-effective."

The Open Cloud Testbed consists of racks of computers located at the University of Illinois at Chicago, the StarLight facility in Chicago, Johns Hopkins University in Baltimore, Maryland, and the University of California at San Diego, all connected by a wide area 10 Gb/s network, and all running a variety of cloud computing services, including cloud storage services and cloud computing services. The technology that makes this possible uses an open architecture design, specifically the open source sector system developed by the NCDM (sector.sf.net).

Although cloud computing is becoming common, processing data by clouds today is almost always done within a single data center. Generally, data intensive computing across geographically distributed data centers is avoided due to the difficulties and cost of moving large amounts of data over long distances. Sector employs an alternative network protocol called UDT designed to swiftly and smoothly transfer data.

According to Joe Mambretti, director of the International Center of Advanced Internet Research at Northwestern University and co-director of the Open Cloud Testbed, "These innovative technologies provide unique capabilities that will enable new generations of applications that can make discoveries involving large volumes of highly distributed data."

Provided by National Science Foundation

Explore further: Flawed computer chips and how to fix them

Related Stories

Flawed computer chips and how to fix them

January 5, 2018

As tech giants race against the clock to fix major security flaws in microprocessors, many users are wondering what lurks behind unsettling names like "Spectre" or "Meltdown" and what can be done about this latest IT scare.

Bringing social media to unconnected areas

November 27, 2017

The number of connected devices may be on the rise, but large swaths of the global population still live in areas without telecom infrastructure or a reliable internet connection. A group of EPFL researchers, working with ...

Next Mars rover will have 23 'eyes'

October 31, 2017

When NASA's Mars Pathfinder touched down in 1997, it had five cameras: two on a mast that popped up from the lander, and three on NASA's first rover, Sojourner.

Cloud computing: a new horizon

April 16, 2009

The outlook is bleak for laptops, hard drives and desktops - clouds are on the horizon and could change the way we use computers forever. For some, the ‘cloud’ is just the latest technological craze, but for others it ...

Recommended for you

Cryptocurrency rivals snap at Bitcoin's heels

January 14, 2018

Bitcoin may be the most famous cryptocurrency but, despite a dizzying rise, it's not the most lucrative one and far from alone in a universe that counts 1,400 rivals, and counting.

Top takeaways from Consumers Electronics Show

January 13, 2018

The 2018 Consumer Electronics Show, which concluded Friday in Las Vegas, drew some 4,000 exhibitors from dozens of countries and more than 170,000 attendees, showcased some of the latest from the technology world.

Finnish firm detects new Intel security flaw

January 12, 2018

A new security flaw has been found in Intel hardware which could enable hackers to access corporate laptops remotely, Finnish cybersecurity specialist F-Secure said on Friday.

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.