Moving data at the speed of science: Berkeley Lab lays foundation for 100 Gbps prototype network

Jul 13, 2011 by Jon Bashor
In its initial phase, the Advanced Networking Initiative (ANI) 100 gigabit-per second (100 Gbps) prototype network will connect the three DOE unclassified supercomputing centers: the National Energy Research Scientific Computing Center (NERSC) at Berkeley Lab, Oak Ridge Leadership Computing Facility (OLCF), and Argonne Leadership Computing Facility (ALCF), as well as the Manhattan Landing International Exchange Point (MANLAN). This prototype will lead to a nationwide 100 Gbps scientific network, a key step toward the DOE's vision of a 1 terabit network connecting its exascale supercomputers.

The DOE's Lawrence Berkeley National Laboratory (Berkeley Lab) today announced a major step toward creating one of the world's fastest scientific networks to accelerate research in fields ranging from advanced energy solutions to particle physics. Known as the Advanced Networking Initiative (ANI), the effort represents a $62 million multi-year investment by the DOE Office of Science in next-generation networking technology.

"As science becomes increasingly data-driven and global in scale, it's critical that we create an infrastructure that will enable our scientists to collaborate and compete successfully in the search for solutions to some of the world's biggest challenges in energy," said DOE Office of Science Director William F. Brinkman. "The Advanced Networking Initiative is the kind of investment that will help secure and maintain America's scientific pre-eminence and improve the quality of life for all of us."

Under the subcontracting agreement that Berkeley Lab signed with Internet2, the Department of Energy's Energy Sciences Network (ESnet) staff will work with Internet2 to develop a 100 gigabit- per-second (100 Gbps) prototype network, increasing the information-carrying capacity of DOE's present scientific network by several orders of magnitude.

"Berkeley Lab has long been a key part of the nation's scientific leadership in physics, chemistry, and energy research," said Paul Alivisatos, director of Lawrence Berkeley National Laboratory. "Berkeley Lab has continued this tradition through advanced computation and networking used at the forefront of modern scientific inquiry. It's fitting that, when it came to building the next-generation network, the DOE turned to Berkeley Lab and ESnet to ensure that ANI meets the needs of the U.S. research community to work with the increasingly complex data flows associated with tackling our nation's most urgent challenges and leading the world in collaborative, discovery-class research."

The prototype network is a crucial bridge to a planned nationwide 100 Gbps scientific network that will support thousands of DOE scientists in their research on environmental modeling, developing energy solutions, and exploring the fundamental nature of the universe, as well as in accessing data from one-of-a-kind experimental facilities such as the Large Hadron Collider. A portion of the funding also goes to the development of a national-scale network testbed, made available to researchers and industry for experiments with new network technologies, protocols and applications to help them get up to speed on the new 100 Gbps capability.

To build the national network, Internet2 will use fiber strands on Colorado-based telecommunications firm Level 3 Communications' Tier 1 fiber-optic network, and will provide cutting-edge optical networking equipment from Ciena Corporation, a Maryland-based provider of network infrastructure solutions. By collaborating on complementary projects, ESnet and Internet2 hope to leverage both the ANI investment and Internet2's own investments to maximize the value of both, to deliver unprecedented network capabilities to an increased pool of potential users.

The 100 Gbps prototype network, including hardware and dark fiber, will initially connect three DOE unclassified supercomputing centers: the National Energy Research Scientific Computing Center (NERSC) at Berkeley Lab, Oak Ridge Leadership Computing Facility (OLCF), and Argonne Leadership Computing Facility (ALCF), as well as the Manhattan Landing International Exchange Point (MANLAN).

"Internet2 is excited by the synergies created through this joint project, and how it builds on an already solid relationship with Berkeley Lab to create a flagship scientific network of unrivaled speed and capacity," said Dave Lambert, Internet2 president and CEO. "We believe the new ANI network will not only play a vital role in helping our community fulfill their research, education and service goals, but will create a host of economic opportunities and assure the U.S. a leading role in global science for many years to come."

An Investment in Scientific Competitiveness

This kind of stimulus investment in IT infrastructure is intended to spur U.S. scientific competitiveness by leading to accelerated development and wider deployment of future technologies, speeding the exchange of data and creating new ripples of scientific innovation—and the jobs that come with it. According to Steve Cotter, ESnet Department Head, this ANI funding is enabling Berkeley Lab and Internet2 to advance 100 Gbps technology development, lower its cost, and speed its adoption by the scientific community.

Networks like ESnet are ramping up capacity as the volume of data generated by scientific research rises exponentially. For some scientists, ANI's 100 Gbps capabilities cannot come fast enough.

"We now have many more people wanting to access and then replicate climate data than in previous years," said Dean Williams, climate scientist at Lawrence Livermore National Laboratory.

Long-Range Benefits

As part of the agreement, the Berkeley Lab has negotiated a 20-year lease on dark fiber capacity to be leveraged for so-called "disruptive" research, allowing DOE, industry and university researchers to use the facility to investigate new networking technologies and protocols, and effectively locking in the cost of future network infrastructure needs for the next two decades.

The ANI network will also jump-start ESnet's research in support of "green" energy-efficient networking–a hot priority as data volumes and network and supercomputing center energy demands keep rising. With Berkeley Lab's commitment to lead efforts in carbon reduction and energy efficiency for research, the ANI 100 Gbps prototype will be instrumental in providing measurements of energy consumption on a real-time basis. ESnet intends to monitor and archive energy usage, making this data available to researchers so they can better understand energy consumption as they develop the next generation of energy-smart networking devices.

Explore further: One man trying to bring Broadway into 21st century

More information:

Related Stories

New technology enables high-speed data transfer

Jun 18, 2009

GridFTP, a protocol developed by researchers at Argonne National Laboratory, has been used to transfer unprecedented amounts of data over the Department of Energy's (DOE) Energy Sciences Network (ESnet), which provides a ...

California is the primary US stop for LHC's ALICE data

Jul 30, 2010

For approximately one month a year, the nuclei of lead atoms traveling near the speed of light will collide in the Large Hadron Collider's (LHC) ALICE experiment, generating a fireball about 100,000 times hotter than the ...

New Internet2 Land-Speed Record: 6.63 Gigabits per Second

Sep 02, 2004

Scientists at the California Institute of Technology (Caltech) and the European Organization for Nuclear Research (CERN), along with colleagues at AMD, Cisco, Microsoft Research, Newisys, and S2io have set a new Internet2 land-speed reco ...

NERSC supercomputing center breaks the petaflops barrier

Nov 16, 2010

The Department of Energy's National Energy Research Scientific Computing Center (NERSC), already one of the world's leading centers for scientific productivity, is now home to the fifth most powerful supercomputer ...

Recommended for you

Cutting congestion on the data network highway

Sep 12, 2014

Perhaps no other consumer-driven technology has made such incredible advances in such a relatively short space of time as the mobile phone. Today's smartphones are used to stream videos, access social media ...

T-Mobile to sell phones that call, text on Wi-Fi (Update)

Sep 10, 2014

T-Mobile will sell more than 100 smartphone models with a built-in feature that taps into Wi-Fi networks to make phone calls and send texts when customers can't connect to the wireless carrier's cellular network.

User comments : 7

Adjust slider to filter visible comments by rank

Display comments: newest first

1 / 5 (1) Jul 13, 2011
They need to have X-Prizes associated with these networks for more incentive. This should enhance R&D results. Also, it looks like the north central area of the country won't be included according to this map.
5 / 5 (2) Jul 13, 2011
Humm... lets see. 8 bits in a byte. so by my calculations it will take 12 seconds to reach my 150gb data cap imposed by my ISP.
3 / 5 (2) Jul 13, 2011
@ JimB135 but your taxes are so low!
not rated yet Jul 13, 2011
The benefits of this will be primarily limited to research facilities.

Down the road, we may experience some benefits if a more efficient or reliable or safe protocol is developed from this, but your ISP is already connected to a network with higher bandwidth (one trunk line has a 1.6 terabit capacity - wikipedia). It's just split up to you and millions of other people.
not rated yet Jul 13, 2011
Its really cool to see how much data is transferred just for research.

Here are my ideas for a high-bandwidth decentralized wireless internet: http://episin.blo...ost.html
not rated yet Jul 14, 2011
Did they mention 1 terabit network ?

not rated yet Jul 14, 2011
with 1 terabit (minus the overhead) you'd have the speed to cluster several supercomputers from coast to coast as one virtual machine, granted, we already have clusters but usually within the same campus/building