Transferring data files may seem a routine task to some, but to researchers, it's a task that becomes arduous when large amounts of computational data are involved—especially when considering large distances and wide area networks.
"When you use a wide area network, a network that spans a large geographical distance beyond a city for example, the performance can drop off greatly," says Paul Lu, a professor in the Department of Computing Science. "There are more pieces of equipment in the way—middleboxes, routers—and everyone of those pieces slows down the ability to transfer data quickly."
The complexity in transferring the data often comes down to its sheer size; it can consist of hundreds of gigabytes to terabytes. When transferred over a typical IP-based network, a transfer of this magnitude can take anywhere from a few hours to several days to complete. Sometimes opting for an alternate data transfer method, researchers will elect to ship hard drives to fellow colleagues.
The prospect of transferring large data over a powerful network—with high rates of speed—is a concept coming close to fruition, thanks to the assistance of local Edmonton-based vendor, Obsidian Strategics.
In a first-of-its-kind data transfer demonstration by the University of Alberta, Paul Lu facilitated a multi-collaborative effort at SC15 in Austin, Texas this past November that brought together Obsidian Strategics, CANARIE, Cybera, iCAIR, Gane Wong of the Department of Biological Sciences and Information Services & Technology to demonstrate the capabilities of InfiniBand—a technology which permits low-latency, high-bandwidth connections. The goal: to perform a large data transfer between Austin and Edmonton.
Stationed on the showroom floor in Austin, Texas and at the University of Alberta in Edmonton, each transfer site was configured with a dedicated server and solid state drive rated to transfer data at a rate of 900MB/s. Dr. David Southwell, director and chief visionary officer and Jason Gunthorpe, director and chief technology officer of Obsidian Strategics configured each site with a Longbow E100—a technology which extends the range of InfiniBand across a Wide Area Network.
"With the participation of iCAIR they routed the network from Austin to Chicago, CANARIE routed it to the province of Alberta, and Cybera picked it up as our Alberta network provider," says Lu. "We received a very high performance network connection between those two sites."
By utilizing a mirrored setup of servers connected by the Longbow E100 units across a dedicated encrypted network, researchers were able to transfer 300GB of genetics research data over 3,500km at a higher transfer rate than originally anticipated—a rate roughly 37 times faster than most home internet systems.
"When they performed testing [the transfer rate] was reaching 928MB/s," says Lu. "In fairness it could have gone faster; the bottleneck was the disk drive."
Data used throughout the demonstration was provided by Gane Wong, Jordan Patterson and Dr. Juan Jovel and pertained to research on gut microbiome and included associated sequencing data using next generation sequencing. Lu explains through advancements in chemistry, physics and computing, researchers are able to generate DNA sequencing for a fraction of the cost of a few years ago. Examining the gut microbiome from a wide populous of patients enables researchers to study connections between gut microbiome and gain insight into the fundamental processes of health and disease.
Advancements in data transfer rates provide researchers, such as Gane Wong and Paul Lu, the opportunity to analyze and react to complex data sets faster and provides the added benefit of fostering and supporting collaborations with colleagues around the world.
"In terms of science, getting data very quickly allows you to move to the next step," says Lu. "Sometimes [waiting for data transfers to complete] slows down the science—you want to get the data and be able to start analyzing it, and if something's wrong you want to go back and say, 'No, let's redo it, let's change it this way.'"
"This allows our researchers to be involved in global research programs no matter where they occur; our physical location is immaterial to their ability to participate," says Scott Delinger, director of research computing in Information Services & Technology.
Provided by University of Alberta
