DESY and IBM develop big data architecture for science

Aug 22, 2014 by Chaiti Sen
X-ray scan of biological cells. Credit: Britta Weinhausen/University of Göttingen

IBM today announced it is collaborating with Deutsches Elektronen-Synchrotron (DESY), a leading national research center in Germany, to speed up management and storage of massive volumes of x-ray data. The planned Big Data and Analytics architecture based on IBM software defined technology can handle more than 20 gigabyte per second of data at peak performance and help scientists worldwide gain faster insights into the atomic structure of novel semiconductors, catalysts, biological cells and other samples.

DESY's 1.7 mile-long PETRA III accelerator is a super microscope that speeds up electrically charged particles nearly to the speed of light – approximately 186,000 miles per second – and sends them through a tight magnetic slalom course to generate the most brilliant x-ray radiation of its kind. This synchrotron radiation is used by more than 2,000 scientists each year to examine the internal structure of a variety of materials with atomic resolution. A key challenge in this process is storing and handling huge volumes of X-ray data. .

"A typical detector generates a data stream of about 5 Gigabit per second, which is about the data volume of one complete CD-ROM per second," said Dr. Volker Gülzow, head of DESY IT. "And at PETRA III we do not have just one detector, but 14 beamlines equipped with many detectors, and they are currently being extended to 24. All this Big Data must be stored and handled reliably."

IBM Software Defined Storage technology code name Elastic Storage allows DESY to employ to store and handle the massive data flowing from PETRA III every second. Credit: DESY

DESY is addressing this Big Data challenge with the help of IBM Research and IBM Software Defined Storage technology code name Elastic Storage that can scale easily to store and handle more than 20 Gigabyte of data flowing every second from PETRA III. Elastic Storage can provide scientists with high-speed access to increasing volumes of research data. This architecture will allow DESY to develop an open ecosystem for research and offer analysis-as-a-service and cloud solutions to its users worldwide.

"IBM's software defined technologies can provide DESY the scalability, speed and agility it requires to morph into a real-time analytics service provider." said Jamie Thomas, General Manager Storage and Software Defined Systems, IBM. "IBM can take the experience gained at DESY and transfer it to other fields of data intensive science such as astronomy, climate research and geophysics and design storage architectures for the analysis of data generated by distributed detectors and sensors."

The scalability of the system can support DESY and a number of international partners that are currently building the X-ray laser European XFEL, a research light source that will generate even more data. "We expect about 100 Petabyte per year from the European XFEL," said Dr. Gülzow. That is comparable to the yearly data volume produced at the world's largest particle accelerator, the Large Hadron Collider (LHC) at the research center CERN in Geneva.

DESY is one of the world's leading accelerator centers and a member of the Helmholtz Association. It develops, builds and operates large particle accelerators used to investigate the structure of matter. DESY is housed in Hamburg and Zeuthen in Germany and is home to 3000 scientists from over 40 countries a year.

Explore further: First H.E.S.S. II data reveal promising pulsar signal

Related Stories

First H.E.S.S. II data reveal promising pulsar signal

Jul 08, 2014

Following the intensive commissioning phase of the largest Cherenkov telescope worldwide (H.E.S.S. CT5), the H.E.S.S. collaboration has announced the detection of cosmic gamma rays of 30 gigaelectronvolts ...

Recommended for you

Five properties of physics that affect your gas mileage

29 minutes ago

Physics is inescapable. It's everywhere, making your Frisbees fly, your toilets flush and your pasta water boil at a lower temperature at altitude. We've harnessed these forces, along with chemistry and engineering, ...

Encryption made easier: Just talk like a parent

1 hour ago

Encrypting emails can be tedious, difficult and very confusing. And even for those who have mastered the process, it's useless unless the intended recipient has the correct software to decode the message. ...

SemanticPaint system labels environment quickly online

3 hours ago

Ten researchers from University of Oxford, Microsoft Research Cambridge, Stanford, and Nankai University have presented a new approach to 3D scene understanding with a system which they dubbed SemanticPaint. ...

User comments : 0

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.