DESY and IBM develop big data architecture for science

August 22, 2014 by Chaiti Sen
X-ray scan of biological cells. Credit: Britta Weinhausen/University of Göttingen

IBM today announced it is collaborating with Deutsches Elektronen-Synchrotron (DESY), a leading national research center in Germany, to speed up management and storage of massive volumes of x-ray data. The planned Big Data and Analytics architecture based on IBM software defined technology can handle more than 20 gigabyte per second of data at peak performance and help scientists worldwide gain faster insights into the atomic structure of novel semiconductors, catalysts, biological cells and other samples.

DESY's 1.7 mile-long PETRA III accelerator is a super microscope that speeds up electrically charged particles nearly to the speed of light – approximately 186,000 miles per second – and sends them through a tight magnetic slalom course to generate the most brilliant x-ray radiation of its kind. This synchrotron radiation is used by more than 2,000 scientists each year to examine the internal structure of a variety of materials with atomic resolution. A key challenge in this process is storing and handling huge volumes of X-ray data. .

"A typical detector generates a data stream of about 5 Gigabit per second, which is about the data volume of one complete CD-ROM per second," said Dr. Volker Gülzow, head of DESY IT. "And at PETRA III we do not have just one detector, but 14 beamlines equipped with many detectors, and they are currently being extended to 24. All this Big Data must be stored and handled reliably."

IBM Software Defined Storage technology code name Elastic Storage allows DESY to employ to store and handle the massive data flowing from PETRA III every second. Credit: DESY

DESY is addressing this Big Data challenge with the help of IBM Research and IBM Software Defined Storage technology code name Elastic Storage that can scale easily to store and handle more than 20 Gigabyte of data flowing every second from PETRA III. Elastic Storage can provide scientists with high-speed access to increasing volumes of research data. This architecture will allow DESY to develop an open ecosystem for research and offer analysis-as-a-service and cloud solutions to its users worldwide.

"IBM's software defined technologies can provide DESY the scalability, speed and agility it requires to morph into a real-time analytics service provider." said Jamie Thomas, General Manager Storage and Software Defined Systems, IBM. "IBM can take the experience gained at DESY and transfer it to other fields of data intensive science such as astronomy, climate research and geophysics and design storage architectures for the analysis of data generated by distributed detectors and sensors."

The scalability of the system can support DESY and a number of international partners that are currently building the X-ray laser European XFEL, a research light source that will generate even more data. "We expect about 100 Petabyte per year from the European XFEL," said Dr. Gülzow. That is comparable to the yearly data volume produced at the world's largest particle accelerator, the Large Hadron Collider (LHC) at the research center CERN in Geneva.

DESY is one of the world's leading accelerator centers and a member of the Helmholtz Association. It develops, builds and operates large particle accelerators used to investigate the structure of matter. DESY is housed in Hamburg and Zeuthen in Germany and is home to 3000 scientists from over 40 countries a year.

Explore further: DESY light source PETRA III now has world's sharpest X-ray eyes

Related Stories

Novel material stores unusually large amounts of hydrogen

November 20, 2013

An international team of researchers has synthesized a new material that stores an unusually large amount of hydrogen. Performing high-pressure X-ray studies at DESY's PETRA III and other light sources, the scientists detected ...

Solar cell degradation observed directly for the first time

December 9, 2013

With the help of DESY's X-ray light source PETRA III, researchers of Technische Universität München have, for the first time, watched organic solar cells degrade in real time. This work could open new approaches to increasing ...

First H.E.S.S. II data reveal promising pulsar signal

July 8, 2014

Following the intensive commissioning phase of the largest Cherenkov telescope worldwide (H.E.S.S. CT5), the H.E.S.S. collaboration has announced the detection of cosmic gamma rays of 30 gigaelectronvolts (GeV). The High ...

Recommended for you

The ethics of robot love

November 25, 2015

There was to have been a conference in Malaysia last week called Love and Sex with Robots but it was cancelled. Malaysian police branded it "illegal" and "ridiculous". "There is nothing scientific about sex with robots," ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.