DESY and IBM develop big data architecture for science

Aug 22, 2014 by Chaiti Sen
X-ray scan of biological cells. Credit: Britta Weinhausen/University of Göttingen

IBM today announced it is collaborating with Deutsches Elektronen-Synchrotron (DESY), a leading national research center in Germany, to speed up management and storage of massive volumes of x-ray data. The planned Big Data and Analytics architecture based on IBM software defined technology can handle more than 20 gigabyte per second of data at peak performance and help scientists worldwide gain faster insights into the atomic structure of novel semiconductors, catalysts, biological cells and other samples.

DESY's 1.7 mile-long PETRA III accelerator is a super microscope that speeds up electrically charged particles nearly to the speed of light – approximately 186,000 miles per second – and sends them through a tight magnetic slalom course to generate the most brilliant x-ray radiation of its kind. This synchrotron radiation is used by more than 2,000 scientists each year to examine the internal structure of a variety of materials with atomic resolution. A key challenge in this process is storing and handling huge volumes of X-ray data. .

"A typical detector generates a data stream of about 5 Gigabit per second, which is about the data volume of one complete CD-ROM per second," said Dr. Volker Gülzow, head of DESY IT. "And at PETRA III we do not have just one detector, but 14 beamlines equipped with many detectors, and they are currently being extended to 24. All this Big Data must be stored and handled reliably."

IBM Software Defined Storage technology code name Elastic Storage allows DESY to employ to store and handle the massive data flowing from PETRA III every second. Credit: DESY

DESY is addressing this Big Data challenge with the help of IBM Research and IBM Software Defined Storage technology code name Elastic Storage that can scale easily to store and handle more than 20 Gigabyte of data flowing every second from PETRA III. Elastic Storage can provide scientists with high-speed access to increasing volumes of research data. This architecture will allow DESY to develop an open ecosystem for research and offer analysis-as-a-service and cloud solutions to its users worldwide.

"IBM's software defined technologies can provide DESY the scalability, speed and agility it requires to morph into a real-time analytics service provider." said Jamie Thomas, General Manager Storage and Software Defined Systems, IBM. "IBM can take the experience gained at DESY and transfer it to other fields of data intensive science such as astronomy, climate research and geophysics and design storage architectures for the analysis of data generated by distributed detectors and sensors."

The scalability of the system can support DESY and a number of international partners that are currently building the X-ray laser European XFEL, a research light source that will generate even more data. "We expect about 100 Petabyte per year from the European XFEL," said Dr. Gülzow. That is comparable to the yearly data volume produced at the world's largest particle accelerator, the Large Hadron Collider (LHC) at the research center CERN in Geneva.

DESY is one of the world's leading accelerator centers and a member of the Helmholtz Association. It develops, builds and operates large particle accelerators used to investigate the structure of matter. DESY is housed in Hamburg and Zeuthen in Germany and is home to 3000 scientists from over 40 countries a year.

Explore further: First H.E.S.S. II data reveal promising pulsar signal

add to favorites email to friend print save as pdf

Related Stories

First H.E.S.S. II data reveal promising pulsar signal

Jul 08, 2014

Following the intensive commissioning phase of the largest Cherenkov telescope worldwide (H.E.S.S. CT5), the H.E.S.S. collaboration has announced the detection of cosmic gamma rays of 30 gigaelectronvolts ...

Recommended for you

Watches, robots suitcases: mobile gadget highlights

4 hours ago

Tech companies showcased countless connected gadgets at the world's biggest wireless telecom fair, the Mobile World Congress in Barcelona, which wrapped up on Thursday. Here is a selection of highlights:

Cebit 2015: DIY printing custom touch-sensitive displays

6 hours ago

Computer scientists from Saarbrücken have developed a technique that could enable virtually anyone to print out customized displays of their own in future—in all shapes and sizes and onto various materials. ...

Power-generating urinal pioneered in Britain

6 hours ago

British scientists on Thursday unveiled a toilet that unlocks energy stored within urine to generate electricity, which they hope could be used to light remote places such as refugee camps.

User comments : 0

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.