What to do with 15 million gigabytes of data

Nov 03, 2008

When it is fully up and running, the four massive detectors on the new Large Hadron Collider (LHC) at the CERN particle-physics lab near Geneva are expected to produce up to 15 million gigabytes, aka 15 petabytes, of data every year. Andreas Hirstius, manager of CERN Openlab and the CERN School of Computing, explains in November's Physics World how computer scientists have risen to the challenge of dealing with this unprecedented volume of data.

When CERN staff first considered how they might deal with the large volume of data that the huge collider would produce when its two beams of protons collide, in the mid-1990s, a single gigabyte of disk space still cost a few hundred dollars and CERN's total external connectivity was equivalent to just one of today's broadband connections.

It quickly became clear that computing power at CERN, even taking Moore's Law into account, would be significantly less than that required to analyse LHC data. The solution, it transpired during the 1990s, was to turn to "high-throughput computing" where the focus is not on shifting data as quickly as possible from A to B but rather from shifting as much information as possible between those two points.

High-performance computing is ideal for particle physics because the data produced in the millions of proton-proton collisions are all independent of one another - and can therefore be handled independently. So, rather than using a massive all-in-one mainframe supercomputer to analyse the results, the data can be sent to separate computers, all connected via a network.

From here sprung the LHC Grid. The Grid, which was officially inaugurated last month, is a tiered structure centred on CERN (Tier-0), which is connected by superfast fibre links to 11 Tier-1 centres at places like the Rutherford Appleton Laboratory (RAL) in the UK and Fermilab in the US. More than one CD's worth of data (about 700 MB) can be sent down these fibres to each of the Tier-1 centres every second.

Tier 1 centres then feed down to another 250 regional Tier-2 centres that are in turn accessed by individual researchers through university computer clusters and desktops and laptops (Tier-3).

As Andreas Hirstius writes, "The LHC challenge presented to CERN's computer scientists was as big as the challenges to its engineers and physicists. The computer scientists managed to develop a computing infrastructure that can handle huge amounts of data, thereby fulfilling all of the physicists' requirements and in some cases even going beyond them."

Source: Institute of Physics

Explore further: Researchers find first direct evidence of 'spin symmetry' in atoms

add to favorites email to friend print save as pdf

Related Stories

CERN prepares its long-term future

Feb 06, 2014

Particle physics takes the long-term view. Originally conceived in the 1980s, the LHC took another 25 years to come into being. This accelerator, which is unlike any other, is just at the start of a programme ...

Recommended for you

Water window imaging opportunity

10 hours ago

Ever heard of the water window? It consists of radiations in the 3.3 to 4.4 nanometre range, which are not absorbed by the water in biological tissues. New theoretical findings show that it is possible to ...

User comments : 5

Adjust slider to filter visible comments by rank

Display comments: newest first

kerry
4.8 / 5 (4) Nov 03, 2008
Awesome! It's great to see scientists and engineers from all different fields coming together to work towards a single goal. This makes me happy =)
physpuppy
5 / 5 (1) Nov 03, 2008
Science has become very inter-disciplinary. While it is possible to study your own little niche and stay there, to be successful you need to branch out (or collaborate with other fields). Some of the most successful scientists are those who are excellent in their field but look at other fields for ideas.
QubitTamer
5 / 5 (3) Nov 03, 2008
Sounds like a very smart plan and a very cool network they built.

Too bad the whole place will be destroyed soon by strangelets or stable quantum black holes that don't evaporate via Hawking radiation.

I am kidding... I hope!
tkjtkj
2 / 5 (1) Nov 04, 2008
This story is a year old.
It is , no doubt,even older for
the people working on the issue.

weewilly
not rated yet Nov 04, 2008
As a tree reaches up and out to intertwine with the atmosphere and it's root system grows a strong anchor into the earth so will it be with the sciences, growing, helping, supporting one another.