Pawsey powerhouse supercomputer crunches pre-SKA data torrent

Jul 23, 2013
Antenna tiles in the core region of the Murchison Widefield Array. Credit: ICRAR 

High-performance computing specialists from Perth's International Centre for Radio Astronomy Research (ICRAR) today became the first users of one of Australia's leading supercomputing facilities – the Pawsey Centre – ahead of its official opening later this year.

The recent of the Murchison Widefield Array (MWA) – a radio telescope based in Western Australia's Mid West – marked the start of an impressive flow of that will be stored in the iVEC-managed Pawsey Centre in Kensington for later use by researchers around the world.

"We now have more than 400 per second of MWA data streaming along the National Broadband Network from the desert 800km away," said Professor Andreas Wicenec, from The University of Western Australia node of ICRAR.

The Murchison Widefield Array is the first Square Kilometre Array precursor to enter full operations, generating a vast torrent of information that needs to be stored for later retrieval by researchers.

"To store the Big Data the MWA produces, you'd need almost three 1TB hard drives every two hours," said Prof Wicenec. "The technical challenge isn't just in saving the observations but how you then distribute them to astronomers from the MWA team in far-flung places so they can start using it."

There are currently two links between the data stores in Perth and MWA researchers at the Massachusetts Institute of Technology (MIT) in the United States and the Victoria University of Wellington in New Zealand. A future link to India – another MWA partner – will also be created.

"Not everyone needs all of the MWA data," said Professor Wicenec. "For example, MIT researchers are interested in the early Universe so we use filtering techniques to control what data is copied from the Pawsey Centre archive to the MIT machines. So far, more than 150TB of data has been transferred automatically to the MIT store, with a stream of up to 4TB a day increasing that value."

Professor Wicenec said the MWA is producing so much information that it would be impossible to manually decide where to send what, which is where a sophisticated archiving system – the open-source Next Generation Archive System (NGAS) – comes in.

"Controlling data for a widely distributed user group on this scale is a challenge that's being faced more and more frequently in science and other fields, but nothing suitable existed that could solve this problem for us," said UWA Associate Professor Chen Wu.

NGAS was initially developed by Professor Wicenec while he was at the European Southern Observatory and later modified by the ICRAR team to meet the MWA data challenge.

Associate Professor Wu said NGAS is very advanced – it doesn't matter where data is stored, you simply ask the system for what you want and it either provides it from the local store or retrieves it from the full archive back in Perth through a highly efficient dataflow management system.

About half of all MWA computing occurs on site in the Murchison, where signals from radio telescope antennas are combined and processed in a powerful system of computers called a correlator. What's left to do in Perth is produce images, and manage storage and distribution by the archive system so MWA astronomers can analyse the collected data.

Optical fibre links the Murchison Radio-astronomy Observatory (MRO) – where the MWA is situated – to the Pawsey Centre in Perth. Data travels down a dedicated 10 Gigabit per second connection between the MRO and Geraldton, and the trip to Perth is completed on Australia's new high-speed National Broadband Network.

The MWA will store about 3 Petabytes (3000TB) at the Pawsey Centre each year, equivalent to the MWA observing about a quarter of the time. Another section of the Pawsey Centre will be a supercomputing facility that includes computing for Australia's other SKA precursor, CSIRO's Australian Square Kilometre Array Pathfinder (ASKAP), and projects from geoscience and other computationally intensive fields.

"We're really impressed with iVEC and the staff from Cray and SGI," said Prof Wicenec. "We've been pushing to have resources ready as soon as possible so we could take advantage of Pawsey's capabilities for the MWA. They have provided us with very early access to the facility, which has let us interact directly with the experts and optimise the integration and setup of complex hardware and software systems."

"We're very grateful for the support from everyone in the team."

Explore further: Can astronomy explain the biblical Star of Bethlehem?

Provided by International Centre for Radio Astronomy Research

5 /5 (4 votes)
add to favorites email to friend print save as pdf

Related Stories

A new way to see space – even its junk

Jul 18, 2013

When a large radio telescope in the Australian outback was unveiled last week its improved sensitivity was immediately apparent. It transformed images of supernova remnants taken with last year's equipment ...

New radio telescope could save world billions

Nov 30, 2012

A small pocket of Western Australia's remote outback is set to become the eye on the sky and could potentially save the world billions of dollars. The Murchison Widefield Array (MWA) radio telescope, unveiled today, Friday ...

Recommended for you

Can astronomy explain the biblical Star of Bethlehem?

Dec 24, 2014

Bright stars top Christmas trees in Christian homes around much of the world. The faithful sing about the Star of Wonder that guided the wise men to a manger in the little town of Bethlehem, where Jesus was ...

Hubbles spies the beautiful galaxy IC 335

Dec 24, 2014

This new NASA/ESA Hubble Space Telescope image shows the galaxy IC 335 in front of a backdrop of distant galaxies. IC 335 is part of a galaxy group containing three other galaxies, and located in the Fornax ...

Image: Multicoloured view of supernova remnant

Dec 22, 2014

Most celestial events unfold over thousands of years or more, making it impossible to follow their evolution on human timescales. Supernovas are notable exceptions, the powerful stellar explosions that make ...

Ultra-luminous X-ray sources in starburst galaxies

Dec 22, 2014

Ultra-luminous X-ray sources (ULXs) are point sources in the sky that are so bright in X-rays that each emits more radiation than a million suns emit at all wavelengths. ULXs are rare. Most galaxies (including ...

When a bright light fades

Dec 22, 2014

Astronomer Charles Telesco is primarily interested in the creation of planets and stars. So, when the University of Florida's giant telescope was pointed at a star undergoing a magnificent and explosive death, ...

User comments : 0

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.