Pawsey powerhouse supercomputer crunches pre-SKA data torrent

Jul 23, 2013
Antenna tiles in the core region of the Murchison Widefield Array. Credit: ICRAR 

High-performance computing specialists from Perth's International Centre for Radio Astronomy Research (ICRAR) today became the first users of one of Australia's leading supercomputing facilities – the Pawsey Centre – ahead of its official opening later this year.

The recent of the Murchison Widefield Array (MWA) – a radio telescope based in Western Australia's Mid West – marked the start of an impressive flow of that will be stored in the iVEC-managed Pawsey Centre in Kensington for later use by researchers around the world.

"We now have more than 400 per second of MWA data streaming along the National Broadband Network from the desert 800km away," said Professor Andreas Wicenec, from The University of Western Australia node of ICRAR.

The Murchison Widefield Array is the first Square Kilometre Array precursor to enter full operations, generating a vast torrent of information that needs to be stored for later retrieval by researchers.

"To store the Big Data the MWA produces, you'd need almost three 1TB hard drives every two hours," said Prof Wicenec. "The technical challenge isn't just in saving the observations but how you then distribute them to astronomers from the MWA team in far-flung places so they can start using it."

There are currently two links between the data stores in Perth and MWA researchers at the Massachusetts Institute of Technology (MIT) in the United States and the Victoria University of Wellington in New Zealand. A future link to India – another MWA partner – will also be created.

"Not everyone needs all of the MWA data," said Professor Wicenec. "For example, MIT researchers are interested in the early Universe so we use filtering techniques to control what data is copied from the Pawsey Centre archive to the MIT machines. So far, more than 150TB of data has been transferred automatically to the MIT store, with a stream of up to 4TB a day increasing that value."

Professor Wicenec said the MWA is producing so much information that it would be impossible to manually decide where to send what, which is where a sophisticated archiving system – the open-source Next Generation Archive System (NGAS) – comes in.

"Controlling data for a widely distributed user group on this scale is a challenge that's being faced more and more frequently in science and other fields, but nothing suitable existed that could solve this problem for us," said UWA Associate Professor Chen Wu.

NGAS was initially developed by Professor Wicenec while he was at the European Southern Observatory and later modified by the ICRAR team to meet the MWA data challenge.

Associate Professor Wu said NGAS is very advanced – it doesn't matter where data is stored, you simply ask the system for what you want and it either provides it from the local store or retrieves it from the full archive back in Perth through a highly efficient dataflow management system.

About half of all MWA computing occurs on site in the Murchison, where signals from radio telescope antennas are combined and processed in a powerful system of computers called a correlator. What's left to do in Perth is produce images, and manage storage and distribution by the archive system so MWA astronomers can analyse the collected data.

Optical fibre links the Murchison Radio-astronomy Observatory (MRO) – where the MWA is situated – to the Pawsey Centre in Perth. Data travels down a dedicated 10 Gigabit per second connection between the MRO and Geraldton, and the trip to Perth is completed on Australia's new high-speed National Broadband Network.

The MWA will store about 3 Petabytes (3000TB) at the Pawsey Centre each year, equivalent to the MWA observing about a quarter of the time. Another section of the Pawsey Centre will be a supercomputing facility that includes computing for Australia's other SKA precursor, CSIRO's Australian Square Kilometre Array Pathfinder (ASKAP), and projects from geoscience and other computationally intensive fields.

"We're really impressed with iVEC and the staff from Cray and SGI," said Prof Wicenec. "We've been pushing to have resources ready as soon as possible so we could take advantage of Pawsey's capabilities for the MWA. They have provided us with very early access to the facility, which has let us interact directly with the experts and optimise the integration and setup of complex hardware and software systems."

"We're very grateful for the support from everyone in the team."

Explore further: Murchison Widefield Array: Square Kilometre Array precursor debuts

Provided by International Centre for Radio Astronomy Research

5 /5 (4 votes)
add to favorites email to friend print save as pdf

Related Stories

A new way to see space – even its junk

Jul 18, 2013

When a large radio telescope in the Australian outback was unveiled last week its improved sensitivity was immediately apparent. It transformed images of supernova remnants taken with last year's equipment ...

New radio telescope could save world billions

Nov 30, 2012

A small pocket of Western Australia's remote outback is set to become the eye on the sky and could potentially save the world billions of dollars. The Murchison Widefield Array (MWA) radio telescope, unveiled today, Friday ...

Recommended for you

Quest for extraterrestrial life not over, experts say

Apr 18, 2014

The discovery of an Earth-sized planet in the "habitable" zone of a distant star, though exciting, is still a long way from pointing to the existence of extraterrestrial life, experts said Friday. ...

Continents may be a key feature of Super-Earths

Apr 18, 2014

Huge Earth-like planets that have both continents and oceans may be better at harboring extraterrestrial life than those that are water-only worlds. A new study gives hope for the possibility that many super-Earth ...

Exoplanets soon to gleam in the eye of NESSI

Apr 18, 2014

(Phys.org) —The New Mexico Exoplanet Spectroscopic Survey Instrument (NESSI) will soon get its first "taste" of exoplanets, helping astronomers decipher their chemical composition. Exoplanets are planets ...

User comments : 0

More news stories

Easter morning delivery for space station

Space station astronauts got a special Easter treat: a cargo ship full of supplies. The shipment arrived Sunday morning via the SpaceX company's Dragon cargo capsule.

Making graphene in your kitchen

Graphene has been touted as a wonder material—the world's thinnest substance, but super-strong. Now scientists say it is so easy to make you could produce some in your kitchen.