Radio astronomy backed by big data projects

April 21, 2015
Radio astronomy backed by big data projects
This set up means the MWA can produce data at a rate almost 8,700 times faster than the average Australian internet connection can download. Credit: WA Department of Commerce

As the leading edge of the Square Kilometre Array (SKA) project, the Murchison Widefield Array (MWA) radio telescope is at the forefront of the big data challenges facing radio astronomy, presenting and solving issues that will help research and industry across the world for years to come.

The MWA is a Curtin University-led project consisting of antennas in Murchison Shire (800 km north of Perth), data connectivity to Perth, and a large-scale data archive based at the Pawsey Supercomputing Centre.

The MWA is designed to look back in time, to study the formation of the first stars and galaxies in the universe, less than one billion years after the Big Bang (the universe is 13.8 billion years old).

It is a smaller scale prototype that is testing the types of challenges and technologies that will face the SKA proper, as well as producing scientifically valuable results in its own right.

After approximately 18 months of operations, the MWA has collected over 4 petabytes of data, which is enough data to fill 8 million average computer hard drives.

It is therefore one of the first astronomy facilities to enter the era of .

The systems in the Murchison produce approximately 60 gigabits per second data streams (the average Australian internet is 6.9Mbps) that are processed in real-time on site, using GPU-based signal processing as the first stage in a hierarchical data processing strategy.

This set up means the MWA can produce data at a rate almost 8,700 times faster than the average Australian internet connection can download.

The output data streams from this processing are approximately an order of magnitude smaller than the input and are transmitted over a dedicated 10 Gbps optical fibre network to the Pawsey Supercomputing Centre in Perth.

Radio astronomy backed by big data projects
The MWA is designed to look back in time, to study the formation of the first stars and galaxies in the universe, less than one billion years after the Big Bang (the universe is 13.8 billion years old). Credit: Murchison Widefield Array

At the centre, the archiving and curation of accumulated datasets represents the next big data challenge.

Ensuring the integrity and discoverability of the archive requires significant effort for such a large-scale dataset.

Finally, the Pawsey Supercomputing Centre allows scientists around the world to access MWA data from the archive and process them.

This is the data analytics aspect of big data, where scientists invent new algorithms to manipulate data into images of the sky, catalogues of objects, and extract highly abstracted representations of the data, searching for subtle statistical signals within.

This is similar to the analytics problems that the commercial world is familiar with, discovering the information due to a small handful of important variables in amongst a very large number of possible variables, across a large volume of data.

The MWA is one of three precursor telescopes for the much larger Square Kilometre Array (SKA), approximately half of which will be built in the Murchison within a decade.

The MWA is the first to be fully operational, placing Western Australian astronomers and engineers at the forefront of international SKA developments and engaging with industry partners such as CISCO, Woodside and SIRCA.

Explore further: Pawsey powerhouse supercomputer crunches pre-SKA data torrent

Related Stories

Telescope to track space junk using youth radio station

November 29, 2013

A combination of pop songs, talkback radio and cutting-edge science has enabled Australian astronomers to identify a way to prevent catastrophic, multi-billion dollar space junk collisions, a new study has revealed.

Recommended for you

Freeze-dried food and 1 bathroom: 6 simulate Mars in dome

January 20, 2017

Crammed into a dome with one bathroom, six scientists will spend eight months munching on mostly freeze-dried foods—with a rare treat of Spam—and have only their small sleeping quarters to retreat to for solace.

Image: Wavemaker moon Daphnis

January 20, 2017

The wavemaker moon, Daphnis, is featured in this view, taken as NASA's Cassini spacecraft made one of its ring-grazing passes over the outer edges of Saturn's rings on Jan. 16, 2017. This is the closest view of the small ...

Video: A colorful 'landing' on Pluto

January 20, 2017

What would it be like to actually land on Pluto? This movie was made from more than 100 images taken by NASA's New Horizons spacecraft over six weeks of approach and close flyby in the summer of 2015. The video offers a trip ...

The evolution of massive galaxy clusters

January 20, 2017

Galaxy clusters have long been recognized as important laboratories for the study of galaxy formation and evolution. The advent of the new generation of millimeter and submillimeter wave survey telescopes, like the South ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.