Science powerhouses unite to help search for gravitational waves

Science powerhouses unite to help search for gravitational waves
Scientists from Caltech, Syracuse University and Abilene Christian University worked with computing experts at the Texas Advanced Computing Center to develop and test new methods for transmitting and analyzing data with an eye towards Advanced LIGO's future operations. Credit: LIGO

Scientists since Albert Einstein have believed that when major galactic events like supernova explosions or black hole mergers occur in the universe, they leave a trace. That trace, it is believed, takes the form of gravitational waves—ripples in the curvature of space-time that travel outward from the source.

For over a decade, scientists and engineers have engaged in one of the most ambitious research efforts ever undertaken: to design, build and operate the Laser Interferometer Gravitational Observatory (LIGO) in order to identify signs of gravitational waves.

From 2002 to 2010, hundreds of scientists worked together to bring the experiment to life. It required two gravitational wave observatories, located 1,865 miles apart and working in unison: the LIGO Livingston Observatory in Livingston, La., and the LIGO Hanford Observatory, located near Richland, Wash.

LIGO was developed and is managed by the California Institute of Technology and MIT, and is funded by the National Science Foundation (NSF).

Since gravitational waves are expected to travel at the speed of light, the distance between the sites corresponds to a difference in arrival times of about ten milliseconds. By triangulating the signals, this small difference in detection can determine the source of the wave in the sky.

"Gravitational waves offer an unprecedented opportunity to see the universe from a new perspective, providing access to astrophysical insights that are available in no other way," said Stuart Anderson, a Caltech research manager for LIGO.

To date, no waves have been detected. Yet most astronomers believe they're still out there, waiting to be discovered with use of the right tools. For that reason, the National Science Foundation, the U.K. Science and Technology Facilities Council, the German Max Planck Society and the Australian Research Council have supported the creation of a follow-up to LIGO called Advanced LIGO.

Advanced LIGO replaces LIGO's original detectors with ones ten times more sensitive. It is expected to go online in 2015.

"Advanced LIGO's goal is to pioneer a new field of gravitational-wave astronomy, and its planned sensitivity will allow this very significant step beyond first detections," Anderson said. "The massive, relativistic constituents, causing violent perturbations of space-time itself, will be revealed through this unique probe."

Big Data and detection

In addition to all of the astronomical considerations that must be attended to in a project like this—the placement of the observatories, the sensitivity of the detectors, the modeling of the expected signals—Advanced LIGO is also a "Big Data" problem.

The observatories take in huge volumes of data that then must be analyzed to determine their meaning. In the case of a positive identification of a gravitational wave candidate, the analysis must happen quickly, so the location of the waves can be identified and used for further study.

Science powerhouses unite to help search for gravitational waves
Advanced LIGO will take in huge volumes of data that must be analyzed to determine their meaning. Researchers estimated that ALIGO would generate more than 1 petabyte of data a year, the equivalent of 13.3 years of high-definition video. Credit: MIT/CalTech LIGO

Developing an offline system for binary neutron star and black hole detection was one of the many goals of the Advanced LIGO project, Anderson said. "We wanted to dig deeply into signals, since we have a good idea of what those signals are."

Since gravitational waves are still in their theoretical infancy, scientists require additional information to learn about their existence. Combining observations of electromagnetic radiation, neutrinos or —a process known as multi-messenger astronomy—provides complementary information, but it makes a messy data problem even messier.

When they were planning Advanced LIGO, the researchers realized they needed more computing and data processing power than they had used previously. But how much more did they need? And where would it come from?

By moving to more a sensitive detector, they had to increase their computing needs by more than an order of magnitude. They estimated that ALIGO would generate more than 1 petabyte of data a year, the equivalent of 13.3 years of high-definition video.

Discussing the matter with program officers at NSF, they decided to see if the Extreme Science and Engineering Discovery Environment (XSEDE) could help.

XSEDE, another NSF-supported project, oversees the nation's network of supercomputers and digital resources. These resources allow scientists to solve otherwise impossible computing problems and pursue really big science. Perhaps it could do the same for Advanced LIGO.

Scientists from Caltech, Syracuse University and Abilene Christian University worked with computing experts at the Texas Advanced Computing Center (TACC) to develop and test new methods for transmitting and analyzing data with an eye towards Advanced LIGO's future operations.

To achieve their goal, the team overlaid LIGO's computing environment on top of the Stampede supercomputer at TACC, one of the most powerful in the world. They then used a tool called Condor to break their large computing problems into smaller parts that could be distributed and solved on the supercomputer's individual nodes. (A node consists of a number of processors connected together and to the larger system.)

Miron Livney's High Throughput Condor group at the University of Wisconsin-Madison provided critical support (including software updates) to allow this computational challenge to work, and the members of the Advanced LIGO team were given priority access to the supercomputer so their jobs could run efficiently with few interruptions.

At first, they used a data analysis code from the initial LIGO for the experiment, since it was easy to validate. However over time, they began using the new Advanced LIGO version of the analysis code. Through detailed benchmarking, they were able to verify the reliability of the process and offer a preliminary sense of how much computing and data-crunching capability they needed to transmit, store and analyze data from the observatories for use by the astronomy community.

Science powerhouses unite to help search for gravitational waves
LIGO, the Laser Interferometer Gravitational-Wave Observatory, is a large-scale physics experiment that aims to directly detect gravitational waves. Here is the LIGO Livingston Observatory in Louisiana. Credit: MIT/CalTech LIGO
"The XSEDE and TACC teams were extremely helpful in collaborating on a successful port of a large scale LIGO data analysis pipeline to the Stampede system—all the way from network transfers of instrument data, through large scale number crunching, to generating and presenting web accessible scientific results," Anderson said.

"The solution we implemented allowed individual LIGO data analysts to use their existing computer identities and software tools to quickly and efficiently leverage the Stampede computing resources. I am very excited by the possibility of using XSEDE resources to maximize the scientific reach of the Advanced LIGO experiment."

Accelerating the Advanced LIGO code

Not only were the researchers able to prove they could compute on a big national supercomputer. In the process of collaborating with experts at TACC, they ended up making the Advanced LIGO software four times more efficient, ultimately saving money and time.

"We were able to reproduce the answers from previous experiments and also took advantage of our local experts to speed up the LIGO codes," said Yaakoub El Khamra, a research engineer at TACC who collaborated on the project. "The effort was an unmitigated rousing success."

According to Peter Couvares, a computer scientist from Syracuse University who worked on the project, the experience showed LIGO researchers that large high-performance computing systems like Stampede can work as well as the more familiar high-throughput computing systems that use a number of clusters distributed across LIGO Scientific Collaboration institutions.

"Given that our foremost computing challenge has always been to enable scientists to automatically and reliably execute large complex workflows consisting of millions of jobs and big data, NSF-supported tools like Condor and Pegasus have been invaluable," Couvares said. "We're increasingly realizing the value of HPC expertise at places like TACC to optimize our codes to make efficient use of shared XSEDE resources like Stampede."

Borne out in tests run over the last few months, the results proved that using the nation's supercomputers for gravitational wave analyses is feasible. In fact, in the end, the team was able to run more than 10,000 analysis jobs simultaneously on Stampede.

They presented their results at the HTCondor Week 2014 workshop in April.

"TACC staff not only got the LIGO team up and running on Stampede, they helped make the LIGO code substantially more efficient," said Dan Stanzione, TACC director. "Over the course of a year of computing with advanced LIGO, that could equate to millions of dollars in savings, or a huge increase in the amount of processing we can do on the same budget. This shows the value of software engineering being interjected into the process."


Explore further

Tabletop experiment could detect gravitational waves

Citation: Science powerhouses unite to help search for gravitational waves (2014, December 5) retrieved 23 May 2019 from https://phys.org/news/2014-12-science-powerhouses-gravitational.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.
0 shares

Feedback to editors

User comments

Dec 05, 2014
The notion that a change in the structure of mass occupying a location in space will be detectable by a gravitational fluctuation in our reference frame is yet another bastard child of theoretical math. The gravity fields of two merging objects are indistinguishable on the cosmic scale.
Your use of the term "reference frame" makes it sound like you know what you're talking about, but the 'notion' part of your objection indicates that you lack understanding regarding the nature of gravitational waves. The waves are generated by an acceleration of mass. Your objection about 'gravity fields' is a first-order complaint, but the gravitational waves are a higher-order effect—e.g., the radiation is quadrupolar and the polarizations are at a 45-degree angle instead of 90 (as with light).

Dec 06, 2014
Why are we looking at space the same way we look at water? I think the premise that any sort of mass displacement be it an exploding star or some other sort of large scale change in a celestial massive body will produce long range g waves which will be detectable via an Interferometer is nothing more than a guess. A SWAG!

Dec 06, 2014
Any possible g wave would fall off sharply from its source via 1/r sup 2 for the single initial pulse, it wouldn't be like a spring unless the mass is undulating someway...this means that by the distance of a light year away from such an event, for example, that pulse and any g wave produced by it would have been stretched out …smeared out into space-time, regardless of inertial frames of observation, unless of course the "graviton" which is suppose to be massless could hold the energy of the initial pulse but then this would pose a problem as the cosmos would be full of detectable g waves from many and all large scale event sources regardless of the degree or order of the Qs of detection.

What a waste of $$$. Some dinosaur professor(s) with his or her head in the theoretical sand is most likely going to coast to retirement on the funding for this project.

Dec 06, 2014
This is a good example of someone who thinks math has made something real. And just posted 4 facts about an as of yet fictional construct.

There are a variety of ways to test general relativity. "Both in the weak field limit (as in our solar system) and with the stronger fields present in systems of binary pulsars the predictions of general relativity have been extremely well tested locally."

Your objections aren't supported with maths or experiments, and typify the level of ignorance required to practice pseudoscience.

Maths and experimentation within the context of the scientific method are the sharpest tools I know of for assessing the nature of reality. What's your excuse, reset?

Dec 06, 2014
This comment has been removed by a moderator.

Dec 06, 2014
""Advanced LIGO's goal is to pioneer a new field of gravitational-wave astronomy, and its planned sensitivity will allow this very significant step beyond first detections," "

If the ALIGO comes up empty will it end the search, or will the goal posts be moved.....?

om out

Dec 06, 2014
If the ALIGO comes up empty will it end the search, or will the goal posts be moved.....?

In keeping with your analogy, if ALIGO and other efforts currently underway don't detect GWs, the goal posts will have to be modified instead of moved, into hoops or bored holes in the ground or pentagonal rubber plates or …

In light of the successes and degree to which both quantum mechanics and general relativity have been tested, the doubts and complaints regarding GWs expressed by some here are reminiscent of the objections to searching for the Higgs boson.

Dec 07, 2014
This comment has been removed by a moderator.

Dec 07, 2014
The goals are already shifting, now the physicists propose to search for gravitational waves at THz scale, which can have hardly something common with gravitational waves searched at LIGO. Can you imagine some massive object vibrating at the THz frequency? Me not

The portion of the spectrum they're searching is 1 – 1000 MHz, so that's only up to 1 THz. And the commonality is GWs so that's not a "shift" at all—the difference is only in the candidates for the source of the GWs.
In another words, the physicists will always try to detect something, no matter of its actual properties, because the research industry cannot be stopped and the salary must be payed

Stop playing devil's advocate, iZeph. Who would want to stop the research industry? It's the bomb making industry that needs to be stopped.

Dec 10, 2014
This comment has been removed by a moderator.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more