LIGO supercomputer upgrade will speed up groundbreaking astrophysics research
In 2016, an international team of scientists found definitive evidence—tiny ripples in space known as gravitational waves—to support one of the last remaining untested predictions of Einstein's theory of general relativity. The team used the Laser Interferometer Gravitational-Wave Observatory (LIGO), which has since made several gravitational wave discoveries. Each discovery was possible in part because of a global network of supercomputer clusters, one of which is housed at Penn State. Researchers use this network, known as the LIGO Data Grid, to analyze the gravitational wave data.
Penn State recently invested in an upgrade to its portion of the data grid that will roughly quadruple the cluster's capacity for conducting cutting-edge astronomy and astrophysics research. The new cluster, 192 servers working in tandem, is administered by the Institute for CyberScience (ICS). Bangalore Sathyaprakash, professor of astronomy and astrophysics and Elsbach Professor of Physics; and Chad Hanna, associate professor of physics and astronomy and astrophysics, and ICS co-hired faculty member, are the primary researchers who will be using the new system with their research team and collaborators.
Speeding up faculty and student research
"At Penn State we're involved in all aspects of gravitational wave astronomy, which we use to learn about the universe," said Sathyaprakash. "Until the discovery of gravitational waves, the only way we could observe the universe was using light, radio waves or gamma rays, which all belong to the electromagnetic spectrum. Gravitational waves allow us to create a complementary picture of the universe and reveal processes and phenomena that might not otherwise be revealed through electromagnetic observation."
The new cluster will vastly increase the speed at which researchers can complete analysis, according to Chad Hanna. He and colleagues recently finished the first study that used data housed on Penn State's LIGO cluster. The team designed an experiment to quantify the number of binary black holes in the universe that have less mass than the Sun, which may have implications for the amount of dark matter in the universe.
"Our first study that solely used the Penn State LIGO cluster took 12 weeks," said Hanna. "If we were to complete that same investigation on the upgraded cluster today, it would only take three weeks."
The upgrade boosts the cluster from 1,152 compute cores to 4,608 cores, which will allow more researchers to use the system simultaneously. For reference, this is roughly equivalent to more than 1,000 desktop computers working in unison.
"I'm most excited about the extra machines," said Ryan Magee, graduate student in physics. "It allows for multiple analyses to run at once without much bottlenecking."
Magee plans to use the cluster to search for sub-solar mass compact objects in the universe, he said, because "they are not produced by stellar mechanisms, so it would be a hint of new physics."
Researchers at all levels will be using the new resource, including undergraduate students like Phoebe McClincy, a sophomore studying astronomy and astrophysics, and a Millennium Scholar. McClincy was first exposed to gravitational wave research as a high school student attending a Penn State summer camp led by Hanna.
"During that summer camp I was actually afforded the opportunity to visit the cluster, and I remember thinking it was really cool and fascinating to see the other side of the computer," said McClincy, now a member of Hanna's research team. "I've always thought tech like this is amazing, so I can't wait to see what can be done now that it will be even more advanced."
Building capacity for future LIGO discoveries
The first iteration of LIGO's observatories collected data from 2002 to 2010 but did not detect any gravitational waves. Upgrading the observatories to their current state, known as Advanced LIGO, greatly increased their detection capabilities, and, as a result, the system has detected six gravitational wave events since 2016.
Sathyaprakash said there are plans to continue enhancing the detection capabilities of gravitational wave observatories, which will pose both opportunities and challenges for researchers.
"When advanced LIGO reaches its design sensitivity, we will observe binary black hole collisions as far as tens of billions of light years and binary neutron star mergers billions of light years away. With the construction in the 2030s of new detectors that are 10 times more sensitive than the current ones, we will be able to observe the entire universe in gravitational waves for black holes and most of the universe for neutron stars," he said.
Coming with that will be challenges in collecting, storing and analyzing huge amounts of data. It has taken between one and three months to analyze each gravitational wave detected to date.
"With advanced LIGO we expect to observe one event every day or every other day, this will offer a huge computational challenge, and so every bit helps," he said. "With this new LIGO cluster, what we've done is to secure enough resources to be completely independent in doing our analyses. ICS and Penn State are enabling this challenging science. Without this new cluster, we would be very severely hampered from doing the science that we want to do."