Team creates Milky Way structure simulations

November 18, 2014 by Katie Elyce Jones, Oak Ridge National Laboratory
Milky Way galaxy simulation developed by a team of researchers who demonstrated scalability of their Bonsai code to nearly 25 petaflops on Oak Ridge National Laboratory’s Titan supercomputer, resulting in a Gordon Bell Prize finalist nomination. Credit: SURFsara, J. Bédorf, and NVIDIA.

If you took a photograph of the Milky Way galaxy today from a distance, the photo would show a spiral galaxy with a bright, central bar (sometimes called a bulge) of dense star populations. The Sun—very difficult to see in your photo—would be located outside this bar near one of the spiral arms composed of stars and interstellar dust. Beyond the visible galaxy would be a dark matter halo—invisible to your camera but important nonetheless because it keeps everything together by dragging down the rotational velocity of the bar and spiral arms.

Now, if you wanted to go back in time and take a video of the Milky Way forming, you could go back 10 billion years, but many of the galaxy's prominent features would not be recognizable. You would have to wait about 5 billion years to witness the formation of the Earth's solar system. By this point, 4.6 billion years ago, the galaxy looks almost like it does today.

"The grand structure of the galaxy has emerged from the self-organization of the stellar distribution over the past 10 billion years to eventually look like the Milky Way in the photo," said Simon Portegies Zwart of Leiden Observatory in the Netherlands.

This is the timeline a team of researchers from the Netherlands and Japan, including Portegies Zwart, are seeing emerge when they use supercomputers to simulate the Milky Way galaxy's evolution. Using a code developed for GPU supercomputing architectures—including that of the Cray XK7 Titan located at the Department of Energy's Oak Ridge National Laboratory—the team's simulations have earned acceptance as a Gordon Bell Prize finalist. The prize recognizes outstanding achievement in high-performance computing and will be presented by the Association for Computing Machinery at SC14 on November 20.  

"We don't really know how the structure of the galaxy came about," Portegies Zwart said. "What we realized is we can use the positions, velocities, and masses of stars in three-dimensional space to allow the structure to emerge out of the self-gravity of the system."

The challenge of computing galactic structure on a star-by-star basis is, as you might imagine, the sheer number of stars in the Milky Way—at least 100 billion. Therefore, the team needed at least a 100 billion-particle simulation to connect all the dots. Before the development of the team's code, known as Bonsai, the largest galaxy simulation topped out around 100 million—not billion—particles.

The team tested an early version of Bonsai on the Oak Ridge Leadership Computing Facility's Titan, the second-most-powerful supercomputer in the world, to improve scalability in the code. After scaling Bonsai to almost half of Titan's GPU nodes, the team ran Bonsai on the Piz Daint supercomputer at the Swiss National Supercomputing Centre and simulated galaxy formation over 6 billion years with 51 million particles representing the forces of stars and . After a successful Piz Daint run, the team returned to Titan to maximize the code's parallelism.

The Bonsai code demonstrated scalability on 18,600 Titan nodes (96% of the machine's GPU nodes), which would enable an 8 million-year, 242 billion-particle Milky Way simulation. Bonsai achieved nearly 25 petaflops of sustained single-precision, floating point performance on Titan. Single-precision floating-point operations use less memory by representing numbers using 32 bits, whereas double-precision operations represent more precise numbers at the expense of using 64 bits.

"With graduate student Jeroen Bédorf, we started by writing single code for GPUs and deliberately never wrote code on CPUs because we wanted the entire code to run on GPUs to exploit their parallelism," Portegies Zwart said. "The host CPUs are only used to streamline the communication between the nodes and the GPUs. In this way, we can fully optimize the use of GPUs for number crunching and the much slower CPUs to minimize the communication overhead."

Another feature of Bonsai that makes 242 billion-particle simulations feasible is the use of a hierarchal tree-code that eliminates direct gravitational force calculations between each particle (or star) and all the other billions of particles by organizing them into octants that prioritize particle interactions.

The team aims to compare simulation results to new observations coming from the European Space Agency's Gaia satellite that launched last year. The Gaia mission is currently cataloguing star measurements—including distances, velocities, and stellar type—of one billion Milky Way stars.

"One percent of the particles, or stars, in our simulated galaxy should match Gaia data," Portegies Zwart said.

Gaia will also provide data on stars farther than Earth's Solar Neighborhood, or only stars within tens of light-years. When compared with Bonsai simulations, these new observations will help researchers better understand larger galaxy dynamics, such as the interaction of the bar and , in addition to local dynamics taking place around the Solar Neighborhood.

Because the galaxy is a big place with many research teams dedicated to understanding it, Portegies Zwart's team plans to make simulation data and source code resulting from Bonsai projects available to the research community.

Explore further: Video: Guide to our Galaxy

Related Stories

Video: Guide to our Galaxy

November 22, 2013

This virtual journey shows the different components that make up our home galaxy, the Milky Way, which contains about a hundred billion stars.

Image: Galactic wheel of life shines in infrared

October 24, 2014

It might look like a spoked wheel or even a "Chakram" weapon wielded by warriors like "Xena," from the fictional TV show, but this ringed galaxy is actually a vast place of stellar life. A newly released image from NASA's ...

Image: Our flocculent neighbour, the spiral galaxy M33

July 28, 2014

The spiral galaxy M33, also known as the Triangulum Galaxy, is one of our closest cosmic neighbours, just three million light-years away. Home to some forty billion stars, it is the third largest in the Local Group of galaxies ...

Gaia mapping the stars of the Milky Way

June 28, 2013

ESA's billion-star surveyor, Gaia, has completed final preparations in Europe and is ready to depart for its launch site in French Guiana, set to embark on a five-year mission to map the stars with unprecedented precision.

Pinwheeling across the sky

June 11, 2013

( —The face-on Pinwheel spiral galaxy is seen at ultraviolet wavelengths in this image taken by ESA's XMM-Newton space telescope.

Recommended for you

Nanoscale Lamb wave-driven motors in nonliquid environments

March 19, 2019

Light driven movement is challenging in nonliquid environments as micro-sized objects can experience strong dry adhesion to contact surfaces and resist movement. In a recent study, Jinsheng Lu and co-workers at the College ...

OSIRIS-REx reveals asteroid Bennu has big surprises

March 19, 2019

A NASA spacecraft that will return a sample of a near-Earth asteroid named Bennu to Earth in 2023 made the first-ever close-up observations of particle plumes erupting from an asteroid's surface. Bennu also revealed itself ...

The powerful meteor that no one saw (except satellites)

March 19, 2019

At precisely 11:48 am on December 18, 2018, a large space rock heading straight for Earth at a speed of 19 miles per second exploded into a vast ball of fire as it entered the atmosphere, 15.9 miles above the Bering Sea.

1 comment

Adjust slider to filter visible comments by rank

Display comments: newest first

not rated yet Nov 18, 2014
And they did it all upside-down. As a joke? Or for their Southern Hemisphere friends.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.