Oak Ridge to acquire next generation supercomputer

The U.S. Department of Energy's Oak Ridge Leadership Computing Facility (OLCF) has signed a contract with IBM to bring a next-generation supercomputer to Oak Ridge National Laboratory (ORNL). The OLCF's new hybrid CPU/GPU computing system, Summit, will be delivered in 2017.

Summit will provide at least five times the performance of Titan, the OLCF's current leadership system, for a wide range of scientific applications. The hybrid system will support DOE's Office of Science in its broad science and energy mission, addressing the most challenging and impactful science problems for government, academia, and industry.

Among the goals researchers will pursue by applying Summit's capabilities in diverse scientific arenas:

• Combustion science: Creating a fundamental understanding of combustion to increase efficiency by 25-50 percent and lower emissions from internal combustion engines using advanced fuels and new, low-temperature combustion concepts.
• Climate change science: Understanding the dynamic ecological and chemical evolution of the climate system with uncertainty quantification of impacts on regional and decadal scales.
• Energy storage: Gaining a fundamental understanding of chemical reaction processes at the atomic and molecular level required for predictive design of new materials for energy storage and predictive engineering of safe, large-format, durable, rechargeable batteries.
• Nuclear power: Enabling reactor-scale simulations to allow safe, increased nuclear fuel burn times, power upgrades, and reactor lifetime extensions, and in doing so reduce the volume of spent fuel.

The system's vendor, IBM, and major component suppliers, NVIDIA and Mellanox, are all participating in an open architecture technology collaboration known as the OpenPOWER Foundation.

"Summit builds on the hybrid multi-core architecture that the OLCF successfully pioneered with Titan," says Buddy Bland, the director of the Summit project at the OLCF. "The large, powerful nodes allow applications to achieve very high performance without having to scale to hundreds of thousands of Message Passing Interface (MPI) tasks. The combination of very large memory per node and the powerful IBM POWER and NVIDIA processors provides an ideal platform for data analysis as well as computation."

Summit will feature more than 3,400 nodes, each with:

• Multiple IBM POWER9 processors and multiple NVIDIA Volta GPUs
• CPUs and GPUs completely connected with high speed NVLink
• Large coherent memory: more than 512 GB of combined DDR4 and high bandwidth memory - all directly addressable from the CPUs and GPUs
• An additional 800 GB of NVRAM, which can be configured as either a burst buffer or as extended memory
• over 40 TF peak performance

Summit will have a dual-rail Mellanox EDR interconnect configured as a full, non-blocking fat-tree. The file system will be a GPFS Storage Server system with 1TB/s I/O bandwidth and 120 PB of disk capacity.

System software will include: IBM XL, NVIDIA, and PGI environments supporting OpenMP and OpenACC programming, IBM HPC software including Linux, Platform Computing LSF scheduler, resource manager, system management, and GPFS parallel file system.

"High-performance computing has become a key part of technology advancement and scientific discovery," says Jim Hack, Director of the National Center for Computational Sciences at ORNL. "The OLCF is being used to solve some of the most demanding and important science problems in the world. Summit will allow us to continue in this mission through the end of the decade."