The US Department of Energy’s (DOE) Oak Ridge National Laboratory (ORNL) has launched Titan, a 20 petaflop (20,000 trillion calculations per second) supercomputer that will be 10 times more powerful than ORNL’s last world-leading system, Jaguar, while overcoming power and space limitations inherent in the previous generation of high-performance computers. Titan, supported by the DOE, will provide unprecedented computing power for research in energy, climate change, efficient engines, materials and other disciplines. (20 petaflops is on par with each of the world’s 7 billion people being able to carry out 3 million calculations per second, ORNL points out.)
The Cray XK7 system contains 18,688 nodes, with each holding a 16-core AMD Opteron 6274 processor and an NVIDIA Tesla K20 graphics processing unit (GPU) accelerator. Titan also has more than 700 terabytes of memory. The combination of central processing units, the traditional foundation of high-performance computers, and more recent GPUs will allow Titan to occupy the same space as its Jaguar predecessor while using only marginally more electricity.
One challenge in supercomputers today is power consumption. Combining GPUs and CPUs in a single system requires less power than CPUs alone and is a responsible move toward lowering our carbon footprint. Titan will provide unprecedented computing power for research in energy, climate change, materials and other disciplines to enable scientific leadership.—Jeff Nichols, associate laboratory director for computing and computational sciences
Because they handle hundreds of calculations simultaneously, GPUs can go through many more than CPUs in a given time. By relying on its 299,008 CPU cores to guide simulations and allowing its new NVIDIA GPUs to do the heavy lifting, Titan will enable researchers to run scientific calculations with greater speed and accuracy.
Titan will be open to select projects while ORNL and Cray work through the process for final system acceptance. The lion’s share of access to Titan in the coming year will come from the Department of Energy’s Innovative and Novel Computational Impact on Theory and Experiment program INCITE. Researchers have been preparing for Titan and its hybrid architecture for the past two years.
The Oak Ridge Leadership Computing Facility (OLCF) created the Center for Accelerated Application Readiness (CAAR), a collaboration among application developers, Titan’s manufacturer Cray, GPU manufacturer NVIDIA, and the OLCF’s scientific computing experts.
CAAR has been working for nearly 2 years to establish best practices for code-writers. The center is divided into five teams working with five of the OLCF’s most advanced and representative applications. Essentially these, and other potential applications, need to be able to keep the GPUs in Titan busy.
CAAR applications include the combustion code S3D; LSMS, which studies magnetic systems; LAMMPS, a bioenergy and molecular dynamics application; Denovo, which investigates nuclear reactors; and CAM-SE, a code that explores climate change.
Combustion. The S3D application models the underlying turbulent combustion of fuels in an internal combustion engine. Titan will allow researchers to model large-molecule hydrocarbon fuels such as the gasoline surrogate isooctane; commercially important oxygenated alcohols such as ethanol and butanol; and biofuel surrogates that blend methyl butanoate, methyl decanoate and n-heptane.
In particular, these simulations will enable us to understand the complexities associated with strong coupling between fuel chemistry and turbulence at low preignition temperatures. These complexities pose challenges, but also opportunities, as the strong sensitivities to both the fuel chemistry and to the fluid flows provide multiple control options which may lead to the design of a high-efficiency, low-emission, optimally combined engine-fuel system.—Jacqueline Chen of Sandia National Laboratories
Materials Science. The magnetic properties of materials hold the key to major advances in technology. The application WL-LSMS provides a nanoscale analysis of important materials such as steels, iron-nickel alloys and advanced permanent magnets that will help drive future electric motors and generators. Titan will allow researchers to improve the calculations of a material’s magnetic states as they vary by temperature.
Nuclear Energy. Nuclear researchers use the Denovo application to, among other things, model the behavior of neutrons in a nuclear power reactor. Titan will allow Denovo to simulate a fuel rod through one round of use in a reactor core in 13 hours; this job took 60 hours on the Jaguar system.
Climate Change. The Community Atmosphere Model-Spectral Element simulates long-term global climate. Improved atmospheric modeling under Titan will help researchers better understand future air quality as well as the effect of particles suspended in the air.
Using a grid of 14-kilometer cells, the new system will be able to simulate from one to five years per day of computing time, up from the three months or so that Jaguar was able to churn through in a day.