A new earth modeling system will support weather-scale resolution and use advanced computers to simulate aspects of Earth’s variability and anticipate decadal changes that will impact the US energy sector in coming years. After four years of development, the Energy Exascale Earth System Model (E3SM) is being released to the broader scientific community this month.

The E3SM project is supported by the Department of Energy’s Office of Science in the Biological and Environmental Research Office. The E3SM release will include model code and documentation, as well as output from an initial set of benchmark simulations.

The high-resolution E3SM earth system model simulates the strongest storms with surface winds exceeding 150 mph—hurricanes that leave cold wakes that are 2 to 4 degrees Celsius cooler than their surroundings. This simulation represents how sea surface temperature changes evolve as a hurricane (seen here approaching the US East Coast) moves across the Atlantic and how the resultant cold wake affects subsequent intensification of the next hurricane. Source: LLNL. Click to enlarge.

Over the past three years, E3SM Phase I developed the first version (v1) of the E3SM model. Phase II is expanding and extends this work by further exercising the low- and high-resolution versions of v1, adding available improvements and exercising the model with regional refinement over North America for v2, and significantly improving model performance, infrastructure, and Earth system capabilities in Phase II by developing v3-v4 (for use in a Phase III simulation campaign).

Three new model components are included in the E3SM v1 configuration: Model for Prediction Across Scales (MPAS) Ocean, MPAS Sea Ice, and MPAS Land Ice. Pictured here is global E3SM simulation showing eddy activity. Source: E3SM. Click to enlarge.

Phase II will continue to push the envelope in terms of model resolution beyond the current hydrostatic limit in the atmosphere. During this period, the coupled system will run efficiently on the most powerful DOE High Performance Computational Facility computers.

The E3SM project will soon convert to an “open development” project. This means that not only released version 1 code, but also the developing code, will be visible to those with an interest in the ongoing code evolution. This conversion to open code will allow collaboration with more groups and centers.

The Earth, with its myriad interactions of atmosphere, oceans, land and ice components, presents an extraordinarily complex system for investigation. Earth system simulation involves solving approximations of physical, chemical and biological governing equations on spatial grids at resolutions that are as fine in scale as computing resources will allow.

The goal of the E3SM project is to simulate reliably aspects of earth system variability and to project decadal changes that will critically impact the US energy sector in the near future. These critical factors include a) regional air/water temperatures, which can strain energy grids; b) water availability, which affects power plant operations; c) extreme water-cycle events (e.g. floods and droughts), which impact infrastructure and bio-energy; and d) sea-level rise and coastal flooding, which threaten coastal infrastructure.

The project is developing an earth system model (ESM) that has not been possible because of limitations in current computing technologies. Meeting this goal will require advances on three frontiers: 1) better resolving earth system processes through a strategic combination of developing new processes in the model, increased model resolution and enhanced computational performance; 2) representing more realistically the two-way interactions between human activities and natural processes, especially where these interactions affect US energy needs; and 3) ensemble modeling to quantify uncertainty of model simulations and projections.

The quality and quantity of observations really makes us constrain the models. With the new system, we’ll be able to more realistically simulate the present, which gives us more confidence to simulate the future.

—David Bader, Lawrence Livermore National Laboratory (LLNL) scientist and lead of the E3SM project

Simulating atmospheric and oceanic fluid dynamics with fine spatial resolution is especially challenging for ESMs.

The E3SM project is positioned on the forefront of this research challenge, acting on behalf of an international ESM effort. Increasing the number of earth system days simulated per day of computing time is a prerequisite for achieving the E3SM project goal. It also is important for E3SM to effectively use the diverse computer architectures that the DOE Advanced Scientific Computing Research (ASCR) Office procures to be prepared for the uncertain future of next-generation machines.

A long-term aim of the E3SM project is to use exascale machines to be procured over the next five years. The development of the E3SM is proceeding in tandem with the Exascale Computing Initiative (ECI). This represents a thousand-fold increase in performance over that of the most advanced computers from a decade ago.

This model adds a much more complete representation between interactions of the energy system and the earth system. The increase in computing power allows us to add more detail to processes and interactions that results in more accurate and useful simulations than previous models.

—David Bader

To address the diverse critical factors impacting the US energy sector, the E3SM project is dedicated to answering three overarching scientific questions that drive its numerical experimentation initiatives:

• Water Cycle: How does the hydrological cycle interact with the rest of the human-earth system on local to global scales to determine water availability and water cycle extremes?

• Biogeochemistry: How do biogeochemical cycles interact with other earth system components to influence the energy sector?

• Cryosphere Systems: How do rapid changes in cryosphere (continental and ocean ice) systems evolve with the earth system, and contribute to sea-level rise and increased coastal vulnerability?

In the E3SM, all model components (atmosphere, ocean, land, ice) are able to employ variable resolution to focus computing power on fine-scale processes in regions of particular interest. This is implemented using advanced mesh-designs that smoothly taper the grid-scale from the coarser outer region to the more refined region.

The E3SM project includes more than 100 scientists and software engineers at multiple DOE laboratories as well as several universities; the DOE laboratories include Argonne, Brookhaven(link is external), LLNL, Lawrence Berkeley, Los Alamos, Oak Ridge, Pacific Northwest and Sandia national laboratories. In recognition of unifying the DOE earth system modeling community to perform high-resolution coupled simulations, the E3SM executive committee was awarded the Secretary of Energy’s Achievement Award in 2015.

In addition, the E3SM project benefits from-DOE programmatic collaborations including the Exascale Computing Project(link is external) (ECP) and programs in Scientific Discovery Through Advanced Computing (SciDAC), Climate Model Development and Validation (CMDV), Atmospheric Radiation Measurement (ARM), Program for Climate Model Diagnosis and Intercomparison (PCMDI), International Land Model Benchmarking Project (iLAMB), Community Earth System Model Community Earth System Model (CESM) and Next Generation Ecosystem Experiments (NGEE) for the Arctic and the Tropics.