LLNL logo Lawrence Livermore National Laboratory

Co-design at Lawrence Livermore National Lab

Livermore Unstructured Lagrangian Explicit Shock Hydrodynamics (LULESH)



The Shock Hydrodynamics Challenge Problem was originally defined and implemented by LLNL as one of five challenge problems in the DARPA UHPC program and has since become a widely studied proxy application in DOE co-design efforts for exascale. It has been ported to a number of programming models and optimized for a number of advanced platforms – many of those versions are available in the downloads section below. This page gives a brief description of why the problem was chosen. LULESH Ports details other versions and performance tuning work done on the initial version of LULESH, while LULESH 2.0 Changes details the changes made from LULESH 1.0 to LULESH 2.0. A presentation describing the OpenACC port is found here.


What LULESH Models

lulesh visualization Computer simulations of a wide variety of science and engineering problems require modeling hydrodynamics, which describes the motion of materials relative to each other when subject to forces. Many important simulation problems of interest to DOE involve complex multi-material systems that undergo large deformations. LULESH is a highly simplified application, hard-coded to only solve a simple Sedov blast problem with analytic answers – but represents the numerical algorithms, data motion, and programming style typical in scientific C or C++ based applications.

LULESH represents a typical hydrocode, like ALE3D. LULESH approximates the hydrodynamics equations discretely by partitioning the spatial problem domain into a collection of volumetric elements defined by a mesh. A node on the mesh is a point where mesh lines intersect. LULESH is built on the concept of an unstructured hex mesh. Instead, indirection arrays that define mesh relationships are used. The default test case for LULESH appears to be a regular cartesian mesh, but this is for simplicity only – it is important to retain the unstructured data structures as they are representative of what a more complex geometry will require. When modifying LULESH it is important to not take advantage of this or other simplifications in the application. We have tried to spell these out in this technical report, but the list may not be complete.

ALE3D Software Architecture chart showing what LULESH models

Software Downloads

Below is the release version of LULESH 2.0. From a single source it can be built as a serial, OpenMP, MPI or MPI + OpenMP code. Details on the changes in going from LULESH 1.0 to LULESH 2.0 are found in LULESH 2.0 Changes. Both the 1.0 and 2.0 versions of the software were released under the following UCRL: LLNL–CODE-461231. Currently the only port of LULESH 2.0 is to OpenACC.

Lulesh Models Downloads Version
CPU Models .tgz includes Serial, OMP, MPI, MPI+OMP 2.0.3
GPU Model Cuda | OpenACC | OpenMP_4.0 | Open CL 2.0.2
Other Models AMP 2.0.2
Baseline Implementations

The table below contains a collection of significant LULESH versions. More details about these can be found in LULESH Programming Model and Performance Ports Overview. Other tuned versions are also available for experimentation here.

Lulesh Models Downloads Version
CPU Models  Serial  |  OMP   |  MPI  |  MPI + OMP | Fortran 1.0
GPU Models Cuda_Fermi | CUDA_Kepler | CUDA_MPI_OneD_Kepler 1.0
Other Models A++ |  Charm ++ | Chapel | Loci | Liszt | Nabla | Rust 1.0
Fully Unstructured Unstructured 1.0
Vista Database Vista 1.0
One Dimensional Communication Pattern OMP_Z 1.0
Fault Tolerant Checkpoint OMP_Perm 1.0
Best BlueGene/Q versions OMP  1.0
Best Intel SandyBridge versions with ICC compiler OMP  1.0
Baseline Implementations Added Functionality Tuned Implementations


To cite the algorithms and physics modeled in LULESH please use:
Hydrodynamics Challenge Problem
, Lawrence Livermore National Laboratory. Technical Report,
LLNL-TR-490254. [BibTEX]

To cite the LULESH 2.0 code and versions of it please use:
I. Karlin, J. Keasler, R. Neely. LULESH 2.0 Updates and Changes. August 2013, pages 1-9,
LLNL-TR-641973. [BibTEX]

To cite specific LULESH 1.0 code versions found on this website please use:
I. Karlin, A. Bhatele, B. Chamberlain, J. Cohen, Z. Devito, M. Gokhale, R. Haque, R. Hornung, J. Keasler, D. Laney, E. Luke, S. Lloyd, J. McGraw R. Neely, D. Richards, M. Schulz, C. H. Still, F. Wang, D. Wong. LULESH Programming Model and Performance Ports Overview, December 2012, pages 1-17,
LLNL-TR-608824. [BibTEX]

Refereed LULESH Publications:
I. Karlin, A. Bhatele, J. Keasler, B. L. Chamberlain, J. Cohen, Z DeVito, R. Haque, D. Laney, E. Luke, F. Wang, D. Richards, M. Schulz, C. H. Still, Exploring Traditional and Emerging Parallel Programming Models using a Proxy Application. In Proceedings of 27th IEEE International Parallel & Distributed Processing Symposium, Boston, MA, pages 1-14, (Best Paper) LLNL-CONF-586774. [BibTEX]
Presentation Slides: LLNL-PRES-637084

Summer Co-design School

Join us!

For primary information about LULESH, contact: proxyapp-info@llnl.gov