Summer 2014 Scalability Study

This page presents a collection of small studies over the performance of the Proteus modeling toolkit.

Currently, the study focuses on the behavior of the 2D and 3D dambreak benchmarks from the Proteus-MPRANS private repository. The study should still be of general value in tuning Proteus on supercomputers.

Performance Profile

Prior to running the scalability study, I completed several small performance profiles.

Here's an example snippet for running a performance profile on Garnet.

source /opt/modules/default/init/bash
module swap PrgEnv-pgi PrgEnv-gnu
module unload xt-libsci
module load acml
source /u/aron/proteus/garnet.gnu/bin/proteus_env.sh
PARUN=/u/aron/proteus/garnet.gnu/bin/parun
PROTEUS_PYTHON=$(which python)

cd $PBS_O_WORKDIR
export MPICH_RANK_REORDER_METHOD=2
aprun -n 8 -N 4 ${PROTEUS_PYTHON} ${PARUN} dambreak_so.py -l 5 -v -p -O ../inputTemplates/petsc.options.schur_upper_a11_asm_boomeramg 

Note that the -p flag is needed to enable the profile. The performance profile is tabulated and visualized using gprof2dot:

gprof2dot -f pstats dambreak_p_run_prof0 | dot -Tpng -o garnet_dambreak_p_run_prof0.png

Note that the solver is dominating the simulation's computation time. This is a well-known trend in high performance computing of numerical models, and is the reason considerable research effort has been poured into the development of scalable preconditioners for the solution of systems of nonlinear equations arising from discretization of important physical models (such as the Navier-Stokes equations).

Scalability Studies

All scalability studies were performed on the Garnet supercomputer at ERDC. Reproducible results are recording into a version controlled Git repository publicly available here: https://github.com/erdc-cm/proteus-profile

Proteus uses PETSc as its underlying interface for scalably solving systems of linear equations (Proteus implements its own Newton solver). PETSc provides its own domain decomposition algorithms and preconditioners, and also interfaces to several major libraries for solving systems of equations, such as Hypre

2D Strong Scalability Study

The first study run was a strong scalability test. This test answers the question, "Can I obtain solutions faster by adding more processors to the problem?" In an ideal world, doubling the number of processors will double the speed of a simulation. In real world scenarios, a number of factors prevent this from happening.

2D/3D Weak Scalability Study

The 2D and 3D scalability study was conducted on the Garnet supercomputer system at ERDC, using the Proteus-MPRANS dambreak benchmark.

The first

Further Reading and References