V2.14.x (#645)

* Updating development version to 2.13-dev * Release guidelines update * Add Changes to Changelog (#604) * Remoev UWGeodynamics installation from dockerfiles * Petsc 3.17 compatibility and removing duplicate examples python scripts * Add petsc variants to conda recipe * Add python 3.10 conda builds to actions. * Change PETSC and MPICH versions * Fix MPICH complaining... * Getting Dockerfiles ready: Ubuntu 22.04LTS and python 3.10. Note that this will fail for now because of h5py not being compatible. PR is on its way. * Update README * Update README * Transfer UWGeo documentation to Underworld * Remove python3-numpy-dbg for now * Fix Documentation Links (UWGeodynamics) Fix visualisation imports * Fix imports (UWGeodynamics) * Change Depth documentation toctree * UWGeodynamics Merging * Run tests * Move tests * H5py 3.7 is released, Use it in docker. * Typo in Petsc Dockerfile * Getting UWGeo internals to use uw2 mpi4py interface (#612) * Getting UWGeo internals to use uw2 mpi4py interface * Update CHANGES.md * ask per 2.13 release (#615) 2.13 release Co-authored-by: Romain Beucher <romain.beucher@anu.edu.au> * Update _version.py * Fix for analytic ID, which changed during cmake update. Ganalytics should now be right * Typo fix for 3.10 * Dockerfile improvement - should be a single layer command * safer #define when not using an official release of petsc * Update PetscOptionsBegin() to be dev compatible. see https://gitlab.com/petsc/petsc/-/blob/f3ed01cd9ab452072c1a44e935347389f67d72c6/doc/docs/changes/dev.rst * Petsc 3.18 updates Petsc error handling updates only. * Dockerfile updates. * Adding mpi dockerfiles - only openmpi for now, mpich coming soon. * Removing `base` image for now. * Resultant UW image is ~2.4Gb rather than 3Gb. * Add some documentation * Activate UWGeo testing * Updates github actions for Dockerfile change * Update CI.yml Fix typo * Update CI.yml * Update CI.yml * Update CI.yml * Update CI.yml * Update CI.yml * Update CI.yml * Update CI.yml * Update CI.yml * Update CI.yml * Update CI.yml * Update CI.yml * Update CI.yml * Update CI.yml * Update CI.yml * Update CI.yml * Update CI.yml * Updating versions of github actions * Fixing petsc image name * Rebuild UW is the dockerfile changes * Add checks * Forgot github action variable * checks variable is working or not? * Can't get underworld check in github action to work ???!!! * Fix? * Typo fix * Update CI.yml debug * Update CI.yml always! * Update CI.yml * Update CI.yml * Update CI.yml * Update CI.yml * Including vim in find docker image * Better way to look for h5 units. Previous h5py interface was causing erros on the c level. * Updates to build docker * Enable tests round 1 * Update CI.yml * Update CI.yml * Renmae UWGeo test, they take a while, might reintroduce later * Revert "Renmae UWGeo test, they take a while, might reintroduce later" This reverts commit d8a2268. * Smarter way to disable UWgeo tests for CI * Typo fix * Typo #2 * opss, fixes broken units check, thanks tests ;) * Typo #3 * type #4 * Enable multi arch build !!! * Update for dockerfiles and macos * Rename docker images * Update Dockerfile * Update Dockerfile new docker image names * Update Dockerfile new docker image names * CHAGES.md and checkin docker-builder.sh for manual docker image creation * docs/development/docker/docker-builder.sh * Updates for conda - still testing * Update CI.yml * Update CI.yml * Update CI.yml * Update CI.yml * Testing conda-forge connection to github actions * Making petsc version a build-arg for the dockerfiles. * Typo and adding UW_VERSION to github actions too. * Update CI.yml Can't resolve tag in `container` of github actions!!! * Disable arm build for underworld image - easier for now. * Dockerfile go * Work please * Dockerfile madness * Pin down psutil as a dependency was updating it - breaking jupyter-lab * Updates * psutil set to 5.9.3 for github action builds * Touch Lavavu * Dockerfile touch ups * remove psutil confusion * setuptools 65.6.0 has an error, so use 65.5.1 * update lavavu * fix * Fix typo in docker label * v2.14 branch upto with dev development (#640) * Updating development version to 2.13-dev * Release guidelines update * Add Changes to Changelog (#604) * Remoev UWGeodynamics installation from dockerfiles * Petsc 3.17 compatibility and removing duplicate examples python scripts * Add petsc variants to conda recipe * Add python 3.10 conda builds to actions. * Change PETSC and MPICH versions * Fix MPICH complaining... * Getting Dockerfiles ready: Ubuntu 22.04LTS and python 3.10. Note that this will fail for now because of h5py not being compatible. PR is on its way. * Update README * Update README * Transfer UWGeo documentation to Underworld * Remove python3-numpy-dbg for now * Fix Documentation Links (UWGeodynamics) Fix visualisation imports * Fix imports (UWGeodynamics) * Change Depth documentation toctree * UWGeodynamics Merging * Run tests * Move tests * H5py 3.7 is released, Use it in docker. * Typo in Petsc Dockerfile * Getting UWGeo internals to use uw2 mpi4py interface (#612) * Getting UWGeo internals to use uw2 mpi4py interface * Update CHANGES.md * ask per 2.13 release (#615) 2.13 release Co-authored-by: Romain Beucher <romain.beucher@anu.edu.au> * Update _version.py * Fix for analytic ID, which changed during cmake update. Ganalytics should now be right * Typo fix for 3.10 * Dockerfile improvement - should be a single layer command * safer #define when not using an official release of petsc * Update PetscOptionsBegin() to be dev compatible. see https://gitlab.com/petsc/petsc/-/blob/f3ed01cd9ab452072c1a44e935347389f67d72c6/doc/docs/changes/dev.rst * Petsc 3.18 updates Petsc error handling updates only. Co-authored-by: Romain Beucher <romain.beucher@anu.edu.au> * Add build-args for openmpi Docker creation * Dockerfile updates (#1) * Dockerfile updates. * Adding mpi dockerfiles - only openmpi for now, mpich coming soon. * Removing `base` image for now. * Resultant UW image is ~2.4Gb rather than 3Gb. * Add some documentation * Activate UWGeo testing * Updates github actions for Dockerfile change * Update CI.yml Fix typo * Update CI.yml * Update CI.yml * Update CI.yml * Update CI.yml * Update CI.yml * Update CI.yml * Update CI.yml * Update CI.yml * Update CI.yml * Update CI.yml * Update CI.yml * Update CI.yml * Update CI.yml * Update CI.yml * Update CI.yml * Update CI.yml * Updating versions of github actions * Fixing petsc image name * Rebuild UW is the dockerfile changes * Add checks * Forgot github action variable * checks variable is working or not? * Can't get underworld check in github action to work ???!!! * Fix? * Typo fix * Update CI.yml debug * Update CI.yml always! * Update CI.yml * Update CI.yml * Update CI.yml * Update CI.yml * Including vim in find docker image * Better way to look for h5 units. Previous h5py interface was causing erros on the c level. * Updates to build docker * Enable tests round 1 * Update CI.yml * Update CI.yml * Renmae UWGeo test, they take a while, might reintroduce later * Revert "Renmae UWGeo test, they take a while, might reintroduce later" This reverts commit d8a2268. * Smarter way to disable UWgeo tests for CI * Typo fix * Typo #2 * opss, fixes broken units check, thanks tests ;) * Typo #3 * type #4 * Enable multi arch build !!! * Update for dockerfiles and macos * Rename docker images * Update Dockerfile * Update Dockerfile new docker image names * Update Dockerfile new docker image names * CHAGES.md and checkin docker-builder.sh for manual docker image creation * docs/development/docker/docker-builder.sh * Updates for conda - still testing * Update CI.yml * Update CI.yml * Update CI.yml * Update CI.yml * Testing conda-forge connection to github actions * Making petsc version a build-arg for the dockerfiles. * Typo and adding UW_VERSION to github actions too. * Update CI.yml Can't resolve tag in `container` of github actions!!! * Disable arm build for underworld image - easier for now. * Dockerfile go * Work please * Dockerfile madness * Pin down psutil as a dependency was updating it - breaking jupyter-lab * Updates * psutil set to 5.9.3 for github action builds * Touch Lavavu * Dockerfile touch ups * remove psutil confusion * setuptools 65.6.0 has an error, so use 65.5.1 * update lavavu * fix * Fix typo in docker label * v2.14 branch upto with dev development (#640) * Updating development version to 2.13-dev * Release guidelines update * Add Changes to Changelog (#604) * Remoev UWGeodynamics installation from dockerfiles * Petsc 3.17 compatibility and removing duplicate examples python scripts * Add petsc variants to conda recipe * Add python 3.10 conda builds to actions. * Change PETSC and MPICH versions * Fix MPICH complaining... * Getting Dockerfiles ready: Ubuntu 22.04LTS and python 3.10. Note that this will fail for now because of h5py not being compatible. PR is on its way. * Update README * Update README * Transfer UWGeo documentation to Underworld * Remove python3-numpy-dbg for now * Fix Documentation Links (UWGeodynamics) Fix visualisation imports * Fix imports (UWGeodynamics) * Change Depth documentation toctree * UWGeodynamics Merging * Run tests * Move tests * H5py 3.7 is released, Use it in docker. * Typo in Petsc Dockerfile * Getting UWGeo internals to use uw2 mpi4py interface (#612) * Getting UWGeo internals to use uw2 mpi4py interface * Update CHANGES.md * ask per 2.13 release (#615) 2.13 release Co-authored-by: Romain Beucher <romain.beucher@anu.edu.au> * Update _version.py * Fix for analytic ID, which changed during cmake update. Ganalytics should now be right * Typo fix for 3.10 * Dockerfile improvement - should be a single layer command * safer #define when not using an official release of petsc * Update PetscOptionsBegin() to be dev compatible. see https://gitlab.com/petsc/petsc/-/blob/f3ed01cd9ab452072c1a44e935347389f67d72c6/doc/docs/changes/dev.rst * Petsc 3.18 updates Petsc error handling updates only. Co-authored-by: Romain Beucher <romain.beucher@anu.edu.au> * Add build-args for openmpi Docker creation Co-authored-by: Julian Giordani <jgiordani@mini.staff.wireless.sydney.edu.au> Co-authored-by: Romain Beucher <romain.beucher@anu.edu.au> * Enable the default args * disable petsc in github workflows * Update to underworldcode docker hub * update for 2.14 release (#642) * Images will push to underworldcode docker hub now * Change to kick off underworld2 image build * fix spacing in version file * Update CHANGES.md * Adding mpich Dockerfile * Disable github actions build of petsc and uw * Adding ARG smarts to petsc and UW image. Now MPI_IMAGE for petsc/Dockerfile and PETSC_IMAGE for uw2/Dockerfile are options that can we used to specify the exact docker image name. This will be used by github actions in future. * Update CI.yml * Update CI.yml Fix indenting of yaml * Update CI.yml * workdispatch only for openmpi/mpich and lavavu * Update CI.yml Adding MPICH_VERSION as an env var * Clean up docker-builder script * Smarts for docker-builder.sh Co-authored-by: Romain Beucher <romain.beucher@anu.edu.au> Co-authored-by: Julian Giordani <jgiordani@mini.staff.wireless.sydney.edu.au>
underworldcode · Dec 14, 2022 · a75ed03 · a75ed03
1 parent ff32a86
commit a75ed03
Show file tree

Hide file tree

Showing 27 changed files with 956 additions and 402 deletions.
diff --git a/.github/workflows/CI.yml b/.github/workflows/CI.yml
diff --git a/CHANGES.md b/CHANGES.md
@@ -1,6 +1,18 @@
 CHANGES: Underworld2
 =======================
 
+Release 2.14 [2022-11-29]
+---------------------------
+New:
+ * New Dockerfile configuration resulting in smaller docker images.
+ * Move to Petsc-3.18.1.
+
+Changes:
+ * Multiple mpi implementation images for HPC usage (Gadi and Setonix).
+
+Fixes:
+ * Revert to HDF5 1.10. Fix for Checkpointing issues.
+
 Release 2.13.1 [2022-08-15]
 ---------------------------
 Fixes:

diff --git a/conda/conda_build_config.yaml b/conda/conda_build_config.yaml
@@ -2,5 +2,4 @@
     - mpich
     - openmpi [linux]
  petsc:
-    - 3.16
-    - 3.17
+    - 3.18.1
diff --git a/conda/meta.yaml b/conda/meta.yaml
@@ -1,5 +1,5 @@
 {% set name = "underworld" %}
-{% set version = "2.13.0" %}
+{% set version = "2.14.0" %}
 {% set sha256 = "fdc6c7ae1034b5bd6159b465ae44d75a19ca9bb44021e9d16c9eafb6bced2e12" %}
 {% set build = 0 %}
 
@@ -8,6 +8,8 @@ package:
   version: {{ version }}
 
 source:
+  # git_url: https://github.com/julesghub/underworld2.git
+  # git_rev: dockerfile-updates
   url: https://github.com/underworldcode/underworld2/archive/refs/tags/v{{ version }}b.tar.gz
   sha256: {{ sha256 }}
 

diff --git a/docs/development/docker.md b/docs/development/docker.md
@@ -0,0 +1,67 @@
+Underworld Docker Schema
+--------------------------------
+
+https://hub.docker.com/u/underworldcode/
+
+**lavavu**: 
+
+    This Dockerfile generates the software stack required by `lavavu`. The resulting image can operate
+    stand alone to run `lavavu` within `Jupyter`. This image should be rebuilt after a new release is published.  
+
+**petsc**: 
+
+    This Dockerfile generates the software stack required by `petsc`. It includes `MPI`. The resulting image can operate
+    stand alone to, for example, run `petsc4py`. This image should be rebuilt after a new release is published.  
+
+**base**:
+
+    This Dockerfile generates the software stack required by Underworld. The corresponding image is only built 
+    when explicitly triggered by a developer. At the beginning of a new release cycle (ie, just after a release has been
+    made), this image is regenerated rebuilt to update software stack. The `lavavu` and `petsc` images should
+    be rebuilt first. 
+
+**underworld2**:
+
+    Inherits from `base`.
+    This image is automatically built (tag:dev) whenever a commit is pushed to the Underworld Github
+    repository (branch: development). It inherits from an SHA digest pinned `base` image so that the full 
+    software stack is fixed within any release cycle. This also has the added benefit of recording the `base`
+    image's SHA digest to the git repo, and therefore allowing the **exact** software stack to be
+    reproduced (ie, pulled down from Docker Hub) at a later date if necessary. At the beginning
+    of a release cycle, the `base` image is regenerated, and the `underworld2` image is
+    then pinned to the new version of the `base` image. This allows thorough testing by the dev
+    team before the next release.
+
+    Stable releases are built manually and pushed up to this registry with appropriate tags.
+
+Releases
+-----------
+
+Note that because we cannot yet build ARM images automatically using github actions.
+We need to build and push them manually from the mac mini (M1) making sure that we properly tag and create the docker manifest.
+
+    ```
+    $ docker manifest create underworldcode/petsc --amend underworldcode/petsc:amd64 --amend underworldcode/petsc:arm64
+    $ docker manifest push underworldcode/petsc
+    ```
+
+Example, if `2.5` release:
+
+** Tag the release locally:
+```
+$ docker tag 6649e5e26534 underworldcode/underworld2:2.5.0b
+```
+** Push the tagged image
+```
+$ docker login
+$ docker push underworldcode/underworld2:2.5.0b
+```
+
+Security
+---------
+
+All dockers are designed to be run as non-root user. So `apt-get install` won't function.
+Care has been taken to minimise optional packages, i.e. `wget`. This minimises the overall size of the docker image size,
+and diminishes potential security weaknesses.
+That said `pip` is enabled in every image, so python packages can be downloaded into a container runtime.
+
diff --git a/docs/development/docker/docker-builder.sh b/docs/development/docker/docker-builder.sh
@@ -0,0 +1,42 @@
+#!/usr/bin/bash
+
+set -e
+
+
+# Example Usage:
+#   - Run from underworld2 repository head
+#   - mpi and lavavu dockers are automatically generated via github actions
+#   - petsc and underworld2 must be created by runn the following script.
+
+
+ARCH=$(uname -m)
+echo "Will build docker image locally for architecture type: $ARCH"
+echo "************************************************************\n"
+
+## The mpi and lavavu images should be automatically made via github actions
+#docker build . --pull -f ./docs/development/docker/mpi/Dockerfile.openmpi -t underworldcode/openmpi:4.1.4-$ARCH
+#docker build . --pull -f ./docs/development/docker/lavavu/Dockerfile -t underworldcode/lavavu:$ARCH
+
+docker build . --pull \
+  -f ./docs/development/docker/petsc/Dockerfile \
+  --build-arg MPI_IMAGE="underworldcode/openmpi:4.1.4" \
+  -t underworldcode/petsc:3.18.1-$ARCH
+
+# don't use pull here as we want the petsc image above
+docker build . \
+  --build-arg PETSC_IMAGE="underworldcode/petsc:3.18.1-$ARCH" \
+  -f ./docs/development/docker/underworld2/Dockerfile \
+  -t underworldcode/underworld2:2.14.0b-$ARCH
+
+
+docker push underworldcode/petsc:3.18.1-$ARCH 
+docker push underworldcode/underworld2:2.14.0b-$ARCH
+
+#### if updates for both arm64 and x86_64 build manifest, ie
+# docker manifest create underworldcode/petsc:3.18.1 \
+#  -a underworldcode/petsc:3.18.1-x86_64 \
+#  -a underworldcode/petsc:3.18.1-arm64
+#
+# docker manifest push underworldcode/petsc:3.18.1
+#
+# in future this should be automated
diff --git a/docs/development/docker/lavavu/Dockerfile b/docs/development/docker/lavavu/Dockerfile
@@ -6,7 +6,6 @@ ENV PYVER=3.10
 ENV VIRTUAL_ENV=/opt/venv
 # The following ensures that the venv takes precedence if available
 ENV PATH=${VIRTUAL_ENV}/bin:$PATH
-
 # The following ensures venv packages are available when using system python (such as from jupyter)
 ENV PYTHONPATH=${PYTHONPATH}:${VIRTUAL_ENV}/lib/python${PYVER}/site-packages
 # add joyvan user, volume mount and expose port 8888
@@ -43,14 +42,6 @@ RUN apt-get update -qq \
 &&  apt-get clean \
 &&  rm -rf /var/lib/apt/lists/*
 
-RUN pip3 install -U setuptools  \
-&&  pip3 install --no-cache-dir \
-        packaging \
-        appdirs \
-        jupyter \
-        pillow \
-        ipython
-
 FROM base_runtime AS build_base
 # FROM base_runtime AS build_base
 # install build requirements
@@ -72,7 +63,17 @@ RUN DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends \
         zlib1g-dev \
         cmake \
         libopenblas-dev \
-        libz-dev
+        libz-dev \
+        gcc \
+        python3-dev
+
+RUN pip3 install -U setuptools  \
+&&  pip3 install --no-cache-dir \
+        packaging \
+        appdirs \
+        jupyter \
+        pillow \
+        ipython
 
 # lavavu
 # create a virtualenv to put new python modules

diff --git a/docs/development/docker/mpi/Dockerfile.mpich b/docs/development/docker/mpi/Dockerfile.mpich
@@ -0,0 +1,87 @@
+#####################################################################
+# Multi stage Dockerfile structure:
+# 1. runtime 
+# 2. build
+# 3. final == runtime + min. build
+#
+# It begins with layers for runtime execution. 
+# The runtime environment (packages, permissions, ENV vars.) 
+# are consistent accross all layer of this Dockerfile. 
+# The build layer takes the runtime layer and builds the software
+# stack in /usr/local.
+# The final image is a composite of the runtime layer and 
+# minimal sections of the build layer.
+#####################################################################
+
+ARG MPICH_VERSION="3.4.3"
+
+FROM ubuntu:22.04 as runtime
+LABEL maintainer="https://github.com/underworldcode/"
+
+################
+## 1. Runtime ##
+################
+# Dockerfile ENV vars - for all image stages
+ENV LANG=C.UTF-8
+# openmpi lib will be install at /usr/local/lib
+ENV LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
+# add user jovyan
+ENV NB_USER jovyan
+ENV NB_HOME /home/$NB_USER
+RUN useradd -m -s /bin/bash -N $NB_USER
+
+#install runtime packages
+RUN apt-get update -qq \
+&&  DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends \
+        ssh \
+        bash \
+&&  apt-get clean \
+&&  rm -rf /var/lib/apt/lists/*
+
+################
+## 2. Build   ##
+################
+FROM runtime as build
+
+ARG MPICH_VERSION
+# Build options for Dockerfile
+ARG MPICH_CONFIGURE_OPTIONS="--enable-fast=all,O3 --prefix=/usr/local --with-device=ch4:ofi FFLAGS=-fallow-argument-mismatch FCFLAGS=-fallow-argument-mismatch"
+ARG MPICH_MAKE_OPTIONS="-j4"
+
+RUN apt-get update -qq \
+&&  DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends \
+        wget \
+        gcc \
+        gfortran \
+        g++ \
+        make \
+        file \
+&&  apt-get clean \
+&&  rm -rf /var/lib/apt/lists/*
+
+# build mpi
+RUN mkdir -p /tmp/src
+WORKDIR /tmp/src
+RUN wget http://www.mpich.org/static/downloads/${MPICH_VERSION}/mpich-${MPICH_VERSION}.tar.gz --no-check-certificate \
+&& tar -zxf mpich-${MPICH_VERSION}.tar.gz 
+WORKDIR /tmp/src/mpich-${MPICH_VERSION}              
+RUN ./configure ${MPICH_CONFIGURE_OPTIONS} \
+ && make ${MPICH_MAKE_OPTIONS} \
+ && make install \
+ && ldconfig \
+ && rm -rf /tmp/src/
+
+# record build packages used
+RUN apt-mark showmanual > /opt/installed.txt
+
+################
+## 3. Final   ##
+################
+FROM runtime as final
+
+COPY --from=build /usr/local /usr/local
+COPY --from=build /opt/installed.txt /opt/installed.txt
+
+# switch to user and workspace
+WORKDIR $NB_HOME
+USER $NB_USER
diff --git a/docs/development/docker/mpi/Dockerfile.openmpi b/docs/development/docker/mpi/Dockerfile.openmpi
@@ -0,0 +1,91 @@
+#####################################################################
+# Multi stage Dockerfile structure:
+# 1. runtime 
+# 2. build
+# 3. final == runtime + min. build
+#
+# It begins with layers for runtime execution. 
+# The runtime environment (packages, permissions, ENV vars.) 
+# are consistent accross all layer of this Dockerfile. 
+# The build layer takes the runtime layer and builds the software
+# stack in /usr/local.
+# The final image is a composite of the runtime layer and 
+# minimal sections of the build layer.
+#####################################################################
+
+# https://docs.docker.com/engine/reference/builder/#understand-how-arg-and-from-interact
+ARG OMPI_VERSION=4.1.4
+
+FROM ubuntu:22.04 as runtime
+LABEL maintainer="https://github.com/underworldcode/"
+
+################
+## 1. Runtime ##
+################
+# Dockerfile ENV vars - for all image stages
+ENV LANG=C.UTF-8
+# openmpi lib will be install at /usr/local/lib
+ENV LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
+# add user jovyan
+ENV NB_USER jovyan
+ENV NB_HOME /home/$NB_USER
+RUN useradd -m -s /bin/bash -N $NB_USER
+
+# install runtime packages
+RUN apt-get update -qq \
+&&  DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends \
+        ssh \
+        bash \
+&&  apt-get clean \
+&&  rm -rf /var/lib/apt/lists/*
+
+
+################
+## 2. Build   ##
+################
+FROM runtime as build
+
+# Build options for for openmpi
+ARG OMPI_VERSION
+ARG OMPI_MAJOR_VERSION="v4.1"
+ARG OMPI_CONFIGURE_OPTIONS="--prefix=/usr/local"
+ARG OMPI_MAKE_OPTIONS="-j4"
+
+# apt get install dependency packages
+RUN apt-get update -qq \
+&&  DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends \
+        wget \
+        gcc \
+        gfortran \
+        g++ \
+        make \
+        file \
+&&  apt-get clean \
+&&  rm -rf /var/lib/apt/lists/*
+
+# build mpi and remove tarball at the end
+RUN mkdir -p /tmp/src
+WORKDIR /tmp/src
+RUN wget https://download.open-mpi.org/release/open-mpi/${OMPI_MAJOR_VERSION}/openmpi-${OMPI_VERSION}.tar.gz --no-check-certificate \
+&& tar -zxf openmpi-${OMPI_VERSION}.tar.gz 
+WORKDIR /tmp/src/openmpi-${OMPI_VERSION}              
+RUN ./configure ${OMPI_CONFIGURE_OPTIONS} \
+&&  make ${OMPI_MAKE_OPTIONS} \
+&&  make install \
+&&  rm -rf /tmp/src/
+
+# record build packages used
+RUN apt-mark showmanual > /opt/installed.txt
+
+
+################
+## 2. Final   ##
+################
+FROM runtime as final
+
+COPY --from=build /usr/local /usr/local
+COPY --from=build /opt/installed.txt /opt/installed.txt
+
+# switch to user and workspace
+WORKDIR $NB_HOME
+USER $NB_USER