0.10.x merge into main (#1235)

* run CI on new 0.10.x branch (#1212) * run CI on new 0.10.x branch * Update .github/workflows/CI.yaml Co-authored-by: Iván Pulido <2949729+ijpulidos@users.noreply.github.com> --------- Co-authored-by: Iván Pulido <2949729+ijpulidos@users.noreply.github.com> * print perses version on startup (#1176) * Update comments in `RESTCapableHybridTopologyFactory` (#1189) * update comments * bump ci --------- Co-authored-by: Iván Pulido <2949729+ijpulidos@users.noreply.github.com> Co-authored-by: Mike Henry <11765982+mikemhenry@users.noreply.github.com> * use python executable from env (#1174) * use python executable from env * run tests using the shell * parmed 4 seems to be giving us some issues * actually I think we want the newer parmed * pin pymbar for now * ooof, parmed != pymbar * add some more debugging to figure out how we are getting old parmed * Update devtools/conda-envs/test_env.yaml * see if shell=True fixes it * removed parmed by mistake * Support pymbar 4 (#1173) * switch to using pymbar 3 & 4 support from openmmtools * fix typo on import * fix yaml * switch to using pymbar 4 * missed a pymbar import * missed another one * bump ci * missed a import * go back to how it was * Update devtools/conda-envs/test_env.yaml --------- Co-authored-by: Iván Pulido <2949729+ijpulidos@users.noreply.github.com> * Remove example testing (#1214) * Store initial and final topologies for all phases -- small molecule pipeline (#1210) * Initial and final topologies serialized per phase. * Using properties instead of private attrs. * Fix test * Remove uneeded code/attributes * bump ci * Store phase topologies separately * Fix tests. vacuum topologies expected. * Better docstring. --------- Co-authored-by: Mike Henry <11765982+mikemhenry@users.noreply.github.com> * Improve docker building (#1200) * Added note about example for adding oe license * make docker file much simpler * build images in CI * forgot to add conda-forge * fix permissions on a step * get oe_license file mounted in docker container * mount path must be absolute * setup singulairty to test + fix testing on docker image * add some testing deps to the image * add -v for tests, fix envar * tests are failing, want to test rest of pipeline * use latest openmm version * hardcode perses version for now * bump ci * make sure we can make openeye dir * support a dev build as well * don't hardcode value * fix name clash * forgot to add conda-forge * bump ci * test docker image and fix missing deps * install ps * also push latest tag * don't build on tag since the conda-forge package won't exist yet * don't test the examples * Remove docker deb build, we can do these ourselves * better document container use * build a latest version for apptainer * build with 11.2 to make things more compatable * skip docker test to see if other bits build okay * see if I can get the step to fail if there is an error * skip docker tests to make sure apptainer builds okay * Add mpiplus and mpi4py to docker image * give correct path to oe license * add mpi stuff to docker * clean up diskspace before build * skip tests for singularity now that the only failures come from a package bug * Clean examples -- CLI protein-ligand example for Tyk2 (#1223) * Improving examples dir structure and readme * Adding Tyk2 CLI example * removing new-cli/ripk2 example (deprecated) * fix typo for link * Clarify tyk2 cli example docs * Realtime analysis interval to default to checkpoint interval (#1227) * CI miscellaneous fixes (#1217) * CI minor fixes. Allow codecov to fail. * bump ci --------- Co-authored-by: Mike Henry <11765982+mikemhenry@users.noreply.github.com> * Changing offline freq default to checkpoint interval * Fixing input yaml for example * commenting offline-freq param --------- Co-authored-by: Mike Henry <11765982+mikemhenry@users.noreply.github.com> * MPI example with dipeptide mutation (#1228) * peptide mutation MPI example added * better documentation of motivation * Fix/issue 1194 (#1230) * set cutoff distance in sterics_custom_nonbonded_force * matching cutoff for custom forces. Improving logic. * test for HTF nonbonded cutoff --------- Co-authored-by: Iván Pulido <2949729+ijpulidos@users.noreply.github.com> * Fix/issue 1196 (#1229) * CI miscellaneous fixes (#1217) * CI minor fixes. Allow codecov to fail. * bump ci --------- Co-authored-by: Mike Henry <11765982+mikemhenry@users.noreply.github.com> * added dels to contexts * Update perses/app/setup_relative_calculation.py --------- Co-authored-by: Iván Pulido <2949729+ijpulidos@users.noreply.github.com> * Fix spectator support (#1233) * fix spectator support. Enabling test. * Test to run on GPU CI. * pin <4 for pymbar on GPU --------- Co-authored-by: Iván Pulido <2949729+ijpulidos@users.noreply.github.com> Co-authored-by: Ivy Zhang <35546250+zhang-ivy@users.noreply.github.com>
choderalab · Sep 14, 2023 · 2329fed · 2329fed
1 parent ae55801
commit 2329fed
Show file tree

Hide file tree

Showing 49 changed files with 8,430 additions and 13,528 deletions.
diff --git a/.github/workflows/CI.yaml b/.github/workflows/CI.yaml
@@ -5,12 +5,14 @@ on:
   pull_request:
     branches:
       - "main"
+      - "0.10.*"
   schedule:
     # nightly tests
     - cron: "0 0 * * *"
   push:
     branches:
       - main
+      - "0.10.x"
 
 concurrency:
   group: "${{ github.workflow }}-${{ github.ref }}"

diff --git a/.github/workflows/docker.yaml b/.github/workflows/docker.yaml
@@ -0,0 +1,164 @@
+# This workflow uses actions that are not certified by GitHub.
+# They are provided by a third-party and are governed by
+# separate terms of service, privacy policy, and support
+# documentation.
+
+# GitHub recommends pinning actions to a commit SHA.
+# To get a newer version, you will need to update the SHA.
+# You can also reference a tag or branch, but the action may change without warning.
+
+name: Create and publish a Docker image
+
+on:
+  push:
+    branches:
+      - feat/improve_docker_building 
+  workflow_dispatch:
+
+defaults:
+  run:
+    shell: bash -l {0}
+
+env:
+  REGISTRY: ghcr.io
+  IMAGE_NAME: choderalab/perses
+
+jobs:
+  build-and-push-image:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      packages: write
+
+    steps:
+      - name: Free disk space
+        run: |
+          run: |
+          sudo docker rmi $(docker image ls -aq) >/dev/null 2>&1 || true
+          sudo rm -rf \
+            /usr/share/dotnet /usr/local/lib/android /opt/ghc \
+            /usr/local/share/powershell /usr/share/swift /usr/local/.ghcup \
+            /usr/lib/jvm || true
+          echo "some directories deleted"
+          sudo apt install aptitude -y >/dev/null 2>&1
+          sudo aptitude purge aria2 ansible azure-cli shellcheck rpm xorriso zsync \
+            esl-erlang firefox gfortran-8 gfortran-9 google-chrome-stable \
+            google-cloud-sdk imagemagick \
+            libmagickcore-dev libmagickwand-dev libmagic-dev ant ant-optional kubectl \
+            mercurial apt-transport-https mono-complete libmysqlclient \
+            unixodbc-dev yarn chrpath libssl-dev libxft-dev \
+            libfreetype6 libfreetype6-dev libfontconfig1 libfontconfig1-dev \
+            snmp pollinate libpq-dev postgresql-client powershell ruby-full \
+            sphinxsearch subversion mongodb-org azure-cli microsoft-edge-stable \
+            -y -f >/dev/null 2>&1
+          sudo aptitude purge google-cloud-sdk -f -y >/dev/null 2>&1
+          sudo aptitude purge microsoft-edge-stable -f -y >/dev/null 2>&1 || true
+          sudo apt purge microsoft-edge-stable -f -y >/dev/null 2>&1 || true
+          sudo aptitude purge '~n ^mysql' -f -y >/dev/null 2>&1
+          sudo aptitude purge '~n ^php' -f -y >/dev/null 2>&1
+          sudo aptitude purge '~n ^dotnet' -f -y >/dev/null 2>&1
+          sudo apt-get autoremove -y >/dev/null 2>&1
+          sudo apt-get autoclean -y >/dev/null 2>&1
+          echo "some packages purged"
+
+      - name: Checkout repository
+        uses: actions/checkout@v3
+        with:
+          fetch-depth: 0
+
+      - name: Get Latest Version
+        id: latest-version
+        run: |
+          LATEST_TAG=$(git describe --tags $(git rev-list --tags --max-count=1))
+          echo $LATEST_TAG
+          echo "LATEST_TAG=$LATEST_TAG" >> $GITHUB_OUTPUT
+          # In this case the version is the same as the latest_tag
+          VERSION=$LATEST_TAG
+          echo $VERSION
+          echo "VERSION=$VERSION" >> $GITHUB_OUTPUT
+
+      - name: Print Latest Version
+        run: echo ${{ steps.latest-version.outputs.VERSION }}
+
+      - name: Create fully qualified image registry path
+        id: fqirp
+        run: |
+          FQIRP=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ steps.latest-version.outputs.VERSION }}
+          echo "FQIRP=$FQIRP" >> $GITHUB_OUTPUT
+
+      - name: Print FQIRP
+        run: echo ${{ steps.fqirp.outputs.FQIRP  }}
+
+      - name: Log in to the Container registry
+        uses: docker/login-action@f054a8b539a109f9f41c372932f1ae047eff08c9
+        with:
+          registry: ${{ env.REGISTRY }}
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Extract metadata (tags, labels) for Docker
+        id: meta
+        uses: docker/metadata-action@98669ae865ea3cffbcbaa878cf57c20bbf1c6c38
+        with:
+          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
+          tags: |
+            type=schedule,pattern=nightly,enable=true,priority=1000
+            type=ref,event=branch,enable=true,priority=600
+            type=ref,event=tag,enable=true,priority=600
+            type=ref,event=pr,prefix=pr-,enable=true,priority=600
+            type=semver,pattern={{major}}.{{minor}}
+            type=semver,pattern={{version}}
+            type=sha
+            ${{ steps.latest-version.outputs.VERSION }}
+            latest
+
+      - name: Build and export to Docker
+        uses: docker/build-push-action@v4
+        with:
+          context: .
+          file: docker/Dockerfile
+          load: true
+          push: false
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}
+          build-args: |
+            VERSION=${{ steps.latest-version.outputs.VERSION }}
+
+      - name: Test image
+        run: |
+          mkdir oe_license_dir
+          # Write out license to a file
+          echo "${{ secrets.OE_LICENSE }}" > oe_license_dir/oe_license.txt
+          docker run --rm ${{ steps.fqirp.outputs.FQIRP }} perses-cli --help
+          # Mount oe_license_dir (which should have the license file) to /openeye on container for testing
+          #docker run -e GITHUB_ACTIONS=true --mount type=bind,source=$PWD/oe_license_dir/,target=/openeye/,readonly --rm ${{ steps.fqirp.outputs.FQIRP }} pytest --pyargs perses -a "not advanced" -n auto -m "not gpu_needed" -v
+
+      - name: Push Docker image
+        uses: docker/build-push-action@ad44023a93711e3deb337508980b4b5e9bcdc5dc
+        with:
+          context: .
+          file: docker/Dockerfile
+          push: true
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}
+          build-args: |
+            VERSION=${{ steps.latest-version.outputs.VERSION }}
+
+      - name: Setup Apptainer
+        uses: eWaterCycle/setup-apptainer@v2
+        with:
+          apptainer-version: 1.1.2
+
+      - name: Build Apptainer Image
+        run: singularity build perses_${{ steps.latest-version.outputs.VERSION }}.sif docker-daemon:${{ steps.fqirp.outputs.FQIRP }}
+
+      - name: Test & Push Apptainer Image
+        run: |
+          set -e 
+          mkdir test_apptainer
+          cd test_apptainer
+          singularity run ../perses_${{ steps.latest-version.outputs.VERSION }}.sif perses-cli --help
+          #singularity run --bind ../oe_license_dir/:/openeye/ --env "GITHUB_ACTIONS=true" ../perses_${{ steps.latest-version.outputs.VERSION }}.sif pytest --pyargs perses -a "not advanced" -n auto -m "not gpu_needed" -v
+          echo ${{ secrets.GITHUB_TOKEN }} | singularity remote login -u ${{ secrets.GHCR_USERNAME }} --password-stdin oras://ghcr.io
+          singularity push ../perses_${{ steps.latest-version.outputs.VERSION }}.sif oras://${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ steps.latest-version.outputs.VERSION }}-apptainer
+          singularity push ../perses_${{ steps.latest-version.outputs.VERSION }}.sif oras://${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest-apptainer
diff --git a/.github/workflows/self-hosted-gpu-test.yml b/.github/workflows/self-hosted-gpu-test.yml
@@ -3,6 +3,7 @@ on:
   push:
     branches:
       - main
+      - "0.10.x"
   workflow_dispatch:
   schedule:
     # nightly tests
@@ -68,6 +69,7 @@ jobs:
             python==3.9
             openmm>=8.0
             cudatoolkit==11.7
+            pymbar<4
       
       - name: Additional info about the build
         run: |

diff --git a/devtools/conda-envs/test_env.yaml b/devtools/conda-envs/test_env.yaml
@@ -30,11 +30,11 @@ dependencies:
   - openmmforcefields >=0.9.0
   - openmmtools >=0.23 # may need to sort out ambermini/ambertools/parmed dependencies
   - openmoltools # may need to sort out ambermini/ambertools/parmed dependencies (we don't want ambertools)
-  - parmed # may need to sort out ambermini/ambertools/parmed dependencies
+  - parmed
   - pdbfixer
   - pip
   - progressbar2
-  - pymbar <4
+  - pymbar
   - pytest
   - pytest-attrib
   - pytest-cov

diff --git a/docker/Dockerfile b/docker/Dockerfile
@@ -0,0 +1,33 @@
+FROM mambaorg/micromamba:latest
+
+LABEL org.opencontainers.image.source = "https://github.com/choderalab/perses"
+LABEL org.opencontainers.image.description="Experiments with expanded ensemble simulation to explore chemical and mutational space."
+LABEL org.opencontainers.image.authors = "John D. Chodera"
+LABEL org.opencontainers.image.licenses=MIT
+# perses version we want to build
+ARG VERSION
+
+# install ps
+USER root
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    procps \
+    && rm -rf /var/lib/apt/lists/*
+USER $MAMBA_USER
+
+# Don't buffer stdout & stderr streams, so if there is a crash no partial buffer output is lost
+# https://docs.python.org/3/using/cmdline.html#cmdoption-u
+ENV PYTHONUNBUFFERED=1
+
+RUN micromamba install -y -n base -c conda-forge -c openeye openeye-toolkits git pytest-xdist pytest-attrib mpiplus mpich mpi mpi4py clusterutils  "perses==$VERSION" cudatoolkit=11.2 && \
+    micromamba clean --all --yes
+
+# Make directory and tell openeye where to find
+# license file
+USER root
+RUN mkdir /openeye
+USER $MAMBA_USER
+ENV OE_LICENSE=/openeye/oe_license.txt
+
+# Ensure that conda environment is automatically activated
+# https://github.com/mamba-org/micromamba-docker#running-commands-in-dockerfile-within-the-conda-environment
+ARG MAMBA_DOCKERFILE_ACTIVATE=1
diff --git a/docker/Dockerfile-dev b/docker/Dockerfile-dev
diff --git a/docker/Dockerfile-dev-cuda-11 b/docker/Dockerfile-dev-cuda-11
diff --git a/docker/Dockerfile-dev-cuda-11-openmm-dev b/docker/Dockerfile-dev-cuda-11-openmm-dev