Skip to content

Conversation

avivajpeyi
Copy link
Collaborator

@avivajpeyi avivajpeyi commented Aug 19, 2025

The linux installs take up to 4 min... Maybe we can use the COMPAS docker-container to speed things up?
Requires #1426

Screenshot from 2025-08-19 20-27-19

@avivajpeyi
Copy link
Collaborator Author

Ok for future ref, the clean way fails

    name: Build COMPAS
    runs-on: ubuntu-22.04
    container: 
      image: teamcompas/compas:latest
      options: --user root  

with error

>> Run actions/checkout@v4
..
/usr/bin/docker exec d06cab306f0aa228341e70597fec289684cbb920669bdbb9620aeff11937979d sh -c "cat /etc/release | grep ^ID"
/e/node20/bin/node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /e/node20/bin/node)

This means that the teamcompas/compas:latest container has an older version of glibc that's incompatible with GitHub Actions' Node.js .... What the heck.

But we can do something hacky instead:

run: |
          echo "Building COMPAS using teamcompas/compas:latest container..."
          docker run --rm \
            -v ${{ github.workspace }}:/workspace \
            -w /workspace \
            teamcompas/compas:latest \
            sh -c "cd src && make clean 2>/dev/null || echo 'No existing build to clean' && make -j \$(nproc) -f Makefile && ./COMPAS -v"

This basically gets the compas image and builds using the image, rather than using the image as a container.
Ie just uses the installs present in the COMPAS image, but then still builds on the host.

Kindof like a module load on slurm.

The reason we'd want this is to speed up the CI -- now we wont need to spend the 3 min doing Linux installs.

@avivajpeyi
Copy link
Collaborator Author

Darn! So close!

The COMPAS build works


COMPAS v03.23.01 (gsl v2.4, boost v1.65.1, HDF5 v1.10.0)
Compact Object Mergers: Population Astrophysics and Statistics
by Team COMPAS (http://compas.science/index.html)
A binary star simulator

Go to https://compas.readthedocs.io/en/latest/index.html for the online documentation
Check https://compas.readthedocs.io/en/latest/pages/whats-new.html to see what's new in the latest release

COMPAS build completed successfully!

But then i think we're still missing some libraires (the hdf5 lib) when we run COMPAS detailed plotter:

Run export COMPAS_EXECUTABLE_PATH=/home/runner/work/COMPAS/COMPAS/src/COMPAS
Contents of run.sh:
#bin/bash
echo ">>>  GENERATING TEST COMPAS DATA <<<"
# if $COMPAS_EXECUTABLE_PATH not set, set to ../../src/COMPAS
COMPAS_EXECUTABLE_PATH=${COMPAS_EXECUTABLE_PATH:-../../src/COMPAS}
$COMPAS_EXECUTABLE_PATH \
  -n 2 \
  --initial-mass-1 35 \
  --initial-mass-2 31 \
  -a 3.5 \
  --random-seed 0 \
  --metallicity 0.001 \
  --detailed-output \
  > compas_run.log
cat compas_run.log
echo "Generating detailed evolution plot"
compas_plot_detailed_evolution "./COMPAS_Output/Detailed_Output/BSE_Detailed_Output_0.h5" --dont-show >> detailed_evolution.log
echo "Out files:"
ls -l
echo ">>> DONE <<<"
Running COMPAS example...
>>>  GENERATING TEST COMPAS DATA <<<
/home/runner/work/COMPAS/COMPAS/src/COMPAS: error while loading shared libraries: libhdf5_serial.so.100: cannot open shared object file: No such file or directory
Generating detailed evolution plot
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.9.23/x64/bin/compas_plot_detailed_evolution", line 7, in <module>
    sys.exit(main())
  File "/home/runner/work/COMPAS/COMPAS/compas_python_utils/detailed_evolution_plotter/plot_detailed_evolution.py", line 28, in main
    run_main_plotter(args.data_path, outdir=args.outdir, show=args.dont_show, use_latex=True)
  File "/home/runner/work/COMPAS/COMPAS/compas_python_utils/detailed_evolution_plotter/plot_detailed_evolution.py", line 34, in run_main_plotter
    RawData = h5.File(data_path, 'r')
  File "/opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/site-packages/h5py/_hl/files.py", line 564, in __init__
    fid = make_fid(name, mode, userblock_size, fapl, fcpl, swmr=swmr)
  File "/opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/site-packages/h5py/_hl/files.py", line 238, in make_fid
    fid = h5f.open(name, flags, fapl=fapl)
  File "h5py/_objects.pyx", line 56, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 57, in h5py._objects.with_phil.wrapper
  File "h5py/h5f.pyx", line 102, in h5py.h5f.open
FileNotFoundError: [Errno 2] Unable to synchronously open file (unable to open file: name = './COMPAS_Output/Detailed_Output/BSE_Detailed_Output_0.h5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)
Out files:
total 8
-rw-r--r-- 1 runner docker 424 Aug 19 08:00 README.md
-rw-r--r-- 1 runner docker   0 Aug 19 08:06 compas_run.log
-rw-r--r-- 1 runner docker   0 Aug 19 08:06 detailed_evolution.log
-rwxr-xr-x 1 runner docker 595 Aug 19 08:00 run.sh
>>> DONE <<<

rror while loading shared libraries: libhdf5_serial.so.100: cannot open shared object file: No such file or directory

@avivajpeyi
Copy link
Collaborator Author

avivajpeyi commented Aug 19, 2025

@jeffriley , any tips? Im guessing its cause we might need the docker env to run compas as well?

@avivajpeyi
Copy link
Collaborator Author

Arrrghh, that was working, but now its an issue with the image having an old python...

=== Installing Python dependencies ===
Collecting pip
  Downloading https://files.pythonhosted.org/packages/a4/6d/6463d49a933f547439d6b5b98b46af8742cc03ae83543e4d7688c2420f8b/pip-21.3.1-py3-none-any.whl (1.7MB)
Installing collected packages: pip
  Found existing installation: pip 9.0.1
    Not uninstalling pip at /usr/lib/python3/dist-packages, outside environment /usr
Successfully installed pip-21.3.1
Obtaining file:///workspace
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'error'
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/workspace/setup.py'"'"'; __file__='"'"'/workspace/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-luksy427
       cwd: /workspace/
  Complete output (1 lines):
  Python < 3.8 is not supported, aborting setup

Copy link

badge

✅ COMPAS Build Successful!

Item Value
Commit eca86c8
Logs View workflow

📊 Detailed Evolution Plot

Click to view evolution plot

Evolution Plot


Generated by COMPAS CI

Copy link

badge

✅ COMPAS Build Successful!

Item Value
Commit 2b40b40
Logs View workflow

Detailed Evolution Plot

Click to view evolution plot

Evolution Plot


Generated by COMPAS CI

@avivajpeyi avivajpeyi marked this pull request as ready for review August 20, 2025 05:06
@avivajpeyi
Copy link
Collaborator Author

Using containers speeds up the CI by ~4 min!
Screenshot from 2025-08-20 17-06-34

Copy link
Collaborator

@ilyamandel ilyamandel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @avivajpeyi ,
Happy to approve, leave it to you to decide when it's appropriate to pull into dev.

@ilyamandel
Copy link
Collaborator

@avivajpeyi -- a ping that this has been approved and is waiting for you to decide whether to pull it.

@avivajpeyi
Copy link
Collaborator Author

Ah woops! sorry @ilyamandel .

Ill double-check this and merge it tomorrow if it makes sense

Copy link

✅ COMPAS Build Successful!

Item Value
Commit 88dcae8
Logs View workflow

📊 Detailed Evolution Plot

Click to view evolution plot


Generated by COMPAS CI with CML

Copy link

✅ COMPAS Build Successful!

Item Value
Commit 7cf9306
Logs View workflow

Detailed Evolution Plot

Click to view evolution plot


Generated by COMPAS CI

@avivajpeyi avivajpeyi merged commit 428dc2d into dev Sep 11, 2025
4 checks passed
@avivajpeyi avivajpeyi deleted the use_compas_docker_image_to_avoid_linux_installs_on_ci branch September 11, 2025 01:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants