Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use newer OpenMPI version on Ubuntu in CI #3121

Draft
wants to merge 23 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 15 additions & 3 deletions .github/workflows/coverage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,15 +37,27 @@ jobs:
env:
DISPLAY: ${{ ':0' }}
MUSIC_INSTALL_DIR: /opt/MUSIC
MUSIC_VERSION: 1.2.0
MUSIC_VERSION: 1.2.1

steps:

- name: Install apt packages
run: |
sudo apt-get install xfonts-100dpi build-essential doxygen lcov libboost-all-dev libopenmpi-dev libmpich-dev libx11-dev libxcomposite-dev mpich openmpi-bin gpg ninja-build flex bison libfl-dev
sudo apt-get install xfonts-100dpi build-essential doxygen lcov libboost-all-dev libmpich-dev libx11-dev libxcomposite-dev mpich gpg ninja-build flex bison libfl-dev
shell: bash

- name: Install a new version of OpenMPI
Copy link

@dalcinl dalcinl Oct 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Open MPI does not yet support MPI 4. There is no point on installing it from sources, that will not fix the underlying issue. The right fix is to fix MUSIC as per my advice elsewhere to support both MPI-3 (current Open MPI) and MPI-4 (current MPICH) implementations.

# The version of OpenMPI on Ubuntu 22.04 does not support MPI 4 features, so we install it manually
run: |
sudo apt-get purge -y libopenmpi-dev openmpi-bin
curl -L -o openmpi.tar.gz https://download.open-mpi.org/release/open-mpi/v5.0/openmpi-5.0.5.tar.gz
tar -xf openmpi.tar.gz
cd openmpi-5.0.5/
./configure --prefix=/usr/
make -j
sudo make -j install
working-directory: ${{runner.temp}}

- name: Setup Caliper profiler
run: |
git clone https://github.com/LLNL/Caliper.git
Expand All @@ -58,7 +70,7 @@ jobs:
run: |
python3 -m venv music-venv
source music-venv/bin/activate
python3 -m pip install 'mpi4py<4' cython numpy setuptools
python3 -m pip install mpi4py cython numpy setuptools
sudo mkdir -p $MUSIC_INSTALL_DIR
sudo chown -R $USER $MUSIC_INSTALL_DIR
curl -L -o MUSIC.zip https://github.com/INCF/MUSIC/archive/refs/tags/${MUSIC_VERSION}.zip
Expand Down
14 changes: 13 additions & 1 deletion .github/workflows/neuron-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,18 @@ jobs:
python -m pip install --upgrade -r ci_requirements.txt
python -m pip install --upgrade pip -r nrn_requirements.txt

- name: Install a new version of OpenMPI
# The version of OpenMPI on Ubuntu 22.04 does not support MPI 4 features, so we install it manually
run: |
sudo apt-get purge -y libopenmpi-dev openmpi-bin
curl -L -o openmpi.tar.gz https://download.open-mpi.org/release/open-mpi/v5.0/openmpi-5.0.5.tar.gz
tar -xf openmpi.tar.gz
cd openmpi-5.0.5/
./configure --prefix=/usr/
make -j
sudo make -j install
working-directory: ${{runner.temp}}

- name: Install a new automake
# A automake >= 1.16.5 is needed for python 3.12 because it generates a python script
# called py-compile and the original one is not supporting this version of python
Expand All @@ -207,7 +219,7 @@ jobs:
run: |
python3 -m venv music-venv
source music-venv/bin/activate
python3 -m pip install 'mpi4py<4' cython numpy setuptools
python3 -m pip install mpi4py cython numpy setuptools
sudo mkdir -p $MUSIC_INSTALL_DIR
sudo chown -R $USER $MUSIC_INSTALL_DIR
curl -L -o MUSIC.zip https://github.com/INCF/MUSIC/archive/refs/tags/${MUSIC_VERSION}.zip
Expand Down
2 changes: 1 addition & 1 deletion nrn_requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,6 @@ cython
packaging
pytest<=8.1.1 # potential bug from 8.2.0 due to parallelism?
pytest-cov
mpi4py<4 # MUSIC not compatible with MPI 4
mpi4py
numpy
find_libpython
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ class Components:
Components.MUSIC = True
sys.argv.remove("--enable-music")

if Components.RX3D:
if Components.RX3D or Components.MUSIC:
try:
from Cython.Distutils import Extension as CyExtension
from Cython.Distutils import build_ext
Expand Down
Loading