Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build and test new wheels (Python 3.10+ and manylinux tag 2.28 instead of 2.24) #165

Merged
merged 5 commits into from
Aug 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/workflows/actions.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,8 @@ jobs:
run: |
set -xe
COMMIT_MSG=$(git log --no-merges -1 --oneline)
echo "::set-output name=message::$COMMIT_MSG"
echo "message=$COMMIT_MSG" >> $GITHUB_OUTPUT
echo github.ref ${{ github.ref }}

trexio_ubuntu:
name: x86 Ubuntu 20.04
Expand Down
56 changes: 28 additions & 28 deletions .github/workflows/build-wheels.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ on:
# Run this workflow after the TREXIO CI completed
workflow_run:
workflows: [ "TREXIO CI" ]
branches: [ master ]
branches: [ test-new-wheels ]
types:
- completed

Expand All @@ -16,12 +16,12 @@ jobs:
get_commit_message:
if: ${{ github.event.workflow_run.conclusion == 'success' }}
name: Get commit message
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
outputs:
message: ${{ steps.commit_message.outputs.message }}
steps:
- name: Checkout the repo
uses: actions/checkout@v3
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
# Gets the correct commit message for pull request
with:
ref: ${{ github.event.pull_request.head.sha }}
Expand All @@ -30,28 +30,28 @@ jobs:
run: |
set -xe
COMMIT_MSG=$(git log --no-merges -1 --oneline)
echo "::set-output name=message::$COMMIT_MSG"

echo "message=$COMMIT_MSG" >> $GITHUB_OUTPUT
echo github.ref ${{ github.ref }}

build_linux_wheels:
name: Build Linux wheels for different versions of CPython on manylinux_x86_64
needs: get_commit_message
if: >-
contains(needs.get_commit_message.outputs.message, '[wheel build]') ||
(github.repository == 'TREX-CoE/trexio' && startsWith(github.ref, 'refs/tags/v'))
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
strategy:
matrix:
manylinux_tag: [2014_x86_64, 2_24_x86_64]
manylinux_tag: [2014_x86_64, 2_28_x86_64]

steps:
- name: Checkout the branch
uses: actions/checkout@v3
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11

- name: Set up Python
uses: actions/setup-python@v4
uses: actions/setup-python@39cd14951b08e74b54015e9e001cdefcf80e669f
with:
python-version: '3.9'
python-version: '3.10'

- name: Install build dependencies
run: python -m pip install -U setuptools
Expand All @@ -65,7 +65,7 @@ jobs:
# Conventional download-artifact action does not work for artifact produced in a different workflow,
# which is the case here (TREXIO CI produced the Python API distribution tarball)
- name: Download the Python API distribution tarball
uses: dawidd6/action-download-artifact@v2
uses: dawidd6/action-download-artifact@v3
with:
# Specify the name of the workflow file which uploaded the tarball
workflow: actions.yml
Expand All @@ -89,7 +89,7 @@ jobs:
working-directory: python

- name: Upload produced wheels as artifacts
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@834a144ee995460fba8ed112a2fc961b36a5ec5a
with:
name: pytrexio-manylinux-${{ matrix.manylinux_tag }}
path: ./python/wheelhouse/*.whl
Expand All @@ -98,27 +98,27 @@ jobs:
publish_wheels:
name: Publish all wheels on PyPI
needs: [build_linux_wheels]
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04

steps:
- name: Checkout the branch
uses: actions/checkout@v3
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11

- name: Set up Python
uses: actions/setup-python@v4
uses: actions/setup-python@39cd14951b08e74b54015e9e001cdefcf80e669f
with:
python-version: '3.9'
python-version: '3.10'

- name: Install build dependencies
run: python -m pip install -U setuptools twine

- name: Download the build artifacts (wheels) of this workflow
uses: actions/download-artifact@v2
uses: actions/download-artifact@v4
with:
path: dist

- name: Download the Python API distribution tarball
uses: dawidd6/action-download-artifact@v2
uses: dawidd6/action-download-artifact@v3
with:
workflow: actions.yml
workflow_conclusion: success
Expand All @@ -135,15 +135,15 @@ jobs:
ls -sh -w 1
working-directory: dist

#- name: Publish distribution 📦 to Test PyPI
# uses: pypa/gh-action-pypi-publish@master
# with:
# password: ${{ secrets.TEST_PYPI_API_TOKEN }}
# repository_url: https://test.pypi.org/legacy/
#verbose: true

- name: Publish distribution 📦 to PyPI
- name: Publish distribution 📦 to Test PyPI
uses: pypa/gh-action-pypi-publish@master
with:
password: ${{ secrets.PYPI_API_TOKEN }}
#if: startsWith(github.ref, 'refs/tags')
password: ${{ secrets.TEST_PYPI_API_TOKEN }}
repository_url: https://test.pypi.org/legacy/
#verbose: true

#- name: Publish distribution 📦 to PyPI
# uses: pypa/gh-action-pypi-publish@master
# with:
# password: ${{ secrets.PYPI_API_TOKEN }}
# #if: startsWith(github.ref, 'refs/tags')
19 changes: 3 additions & 16 deletions python/build_manylinux_wheels.sh
Original file line number Diff line number Diff line change
Expand Up @@ -46,22 +46,9 @@ function build_wheel_for_py()
PYVERSION=${1}

# derive manylinux glibc tag from the PLAT env variable provided to docker run
# this is needed to avoid building wheel for 2010_x86_64 with CPython 3.10
# because NumPy does not have wheels for it
MANYLINUX_TAG=${PLAT:9:4}
if [[ ${PYVERSION} -eq 310 ]] && [[ ${MANYLINUX_TAG} -eq 2010 ]]; then
echo "Skip build of the wheel for CPython 3.10 on manylinux2010_x86_64"
return
fi

# python versions <= 3.7 required additional "m" in the platform tag, e.g. cp37-cp37m
if [[ ${PYVERSION} -eq 36 ]] || [[ ${PYVERSION} -eq 37 ]]; then
PYM="m"
else
PYM=""
fi

CPYTHON="cp${PYVERSION}-cp${PYVERSION}${PYM}"
CPYTHON="cp${PYVERSION}-cp${PYVERSION}"

# create and activate a virtual environment based on CPython version ${PYVERSION}
/opt/python/${CPYTHON}/bin/python3 -m venv --clear trexio-manylinux-py${PYVERSION}
Expand All @@ -88,7 +75,7 @@ function build_wheel_for_py()
pytest -v test/test_api.py

# cleaning
rm -rf -- dist/ build/ trexio.egg-info/
rm -rf -- dist/ build/ trexio.egg-info/
#rm -- test_file_py.h5 unsafe_test_file_py.h5

# deactivate the current environment
Expand All @@ -103,7 +90,7 @@ function build_wheel_for_py()


# build wheels for all versions of CPython in this container
for CPYVERSION in 37 38 39 310
for CPYVERSION in 38 39 310 311 312 313
do
build_wheel_for_py ${CPYVERSION}
done
Loading