Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: build wheels #416

Merged
merged 31 commits into from
Jan 24, 2025
Merged
Show file tree
Hide file tree
Changes from 27 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
9925786
feat: build wheels
beckermr Jan 17, 2025
6f3d8f6
fix: no more mambaforge
beckermr Jan 17, 2025
5f8cdc5
fix: always build wheels for testing
beckermr Jan 17, 2025
aa10815
fix: only publish for release events
beckermr Jan 17, 2025
61676ef
fix: add bzip2 for linux
beckermr Jan 17, 2025
e2659a9
fix: typo in github context name
beckermr Jan 17, 2025
11c6a0a
fix: need single quotes for gha
beckermr Jan 17, 2025
3d8abf7
fix: need to have bzip2 around now I guess?
beckermr Jan 17, 2025
1e59136
fix: use bzip2 from conda
beckermr Jan 17, 2025
8909c8c
prod: update setup-miniconda
beckermr Jan 17, 2025
2d00f8c
fix: use yum instead for bzip2
beckermr Jan 17, 2025
4ee8c05
prod: update gha versions and syntax
beckermr Jan 17, 2025
d8f5a00
fix: ubuntu uses apt :facepalm:
beckermr Jan 17, 2025
f0de506
fix: try this name
beckermr Jan 17, 2025
db0f3d8
fix: use correct bzip2 dev package for wheels just in case
beckermr Jan 17, 2025
0151c02
Add concurrency group to wheel workflow
beckermr Jan 21, 2025
955607d
feat: add dependabot
beckermr Jan 21, 2025
31dcf40
fix: set python version in wheel build
beckermr Jan 21, 2025
16a66d3
fix: spacing
beckermr Jan 21, 2025
d18597e
fix: add support for workflow dispatch
beckermr Jan 22, 2025
6379942
test: add tests for wheels
beckermr Jan 23, 2025
22417b8
test: add tests for other wheels
beckermr Jan 23, 2025
42e1f52
fix: run tests for alpine
beckermr Jan 23, 2025
34b0952
fix: test w/e version of python we have for musl
beckermr Jan 23, 2025
db9f140
debug: what is this
beckermr Jan 23, 2025
d0fa875
fix: use a venv
beckermr Jan 23, 2025
441ff81
fix: buggy activation
beckermr Jan 23, 2025
995befb
Update wheel.yml
beckermr Jan 23, 2025
cab9af6
feat: skip existing things
beckermr Jan 23, 2025
7ef1142
Merge branch 'beckermr-patch-1' of https://github.com/esheldon/fitsio…
beckermr Jan 23, 2025
76758cc
feat: add token secret for publishing
beckermr Jan 23, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
version: 2
updates:
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "monthly"
groups:
github-actions:
patterns:
- '*'
22 changes: 13 additions & 9 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,10 @@ on:
env:
PY_COLORS: "1"

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
tests:
name: tests
Expand All @@ -22,21 +26,15 @@ jobs:
runs-on: ${{ matrix.os }}

steps:
- name: cancel previous runs
uses: styfle/cancel-workflow-action@0.6.0
with:
access_token: ${{ github.token }}

- uses: actions/checkout@v2
- uses: actions/checkout@v4

- uses: conda-incubator/setup-miniconda@v2
- uses: conda-incubator/setup-miniconda@d2e6a045a86077fb6cad6f5adf368e9076ddaa8d # v3.1.0
with:
python-version: ${{ matrix.pyver }}
channels: conda-forge/label/numpy_rc,conda-forge,defaults
channels: conda-forge
channel-priority: strict
show-channel-urls: true
miniforge-version: latest
miniforge-variant: Mambaforge

- name: install conda deps
shell: bash -l {0}
Expand All @@ -47,6 +45,12 @@ jobs:
mamba install importlib_resources
fi

- name: install bzip2 on linux
shell: bash -l {0}
if: matrix.os == 'ubuntu-latest'
run: |
sudo apt-get install libbz2-dev

- name: build external cfitsio
shell: bash -l {0}
run: |
Expand Down
216 changes: 216 additions & 0 deletions .github/workflows/wheel.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,216 @@
# this is based on the wheel workflow in GalSim, but adapted to our needs
name: build wheels and sdist

on:
workflow_dispatch:
inputs:
ref:
description: 'The git ref to build wheels for. This will trigger a pypi upload.'
default: ''
required: false
type: string
cibw_skip:
description: 'Python versions to skip when building wheels.'
default: 'cp36* cp37* pp* cp38*'
required: false
type: string
pull_request: null
release:
types:
- published

concurrency:
group: pypi
cancel-in-progress: false

beckermr marked this conversation as resolved.
Show resolved Hide resolved
env:
PYVER: '3.11'
beckermr marked this conversation as resolved.
Show resolved Hide resolved
CIBW_SKIP_VAL: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.cibw_skip || 'cp36* cp37* pp* cp38*' }}

jobs:
linux-manylinux:
name: linux-manylinux
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.ref || '' }}

- uses: actions/setup-python@v5
beckermr marked this conversation as resolved.
Show resolved Hide resolved
with:
python-version: '${{ env.PYVER }}'

- name: build wheels
uses: pypa/cibuildwheel@v2.21.1
env:
CIBW_BUILD: "*manylinux*"
CIBW_ARCHS: auto64
CIBW_SKIP: ${{ env.CIBW_SKIP_VAL }}
# I think yum might always work here. But leave all options available.
CIBW_BEFORE_ALL: yum install -y bzip2-devel || apt-get install libbz2-dev || apk add --upgrade bzip2-dev
esheldon marked this conversation as resolved.
Show resolved Hide resolved

- name: test wheel for python ${{ env.PYVER }}
run: |
pystr='${{ env.PYVER }}'
pystr=${pystr//./}
python -m pip install pip
pip install numpy pytest
pip install ./wheelhouse/*cp${pystr}*.whl
pytest --pyargs fitsio

- uses: actions/upload-artifact@v4
with:
name: whl-linux
path: ./wheelhouse/*.whl

linux-musl:
name: linux-musl
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- uses: actions/setup-python@v5
beckermr marked this conversation as resolved.
Show resolved Hide resolved
with:
python-version: '${{ env.PYVER }}'

- name: build wheels
uses: pypa/cibuildwheel@v2.21.1
env:
CIBW_BUILD: "*musllinux*"
CIBW_ARCHS: auto64
CIBW_SKIP: ${{ env.CIBW_SKIP_VAL }}
# I think musl always uses apk, but keep all options available.
CIBW_BEFORE_ALL: yum install -y bzip2-devel || apt-get install libbz2-dev || apk add --upgrade bzip2-dev

- uses: jirutka/setup-alpine@v1
with:
packages: "bzip2-dev python3 py3-pip py3-numpy"

- name: test wheel for python
shell: alpine.sh {0}
run: |
set -ex
beckermr marked this conversation as resolved.
Show resolved Hide resolved
python --version
pystr=$(python --version | cut -d' ' -f 2 | cut -d'.' -f 1)$(python --version | cut -d' ' -f 2 | cut -d'.' -f 2)
mkdir test-venv
python3 -m venv test-venv
. test-venv/bin/activate
pip install numpy pytest
pip install ./wheelhouse/*cp${pystr}*musl*.whl
pytest --pyargs fitsio
deactivate

- uses: actions/upload-artifact@v4
with:
name: whl-musl
path: ./wheelhouse/*.whl

osx-intel:
name: osx-intel
runs-on: macos-13
steps:
- uses: actions/checkout@v4

- uses: actions/setup-python@v5
beckermr marked this conversation as resolved.
Show resolved Hide resolved
with:
python-version: '${{ env.PYVER }}'

- name: build wheels
uses: pypa/cibuildwheel@v2.21.1
env:
CIBW_BUILD: "*macosx*"
CIBW_ARCHS: auto64
CIBW_SKIP: ${{ env.CIBW_SKIP_VAL }}
# CIBW_BEFORE_ALL: brew install fftw || true
CIBW_ENVIRONMENT: >-
MACOSX_DEPLOYMENT_TARGET=13.0

- name: test wheel for python ${{ env.PYVER }}
run: |
pystr='${{ env.PYVER }}'
pystr=${pystr//./}
python -m pip install pip
pip install numpy pytest
pip install ./wheelhouse/*cp${pystr}*.whl
pytest --pyargs fitsio

- uses: actions/upload-artifact@v4
with:
name: whl-macos
path: ./wheelhouse/*.whl

osx-arm:
name: osx-arm
runs-on: macos-latest
steps:
- uses: actions/checkout@v4

- uses: actions/setup-python@v5
beckermr marked this conversation as resolved.
Show resolved Hide resolved
with:
python-version: '${{ env.PYVER }}'

- name: build wheels
uses: pypa/cibuildwheel@v2.21.1
env:
CIBW_BUILD: "*macosx*"
CIBW_ARCHS: arm64
CIBW_SKIP: ${{ env.CIBW_SKIP_VAL }}
# CIBW_BEFORE_ALL: brew install llvm libomp fftw eigen
CIBW_ENVIRONMENT: >-
MACOSX_DEPLOYMENT_TARGET=14.7

- name: test wheel for python ${{ env.PYVER }}
run: |
pystr='${{ env.PYVER }}'
pystr=${pystr//./}
python -m pip install pip
pip install numpy pytest
pip install ./wheelhouse/*cp${pystr}*.whl
pytest --pyargs fitsio

- uses: actions/upload-artifact@v4
with:
name: whl-arm
path: ./wheelhouse/*.whl

sdist:
name: sdist
needs: [linux-manylinux, linux-musl, osx-intel, osx-arm]
# Just need to build sdist on a single machine
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- uses: actions/setup-python@v5

- name: Install dependencies
run: |
python -m pip install -U pip
pip install -U numpy setuptools

- name: download wheels
uses: actions/download-artifact@v4
with:
path: ./wheels
pattern: whl-*
merge-multiple: true

- name: build sdist
run: |
python setup.py sdist
ls -l dist
tar tvfz dist/*.tar.gz

- name: copy wheels to dist
run: |
echo ls -l wheels
ls -l wheels
cp wheels/*.whl dist
echo ls -l dist
ls -l dist

- name: publish to pypi
uses: pypa/gh-action-pypi-publish@release/v1
if: github.event_name == 'release' || github.event.action == 'workflow_dispatch'
with:
verbose: true
Loading