Skip to content

Commit

Permalink
Trying to fix CI issues
Browse files Browse the repository at this point in the history
- gcc 13.3.0 seems to have an issue with math.h on macOS. It spews a
  bunch of errors about the API_DEPRECATED macro not being used
  correctly, which is not an issue on our side. gcc 13.2.0 does not
  seem to have this issue, but since we cannot select gcc with such
  granularity we downgrade to gcc 12 and hope this fixes the issue for
  now.

- numpy 2.0.0 has come out which causes runtime issues. Fortunately the
  fix is easy, just build with pybind11 >=2.12.0. However this is still
  an issue for the pdfo compatibility test since pdfo does not support
  numpy 2.0.0, and sometimes the tests install 2.0.0 because it happens
  to be in the pip cache. So we upgrade pybind11 (the new version is
  compatible with 2.0 and 1.0 numpy) and skip the pdfo test if we
  happened to pick up numpy 2.

- macOS builds for building Python wheels started to complain when running
  the "delocate" step that libquadmath, libstdc++, and libgfortran have
  minimum targets of 11.0, whereas the default was 10.9, hence we added
  the relevant env variable for CIBW.

- Notes have been made in build_python.yml for failures related to
  Ubuntu 24.04.

- A couple tests were failing on macOS 14, so I modified the rhoend and
  rhobeg values to try to get it to converge. The test was surprisingly
  sensitive to these values.
  • Loading branch information
nbelakovski committed Jun 16, 2024
1 parent be6c3d0 commit 01b328b
Show file tree
Hide file tree
Showing 6 changed files with 28 additions and 8 deletions.
1 change: 1 addition & 0 deletions .github/actions/spelling/allow.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2261,3 +2261,4 @@ TESTQUAD
WAITALL
xcodebuild
ICX
sw_vers

Check warning

Code scanning / check-spelling

Non-alpha in dictionary Warning

Ignoring entry because it contains non-alpha characters. (non-alpha-in-dictionary)
15 changes: 14 additions & 1 deletion .github/workflows/build_python.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,11 @@ jobs:
fail-fast: false
matrix:
# As of 20240501, macos-11/12/13 are AMD64, and macOS-14 is ARM64.
os: [ubuntu-20.04, ubuntu-22.04, ubuntu-24.04, windows-2019, windows-2022, macos-11, macos-12, macos-13, macos-14]
# As of 20240616, ubuntu 24.04 is failing to copy files into the quay.io/pypa/manylinux2014_i686:2024-05-13-0983f6f container
# It fails with "tar: ./todo: Cannot utime: Function not implemented" for every file in the repo.
# This is strange since it worked for the quay.io/pypa/manylinux2014_x86_64:2024-05-13-0983f6f container.
# I suggest trying ubuntu-24.04 again in a month.
os: [ubuntu-20.04, ubuntu-22.04, windows-2019, windows-2022, macos-11, macos-12, macos-13, macos-14]

steps:
- name: Clone Repository (Latest)
Expand All @@ -46,11 +50,20 @@ jobs:
- name: Checkout pybind11 submodule
run: git submodule update --init python/pybind11

- name: Set the MACOSX_DEPLOYMENT_TARGET
if: ${{ runner.os == 'macOS' }}
run: |
MACOSX_DEPLOYMENT_TARGET=$(sw_vers -productVersion | cut -d'.' -f 1)
echo "MACOSX_DEPLOYMENT_TARGET is $MACOSX_DEPLOYMENT_TARGET"
echo "MACOSX_DEPLOYMENT_TARGET=$MACOSX_DEPLOYMENT_TARGET" >> $GITHUB_ENV
- name: Set up Fortran
uses: fortran-lang/setup-fortran@main
if: ${{ runner.os == 'macOS' }}
with:
compiler: gcc
version: 12

# Copied from https://github.com/scipy/scipy/blob/main/.github/workflows/wheels.yml
# For rtools, see https://github.com/r-windows/rtools-installer/releases, which has been
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/cmake.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,8 @@ jobs:
toolchain:
- {compiler: gcc, version: 11, cflags: '-Wall -Wextra -Wpedantic -Werror', fflags: '-Wall -Wextra -Wpedantic -Werror -fimplicit-none -fcheck=all -fstack-check -Wno-function-elimination'}
- {compiler: gcc, version: 12, cflags: '-Wall -Wextra -Wpedantic -Werror', fflags: '-Wall -Wextra -Wpedantic -Werror -fimplicit-none -fcheck=all -fstack-check -Wno-function-elimination'}
- {compiler: gcc, version: 13, cflags: '-Wall -Wextra -Wpedantic -Werror', fflags: '-Wall -Wextra -Wpedantic -Werror -fimplicit-none -fcheck=all -fstack-check -Wno-function-elimination'}
# As of 20240616 gcc13 has issues importing math.h on macOS.
# - {compiler: gcc, version: 13, cflags: '-Wall -Wextra -Wpedantic -Werror', fflags: '-Wall -Wextra -Wpedantic -Werror -fimplicit-none -fcheck=all -fstack-check -Wno-function-elimination'}
- {compiler: intel-classic, version: '2021.8', cflags: '-diag-disable=10441 -Wall -w3 -Werror-all', fflags: '-warn all -debug extended -fimplicit-none -standard-semantics'}
- {compiler: intel-classic, version: '2021.9', cflags: '-diag-disable=10441 -Wall -w3 -Werror-all', fflags: '-warn all -debug extended -fimplicit-none -standard-semantics'}
- {compiler: intel-classic, version: '2021.10', cflags: '-diag-disable=10441 -Wall -w3 -Werror-all', fflags: '-warn all -debug extended -fimplicit-none -standard-semantics'}
Expand Down
2 changes: 1 addition & 1 deletion python/pybind11
Submodule pybind11 updated 91 files
+1 −1 .github/CONTRIBUTING.md
+9 −1 .github/dependabot.yml
+111 −74 .github/workflows/ci.yml
+3 −3 .github/workflows/configure.yml
+4 −4 .github/workflows/format.yml
+1 −1 .github/workflows/labeler.yml
+8 −8 .github/workflows/pip.yml
+7 −7 .github/workflows/upstream.yml
+21 −19 .pre-commit-config.yaml
+19 −2 .readthedocs.yml
+57 −6 CMakeLists.txt
+5 −4 README.rst
+1 −1 docs/advanced/embedding.rst
+11 −11 docs/advanced/exceptions.rst
+1 −1 docs/advanced/functions.rst
+29 −0 docs/advanced/misc.rst
+1 −1 docs/benchmark.py
+182 −2 docs/changelog.rst
+12 −4 docs/compiling.rst
+86 −42 docs/release.rst
+28 −0 docs/upgrade.rst
+142 −9 include/pybind11/cast.h
+18 −13 include/pybind11/detail/class.h
+16 −4 include/pybind11/detail/common.h
+1 −1 include/pybind11/detail/init.h
+16 −5 include/pybind11/detail/internals.h
+51 −10 include/pybind11/detail/type_caster_base.h
+3 −2 include/pybind11/eigen/tensor.h
+2 −1 include/pybind11/functional.h
+9 −1 include/pybind11/gil.h
+91 −0 include/pybind11/gil_safe_call_once.h
+157 −22 include/pybind11/numpy.h
+121 −48 include/pybind11/pybind11.h
+22 −5 include/pybind11/pytypes.h
+8 −7 include/pybind11/stl.h
+28 −56 include/pybind11/stl_bind.h
+125 −0 include/pybind11/typing.h
+1 −1 noxfile.py
+1 −1 pybind11/_version.py
+3 −1 pybind11/setup_helpers.py
+8 −11 pyproject.toml
+9 −4 tests/CMakeLists.txt
+1 −0 tests/conftest.py
+2 −2 tests/cross_module_interleaved_error_already_set.cpp
+2 −0 tests/extra_python_package/test_files.py
+4 −4 tests/pybind11_cross_module_tests.cpp
+9 −3 tests/pybind11_tests.cpp
+14 −8 tests/requirements.txt
+7 −0 tests/test_buffers.py
+2 −2 tests/test_builtin_casters.py
+7 −0 tests/test_callbacks.py
+14 −0 tests/test_class.py
+1 −2 tests/test_cmake_build/CMakeLists.txt
+3 −3 tests/test_cmake_build/installed_embed/CMakeLists.txt
+3 −3 tests/test_cmake_build/installed_function/CMakeLists.txt
+3 −3 tests/test_cmake_build/installed_target/CMakeLists.txt
+9 −3 tests/test_cmake_build/subdirectory_embed/CMakeLists.txt
+9 −3 tests/test_cmake_build/subdirectory_function/CMakeLists.txt
+9 −3 tests/test_cmake_build/subdirectory_target/CMakeLists.txt
+4 −0 tests/test_constants_and_functions.cpp
+12 −0 tests/test_custom_type_casters.cpp
+17 −0 tests/test_eigen_matrix.cpp
+8 −1 tests/test_eigen_matrix.py
+6 −3 tests/test_enum.py
+54 −13 tests/test_exceptions.cpp
+1 −1 tests/test_exceptions.h
+21 −2 tests/test_exceptions.py
+2 −2 tests/test_factory_constructors.py
+46 −0 tests/test_kwargs_and_defaults.cpp
+37 −1 tests/test_kwargs_and_defaults.py
+19 −15 tests/test_methods_and_attributes.py
+9 −3 tests/test_numpy_array.py
+29 −4 tests/test_numpy_dtypes.cpp
+11 −3 tests/test_numpy_dtypes.py
+45 −0 tests/test_python_multiple_inheritance.cpp
+35 −0 tests/test_python_multiple_inheritance.py
+26 −3 tests/test_pytypes.cpp
+57 −1 tests/test_pytypes.py
+19 −0 tests/test_sequences_and_iterators.cpp
+13 −0 tests/test_sequences_and_iterators.py
+15 −12 tests/test_smart_ptr.cpp
+12 −12 tests/test_stl.py
+80 −0 tests/test_stl_binders.cpp
+44 −6 tests/test_stl_binders.py
+2 −2 tests/test_type_caster_pyobject_ptr.cpp
+24 −1 tools/FindPythonLibsNew.cmake
+29 −1 tools/make_changelog.py
+30 −16 tools/pybind11Common.cmake
+1 −1 tools/pybind11Config.cmake.in
+77 −22 tools/pybind11NewTools.cmake
+13 −7 tools/pybind11Tools.cmake
12 changes: 7 additions & 5 deletions python/tests/test_combining_constraints.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,8 @@ def test_providing_bounds_and_nonlinear_constraints():
nlc = prima_NLC(lambda x: x[0]**2, lb=[25], ub=[100])
bounds = prima_Bounds([None, 1], [None, 1])
x0 = [0, 0]
res = prima_minimize(fun, x0, constraints=nlc, bounds=bounds)
# We start with a higher rhobeg to help the optimizer find the optimal solution
res = prima_minimize(fun, x0, constraints=nlc, bounds=bounds, options={'rhobeg': 1.5})
assert np.isclose(res.x[0], 5, atol=1e-6, rtol=1e-6)
assert np.isclose(res.x[1], 1, atol=1e-6, rtol=1e-6)
assert np.isclose(res.fun, 9, atol=1e-6, rtol=1e-6)
Expand All @@ -48,16 +49,17 @@ def newfun(x):
lc = LC(np.array([1,1,1]), lb=10, ub=15)
x0 = [0, 0, 0]
# macOS seems to stop just short of the optimal solution, so we help it along by
# taking a larger initial trust region radius and requiring a smaller final radius
# taking a larger initial trust region radius and requiring a larger final radius
# before stopping. The different packages have different names for these options.
# This test seems surprisingly sensitive to the initial and final trust region radius.
if package == 'prima':
options = {'rhobeg': 2, 'rhoend': 1e-8}
options = {'rhobeg': 2.2, 'rhoend': 1e-5}
method = None
elif package == 'pdfo':
options = {'radius_init': 2, 'radius_final': 1e-8}
options = {'radius_init': 2.2, 'radius_final': 1e-5}
method = None
elif package == 'scipy':
options = {'rhobeg': 2, 'tol': 1e-8}
options = {'rhobeg': 2.2, 'tol': 1e-5}
# PDFO and PRIMA will select COBYLA but SciPy may select something else, so we tell it to select COBYLA
method = 'COBYLA'
else:
Expand Down
3 changes: 3 additions & 0 deletions python/tests/test_compatibility_pdfo.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,9 @@ def test_pdfo():
scipy = pytest.importorskip("scipy")
if version.parse(scipy.__version__) < version.parse("1.11.0"):
pytest.skip("scipy version too old for this test (its version of COBYLA does not accept bounds)")
numpy = pytest.importorskip("numpy")
if version.parse(numpy.__version__) >= version.parse("2.0.0"):
pytest.skip("numpy version too new for this test (pdfo does not yet support numpy v2)")

from pdfo import pdfo
from scipy.optimize import NonlinearConstraint as NLC, LinearConstraint as LC, Bounds
Expand Down

0 comments on commit 01b328b

Please sign in to comment.