feat: support setup helpers

[henryiii]

Moving to the new setup helpers infrastructure being developed in pybind11 2.6.0. Adding cibuildwheel. Adding GHA checks.

There are three methods to use this; I'm using the setup_requires method; while this is discouraged, it works just about everywhere (at least it does not require pip 10 like the PEP 518 method) and it's more complex than the PEP 518 method, which is basically just a simplification on this.

I'll probably refactor after this gets merged to use PEP 518, but for now, it is good to have this in the history, and it would help to have an actual release of pybind11, since most of the work in the PR is getting pip to install from PyPI, which requires CMake and such. I've disabled Python 3.5 + Windows + cibuildwheel, since it does not correctly use PEP 518 due to "portable" install issues. It can handle PEP 518 only in the main project, not a dependency. (We could enable this by splitting the install into cmake first, then pybind11, but it won't matter once we use a released version, or even a beta version from PyPI).

Actions run can be seen here: https://github.com/henryiii/python_example/actions/runs/261642597

I've also added pip and conda builds to Actions; Travis should be able to be removed after this is merged, and appveyer reduced quite a bit. I don't have macOS & Windows conda builds running yet, due primarily to the fact it's hard to setup pybind11 from source since CMake isn't finding a valid compiler. As soon as we have (even a beta) release of pybind11, then it should be easy to add these back in (and simplify everything else). If we wanted to make it work, the conda-forge "compilers" package would probably set up the correct env variables for it to work.

Here's the contents of the file attached to the run (click "artifact" on that page to download):

python_example-0.0.1-cp27-cp27m-macosx_10_9_x86_64.whl
python_example-0.0.1-cp27-cp27m-manylinux2010_i686.whl
python_example-0.0.1-cp27-cp27m-manylinux2010_x86_64.whl
python_example-0.0.1-cp27-cp27m-win_amd64.whl
python_example-0.0.1-cp27-cp27m-win32.whl
python_example-0.0.1-cp27-cp27mu-manylinux2010_i686.whl
python_example-0.0.1-cp27-cp27mu-manylinux2010_x86_64.whl
python_example-0.0.1-cp35-cp35m-macosx_10_9_x86_64.whl
python_example-0.0.1-cp35-cp35m-manylinux2010_i686.whl
python_example-0.0.1-cp35-cp35m-manylinux2010_x86_64.whl
python_example-0.0.1-cp36-cp36m-macosx_10_9_x86_64.whl
python_example-0.0.1-cp36-cp36m-manylinux2010_i686.whl
python_example-0.0.1-cp36-cp36m-manylinux2010_x86_64.whl
python_example-0.0.1-cp36-cp36m-win_amd64.whl
python_example-0.0.1-cp36-cp36m-win32.whl
python_example-0.0.1-cp37-cp37m-macosx_10_9_x86_64.whl
python_example-0.0.1-cp37-cp37m-manylinux2010_i686.whl
python_example-0.0.1-cp37-cp37m-manylinux2010_x86_64.whl
python_example-0.0.1-cp37-cp37m-win_amd64.whl
python_example-0.0.1-cp37-cp37m-win32.whl
python_example-0.0.1-cp38-cp38-macosx_10_9_x86_64.whl
python_example-0.0.1-cp38-cp38-manylinux2010_i686.whl
python_example-0.0.1-cp38-cp38-manylinux2010_x86_64.whl
python_example-0.0.1-cp38-cp38-win_amd64.whl
python_example-0.0.1-cp38-cp38-win32.whl
python_example-0.0.1-cp39-cp39-macosx_10_9_x86_64.whl
python_example-0.0.1-cp39-cp39-manylinux2010_i686.whl
python_example-0.0.1-cp39-cp39-manylinux2010_x86_64.whl
python_example-0.0.1-cp39-cp39-win_amd64.whl
python_example-0.0.1-cp39-cp39-win32.whl
python_example-0.0.1-pp27-pypy_73-macosx_10_9_x86_64.whl
python_example-0.0.1-pp27-pypy_73-manylinux2010_x86_64.whl
python_example-0.0.1-pp27-pypy_73-win32.whl
python_example-0.0.1-pp36-pypy36_pp73-macosx_10_9_x86_64.whl
python_example-0.0.1-pp36-pypy36_pp73-manylinux2010_x86_64.whl
python_example-0.0.1-pp36-pypy36_pp73-win32.whl
python_example-0.0.1.tar.gz

Note that AppVeyor was completely broken; I've remove conda from it, and fixed it for pip builds.

May help #58 (might need support of minGW in setup helpers)
Closes #48
Closes #44
May help #35
Closes #23
May help #21
Closes #16

[henryiii]

@YannickJadoul, in cibuildwheel:

pip wheel /project -w /tmp/cibuildwheel/built_wheel --no-deps

Why --no-deps? This would work fine with PEP 518, but it seems like it should work with setup_requires/install_requires too? (or is it something else going wrong here?). At this point, since I've tested it with this method, I'd also be okay to switch to the "correct" PEP 518 system.

(Looking at the docs, this seems correct, just slightly odd, unless the worry is that extra wheels will be placed in the final directory, but I think that can happen anyway. Might just chalk it up to another reason PEP 518 is better.)

[YannickJadoul]

Why --no-deps?

As far as I know, --no-deps stops pip wheel from downloading/building wheels for all the install_requires dependencies, and thus having many more files that you're actually not trying to build (for example, my project would drag in a numpy wheel, that's already on PyPI anyway).

When the created wheel gets tested with PyPI, pip install ....whl is ran, which installs all dependencies into the temporary virtual environment; so any wheel/package dependencies are downloaded and installed then.

[henryiii]

Okay, but this a) breaks setup_requires, and b) I really think pyproject.toml dependencies that need to be built into wheels to install get thrown here too (even though they are never installed into the active environment, the same wheelhouse directory used for placing built wheels). I'm not really worked about breaking setup_requires, because it's deprecated and a hack anyway. Putting pyproject.toml requirement's build wheels in is irritating, but the process must have them to build, so I don't think there's a better solution (I usually filter wheels based on project name when uploading) and (python -m build probably should solve this properly, I would think, by keeping the "output" directly separate from the wheelhouse).

[YannickJadoul]

Actions are not yet enabled in this repository?

[henryiii]

I've been building them locally. There might be a setting in the "actions" tab that @wjakob can set, otherwise, it might need to be merged first. I'll trigger a build in my fork again.

[henryiii]

Edit: no, I can see the Actions tab, so I think it has to be in this repo before it will build.

[YannickJadoul]

Let's see when this is merged, then. I thought when Actions were enabled, things just started building when adding a workflow, but I might be mistaken.

[rwgk]

With the usual caveats (I don't know a lot about the details): this is a huge amount of work, going in a good direction, we can always fix details later.

Note that AppVeyor was completely broken

Sounds like it can only get better.

I'll probably refactor after this gets merged to use PEP 518, but for now, it is good to have this in the history,

I think that's a great plan.

[rwgk]

I haven't come across this before ... why is the 2nd macro needed? Is this working around compiler bugs?

[henryiii]

This is an idiom if you need to stringify the contents post macro expansion. Just the first one would not expand the macro inside. It's BOOST_PP_STRINGIZE, basically.

[henryiii]

Okay, let's merge then cleanup as needed. :)