Clarify implications wrt. to auditwheel and delocate bundling of extra binaries

[pombredanne]

The readme at https://github.com/pypa/cibuildwheel/blob/1d2f86e6cf7fe43faef886d32748d6904c0e3a02/README.md#legal-note states:

Legal note
Since cibuildwheel repairs the wheel with delocate or auditwheel, it might automatically bundle dynamically linked libraries from the build machine.
It helps ensure that the library can run without any dependencies outside of the pip toolchain.
This is similar to static linking, so it might have some license implications. Check the license for any code you're pulling in to make sure that's allowed.

This may or may not apply ... but when this happen, how can a cibuildwheel user find out and:

• get the list of "vendored" binaries?
• the download URLs of their exact corresponding binary and source code to meet possible redistribution license obligations?
• include delocated and auditedwheel bundled binaries license?
• What if some bundled binaries have vulnerabilities? How could I know this? (e.g. back to the question, I would need to know what has been bundled exactly)

All these are IMHO important things to figure out for me as I plan to use cibuildwheel to build many binary wheels.
I would like to ensure that this is easy and documented for anyone (including me) and ideally automated and contribute this back here.
(Note: I maintain an ugly set of scripts that build wheels on Linux/mac/Windows there https://github.com/nexB/skeleton/blob/main/etc/scripts/build_wheels.py that I want to mostly drop in favor of this)

@joerick gentle ping... you added this legal note in the first place.

[pombredanne]

There is a related license discussion at #809

[nightlark]

I'm just a cibuildwheel user. I don't think I have a particularly deep knowledge of the cibuildwheel implementation, so there may be ways features in cibuildwheel I'm unaware of that could be harnessed to simplify some of these tasks. This is a bit of a brain dump on issues related to some of these questions that go beyond just cibuildwheel; I think solving them would result in tools that a lot of other people would find useful.

Determining libraries included in wheel

I think the vendoring referred to is happening as a result of cibuildwheel running delocate on macOS and auditwheel on Linux, so the list of "vendored" libraries should match what gets vendored by running those tools... in addition to any libraries manually included by the author of the Python package via something like the MANIFEST.in file if setuptools is used.

If part of the Python package build involves running make or cmake to build native libraries, it is possible those build systems could be set up to automatically download and build dependencies (similar to what https://github.com/scikit-build/ninja-python-distributions and https://github.com/scikit-build/cmake-python-distributions are doing).

Scraping the log output for the build and repair steps for a wheel (e.g. https://github.com/scikit-build/cmake-python-distributions/runs/4973768375?check_suite_focus=true#step:4:720) could be used to figure out some of what is being included, or check the finished wheel file for all the .so/.dylib/*.dll files it contains.

---

Download URLs

It might be tricky to figure out the download URLs as the libraries may have already been installed on the build machine using a package manager (e.g. apt-get) or included as part of the virtual environment created by the CI service being used. If they were downloaded as part of the wheel building process then cibuildwheel probably doesn't have insight into what the build system for a particular wheel is doing to determine that it is downloading libraries.

A possibly crazy idea: hook all common functions/tools related to downloading files (git, curl, wget) on a system to try to automatically compile a list of URLs that were accessed during the wheel build process.

---

Vulnerabilities

For the question of if some bundled libraries have vulnerabilities, the answer is almost certainly yes, which isn't terribly useful. For unknown vulnerabilities, it may be possible to find some of them by running various static and dynamic analysis tools on the software, though I don't think this is practical as a step when building a package.

For specific (known) vulnerabilities the the name of the library and the version are needed to look up CVEs that affect that version of a library. The shared library name could be used to get a good guess at what the name of the library is for looking up CVEs. Determining the version of a library is probably trickier, ELF files don't store version information so it could require some knowledge of what the library is to know if it provides a version function (e.g. to call a getVersion function in libxyz).

If part of building a library (or binary) involved static linking, then looking up CVEs for each static library (and any static libraries used by that static library) should also be done. Determining what static libraries are present in a binary is kind of an open research question (esp. for closed source libraries, since you can't just manually go and look at the source code or build the library yourself). From my experience trying to pinpoint a version is even harder. Header-only libraries or "copy+paste a .c source file" libraries (e.g. imgui) would be a similar case to static linking.

Overall, if everything a Python package depends on is open source, I think it is relatively practical for someone to manually go through and see what the exact version of a library used is (and recursively going in and checking any static libraries/vendored code it has) -- I can imagine some opportunities to automate parts of this process; it would certainly be easier if the C/C++ ecosystem had a standard package/module file like Go, Rust, Node, pip requirements.txt, etc.

[pombredanne]

Determining what static libraries are present in a binary is kind of an open research question (esp. for closed source libraries, since you can't just manually go and look at the source code or build the library yourself).

IMHO there are two levels:

• the code of the wheel beeing built AND any third-party code injected by its own build is one thing. This is out of the control of cibuidlwheel and should be documented by the package author in the first place, otherwise this is the domain of binary analysis and reversing which is not for the faint of heart
• the binary plugs that may be injected by delocate/auditwheel. This is called by cibuildwheel and under its direct control. This is the part that should be traced so it can reported to the user. When I say, I mean introspecting what is done, rather than looking at it after the fact with reversing. This would be simpler, cleaner and accurate too.

[joerick]

@joerick gentle ping... you added this legal note in the first place.

Git blame has done me in! Yes, it's true that I did. Truth be told the reason why that section is worded so vaguely is because I wouldn't feel confident giving firm guidance about how this might apply in a general way. It's going to be specific to each project - the specific dependencies, the specific way that they're used and linked into the wheel.

as I plan to use cibuildwheel to build many binary wheels

This is interesting, but do be aware that this is outside of the user model that we've been designing cibuildwheel for thus far. The reason that I mention this is that these questions feel very project-specific, and just because cibuildwheel is the one that you've invoked to build wheels, that doesn't mean that cibuildwheel is the tool that's pulling in dependencies. That action is performed by other tools - pip, build, the package's build backend, auditwheel/delocate etc. And I'm not sure how cibuildwheel would even go about tracking the dependencies that went into a wheel build - a lot of that is transparent to us, when we just call pip wheel or python -m build on the package dir.

I wonder if there might be an approach that statically examines repos/sdists to track dependencies. I suspect that libraries.io does something like this (example). Github is also starting to track dependencies. Perhaps you could find out how they're doing it?

[pombredanne]

@joerick Thanks! delocate and auditwheels would likely be the primary triggers of binary vendoring. FWIW libraries.io does nothing of the sort (I maintain tools that do binary analysis and there is not a line of code in the libraries.io codebase that would remotely do anything there). I happen to also maintain one of the more popular license detection tools so I am a savvy enough license-wise. And also a package vulnerabilities database so I dabble in this too.

The issue is that binary vendoring means redistribution and this may trigger some license obligations (like at the minimum produce some attribution/credits and license notices, and in some case support copyleft source code redistribution) . And vendoring binaries unknowingly could be a security concern too if the vendored bits have exploitable vulnerabilities.

This means therefore that vendoring information need to bubble up from delocate and auditwheel and then in cibuidlwheel somehow and from then somehow surface in built wheels so that user can known. Ideally automating everything would be the right to do. But first logging/alerting users that something is done would go a long way. Short of this there are thing happening automagically in the build that could a source of some concerns ;)

Note that the origin and source of the vendored binaries has to come from somewhere... and this must be the Docker images you use. Do you keep track or a copy of all the binary packages and their sources somewhere? If yes, that would help a lot and if no, that may be the place where to start as nothing would be much doable without this in the first place.

[joerick]

Note that the origin and source of the vendored binaries has to come from somewhere... and this must be the Docker images you use. Do you keep track or a copy of all the binary packages and their sources somewhere?

Aside from what the project installs themselves during the build, the Docker images are manylinux, so you could check that project to see what's going on there.