Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix multiple pip compile errors #8189

Merged

Conversation

deivid-rodriguez
Copy link
Contributor

@deivid-rodriguez deivid-rodriguez commented Oct 13, 2023

This PR contains a few different bug fixes and refactorings.

I decided to propose them together, because the failing cases I was working with did not get fixed by applying only a subset of the fixes, but they need all of them to start creating sane PRs.

Break up of the different fixes is:

  • Fix regex match of pip-compile errors as a way to retry pip-compile with --no-build-isolation. This error was caused by Pip itself changing the error message, and just catches up with the change. 500455f
  • Copy --no-build-isolation retry code into the file updater. Normally error detection only needs to happen in the update checker, since we fail early when detecting these errors and we don't need to check again in the file updater. We don't reach that far. However, in the case of these retries, if we succeed when retying, we keep going and need the same fallback when updating files. This could be improved by passing around whether --no-build-isolation needs to be used or not from the update checker to the file updater, but for now I just copied the code to detect it. f4fa838fe5dd70aa2ed5eb5097a8f3990a80f9b2
  • Workaround a very common compilation error. PyYAML < 6 is incompatible with Cython 3, which we use since last July. In that PR, this problem was actually surfaced by our specs, but we failed to notice that it was not a specs-only problem where we could upgrade ourselves to PyYAML 6, but a real world user issue. Generally we don't introduce this kind of workaround but in this case I believe it's very wide spread so it's worth introducing it and paying the cost of maintaining this extra code. bb3398a64624f29af16072c48385b8f7f4964496

This is a reproducible example being fixed by this PR:

$ bin/dry-run.rb pip "raiden-network/raiden" --dir="/requirements" --updater-options=grouped_updates_experimental_rules,record_ecosystem_versions,record_update_job_unknown_error,unignore_commands --commit=90cd5a6cc27e31088a39fd35c0dccf89871dfa88 --cache=files --dep aiortc

@deivid-rodriguez deivid-rodriguez requested a review from a team as a code owner October 13, 2023 15:22
@deivid-rodriguez deivid-rodriguez force-pushed the deivid-rodriguez/fix-multiple-pip-compile-errors branch from 70c1eed to bb3398a Compare October 13, 2023 15:38
@deivid-rodriguez deivid-rodriguez force-pushed the deivid-rodriguez/fix-multiple-pip-compile-errors branch from bb3398a to ca2d2f5 Compare October 13, 2023 15:45
This is used to retry without build isolation, but the message has
changed in recent pip versions and was no longer matched. Now, at least
in some cases, the error reads like this:

```
pip._internal.exceptions.InstallationSubprocessError: Getting requirements to build wheel exited with 1
```
The convention is that those always perform some work.
Move the handling to wrap the only code that can raise those errors.
… updater

Otherwise, retries will never succeed since even if the checker manages
to find a target version to update to, the updater will fail to update
the files.
@deivid-rodriguez deivid-rodriguez force-pushed the deivid-rodriguez/fix-multiple-pip-compile-errors branch from ca2d2f5 to 450aa61 Compare October 16, 2023 14:57
@deivid-rodriguez deivid-rodriguez merged commit 2101a2a into main Oct 16, 2023
80 checks passed
@deivid-rodriguez deivid-rodriguez deleted the deivid-rodriguez/fix-multiple-pip-compile-errors branch October 16, 2023 16:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants