Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pin test to legacy resolver to force desired error message #7738

Merged
merged 1 commit into from
Aug 7, 2023

Conversation

jeffwidman
Copy link
Member

@jeffwidman jeffwidman commented Aug 5, 2023

Under pip-tools 7.0.0, the default flipped from --resolver=legacy
to --resolver=backtracking.

As a result, the error message changed. Furthermore, the new error
message is hidden behind an exception thrown by
cython>=3.0.0/pyyaml<6.0.1 conflict (details in yaml/pyyaml#702).
I already fixed a similar error here: 0a8e0b4

So the first thing I did was bump pyyaml to 6.0.1. While not
strictly necessary, it'll make later debugging easier when / if someone
is trying to repro this error under the new backtracking resolver.

Secondly, I specified --resolver=legacy to preserve the old behavior
even after we upgrade pip-tools (#7711).

This is acceptable for the purposes of this test because this isn't
testing the resolver behavior, it's testing that we properly convert the
pip-tools native error to a well-known Dependabot::DependencyFileNotResolvable
error.

I did take a peek at the pip-tools internals to see if this error
messsage was specific solely to the legacy resolver, but the logic
looks pretty generic... ie, this likely could be raised under the new resolver:
https://github.com/jazzband/pip-tools/blob/9e4aeff8f3b5518586091700bec9bbb570154a38/piptools/exceptions.py#L53

This is admittedly a short-term solution because the legacy resolver
is deprecated and will be removed at some point in the future... so at
that point we'll need to find another way to reproduce the error. But
enough other things could change upstream in pip/pip-tools (such as
they might switch the error or remove it when they remove the legacy
resolver) so we can worry about that when we get there rather than
prematurely optimizing.

Technically I suppose we could mock the response
from calling pip-compile, but I hate mocking what I don't own... much
prefer to simply coerce it to return the expected error message.

For future reference, here's the error message under
--resolver=legacy:

```shell
[dependabot-core-dev] ~/python/spec/fixtures/pip_compile_files $ pyenv exec pip-compile --build-isolation --output-file=../requirements/incompatible_versions.txt --no-emit-index-url -P pyyaml==6.0.1 --resolver=legacy incompatible_versions.in
WARNING: the legacy dependency resolver is deprecated and will be removed in future versions of pip-tools.
Using legacy resolver. Consider using backtracking resolver with `--resolver=backtracking`.
Could not find a version that matches pyyaml<5.4,==6.0.1,>=3.10
Tried: 3.10, 3.10, 3.11, 3.11, 3.12, 3.12, 3.13, 5.1, 5.1.1, 5.1.2, 5.2, 5.3, 5.3.1, 5.4, 5.4.1, 6.0, 6.0, 6.0.1, 6.0.1
Skipped pre-versions: 3.13b1, 3.13rc1, 4.2b1, 4.2b2, 4.2b4, 5.1b1, 5.1b3, 5.1b5, 5.2b1, 5.3b1, 5.4b1, 5.4b2, 6.0b1
There are incompatible versions in the resolved dependencies:
  pyyaml==6.0.1
  PyYAML<5.4,>=3.10 (from awscli==1.18.198->-r incompatible_versions.in (line 3))
  pyyaml (from jinja2-cli[yaml]==0.7.0->-r incompatible_versions.in (line 2))

And here's the error message under --resolver=backtracking:

[dependabot-core-dev] ~/python/spec/fixtures/pip_compile_files $ pyenv exec pip-compile --build-isolation --output-file=../requirements/incompatible_versions.txt --no-emit-index-url -P pyyaml==6.0.1 --resolver=backtracking incompatible_versions.in
  ERROR: Cannot install -r incompatible_versions.in (line 3) and jinja2-cli[yaml]==0.7.0 because these package versions have conflicting dependencies.
Traceback (most recent call last):
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 316, in _backjump
    name, candidate = broken_state.mapping.popitem()
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: 'dictionary is empty'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 92, in resolve
    result = self._result = resolver.resolve(
                            ^^^^^^^^^^^^^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 434, in resolve
    success = self._backjump(causes)
              ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 318, in _backjump
    raise ResolutionImpossible(causes)
pip._vendor.resolvelib.resolvers.ResolutionImpossible: [RequirementInformation(requirement=SpecifierRequirement('pyyaml; extra == "yaml"'), parent=ExtrasCandidate(base=LinkCandidate('https://files.pythonhosted.org/packages/83/52/c1bb249f49b204c14bf603e1b1a6dc2db8c3b631c4fe00a2872360085cd6/jinja2_cli-0.7.0-py2.py3-none-any.whl (from https://pypi.org/simple/jinja2-cli/)'), extras=frozenset({'yaml'}))), RequirementInformation(requirement=SpecifierRequirement('PyYAML<5.4,>=3.10; python_version != "3.4"'), parent=LinkCandidate('https://files.pythonhosted.org/packages/4e/68/35522bf3b5ef9186f3cd7cfdc00fe708c019fada077a21009ff78757b0f9/awscli-1.18.198-py2.py3-none-any.whl (from https://pypi.org/simple/awscli/)'))]

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/.pyenv/versions/3.11.4/bin/pip-compile", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/piptools/scripts/compile.py", line 657, in cli
    results = resolver.resolve(max_rounds=max_rounds)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/piptools/resolver.py", line 604, in resolve
    is_resolved = self._do_resolve(
                  ^^^^^^^^^^^^^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/piptools/resolver.py", line 636, in _do_resolve
    resolver.resolve(
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 101, in resolve
    raise error from e
pip._internal.exceptions.DistributionNotFound: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

Furthermore, the cython>=3.0.0/pyyaml<=6.0.1 conflict only appears when I do --resolver=backtracking, otherwise under the legacy resolver I don't see that error, even on older version of pyyaml:

[dependabot-core-dev] ~/python/spec/fixtures/pip_compile_files $ pyenv exec pip-compile --build-isolation --output-file=../requirements/incompatible_versions.txt --no-emit-index-url -P pyyaml==5.4 --resolver=legacy incompatible_versions.in
WARNING: the legacy dependency resolver is deprecated and will be removed in future versions of pip-tools.
Using legacy resolver. Consider using backtracking resolver with `--resolver=backtracking`.
Could not find a version that matches pyyaml<5.4,==5.4,>=3.10
Tried: 3.10, 3.10, 3.11, 3.11, 3.12, 3.12, 3.13, 5.1, 5.1.1, 5.1.2, 5.2, 5.3, 5.3.1, 5.4, 5.4.1, 6.0, 6.0, 6.0.1, 6.0.1
Skipped pre-versions: 3.13b1, 3.13rc1, 4.2b1, 4.2b2, 4.2b4, 5.1b1, 5.1b3, 5.1b5, 5.2b1, 5.3b1, 5.4b1, 5.4b2, 6.0b1
There are incompatible versions in the resolved dependencies:
  pyyaml==5.4
  PyYAML<5.4,>=3.10 (from awscli==1.18.198->-r incompatible_versions.in (line 3))
  pyyaml (from jinja2-cli[yaml]==0.7.0->-r incompatible_versions.in (line 2))

@jeffwidman jeffwidman requested a review from a team as a code owner August 5, 2023 16:06
@jeffwidman jeffwidman marked this pull request as draft August 5, 2023 16:14
@jeffwidman jeffwidman force-pushed the bump-pyyaml-version-due-to-cython-conflict branch 3 times, most recently from cfc34a1 to 4d4117f Compare August 5, 2023 20:32
@jeffwidman jeffwidman changed the title Bump desired pyyaml to latest due to cython conflict Pin test to legacy resolver to force desired error message Aug 5, 2023
@jeffwidman jeffwidman marked this pull request as ready for review August 5, 2023 20:34
@jeffwidman jeffwidman added T: tech-debt ⚙️ L:python:pip-compile Python packages via pip-compile Ecosystems Used by the maintainer team for internal-facing project tracking and removed L: python labels Aug 5, 2023
@jeffwidman jeffwidman self-assigned this Aug 5, 2023
Copy link
Contributor

@deivid-rodriguez deivid-rodriguez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Under `pip-tools` `7.0.0`, the default flipped from `--resolver=legacy`
to `--resolver=backtracking`.

As a result, the error message changed. Furthermore, the new error
message is hidden behind an exception thrown by
`cython>=3.0.0`/`pyyaml<6.0.1` conflict (details in yaml/pyyaml#702).
I already fixed a similar error here: 0a8e0b4

So the first thing I did was bump `pyyaml` to `6.0.1`. While not
strictly necessary, it'll make later debugging easier when / if someone
is trying to repro this error under the new `backtracking` resolver.

Secondly, I specified `--resolver=legacy` to preserve the old behavior
even after we upgrade `pip-tools` (#7711).

This is acceptable for the purposes of this test because this isn't
testing the resolver behavior, it's testing that we properly convert the
`pip-tools` native error to a well-known `Dependabot::DependencyFileNotResolvable`
error.

I did take a peek at the `pip-tools` internals to see if this error
messsage was specific solely to the `legacy` resolver, but the logic
looks pretty generic... ie, this likely could be raised under the new resolver:
https://github.com/jazzband/pip-tools/blob/9e4aeff8f3b5518586091700bec9bbb570154a38/piptools/exceptions.py#L53

This is admittedly a short-term solution because the `legacy` resolver
is deprecated and will be removed at some point in the future... so at
that point we'll need to find another way to reproduce the error. But
enough other things could change upstream in `pip`/`pip-tools` (such as
they might switch the error or remove it when they remove the legacy
resolver) so we can worry about that when we get there rather than
prematurely optimizing.

Technically I suppose we could mock the response
from calling `pip-compile`, but I hate mocking what I don't own... much
prefer to simply coerce it to return the expected error message.

For future reference, here's the error message under
`--resolver=legacy`:
```shell
```shell
[dependabot-core-dev] ~/python/spec/fixtures/pip_compile_files $ pyenv exec pip-compile --build-isolation --output-file=../requirements/incompatible_versions.txt --no-emit-index-url -P pyyaml==6.0.1 --resolver=legacy incompatible_versions.in
WARNING: the legacy dependency resolver is deprecated and will be removed in future versions of pip-tools.
Using legacy resolver. Consider using backtracking resolver with `--resolver=backtracking`.
Could not find a version that matches pyyaml<5.4,==6.0.1,>=3.10
Tried: 3.10, 3.10, 3.11, 3.11, 3.12, 3.12, 3.13, 5.1, 5.1.1, 5.1.2, 5.2, 5.3, 5.3.1, 5.4, 5.4.1, 6.0, 6.0, 6.0.1, 6.0.1
Skipped pre-versions: 3.13b1, 3.13rc1, 4.2b1, 4.2b2, 4.2b4, 5.1b1, 5.1b3, 5.1b5, 5.2b1, 5.3b1, 5.4b1, 5.4b2, 6.0b1
There are incompatible versions in the resolved dependencies:
  pyyaml==6.0.1
  PyYAML<5.4,>=3.10 (from awscli==1.18.198->-r incompatible_versions.in (line 3))
  pyyaml (from jinja2-cli[yaml]==0.7.0->-r incompatible_versions.in (line 2))
```

And here's the error message under `--resolver=backtracking`:
```shell
[dependabot-core-dev] ~/python/spec/fixtures/pip_compile_files $ pyenv exec pip-compile --build-isolation --output-file=../requirements/incompatible_versions.txt --no-emit-index-url -P pyyaml==6.0.1 --resolver=backtracking incompatible_versions.in
  ERROR: Cannot install -r incompatible_versions.in (line 3) and jinja2-cli[yaml]==0.7.0 because these package versions have conflicting dependencies.
Traceback (most recent call last):
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 316, in _backjump
    name, candidate = broken_state.mapping.popitem()
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: 'dictionary is empty'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 92, in resolve
    result = self._result = resolver.resolve(
                            ^^^^^^^^^^^^^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 434, in resolve
    success = self._backjump(causes)
              ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 318, in _backjump
    raise ResolutionImpossible(causes)
pip._vendor.resolvelib.resolvers.ResolutionImpossible: [RequirementInformation(requirement=SpecifierRequirement('pyyaml; extra == "yaml"'), parent=ExtrasCandidate(base=LinkCandidate('https://files.pythonhosted.org/packages/83/52/c1bb249f49b204c14bf603e1b1a6dc2db8c3b631c4fe00a2872360085cd6/jinja2_cli-0.7.0-py2.py3-none-any.whl (from https://pypi.org/simple/jinja2-cli/)'), extras=frozenset({'yaml'}))), RequirementInformation(requirement=SpecifierRequirement('PyYAML<5.4,>=3.10; python_version != "3.4"'), parent=LinkCandidate('https://files.pythonhosted.org/packages/4e/68/35522bf3b5ef9186f3cd7cfdc00fe708c019fada077a21009ff78757b0f9/awscli-1.18.198-py2.py3-none-any.whl (from https://pypi.org/simple/awscli/)'))]

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/.pyenv/versions/3.11.4/bin/pip-compile", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/piptools/scripts/compile.py", line 657, in cli
    results = resolver.resolve(max_rounds=max_rounds)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/piptools/resolver.py", line 604, in resolve
    is_resolved = self._do_resolve(
                  ^^^^^^^^^^^^^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/piptools/resolver.py", line 636, in _do_resolve
    resolver.resolve(
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 101, in resolve
    raise error from e
pip._internal.exceptions.DistributionNotFound: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
```

Furthermore, the `cython>=3.0.0`/`pyyaml<=6.0.1` conflict only appears when I do `--resolver=backtracking`, otherwise under the legacy resolver I don't see that error, even on older version of pyyaml:
```shell
[dependabot-core-dev] ~/python/spec/fixtures/pip_compile_files $ pyenv exec pip-compile --build-isolation --output-file=../requirements/incompatible_versions.txt --no-emit-index-url -P pyyaml==5.4 --resolver=legacy incompatible_versions.in
WARNING: the legacy dependency resolver is deprecated and will be removed in future versions of pip-tools.
Using legacy resolver. Consider using backtracking resolver with `--resolver=backtracking`.
Could not find a version that matches pyyaml<5.4,==5.4,>=3.10
Tried: 3.10, 3.10, 3.11, 3.11, 3.12, 3.12, 3.13, 5.1, 5.1.1, 5.1.2, 5.2, 5.3, 5.3.1, 5.4, 5.4.1, 6.0, 6.0, 6.0.1, 6.0.1
Skipped pre-versions: 3.13b1, 3.13rc1, 4.2b1, 4.2b2, 4.2b4, 5.1b1, 5.1b3, 5.1b5, 5.2b1, 5.3b1, 5.4b1, 5.4b2, 6.0b1
There are incompatible versions in the resolved dependencies:
  pyyaml==5.4
  PyYAML<5.4,>=3.10 (from awscli==1.18.198->-r incompatible_versions.in (line 3))
  pyyaml (from jinja2-cli[yaml]==0.7.0->-r incompatible_versions.in (line 2))
```
@jeffwidman jeffwidman force-pushed the bump-pyyaml-version-due-to-cython-conflict branch from 4d4117f to 10c7544 Compare August 7, 2023 18:12
@jeffwidman jeffwidman enabled auto-merge (squash) August 7, 2023 18:13
@jeffwidman jeffwidman merged commit 4f6ca0f into main Aug 7, 2023
90 checks passed
@jeffwidman jeffwidman deleted the bump-pyyaml-version-due-to-cython-conflict branch August 7, 2023 18:26
brettfo pushed a commit to brettfo/dependabot-core that referenced this pull request Oct 11, 2023
…bot#7738)

Under `pip-tools` `7.0.0`, the default flipped from `--resolver=legacy`
to `--resolver=backtracking`.

As a result, the error message changed. Furthermore, the new error
message is hidden behind an exception thrown by
`cython>=3.0.0`/`pyyaml<6.0.1` conflict (details in yaml/pyyaml#702).
I already fixed a similar error here: dependabot@0a8e0b4

So the first thing I did was bump `pyyaml` to `6.0.1`. While not
strictly necessary, it'll make later debugging easier when / if someone
is trying to repro this error under the new `backtracking` resolver.

Secondly, I specified `--resolver=legacy` to preserve the old behavior
even after we upgrade `pip-tools` (dependabot#7711).

This is acceptable for the purposes of this test because this isn't
testing the resolver behavior, it's testing that we properly convert the
`pip-tools` native error to a well-known `Dependabot::DependencyFileNotResolvable`
error.

I did take a peek at the `pip-tools` internals to see if this error
messsage was specific solely to the `legacy` resolver, but the logic
looks pretty generic... ie, this likely could be raised under the new resolver:
https://github.com/jazzband/pip-tools/blob/9e4aeff8f3b5518586091700bec9bbb570154a38/piptools/exceptions.py#L53

This is admittedly a short-term solution because the `legacy` resolver
is deprecated and will be removed at some point in the future... so at
that point we'll need to find another way to reproduce the error. But
enough other things could change upstream in `pip`/`pip-tools` (such as
they might switch the error or remove it when they remove the legacy
resolver) so we can worry about that when we get there rather than
prematurely optimizing.

Technically I suppose we could mock the response
from calling `pip-compile`, but I hate mocking what I don't own... much
prefer to simply coerce it to return the expected error message.

For future reference, here's the error message under
`--resolver=legacy`:
```shell
```shell
[dependabot-core-dev] ~/python/spec/fixtures/pip_compile_files $ pyenv exec pip-compile --build-isolation --output-file=../requirements/incompatible_versions.txt --no-emit-index-url -P pyyaml==6.0.1 --resolver=legacy incompatible_versions.in
WARNING: the legacy dependency resolver is deprecated and will be removed in future versions of pip-tools.
Using legacy resolver. Consider using backtracking resolver with `--resolver=backtracking`.
Could not find a version that matches pyyaml<5.4,==6.0.1,>=3.10
Tried: 3.10, 3.10, 3.11, 3.11, 3.12, 3.12, 3.13, 5.1, 5.1.1, 5.1.2, 5.2, 5.3, 5.3.1, 5.4, 5.4.1, 6.0, 6.0, 6.0.1, 6.0.1
Skipped pre-versions: 3.13b1, 3.13rc1, 4.2b1, 4.2b2, 4.2b4, 5.1b1, 5.1b3, 5.1b5, 5.2b1, 5.3b1, 5.4b1, 5.4b2, 6.0b1
There are incompatible versions in the resolved dependencies:
  pyyaml==6.0.1
  PyYAML<5.4,>=3.10 (from awscli==1.18.198->-r incompatible_versions.in (line 3))
  pyyaml (from jinja2-cli[yaml]==0.7.0->-r incompatible_versions.in (line 2))
```

And here's the error message under `--resolver=backtracking`:
```shell
[dependabot-core-dev] ~/python/spec/fixtures/pip_compile_files $ pyenv exec pip-compile --build-isolation --output-file=../requirements/incompatible_versions.txt --no-emit-index-url -P pyyaml==6.0.1 --resolver=backtracking incompatible_versions.in
  ERROR: Cannot install -r incompatible_versions.in (line 3) and jinja2-cli[yaml]==0.7.0 because these package versions have conflicting dependencies.
Traceback (most recent call last):
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 316, in _backjump
    name, candidate = broken_state.mapping.popitem()
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: 'dictionary is empty'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 92, in resolve
    result = self._result = resolver.resolve(
                            ^^^^^^^^^^^^^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 434, in resolve
    success = self._backjump(causes)
              ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 318, in _backjump
    raise ResolutionImpossible(causes)
pip._vendor.resolvelib.resolvers.ResolutionImpossible: [RequirementInformation(requirement=SpecifierRequirement('pyyaml; extra == "yaml"'), parent=ExtrasCandidate(base=LinkCandidate('https://files.pythonhosted.org/packages/83/52/c1bb249f49b204c14bf603e1b1a6dc2db8c3b631c4fe00a2872360085cd6/jinja2_cli-0.7.0-py2.py3-none-any.whl (from https://pypi.org/simple/jinja2-cli/)'), extras=frozenset({'yaml'}))), RequirementInformation(requirement=SpecifierRequirement('PyYAML<5.4,>=3.10; python_version != "3.4"'), parent=LinkCandidate('https://files.pythonhosted.org/packages/4e/68/35522bf3b5ef9186f3cd7cfdc00fe708c019fada077a21009ff78757b0f9/awscli-1.18.198-py2.py3-none-any.whl (from https://pypi.org/simple/awscli/)'))]

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/.pyenv/versions/3.11.4/bin/pip-compile", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/piptools/scripts/compile.py", line 657, in cli
    results = resolver.resolve(max_rounds=max_rounds)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/piptools/resolver.py", line 604, in resolve
    is_resolved = self._do_resolve(
                  ^^^^^^^^^^^^^^^^^
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/piptools/resolver.py", line 636, in _do_resolve
    resolver.resolve(
  File "/usr/local/.pyenv/versions/3.11.4/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 101, in resolve
    raise error from e
pip._internal.exceptions.DistributionNotFound: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
```

Furthermore, the `cython>=3.0.0`/`pyyaml<=6.0.1` conflict only appears when I do `--resolver=backtracking`, otherwise under the legacy resolver I don't see that error, even on older version of pyyaml:
```shell
[dependabot-core-dev] ~/python/spec/fixtures/pip_compile_files $ pyenv exec pip-compile --build-isolation --output-file=../requirements/incompatible_versions.txt --no-emit-index-url -P pyyaml==5.4 --resolver=legacy incompatible_versions.in
WARNING: the legacy dependency resolver is deprecated and will be removed in future versions of pip-tools.
Using legacy resolver. Consider using backtracking resolver with `--resolver=backtracking`.
Could not find a version that matches pyyaml<5.4,==5.4,>=3.10
Tried: 3.10, 3.10, 3.11, 3.11, 3.12, 3.12, 3.13, 5.1, 5.1.1, 5.1.2, 5.2, 5.3, 5.3.1, 5.4, 5.4.1, 6.0, 6.0, 6.0.1, 6.0.1
Skipped pre-versions: 3.13b1, 3.13rc1, 4.2b1, 4.2b2, 4.2b4, 5.1b1, 5.1b3, 5.1b5, 5.2b1, 5.3b1, 5.4b1, 5.4b2, 6.0b1
There are incompatible versions in the resolved dependencies:
  pyyaml==5.4
  PyYAML<5.4,>=3.10 (from awscli==1.18.198->-r incompatible_versions.in (line 3))
  pyyaml (from jinja2-cli[yaml]==0.7.0->-r incompatible_versions.in (line 2))
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Ecosystems Used by the maintainer team for internal-facing project tracking L:python:pip-compile Python packages via pip-compile L: python T: tech-debt ⚙️
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants