Versions >=0.3.0 have argparse conflict with package pytest-playwright
#1469
Comments
|
+1 to this, having the same issue |
|
+1 to the above, I have the same issue. |
|
Upon inspection, it seems that the
This is not well documented. If this pytest plugin were to be split into its own package (such as If the langsmith team wishes to keep the pytest plugin embedded in the core @baskaryan you seem to be the one that implemented the pytest plugin? (PR here) What are your thoughts? |
|
I've discovered that pytest plugins can be disabled: https://docs.pytest.org/en/stable/how-to/writing_plugins.html#plugin-discovery-order-at-tool-startup If possible, I think this pytest plugin should be disabled by default, requiring opt-in, since it's included as part of the base |
|
Hey folks, thanks for raising this and for the PR. Will look ASAP |
|
thanks for flagging! did not realize this would lead to conflicts. would prefer one or both of:
for 1. maybe we add a flag for 2. believe we should be able to do something simple like def pytest_addoption(parser):
"""Set a boolean flag for LangSmith output, but skip if --langsmith-output is already defined."""
# 1) Check if --langsmith-output is already present:
for action in parser._actions:
if "--langsmith-output" in action.option_strings:
logger.warning("LangSmith output flag cannot be added because ...")
return
# 2) If it's not found, register the flag
group = parser.getgroup("langsmith", "LangSmith")
group.addoption(
"--langsmith-output",
action="store_true",
default=False,
help="Use LangSmith output (requires 'rich')."
)and then updating the corresponding docs page to highlight the change: https://github.com/langchain-ai/langsmith-docs/blob/5260804cc21542370da306aeca461d59995e6537/docs/evaluation/how_to_guides/pytest.mdx?plain=1#L302 |
|
Fixed in #1482 and shipped in Unfortunately required a breaking change - new flag is |
Problem Description
Langsmith Python SDK versions
>=0.3.0(latest is currently0.3.2) seem to add command-line argument--outputtopytestentrypoint (perhaps all CLI entrypoints?), which conflicts with pytest pluginpytest-playwright(for example) which adds the same command-line argument (see docs). When both packages are installed in the same python environment, this renderspytestcompletely unusable, giving the following error:I must apologize that this may seem simply as a conflict between two packages, but I would say the core issue is here in the
langsmithpackage because:pytestentrypointAdditionally, even if we suppose the
langsmithpackage is justified in interacting with command-line arguments:--outputcommand-line argument is not documented bylangsmith(at least not anywhere that I could find; correct me if I'm wrong)pytest-playwrightgot to it first; they published releases using--outputbeforelangsmithdidPotential Solutions
pytest-langsmithwhich is a proper pytest plugin, and only modify pytest behavior if this plugin is installed; this allows the environment to have regularlangsmithinstalled without it interfering with command-line arguments--outputto--langsmith-outputto reduce chance of collisionsUntil an actual solution is implemented, I will be tied to using
langsmith<0.3.0Full Error Stack Trace
Environment
3.9through3.13)pyproject.toml:
requirements.txt
The text was updated successfully, but these errors were encountered: