Skip to content

skipping: --runxfail breaks pytest.mark.skip location reporting #29

@rowan-stein

Description

@rowan-stein

Issue: skipping: --runxfail breaks pytest.mark.skip location reporting
Pytest versions: 5.4.x, current master

When @pytest.mark.skip/skipif marks are used to skip a test, for example:

import pytest
@pytest.mark.skip
def test_skip_location() -> None:
    assert 0

The expected skip location reported should point to the item itself, and this is indeed what happens when running with pytest -rs:

SKIPPED [1] test_it.py:3: unconditional skip

However, adding pytest -rs --runxfail breaks this:

SKIPPED [1] src/_pytest/skipping.py:238: unconditional skip

The --runxfail is only about xfail and should not affect this at all.


Research specification (by Emerson Gray):

Root cause:

  • In src/_pytest/skipping.py, within pytest_runtest_makereport, there is an unconditional short-circuit when --runxfail is active:
    • elif item.config.option.runxfail: pass # don't interfere
  • This prevents the subsequent skip-location rewrite block from executing, leaving rep.longrepr pointing to where skip() is raised in the plugin (skipping.py), instead of the test item’s location.
  • Without --runxfail, the skip-location rewrite runs and correctly reports the item’s file:line.

Proposed fix (minimal and correct):

  • Remove the broad runxfail short-circuit so the skip-location rewrite can still run when --runxfail is active.
  • Guard only the xfail-specific branches with not item.config.option.runxfail to preserve --runxfail semantics for xfail while leaving skip handling untouched:
    • Change elif call.excinfo and isinstance(call.excinfo.value, xfail.Exception): to elif (not item.config.option.runxfail) and call.excinfo and isinstance(call.excinfo.value, xfail.Exception):
    • Change elif not rep.skipped and xfailed: to elif (not item.config.option.runxfail) and not rep.skipped and xfailed:
  • Keep the existing skip-location rewrite block unchanged; once the broad pass is removed, it will run for skip/skipif marks regardless of --runxfail.

Reproduction steps and observed failure:

  1. Create two tests:
# test_skip.py
import pytest

@pytest.mark.skip(reason="skip me")
def test_skip():
    assert True
# test_skipif.py
import pytest

@pytest.mark.skipif(True, reason="conditional skip")
def test_skipif():
    assert True
  1. Commands and outputs (before fix):
  • pytest -q -rs test_skip.pySKIP [1] test_skip.py:3: skip me (correct)
  • pytest -q -rs test_skip.py --runxfailSKIP [1] src/_pytest/skipping.py:239: skip me (incorrect)
  • pytest -q -rs test_skipif.pySKIP [1] test_skipif.py:4: conditional skip (correct)
  • pytest -q -rs test_skipif.py --runxfailSKIP [1] src/_pytest/skipping.py:239: conditional skip (incorrect)
  1. After applying the fix:
  • pytest -q -rs test_skip.py --runxfailSKIP [1] test_skip.py:3: skip me (correct)
  • pytest -q -rs test_skipif.py --runxfailSKIP [1] test_skipif.py:4: conditional skip (correct)

XFail behavior check:

# test_xfail.py
import pytest

@pytest.mark.xfail(reason="expected failure")
def test_xfail_fails():
    assert False
  • pytest -q -rs test_xfail.py → shows XFAIL as usual.
  • pytest -q -rs --runxfail test_xfail.py → xfail handling disabled; test fails normally without XFAIL/XPASS, as expected.

Edge cases:

  • Skips triggered via marks (skip/skipif) will have locations rewritten to the item under --runxfail.
  • Direct pytest.skip() calls inside tests are unaffected (location stays where called in test).
  • unittest-style tests and unexpected successes remain unaffected.

We will implement this fix via a PR targeting branch pytest-dev__pytest-7432.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions