caplog fixture doesn't restore log level after test (regression in 6.0)

[rowan-stein]

User Request

From the documentation at https://docs.pytest.org/en/6.0.0/logging.html#caplog-fixture: "The log levels set are restored automatically at the end of the test". It used to work, but looks broken in new 6.0 release. Minimal example to reproduce:

def test_foo(caplog):
    caplog.set_level(42)

def test_bar(caplog):
    print(caplog.handler.level)

It prints 0 for pytest<6, 42 after. This probably regressed in commit fcbaab8b0b89abc622dbfb7982cf9bd8c91ef301.

Research Specification

• Area: src/_pytest/logging.py (caplog fixture and logging plugin)
• Regression cause:
  • In pytest 6.0, a single session-scoped caplog handler began to be reused across tests. When no global log_level is configured, the handler's level is not reset between tests. As a result, caplog.set_level(...) in one test leaks into the next test.
  • Fixture finalization restores logger levels changed via logger=..., but not the handler's level.
• Proposed fix:

  • In LoggingPlugin._runtest_for, before entering catching_logs(self.caplog_handler, level=self.log_level), reset the handler level if no global log level is set:

    if self.log_level is None:
        self.caplog_handler.setLevel(logging.NOTSET)

  • Optional safeguard: Track and restore the initial handler level in LogCaptureFixture during finalize (not strictly required if the plugin-level reset is applied).

• Backwards compatibility:
  • Restores pre-6.0 behavior (handler level starts at NOTSET for each test when no global log level is configured).
  • Preserves behavior when a global log level is configured via CLI/ini.

Reproduction Steps and Observed Failure

1. Create tests/test_repro_caplog.py with:

import logging

def test_foo(caplog):
    caplog.set_level(42)

def test_bar(caplog):
    print(caplog.handler.level)

2. Run pytest -q -s tests/test_repro_caplog.py.
3. Observed (pytest 6.0): prints 42.
4. Expected (pytest<6): prints 0.

No exception stack trace is produced; the failure is incorrect state restoration (log handler level persists across tests).

Test Plan

• Add tests ensuring handler level is restored between tests, including:
  • Minimal repro (above) expecting 0 in the second test.
  • Specific logger override via logger=... does not leak handler level.
  • Nested set_level calls and restoration behavior.
  • Ensure behavior remains correct when a global log_level is configured via CLI/ini.