Had a test marked @pytest.mark.xfail because of a known bug. Someone fixed the bug. The test started passing. Nobody un-xfailed it. We kept treating it as “known broken” for three months.

strict=True prevents this:

@pytest.mark.xfail(strict=True, reason="bug #1234")
def test_broken_thing():
    assert something_broken()

If the test unexpectedly passes, pytest marks it as a failure. Which forces someone to investigate: “did the bug get fixed? If so, remove the xfail and link the commit. If not, why did this pass?”

You can set it as the default:

# pytest.ini
[pytest]
xfail_strict = true

And now any xfail without strict=False is strict. I set this in every project now. Too easy to accumulate “known bad” tests that silently become “actually fine” tests and nobody notices.

Adjacent trick: @pytest.mark.skip(reason="known flaky") should almost never be used long-term. Prefer xfail(strict=True) so you’re forced to notice when the problem goes away.

Bug markers being opinionated about “expected to still be broken” vs “working but we forgot” is a small but nice piece of test discipline.