Release 4.2.0: xfail spelling change breaks test suite

[hackebrot]

Hi friends! 👋

Upgrading to pytest 4.2.0 caused failures in the pytest-emoji testsuite, because pytest_report_teststatus now returns XFAIL instead of xfail for the verbose test report.

I'm raising this here as our changelog did not warn about this change (#4667, #4668). I welcome the change for the sake of consistency with other test outcomes, but I'm afraid this might cause some confusion for plugin maintainers.

How do you feel about updating the changelog to warn about this change? I'm happy to submit a pull request for that, if you think it makes sense. 🤔

Here's a minimal testcase, which passes for 4.1.0 but fails for 4.2.0:

import textwrap
import pytest


def test_emoji_disabled_by_default_verbose(testdir):
    testdir.makepyfile(
        textwrap.dedent(
            """
        # -*- coding: utf-8 -*-

        import pytest

        def test_passed():
            assert True

        def test_failed():
            assert False

        @pytest.mark.xfail
        def test_xfailed():
            assert False

        @pytest.mark.xfail
        def test_xpassed():
            assert True
        """
        )
    )

    result = testdir.runpytest("-v", "-o", "console_output_style=classic")

    result.stdout.fnmatch_lines(
        [
            "*::test_passed PASSED",
            "*::test_failed FAILED",
            "*::test_xfailed xfail",
            "*::test_xpassed XPASS",
        ]
    )

    assert result.ret == 1

[nicoddemus]

Howdy Raphael!

How do you feel about updating the changelog to warn about this change? I'm happy to submit a pull request for that, if you think it makes sense.

Sounds good, indeed that slipped and we should make the change clearer in the CHANGELOG. It would be great if you could submit a PR. 👍 🙇

[hackebrot]

Which changelog category would be most fitting?

[nicoddemus]

I think "trivial" is OK?