--junitxml: Escaping < and maybe other special characters

[danwos]

I just tried to parse an junit-xml file, which was created by pytest.
The parser and also my IDE (PyCharm) claim about not escaped chars.

Here is an example, which was generated by pytest --junitxml:

<testsuite errors="0" failures="0" name="pytest" skips="10" tests="10" time="0.054">
    <testcase classname="setup" file="setup.py" line="-1" name="FLAKE8" time="0.000252246856689">
        <skipped message="file(s) previously passed FLAKE8 checks" type="pytest.skip">
        /some/folders/python2.7/site-packages/pytest_flake8.py:106: <py._xmlgen.raw object at 0x7fd5a8a0e950>
        </skipped>
    </testcase>
</testsuite>

The problem is inside the text of the skipped node.
There is a not escaped < (after the path):
/some/folders/python2.7/site-packages/pytest_flake8.py:106: <py._xmlgen...

I'm not sure, if this bug affects skipped tests only or if < is the only problematic char.

If you are not so familiar with allowed chars in xml (like me), here is a nice SO answer:
https://stackoverflow.com/a/1091953

Finally some details about my installed versions:

platform linux -- Python 3.5.2, pytest-3.6.0, py-1.5.3, pluggy-0.6.0

[pytestbot]

GitMate.io thinks possibly related issues are #126 (junitxml doesnt escape everything), #37 (Avoid null-characters in junitxml output), #1218 (junitxml doesn't escape newlines in attribute values), #2649 (Consider escaping more characters when generating node ids), and #404 (junitxml generates invalid XML characters in failure message).

[nicoddemus]

Thanks for the report @danwos!

It does look like a bug. At first it seems to due to some interaction with pytest-flake8, can you confirm that it is installed? (also can also run pytest --version, which also lists all plugins).

[jeffreyrack]

I'm not able to recreate this in my own local environment using Python3.6 && pytest 3.6.1

Test I wrote to try to recreate it:

import sys
import pytest

@pytest.mark.skip(reason='This test should fail because 1 is < 2')
def test_fail():
    a = 1
    assert a >= 2

And this created the following XML:

<?xml version="1.0" encoding="utf-8"?>
<testsuite errors="0" failures="0" name="pytest" skips="1" tests="1" time="0.100">
	<testcase classname="test" file="test.py" line="3" name="test_fail" time="0.0010006427764892578">
		<skipped message="This test should fail because 1 is &lt; 2" type="pytest.skip">test.py:3: &lt;py._xmlgen.raw object at 0x000000AD85EB54E0&gt;</skipped>
	</testcase>
</testsuite>

Is there any easy way for us to be able to recreate this issue?

[danwos]

>>> pytest --version
This is pytest version 3.6.0, imported from ....python3.5/site-packages/pytest.py
setuptools registered plugins:
  pytest-flake8-1.0.1 at .....python3.5/site-packages/pytest_flake8.py

I tried to reproduce the error inside a simpler environment.
And it looks like the pytest-flake8 must be involved to trigger the issue.
Here is my setup:

flake_test.py

def foo():
    pass

Run py.test --flake8 --junitxml pytest.xml twice(!) to get skipped message inside xml-file.
Works only if there is no flake8-issue inside flake_test.py

Result: pytest.xml without escaped <

<testsuite errors="0" failures="0" name="pytest" skips="1" tests="1" time="0.010">
    <testcase classname="flake_test" file="flake_test.py" line="-1" name="FLAKE8" time="0.0003752708435058594">
        <skipped message="file(s) previously passed FLAKE8 checks" type="pytest.skip">
..../python3.5/site-packages/pytest_flake8.py:106: <py._xmlgen.raw object at 0x7f717c7da518>
        </skipped>
    </testcase>
</testsuite>

I'm not sure if this is a pytest-flake8 error, because I guess that there must be some common escape-function inside pytest itself, which cares about the correct handling of the xml-ouput.
No matter if the output comes from pytest or some of its plugins.

Tell mef, if you need some more information.
And thanks for caring!

[RonnyPfannschmidt]

thats even worse - that is a repr(...) of something that should be raw data to the xml contents - massive bug from pylib usage

[RonnyPfannschmidt]

the bug happens in

pytest/src/_pytest/junitxml.py

Lines 226 to 229 in 60273d7

	skipreason = bin_xml_escape(skipreason[9:])
	self.append(
	Junit.skipped(
	"%s:%s: %s" % (filename, lineno, skipreason),

the skipreason is bin escaped into a raw object, but then used as part of a repr call, thats def a misstake in pytest

[RonnyPfannschmidt]

the issue got introduced in e843b02 but didn't manifest, until 580c852

the solution is to use the bin-escape on the formatted string

up for a pr?

[nicoddemus]

Strange, I would expect Junit.skipped to automatically xml-escape anything passed to it as the tag contents.