-
-
Notifications
You must be signed in to change notification settings - Fork 3k
Closed
Description
- a detailed description of the bug or problem you are having
- output of
pip listfrom the virtual environment you are using - pytest and operating system versions
- minimal example if possible
Hello. We have updated Python from 3.11.0b3 to 3.11.0b4 in Fedora Linux 37 and we see some test failures of pytest 7.1.2. I was able to reproduce them from git HEAD @ c2b1d5b.
# on pytest git clone
$ git rev-parse HEAD
c2b1d5b368a1fc829c6c06c700ae2ba9cda55794
$ tox -e py311 -- -k 'test_wrap_session_notify_exception or test_doctest_unexpected_exception'
py311 inst-nodeps: .../pytest/.tox/.tmp/package/1/pytest-7.2.0.dev210+gc2b1d5b36.tar.gz
py311 installed: argcomplete==2.0.0,attrs==21.4.0,certifi==2022.6.15,charset-normalizer==2.1.0,elementpath==2.5.3,hypothesis==6.50.1,idna==3.3,iniconfig==1.1.1,mock==4.0.3,nose==1.3.7,packaging==21.3,pluggy==1.0.0,py==1.11.0,Pygments==2.12.0,pyparsing==3.0.9,pytest @ file://.../pytest/.tox/.tmp/package/1/pytest-7.2.0.dev210%2Bgc2b1d5b36.tar.gz,requests==2.28.1,sortedcontainers==2.4.0,urllib3==1.26.10,xmlschema==1.11.3
py311 run-test-pre: PYTHONHASHSEED='1407110200'
py311 run-test: commands[0] | pytest -k 'test_wrap_session_notify_exception or test_doctest_unexpected_exception'
============================= test session starts ==============================
platform linux -- Python 3.11.0b4, pytest-7.2.0.dev210+gc2b1d5b36, pluggy-1.0.0
cachedir: .tox/py311/.pytest_cache
rootdir: .../pytest, configfile: pyproject.toml, testpaths: testing
plugins: hypothesis-6.50.1
collected 3164 items / 3160 deselected / 4 selected
testing/test_doctest.py F [ 25%]
testing/test_main.py FFF [100%]
=================================== FAILURES ===================================
________________ TestDoctests.test_doctest_unexpected_exception ________________
self = <test_doctest.TestDoctests object at 0x7f73ca605e50>
pytester = <Pytester PosixPath('/tmp/pytest-of-.../pytest-2/test_doctest_unexpected_exception0')>
def test_doctest_unexpected_exception(self, pytester: Pytester):
pytester.maketxtfile(
"""
>>> i = 0
>>> 0 / i
2
"""
)
result = pytester.runpytest("--doctest-modules")
> result.stdout.fnmatch_lines(
[
"test_doctest_unexpected_exception.txt F *",
"",
"*= FAILURES =*",
"*_ [[]doctest[]] test_doctest_unexpected_exception.txt _*",
"001 >>> i = 0",
"002 >>> 0 / i",
"UNEXPECTED EXCEPTION: ZeroDivisionError*",
"Traceback (most recent call last):",
' File "*/doctest.py", line *, in __run',
" *",
*((" *^^^^*",) if sys.version_info >= (3, 11) else ()),
' File "<doctest test_doctest_unexpected_exception.txt[1]>", line 1, in <module>',
"ZeroDivisionError: division by zero",
"*/test_doctest_unexpected_exception.txt:2: UnexpectedException",
],
consecutive=True,
)
E Failed: nomatch: 'test_doctest_unexpected_exception.txt F *'
E and: '============================= test session starts =============================='
E and: 'platform linux -- Python 3.11.0b4, pytest-7.2.0.dev210+gc2b1d5b36, pluggy-1.0.0'
E and: 'rootdir: /tmp/pytest-of-.../pytest-2/test_doctest_unexpected_exception0'
E and: 'collected 1 item'
E and: ''
E fnmatch: 'test_doctest_unexpected_exception.txt F *'
E with: 'test_doctest_unexpected_exception.txt F [100%]'
E exact match: ''
E fnmatch: '*= FAILURES =*'
E with: '=================================== FAILURES ==================================='
E fnmatch: '*_ [[]doctest[]] test_doctest_unexpected_exception.txt _*'
E with: '_______________ [doctest] test_doctest_unexpected_exception.txt ________________'
E exact match: '001 >>> i = 0'
E exact match: '002 >>> 0 / i'
E fnmatch: 'UNEXPECTED EXCEPTION: ZeroDivisionError*'
E with: "UNEXPECTED EXCEPTION: ZeroDivisionError('division by zero')"
E exact match: 'Traceback (most recent call last):'
E fnmatch: ' File "*/doctest.py", line *, in __run'
E with: ' File "/usr/lib64/python3.11/doctest.py", line 1350, in __run'
E fnmatch: ' *'
E with: ' exec(compile(example.source, filename, "single",'
E no consecutive match: ' *^^^^*'
E with: ' File "<doctest test_doctest_unexpected_exception.txt[1]>", line 1, in <module>'
.../pytest/testing/test_doctest.py:214: Failed
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.11.0b4, pytest-7.2.0.dev210+gc2b1d5b36, pluggy-1.0.0
rootdir: /tmp/pytest-of-.../pytest-2/test_doctest_unexpected_exception0
collected 1 item
test_doctest_unexpected_exception.txt F [100%]
=================================== FAILURES ===================================
_______________ [doctest] test_doctest_unexpected_exception.txt ________________
001 >>> i = 0
002 >>> 0 / i
UNEXPECTED EXCEPTION: ZeroDivisionError('division by zero')
Traceback (most recent call last):
File "/usr/lib64/python3.11/doctest.py", line 1350, in __run
exec(compile(example.source, filename, "single",
File "<doctest test_doctest_unexpected_exception.txt[1]>", line 1, in <module>
ZeroDivisionError: division by zero
/tmp/pytest-of-.../pytest-2/test_doctest_unexpected_exception0/test_doctest_unexpected_exception.txt:2: UnexpectedException
=========================== short test summary info ============================
FAILED test_doctest_unexpected_exception.txt::test_doctest_unexpected_exception.txt
============================== 1 failed in 0.00s ===============================
_________________ test_wrap_session_notify_exception[ret_exc0] _________________
ret_exc = (None, <class 'ValueError'>)
pytester = <Pytester PosixPath('/tmp/pytest-of-.../pytest-2/test_wrap_session_notify_exception0')>
@pytest.mark.parametrize(
"ret_exc",
(
pytest.param((None, ValueError)),
pytest.param((42, SystemExit)),
pytest.param((False, SystemExit)),
),
)
def test_wrap_session_notify_exception(ret_exc, pytester: Pytester) -> None:
returncode, exc = ret_exc
c1 = pytester.makeconftest(
"""
import pytest
def pytest_sessionstart():
raise {exc}("boom")
def pytest_internalerror(excrepr, excinfo):
returncode = {returncode!r}
if returncode is not False:
pytest.exit("exiting after %s..." % excinfo.typename, returncode={returncode!r})
""".format(
returncode=returncode, exc=exc.__name__
)
)
result = pytester.runpytest()
if returncode:
assert result.ret == returncode
else:
assert result.ret == ExitCode.INTERNAL_ERROR
assert result.stdout.lines[0] == "INTERNALERROR> Traceback (most recent call last):"
end_lines = (
result.stdout.lines[-4:]
if sys.version_info >= (3, 11)
else result.stdout.lines[-3:]
)
if exc == SystemExit:
assert end_lines == [
f'INTERNALERROR> File "{c1}", line 4, in pytest_sessionstart',
'INTERNALERROR> raise SystemExit("boom")',
*(
("INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^^^^^",)
if sys.version_info >= (3, 11)
else ()
),
"INTERNALERROR> SystemExit: boom",
]
else:
> assert end_lines == [
f'INTERNALERROR> File "{c1}", line 4, in pytest_sessionstart',
'INTERNALERROR> raise ValueError("boom")',
*(
("INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^^^^^",)
if sys.version_info >= (3, 11)
else ()
),
"INTERNALERROR> ValueError: boom",
]
E assert ['INTERNALERR...eError: boom'] == ['INTERNALERR...eError: boom']
E At index 0 diff: 'INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^^^^^^' != 'INTERNALERROR> File "/tmp/pytest-of-.../pytest-2/test_wrap_session_notify_exception0/conftest.py", line 4, in pytest_sessionstart'
E Use -v to get more diff
.../pytest/testing/test_main.py:66: AssertionError
----------------------------- Captured stdout call -----------------------------
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File ".../pytest/.tox/py311/lib/python3.11/site-packages/_pytest/main.py", line 268, in wrap_session
INTERNALERROR> config.hook.pytest_sessionstart(session=session)
INTERNALERROR> File ".../pytest/.tox/py311/lib/python3.11/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR> return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> File ".../pytest/.tox/py311/lib/python3.11/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> File ".../pytest/.tox/py311/lib/python3.11/site-packages/pluggy/_manager.py", line 327, in traced_hookexec
INTERNALERROR> return outcome.get_result()
INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> File ".../pytest/.tox/py311/lib/python3.11/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR> raise ex[1].with_traceback(ex[2])
INTERNALERROR> File ".../pytest/.tox/py311/lib/python3.11/site-packages/pluggy/_result.py", line 33, in from_call
INTERNALERROR> result = func()
INTERNALERROR> ^^^^^^
INTERNALERROR> File ".../pytest/.tox/py311/lib/python3.11/site-packages/pluggy/_manager.py", line 324, in <lambda>
INTERNALERROR> lambda: oldcall(hook_name, hook_impls, kwargs, firstresult)
INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> File ".../pytest/.tox/py311/lib/python3.11/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR> return outcome.get_result()
INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> File ".../pytest/.tox/py311/lib/python3.11/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR> raise ex[1].with_traceback(ex[2])
INTERNALERROR> File ".../pytest/.tox/py311/lib/python3.11/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR> res = hook_impl.function(*args)
INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> File "/tmp/pytest-of-.../pytest-2/test_wrap_session_notify_exception0/conftest.py", line 4, in pytest_sessionstart
INTERNALERROR> raise ValueError("boom")
INTERNALERROR> ValueError: boom
----------------------------- Captured stderr call -----------------------------
Exit: exiting after ValueError...
_________________ test_wrap_session_notify_exception[ret_exc1] _________________
ret_exc = (42, <class 'SystemExit'>)
pytester = <Pytester PosixPath('/tmp/pytest-of-.../pytest-2/test_wrap_session_notify_exception1')>
@pytest.mark.parametrize(
"ret_exc",
(
pytest.param((None, ValueError)),
pytest.param((42, SystemExit)),
pytest.param((False, SystemExit)),
),
)
def test_wrap_session_notify_exception(ret_exc, pytester: Pytester) -> None:
returncode, exc = ret_exc
c1 = pytester.makeconftest(
"""
import pytest
def pytest_sessionstart():
raise {exc}("boom")
def pytest_internalerror(excrepr, excinfo):
returncode = {returncode!r}
if returncode is not False:
pytest.exit("exiting after %s..." % excinfo.typename, returncode={returncode!r})
""".format(
returncode=returncode, exc=exc.__name__
)
)
result = pytester.runpytest()
if returncode:
assert result.ret == returncode
else:
assert result.ret == ExitCode.INTERNAL_ERROR
assert result.stdout.lines[0] == "INTERNALERROR> Traceback (most recent call last):"
end_lines = (
result.stdout.lines[-4:]
if sys.version_info >= (3, 11)
else result.stdout.lines[-3:]
)
if exc == SystemExit:
> assert end_lines == [
f'INTERNALERROR> File "{c1}", line 4, in pytest_sessionstart',
'INTERNALERROR> raise SystemExit("boom")',
*(
("INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^^^^^",)
if sys.version_info >= (3, 11)
else ()
),
"INTERNALERROR> SystemExit: boom",
]
E assert ['INTERNALERR...emExit: boom'] == ['INTERNALERR...emExit: boom']
E At index 0 diff: 'INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^^^^^^' != 'INTERNALERROR> File "/tmp/pytest-of-.../pytest-2/test_wrap_session_notify_exception1/conftest.py", line 4, in pytest_sessionstart'
E Use -v to get more diff
.../pytest/testing/test_main.py:55: AssertionError
----------------------------- Captured stdout call -----------------------------
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File ".../pytest/.tox/py311/lib/python3.11/site-packages/_pytest/main.py", line 268, in wrap_session
INTERNALERROR> config.hook.pytest_sessionstart(session=session)
INTERNALERROR> File ".../pytest/.tox/py311/lib/python3.11/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR> return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> File ".../pytest/.tox/py311/lib/python3.11/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> File ".../pytest/.tox/py311/lib/python3.11/site-packages/pluggy/_manager.py", line 327, in traced_hookexec
INTERNALERROR> return outcome.get_result()
INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> File ".../pytest/.tox/py311/lib/python3.11/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR> raise ex[1].with_traceback(ex[2])
INTERNALERROR> File ".../pytest/.tox/py311/lib/python3.11/site-packages/pluggy/_result.py", line 33, in from_call
INTERNALERROR> result = func()
INTERNALERROR> ^^^^^^
INTERNALERROR> File ".../pytest/.tox/py311/lib/python3.11/site-packages/pluggy/_manager.py", line 324, in <lambda>
INTERNALERROR> lambda: oldcall(hook_name, hook_impls, kwargs, firstresult)
INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> File ".../pytest/.tox/py311/lib/python3.11/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR> return outcome.get_result()
INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> File ".../pytest/.tox/py311/lib/python3.11/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR> raise ex[1].with_traceback(ex[2])
INTERNALERROR> File ".../pytest/.tox/py311/lib/python3.11/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR> res = hook_impl.function(*args)
INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> File "/tmp/pytest-of-.../pytest-2/test_wrap_session_notify_exception1/conftest.py", line 4, in pytest_sessionstart
INTERNALERROR> raise SystemExit("boom")
INTERNALERROR> SystemExit: boom
----------------------------- Captured stderr call -----------------------------
Exit: exiting after SystemExit...
_________________ test_wrap_session_notify_exception[ret_exc2] _________________
ret_exc = (False, <class 'SystemExit'>)
pytester = <Pytester PosixPath('/tmp/pytest-of-.../pytest-2/test_wrap_session_notify_exception2')>
@pytest.mark.parametrize(
"ret_exc",
(
pytest.param((None, ValueError)),
pytest.param((42, SystemExit)),
pytest.param((False, SystemExit)),
),
)
def test_wrap_session_notify_exception(ret_exc, pytester: Pytester) -> None:
returncode, exc = ret_exc
c1 = pytester.makeconftest(
"""
import pytest
def pytest_sessionstart():
raise {exc}("boom")
def pytest_internalerror(excrepr, excinfo):
returncode = {returncode!r}
if returncode is not False:
pytest.exit("exiting after %s..." % excinfo.typename, returncode={returncode!r})
""".format(
returncode=returncode, exc=exc.__name__
)
)
result = pytester.runpytest()
if returncode:
assert result.ret == returncode
else:
assert result.ret == ExitCode.INTERNAL_ERROR
assert result.stdout.lines[0] == "INTERNALERROR> Traceback (most recent call last):"
end_lines = (
result.stdout.lines[-4:]
if sys.version_info >= (3, 11)
else result.stdout.lines[-3:]
)
if exc == SystemExit:
> assert end_lines == [
f'INTERNALERROR> File "{c1}", line 4, in pytest_sessionstart',
'INTERNALERROR> raise SystemExit("boom")',
*(
("INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^^^^^",)
if sys.version_info >= (3, 11)
else ()
),
"INTERNALERROR> SystemExit: boom",
]
E assert ['INTERNALERR...emExit: boom'] == ['INTERNALERR...emExit: boom']
E At index 0 diff: 'INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^^^^^^' != 'INTERNALERROR> File "/tmp/pytest-of-.../pytest-2/test_wrap_session_notify_exception2/conftest.py", line 4, in pytest_sessionstart'
E Use -v to get more diff
.../pytest/testing/test_main.py:55: AssertionError
----------------------------- Captured stdout call -----------------------------
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File ".../pytest/.tox/py311/lib/python3.11/site-packages/_pytest/main.py", line 268, in wrap_session
INTERNALERROR> config.hook.pytest_sessionstart(session=session)
INTERNALERROR> File ".../pytest/.tox/py311/lib/python3.11/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR> return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> File ".../pytest/.tox/py311/lib/python3.11/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> File ".../pytest/.tox/py311/lib/python3.11/site-packages/pluggy/_manager.py", line 327, in traced_hookexec
INTERNALERROR> return outcome.get_result()
INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> File ".../pytest/.tox/py311/lib/python3.11/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR> raise ex[1].with_traceback(ex[2])
INTERNALERROR> File ".../pytest/.tox/py311/lib/python3.11/site-packages/pluggy/_result.py", line 33, in from_call
INTERNALERROR> result = func()
INTERNALERROR> ^^^^^^
INTERNALERROR> File ".../pytest/.tox/py311/lib/python3.11/site-packages/pluggy/_manager.py", line 324, in <lambda>
INTERNALERROR> lambda: oldcall(hook_name, hook_impls, kwargs, firstresult)
INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> File ".../pytest/.tox/py311/lib/python3.11/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR> return outcome.get_result()
INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> File ".../pytest/.tox/py311/lib/python3.11/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR> raise ex[1].with_traceback(ex[2])
INTERNALERROR> File ".../pytest/.tox/py311/lib/python3.11/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR> res = hook_impl.function(*args)
INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> File "/tmp/pytest-of-.../pytest-2/test_wrap_session_notify_exception2/conftest.py", line 4, in pytest_sessionstart
INTERNALERROR> raise SystemExit("boom")
INTERNALERROR> SystemExit: boom
----------------------------- Captured stderr call -----------------------------
mainloop: caught unexpected SystemExit!
=========================== short test summary info ============================
FAILED testing/test_doctest.py::TestDoctests::test_doctest_unexpected_exception
FAILED testing/test_main.py::test_wrap_session_notify_exception[ret_exc0] - a...
FAILED testing/test_main.py::test_wrap_session_notify_exception[ret_exc1] - a...
FAILED testing/test_main.py::test_wrap_session_notify_exception[ret_exc2] - a...
====================== 4 failed, 3160 deselected in 0.65s ======================
ERROR: InvocationError for command .../pytest/.tox/py311/bin/pytest -k 'test_wrap_session_notify_exception or test_doctest_unexpected_exception' (exited with code 1)
___________________________________ summary ____________________________________
ERROR: py311: commands failed
$ .tox/py311/bin/pip list
Package Version
------------------ -----------------------
argcomplete 2.0.0
attrs 21.4.0
certifi 2022.6.15
charset-normalizer 2.1.0
elementpath 2.5.3
hypothesis 6.50.1
idna 3.3
iniconfig 1.1.1
mock 4.0.3
nose 1.3.7
packaging 21.3
pip 21.2.3
pluggy 1.0.0
py 1.11.0
Pygments 2.12.0
pyparsing 3.0.9
pytest 7.2.0.dev210+gc2b1d5b36
requests 2.28.1
setuptools 57.4.0
sortedcontainers 2.4.0
urllib3 1.26.10
wheel 0.36.2
xmlschema 1.11.3
Metadata
Metadata
Assignees
Labels
No labels