doctest: unset RUNNER_CLASS in pytest_unconfigure by blueyed · Pull Request #6039 · pytest-dev/pytest

blueyed · 2019-10-22T20:32:21Z

No description provided.

This is important when used with ``pytester``'s ``runpytest_inprocess``. Since 07f20cc `pytest testing/acceptance_test.py -k test_doctest_id` would fail, since the second run would not consider the exception to be an instance of `doctest.DocTestFailure` anymore, since the module was re-imported, and use another failure message then in the short test summary info (and in the report itself): > FAILED test_doctest_id.txt::test_doctest_id.txt - doctest.DocTestFailure: <Do... while it should be: > FAILED test_doctest_id.txt::test_doctest_id.txt

blueyed · 2019-10-23T01:10:37Z

Ok, that was nasty.. - see the commit message.

blueyed · 2019-10-23T08:49:23Z

To reproduce: pytest testing/acceptance_test.py -k test_doctest_id

RonnyPfannschmidt · 2019-10-23T08:57:59Z

this one is most confusing - this is just caching a lazy import/codegen, does that surround inprocess sys.modules cleanup?

blueyed · 2019-10-23T09:08:23Z

Yes.

RonnyPfannschmidt · 2019-10-23T09:12:01Z

lets adda followup to put this into a per config storage then, or ensure we import doctest in the testsuite instead

blueyed · 2019-10-23T09:19:35Z

https://github.com/pytest-dev/pytest/pull/5847/files#diff-df80246215438974e5e2656c15531696 changes it already a bit - might at least be affected then still (since the class appears to be cached there also/additionally) /cc @bluetech

blueyed · 2019-10-23T09:20:27Z

Should we take this then for now?

RonnyPfannschmidt · 2019-10-23T09:28:50Z

this pr is fine for now, we just should take a look at doing a followup

79: Update pytest to 5.3.0 r=duckinator a=pyup-bot This PR updates [pytest](https://pypi.org/project/pytest) from **5.2.4** to **5.3.0**. <details> <summary>Changelog</summary> ### 5.3.0 ``` ========================= Deprecations ------------ - `6179 <https://github.com/pytest-dev/pytest/issues/6179>`_: The default value of ``junit_family`` option will change to ``xunit2`` in pytest 6.0, given that this is the version supported by default in modern tools that manipulate this type of file. In order to smooth the transition, pytest will issue a warning in case the ``--junitxml`` option is given in the command line but ``junit_family`` is not explicitly configured in ``pytest.ini``. For more information, `see the docs <https://docs.pytest.org/en/latest/deprecations.htmljunit-family-default-value-change-to-xunit2>`__. Features -------- - `4488 <https://github.com/pytest-dev/pytest/issues/4488>`_: The pytest team has created the `pytest-reportlog <https://github.com/pytest-dev/pytest-reportlog>`__ plugin, which provides a new ``--report-log=FILE`` option that writes *report logs* into a file as the test session executes. Each line of the report log contains a self contained JSON object corresponding to a testing event, such as a collection or a test result report. The file is guaranteed to be flushed after writing each line, so systems can read and process events in real-time. The plugin is meant to replace the ``--resultlog`` option, which is deprecated and meant to be removed in a future release. If you use ``--resultlog``, please try out ``pytest-reportlog`` and provide feedback. - `4730 <https://github.com/pytest-dev/pytest/issues/4730>`_: When ``sys.pycache_prefix`` (Python 3.8+) is set, it will be used by pytest to cache test files changed by the assertion rewriting mechanism. This makes it easier to benefit of cached ``.pyc`` files even on file systems without permissions. - `5515 <https://github.com/pytest-dev/pytest/issues/5515>`_: Allow selective auto-indentation of multiline log messages. Adds command line option ``--log-auto-indent``, config option ``log_auto_indent`` and support for per-entry configuration of indentation behavior on calls to ``logging.log()``. Alters the default for auto-indention from ``on`` to ``off``. This restores the older behavior that existed prior to v4.6.0. This reversion to earlier behavior was done because it is better to activate new features that may lead to broken tests explicitly rather than implicitly. - `5914 <https://github.com/pytest-dev/pytest/issues/5914>`_: ``pytester`` learned two new functions, `no_fnmatch_line <https://docs.pytest.org/en/latest/reference.html_pytest.pytester.LineMatcher.no_fnmatch_line>`_ and `no_re_match_line <https://docs.pytest.org/en/latest/reference.html_pytest.pytester.LineMatcher.no_re_match_line>`_. The functions are used to ensure the captured text *does not* match the given pattern. The previous idiom was to use ``re.match``: .. code-block:: python assert re.match(pat, result.stdout.str()) is None Or the ``in`` operator: .. code-block:: python assert text in result.stdout.str() But the new functions produce best output on failure. - `6057 <https://github.com/pytest-dev/pytest/issues/6057>`_: Added tolerances to complex values when printing ``pytest.approx``. For example, ``repr(pytest.approx(3+4j))`` returns ``(3+4j) ± 5e-06 ∠ ±180°``. This is polar notation indicating a circle around the expected value, with a radius of 5e-06. For ``approx`` comparisons to return ``True``, the actual value should fall within this circle. - `6061 <https://github.com/pytest-dev/pytest/issues/6061>`_: Added the pluginmanager as an argument to ``pytest_addoption`` so that hooks can be invoked when setting up command line options. This is useful for having one plugin communicate things to another plugin, such as default values or which set of command line options to add. Improvements ------------ - `5061 <https://github.com/pytest-dev/pytest/issues/5061>`_: Use multiple colors with terminal summary statistics. - `5630 <https://github.com/pytest-dev/pytest/issues/5630>`_: Quitting from debuggers is now properly handled in ``doctest`` items. - `5924 <https://github.com/pytest-dev/pytest/issues/5924>`_: Improved verbose diff output with sequences. Before: .. code-block:: E AssertionError: assert ['version', '...version_info'] == ['version', '...version', ...] E Right contains 3 more items, first extra item: ' ' E Full diff: E - ['version', 'version_info', 'sys.version', 'sys.version_info'] E + ['version', E + 'version_info', E + 'sys.version', E + 'sys.version_info', E + ' ', E + 'sys.version', E + 'sys.version_info'] After: .. code-block:: E AssertionError: assert ['version', '...version_info'] == ['version', '...version', ...] E Right contains 3 more items, first extra item: ' ' E Full diff: E [ E 'version', E 'version_info', E 'sys.version', E 'sys.version_info', E + ' ', E + 'sys.version', E + 'sys.version_info', E ] - `5936 <https://github.com/pytest-dev/pytest/issues/5936>`_: Display untruncated assertion message with ``-vv``. - `5990 <https://github.com/pytest-dev/pytest/issues/5990>`_: Fixed plurality mismatch in test summary (e.g. display "1 error" instead of "1 errors"). - `6008 <https://github.com/pytest-dev/pytest/issues/6008>`_: ``Config.InvocationParams.args`` is now always a ``tuple`` to better convey that it should be immutable and avoid accidental modifications. - `6023 <https://github.com/pytest-dev/pytest/issues/6023>`_: ``pytest.main`` now returns a ``pytest.ExitCode`` instance now, except for when custom exit codes are used (where it returns ``int`` then still). - `6026 <https://github.com/pytest-dev/pytest/issues/6026>`_: Align prefixes in output of pytester's ``LineMatcher``. - `6059 <https://github.com/pytest-dev/pytest/issues/6059>`_: Collection errors are reported as errors (and not failures like before) in the terminal's short test summary. - `6069 <https://github.com/pytest-dev/pytest/issues/6069>`_: ``pytester.spawn`` does not skip/xfail tests on FreeBSD anymore unconditionally. - `6097 <https://github.com/pytest-dev/pytest/issues/6097>`_: The "[XXX%]" indicator in the test summary is now colored according to the final (new) multi-colored line's main color. - `6116 <https://github.com/pytest-dev/pytest/issues/6116>`_: Added ``--co`` as a synonym to ``--collect-only``. - `6148 <https://github.com/pytest-dev/pytest/issues/6148>`_: ``atomicwrites`` is now only used on Windows, fixing a performance regression with assertion rewriting on Unix. - `6152 <https://github.com/pytest-dev/pytest/issues/6152>`_: Now parametrization will use the ``__name__`` attribute of any object for the id, if present. Previously it would only use ``__name__`` for functions and classes. - `6176 <https://github.com/pytest-dev/pytest/issues/6176>`_: Improved failure reporting with pytester's ``Hookrecorder.assertoutcome``. - `6181 <https://github.com/pytest-dev/pytest/issues/6181>`_: The reason for a stopped session, e.g. with ``--maxfail`` / ``-x``, now gets reported in the test summary. - `6206 <https://github.com/pytest-dev/pytest/issues/6206>`_: Improved ``cache.set`` robustness and performance. Bug Fixes --------- - `2049 <https://github.com/pytest-dev/pytest/issues/2049>`_: Fixed ``--setup-plan`` showing inaccurate information about fixture lifetimes. - `2548 <https://github.com/pytest-dev/pytest/issues/2548>`_: Fixed line offset mismatch of skipped tests in terminal summary. - `6039 <https://github.com/pytest-dev/pytest/issues/6039>`_: The ``PytestDoctestRunner`` is now properly invalidated when unconfiguring the doctest plugin. This is important when used with ``pytester``'s ``runpytest_inprocess``. - `6047 <https://github.com/pytest-dev/pytest/issues/6047>`_: BaseExceptions are now handled in ``saferepr``, which includes ``pytest.fail.Exception`` etc. - `6074 <https://github.com/pytest-dev/pytest/issues/6074>`_: pytester: fixed order of arguments in ``rm_rf`` warning when cleaning up temporary directories, and do not emit warnings for errors with ``os.open``. - `6189 <https://github.com/pytest-dev/pytest/issues/6189>`_: Fixed result of ``getmodpath`` method. Trivial/Internal Changes ------------------------ - `4901 <https://github.com/pytest-dev/pytest/issues/4901>`_: ``RunResult`` from ``pytester`` now displays the mnemonic of the ``ret`` attribute when it is a valid ``pytest.ExitCode`` value. ``` </details> <details> <summary>Links</summary> - PyPI: https://pypi.org/project/pytest - Changelog: https://pyup.io/changelogs/pytest/ - Homepage: https://docs.pytest.org/en/latest/ </details> Co-authored-by: pyup-bot <github-bot@pyup.io>

105: Update pytest to 5.3.1 r=rehandalal a=pyup-bot This PR updates [pytest](https://pypi.org/project/pytest) from **5.2.4** to **5.3.1**. <details> <summary>Changelog</summary> ### 5.3.1 ``` ========================= Improvements ------------ - `6231 <https://github.com/pytest-dev/pytest/issues/6231>`_: Improve check for misspelling of ``pytest.mark.parametrize``. - `6257 <https://github.com/pytest-dev/pytest/issues/6257>`_: Handle `exit.Exception` raised in `notify_exception` (via `pytest_internalerror`), e.g. when quitting pdb from post mortem. Bug Fixes --------- - `5914 <https://github.com/pytest-dev/pytest/issues/5914>`_: pytester: fix ``no_fnmatch_line`` when used after positive matching. - `6082 <https://github.com/pytest-dev/pytest/issues/6082>`_: Fix line detection for doctest samples inside ``property`` docstrings, as a workaround to `bpo-17446 <https://bugs.python.org/issue17446>`__. - `6254 <https://github.com/pytest-dev/pytest/issues/6254>`_: Fix compatibility with pytest-parallel (regression in pytest 5.3.0). - `6255 <https://github.com/pytest-dev/pytest/issues/6255>`_: Clear the ``sys.last_traceback``, ``sys.last_type`` and ``sys.last_value`` attributes by deleting them instead of setting them to ``None``. This better matches the behaviour of the Python standard library. ``` ### 5.3.0 ``` ========================= Deprecations ------------ - `6179 <https://github.com/pytest-dev/pytest/issues/6179>`_: The default value of ``junit_family`` option will change to ``xunit2`` in pytest 6.0, given that this is the version supported by default in modern tools that manipulate this type of file. In order to smooth the transition, pytest will issue a warning in case the ``--junitxml`` option is given in the command line but ``junit_family`` is not explicitly configured in ``pytest.ini``. For more information, `see the docs <https://docs.pytest.org/en/latest/deprecations.htmljunit-family-default-value-change-to-xunit2>`__. Features -------- - `4488 <https://github.com/pytest-dev/pytest/issues/4488>`_: The pytest team has created the `pytest-reportlog <https://github.com/pytest-dev/pytest-reportlog>`__ plugin, which provides a new ``--report-log=FILE`` option that writes *report logs* into a file as the test session executes. Each line of the report log contains a self contained JSON object corresponding to a testing event, such as a collection or a test result report. The file is guaranteed to be flushed after writing each line, so systems can read and process events in real-time. The plugin is meant to replace the ``--resultlog`` option, which is deprecated and meant to be removed in a future release. If you use ``--resultlog``, please try out ``pytest-reportlog`` and provide feedback. - `4730 <https://github.com/pytest-dev/pytest/issues/4730>`_: When ``sys.pycache_prefix`` (Python 3.8+) is set, it will be used by pytest to cache test files changed by the assertion rewriting mechanism. This makes it easier to benefit of cached ``.pyc`` files even on file systems without permissions. - `5515 <https://github.com/pytest-dev/pytest/issues/5515>`_: Allow selective auto-indentation of multiline log messages. Adds command line option ``--log-auto-indent``, config option ``log_auto_indent`` and support for per-entry configuration of indentation behavior on calls to ``logging.log()``. Alters the default for auto-indention from ``on`` to ``off``. This restores the older behavior that existed prior to v4.6.0. This reversion to earlier behavior was done because it is better to activate new features that may lead to broken tests explicitly rather than implicitly. - `5914 <https://github.com/pytest-dev/pytest/issues/5914>`_: `testdir <https://docs.pytest.org/en/latest/reference.htmltestdir>`__ learned two new functions, `no_fnmatch_line <https://docs.pytest.org/en/latest/reference.html_pytest.pytester.LineMatcher.no_fnmatch_line>`_ and `no_re_match_line <https://docs.pytest.org/en/latest/reference.html_pytest.pytester.LineMatcher.no_re_match_line>`_. The functions are used to ensure the captured text *does not* match the given pattern. The previous idiom was to use ``re.match``: .. code-block:: python result = testdir.runpytest() assert re.match(pat, result.stdout.str()) is None Or the ``in`` operator: .. code-block:: python result = testdir.runpytest() assert text in result.stdout.str() But the new functions produce best output on failure. - `6057 <https://github.com/pytest-dev/pytest/issues/6057>`_: Added tolerances to complex values when printing ``pytest.approx``. For example, ``repr(pytest.approx(3+4j))`` returns ``(3+4j) ± 5e-06 ∠ ±180°``. This is polar notation indicating a circle around the expected value, with a radius of 5e-06. For ``approx`` comparisons to return ``True``, the actual value should fall within this circle. - `6061 <https://github.com/pytest-dev/pytest/issues/6061>`_: Added the pluginmanager as an argument to ``pytest_addoption`` so that hooks can be invoked when setting up command line options. This is useful for having one plugin communicate things to another plugin, such as default values or which set of command line options to add. Improvements ------------ - `5061 <https://github.com/pytest-dev/pytest/issues/5061>`_: Use multiple colors with terminal summary statistics. - `5630 <https://github.com/pytest-dev/pytest/issues/5630>`_: Quitting from debuggers is now properly handled in ``doctest`` items. - `5924 <https://github.com/pytest-dev/pytest/issues/5924>`_: Improved verbose diff output with sequences. Before: :: E AssertionError: assert ['version', '...version_info'] == ['version', '...version', ...] E Right contains 3 more items, first extra item: ' ' E Full diff: E - ['version', 'version_info', 'sys.version', 'sys.version_info'] E + ['version', E + 'version_info', E + 'sys.version', E + 'sys.version_info', E + ' ', E + 'sys.version', E + 'sys.version_info'] After: :: E AssertionError: assert ['version', '...version_info'] == ['version', '...version', ...] E Right contains 3 more items, first extra item: ' ' E Full diff: E [ E 'version', E 'version_info', E 'sys.version', E 'sys.version_info', E + ' ', E + 'sys.version', E + 'sys.version_info', E ] - `5934 <https://github.com/pytest-dev/pytest/issues/5934>`_: ``repr`` of ``ExceptionInfo`` objects has been improved to honor the ``__repr__`` method of the underlying exception. - `5936 <https://github.com/pytest-dev/pytest/issues/5936>`_: Display untruncated assertion message with ``-vv``. - `5990 <https://github.com/pytest-dev/pytest/issues/5990>`_: Fixed plurality mismatch in test summary (e.g. display "1 error" instead of "1 errors"). - `6008 <https://github.com/pytest-dev/pytest/issues/6008>`_: ``Config.InvocationParams.args`` is now always a ``tuple`` to better convey that it should be immutable and avoid accidental modifications. - `6023 <https://github.com/pytest-dev/pytest/issues/6023>`_: ``pytest.main`` returns a ``pytest.ExitCode`` instance now, except for when custom exit codes are used (where it returns ``int`` then still). - `6026 <https://github.com/pytest-dev/pytest/issues/6026>`_: Align prefixes in output of pytester's ``LineMatcher``. - `6059 <https://github.com/pytest-dev/pytest/issues/6059>`_: Collection errors are reported as errors (and not failures like before) in the terminal's short test summary. - `6069 <https://github.com/pytest-dev/pytest/issues/6069>`_: ``pytester.spawn`` does not skip/xfail tests on FreeBSD anymore unconditionally. - `6097 <https://github.com/pytest-dev/pytest/issues/6097>`_: The "[...%]" indicator in the test summary is now colored according to the final (new) multi-colored line's main color. - `6116 <https://github.com/pytest-dev/pytest/issues/6116>`_: Added ``--co`` as a synonym to ``--collect-only``. - `6148 <https://github.com/pytest-dev/pytest/issues/6148>`_: ``atomicwrites`` is now only used on Windows, fixing a performance regression with assertion rewriting on Unix. - `6152 <https://github.com/pytest-dev/pytest/issues/6152>`_: Now parametrization will use the ``__name__`` attribute of any object for the id, if present. Previously it would only use ``__name__`` for functions and classes. - `6176 <https://github.com/pytest-dev/pytest/issues/6176>`_: Improved failure reporting with pytester's ``Hookrecorder.assertoutcome``. - `6181 <https://github.com/pytest-dev/pytest/issues/6181>`_: The reason for a stopped session, e.g. with ``--maxfail`` / ``-x``, now gets reported in the test summary. - `6206 <https://github.com/pytest-dev/pytest/issues/6206>`_: Improved ``cache.set`` robustness and performance. Bug Fixes --------- - `2049 <https://github.com/pytest-dev/pytest/issues/2049>`_: Fixed ``--setup-plan`` showing inaccurate information about fixture lifetimes. - `2548 <https://github.com/pytest-dev/pytest/issues/2548>`_: Fixed line offset mismatch of skipped tests in terminal summary. - `6039 <https://github.com/pytest-dev/pytest/issues/6039>`_: The ``PytestDoctestRunner`` is now properly invalidated when unconfiguring the doctest plugin. This is important when used with ``pytester``'s ``runpytest_inprocess``. - `6047 <https://github.com/pytest-dev/pytest/issues/6047>`_: BaseExceptions are now handled in ``saferepr``, which includes ``pytest.fail.Exception`` etc. - `6074 <https://github.com/pytest-dev/pytest/issues/6074>`_: pytester: fixed order of arguments in ``rm_rf`` warning when cleaning up temporary directories, and do not emit warnings for errors with ``os.open``. - `6189 <https://github.com/pytest-dev/pytest/issues/6189>`_: Fixed result of ``getmodpath`` method. Trivial/Internal Changes ------------------------ - `4901 <https://github.com/pytest-dev/pytest/issues/4901>`_: ``RunResult`` from ``pytester`` now displays the mnemonic of the ``ret`` attribute when it is a valid ``pytest.ExitCode`` value. ``` </details> <details> <summary>Links</summary> - PyPI: https://pypi.org/project/pytest - Changelog: https://pyup.io/changelogs/pytest/ - Homepage: https://docs.pytest.org/en/latest/ </details> Co-authored-by: pyup-bot <github-bot@pyup.io>

blueyed changed the title ~~test_doctest_id: avoid UnboundLocalError with internal errors~~ [WIP] test_doctest_id: avoid UnboundLocalError with internal errors Oct 22, 2019

blueyed changed the title ~~[WIP] test_doctest_id: avoid UnboundLocalError with internal errors~~ [WIP] Fix test_doctest_id Oct 22, 2019

blueyed force-pushed the test_doctest_id branch from 0b8e817 to 5e7b2ae Compare October 23, 2019 01:09

blueyed changed the title ~~[WIP] Fix test_doctest_id~~ doctest: unset RUNNER_CLASS in pytest_unconfigure Oct 23, 2019

blueyed mentioned this pull request Oct 23, 2019

[WIP] ci: test with wider $COLUMNS #6038

Closed

blueyed merged commit db9e248 into pytest-dev:features Oct 23, 2019

blueyed deleted the test_doctest_id branch October 23, 2019 09:39

blueyed mentioned this pull request Oct 23, 2019

2/X Fix check_untyped_defs = True mypy errors #5847

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

doctest: unset RUNNER_CLASS in pytest_unconfigure#6039

doctest: unset RUNNER_CLASS in pytest_unconfigure#6039
blueyed merged 1 commit intopytest-dev:featuresfrom
blueyed:test_doctest_id

blueyed commented Oct 22, 2019

Uh oh!

blueyed commented Oct 23, 2019

Uh oh!

blueyed commented Oct 23, 2019

Uh oh!

RonnyPfannschmidt commented Oct 23, 2019

Uh oh!

blueyed commented Oct 23, 2019

Uh oh!

RonnyPfannschmidt commented Oct 23, 2019

Uh oh!

blueyed commented Oct 23, 2019

Uh oh!

blueyed commented Oct 23, 2019

Uh oh!

RonnyPfannschmidt commented Oct 23, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

blueyed commented Oct 22, 2019

Uh oh!

blueyed commented Oct 23, 2019

Uh oh!

blueyed commented Oct 23, 2019

Uh oh!

RonnyPfannschmidt commented Oct 23, 2019

Uh oh!

blueyed commented Oct 23, 2019

Uh oh!

RonnyPfannschmidt commented Oct 23, 2019

Uh oh!

blueyed commented Oct 23, 2019

Uh oh!

blueyed commented Oct 23, 2019

Uh oh!

RonnyPfannschmidt commented Oct 23, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants