No module importlib.readers breaks pytest==7.0.0 with python 3.9

[kdelee]

My project is using python 3.9 and we hit this today when 7.0.0 released:

pytest/src/_pytest/assertion/rewrite.py

Lines 279 to 283 in 5eb4d69

	if sys.version_info >= (3, 9):
	def get_resource_reader(self, name: str) -> importlib.abc.TraversableResources: # type: ignore
	from types import SimpleNamespace
	from importlib.readers import FileReader

Because this import does not seem to be working/reference existing code

/var/lib/awx/venv/awx/lib64/python3.9/site-packages/_pytest/assertion/rewrite.py:283: in get_resource_reader
    from importlib.readers import FileReader
E   ModuleNotFoundError: No module named 'importlib.readers'

If you'd like to see where this is happening on our CI, I'm pretty sure you can see the logs for the api-test stage here https://github.com/ansible/awx/runs/5072172116?check_suite_focus=true

Downgrading pytest fixes the issue, so it seems to a problem exclusively in 7.0.0 right now.

• a detailed description of the bug or problem you are having
• output of pip list from the virtual environment you are using piplist.txt
• pytest==7.0.0
• CentOS Stream release 9
• minimal example if possible

This minimal reproducer re-creates the error:
say we have broken.py with the following contents

from pip._internal.req import parse_requirements


def test_broken():
    assert True

then running pytest broken.py results in

==================================================================================== test session starts =====================================================================================
platform linux -- Python 3.9.10, pytest-7.0.0, pluggy-1.0.0
django: settings: awx.settings.development (from ini)
rootdir: /awx_devel, configfile: pytest.ini
plugins: mock-1.11.1, forked-1.4.0, django-4.5.2, xdist-1.34.0, cov-3.0.0, timeout-2.1.0
collected 0 items / 1 error                                                                                                                                                                  

=========================================================================================== ERRORS ===========================================================================================
_________________________________________________________________________________ ERROR collecting broken.py _________________________________________________________________________________
ImportError while importing test module '/awx_devel/broken.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
broken.py:1: in <module>
    from pip._internal.req import parse_requirements
/var/lib/awx/venv/awx/lib64/python3.9/site-packages/pip/_internal/req/__init__.py:5: in <module>
    from pip._internal.utils.logging import indent_log
/var/lib/awx/venv/awx/lib64/python3.9/site-packages/pip/_internal/utils/logging.py:23: in <module>
    from pip._internal.exceptions import DiagnosticPipError
/var/lib/awx/venv/awx/lib64/python3.9/site-packages/pip/_internal/exceptions.py:13: in <module>
    from pip._vendor.requests.models import Request, Response
/var/lib/awx/venv/awx/lib64/python3.9/site-packages/pip/_vendor/requests/__init__.py:135: in <module>
    from . import utils
/var/lib/awx/venv/awx/lib64/python3.9/site-packages/pip/_vendor/requests/utils.py:42: in <module>
    DEFAULT_CA_BUNDLE_PATH = certs.where()
/var/lib/awx/venv/awx/lib64/python3.9/site-packages/pip/_vendor/certifi/core.py:50: in where
    _CACERT_CTX = get_path("pip._vendor.certifi", "cacert.pem")
/usr/lib64/python3.9/importlib/resources.py:161: in path
    reader = _get_resource_reader(_get_package(package))
/usr/lib64/python3.9/importlib/resources.py:76: in _get_resource_reader
    spec.loader.get_resource_reader(spec.name))
E   ModuleNotFoundError: No module named 'importlib.readers'
================================================================================== short test summary info ===================================================================================
ERROR broken.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
================================================================================ 5 warnings, 1 error in 1.15s ================================================================================

[tsibley]

I also just ran into this today and came here to report the same issue.

[mdmintz]

Here's the temporary fix for requirements.txt files:

pytest==7.0.0;python_version>="3.6" and python_version<"3.9"
pytest==6.2.5;python_version>="3.9" and python_version<"3.10"
pytest==7.0.0;python_version>="3.10"

From my testing, only Python 3.9 seems to be effected. The improvements of pytest 7.0.0 are so nice, that this temporary workaround may be good enough until the next release is out with the fix. Just because Python 3.9 needs to sit this one out doesn't mean that 3.6, 3.7, 3.8, and 3.10 need to sit out with it. 🙂

[tsibley]

@mdmintz Ah, I also took the same approach of using environment markers in the dependency specs, but used a simpler spec:

pytest; python_version != '3.9'
pytest !=7.0.0; python_version == '3.9'

The hope is that the next release of pytest will fix this bug and thus the spec will be self-healing and allow 3.9 to use 7.0.x. (And CI will break again for 3.9 if it's not fixed in the next release.)