0.920: mypy failure claiming __path__ already exists

[ardumont]

Hello,

Thanks for mypy!

I discovered a recent change in behavior regarding the following code [1]. Which worked
well under mypy up to 0.910 and is currently breaking on the new release 0.920.

I don't find any issue related to this nor any changelog entries which mentions anything
about this. I've marked this as a bug since the behavior changed but feel free to
recategorize if i'm wrong.

Do you have any hints as to what's the problem here?

Note that this is pkgutil namespace init.py that's replicated in twenty modules or
so. But currently, that problem appears in each module separately (see for example those
builds [3] [4]).

[1]

$ cat swh/__init__.py
from pkgutil import extend_path
from typing import List

__path__: List[str] = extend_path(__path__, __name__)

[2]

$ uname -a
Linux yavin4 5.10.0-9-amd64 #1 SMP Debian 5.10.70-1 (2021-09-30) x86_64 GNU/Linux
$ python --version
Python 3.9.2
$ mypy --version
mypy 0.910
$ mypy swh
Success: no issues found in 10 source files
$ pip install mypy --upgrade
Collecting mypy
  Using cached mypy-0.920-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (24.6 MB)
Requirement already satisfied, skipping upgrade: mypy-extensions<0.5.0,>=0.4.3 in /home/tony/.virtualenvs/swh/lib/python3.9/site-packages (from mypy) (0.4.3)
Requirement already satisfied, skipping upgrade: typing-extensions>=3.7.4 in /home/tony/.virtualenvs/swh/lib/python3.9/site-packages (from mypy) (3.10.0.2)
Collecting tomli<3.0.0,>=1.1.0
  Using cached tomli-2.0.0-py3-none-any.whl (12 kB)
Installing collected packages: tomli, mypy
  Attempting uninstall: mypy
    Found existing installation: mypy 0.910
    Uninstalling mypy-0.910:
      Successfully uninstalled mypy-0.910
Successfully installed mypy-0.920 tomli-2.0.0
$ mypy --version
mypy 0.920
$ mypy swh
swh/loader/__init__.py:4: error: Name "__path__" already defined (possibly by an import)
swh/__init__.py:4: error: Name "__path__" already defined (possibly by an import)
Found 2 errors in 2 files (checked 10 source files)

[3] https://jenkins.softwareheritage.org/view/swh-draft/job/DCORE/job/tests/1577/console

[4] https://jenkins.softwareheritage.org/view/swh-draft/job/DLDBASE/job/tests/1738/console

[cdce8p]

Try removing the type annotation from __path__. That should fix the issue. For reference: #9454

[ardumont]

That does indeed work [1].

As it's more glue work than actual code, I guess it's ok to apply your suggestion.
Thanks.

$ cat swh/__init__.py
from pkgutil import extend_path

__path__ = extend_path(__path__, __name__)
$ cat swh/loader/__init__.py
from pkgutil import extend_path

__path__ = extend_path(__path__, __name__)
$ mypy --version
mypy 0.920
$ mypy swh
Success: no issues found in 10 source files

[cdce8p]

Actually mypy will still infer __path__ to be list[str] with #9454, so you don't loss anything. Try adding reveal_type(__path__) after extend_path and run mypy. Just make sure to remove reveal_type afterwards, it will crash Python otherwise 😉

[ardumont]

Thanks for the reveal_type hint!

Correct:

$ cat swh/__init__.py
from pkgutil import extend_path

__path__ = extend_path(__path__, __name__)

print(reveal_type(__path__))
$ mypy swh
swh/__init__.py:5: note: Revealed type is "builtins.list[builtins.str]"

FWIW, I'm gonna apply the suggestion [1] to our swh stack of code to unstuck our current
failing builds.

Thanks again.

[1] https://forge.softwareheritage.org/D6853

[cdce8p]

You don't need to wrap reveal_type in print. It's not a Python builtin, just a signal for type checkers to emit a note. That's why it would fail at runtime.

As the issue is solved, can you close it?

[ardumont]

Yes, I was coming back to close it.
It's all done.

Thanks again.