Skip to content

Improve the output of tests/stubtest_third_party.py when it fails#11763

Merged
AlexWaygood merged 3 commits intopython:mainfrom
AlexWaygood:helpful-tests
Apr 14, 2024
Merged

Improve the output of tests/stubtest_third_party.py when it fails#11763
AlexWaygood merged 3 commits intopython:mainfrom
AlexWaygood:helpful-tests

Conversation

@AlexWaygood
Copy link
Copy Markdown
Member

Mostly changes to enable us to directly click through from the GitHub logs if the daily test fails, rather than having to manually figure out what the upstream repository is or where the typeshed stubs are located.

Previous output if it failed:
(typeshed) (main)⚡ % py tests/stubtest_third_party.py requests                                                                       ~/dev/typeshed
requests... Note: requests is not currently tested on darwin in typeshed's CI.
fail

/var/folders/gd/1czdxc454j15y8gns2krv8vc0000gn/T/tmpq5w04nqe/bin/pip install requests[socks]==2.31.* mypy==1.9.0 urllib3>=2
MYPYPATH=/Users/alexw/dev/typeshed/stubs/requests /var/folders/gd/1czdxc454j15y8gns2krv8vc0000gn/T/tmpq5w04nqe/bin/python -m mypy.stubtest --custom-typeshed-dir /Users/alexw/dev/typeshed requests --allowlist /Users/alexw/dev/typeshed/stubs/requests/@tests/stubtest_allowlist.txt

error: requests.FOO is not present at runtime
Stub: in file /Users/alexw/dev/typeshed/stubs/requests/requests/__init__.pyi:40
builtins.str
Runtime:
MISSING

note: unused allowlist entry requests.compat.bytes.__buffer__
Found 2 errors (checked 18 modules)

Python version: 
Python 3.12.2 (main, Feb 15 2024, 19:30:27) [Clang 15.0.0 (clang-1500.1.0.2.5)]

Ran with the following environment:
certifi==2024.2.2
charset-normalizer==3.3.2
idna==3.7
mypy==1.9.0
mypy-extensions==1.0.0
pip==24.0
PySocks==1.7.1
requests==2.31.0
typing_extensions==4.11.0
urllib3==2.2.1

To fix "unused allowlist" errors, remove the corresponding entries from /Users/alexw/dev/typeshed/stubs/requests/@tests/stubtest_allowlist.txt
Screenshot

image

New output
(typeshed) (helpful-tests)⚡ % py tests/stubtest_third_party.py requests                                                              ~/dev/typeshed
requests... Note: requests is not currently tested on darwin in typeshed's CI.
fail

**********************************************************************
Commands run:

/var/folders/gd/1czdxc454j15y8gns2krv8vc0000gn/T/tmpi8lk5p38/bin/pip install requests[socks]==2.31.* mypy==1.9.0 urllib3>=2
MYPYPATH=/Users/alexw/dev/typeshed/stubs/requests /var/folders/gd/1czdxc454j15y8gns2krv8vc0000gn/T/tmpi8lk5p38/bin/python -m mypy.stubtest --custom-typeshed-dir /Users/alexw/dev/typeshed requests --allowlist /Users/alexw/dev/typeshed/stubs/requests/@tests/stubtest_allowlist.txt

**********************************************************************
Command output:

error: requests.FOO is not present at runtime
Stub: in file /Users/alexw/dev/typeshed/stubs/requests/requests/__init__.pyi:40
builtins.str
Runtime:
MISSING

note: unused allowlist entry requests.compat.bytes.__buffer__
Found 2 errors (checked 18 modules)

**********************************************************************
Upstream repository: https://github.com/psf/requests
Typeshed source code: https://github.com/python/typeshed/tree/main/stubs/requests
Python version: Python 3.12.2 (main, Feb 15 2024, 19:30:27) [Clang 15.0.0 (clang-1500.1.0.2.5)]

Ran with the following environment:
certifi==2024.2.2
charset-normalizer==3.3.2
idna==3.7
mypy==1.9.0
mypy-extensions==1.0.0
pip==24.0
PySocks==1.7.1
requests==2.31.0
typing_extensions==4.11.0
urllib3==2.2.1

To fix "unused allowlist" errors, remove the corresponding entries from stubs/requests/@tests/stubtest_allowlist.txt

**********************************************************************
New screenshot

image

Copy link
Copy Markdown
Collaborator

@Avasam Avasam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just wondering if you meant to include the stubsabot change here. Either way I'm perfectly fine with these changes. Really helps readability.

@AlexWaygood AlexWaygood merged commit 7c8e82f into python:main Apr 14, 2024
@AlexWaygood AlexWaygood deleted the helpful-tests branch April 14, 2024 22:23
@Avasam
Copy link
Copy Markdown
Collaborator

Avasam commented Apr 20, 2024

@AlexWaygood
Copy link
Copy Markdown
Member Author

Yeah, I saw... Will try to fix today :/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants