-
-
Notifications
You must be signed in to change notification settings - Fork 3.1k
Closed
Description
The "pythoneval" tests are by far the biggest contributor to the time it takes to run our test suite. For each of 171 sample programs, we type-check them and then run them through the Python interpreter.
From discussion in #1668, running the programs is no longer important for our testing. So cut it out. This will mean
- cutting out that step in
testpythoneval.py, - cutting out the program output (as opposed to the type-checker output) from all the test cases, and
- cutting out entirely any cases that no longer seem to have any meaningful content when they're only for the type-checker.
Some of the test cases may exercise features of typing in interesting ways -- any of those should be moved to the typing repo's own test suite if they aren't covered there already.
After this is done, we can probably get a big further speedup by folding the remaining test cases into just more cases for testcheck.py, so that we're not forking a mypy subprocess for each one. Will make a separate issue for that.