./runtests.sh does not pick up on missing imports

[rijobro]

Describe the bug
As can be seen here: https://github.com/Project-MONAI/MONAI/runs/4802838071,
the .runtests.sh passes (the job named flake8), yet the tests fail, due to missing imports: https://github.com/Project-MONAI/MONAI/runs/4802838468?check_suite_focus=true#step:10:11980.

Expected behavior
Our runtests.sh script should pick up on these.

[wyli]

Hi @rijobro, that specific part is not actually via runtests.sh:

MONAI/.github/workflows/pythonapp.yml

Lines 101 to 107 in f089659

	- name: Run quick tests (CPU ${{ runner.os }})
	run: |
	python -c 'import torch; print(torch.__version__); print(torch.rand(5,3))'
	python -c "import monai; monai.config.print_config()"
	python -m unittest -v
	env:
	QUICKTEST: True

If you want to debug locally, the command would be: QUICKTEST=True python -m unittest -v

[rijobro]

We use mypy to check for missing imports on the main code, but runningmypy tests/test_*.py reported no errors strangely.

I think I've fixed it by using pylint (extra dependency) that also runs on the tests folder. Will submit a PR for review.

[wyli]

I see, probably the flake8 config suppressed warnings

MONAI/setup.cfg

Line 101 in f089659

ignore =

[rijobro]

I should have specified that after some debugging, I found that missing imports are detected for files inside of the subfolder monai, but not detected in other subfolders, such as tests.

[wyli]

Yes, I found the same issue a while ago #3462 but didn't look into the root cause. now I know it's not detected because of ignoring F821

MONAI/setup.cfg

Line 108 in b9ef673

F821

#3659 addresses it, and it's also good to have the pylint checks, thanks!

[rijobro]

Great, good hunting!