Tests not Reporting Status if One of the Test Folder are Outside of Workspace Folder

[AngellusMortis]

Environment data

• VS Code version: 1.35 (Insiders)
• Extension version (available under the Extensions sidebar): 2019.4.12954
• OS and version: Ubuntu 18.04 LTS (interrupter is running inside of Debian 9 container via Remote Container feature)
• Python version (& distribution if applicable, e.g. Anaconda): 3.7
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): Remote Container, system Python
• Relevant/affected Python packages and their versions: pytest?

Expected behavior

After running tests via PyTest, the status symbol next to them should change from a ? to either based on test success or failure.

Actual behavior

After running tests via PyTest, the status symbol never stop showing as ?.

Steps to reproduce:

1. Create Python project with tests
2. Create a second Python project with tests next to the first one
3. Open the second Python project as project folder in VS Code
4. Configure PyTest to run tests for both projects

The use case for this is that we have shared "framework" of code that all of our projects are built that has a set of test that get run to make sure our code does not break the framework. I do not want VS Code running full linting/code checks on the framework because that is not the code I am working on. Just need to make sure tests pass.

Ex. (assuming 1. is at /project1 and 2. is at /project2, and /project2 is the folder you have open in VS Code)

{
    "python.testing.pyTestEnabled": true,
    "python.testing.pyTestArgs": [
       "/project1",
       "/project2",
    ],
}

Also, if project1 is not pip install -e in your environment, you will need to set a $PYTHONPATH environment variable.

Logs

Nothing useful. Just a lot of "Reloading modules... done. Analysis restarted." Python Test Log has nothing but standard PyTest output, no types of errors. Console under Developer tools has nothing useful enough, just "[Extension Host] Python Extension: Cached data exists getEnvironmentVariables, <workspace_path>"

[DonJayamanne]

Please could you provide a sample repo so we can replicate this issue.

[AngellusMortis]

I figured out this is two pronged. If you have pytest.ini in a parent directory (outside of workspace) or you have tests that runs outside of workspace, it causes test results to not show up correctly.

I will try to slap together a sample repo this weekend if I can.

[AngellusMortis]

Sorry it took so long for me to make this, but here is a repo you should be able to reproduce the issue with.

https://github.com/AngellusMortis/pytest-vscode

I have a Windows 10 machine, so I generally only use the Remote Development extensions now. I cannot promise it is still reproducible without them, but I imagine it should be.