Workaround for #3742

[nicoddemus]

I did not have time to investigate the issue yet, but took a few minutes to produce a failing test for #3742.

At first I'm not sure how to fix the regression, so I would appreciate suggestions here.

cc @jonozzz

[nicoddemus]

This patch fixes the test and breaks another test:

diff --git a/src/_pytest/main.py b/src/_pytest/main.py
index 9599fa16..1aee4bfd 100644
--- a/src/_pytest/main.py
+++ b/src/_pytest/main.py
@@ -482,6 +482,7 @@ class Session(nodes.FSCollector):
             self.trace.root.indent -= 1

     def _collect(self, arg):
+        from _pytest.python import Package
         names = self._parsearg(arg)
         argpath = names.pop(0)
         paths = []
@@ -503,7 +504,7 @@ class Session(nodes.FSCollector):
                             root = self._node_cache[pkginit]
                         else:
                             col = root._collectfile(pkginit)
-                            if col:
+                            if col and isinstance(col, Package):
                                 root = col[0]
                                 self._node_cache[root.fspath] = root

With the above patch, all tests pass except for this one:

________________________________ Test_getinitialnodes.test_pkgfile ________________________________
[gw4] win32 -- Python 3.6.3 c:\pytest\.env36\scripts\python.exe

self = <test_collection.Test_getinitialnodes object at 0x00000283A5DB5630>
testdir = <Testdir local('C:\\Users\\Bruno\\AppData\\Local\\Temp\\pytest-of-Bruno\\pytest-535\\popen-gw4\\test_pkgfile0')>

    def test_pkgfile(self, testdir):
        tmpdir = testdir.tmpdir
        subdir = tmpdir.join("subdir")
        x = subdir.ensure("x.py")
        subdir.ensure("__init__.py")
        with subdir.as_cwd():
            config = testdir.parseconfigure(x)
        col = testdir.getnode(config, x)
        assert isinstance(col, pytest.Module)
        assert col.name == "x.py"
>       assert col.parent.parent.parent is None
E       AttributeError: 'NoneType' object has no attribute 'parent'

c:\pytest\testing\test_collection.py:650: AttributeError

But this test has been changed in c02e8d8 as part of the package-scope fixtures support.

For the record the patch above feels like a hack, but I don't have time today to investigate this further. I would appreciate further suggestions here, and if someone finds a proper solution feel free to push to this branch.

I would like to get 3.7.1 out with a fix ASAP given that this broke a lot of plugins out there.

[nicoddemus]

Decided to push the patch above so we can see how it fares under CI and gather feedback.

Unless someone comes up with a better solution I'm considering releasing this fix as it is, at least until we have some time to work on a long term solution.

[jonozzz]

I've started looking at this issue today. I applied that patch and it does seem to fix it, however it'll skip the init.py files (tested with flake8 only).

It does look like a hack and I'll see if there's a better fix.

[nicoddemus]

Thanks for the feedback @jonozzz!

[coveralls]

Coverage increased (+0.02%) to 92.514% when pulling 8c9efd8 on nicoddemus:collect-file-bug into 2534193 on pytest-dev:master.

[RonnyPfannschmidt]

@jonozzz i beleive the deeper issue comes from packags being collected on the same level as their modules and then dancing a dance around the collectors - a package should be collected from the fspath of a folder

its not yet clear how to reintegrate that propperly

[nicoddemus]

Also need to check if this fixes #3749

[nicoddemus]

@RonnyPfannschmidt, you marked test_collect_custom_items as xfail, but this test reproduces the problem and this patch fixes it. 😅

[RonnyPfannschmidt]

yikes, my bad

[nicoddemus]

I guess we can merge this once CI passes?

We definitely should investigate a better solution though.

[nicoddemus]

@RonnyPfannschmidt want to merge this?

[cybergrind]

@nicoddemus seems that it doesn't work for pytest-asyncio.
https://travis-ci.org/tipsi/aiozk/builds/411126128#L2148

[RonnyPfannschmidt]

@cybergrind this one does not even try to sort out that issue - #3754 does

[RonnyPfannschmidt]

this one is part of creating headaches - in future we def need to sort that out

[nicoddemus]

Sure, I'm hoping @jonozzz can come up with a good solution to this. 😁

[jonozzz]

Totally agree, this needs a major rework, at least the code that went into main.py.

[cybergrind]

Oh, just read to message about async_generator and scrolled down to possible patch. Will be more careful next time

[drdavella]

We were bitten by the bug reported in #3742, but the fix has led to a new issue: scientific-python/pytest-doctestplus#22.

I haven't done enough investigation to determine if there's a workaround on our end, but I just wanted to report it here in case anyone else is experiencing similar problems as of 3.7.1.