tests: improve test for nose.raises

[blueyed]

This should probably get transferred into a pytest.fail really, but
tests/documents the current behavior.

[blueyed]

Without the cast here it would display "test_raises.py::test_raises_runtimeerror <- /…/test_raises0/test_raises.py PASSED` here.

[nicoddemus]

Hmm sorry, what "cast" you mean?

[blueyed]

@nicoddemus https://github.com/blueyed/pytest/blob/156a41a855d28ae9f6d83545af69b51fb26f199e/src/_pytest/terminal.py#L747-L752

Since location fails to make the path relative it is absolute, and then triggers that code.

[nicoddemus]

LGTM but @bluetech specifically said he would like to revisit his original commit instead of reverting it, so let's wait for his review.

[blueyed]

This might be triggered via

pytest/src/_pytest/python.py

Lines 288 to 292 in 156a41a

	# nose compatibility
	fspath = sys.modules[obj.__module__].__file__
	if fspath.endswith(".pyc"):
	fspath = fspath[:-1]
	lineno = compat_co_firstlineno

.

[nicoddemus]

Thanks I was wondering this myself because a cursory look at nose.py did not bring anything to light.

[bluetech]

Thanks @blueyed for hardening the test! The test improvements looks good to me, the rest I can't quite assess ATM.

From my point of view I've come up with similar / the same like he did initially, except for that I've noticed (and IIRC mentioned in the review) that it might not be 100% correct, what we've seen now.

You did -- the revert PR message explains why I was sure the assert was safe, but I'll be more careful with these. Amusingly, this is just the sort of bugs that types help prevent, and we'll get there eventually, but the road can be bumpy :)

---

One thing I'm not sure about in general regarding this part of the code is: are custom Items a thing? i.e. are there plugins which create subclasses of Item? I see that at least the internal doctest plugin does this, so I assume yes. And if so, is it considered to be public API? Is reportinfo public API (not allowed to break unscrupulously)?

[blueyed]

One thing I'm not sure about in general regarding this part of the code is: are custom Items a thing? i.e. are there plugins which create subclasses of Item? I see that at least the internal doctest plugin does this, so I assume yes. And if so, is it considered to be public API? Is reportinfo public API (not allowed to break unscrupulously)?

Yes, I would say it is public, and therefore we need to handle strs returned from it, as does pytest itself internally still.
With your revert I've left out the adjustment to the test (test_itemreport_reportinfo) that you had to adjust, which kind of showed the issue already.
With the hardened test -vv triggers similar code.

[bluetech]

Yes, I would say it is public, and therefore we need to handle strs returned from it, as does pytest itself internally still.

OK. I think it will be better to use the same signature for all reportinfo functions, then. Here are some suggestions (untested...):

diff --git a/src/_pytest/doctest.py b/src/_pytest/doctest.py
index d7ca888cc..ffc1890d4 100644
--- a/src/_pytest/doctest.py
+++ b/src/_pytest/doctest.py
@@ -308,7 +308,7 @@ class DoctestItem(pytest.Item):
         else:
             return super().repr_failure(excinfo)
 
-    def reportinfo(self) -> Tuple[py.path.local, int, str]:
+    def reportinfo(self) -> Tuple[Union[str, py.path.local], int, str]:
         return self.fspath, self.dtest.lineno, "[doctest] %s" % self.name
 
 
diff --git a/src/_pytest/nodes.py b/src/_pytest/nodes.py
index 080f079cd..ab976efae 100644
--- a/src/_pytest/nodes.py
+++ b/src/_pytest/nodes.py
@@ -462,9 +462,10 @@ class Item(Node):
     @cached_property
     def location(self) -> Tuple[str, Optional[int], str]:
         location = self.reportinfo()
-        fspath = location[0]
-        if not isinstance(fspath, py.path.local):
-            fspath = py.path.local(fspath)
-        fspath = self.session._node_location_to_relpath(fspath)
+        if isinstance(location[0], py.path.local):
+            fspath = location[0]
+        else:
+            fspath = py.path.local(location[0])
+        relfspath = self.session._node_location_to_relpath(fspath)
         assert type(location[2]) is str
-        return (fspath, location[1], location[2])
+        return (relfspath, location[1], location[2])
diff --git a/src/_pytest/python.py b/src/_pytest/python.py
index 3d2916c83..82dca3bcc 100644
--- a/src/_pytest/python.py
+++ b/src/_pytest/python.py
@@ -369,7 +369,12 @@ class PyCollector(PyobjMixin, nodes.Collector):
                 if not isinstance(res, list):
                     res = [res]
                 values.extend(res)
-        values.sort(key=lambda item: item.reportinfo()[:2])
+
+        def sort_key(item):
+            fspath, lineno, _ = item.reportinfo()
+            return (str(fspath), lineno)
+
+        values.sort(key=sort_key)
         return values
 
     def _makeitem(self, name, obj):

[blueyed]

@bluetech

I think it will be better to use the same signature for all reportinfo functions, then.

Why? If it is known to return a str only that should be reflected in the signature, no?

[bluetech]

I think it adds to the confusion. When all the subclasses publicize the same interface that the base class dictates, there is less chance of errors creeping in when the types are interchanged. But you can leave it as just py.path.local if you disagree.

[blueyed]

@bluetech
I think it makes it clearer in general - especially with regard to removing the py.path.local instances in the long run it is good to know where only strs are returned by now already.
So I'd rather leave it as-is (i.e. more strict / describing the actual return values).

[blueyed]

@bluetech are you OK with it then?

[bluetech]

LGTM with a couple of suggestions.

I also think we should fold this change in here as well:

diff --git a/src/_pytest/python.py b/src/_pytest/python.py
index 3d2916c83..82dca3bcc 100644
--- a/src/_pytest/python.py
+++ b/src/_pytest/python.py
@@ -369,7 +369,12 @@ class PyCollector(PyobjMixin, nodes.Collector):
                 if not isinstance(res, list):
                     res = [res]
                 values.extend(res)
-        values.sort(key=lambda item: item.reportinfo()[:2])
+
+        def sort_key(item):
+            fspath, lineno, _ = item.reportinfo()
+            return (str(fspath), lineno)
+
+        values.sort(key=sort_key)
         return values
 
     def _makeitem(self, name, obj):

The idea is that since item.reportinfo()[1] is Union[str, py.path.local], sorting by it can compare str with py.path.local. While this does the right thing, I find it distasteful, so I prefer that the str(fspath) is explicit.

[bluetech]

Thanks, LGTM now, if CI passes.