invalid knowledge of `self` variables retained after self call, unreachable false positive, invalid operations allowed

[mortoray]

The --warn-unreachable check is reporting a false positive for a branch on an instance field that it thinks it knows the value of, but it cannot. This is when a function calls another member function prior to the branch.

from typing import *

class Unreachable:
	def member_function(self) -> None:
		self.path: Optional[str] = None
		self.other()
		if self.path:
			print(self.path)
			
	def other(self) -> None:
		self.path = 'hi'
		
Unreachable().member_function()

The call to print(self.path) is marked as unreachable.

I presume this happens since mypy is seeing the None assignment to self.path, no intervening assignment, and thus assuming it is still None by the if self.path condition.

The call to self.other() however should invalidate any knowledge mypy has about self. instance variables. It cannot know how the other function modifies self.

Note: the error also happens inside an __init__ function, which is where I discovered it. I wanted to see if it was more general.

mypy: 0.770

[mortoray]

The problem is likely more generic than the warn-unreachable problem, since it can trigger other errors as well. Here's an expanded test case:

from typing import *

class Unreachable:
	path: Optional[str]
	vary: Union[int, str]
	
	def member_function(self) -> None:
		self.path = None
		self.other()
		if self.path:
			print(self.path)
			
	def other(self) -> None:
		self.path = 'hi'
		self.vary = 'hello'
		
	def alter_type(self) -> None:
		self.vary = 13
		self.other()
		if isinstance(self.vary, str):
			print(self.vary)
			
	
Unreachable().member_function()
Unreachable().alter_type()

The instance line is also triggering the error Subclass of "int" and "str" cannot exist: would have incompatible method signatures. As well as triggering the unreachable on the print statement two lines after.

So it does appear mypy is retaining knowledge about assignments to self fields after an intervening call to a function on self, which is wrong. Either it can know nothing about the values of self, or it needs to consider the other call and annotate with knowledge of what it can possible modify.

[mortoray]

Here's an example that shows a more problematic case, where it falsely assumes a type and allows an invalid statement to go unnoticed:

from typing import *

class Unreachable:
	vary: Union[int, str]
	
	def member_function(self) -> None:
		self.vary = 13
		self.other()
		
		value: int = self.vary + 1
		print(value)
			
	def other(self) -> None:
		self.vary = 'hello'
			
	
Unreachable().member_function()

value: int = self.vary + 1 should flag an error about Unsupported operand types for + ("str" and "int") but it does not, since mypy is falsely believing that self.vary is the integer 13 at this point.

[JukkaL]

This is a known issue, but I'm not sure if there's anything we can reasonably do to make this better.

Almost any operation could change the type of almost any attribute in Python, so mypy optimistically assumes that function calls don't change the types of attributes. For example, this code is not safe:

if x.y is not None and x.z is not None:
    print(x.y + x.z)

It's possible that x.z would actually be a property that modifies x.y to be None! Even local variables can be modified by called functions.

Because of the dynamic nature of Python, the only reasonable thing mypy can do is to be somewhat optimistic. This works maybe 99% of the time. I guess we might plausibly add a strictness option to only narrow down types of local variables. If this was enabled, self.x = y wouldn't narrow down the type of self.x, and neither would if isinstance(self.x, ...).

In your example, my suggestion is to not use isinstance(x.y) but instead do something like t = x.y and isinstance(t, ...).

[mortoray]

I feel as though this is different than the situations where a type is inferred, because I've provided an explicit type here:

self.path: Optional[str] = None

So I think your comment about a strictness option and local variables would make sense. I'd prefer not to narrow down the type of self.

Indeed, I would even be okay with an error on your if-statement. By comparison, the TypeScript compiler will complain about such things. If you wish to narrow a type you need to assign to a temporary where it's guaranteed the type will be narrowed. Access to field variables of other objects shouldn't be narrows in context since it's not possible to know if it remains the same or not.

[hauntsaninja]

This is a won't fix. Never narrowing is not an option. Non-local analysis is expensive. I'm not aware of another Python static type checker that handles this in the ideal way.