How should type checkers type check equality?

[JukkaL]

Right now typeshed defines equality to be defined for each pair of types, even though something like n == (m,) is almost certainly a bug if both n and m are integers, for example. There is no runtime error, so not rejecting it would also be somewhat reasonable (also, a subclass of int might plausibly return True when compared against a tuple, though this more of a theoretical concern). It's unclear how type checkers should deal with these. As equality checking is very common, and anecdotally errors related to equality checking are not infrequent, we may want to spell out how this should be done in PEP 484, so that all tools would handle these consistently.

Here are some options:

• Leave it up to individual type checkers to figure this out, and make the signatures of __eq__ and __ne__ accept arbitrary objects.
• Make the signatures of __eq__ and __ne__ accept arbitrary objects, but require type checkers to have special, well-defined rules for type checking == and != operations so that we can reject n == (m,) and other similar cases consistently.
• Encode the information about what a particular type can be equal to in the signature of __eq__ and __ne__. This may require special rules for these methods, since we may want to narrow down the argument type in a subclass (as it should be possible to compare object to object), but normally method argument types vary contravariantly so this is not allowed. For example, int.__eq__ would only accept int arguments, since this seems to return NotImplemented for other argument types. It's still unclear what the exact rules for type checking __eq__ and __ne__ should be.
• Add a new type system concept to PEP 484 and typing to represent "comparable types" or "equatable types". For example, I might want to define function isequal(x, y) that takes two arguments that can be of arbitrary types as long as they might compare equal to each other. isequal(1, 1.1) should be okay but isequal(1, 'x') should be rejected. We'd then use this concept to type check == and != as well. This could be useful for things like __contains__. If the result of an in operation is known to be False statically, we should perhaps reject it as well.

Also see python/mypy#1271 and the conversation in python/typeshed#874.

[vlasovskikh]

If we warn about 2 == '3' even though it's OK at run-time, the users of type checkers will see a false error. I believe a type checker could have a non-standard flag to check if objects are comparable to the objects of the same sub- / super- / type.

I would suggest adding strict types for equality methods to typeshed stubs only if they are really strict for these objects at run-time.

[markshannon]

I think it is OK to warn that 2 == '3' is always false. Whether it is a type error is debatable.

Adding special cases like this to typecheckers doesn't seem sustainable,
mypy can do what it wants, but we really do not need more special cases in the PEP.

If we want to distinguish between always True or always False and other comparisons, then the best approach is add the information to typeshed.
I think the problem is that the correct formulations are convoluted and no one wants to write them by hand. (including me).

For example, if tuple.__eq__ were

     def __eq__(self, x:object) -> bool:
         return False
    @overload
    def __eq__(self, x: Tuple[_T_co, ...]) -> bool: ...

then a simple checker would just see def __eq__(self, x:object) -> bool, but a more sophisticated checker would be able to tell that (where t is a tuple):
t == None is always false, but that t = (None, None) is OK.

[JukkaL]

If we let each type checker figure out this separately, we still have the question of how to annotate __eq__ and friends in typeshed.

We have at least these options:

1. Use def __eq__(self, x: object) -> bool: ..., even though the method may also return NotImplemented. This is what typeshed has right now. The rationale for ignoring NotImplemented is that object().__eq__ in Python 3 may return NotImplemented for some instances of object but not others, i.e. we have no way of specifying the signature precisely if we insist on a bool return type: o.__eq__(o) evaluates to True (assuming o = object()) but o.__eq__(object()) evaluates to NotImplemented, not False. Also, object().__eq__(1) is NotImplemented.
2. Use def __eq__(self, x: Foo) -> bool: ... where Foo describes for which types the method returns a non-NotImplemented value. As discussed above, we can't actually do this consistently for object at least, and this doesn't mix well with how signature compatibility of overloads is normally checked.
3. Usedef __eq__(self, x: object) -> Union[bool, NotImplemented]: ... or similar. This would be kind of consistent with how things actually work, but feels inconsistent with other, non-comparison operator methods. This is also a pretty awkward signature to write in user-defined classes that define __eq__. We could have something similar to Optional[x] that could be used as a shorthand for Union[x, NotImplemented], though.
4. Like one of the above, but optionally use an overload with explicit return False (or maybe even return True) in some overload items to help type checkers give warnings about always-false equality comparisons. (Idea by @markshannon)

I think that the existing approach (1) is good enough, and the alternatives aren't (at least significantly) better. However, (3) or (3) + (4) would arguably be more correct ways to do this.

[ilevkivskyi]

I like the idea of @markshannon. It is practical and simple to understand for a reader. Alternatively, one can use self type in the typeshed for object.__eq__ but I am not sure mypy fully supports self types already.

[ilevkivskyi]

Note that mypy now has an experimental --strict-equality flag for some always false checks.

[srittau]

Is this something that needs more discussion? I am going to close this for now, but please reopen or bring it up on typing-sig if you still think this is an issue.