From 527e86f5dfb6d2090189435ed1db961ce0575b57 Mon Sep 17 00:00:00 2001 From: jorenham Date: Sun, 25 Jul 2021 02:34:34 +0200 Subject: [PATCH 01/11] Variance operators for generic type variables: +T and -T First version of the abstract, rationale and proposal --- pep-9999.rst | 87 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 87 insertions(+) create mode 100644 pep-9999.rst diff --git a/pep-9999.rst b/pep-9999.rst new file mode 100644 index 00000000000..25887153b6f --- /dev/null +++ b/pep-9999.rst @@ -0,0 +1,87 @@ +PEP: 9999 +Title: Variance operators for generic type variables: +T and -T +Author: Joren Hammudoglu +Status: Active +Type: Process +Content-Type: text/x-rst +Created: 24-Jul-2021 +Post-History: 24-Jul-2021 + + +.. note:: + For those who have written a PEP before, there is a template_. + +Abstract +======== + +This PEP proposes a concise syntax for specifying co- and contravariant +type variables when defining generic classes. + + + +Motivation +========= + +PEP 484 [1] proposes + + +Rationale +========= + + + +Proposal +======== + + +Specification +============= + + +How to Teach This +================= + + +Suggested Sections +================== + + +Rejected Ideas +============== + + +References +========== + +.. [1] PEP 484, Type Hints, BDFL, Lehtosalo, Langa + https://www.python.org/dev/peps/pep-0484/ +.. [2] Scala Variance + https://docs.scala-lang.org/scala3/book/types-variance.html + + +Copyright +========= + +This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive. + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: + + + +Copyright +========= + +This document is placed in the public domain or under the +CC0-1.0-Universal license, whichever is more permissive. + + +Various sections are found to be common across PEPs and are outlined in +PEP 1 [1]_. Those sections are provided here for convenience. From fdcf4d9b2e27184ac04681b1aeeb62b3d1ea40f4 Mon Sep 17 00:00:00 2001 From: jorenham Date: Sun, 25 Jul 2021 02:43:38 +0200 Subject: [PATCH 02/11] Syntax fix --- pep-9999.rst | 85 +++++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 74 insertions(+), 11 deletions(-) diff --git a/pep-9999.rst b/pep-9999.rst index 25887153b6f..2f6dbbbb9e4 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -2,42 +2,105 @@ PEP: 9999 Title: Variance operators for generic type variables: +T and -T Author: Joren Hammudoglu Status: Active -Type: Process +Type: Standards Track Content-Type: text/x-rst Created: 24-Jul-2021 Post-History: 24-Jul-2021 -.. note:: - For those who have written a PEP before, there is a template_. - Abstract ======== -This PEP proposes a concise syntax for specifying co- and contravariant -type variables when defining generic classes. +PEP 484 proposes a syntax for defining the `covariance and contravariance +`_ +of `user-defined generic types +`_:: + T_co = typing.TypeVar('T_co', covariant=True) + T_contra = typing.TypeVar('T_contra', contravariant=True) -Motivation -========= + # a covariant generic type + class EventDispatcher(typing.Generic[T_co]): + def dispatch() -> T_co: ... + + # a contravariant generic type + class EventListener(typing.Generic[T_contra]): + def receive(event: T_contra) -> None: ... + + +This PEP proposes a concise syntax for specifying the variance of +generic types by overloading the ``+`` and ``-`` prefix operators +on type variables:: + + T = typing.TypeVar('T') + + + # a covariant generic type + class EventDispatcher(typing.Generic[+T]): + def dispatch() -> T: ... + + # a contravariant generic type + class EventListener(typing.Generic[-T]): + def receive(event: T) -> None: ... -PEP 484 [1] proposes Rationale ========= +Although defining the variance of generic type by using type variables +that are constructed with the ``covariant=True`` or ``contravariant=True`` +keyword arguments, it has downsides. + +This is why type checkers do no allow using co- or contravariant type +variables outside of generic classes. + + + - In PEP 484 it states: + + Covariance or contravariance is not a property of a type variable, + but a property of a generic class defined using this variable. + + Instead of defining variance on the type variable itself, it would + make more sense to mark the type variable as variant when defining + the generic type, and only there. + + - Because variance is a property of the generic type itself, type + checkers do no allow using co- or contravariant type variables + outside of generic classes. But as the variance is currently a + property of the type variables, the must be used within variant + generic classes. + + - Creating separate type variables for different variances often + results in up to 3x code duplication, especially for type variables + with constraints or an upper bound. + + - Although the terms "covariant" "contravariant" are descriptive, they + are verbose, and challenging concepts for people new to typed + languages. Proposal ======== +All of these issues can be solved by overloading the ``+`` and ``-`` +prefix operators of ``typing.TypeVar`` instances, so that ``+T`` is an +alias for a covariant type variable, and ``-T`` for a contravariant +one. These variance operators should only be used within the parameter +list of `typing.Generic` and `typing.Protocol` when subclassing them. + +This syntax is practically identical to type parameter variance in the +Scala programming language [2]_. + + Specification ============= + + How to Teach This ================= @@ -53,8 +116,8 @@ Rejected Ideas References ========== -.. [1] PEP 484, Type Hints, BDFL, Lehtosalo, Langa - https://www.python.org/dev/peps/pep-0484/ +.. [1] mypy + http://mypy-lang.org/ .. [2] Scala Variance https://docs.scala-lang.org/scala3/book/types-variance.html From ac952a5a807900279b4fd2ed244970462d624ea4 Mon Sep 17 00:00:00 2001 From: jorenham Date: Sun, 25 Jul 2021 02:45:57 +0200 Subject: [PATCH 03/11] enumeration indentation fix --- pep-9999.rst | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/pep-9999.rst b/pep-9999.rst index 2f6dbbbb9e4..925c5bb28f0 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -57,28 +57,28 @@ This is why type checkers do no allow using co- or contravariant type variables outside of generic classes. - - In PEP 484 it states: +- In PEP 484 it states: Covariance or contravariance is not a property of a type variable, but a property of a generic class defined using this variable. - Instead of defining variance on the type variable itself, it would - make more sense to mark the type variable as variant when defining - the generic type, and only there. + Instead of defining variance on the type variable itself, it would + make more sense to mark the type variable as variant when defining + the generic type, and only there. - - Because variance is a property of the generic type itself, type - checkers do no allow using co- or contravariant type variables - outside of generic classes. But as the variance is currently a - property of the type variables, the must be used within variant - generic classes. +- Because variance is a property of the generic type itself, type + checkers do no allow using co- or contravariant type variables + outside of generic classes. But as the variance is currently a + property of the type variables, the must be used within variant + generic classes. - - Creating separate type variables for different variances often - results in up to 3x code duplication, especially for type variables - with constraints or an upper bound. +- Creating separate type variables for different variances often + results in up to 3x code duplication, especially for type variables + with constraints or an upper bound. - - Although the terms "covariant" "contravariant" are descriptive, they - are verbose, and challenging concepts for people new to typed - languages. +- Although the terms "covariant" "contravariant" are descriptive, they + are verbose, and challenging concepts for people new to typed + languages. Proposal From 3cdd57e2847deb59ae108726845294fc32920ae0 Mon Sep 17 00:00:00 2001 From: jorenham Date: Mon, 2 Aug 2021 16:03:18 +0200 Subject: [PATCH 04/11] `Specification` and `Rejected Ideas` sections --- pep-9999.rst | 113 +++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 88 insertions(+), 25 deletions(-) diff --git a/pep-9999.rst b/pep-9999.rst index 925c5bb28f0..d9470600e8a 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -1,5 +1,5 @@ PEP: 9999 -Title: Variance operators for generic type variables: +T and -T +Title: Inline variance operators for generic type variables: ``+T`` and ``-T`` Author: Joren Hammudoglu Status: Active Type: Standards Track @@ -76,10 +76,6 @@ variables outside of generic classes. results in up to 3x code duplication, especially for type variables with constraints or an upper bound. -- Although the terms "covariant" "contravariant" are descriptive, they - are verbose, and challenging concepts for people new to typed - languages. - Proposal ======== @@ -92,33 +88,112 @@ one. These variance operators should only be used within the parameter list of `typing.Generic` and `typing.Protocol` when subclassing them. This syntax is practically identical to type parameter variance in the -Scala programming language [2]_. +Scala programming language [1]_. + Specification ============= +The new type variable variance syntax should be used within the generic +parameter list of `typing.Generic` or `typing.Protocol` base classes, +and only there. It is not allowed to be used on type variables that +are already marked as covariant or contravariant, either with +`covariant=True`, `contravariant=True` or with the new `+T` or `-T` +syntax. + +Simplified Syntax +----------------- + +Instead of specifying the variance on the typevar itself:: + + T_co = typing.TypeVar('T_co', covariant=True) + T_contra = typing.TypeVar('T_contra', contravariant=True) + + class SupportsInvertOld(typing.Protocol[T_co]): + def __invert__(self) -> T_co: ... + + class SupportsPartialOrderOld(typing.Protocol[T_contra]): + def __le__(self, other: T_contra): ... + + +The new syntax uses the `+` and `-` prefix operators to specify +covariant and contravariant generic types inline:: + + T = typing.TypeVar('T') + + class SupportsInvert(typing.Protocol[+T]): + def __invert__(self) -> T: ... + + class SupportsPartialOrder(typing.Protocol[-T]): + def __le__(self, other: T): ... + + +Differences with current syntax +------------------------------- + +The new typevar operators return a transparent wrapper around the +original type variable, which can be accessed with the `__orig__` +attribute on the returned wrapper. e.g.:: + + (+T).__orig__ is T + (+T).__covariant__ == not T.__covariant + (+T).__contravariant__ is False + (+T).__name__ == T.__name__ + (+T).__constraints__ == T.__constraints__ + (+T).__bound__ is T.__bound__ + +Thus, type variables defined with `covariant=True` and +`contravariant=True`, are not equivalent to `+T` and `-T`. -How to Teach This -================= +`+T` and `-T` are not valid type annotations, and should only be used +within the generic type parameter list of generic classes. e.g.:: + class Spam(typing.Generic[+KT]): ... + class Eggs(typing.Protocol[KT, +VT]): ... + class OrderedSet(typing.AbstractSet[+T]): ... + +are all valid uses. + + +All variance rules that apply to user-defined classes should apply +in the same way with the new syntax, as they do with the current syntax, +and vice-versa. -Suggested Sections -================== Rejected Ideas ============== +For more detauls about discussions, see links below: + +- `Discussion in python/typing `_ + +1. Using `T_co = +TypeVar('T_co')` instead of `T_co = TypeVar('T_co', covariant=True)` +-------------------------------------------------------------------------------------- + +PROS: + +- This requires minimal changes to the syntax +- Replaces the need to type `covariant=True` or `contravariant=True` + with a concise operator. + + +CONS: + + - The `+` and `-` copy the type variable, but type variables should + be unique. + - It is not obvious what to do with the name of the type variable. + - Co- and contravariance are properties of the generic class, not of + the individual type variables. + References ========== -.. [1] mypy - http://mypy-lang.org/ -.. [2] Scala Variance +.. [1] Scala Variance https://docs.scala-lang.org/scala3/book/types-variance.html @@ -136,15 +211,3 @@ This document is placed in the public domain or under the CC0-1.0-Universal lice fill-column: 70 coding: utf-8 End: - - - -Copyright -========= - -This document is placed in the public domain or under the -CC0-1.0-Universal license, whichever is more permissive. - - -Various sections are found to be common across PEPs and are outlined in -PEP 1 [1]_. Those sections are provided here for convenience. From 15aedb5beca94db13b10dee7889a5a854224bb24 Mon Sep 17 00:00:00 2001 From: jorenham Date: Mon, 2 Aug 2021 16:05:30 +0200 Subject: [PATCH 05/11] Indentation fix --- pep-9999.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/pep-9999.rst b/pep-9999.rst index d9470600e8a..a25e9b3338f 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -183,11 +183,11 @@ PROS: CONS: - - The `+` and `-` copy the type variable, but type variables should - be unique. - - It is not obvious what to do with the name of the type variable. - - Co- and contravariance are properties of the generic class, not of - the individual type variables. +- The `+` and `-` copy the type variable, but type variables should + be unique. +- It is not obvious what to do with the name of the type variable. +- Co- and contravariance are properties of the generic class, not of + the individual type variables. References From a717746c7e0eba04452a19e2632d744af92a0eed Mon Sep 17 00:00:00 2001 From: jorenham Date: Mon, 2 Aug 2021 16:31:45 +0200 Subject: [PATCH 06/11] double backtick fixes --- pep-9999.rst | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/pep-9999.rst b/pep-9999.rst index a25e9b3338f..6db77c20419 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -85,7 +85,7 @@ All of these issues can be solved by overloading the ``+`` and ``-`` prefix operators of ``typing.TypeVar`` instances, so that ``+T`` is an alias for a covariant type variable, and ``-T`` for a contravariant one. These variance operators should only be used within the parameter -list of `typing.Generic` and `typing.Protocol` when subclassing them. +list of ``typing.Generic`` and ``typing.Protocol`` when subclassing them. This syntax is practically identical to type parameter variance in the Scala programming language [1]_. @@ -96,11 +96,11 @@ Specification ============= The new type variable variance syntax should be used within the generic -parameter list of `typing.Generic` or `typing.Protocol` base classes, +parameter list of ``typing.Generic`` or ``typing.Protocol`` base classes, and only there. It is not allowed to be used on type variables that are already marked as covariant or contravariant, either with -`covariant=True`, `contravariant=True` or with the new `+T` or `-T` -syntax. +``covariant=True``, ``contravariant=True`` or with the new ``+T`` or +``-T`` syntax. Simplified Syntax ----------------- @@ -117,7 +117,7 @@ Instead of specifying the variance on the typevar itself:: def __le__(self, other: T_contra): ... -The new syntax uses the `+` and `-` prefix operators to specify +The new syntax uses the ``+`` and ``-`` prefix operators to specify covariant and contravariant generic types inline:: T = typing.TypeVar('T') @@ -133,7 +133,7 @@ Differences with current syntax ------------------------------- The new typevar operators return a transparent wrapper around the -original type variable, which can be accessed with the `__orig__` +original type variable, which can be accessed with the ``__orig__`` attribute on the returned wrapper. e.g.:: (+T).__orig__ is T @@ -144,12 +144,12 @@ attribute on the returned wrapper. e.g.:: (+T).__bound__ is T.__bound__ -Thus, type variables defined with `covariant=True` and -`contravariant=True`, are not equivalent to `+T` and `-T`. +Thus, type variables defined with ``covariant=True`` and +``contravariant=True``, are not equivalent to ``+T`` and ``-T``. -`+T` and `-T` are not valid type annotations, and should only be used -within the generic type parameter list of generic classes. e.g.:: +``+T`` and ``-T`` are not valid type annotations, and should only be +used within the generic type parameter list of generic classes. e.g.:: class Spam(typing.Generic[+KT]): ... class Eggs(typing.Protocol[KT, +VT]): ... @@ -171,20 +171,20 @@ For more detauls about discussions, see links below: - `Discussion in python/typing `_ -1. Using `T_co = +TypeVar('T_co')` instead of `T_co = TypeVar('T_co', covariant=True)` +1. Using ``T_co = +TypeVar('T_co')`` instead of ``T_co = TypeVar('T_co', covariant=True)`` -------------------------------------------------------------------------------------- PROS: - This requires minimal changes to the syntax -- Replaces the need to type `covariant=True` or `contravariant=True` +- Replaces the need to type ``covariant=True`` or ``contravariant=True`` with a concise operator. CONS: -- The `+` and `-` copy the type variable, but type variables should - be unique. +- The ``+`` and ``-`` copy the type variable, but type variables + should be unique. - It is not obvious what to do with the name of the type variable. - Co- and contravariance are properties of the generic class, not of the individual type variables. From 4805514a193f7ae2b430b6f2c7f6e0ac68a8f29c Mon Sep 17 00:00:00 2001 From: jorenham Date: Mon, 2 Aug 2021 16:54:41 +0200 Subject: [PATCH 07/11] Applied the suggestions of @JelleZijlstra Syntax fix in the `Rejected Ideas` subheading --- pep-9999.rst | 30 ++++++++++++++---------------- 1 file changed, 14 insertions(+), 16 deletions(-) diff --git a/pep-9999.rst b/pep-9999.rst index 6db77c20419..a5c05d188de 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -51,26 +51,24 @@ Rationale Although defining the variance of generic type by using type variables that are constructed with the ``covariant=True`` or ``contravariant=True`` -keyword arguments, it has downsides. - -This is why type checkers do no allow using co- or contravariant type -variables outside of generic classes. - +keyword arguments works well enough, it has downsides: - In PEP 484 it states: Covariance or contravariance is not a property of a type variable, but a property of a generic class defined using this variable. - Instead of defining variance on the type variable itself, it would - make more sense to mark the type variable as variant when defining - the generic type, and only there. + This is why type checkers do no allow using co- or contravariant type + variables outside of generic classes. + So, instead of defining variance on the type variable itself, it + would make more sense to mark the type variable's variance when + defining the generic type, and only there. - Because variance is a property of the generic type itself, type - checkers do no allow using co- or contravariant type variables + checkers do not allow using co- or contravariant type variables outside of generic classes. But as the variance is currently a - property of the type variables, the must be used within variant - generic classes. + property of the type variables, they must be used within generic + classes. - Creating separate type variables for different variances often results in up to 3x code duplication, especially for type variables @@ -137,7 +135,7 @@ original type variable, which can be accessed with the ``__orig__`` attribute on the returned wrapper. e.g.:: (+T).__orig__ is T - (+T).__covariant__ == not T.__covariant + (+T).__covariant__ is True (+T).__contravariant__ is False (+T).__name__ == T.__name__ (+T).__constraints__ == T.__constraints__ @@ -149,13 +147,13 @@ Thus, type variables defined with ``covariant=True`` and ``+T`` and ``-T`` are not valid type annotations, and should only be -used within the generic type parameter list of generic classes. e.g.:: +used within the generic type parameter list of ``typing.Generic`` +or ``typing.Protocol``. e.g.:: class Spam(typing.Generic[+KT]): ... class Eggs(typing.Protocol[KT, +VT]): ... - class OrderedSet(typing.AbstractSet[+T]): ... -are all valid uses. +are valid uses. All variance rules that apply to user-defined classes should apply @@ -172,7 +170,7 @@ For more detauls about discussions, see links below: - `Discussion in python/typing `_ 1. Using ``T_co = +TypeVar('T_co')`` instead of ``T_co = TypeVar('T_co', covariant=True)`` --------------------------------------------------------------------------------------- +------------------------------------------------------------------------------------------ PROS: From ad351c5690378d7ef0bd4c1cb8ae0db0ac194540 Mon Sep 17 00:00:00 2001 From: jorenham Date: Mon, 2 Aug 2021 17:36:17 +0200 Subject: [PATCH 08/11] `__origin__` typo fix --- pep-9999.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pep-9999.rst b/pep-9999.rst index a5c05d188de..0f0dcab99a0 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -131,7 +131,7 @@ Differences with current syntax ------------------------------- The new typevar operators return a transparent wrapper around the -original type variable, which can be accessed with the ``__orig__`` +original type variable, which can be accessed with the ``__origin__`` attribute on the returned wrapper. e.g.:: (+T).__orig__ is T From 20d2dbfb60bc10fd18c38349622548bc5ed5c2ef Mon Sep 17 00:00:00 2001 From: jorenham Date: Mon, 2 Aug 2021 17:48:06 +0200 Subject: [PATCH 09/11] Included an example of a subclass an existing generic class --- pep-9999.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/pep-9999.rst b/pep-9999.rst index 0f0dcab99a0..f94a8894d1d 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -151,7 +151,8 @@ used within the generic type parameter list of ``typing.Generic`` or ``typing.Protocol``. e.g.:: class Spam(typing.Generic[+KT]): ... - class Eggs(typing.Protocol[KT, +VT]): ... + class Eggs(typing.Protocol[-KT, +VT]): ... + class HamSet(typing.Sequence[T], typing.Generic[+T]): ... are valid uses. From e0cb451655349ddd497ddf5c6083c50d0b98be46 Mon Sep 17 00:00:00 2001 From: jorenham Date: Mon, 2 Aug 2021 17:49:03 +0200 Subject: [PATCH 10/11] `__origin__` typo fix (again) --- pep-9999.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pep-9999.rst b/pep-9999.rst index f94a8894d1d..cb9f9cd11fc 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -134,7 +134,7 @@ The new typevar operators return a transparent wrapper around the original type variable, which can be accessed with the ``__origin__`` attribute on the returned wrapper. e.g.:: - (+T).__orig__ is T + (+T).__origin__ is T (+T).__covariant__ is True (+T).__contravariant__ is False (+T).__name__ == T.__name__ From afef4d7c2998567fc51b4235b2918f7d09094341 Mon Sep 17 00:00:00 2001 From: jorenham Date: Mon, 2 Aug 2021 21:30:07 +0200 Subject: [PATCH 11/11] Corrected the status to "Draft" --- pep-9999.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pep-9999.rst b/pep-9999.rst index cb9f9cd11fc..22fe2837e97 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -1,7 +1,7 @@ PEP: 9999 Title: Inline variance operators for generic type variables: ``+T`` and ``-T`` Author: Joren Hammudoglu -Status: Active +Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 24-Jul-2021 @@ -57,7 +57,7 @@ keyword arguments works well enough, it has downsides: Covariance or contravariance is not a property of a type variable, but a property of a generic class defined using this variable. - + This is why type checkers do no allow using co- or contravariant type variables outside of generic classes. So, instead of defining variance on the type variable itself, it