From 3d93c34ea944215b2adcf571212591696d00b7b7 Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Fri, 31 Dec 2021 00:00:22 +0100 Subject: [PATCH 01/11] Add draft of ISOLATE_STDLIB_MODULES.md --- ISOLATE_STDLIB_MODULES.md | 71 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 71 insertions(+) create mode 100644 ISOLATE_STDLIB_MODULES.md diff --git a/ISOLATE_STDLIB_MODULES.md b/ISOLATE_STDLIB_MODULES.md new file mode 100644 index 0000000..c2b84bb --- /dev/null +++ b/ISOLATE_STDLIB_MODULES.md @@ -0,0 +1,71 @@ +# Isolating modules in the standard library + +This document serves as a checklist for how to isolate a module in the standard +library. + +As of Python 3.11, a lot of modules have been adapted to multi-phase +initialisation, and a lot of standard library types have been converted from +static to heap types. This process has not been straight-forward, and most of +the issues relate to how heap types differ from static types. Some performance +issues have also been encountered when converting global state to module state. + +Before undertaking this, you should familiarise yourself with the following +PEPs: + +- [PEP 384](https://www.python.org/dev/peps/pep-0384/) +- [PEP 489](https://www.python.org/dev/peps/pep-0489/) +- [PEP 573](https://www.python.org/dev/peps/pep-0573/) +- [PEP 630](https://www.python.org/dev/peps/pep-0630/) + + +## Part 1: Preparation + +1. Open a discussion, either on the bug tracker or on Discourse. Involve the + module maintainer and/or code owner. Explain the reason and rationale for + the changes. +2. Identify global state performance bottlenecks, if there are such. Create a + proof-of-concept implementation and measure the performance impact. `pyperf` + is a good tool for benchmarking. +3. Create an implementation plan. For small modules with few types, a single PR + may do the job. For larger modules with lots of types, and possibly also + external library callbacks, multiple PR's will be needed. + + +## Part 2: Implementation + +Note: this is a suggested implementation plan, based on lessons learned with +other modules. + +1. Add Argument Clinic where possible; it enables you to easily use the + defining class to fetch module state from type methods. +2. Prepare for module state; establish a module state `struct`, add an instance + as a static global variable, and create helper stubs for fetching the module + state. +3. Add relevant global variables to the module state `struct` and modify code + that accesses the global state to use the module state helpers instead. This + step may be broken into several PR's. +4. Convert heap types to static types. +5. Convert the global module state struct to true module state. +6. Implement multi-phase initialisation. + +Preferably, steps 4 through 6 should all land early in a single alpha +development phase. + + +## Got'chas + +- All heap types **must fully implement the GC protocol**. See + [bpo-42972](https://bugs.python.org/issue42972). +- All standard library types should remain immutable. Heap types are mutable by + default, static types are not. Use `Py_TPFLAGS_IMMUTABLE_TYPE` to retain + immutability. See [bpo-43908](https://bugs.python.org/issue43908). +- Static type with tp_new = NULL does not have public constructor, but heap + type inherits constructor from base class. Make sure types that previously + were impossible to instantiate, retain that feature; use + `Py_TPFLAGS_DISALLOW_INSTANTIATION`. Add tests using + `test.support.check_disallow_instantiation()`. See + [bpo-43916](https://bugs.python.org/issue43916). +- Use strong "back-refs" to the module object to ensure the module state + pointer never outlives objects that access module state. Keep this in mind + for external library callbacks that access module state. + From d9b2c6d4ea644bcc6fb587ef79cf41ccf1ecb83d Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Wed, 16 Feb 2022 18:28:17 +0100 Subject: [PATCH 02/11] Add ISOLATE_STDLIB_MODULES.rst --- ISOLATE_STDLIB_MODULES.rst | 240 +++++++++++++++++++++++++++++++++++++ 1 file changed, 240 insertions(+) create mode 100644 ISOLATE_STDLIB_MODULES.rst diff --git a/ISOLATE_STDLIB_MODULES.rst b/ISOLATE_STDLIB_MODULES.rst new file mode 100644 index 0000000..6b5d113 --- /dev/null +++ b/ISOLATE_STDLIB_MODULES.rst @@ -0,0 +1,240 @@ +PEP: XXX +Title: Isolating modules in the standard library +Author: Petr Viktorin , Erlend Egeberg Aasland +Discussions-To: XXX +Status: Draft +Type: Standards +Requires: 489, 573, 630 +Created: 16-Feb-2022 +Python-Version: 3.11 +Post-History: XXX + + +Abstract +======== + +Extensions in the standard library will be converted to multi-phase +initialization (:pep:`489`) and where possible, all state will be +stored on module objects rather than in process-global variables. + + +Note on Backdating +================== + +Much of this proposal has already been implemented, partly to solve related +issues and partly because of misunderstandings and lack of coordination. + +We submit this PEP to explain the changes, seek consensus on +whether they are good, propose changing modules that do not exhibit +issues, and set best practices for new modules. + +We are aware that the pre-existing changes might need to be reverted if issues +are found. + +The PEP takes the form of the proposal that should have been submitted, +talking about the situation before the proposed changes as current. + + +Motivation & Rationale +====================== + +The informational :pep:`630` describes the background, motivation, rationale, +implications and implementation notes of the proposed changes as they apply +generally to any extension module (not just the standard library). + +It is an integral part of this proposal. Read it first. + +This PEP discusses specifics of the standard library. + + +Specification +============= + +The body of :pep:`630` will be converted to a HOWTO in the Python +documentation, and PEP will be retired (marked Final). + +All extension modules in the standard library will be converted to multi-phase +initialization introduced in :pep:`484`. + +All stdlib extension modules will be *isolated*. That is: + +- Types, functions and other objects defined by the module will either be + immutable, or not shared with other module instances. + +- State specific to the module will not be shared with other module instances, + unless it represents global state. + For example, ``_csv.field_size_limit`` will get/set a module-specific + number. On the other hand, functions like ``readline.get_history_item`` or + ``os.getpid`` will continue to work with state that is process-global + (external to the module, and possibly shared across other libraries, including + non-Python ones). + +Conversion to heap types +------------------------ + +Types whose methods need access to “their” module instance will be converted +to heap types following :pep:`630`, with the following considerations: + +- All heap types **must fully implement the GC protocol**. See + [bpo-42972](https://bugs.python.org/issue42972). + (XXX move this to PEP 630) + +- All standard library types that used to be static types should remain + immutable. Heap types must be defined with the `Py_TPFLAGS_IMMUTABLE_TYPE` + flag to retain immutability. + See `bpo-43908 `__. + +- A static type with ``tp_new = NULL`` does not have public constructor, but + heap types inherit the constructor from the base class. Make sure types that + previously were impossible to instantiate retain that feature; use + `Py_TPFLAGS_DISALLOW_INSTANTIATION`. Add tests using + `test.support.check_disallow_instantiation()`. See + `bpo-43916 `__. + +- Converted heap types may unintentionally become serializable + (``pickle``-able). Test that calling ``pickle.dumps`` has the same result + before and after conversion, and if the test fails, add a ``__reduce__`` + method that raises ``TypeError``. See `PR-21002 `__ + for an example. + +- Use strong "back-refs" to the module object to ensure the module state + pointer never outlives objects that access module state. Keep this in mind + for external library callbacks that access module state. + (XXX move this to PEP 630) + +These issues will be added to the Devguide to help any future conversions. + +If another kind of issue is found, the module in questions should be unchanged +until a solution is found and added to the Devguide, and already +converted modules are checked and fixed. + +Static types that do not need module state access and have no other reason to +be converted should stay static. + + +Testing +------- + +XXX How should this be tested? + + +Process +------- + +The following process should be added to the Devguide, and stay there until +all modules are converted. +Any new findings should be documented there or in the general HOWTO. + +Part 1: Preparation +................... + +1. Open a discussion, either on the bug tracker or on Discourse. Involve the + module maintainer and/or code owner. Explain the reason and rationale for + the changes. +2. Identify global state performance bottlenecks, if there are such. Create a + proof-of-concept implementation and measure the performance impact. `pyperf` + is a good tool for benchmarking. +3. Create an implementation plan. For small modules with few types, a single PR + may do the job. For larger modules with lots of types, and possibly also + external library callbacks, multiple PR's will be needed. + + +Part 2: Implementation +...................... + +Note: this is a suggested implementation plan for a complex module, based on +lessons learned with other modules. Feel free to simplify it for +smaller modules. + +1. Add Argument Clinic where possible; it enables you to easily use the + defining class to fetch module state from type methods. +2. Prepare for module state; establish a module state `struct`, add an instance + as a static global variable, and create helper stubs for fetching the module + state. +3. Add relevant global variables to the module state `struct` and modify code + that accesses the global state to use the module state helpers instead. This + step may be broken into several PR's. +4. Where necessary, convert heap types to static types. +5. Convert the global module state struct to true module state. +6. Implement multi-phase initialisation. + +Preferably, steps 4 through 6 should all land early in a single alpha +development phase. + + +Backwards Compatibility +======================= + +Extension modules in the standard library will now be loadable more than once. +For example, deleting such a module from ``sys.modules`` and re-importing it +will result in a fresh module instance, isolated from any previously loaded +instances. + +This may affect code that expected the previous behavior: globals of +extension modules were shallowly copied from the first loaded module. + + +Security Implications +===================== + +None known. + + +How to Teach This +================= + +A large part of this proposal is a HOWTO aimed at experienced users, +which will be moved to the documentation. + +Beginners should not be affected. + + +Reference Implementation +======================== + +Main branch, particularly commits for these issues: + +- https://bugs.python.org/issue40077 +- https://bugs.python.org/issue46417 +- https://bugs.python.org/issue1635741 + + +Rejected Ideas +============== + +Keep things as they were and not break anything +----------------------------------------------- + +XXX Someone write something here please + + +Open Issues +=========== + +See XXX above. + +Types that do not need module state access +------------------------------------------ + +If Python gets a per-interpreter GIL, these types will need to either + +- be converted to heap types, or +- become immortal (see :pep:`683`). + + +Copyright +========= + +This document is placed in the public domain or under the +CC0-1.0-Universal license, whichever is more permissive. + + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: From 67ae085ccbe519e02ea09c3981d61143a2dd9c85 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Wed, 16 Mar 2022 16:21:40 +0100 Subject: [PATCH 03/11] Apply suggestions from code review Co-authored-by: Erlend Egeberg Aasland --- ISOLATE_STDLIB_MODULES.rst | 23 +++++++++++------------ 1 file changed, 11 insertions(+), 12 deletions(-) diff --git a/ISOLATE_STDLIB_MODULES.rst b/ISOLATE_STDLIB_MODULES.rst index 6b5d113..cfd3c93 100644 --- a/ISOLATE_STDLIB_MODULES.rst +++ b/ISOLATE_STDLIB_MODULES.rst @@ -72,7 +72,7 @@ All stdlib extension modules will be *isolated*. That is: Conversion to heap types ------------------------ -Types whose methods need access to “their” module instance will be converted +Types whose methods need access to their module instance will be converted to heap types following :pep:`630`, with the following considerations: - All heap types **must fully implement the GC protocol**. See @@ -84,7 +84,7 @@ to heap types following :pep:`630`, with the following considerations: flag to retain immutability. See `bpo-43908 `__. -- A static type with ``tp_new = NULL`` does not have public constructor, but +- A static type with ``tp_new = NULL`` does not have a public constructor, but heap types inherit the constructor from the base class. Make sure types that previously were impossible to instantiate retain that feature; use `Py_TPFLAGS_DISALLOW_INSTANTIATION`. Add tests using @@ -104,12 +104,12 @@ to heap types following :pep:`630`, with the following considerations: These issues will be added to the Devguide to help any future conversions. -If another kind of issue is found, the module in questions should be unchanged +If another kind of issue is found, the module in question should be unchanged until a solution is found and added to the Devguide, and already converted modules are checked and fixed. -Static types that do not need module state access and have no other reason to -be converted should stay static. +Static types that do not need module state access, and have no other reason to +be converted, should stay static. Testing @@ -121,7 +121,7 @@ XXX How should this be tested? Process ------- -The following process should be added to the Devguide, and stay there until +The following process should be added to the Devguide, and remain until all modules are converted. Any new findings should be documented there or in the general HOWTO. @@ -131,9 +131,9 @@ Part 1: Preparation 1. Open a discussion, either on the bug tracker or on Discourse. Involve the module maintainer and/or code owner. Explain the reason and rationale for the changes. -2. Identify global state performance bottlenecks, if there are such. Create a - proof-of-concept implementation and measure the performance impact. `pyperf` - is a good tool for benchmarking. +2. Identify global state performance bottlenecks. + Create a proof-of-concept implementation, and measure the performance impact. + `pyperf` is a good tool for benchmarking. 3. Create an implementation plan. For small modules with few types, a single PR may do the job. For larger modules with lots of types, and possibly also external library callbacks, multiple PR's will be needed. @@ -151,15 +151,14 @@ smaller modules. 2. Prepare for module state; establish a module state `struct`, add an instance as a static global variable, and create helper stubs for fetching the module state. -3. Add relevant global variables to the module state `struct` and modify code +3. Add relevant global variables to the module state `struct`, and modify code that accesses the global state to use the module state helpers instead. This step may be broken into several PR's. 4. Where necessary, convert heap types to static types. 5. Convert the global module state struct to true module state. 6. Implement multi-phase initialisation. -Preferably, steps 4 through 6 should all land early in a single alpha -development phase. +Steps 4 through 6 should preferably land in a single alpha development phase. Backwards Compatibility From e872073d427b444a2f4b9e7059ad6a5d12379fbf Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Wed, 16 Mar 2022 17:03:02 +0100 Subject: [PATCH 04/11] Address issues from review --- ISOLATE_STDLIB_MODULES.rst | 40 ++++++++++++++++++++------------------ 1 file changed, 21 insertions(+), 19 deletions(-) diff --git a/ISOLATE_STDLIB_MODULES.rst b/ISOLATE_STDLIB_MODULES.rst index cfd3c93..6754c08 100644 --- a/ISOLATE_STDLIB_MODULES.rst +++ b/ISOLATE_STDLIB_MODULES.rst @@ -1,6 +1,6 @@ PEP: XXX Title: Isolating modules in the standard library -Author: Petr Viktorin , Erlend Egeberg Aasland +Author: Erlend Egeberg Aasland , Petr Viktorin Discussions-To: XXX Status: Draft Type: Standards @@ -25,11 +25,8 @@ Much of this proposal has already been implemented, partly to solve related issues and partly because of misunderstandings and lack of coordination. We submit this PEP to explain the changes, seek consensus on -whether they are good, propose changing modules that do not exhibit -issues, and set best practices for new modules. - -We are aware that the pre-existing changes might need to be reverted if issues -are found. +whether they are good, propose the remaining changes, +and set best practices for new modules. The PEP takes the form of the proposal that should have been submitted, talking about the situation before the proposed changes as current. @@ -51,7 +48,7 @@ Specification ============= The body of :pep:`630` will be converted to a HOWTO in the Python -documentation, and PEP will be retired (marked Final). +documentation, and that PEP will be retired (marked Final). All extension modules in the standard library will be converted to multi-phase initialization introduced in :pep:`484`. @@ -63,6 +60,7 @@ All stdlib extension modules will be *isolated*. That is: - State specific to the module will not be shared with other module instances, unless it represents global state. + For example, ``_csv.field_size_limit`` will get/set a module-specific number. On the other hand, functions like ``readline.get_history_item`` or ``os.getpid`` will continue to work with state that is process-global @@ -84,6 +82,9 @@ to heap types following :pep:`630`, with the following considerations: flag to retain immutability. See `bpo-43908 `__. + Tests should ensure ``TypeError`` is raised when trying to create a new + attribute of an immutable type. + - A static type with ``tp_new = NULL`` does not have a public constructor, but heap types inherit the constructor from the base class. Make sure types that previously were impossible to instantiate retain that feature; use @@ -112,12 +113,6 @@ Static types that do not need module state access, and have no other reason to be converted, should stay static. -Testing -------- - -XXX How should this be tested? - - Process ------- @@ -191,18 +186,25 @@ Beginners should not be affected. Reference Implementation ======================== -Main branch, particularly commits for these issues: +Most of the changes are now in the main branch, as commits for these issues: + +- `bpo-40077, Convert static types to heap types: use PyType_FromSpec() `_ +- `bpo-40077, Clear static types in Py_Finalize() for embedded Python `_ +- `bpo-1635741, Py_Finalize() doesn't clear all Python objects at exit `_ + +As an example, changes and fix-ups done in the `_csv` module are: -- https://bugs.python.org/issue40077 -- https://bugs.python.org/issue46417 -- https://bugs.python.org/issue1635741 +- `GH-23224, Remove static state from the _csv module `_ +- `GH-26008, Allow subclassing of ``csv.Error`` `_ +- `GH-26074, Add GC support to _csv heap types `_ +- `GH-26351, Make heap types converted during 3.10 alpha immutable `_ Rejected Ideas ============== -Keep things as they were and not break anything ------------------------------------------------ +Do not isolate modules in the standard library +---------------------------------------------- XXX Someone write something here please From 6372904c2d5a9841e0bcfd099c1dc9c469eb4860 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Mon, 21 Mar 2022 17:55:44 +0100 Subject: [PATCH 05/11] Apply suggestions from code review Co-authored-by: Erlend Egeberg Aasland --- ISOLATE_STDLIB_MODULES.rst | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/ISOLATE_STDLIB_MODULES.rst b/ISOLATE_STDLIB_MODULES.rst index 6754c08..3275930 100644 --- a/ISOLATE_STDLIB_MODULES.rst +++ b/ISOLATE_STDLIB_MODULES.rst @@ -28,8 +28,6 @@ We submit this PEP to explain the changes, seek consensus on whether they are good, propose the remaining changes, and set best practices for new modules. -The PEP takes the form of the proposal that should have been submitted, -talking about the situation before the proposed changes as current. Motivation & Rationale @@ -189,7 +187,7 @@ Reference Implementation Most of the changes are now in the main branch, as commits for these issues: - `bpo-40077, Convert static types to heap types: use PyType_FromSpec() `_ -- `bpo-40077, Clear static types in Py_Finalize() for embedded Python `_ +- `bpo-46417, Clear static types in Py_Finalize() for embedded Python `_ - `bpo-1635741, Py_Finalize() doesn't clear all Python objects at exit `_ As an example, changes and fix-ups done in the `_csv` module are: From cc1619079998c6bbf40642062d2e4b3755f178f7 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Mon, 21 Mar 2022 17:53:49 +0100 Subject: [PATCH 06/11] Remove section on Types that do not need module state access --- ISOLATE_STDLIB_MODULES.rst | 8 -------- 1 file changed, 8 deletions(-) diff --git a/ISOLATE_STDLIB_MODULES.rst b/ISOLATE_STDLIB_MODULES.rst index 3275930..556528d 100644 --- a/ISOLATE_STDLIB_MODULES.rst +++ b/ISOLATE_STDLIB_MODULES.rst @@ -212,14 +212,6 @@ Open Issues See XXX above. -Types that do not need module state access ------------------------------------------- - -If Python gets a per-interpreter GIL, these types will need to either - -- be converted to heap types, or -- become immortal (see :pep:`683`). - Copyright ========= From 5bb3b8ae3e29f67f884333d3e0b56d87d3fcc3c7 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Mon, 21 Mar 2022 17:55:16 +0100 Subject: [PATCH 07/11] Remove now-empty Open Issues section --- ISOLATE_STDLIB_MODULES.rst | 6 ------ 1 file changed, 6 deletions(-) diff --git a/ISOLATE_STDLIB_MODULES.rst b/ISOLATE_STDLIB_MODULES.rst index 556528d..c22d6c6 100644 --- a/ISOLATE_STDLIB_MODULES.rst +++ b/ISOLATE_STDLIB_MODULES.rst @@ -207,12 +207,6 @@ Do not isolate modules in the standard library XXX Someone write something here please -Open Issues -=========== - -See XXX above. - - Copyright ========= From 812b6465d7c3bd3e45bc78bf9417baf56417501c Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Mon, 21 Mar 2022 17:57:27 +0100 Subject: [PATCH 08/11] Remove the Markdown file --- ISOLATE_STDLIB_MODULES.md | 71 --------------------------------------- 1 file changed, 71 deletions(-) delete mode 100644 ISOLATE_STDLIB_MODULES.md diff --git a/ISOLATE_STDLIB_MODULES.md b/ISOLATE_STDLIB_MODULES.md deleted file mode 100644 index c2b84bb..0000000 --- a/ISOLATE_STDLIB_MODULES.md +++ /dev/null @@ -1,71 +0,0 @@ -# Isolating modules in the standard library - -This document serves as a checklist for how to isolate a module in the standard -library. - -As of Python 3.11, a lot of modules have been adapted to multi-phase -initialisation, and a lot of standard library types have been converted from -static to heap types. This process has not been straight-forward, and most of -the issues relate to how heap types differ from static types. Some performance -issues have also been encountered when converting global state to module state. - -Before undertaking this, you should familiarise yourself with the following -PEPs: - -- [PEP 384](https://www.python.org/dev/peps/pep-0384/) -- [PEP 489](https://www.python.org/dev/peps/pep-0489/) -- [PEP 573](https://www.python.org/dev/peps/pep-0573/) -- [PEP 630](https://www.python.org/dev/peps/pep-0630/) - - -## Part 1: Preparation - -1. Open a discussion, either on the bug tracker or on Discourse. Involve the - module maintainer and/or code owner. Explain the reason and rationale for - the changes. -2. Identify global state performance bottlenecks, if there are such. Create a - proof-of-concept implementation and measure the performance impact. `pyperf` - is a good tool for benchmarking. -3. Create an implementation plan. For small modules with few types, a single PR - may do the job. For larger modules with lots of types, and possibly also - external library callbacks, multiple PR's will be needed. - - -## Part 2: Implementation - -Note: this is a suggested implementation plan, based on lessons learned with -other modules. - -1. Add Argument Clinic where possible; it enables you to easily use the - defining class to fetch module state from type methods. -2. Prepare for module state; establish a module state `struct`, add an instance - as a static global variable, and create helper stubs for fetching the module - state. -3. Add relevant global variables to the module state `struct` and modify code - that accesses the global state to use the module state helpers instead. This - step may be broken into several PR's. -4. Convert heap types to static types. -5. Convert the global module state struct to true module state. -6. Implement multi-phase initialisation. - -Preferably, steps 4 through 6 should all land early in a single alpha -development phase. - - -## Got'chas - -- All heap types **must fully implement the GC protocol**. See - [bpo-42972](https://bugs.python.org/issue42972). -- All standard library types should remain immutable. Heap types are mutable by - default, static types are not. Use `Py_TPFLAGS_IMMUTABLE_TYPE` to retain - immutability. See [bpo-43908](https://bugs.python.org/issue43908). -- Static type with tp_new = NULL does not have public constructor, but heap - type inherits constructor from base class. Make sure types that previously - were impossible to instantiate, retain that feature; use - `Py_TPFLAGS_DISALLOW_INSTANTIATION`. Add tests using - `test.support.check_disallow_instantiation()`. See - [bpo-43916](https://bugs.python.org/issue43916). -- Use strong "back-refs" to the module object to ensure the module state - pointer never outlives objects that access module state. Keep this in mind - for external library callbacks that access module state. - From 10c6518a9547784f4d93a8b273572f88b5879c0e Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Mon, 21 Mar 2022 18:02:02 +0100 Subject: [PATCH 09/11] Remove genetal issues that were added to PEP 630 --- ISOLATE_STDLIB_MODULES.rst | 9 --------- 1 file changed, 9 deletions(-) diff --git a/ISOLATE_STDLIB_MODULES.rst b/ISOLATE_STDLIB_MODULES.rst index c22d6c6..8a20ee6 100644 --- a/ISOLATE_STDLIB_MODULES.rst +++ b/ISOLATE_STDLIB_MODULES.rst @@ -71,10 +71,6 @@ Conversion to heap types Types whose methods need access to their module instance will be converted to heap types following :pep:`630`, with the following considerations: -- All heap types **must fully implement the GC protocol**. See - [bpo-42972](https://bugs.python.org/issue42972). - (XXX move this to PEP 630) - - All standard library types that used to be static types should remain immutable. Heap types must be defined with the `Py_TPFLAGS_IMMUTABLE_TYPE` flag to retain immutability. @@ -96,11 +92,6 @@ to heap types following :pep:`630`, with the following considerations: method that raises ``TypeError``. See `PR-21002 `__ for an example. -- Use strong "back-refs" to the module object to ensure the module state - pointer never outlives objects that access module state. Keep this in mind - for external library callbacks that access module state. - (XXX move this to PEP 630) - These issues will be added to the Devguide to help any future conversions. If another kind of issue is found, the module in question should be unchanged From 936768e844e5cde5dc1e45a7ae9c1cefb13e5b45 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Wed, 23 Mar 2022 15:02:26 +0100 Subject: [PATCH 10/11] Remove Rejected ideas section --- ISOLATE_STDLIB_MODULES.rst | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/ISOLATE_STDLIB_MODULES.rst b/ISOLATE_STDLIB_MODULES.rst index 8a20ee6..80ac153 100644 --- a/ISOLATE_STDLIB_MODULES.rst +++ b/ISOLATE_STDLIB_MODULES.rst @@ -189,15 +189,6 @@ As an example, changes and fix-ups done in the `_csv` module are: - `GH-26351, Make heap types converted during 3.10 alpha immutable `_ -Rejected Ideas -============== - -Do not isolate modules in the standard library ----------------------------------------------- - -XXX Someone write something here please - - Copyright ========= @@ -205,7 +196,6 @@ This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive. - .. Local Variables: mode: indented-text From 1d74c85d456c850438ac3422aa7c6b8dee49b5b0 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Mon, 4 Apr 2022 15:26:47 +0200 Subject: [PATCH 11/11] Update ISOLATE_STDLIB_MODULES.rst Co-authored-by: Erlend Egeberg Aasland --- ISOLATE_STDLIB_MODULES.rst | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/ISOLATE_STDLIB_MODULES.rst b/ISOLATE_STDLIB_MODULES.rst index 80ac153..85e05f1 100644 --- a/ISOLATE_STDLIB_MODULES.rst +++ b/ISOLATE_STDLIB_MODULES.rst @@ -21,9 +21,7 @@ stored on module objects rather than in process-global variables. Note on Backdating ================== -Much of this proposal has already been implemented, partly to solve related -issues and partly because of misunderstandings and lack of coordination. - +Much of this proposal has already been implemented. We submit this PEP to explain the changes, seek consensus on whether they are good, propose the remaining changes, and set best practices for new modules.