diff --git a/guide/src/class.md b/guide/src/class.md index 8772e857d7e..5a360a35663 100644 --- a/guide/src/class.md +++ b/guide/src/class.md @@ -2,7 +2,7 @@ PyO3 exposes a group of attributes powered by Rust's proc macro system for defining Python classes as Rust structs. -The main attribute is `#[pyclass]`, which is placed upon a Rust `struct` or `enum` to generate a Python type for it. They will usually also have *one* `#[pymethods]`-annotated `impl` block for the struct, which is used to define Python methods and constants for the generated Python type. (If the [`multiple-pymethods`] feature is enabled, each `#[pyclass]` is allowed to have multiple `#[pymethods]` blocks.) `#[pymethods]` may also have implementations for Python magic methods such as `__str__`. +The main attribute is `#[pyclass]`, which is placed upon a Rust `struct` or `enum` to generate a Python type for it. They will usually also have _one_ `#[pymethods]`-annotated `impl` block for the struct, which is used to define Python methods and constants for the generated Python type. (If the [`multiple-pymethods`] feature is enabled, each `#[pyclass]` is allowed to have multiple `#[pymethods]` blocks.) `#[pymethods]` may also have implementations for Python magic methods such as `__str__`. This chapter will discuss the functionality and configuration these attributes offer. Below is a list of links to the relevant section of this chapter for each: @@ -22,6 +22,7 @@ This chapter will discuss the functionality and configuration these attributes o ## Defining a new class To define a custom Python class, add the `#[pyclass]` attribute to a Rust struct or enum. + ```rust # #![allow(dead_code)] use pyo3::prelude::*; @@ -202,13 +203,14 @@ fn my_module(m: &Bound<'_, PyModule>) -> PyResult<()> { It is often useful to turn a `#[pyclass]` type `T` into a Python object and access it from Rust code. The [`Py`] and [`Bound<'py, T>`] smart pointers are the ways to represent a Python object in PyO3's API. More detail can be found about them [in the Python objects](./types.md#pyo3s-smart-pointers) section of the guide. -Most Python objects do not offer exclusive (`&mut`) access (see the [section on Python's memory model](./python-from-rust.md#pythons-memory-model)). However, Rust structs wrapped as Python objects (called `pyclass` types) often *do* need `&mut` access. Due to the GIL, PyO3 *can* guarantee exclusive access to them. +Most Python objects do not offer exclusive (`&mut`) access (see the [section on Python's memory model](./python-from-rust.md#pythons-memory-model)). However, Rust structs wrapped as Python objects (called `pyclass` types) often _do_ need `&mut` access. Due to the GIL, PyO3 _can_ guarantee exclusive access to them. The Rust borrow checker cannot reason about `&mut` references once an object's ownership has been passed to the Python interpreter. This means that borrow checking is done at runtime using with a scheme very similar to `std::cell::RefCell`. This is known as [interior mutability](https://doc.rust-lang.org/book/ch15-05-interior-mutability.html). Users who are familiar with `RefCell` can use `Py` and `Bound<'py, T>` just like `RefCell`. For users who are not very familiar with `RefCell`, here is a reminder of Rust's rules of borrowing: + - At any given time, you can have either (but not both of) one mutable reference or any number of immutable references. - References can never outlast the data they refer to. @@ -309,11 +311,11 @@ Generally, `#[new]` methods have to return `T: Into>` o For constructors that may fail, you should wrap the return type in a PyResult as well. Consult the table below to determine which type your constructor should return: -| | **Cannot fail** | **May fail** | -|-----------------------------|---------------------------|-----------------------------------| -|**No inheritance** | `T` | `PyResult` | -|**Inheritance(T Inherits U)**| `(T, U)` | `PyResult<(T, U)>` | -|**Inheritance(General Case)**| [`PyClassInitializer`] | `PyResult>` | +| | **Cannot fail** | **May fail** | +| ----------------------------- | ------------------------- | --------------------------------- | +| **No inheritance** | `T` | `PyResult` | +| **Inheritance(T Inherits U)** | `(T, U)` | `PyResult<(T, U)>` | +| **Inheritance(General Case)** | [`PyClassInitializer`] | `PyResult>` | ## Inheritance @@ -323,7 +325,6 @@ Currently, only classes defined in Rust and builtins provided by PyO3 can be inh from; inheriting from other classes defined in Python is not yet supported ([#991](https://github.com/PyO3/pyo3/issues/991)). - For convenience, `(T, U)` implements `Into>` where `U` is the base class of `T`. But for a more deeply nested inheritance, you have to return `PyClassInitializer` @@ -370,7 +371,7 @@ impl SubClass { (SubClass { val2: 15 }, BaseClass::new()) } - fn method2(self_: PyRef<'_, Self>) -> PyResult { + fn method2(self_: PyRef<'_, '_, Self>) -> PyResult { let super_ = self_.as_super(); // Get &PyRef super_.method1().map(|x| x * self_.val2) } @@ -388,24 +389,24 @@ impl SubSubClass { PyClassInitializer::from(SubClass::new()).add_subclass(SubSubClass { val3: 20 }) } - fn method3(self_: PyRef<'_, Self>) -> PyResult { + fn method3(self_: PyRef<'_, '_, Self>) -> PyResult { let base = self_.as_super().as_super(); // Get &PyRef<'_, BaseClass> base.method1().map(|x| x * self_.val3) } - fn method4(self_: PyRef<'_, Self>) -> PyResult { + fn method4(self_: PyRef<'_, '_, Self>) -> PyResult { let v = self_.val3; let super_ = self_.into_super(); // Get PyRef<'_, SubClass> SubClass::method2(super_).map(|x| x * v) } - fn get_values(self_: PyRef<'_, Self>) -> (usize, usize, usize) { + fn get_values(self_: PyRef<'_, '_, Self>) -> (usize, usize, usize) { let val1 = self_.as_super().as_super().val1; let val2 = self_.as_super().val2; (val1, val2, self_.val3) } - fn double_values(mut self_: PyRefMut<'_, Self>) { + fn double_values(mut self_: PyRefMut<'_, '_, Self>) { self_.as_super().as_super().val1 *= 2; self_.as_super().val2 *= 2; self_.val3 *= 2; @@ -481,6 +482,7 @@ impl DictWithCounter { ``` If `SubClass` does not provide a base class initialization, the compilation fails. + ```rust,compile_fail # use pyo3::prelude::*; @@ -504,7 +506,7 @@ impl SubClass { ``` The `__new__` constructor of a native base class is called implicitly when -creating a new instance from Python. Be sure to accept arguments in the +creating a new instance from Python. Be sure to accept arguments in the `#[new]` method that you want the base class to get, even if they are not used in that `fn`: @@ -542,6 +544,7 @@ initial items, such as `MyDict(item_sequence)` or `MyDict(a=1, b=2)`. ## Object properties PyO3 supports two ways to add properties to your `#[pyclass]`: + - For simple struct fields with no side effects, a `#[pyo3(get, set)]` attribute can be added directly to the field definition in the `#[pyclass]`. - For properties which require computation you can define `#[getter]` and `#[setter]` functions in the [`#[pymethods]`](#instance-methods) block. @@ -566,6 +569,7 @@ The above would make the `num` field available for reading and writing as a `sel Properties can be readonly or writeonly by using just `#[pyo3(get)]` or `#[pyo3(set)]` respectively. To use these annotations, your field type must implement some conversion traits: + - For `get` the field type must implement both `IntoPy` and `Clone`. - For `set` the field type must implement `FromPyObject`. @@ -733,15 +737,16 @@ impl MyClass { Declares a class method callable from Python. -* The first parameter is the type object of the class on which the method is called. +- The first parameter is the type object of the class on which the method is called. This may be the type object of a derived class. -* The first parameter implicitly has type `&Bound<'_, PyType>`. -* For details on `parameter-list`, see the documentation of `Method arguments` section. -* The return type must be `PyResult` or `T` for some `T` that implements `IntoPy`. +- The first parameter implicitly has type `&Bound<'_, PyType>`. +- For details on `parameter-list`, see the documentation of `Method arguments` section. +- The return type must be `PyResult` or `T` for some `T` that implements `IntoPy`. ### Constructors which accept a class argument To create a constructor which takes a positional class argument, you can combine the `#[classmethod]` and `#[new]` modifiers: + ```rust # #![allow(dead_code)] # use pyo3::prelude::*; @@ -807,7 +812,7 @@ Python::with_gil(|py| { ``` > Note: if the method has a `Result` return type and returns an `Err`, PyO3 will panic during -class creation. +> class creation. If the class attribute is defined with `const` code only, one can also annotate associated constants: @@ -844,7 +849,7 @@ fn increment_field(my_class: &mut MyClass) { // Take a reference wrapper when borrowing should be automatic, // but interaction with the underlying `Bound` is desired. #[pyfunction] -fn print_field(my_class: PyRef<'_, MyClass>) { +fn print_field(my_class: PyRef<'_, '_, MyClass>) { println!("{}", my_class.my_field); } @@ -1193,7 +1198,7 @@ Python::with_gil(|py| { ``` Ordering of enum variants is optionally added using `#[pyo3(ord)]`. -*Note: Implementation of the `PartialOrd` trait is required when passing the `ord` argument. If not implemented, a compile time error is raised.* +_Note: Implementation of the `PartialOrd` trait is required when passing the `ord` argument. If not implemented, a compile time error is raised._ ```rust # use pyo3::prelude::*; @@ -1390,22 +1395,22 @@ impl pyo3::PyClass for MyClass { type Frozen = pyo3::pyclass::boolean_struct::False; } -impl<'a, 'py> pyo3::impl_::extract_argument::PyFunctionArgument<'a, 'py, false> for &'a MyClass +impl<'a, 'holder, 'py> pyo3::impl_::extract_argument::PyFunctionArgument<'a, 'holder, 'py, false> for &'holder MyClass { - type Holder = ::std::option::Option>; + type Holder = ::std::option::Option>; #[inline] - fn extract(obj: &'a pyo3::Bound<'py, PyAny>, holder: &'a mut Self::Holder) -> pyo3::PyResult { + fn extract(obj: &'a pyo3::Bound<'py, PyAny>, holder: &'holder mut Self::Holder) -> pyo3::PyResult { pyo3::impl_::extract_argument::extract_pyclass_ref(obj, holder) } } -impl<'a, 'py> pyo3::impl_::extract_argument::PyFunctionArgument<'a, 'py, false> for &'a mut MyClass +impl<'a, 'holder, 'py> pyo3::impl_::extract_argument::PyFunctionArgument<'a, 'holder, 'py, false> for &'holder mut MyClass { - type Holder = ::std::option::Option>; + type Holder = ::std::option::Option>; #[inline] - fn extract(obj: &'a pyo3::Bound<'py, PyAny>, holder: &'a mut Self::Holder) -> pyo3::PyResult { + fn extract(obj: &'a pyo3::Bound<'py, PyAny>, holder: &'holder mut Self::Holder) -> pyo3::PyResult { pyo3::impl_::extract_argument::extract_pyclass_ref_mut(obj, holder) } } @@ -1452,22 +1457,16 @@ impl pyo3::impl_::pyclass::PyClassImpl for MyClass { # } ``` - [`PyTypeInfo`]: {{#PYO3_DOCS_URL}}/pyo3/type_object/trait.PyTypeInfo.html - [`Py`]: {{#PYO3_DOCS_URL}}/pyo3/struct.Py.html [`Bound<'_, T>`]: {{#PYO3_DOCS_URL}}/pyo3/struct.Bound.html [`PyClass`]: {{#PYO3_DOCS_URL}}/pyo3/pyclass/trait.PyClass.html [`PyRef`]: {{#PYO3_DOCS_URL}}/pyo3/pycell/struct.PyRef.html [`PyRefMut`]: {{#PYO3_DOCS_URL}}/pyo3/pycell/struct.PyRefMut.html [`PyClassInitializer`]: {{#PYO3_DOCS_URL}}/pyo3/pyclass_init/struct.PyClassInitializer.html - [`Arc`]: https://doc.rust-lang.org/std/sync/struct.Arc.html [`RefCell`]: https://doc.rust-lang.org/std/cell/struct.RefCell.html - [classattr]: https://docs.python.org/3/tutorial/classes.html#class-and-instance-variables - [`multiple-pymethods`]: features.md#multiple-pymethods - [lifetime-elision]: https://doc.rust-lang.org/reference/lifetime-elision.html [compiler-error-e0106]: https://doc.rust-lang.org/error_codes/E0106.html diff --git a/guide/src/class/numeric.md b/guide/src/class/numeric.md index 124bb8d27a6..0502cf1b8d8 100644 --- a/guide/src/class/numeric.md +++ b/guide/src/class/numeric.md @@ -3,11 +3,12 @@ At this point we have a `Number` class that we can't actually do any math on! Before proceeding, we should think about how we want to handle overflows. There are three obvious solutions: + - We can have infinite precision just like Python's `int`. However that would be quite boring - we'd - be reinventing the wheel. + be reinventing the wheel. - We can raise exceptions whenever `Number` overflows, but that makes the API painful to use. - We can wrap around the boundary of `i32`. This is the approach we'll take here. To do that we'll just forward to `i32`'s - `wrapping_*` methods. + `wrapping_*` methods. ### Fixing our constructor @@ -42,6 +43,7 @@ fn wrap(obj: &Bound<'_, PyAny>) -> PyResult { Ok(val as i32) } ``` + We also add documentation, via `///` comments, which are visible to Python users. ```rust,no_run @@ -68,7 +70,6 @@ impl Number { } ``` - With that out of the way, let's implement some operators: ```rust,no_run use pyo3::exceptions::{PyZeroDivisionError, PyValueError}; @@ -132,7 +133,7 @@ impl Number { # #[pymethods] impl Number { - fn __pos__(slf: PyRef<'_, Self>) -> PyRef<'_, Self> { + fn __pos__<'a, 'py>(slf: PyRef<'a, 'py, Self>) -> PyRef<'a, 'py, Self> { slf } @@ -178,7 +179,7 @@ impl Number { We do not implement the in-place operations like `__iadd__` because we do not wish to mutate `Number`. Similarly we're not interested in supporting operations with different types, so we do not implement - the reflected operations like `__radd__` either. +the reflected operations like `__radd__` either. Now Python can use our `Number` class: @@ -405,13 +406,15 @@ function that does: unsigned long PyLong_AsUnsignedLongMask(PyObject *obj) ``` -We can call this function from Rust by using [`pyo3::ffi::PyLong_AsUnsignedLongMask`]. This is an *unsafe* +We can call this function from Rust by using [`pyo3::ffi::PyLong_AsUnsignedLongMask`]. This is an _unsafe_ function, which means we have to use an unsafe block to call it and take responsibility for upholding the contracts of this function. Let's review those contracts: + - The GIL must be held. If it's not, calling this function causes a data race. - The pointer must be valid, i.e. it must be properly aligned and point to a valid Python object. Let's create that helper function. The signature has to be `fn(&Bound<'_, PyAny>) -> PyResult`. + - `&Bound<'_, PyAny>` represents a checked borrowed reference, so the pointer derived from it is valid (and not null). - Whenever we have borrowed references to Python objects in scope, it is guaranteed that the GIL is held. This reference is also where we can get a [`Python`] token to use in our call to [`PyErr::take`]. diff --git a/guide/src/class/protocols.md b/guide/src/class/protocols.md index 44aa9199e36..7e8729a3883 100644 --- a/guide/src/class/protocols.md +++ b/guide/src/class/protocols.md @@ -3,6 +3,7 @@ Python's object model defines several protocols for different object behavior, such as the sequence, mapping, and number protocols. Python classes support these protocols by implementing "magic" methods, such as `__str__` or `__repr__`. Because of the double-underscores surrounding their name, these are also known as "dunder" methods. PyO3 makes it possible for every magic method to be implemented in `#[pymethods]` just as they would be done in a regular Python class, with a few notable differences: + - `__new__` and `__init__` are replaced by the [`#[new]` attribute](../class.md#constructor). - `__del__` is not yet supported, but may be in the future. - `__buffer__` and `__release_buffer__` are currently not supported and instead PyO3 supports [`__getbuffer__` and `__releasebuffer__`](#buffer-objects) methods (these predate [PEP 688](https://peps.python.org/pep-0688/#python-level-buffer-protocol)), again this may change in the future. @@ -18,138 +19,141 @@ If a function name in `#[pymethods]` is a magic method which is known to need sp The magic methods handled by PyO3 are very similar to the standard Python ones on [this page](https://docs.python.org/3/reference/datamodel.html#special-method-names) - in particular they are the subset which have slots as [defined here](https://docs.python.org/3/c-api/typeobj.html). When PyO3 handles a magic method, a couple of changes apply compared to other `#[pymethods]`: - - The Rust function signature is restricted to match the magic method. - - The `#[pyo3(signature = (...)]` and `#[pyo3(text_signature = "...")]` attributes are not allowed. - -The following sections list all magic methods for which PyO3 implements the necessary special handling. The -given signatures should be interpreted as follows: - - All methods take a receiver as first argument, shown as ``. It can be - `&self`, `&mut self` or a `Bound` reference like `self_: PyRef<'_, Self>` and - `self_: PyRefMut<'_, Self>`, as described [here](../class.md#inheritance). - - An optional `Python<'py>` argument is always allowed as the first argument. - - Return values can be optionally wrapped in `PyResult`. - - `object` means that any type is allowed that can be extracted from a Python - object (if argument) or converted to a Python object (if return value). - - Other types must match what's given, e.g. `pyo3::basic::CompareOp` for - `__richcmp__`'s second argument. - - For the comparison and arithmetic methods, extraction errors are not - propagated as exceptions, but lead to a return of `NotImplemented`. - - For some magic methods, the return values are not restricted by PyO3, but - checked by the Python interpreter. For example, `__str__` needs to return a - string object. This is indicated by `object (Python type)`. - -### Basic object customization - - `__str__() -> object (str)` - - `__repr__() -> object (str)` +- The Rust function signature is restricted to match the magic method. +- The `#[pyo3(signature = (...)]` and `#[pyo3(text_signature = "...")]` attributes are not allowed. - - `__hash__() -> isize` - - Objects that compare equal must have the same hash value. Any type up to 64 bits may be returned instead of `isize`, PyO3 will convert to an isize automatically (wrapping unsigned types like `u64` and `usize`). -
- Disabling Python's default hash - By default, all `#[pyclass]` types have a default hash implementation from Python. Types which should not be hashable can override this by setting `__hash__` to `None`. This is the same mechanism as for a pure-Python class. This is done like so: - - ```rust,no_run - # use pyo3::prelude::*; - # - #[pyclass] - struct NotHashable {} - - #[pymethods] - impl NotHashable { - #[classattr] - const __hash__: Option = None; - } - ``` -
- - - `__lt__(, object) -> object` - - `__le__(, object) -> object` - - `__eq__(, object) -> object` - - `__ne__(, object) -> object` - - `__gt__(, object) -> object` - - `__ge__(, object) -> object` - - The implementations of Python's "rich comparison" operators `<`, `<=`, `==`, `!=`, `>` and `>=` respectively. - - _Note that implementing any of these methods will cause Python not to generate a default `__hash__` implementation, so consider also implementing `__hash__`._ -
- Return type - The return type will normally be `bool` or `PyResult`, however any Python object can be returned. -
- - - `__richcmp__(, object, pyo3::basic::CompareOp) -> object` - - Implements Python comparison operations (`==`, `!=`, `<`, `<=`, `>`, and `>=`) in a single method. - The `CompareOp` argument indicates the comparison operation being performed. You can use - [`CompareOp::matches`] to adapt a Rust `std::cmp::Ordering` result to the requested comparison. - - _This method cannot be implemented in combination with any of `__lt__`, `__le__`, `__eq__`, `__ne__`, `__gt__`, or `__ge__`._ - - _Note that implementing `__richcmp__` will cause Python not to generate a default `__hash__` implementation, so consider implementing `__hash__` when implementing `__richcmp__`._ -
- Return type - The return type will normally be `PyResult`, but any Python object can be returned. - - If you want to leave some operations unimplemented, you can return `py.NotImplemented()` - for some of the operations: - - ```rust,no_run - use pyo3::class::basic::CompareOp; - use pyo3::types::PyNotImplemented; - - # use pyo3::prelude::*; - # use pyo3::BoundObject; - # - # #[pyclass] - # struct Number(i32); - # - #[pymethods] - impl Number { - fn __richcmp__<'py>(&self, other: &Self, op: CompareOp, py: Python<'py>) -> PyResult> { - match op { - CompareOp::Eq => Ok((self.0 == other.0).into_pyobject(py)?.into_any()), - CompareOp::Ne => Ok((self.0 != other.0).into_pyobject(py)?.into_any()), - _ => Ok(PyNotImplemented::get(py).into_any()), - } - } - } - ``` - - If the second argument `object` is not of the type specified in the - signature, the generated code will automatically `return NotImplemented`. -
- - - `__getattr__(, object) -> object` - - `__getattribute__(, object) -> object` -
- Differences between `__getattr__` and `__getattribute__` - As in Python, `__getattr__` is only called if the attribute is not found - by normal attribute lookup. `__getattribute__`, on the other hand, is - called for *every* attribute access. If it wants to access existing - attributes on `self`, it needs to be very careful not to introduce - infinite recursion, and use `baseclass.__getattribute__()`. -
- - - `__setattr__(, value: object) -> ()` - - `__delattr__(, object) -> ()` - - Overrides attribute access. +The following sections list all magic methods for which PyO3 implements the necessary special handling. The +given signatures should be interpreted as follows: - - `__bool__() -> bool` +- All methods take a receiver as first argument, shown as ``. It can be + `&self`, `&mut self` or a `Bound` reference like `self_: PyRef<'_, Self>` and + `self_: PyRefMut<'_, Self>`, as described [here](../class.md#inheritance). +- An optional `Python<'py>` argument is always allowed as the first argument. +- Return values can be optionally wrapped in `PyResult`. +- `object` means that any type is allowed that can be extracted from a Python + object (if argument) or converted to a Python object (if return value). +- Other types must match what's given, e.g. `pyo3::basic::CompareOp` for + `__richcmp__`'s second argument. +- For the comparison and arithmetic methods, extraction errors are not + propagated as exceptions, but lead to a return of `NotImplemented`. +- For some magic methods, the return values are not restricted by PyO3, but + checked by the Python interpreter. For example, `__str__` needs to return a + string object. This is indicated by `object (Python type)`. - Determines the "truthyness" of an object. +### Basic object customization - - `__call__(, ...) -> object` - here, any argument list can be defined - as for normal `pymethods` +- `__str__() -> object (str)` +- `__repr__() -> object (str)` + +- `__hash__() -> isize` + + Objects that compare equal must have the same hash value. Any type up to 64 bits may be returned instead of `isize`, PyO3 will convert to an isize automatically (wrapping unsigned types like `u64` and `usize`). +
+ Disabling Python's default hash + By default, all `#[pyclass]` types have a default hash implementation from Python. Types which should not be hashable can override this by setting `__hash__` to `None`. This is the same mechanism as for a pure-Python class. This is done like so: + + ```rust,no_run + # use pyo3::prelude::*; + # + #[pyclass] + struct NotHashable {} + + #[pymethods] + impl NotHashable { + #[classattr] + const __hash__: Option = None; + } + ``` + +
+ +- `__lt__(, object) -> object` +- `__le__(, object) -> object` +- `__eq__(, object) -> object` +- `__ne__(, object) -> object` +- `__gt__(, object) -> object` +- `__ge__(, object) -> object` + + The implementations of Python's "rich comparison" operators `<`, `<=`, `==`, `!=`, `>` and `>=` respectively. + + _Note that implementing any of these methods will cause Python not to generate a default `__hash__` implementation, so consider also implementing `__hash__`._ +
+ Return type + The return type will normally be `bool` or `PyResult`, however any Python object can be returned. +
+ +- `__richcmp__(, object, pyo3::basic::CompareOp) -> object` + + Implements Python comparison operations (`==`, `!=`, `<`, `<=`, `>`, and `>=`) in a single method. + The `CompareOp` argument indicates the comparison operation being performed. You can use + [`CompareOp::matches`] to adapt a Rust `std::cmp::Ordering` result to the requested comparison. + + _This method cannot be implemented in combination with any of `__lt__`, `__le__`, `__eq__`, `__ne__`, `__gt__`, or `__ge__`._ + + _Note that implementing `__richcmp__` will cause Python not to generate a default `__hash__` implementation, so consider implementing `__hash__` when implementing `__richcmp__`._ +
+ Return type + The return type will normally be `PyResult`, but any Python object can be returned. + + If you want to leave some operations unimplemented, you can return `py.NotImplemented()` + for some of the operations: + + ```rust,no_run + use pyo3::class::basic::CompareOp; + use pyo3::types::PyNotImplemented; + + # use pyo3::prelude::*; + # use pyo3::BoundObject; + # + # #[pyclass] + # struct Number(i32); + # + #[pymethods] + impl Number { + fn __richcmp__<'py>(&self, other: &Self, op: CompareOp, py: Python<'py>) -> PyResult> { + match op { + CompareOp::Eq => Ok((self.0 == other.0).into_pyobject(py)?.into_any()), + CompareOp::Ne => Ok((self.0 != other.0).into_pyobject(py)?.into_any()), + _ => Ok(PyNotImplemented::get(py).into_any()), + } + } + } + ``` + + If the second argument `object` is not of the type specified in the + signature, the generated code will automatically `return NotImplemented`. +
+ +- `__getattr__(, object) -> object` +- `__getattribute__(, object) -> object` +
+ Differences between `__getattr__` and `__getattribute__` + As in Python, `__getattr__` is only called if the attribute is not found + by normal attribute lookup. `__getattribute__`, on the other hand, is + called for *every* attribute access. If it wants to access existing + attributes on `self`, it needs to be very careful not to introduce + infinite recursion, and use `baseclass.__getattribute__()`. +
+ +- `__setattr__(, value: object) -> ()` +- `__delattr__(, object) -> ()` + + Overrides attribute access. + +- `__bool__() -> bool` + + Determines the "truthyness" of an object. + +- `__call__(, ...) -> object` - here, any argument list can be defined + as for normal `pymethods` ### Iterable objects Iterators can be defined using these methods: - - `__iter__() -> object` - - `__next__() -> Option or IterNextOutput` ([see details](#returning-a-value-from-iteration)) +- `__iter__() -> object` +- `__next__() -> Option or IterNextOutput` ([see details](#returning-a-value-from-iteration)) Returning `None` from `__next__` indicates that that there are no further items. @@ -167,17 +171,17 @@ struct MyIterator { #[pymethods] impl MyIterator { - fn __iter__(slf: PyRef<'_, Self>) -> PyRef<'_, Self> { + fn __iter__<'a, 'py>(slf: PyRef<'a, 'py, Self>) -> PyRef<'a, 'py, Self> { slf } - fn __next__(slf: PyRefMut<'_, Self>) -> Option { + fn __next__(slf: PyRefMut<'_, '_, Self>) -> Option { slf.iter.lock().unwrap().next() } } ``` In many cases you'll have a distinction between the type being iterated over -(i.e. the *iterable*) and the iterator it provides. In this case, the iterable +(i.e. the _iterable_) and the iterator it provides. In this case, the iterable only needs to implement `__iter__()` while the iterator must implement both `__iter__()` and `__next__()`. For example: @@ -191,11 +195,11 @@ struct Iter { #[pymethods] impl Iter { - fn __iter__(slf: PyRef<'_, Self>) -> PyRef<'_, Self> { + fn __iter__<'a, 'py>(slf: PyRef<'a, 'py, Self>) -> PyRef<'a, 'py, Self> { slf } - fn __next__(mut slf: PyRefMut<'_, Self>) -> Option { + fn __next__(mut slf: PyRefMut<'_, '_, Self>) -> Option { slf.inner.next() } } @@ -207,7 +211,7 @@ struct Container { #[pymethods] impl Container { - fn __iter__(slf: PyRef<'_, Self>) -> PyResult> { + fn __iter__(slf: PyRef<'_, '_, Self>) -> PyResult> { let iter = Iter { inner: slf.iter.clone().into_iter(), }; @@ -229,15 +233,15 @@ documentation](https://docs.python.org/library/stdtypes.html#iterator-types). #### Returning a value from iteration This guide has so far shown how to use `Option` to implement yielding values -during iteration. In Python a generator can also return a value. This is done by +during iteration. In Python a generator can also return a value. This is done by raising a `StopIteration` exception. To express this in Rust, return `PyResult::Err` with a `PyStopIteration` as the error. ### Awaitable objects - - `__await__() -> object` - - `__aiter__() -> object` - - `__anext__() -> Option` +- `__await__() -> object` +- `__aiter__() -> object` +- `__anext__() -> Option` ### Mapping & Sequence types @@ -248,6 +252,7 @@ The Python C-API which PyO3 is built upon has separate "slots" for sequences and By default PyO3 reproduces the Python behaviour of filling both mapping and sequence slots. This makes sense for the "simple" case which matches Python, and also for sequences, where the mapping slot is used anyway to implement slice indexing. Mapping types usually will not want the sequence slots filled. Having them filled will lead to outcomes which may be unwanted, such as: + - The mapping type will successfully cast to [`PySequence`]. This may lead to consumers of the type handling it incorrectly. - Python provides a default implementation of `__iter__` for sequences, which calls `__getitem__` with consecutive positive integers starting from 0 until an `IndexError` is returned. Unless the mapping only contains consecutive positive integer keys, this `__iter__` implementation will likely not be the intended behavior. @@ -255,91 +260,92 @@ Use the `#[pyclass(mapping)]` annotation to instruct PyO3 to only fill the mappi Use the `#[pyclass(sequence)]` annotation to instruct PyO3 to fill the `sq_length` slot instead of the `mp_length` slot for `__len__`. This will help libraries such as `numpy` recognise the class as a sequence, however will also cause CPython to automatically add the sequence length to any negative indices before passing them to `__getitem__`. (`__getitem__`, `__setitem__` and `__delitem__` mapping slots are still used for sequences, for slice operations.) - - `__len__() -> usize` +- `__len__() -> usize` - Implements the built-in function `len()`. + Implements the built-in function `len()`. - - `__contains__(, object) -> bool` +- `__contains__(, object) -> bool` - Implements membership test operators. - Should return true if `item` is in `self`, false otherwise. - For objects that don’t define `__contains__()`, the membership test simply - traverses the sequence until it finds a match. + Implements membership test operators. + Should return true if `item` is in `self`, false otherwise. + For objects that don’t define `__contains__()`, the membership test simply + traverses the sequence until it finds a match. -
- Disabling Python's default contains +
+ Disabling Python's default contains - By default, all `#[pyclass]` types with an `__iter__` method support a - default implementation of the `in` operator. Types which do not want this - can override this by setting `__contains__` to `None`. This is the same - mechanism as for a pure-Python class. This is done like so: + By default, all `#[pyclass]` types with an `__iter__` method support a + default implementation of the `in` operator. Types which do not want this + can override this by setting `__contains__` to `None`. This is the same + mechanism as for a pure-Python class. This is done like so: - ```rust,no_run - # use pyo3::prelude::*; - # - #[pyclass] - struct NoContains {} + ```rust,no_run + # use pyo3::prelude::*; + # + #[pyclass] + struct NoContains {} - #[pymethods] - impl NoContains { - #[classattr] - const __contains__: Option = None; - } - ``` -
+ #[pymethods] + impl NoContains { + #[classattr] + const __contains__: Option = None; + } + ``` + +
- - `__getitem__(, object) -> object` +- `__getitem__(, object) -> object` - Implements retrieval of the `self[a]` element. + Implements retrieval of the `self[a]` element. - *Note:* Negative integer indexes are not handled specially by PyO3. - However, for classes with `#[pyclass(sequence)]`, when a negative index is - accessed via `PySequence::get_item`, the underlying C API already adjusts - the index to be positive. + _Note:_ Negative integer indexes are not handled specially by PyO3. + However, for classes with `#[pyclass(sequence)]`, when a negative index is + accessed via `PySequence::get_item`, the underlying C API already adjusts + the index to be positive. - - `__setitem__(, object, object) -> ()` +- `__setitem__(, object, object) -> ()` - Implements assignment to the `self[a]` element. - Should only be implemented if elements can be replaced. + Implements assignment to the `self[a]` element. + Should only be implemented if elements can be replaced. - Same behavior regarding negative indices as for `__getitem__`. + Same behavior regarding negative indices as for `__getitem__`. - - `__delitem__(, object) -> ()` +- `__delitem__(, object) -> ()` - Implements deletion of the `self[a]` element. - Should only be implemented if elements can be deleted. + Implements deletion of the `self[a]` element. + Should only be implemented if elements can be deleted. - Same behavior regarding negative indices as for `__getitem__`. + Same behavior regarding negative indices as for `__getitem__`. - * `fn __concat__(&self, other: impl FromPyObject) -> PyResult` +* `fn __concat__(&self, other: impl FromPyObject) -> PyResult` - Concatenates two sequences. - Used by the `+` operator, after trying the numeric addition via - the `__add__` and `__radd__` methods. + Concatenates two sequences. + Used by the `+` operator, after trying the numeric addition via + the `__add__` and `__radd__` methods. - * `fn __repeat__(&self, count: isize) -> PyResult` +* `fn __repeat__(&self, count: isize) -> PyResult` - Repeats the sequence `count` times. - Used by the `*` operator, after trying the numeric multiplication via - the `__mul__` and `__rmul__` methods. + Repeats the sequence `count` times. + Used by the `*` operator, after trying the numeric multiplication via + the `__mul__` and `__rmul__` methods. - * `fn __inplace_concat__(&self, other: impl FromPyObject) -> PyResult` +* `fn __inplace_concat__(&self, other: impl FromPyObject) -> PyResult` - Concatenates two sequences. - Used by the `+=` operator, after trying the numeric addition via - the `__iadd__` method. + Concatenates two sequences. + Used by the `+=` operator, after trying the numeric addition via + the `__iadd__` method. - * `fn __inplace_repeat__(&self, count: isize) -> PyResult` +* `fn __inplace_repeat__(&self, count: isize) -> PyResult` - Concatenates two sequences. - Used by the `*=` operator, after trying the numeric multiplication via - the `__imul__` method. + Concatenates two sequences. + Used by the `*=` operator, after trying the numeric multiplication via + the `__imul__` method. ### Descriptors - - `__get__(, object, object) -> object` - - `__set__(, object, object) -> ()` - - `__delete__(, object) -> ()` +- `__get__(, object, object) -> object` +- `__set__(, object, object) -> ()` +- `__delete__(, object) -> ()` ### Numeric types @@ -349,84 +355,84 @@ Binary arithmetic operations (`+`, `-`, `*`, `@`, `/`, `//`, `%`, `divmod()`, (If the `object` is not of the type specified in the signature, the generated code will automatically `return NotImplemented`.) - - `__add__(, object) -> object` - - `__radd__(, object) -> object` - - `__sub__(, object) -> object` - - `__rsub__(, object) -> object` - - `__mul__(, object) -> object` - - `__rmul__(, object) -> object` - - `__matmul__(, object) -> object` - - `__rmatmul__(, object) -> object` - - `__floordiv__(, object) -> object` - - `__rfloordiv__(, object) -> object` - - `__truediv__(, object) -> object` - - `__rtruediv__(, object) -> object` - - `__divmod__(, object) -> object` - - `__rdivmod__(, object) -> object` - - `__mod__(, object) -> object` - - `__rmod__(, object) -> object` - - `__lshift__(, object) -> object` - - `__rlshift__(, object) -> object` - - `__rshift__(, object) -> object` - - `__rrshift__(, object) -> object` - - `__and__(, object) -> object` - - `__rand__(, object) -> object` - - `__xor__(, object) -> object` - - `__rxor__(, object) -> object` - - `__or__(, object) -> object` - - `__ror__(, object) -> object` - - `__pow__(, object, object) -> object` - - `__rpow__(, object, object) -> object` +- `__add__(, object) -> object` +- `__radd__(, object) -> object` +- `__sub__(, object) -> object` +- `__rsub__(, object) -> object` +- `__mul__(, object) -> object` +- `__rmul__(, object) -> object` +- `__matmul__(, object) -> object` +- `__rmatmul__(, object) -> object` +- `__floordiv__(, object) -> object` +- `__rfloordiv__(, object) -> object` +- `__truediv__(, object) -> object` +- `__rtruediv__(, object) -> object` +- `__divmod__(, object) -> object` +- `__rdivmod__(, object) -> object` +- `__mod__(, object) -> object` +- `__rmod__(, object) -> object` +- `__lshift__(, object) -> object` +- `__rlshift__(, object) -> object` +- `__rshift__(, object) -> object` +- `__rrshift__(, object) -> object` +- `__and__(, object) -> object` +- `__rand__(, object) -> object` +- `__xor__(, object) -> object` +- `__rxor__(, object) -> object` +- `__or__(, object) -> object` +- `__ror__(, object) -> object` +- `__pow__(, object, object) -> object` +- `__rpow__(, object, object) -> object` In-place assignment operations (`+=`, `-=`, `*=`, `@=`, `/=`, `//=`, `%=`, `**=`, `<<=`, `>>=`, `&=`, `^=`, `|=`): - - `__iadd__(, object) -> ()` - - `__isub__(, object) -> ()` - - `__imul__(, object) -> ()` - - `__imatmul__(, object) -> ()` - - `__itruediv__(, object) -> ()` - - `__ifloordiv__(, object) -> ()` - - `__imod__(, object) -> ()` - - `__ipow__(, object, object) -> ()` - - `__ilshift__(, object) -> ()` - - `__irshift__(, object) -> ()` - - `__iand__(, object) -> ()` - - `__ixor__(, object) -> ()` - - `__ior__(, object) -> ()` +- `__iadd__(, object) -> ()` +- `__isub__(, object) -> ()` +- `__imul__(, object) -> ()` +- `__imatmul__(, object) -> ()` +- `__itruediv__(, object) -> ()` +- `__ifloordiv__(, object) -> ()` +- `__imod__(, object) -> ()` +- `__ipow__(, object, object) -> ()` +- `__ilshift__(, object) -> ()` +- `__irshift__(, object) -> ()` +- `__iand__(, object) -> ()` +- `__ixor__(, object) -> ()` +- `__ior__(, object) -> ()` Unary operations (`-`, `+`, `abs()` and `~`): - - `__pos__() -> object` - - `__neg__() -> object` - - `__abs__() -> object` - - `__invert__() -> object` +- `__pos__() -> object` +- `__neg__() -> object` +- `__abs__() -> object` +- `__invert__() -> object` Coercions: - - `__index__() -> object (int)` - - `__int__() -> object (int)` - - `__float__() -> object (float)` +- `__index__() -> object (int)` +- `__int__() -> object (int)` +- `__float__() -> object (float)` ### Buffer objects - - `__getbuffer__(, *mut ffi::Py_buffer, flags) -> ()` - - `__releasebuffer__(, *mut ffi::Py_buffer) -> ()` - Errors returned from `__releasebuffer__` will be sent to `sys.unraiseablehook`. It is strongly advised to never return an error from `__releasebuffer__`, and if it really is necessary, to make best effort to perform any required freeing operations before returning. `__releasebuffer__` will not be called a second time; anything not freed will be leaked. +- `__getbuffer__(, *mut ffi::Py_buffer, flags) -> ()` +- `__releasebuffer__(, *mut ffi::Py_buffer) -> ()` + Errors returned from `__releasebuffer__` will be sent to `sys.unraiseablehook`. It is strongly advised to never return an error from `__releasebuffer__`, and if it really is necessary, to make best effort to perform any required freeing operations before returning. `__releasebuffer__` will not be called a second time; anything not freed will be leaked. ### Garbage Collector Integration If your type owns references to other Python objects, you will need to integrate -with Python's garbage collector so that the GC is aware of those references. To -do this, implement the two methods `__traverse__` and `__clear__`. These +with Python's garbage collector so that the GC is aware of those references. To +do this, implement the two methods `__traverse__` and `__clear__`. These correspond to the slots `tp_traverse` and `tp_clear` in the Python C API. `__traverse__` must call `visit.call()` for each reference to another Python -object. `__clear__` must clear out any mutable references to other Python +object. `__clear__` must clear out any mutable references to other Python objects (thus breaking reference cycles). Immutable references do not have to be cleared, as every cycle must contain at least one mutable reference. - - `__traverse__(, pyo3::class::gc::PyVisit<'_>) -> Result<(), pyo3::class::gc::PyTraverseError>` - - `__clear__() -> ()` +- `__traverse__(, pyo3::class::gc::PyVisit<'_>) -> Result<(), pyo3::class::gc::PyTraverseError>` +- `__clear__() -> ()` Example: diff --git a/guide/src/migration.md b/guide/src/migration.md index 6b1adb8e803..6049ebbcd41 100644 --- a/guide/src/migration.md +++ b/guide/src/migration.md @@ -14,6 +14,121 @@ The `AsPyPointer` trait is mostly a leftover from the now removed gil-refs API. `PyAnyMethods::is`/`Py::is` has been updated to take `T: AsRef>>`. Additionally `AsRef>>` implementations were added for `Py`, `Bound` and `Borrowed`. Because of the existing `AsRef> for Bound` implementation this may cause inference issues in non-generic code. This can be easily migrated by switching to `as_any` instead of `as_ref` for these calls. +### `FromPyObject` gains additional lifetime +
+Click to expand + +With the removal of the `gil-ref` API it is now possible to fully split the Python GIL lifetime +`'py` and the input lifetime `'a`. This allows borrowing from the input data without extending the +GIL lifetime. + +`FromPyObject` now takes an additional lifetime `'a` describing the input lifetime. The argument +type of the `extract` method changed from `&Bound<'py, PyAny>` to `Borrowed<'a, 'py, PyAny>`. This was +done because `&'a Bound<'py, PyAny>` would have an implicit restriction `'py: 'a` due to the reference type. `extract_bound` with its +old signature is deprecated, but still available during migration. + +This new form was partly implemented already in 0.22 using the internal `FromPyObjectBound` trait and +is now extended to all types. + +Most implementations can just add an elided lifetime to migrate. + +Before: +```rust,ignore +impl<'py> FromPyObject<'py> for IpAddr { + fn extract_bound(obj: &Bound<'py, PyAny>) -> PyResult { + ... + } +} +``` + +After +```rust,ignore +impl<'py> FromPyObject<'_, 'py> for IpAddr { + fn extract(obj: Borrowed<'_, 'py, PyAny>) -> PyResult { + ... + // since `Borrowed` derefs to `&Bound`, the body often + // needs no changes, or adding an occasional `&` + } +} +``` + +Occasionally, more steps are necessary. For generic types, the bounds need to be adjusted. The +correct bound depends on how the type is used. + +For simple wrapper types usually it's possible to just forward the bound. + +Before: +```rust,ignore +struct MyWrapper(T); + +impl<'py, T> FromPyObject<'py> for MyWrapper +where + T: FromPyObject<'py> +{ + fn extract_bound(obj: &Bound<'py, PyAny>) -> PyResult { + ob.extract().map(MyWrapper) + } +} +``` + +After: +```rust +# use pyo3::prelude::*; +# pub struct MyWrapper(T); +impl<'a, 'py, T> FromPyObject<'a, 'py> for MyWrapper +where + T: FromPyObject<'a, 'py> +{ + fn extract(obj: Borrowed<'a, 'py, PyAny>) -> PyResult { + obj.extract().map(MyWrapper) + } +} +``` + +Container types that need to create temporary Python references during extraction, for example +extracing from a `PyList`, require a stronger bound. For these the `FromPyObjectOwned` trait was +introduced. It is automatically implemented for any type that implements `FromPyObject` and does not +borrow from the input. It is intended to be used as a trait bound in these situations. + +Before: +```rust,ignore +struct MyVec(Vec); +impl<'py, T> FromPyObject<'py> for Vec +where + T: FromPyObject<'py>, +{ + fn extract_bound(obj: &Bound<'py, PyAny>) -> PyResult { + let mut v = MyVec(Vec::new()); + for item in obj.try_iter()? { + v.0.push(item?.extract::()?); + } + Ok(v) + } +} +``` + +After: +```rust +# use pyo3::prelude::*; +# pub struct MyVec(Vec); +impl<'py, T> FromPyObject<'_, 'py> for MyVec +where + T: FromPyObjectOwned<'py> // 👈 can only extract owned values, because each `item` below + // is a temporary short lived owned reference +{ + fn extract(obj: Borrowed<'_, 'py, PyAny>) -> PyResult { + let mut v = MyVec(Vec::new()); + for item in obj.try_iter()? { + v.0.push(item?.extract::()?); + } + Ok(v) + } +} +``` + +This is very similar to `serde`s `Deserialize` and `DeserializeOwned` traits. +
+ ## from 0.22.* to 0.23
Click to expand diff --git a/newsfragments/4390.added.md b/newsfragments/4390.added.md new file mode 100644 index 00000000000..c234b0b6547 --- /dev/null +++ b/newsfragments/4390.added.md @@ -0,0 +1,2 @@ +added `FromPyObjectOwned` as more convenient trait bound +added `Borrowed::extract`, same as `PyAnyMethods::extract`, but does not restrict the lifetime by deref \ No newline at end of file diff --git a/newsfragments/4390.changed.md b/newsfragments/4390.changed.md new file mode 100644 index 00000000000..50c9ffb7fec --- /dev/null +++ b/newsfragments/4390.changed.md @@ -0,0 +1,3 @@ +added second lifetime to `FromPyObject` +reintroduced `extract` method +deprecated `extract_bound` method \ No newline at end of file diff --git a/newsfragments/4390.removed.md b/newsfragments/4390.removed.md new file mode 100644 index 00000000000..a40ee2db268 --- /dev/null +++ b/newsfragments/4390.removed.md @@ -0,0 +1 @@ +removed `FromPyObjectBound` \ No newline at end of file diff --git a/pyo3-macros-backend/src/frompyobject.rs b/pyo3-macros-backend/src/frompyobject.rs index 68f95e794a8..4784d2a9f9f 100644 --- a/pyo3-macros-backend/src/frompyobject.rs +++ b/pyo3-macros-backend/src/frompyobject.rs @@ -712,7 +712,7 @@ pub fn build_derive_from_pyobject(tokens: &DeriveInput) -> Result { let gen_ident = ¶m.ident; where_clause .predicates - .push(parse_quote!(#gen_ident: #pyo3_path::FromPyObject<'py>)) + .push(parse_quote!(#gen_ident: #pyo3_path::conversion::FromPyObjectOwned<#lt_param>)) } let derives = match &tokens.data { @@ -740,8 +740,9 @@ pub fn build_derive_from_pyobject(tokens: &DeriveInput) -> Result { let ident = &tokens.ident; Ok(quote!( #[automatically_derived] - impl #impl_generics #pyo3_path::FromPyObject<#lt_param> for #ident #ty_generics #where_clause { - fn extract_bound(obj: &#pyo3_path::Bound<#lt_param, #pyo3_path::PyAny>) -> #pyo3_path::PyResult { + impl #impl_generics #pyo3_path::FromPyObject<'_, #lt_param> for #ident #ty_generics #where_clause { + fn extract(obj: #pyo3_path::Borrowed<'_, #lt_param, #pyo3_path::PyAny>) -> #pyo3_path::PyResult { + let obj: &#pyo3_path::Bound<'_, _> = &*obj; #derives } } diff --git a/pyo3-macros-backend/src/method.rs b/pyo3-macros-backend/src/method.rs index dfcd5dd2203..dccace7c352 100644 --- a/pyo3-macros-backend/src/method.rs +++ b/pyo3-macros-backend/src/method.rs @@ -1126,7 +1126,7 @@ fn parse_method_attributes(attrs: &mut Vec) -> Result` or `slf: PyRefMut<'_, '_, Self>`."; fn ensure_signatures_on_valid_method( fn_type: &FnType, diff --git a/pyo3-macros-backend/src/pyclass.rs b/pyo3-macros-backend/src/pyclass.rs index 82d4d54c7ec..c9ebe5d20af 100644 --- a/pyo3-macros-backend/src/pyclass.rs +++ b/pyo3-macros-backend/src/pyclass.rs @@ -2071,34 +2071,34 @@ impl<'a> PyClassImplsBuilder<'a> { let cls = self.cls; if self.attr.options.frozen.is_some() { quote! { - impl<'a, 'py> #pyo3_path::impl_::extract_argument::PyFunctionArgument<'a, 'py, false> for &'a #cls + impl<'a, 'holder, 'py> #pyo3_path::impl_::extract_argument::PyFunctionArgument<'a, 'holder, 'py, false> for &'holder #cls { - type Holder = ::std::option::Option<#pyo3_path::PyRef<'py, #cls>>; + type Holder = ::std::option::Option<#pyo3_path::PyRef<'a, 'py, #cls>>; #[inline] - fn extract(obj: &'a #pyo3_path::Bound<'py, #pyo3_path::PyAny>, holder: &'a mut Self::Holder) -> #pyo3_path::PyResult { + fn extract(obj: &'a #pyo3_path::Bound<'py, #pyo3_path::PyAny>, holder: &'holder mut Self::Holder) -> #pyo3_path::PyResult { #pyo3_path::impl_::extract_argument::extract_pyclass_ref(obj, holder) } } } } else { quote! { - impl<'a, 'py> #pyo3_path::impl_::extract_argument::PyFunctionArgument<'a, 'py, false> for &'a #cls + impl<'a, 'holder, 'py> #pyo3_path::impl_::extract_argument::PyFunctionArgument<'a, 'holder, 'py, false> for &'holder #cls { - type Holder = ::std::option::Option<#pyo3_path::PyRef<'py, #cls>>; + type Holder = ::std::option::Option<#pyo3_path::PyRef<'a, 'py, #cls>>; #[inline] - fn extract(obj: &'a #pyo3_path::Bound<'py, #pyo3_path::PyAny>, holder: &'a mut Self::Holder) -> #pyo3_path::PyResult { + fn extract(obj: &'a #pyo3_path::Bound<'py, #pyo3_path::PyAny>, holder: &'holder mut Self::Holder) -> #pyo3_path::PyResult { #pyo3_path::impl_::extract_argument::extract_pyclass_ref(obj, holder) } } - impl<'a, 'py> #pyo3_path::impl_::extract_argument::PyFunctionArgument<'a, 'py, false> for &'a mut #cls + impl<'a, 'holder, 'py> #pyo3_path::impl_::extract_argument::PyFunctionArgument<'a, 'holder, 'py, false> for &'holder mut #cls { - type Holder = ::std::option::Option<#pyo3_path::PyRefMut<'py, #cls>>; + type Holder = ::std::option::Option<#pyo3_path::PyRefMut<'a, 'py, #cls>>; #[inline] - fn extract(obj: &'a #pyo3_path::Bound<'py, #pyo3_path::PyAny>, holder: &'a mut Self::Holder) -> #pyo3_path::PyResult { + fn extract(obj: &'a #pyo3_path::Bound<'py, #pyo3_path::PyAny>, holder: &'holder mut Self::Holder) -> #pyo3_path::PyResult { #pyo3_path::impl_::extract_argument::extract_pyclass_ref_mut(obj, holder) } } diff --git a/pyo3-macros-backend/src/pymethod.rs b/pyo3-macros-backend/src/pymethod.rs index b4637b48012..a3138474d85 100644 --- a/pyo3-macros-backend/src/pymethod.rs +++ b/pyo3-macros-backend/src/pymethod.rs @@ -1142,7 +1142,7 @@ impl Ty { extract_error_mode, holders, arg, - quote! { #ident.as_ptr() }, + quote! { #ident }, ctx ), Ty::IPowModulo => extract_object( @@ -1470,6 +1470,15 @@ impl SlotFragmentDef { let method = syn::Ident::new(fragment, Span::call_site()); let wrapper_ident = format_ident!("__pymethod_{}__", fragment); let arg_types: &Vec<_> = &arguments.iter().map(|arg| arg.ffi_type(ctx)).collect(); + let nn = arguments + .iter() + .enumerate() + .filter(|(_, arg)| matches!(arg, Ty::NonNullObject)) + .map(|(i, _)| { + let i = format_ident!("arg{}", i); + quote! { let #i = #i.as_ptr(); } + }) + .collect::(); let arg_idents: &Vec<_> = &(0..arguments.len()) .map(|i| format_ident!("arg{}", i)) .collect(); @@ -1494,6 +1503,7 @@ impl SlotFragmentDef { #(#arg_idents: #arg_types),* ) -> #pyo3_path::PyResult<#ret_ty> { let _slf = _raw_slf; + #nn #holders #body } diff --git a/pytests/src/awaitable.rs b/pytests/src/awaitable.rs index 01a93c70a0d..451a925f9bd 100644 --- a/pytests/src/awaitable.rs +++ b/pytests/src/awaitable.rs @@ -23,11 +23,11 @@ impl IterAwaitable { } } - fn __await__(pyself: PyRef<'_, Self>) -> PyRef<'_, Self> { + fn __await__<'a, 'py>(pyself: PyRef<'a, 'py, Self>) -> PyRef<'a, 'py, Self> { pyself } - fn __iter__(pyself: PyRef<'_, Self>) -> PyRef<'_, Self> { + fn __iter__<'a, 'py>(pyself: PyRef<'a, 'py, Self>) -> PyRef<'a, 'py, Self> { pyself } @@ -59,15 +59,15 @@ impl FutureAwaitable { } } - fn __await__(pyself: PyRef<'_, Self>) -> PyRef<'_, Self> { + fn __await__<'a, 'py>(pyself: PyRef<'a, 'py, Self>) -> PyRef<'a, 'py, Self> { pyself } - fn __iter__(pyself: PyRef<'_, Self>) -> PyRef<'_, Self> { + fn __iter__<'a, 'py>(pyself: PyRef<'a, 'py, Self>) -> PyRef<'a, 'py, Self> { pyself } - fn __next__(mut pyself: PyRefMut<'_, Self>) -> PyResult> { + fn __next__<'a, 'py>(mut pyself: PyRefMut<'a, 'py, Self>) -> PyResult> { match pyself.result { Some(_) => match pyself.result.take().unwrap() { Ok(v) => Err(PyStopIteration::new_err(v)), diff --git a/src/buffer.rs b/src/buffer.rs index c5e5de568eb..188b0a50230 100644 --- a/src/buffer.rs +++ b/src/buffer.rs @@ -18,8 +18,8 @@ // DEALINGS IN THE SOFTWARE. //! `PyBuffer` implementation -use crate::Bound; use crate::{err, exceptions::PyBufferError, ffi, FromPyObject, PyAny, PyResult, Python}; +use crate::{Borrowed, Bound}; use std::marker::PhantomData; use std::os::raw; use std::pin::Pin; @@ -182,9 +182,9 @@ pub unsafe trait Element: Copy { fn is_compatible_format(format: &CStr) -> bool; } -impl FromPyObject<'_> for PyBuffer { - fn extract_bound(obj: &Bound<'_, PyAny>) -> PyResult> { - Self::get(obj) +impl FromPyObject<'_, '_> for PyBuffer { + fn extract(obj: Borrowed<'_, '_, PyAny>) -> PyResult> { + Self::get(&obj) } } diff --git a/src/conversion.rs b/src/conversion.rs index 165175fae54..0a5ce374e08 100644 --- a/src/conversion.rs +++ b/src/conversion.rs @@ -3,7 +3,6 @@ use crate::err::PyResult; #[cfg(feature = "experimental-inspect")] use crate::inspect::types::TypeInfo; use crate::pyclass::boolean_struct::False; -use crate::types::any::PyAnyMethods; use crate::types::PyTuple; use crate::{Borrowed, Bound, BoundObject, Py, PyAny, PyClass, PyErr, PyRef, PyRefMut, Python}; use std::convert::Infallible; @@ -235,7 +234,8 @@ impl<'py, T> IntoPyObjectExt<'py> for T where T: IntoPyObject<'py> {} /// Extract a type from a Python object. /// /// -/// Normal usage is through the `extract` methods on [`Bound`] and [`Py`], which forward to this trait. +/// Normal usage is through the `extract` methods on [`Bound`], [`Borrowed`] and [`Py`], which +/// forward to this trait. /// /// # Examples /// @@ -259,29 +259,38 @@ impl<'py, T> IntoPyObjectExt<'py> for T where T: IntoPyObject<'py> {} /// # } /// ``` /// -// /// FIXME: until `FromPyObject` can pick up a second lifetime, the below commentary is no longer -// /// true. Update and restore this documentation at that time. -// /// -// /// Note: depending on the implementation, the lifetime of the extracted result may -// /// depend on the lifetime of the `obj` or the `prepared` variable. -// /// -// /// For example, when extracting `&str` from a Python byte string, the resulting string slice will -// /// point to the existing string data (lifetime: `'py`). -// /// On the other hand, when extracting `&str` from a Python Unicode string, the preparation step -// /// will convert the string to UTF-8, and the resulting string slice will have lifetime `'prepared`. -// /// Since which case applies depends on the runtime type of the Python object, -// /// both the `obj` and `prepared` variables must outlive the resulting string slice. +/// Note: Depending on the Python version and implementation, some [`FromPyObject`] implementations +/// may produce a result that borrows into the Python type. This is described by the input lifetime +/// `'a` of `obj`. /// -/// During the migration of PyO3 from the "GIL Refs" API to the `Bound` smart pointer, this trait -/// has two methods `extract` and `extract_bound` which are defaulted to call each other. To avoid -/// infinite recursion, implementors must implement at least one of these methods. The recommendation -/// is to implement `extract_bound` and leave `extract` as the default implementation. -pub trait FromPyObject<'py>: Sized { +/// Types that must not borrow from the input can use [`FromPyObjectOwned`] as a restriction. This +/// is most often the case for collection types. See its documentation for more details. +/// +/// # Details +/// [`Cow<'a, str>`] is an example of an output type that may or may not borrow from the input +/// lifetime `'a`. Which variant will be produced depends on the runtime type of the Python object. +/// For a Python byte string, the existing string data can be borrowed for `'a` into a +/// [`Cow::Borrowed`]. For a Python Unicode string, the data may have to be reencoded to UTF-8, and +/// copied into a [`Cow::Owned`]. It does _not_ depend on the Python lifetime `'py`. +/// +/// The output type may also depend on the Python lifetime `'py`. This allows the output type to +/// keep interacting with the Python interpreter. See also [`Bound<'py, T>`]. +/// +/// [`Cow<'a, str>`]: std::borrow::Cow +/// [`Cow::Borrowed`]: std::borrow::Cow::Borrowed +/// [`Cow::Owned`]: std::borrow::Cow::Owned +pub trait FromPyObject<'a, 'py>: Sized { /// Extracts `Self` from the bound smart pointer `obj`. /// - /// Implementors are encouraged to implement this method and leave `extract` defaulted, as - /// this will be most compatible with PyO3's future API. - fn extract_bound(ob: &Bound<'py, PyAny>) -> PyResult; + /// Users are advised against calling this method directly: instead, use this via + /// [`Bound<'_, PyAny>::extract`] or [`Py::extract`]. + fn extract(obj: Borrowed<'a, 'py, PyAny>) -> PyResult; + + /// Deprecated name for [`FromPyObject::extract`] + #[deprecated(since = "0.23.0", note = "replaced by `FromPyObject::extract`")] + fn extract_bound(ob: &'a Bound<'py, PyAny>) -> PyResult { + Self::extract(ob.as_borrowed()) + } /// Extracts the type hint information for this type when it appears as an argument. /// @@ -297,106 +306,77 @@ pub trait FromPyObject<'py>: Sized { } } -mod from_py_object_bound_sealed { - /// Private seal for the `FromPyObjectBound` trait. - /// - /// This prevents downstream types from implementing the trait before - /// PyO3 is ready to declare the trait as public API. - pub trait Sealed {} - - // This generic implementation is why the seal is separate from - // `crate::sealed::Sealed`. - impl<'py, T> Sealed for T where T: super::FromPyObject<'py> {} - impl Sealed for &'_ str {} - impl Sealed for std::borrow::Cow<'_, str> {} - impl Sealed for &'_ [u8] {} - impl Sealed for std::borrow::Cow<'_, [u8]> {} -} - -/// Expected form of [`FromPyObject`] to be used in a future PyO3 release. +/// A data structure that can be extracted without borrowing any data from the input. /// -/// The difference between this and `FromPyObject` is that this trait takes an -/// additional lifetime `'a`, which is the lifetime of the input `Bound`. +/// This is primarily useful for trait bounds. For example a [`FromPyObject`] implementation of a +/// wrapper type may be able to borrow data from the input, but a [`FromPyObject`] implementation of +/// a collection type may only extract owned data. /// -/// This allows implementations for `&'a str` and `&'a [u8]`, which could not -/// be expressed by the existing `FromPyObject` trait once the GIL Refs API was -/// removed. +/// For example [`PyList`] will not hand out references tied to its own lifetime, but "owned" +/// references independent of it. (Similar to [`Vec>`] where you clone the [`Arc`] out). +/// This makes it impossible to collect borrowed types in a collection, since they would not borrow +/// from the original [`PyList`], but the much shorter lived element reference. See the example +/// below. /// -/// # Usage +/// ``` +/// # use pyo3::prelude::*; +/// pub struct MyWrapper(T); /// -/// Users are prevented from implementing this trait, instead they should implement -/// the normal `FromPyObject` trait. This trait has a blanket implementation -/// for `T: FromPyObject`. +/// impl<'a, 'py, T> FromPyObject<'a, 'py> for MyWrapper +/// where +/// T: FromPyObject<'a, 'py> +/// { +/// fn extract(obj: Borrowed<'a, 'py, PyAny>) -> PyResult { +/// obj.extract().map(MyWrapper) +/// } +/// } /// -/// The only case where this trait may have a use case to be implemented is when the -/// lifetime of the extracted value is tied to the lifetime `'a` of the input `Bound` -/// instead of the GIL lifetime `py`, as is the case for the `&'a str` implementation. +/// pub struct MyVec(Vec); /// -/// Please contact the PyO3 maintainers if you believe you have a use case for implementing -/// this trait before PyO3 is ready to change the main `FromPyObject` trait to take an -/// additional lifetime. +/// impl<'py, T> FromPyObject<'_, 'py> for MyVec +/// where +/// T: FromPyObjectOwned<'py> // 👈 can only extract owned values, because each `item` below +/// // is a temporary short lived owned reference +/// { +/// fn extract(obj: Borrowed<'_, 'py, PyAny>) -> PyResult { +/// let mut v = MyVec(Vec::new()); +/// for item in obj.try_iter()? { +/// v.0.push(item?.extract::()?); +/// } +/// Ok(v) +/// } +/// } +/// ``` /// -/// Similarly, users should typically not call these trait methods and should instead -/// use this via the `extract` method on `Bound` and `Py`. -pub trait FromPyObjectBound<'a, 'py>: Sized + from_py_object_bound_sealed::Sealed { - /// Extracts `Self` from the bound smart pointer `obj`. - /// - /// Users are advised against calling this method directly: instead, use this via - /// [`Bound<'_, PyAny>::extract`] or [`Py::extract`]. - fn from_py_object_bound(ob: Borrowed<'a, 'py, PyAny>) -> PyResult; - - /// Extracts the type hint information for this type when it appears as an argument. - /// - /// For example, `Vec` would return `Sequence[int]`. - /// The default implementation returns `Any`, which is correct for any type. - /// - /// For most types, the return value for this method will be identical to that of - /// [`IntoPyObject::type_output`]. It may be different for some types, such as `Dict`, - /// to allow duck-typing: functions return `Dict` but take `Mapping` as argument. - #[cfg(feature = "experimental-inspect")] - fn type_input() -> TypeInfo { - TypeInfo::Any - } -} - -impl<'py, T> FromPyObjectBound<'_, 'py> for T -where - T: FromPyObject<'py>, -{ - fn from_py_object_bound(ob: Borrowed<'_, 'py, PyAny>) -> PyResult { - Self::extract_bound(&ob) - } - - #[cfg(feature = "experimental-inspect")] - fn type_input() -> TypeInfo { - ::type_input() - } -} +/// [`PyList`]: crate::types::PyList +/// [`Arc`]: std::sync::Arc +pub trait FromPyObjectOwned<'py>: for<'a> FromPyObject<'a, 'py> {} +impl<'py, T> FromPyObjectOwned<'py> for T where T: for<'a> FromPyObject<'a, 'py> {} -impl FromPyObject<'_> for T +impl FromPyObject<'_, '_> for T where T: PyClass + Clone, { - fn extract_bound(obj: &Bound<'_, PyAny>) -> PyResult { + fn extract(obj: Borrowed<'_, '_, PyAny>) -> PyResult { let bound = obj.downcast::()?; Ok(bound.try_borrow()?.clone()) } } -impl<'py, T> FromPyObject<'py> for PyRef<'py, T> +impl<'a, 'py, T> FromPyObject<'a, 'py> for PyRef<'a, 'py, T> where T: PyClass, { - fn extract_bound(obj: &Bound<'py, PyAny>) -> PyResult { + fn extract(obj: Borrowed<'a, 'py, PyAny>) -> PyResult { obj.downcast::()?.try_borrow().map_err(Into::into) } } -impl<'py, T> FromPyObject<'py> for PyRefMut<'py, T> +impl<'a, 'py, T> FromPyObject<'a, 'py> for PyRefMut<'a, 'py, T> where T: PyClass, { - fn extract_bound(obj: &Bound<'py, PyAny>) -> PyResult { + fn extract(obj: Borrowed<'a, 'py, PyAny>) -> PyResult { obj.downcast::()?.try_borrow_mut().map_err(Into::into) } } diff --git a/src/conversions/bigdecimal.rs b/src/conversions/bigdecimal.rs index 446e38bf6c5..357c156e93d 100644 --- a/src/conversions/bigdecimal.rs +++ b/src/conversions/bigdecimal.rs @@ -55,7 +55,7 @@ use crate::{ exceptions::PyValueError, sync::GILOnceCell, types::{PyAnyMethods, PyStringMethods, PyType}, - Bound, FromPyObject, IntoPyObject, Py, PyAny, PyErr, PyResult, Python, + Borrowed, Bound, FromPyObject, IntoPyObject, Py, PyAny, PyErr, PyResult, Python, }; use bigdecimal::BigDecimal; @@ -65,8 +65,8 @@ fn get_decimal_cls(py: Python<'_>) -> PyResult<&Bound<'_, PyType>> { DECIMAL_CLS.import(py, "decimal", "Decimal") } -impl FromPyObject<'_> for BigDecimal { - fn extract_bound(obj: &Bound<'_, PyAny>) -> PyResult { +impl FromPyObject<'_, '_> for BigDecimal { + fn extract(obj: Borrowed<'_, '_, PyAny>) -> PyResult { let py_str = &obj.str()?; let rs_str = &py_str.to_cow()?; BigDecimal::from_str(rs_str).map_err(|e| PyValueError::new_err(e.to_string())) diff --git a/src/conversions/chrono.rs b/src/conversions/chrono.rs index 165fc5b9c43..f4a1534148c 100644 --- a/src/conversions/chrono.rs +++ b/src/conversions/chrono.rs @@ -41,7 +41,7 @@ //! } //! ``` -use crate::conversion::IntoPyObject; +use crate::conversion::{FromPyObjectOwned, IntoPyObject}; use crate::exceptions::{PyTypeError, PyUserWarning, PyValueError}; #[cfg(Py_LIMITED_API)] use crate::intern; @@ -100,8 +100,8 @@ impl<'py> IntoPyObject<'py> for &Duration { } } -impl FromPyObject<'_> for Duration { - fn extract_bound(ob: &Bound<'_, PyAny>) -> PyResult { +impl FromPyObject<'_, '_> for Duration { + fn extract(ob: Borrowed<'_, '_, PyAny>) -> PyResult { let delta = ob.downcast::()?; // Python size are much lower than rust size so we do not need bound checks. // 0 <= microseconds < 1000000 @@ -154,9 +154,9 @@ impl<'py> IntoPyObject<'py> for &NaiveDate { } } -impl FromPyObject<'_> for NaiveDate { - fn extract_bound(ob: &Bound<'_, PyAny>) -> PyResult { - let date = ob.downcast::()?; +impl FromPyObject<'_, '_> for NaiveDate { + fn extract(ob: Borrowed<'_, '_, PyAny>) -> PyResult { + let date = &*ob.downcast::()?; py_date_to_naive_date(date) } } @@ -196,9 +196,9 @@ impl<'py> IntoPyObject<'py> for &NaiveTime { } } -impl FromPyObject<'_> for NaiveTime { - fn extract_bound(ob: &Bound<'_, PyAny>) -> PyResult { - let time = ob.downcast::()?; +impl FromPyObject<'_, '_> for NaiveTime { + fn extract(ob: Borrowed<'_, '_, PyAny>) -> PyResult { + let time = &*ob.downcast::()?; py_time_to_naive_time(time) } } @@ -239,9 +239,9 @@ impl<'py> IntoPyObject<'py> for &NaiveDateTime { } } -impl FromPyObject<'_> for NaiveDateTime { - fn extract_bound(dt: &Bound<'_, PyAny>) -> PyResult { - let dt = dt.downcast::()?; +impl FromPyObject<'_, '_> for NaiveDateTime { + fn extract(dt: Borrowed<'_, '_, PyAny>) -> PyResult { + let dt = &*dt.downcast::()?; // If the user tries to convert a timezone aware datetime into a naive one, // we return a hard error. We could silently remove tzinfo, or assume local timezone @@ -316,9 +316,12 @@ where } } -impl FromPyObject<'py>> FromPyObject<'_> for DateTime { - fn extract_bound(dt: &Bound<'_, PyAny>) -> PyResult> { - let dt = dt.downcast::()?; +impl<'py, Tz> FromPyObject<'_, 'py> for DateTime +where + Tz: TimeZone + FromPyObjectOwned<'py>, +{ + fn extract(dt: Borrowed<'_, 'py, PyAny>) -> PyResult> { + let dt = &*dt.downcast::()?; let tzinfo = dt.get_tzinfo(); let tz = if let Some(tzinfo) = tzinfo { @@ -375,12 +378,12 @@ impl<'py> IntoPyObject<'py> for &FixedOffset { } } -impl FromPyObject<'_> for FixedOffset { +impl FromPyObject<'_, '_> for FixedOffset { /// Convert python tzinfo to rust [`FixedOffset`]. /// /// Note that the conversion will result in precision lost in microseconds as chrono offset /// does not supports microseconds. - fn extract_bound(ob: &Bound<'_, PyAny>) -> PyResult { + fn extract(ob: Borrowed<'_, '_, PyAny>) -> PyResult { let ob = ob.downcast::()?; // Passing Python's None to the `utcoffset` function will only @@ -424,8 +427,8 @@ impl<'py> IntoPyObject<'py> for &Utc { } } -impl FromPyObject<'_> for Utc { - fn extract_bound(ob: &Bound<'_, PyAny>) -> PyResult { +impl FromPyObject<'_, '_> for Utc { + fn extract(ob: Borrowed<'_, '_, PyAny>) -> PyResult { let py_utc = PyTzInfo::utc(ob.py())?; if ob.eq(py_utc)? { Ok(Utc) @@ -488,7 +491,9 @@ fn warn_truncated_leap_second(obj: &Bound<'_, PyAny>) { } #[cfg(not(Py_LIMITED_API))] -fn py_date_to_naive_date(py_date: &impl PyDateAccess) -> PyResult { +fn py_date_to_naive_date( + py_date: impl std::ops::Deref, +) -> PyResult { NaiveDate::from_ymd_opt( py_date.get_year(), py_date.get_month().into(), @@ -508,7 +513,9 @@ fn py_date_to_naive_date(py_date: &Bound<'_, PyAny>) -> PyResult { } #[cfg(not(Py_LIMITED_API))] -fn py_time_to_naive_time(py_time: &impl PyTimeAccess) -> PyResult { +fn py_time_to_naive_time( + py_time: impl std::ops::Deref, +) -> PyResult { NaiveTime::from_hms_micro_opt( py_time.get_hour().into(), py_time.get_minute().into(), diff --git a/src/conversions/chrono_tz.rs b/src/conversions/chrono_tz.rs index ee7bd504888..224eda2390a 100644 --- a/src/conversions/chrono_tz.rs +++ b/src/conversions/chrono_tz.rs @@ -38,7 +38,7 @@ use crate::conversion::IntoPyObject; use crate::exceptions::PyValueError; use crate::pybacked::PyBackedStr; use crate::types::{any::PyAnyMethods, PyTzInfo}; -use crate::{intern, Bound, FromPyObject, PyAny, PyErr, PyResult, Python}; +use crate::{intern, Borrowed, Bound, FromPyObject, PyAny, PyErr, PyResult, Python}; use chrono_tz::Tz; use std::str::FromStr; @@ -63,8 +63,8 @@ impl<'py> IntoPyObject<'py> for &Tz { } } -impl FromPyObject<'_> for Tz { - fn extract_bound(ob: &Bound<'_, PyAny>) -> PyResult { +impl FromPyObject<'_, '_> for Tz { + fn extract(ob: Borrowed<'_, '_, PyAny>) -> PyResult { Tz::from_str( &ob.getattr(intern!(ob.py(), "key"))? .extract::()?, @@ -78,6 +78,7 @@ mod tests { use super::*; use crate::prelude::PyAnyMethods; use crate::types::PyTzInfo; + use crate::Bound; use crate::Python; use chrono::{DateTime, Utc}; use chrono_tz::Tz; diff --git a/src/conversions/either.rs b/src/conversions/either.rs index 7e8e3bfc2dc..b926b58ae9d 100644 --- a/src/conversions/either.rs +++ b/src/conversions/either.rs @@ -47,8 +47,8 @@ #[cfg(feature = "experimental-inspect")] use crate::inspect::types::TypeInfo; use crate::{ - exceptions::PyTypeError, types::any::PyAnyMethods, Bound, FromPyObject, IntoPyObject, - IntoPyObjectExt, PyAny, PyErr, PyResult, Python, + exceptions::PyTypeError, Borrowed, Bound, FromPyObject, IntoPyObject, IntoPyObjectExt, PyAny, + PyErr, PyResult, Python, }; use either::Either; @@ -89,13 +89,13 @@ where } #[cfg_attr(docsrs, doc(cfg(feature = "either")))] -impl<'py, L, R> FromPyObject<'py> for Either +impl<'a, 'py, L, R> FromPyObject<'a, 'py> for Either where - L: FromPyObject<'py>, - R: FromPyObject<'py>, + L: FromPyObject<'a, 'py>, + R: FromPyObject<'a, 'py>, { #[inline] - fn extract_bound(obj: &Bound<'py, PyAny>) -> PyResult { + fn extract(obj: Borrowed<'a, 'py, PyAny>) -> PyResult { if let Ok(l) = obj.extract::() { Ok(Either::Left(l)) } else if let Ok(r) = obj.extract::() { diff --git a/src/conversions/hashbrown.rs b/src/conversions/hashbrown.rs index c302d28d2a1..f784e8b64db 100644 --- a/src/conversions/hashbrown.rs +++ b/src/conversions/hashbrown.rs @@ -17,7 +17,7 @@ //! Note that you must use compatible versions of hashbrown and PyO3. //! The required hashbrown version may vary based on the version of PyO3. use crate::{ - conversion::IntoPyObject, + conversion::{FromPyObjectOwned, IntoPyObject}, types::{ any::PyAnyMethods, dict::PyDictMethods, @@ -25,7 +25,7 @@ use crate::{ set::{try_new_from_iter, PySetMethods}, PyDict, PyFrozenSet, PySet, }, - Bound, FromPyObject, PyAny, PyErr, PyResult, Python, + Borrowed, Bound, FromPyObject, PyAny, PyErr, PyResult, Python, }; use std::{cmp, hash}; @@ -67,16 +67,16 @@ where } } -impl<'py, K, V, S> FromPyObject<'py> for hashbrown::HashMap +impl<'py, K, V, S> FromPyObject<'_, 'py> for hashbrown::HashMap where - K: FromPyObject<'py> + cmp::Eq + hash::Hash, - V: FromPyObject<'py>, + K: FromPyObjectOwned<'py> + cmp::Eq + hash::Hash, + V: FromPyObjectOwned<'py>, S: hash::BuildHasher + Default, { - fn extract_bound(ob: &Bound<'py, PyAny>) -> Result { + fn extract(ob: Borrowed<'_, 'py, PyAny>) -> Result { let dict = ob.downcast::()?; let mut ret = hashbrown::HashMap::with_capacity_and_hasher(dict.len(), S::default()); - for (k, v) in dict { + for (k, v) in dict.iter() { ret.insert(k.extract()?, v.extract()?); } Ok(ret) @@ -111,12 +111,12 @@ where } } -impl<'py, K, S> FromPyObject<'py> for hashbrown::HashSet +impl<'py, K, S> FromPyObject<'_, 'py> for hashbrown::HashSet where - K: FromPyObject<'py> + cmp::Eq + hash::Hash, + K: FromPyObjectOwned<'py> + cmp::Eq + hash::Hash, S: hash::BuildHasher + Default, { - fn extract_bound(ob: &Bound<'py, PyAny>) -> PyResult { + fn extract(ob: Borrowed<'_, 'py, PyAny>) -> PyResult { match ob.downcast::() { Ok(set) => set.iter().map(|any| any.extract()).collect(), Err(err) => { diff --git a/src/conversions/indexmap.rs b/src/conversions/indexmap.rs index d6995c14db2..9bf2c6e7af5 100644 --- a/src/conversions/indexmap.rs +++ b/src/conversions/indexmap.rs @@ -87,9 +87,9 @@ //! # if another hash table was used, the order could be random //! ``` -use crate::conversion::IntoPyObject; +use crate::conversion::{FromPyObjectOwned, IntoPyObject}; use crate::types::*; -use crate::{Bound, FromPyObject, PyErr, Python}; +use crate::{Borrowed, Bound, FromPyObject, PyErr, Python}; use std::{cmp, hash}; impl<'py, K, V, H> IntoPyObject<'py> for indexmap::IndexMap @@ -130,16 +130,16 @@ where } } -impl<'py, K, V, S> FromPyObject<'py> for indexmap::IndexMap +impl<'py, K, V, S> FromPyObject<'_, 'py> for indexmap::IndexMap where - K: FromPyObject<'py> + cmp::Eq + hash::Hash, - V: FromPyObject<'py>, + K: FromPyObjectOwned<'py> + cmp::Eq + hash::Hash, + V: FromPyObjectOwned<'py>, S: hash::BuildHasher + Default, { - fn extract_bound(ob: &Bound<'py, PyAny>) -> Result { + fn extract(ob: Borrowed<'_, 'py, PyAny>) -> Result { let dict = ob.downcast::()?; let mut ret = indexmap::IndexMap::with_capacity_and_hasher(dict.len(), S::default()); - for (k, v) in dict { + for (k, v) in dict.iter() { ret.insert(k.extract()?, v.extract()?); } Ok(ret) diff --git a/src/conversions/jiff.rs b/src/conversions/jiff.rs index b92787a70b8..e9dd1e0967b 100644 --- a/src/conversions/jiff.rs +++ b/src/conversions/jiff.rs @@ -52,7 +52,7 @@ use crate::types::{PyAnyMethods, PyNone}; use crate::types::{PyDate, PyDateTime, PyDelta, PyTime, PyTzInfo, PyTzInfoAccess}; #[cfg(not(Py_LIMITED_API))] use crate::types::{PyDateAccess, PyDeltaAccess, PyTimeAccess}; -use crate::{intern, Bound, FromPyObject, IntoPyObject, PyAny, PyErr, PyResult, Python}; +use crate::{intern, Borrowed, Bound, FromPyObject, IntoPyObject, PyAny, PyErr, PyResult, Python}; use jiff::civil::{Date, DateTime, Time}; use jiff::tz::{Offset, TimeZone}; use jiff::{SignedDuration, Span, Timestamp, Zoned}; @@ -123,8 +123,8 @@ impl<'py> IntoPyObject<'py> for &Timestamp { } } -impl<'py> FromPyObject<'py> for Timestamp { - fn extract_bound(ob: &Bound<'py, PyAny>) -> PyResult { +impl<'py> FromPyObject<'_, 'py> for Timestamp { + fn extract(ob: Borrowed<'_, 'py, PyAny>) -> PyResult { let zoned = ob.extract::()?; Ok(zoned.timestamp()) } @@ -155,8 +155,8 @@ impl<'py> IntoPyObject<'py> for &Date { } } -impl<'py> FromPyObject<'py> for Date { - fn extract_bound(ob: &Bound<'py, PyAny>) -> PyResult { +impl<'py> FromPyObject<'_, 'py> for Date { + fn extract(ob: Borrowed<'_, 'py, PyAny>) -> PyResult { let date = ob.downcast::()?; #[cfg(not(Py_LIMITED_API))] @@ -207,11 +207,11 @@ impl<'py> IntoPyObject<'py> for &Time { } } -impl<'py> FromPyObject<'py> for Time { - fn extract_bound(ob: &Bound<'py, PyAny>) -> PyResult { +impl<'py> FromPyObject<'_, 'py> for Time { + fn extract(ob: Borrowed<'_, 'py, PyAny>) -> PyResult { let ob = ob.downcast::()?; - - pytime_to_time(ob) + #[allow(clippy::explicit_auto_deref)] + pytime_to_time(&*ob) } } @@ -235,8 +235,8 @@ impl<'py> IntoPyObject<'py> for &DateTime { } } -impl<'py> FromPyObject<'py> for DateTime { - fn extract_bound(dt: &Bound<'py, PyAny>) -> PyResult { +impl<'py> FromPyObject<'_, 'py> for DateTime { + fn extract(dt: Borrowed<'_, 'py, PyAny>) -> PyResult { let dt = dt.downcast::()?; let has_tzinfo = dt.get_tzinfo().is_some(); @@ -244,7 +244,8 @@ impl<'py> FromPyObject<'py> for DateTime { return Err(PyTypeError::new_err("expected a datetime without tzinfo")); } - Ok(DateTime::from_parts(dt.extract()?, pytime_to_time(dt)?)) + #[allow(clippy::explicit_auto_deref)] + Ok(DateTime::from_parts(dt.extract()?, pytime_to_time(&*dt)?)) } } @@ -283,8 +284,8 @@ impl<'py> IntoPyObject<'py> for &Zoned { } } -impl<'py> FromPyObject<'py> for Zoned { - fn extract_bound(dt: &Bound<'py, PyAny>) -> PyResult { +impl<'py> FromPyObject<'_, 'py> for Zoned { + fn extract(dt: Borrowed<'_, 'py, PyAny>) -> PyResult { let dt = dt.downcast::()?; let tz = dt @@ -295,7 +296,8 @@ impl<'py> FromPyObject<'py> for Zoned { "expected a datetime with non-None tzinfo", )) })?; - let datetime = DateTime::from_parts(dt.extract()?, pytime_to_time(dt)?); + #[allow(clippy::explicit_auto_deref)] + let datetime = DateTime::from_parts(dt.extract()?, pytime_to_time(&*dt)?); let zoned = tz.into_ambiguous_zoned(datetime); #[cfg(not(Py_LIMITED_API))] @@ -341,8 +343,8 @@ impl<'py> IntoPyObject<'py> for &TimeZone { } } -impl<'py> FromPyObject<'py> for TimeZone { - fn extract_bound(ob: &Bound<'py, PyAny>) -> PyResult { +impl<'py> FromPyObject<'_, 'py> for TimeZone { + fn extract(ob: Borrowed<'_, 'py, PyAny>) -> PyResult { let ob = ob.downcast::()?; let attr = intern!(ob.py(), "key"); @@ -378,8 +380,8 @@ impl<'py> IntoPyObject<'py> for Offset { } } -impl<'py> FromPyObject<'py> for Offset { - fn extract_bound(ob: &Bound<'py, PyAny>) -> PyResult { +impl<'py> FromPyObject<'_, 'py> for Offset { + fn extract(ob: Borrowed<'_, 'py, PyAny>) -> PyResult { let py = ob.py(); let ob = ob.downcast::()?; @@ -427,8 +429,8 @@ impl<'py> IntoPyObject<'py> for SignedDuration { } } -impl<'py> FromPyObject<'py> for SignedDuration { - fn extract_bound(ob: &Bound<'py, PyAny>) -> PyResult { +impl<'py> FromPyObject<'_, 'py> for SignedDuration { + fn extract(ob: Borrowed<'_, 'py, PyAny>) -> PyResult { let delta = ob.downcast::()?; #[cfg(not(Py_LIMITED_API))] @@ -452,8 +454,8 @@ impl<'py> FromPyObject<'py> for SignedDuration { } } -impl<'py> FromPyObject<'py> for Span { - fn extract_bound(ob: &Bound<'py, PyAny>) -> PyResult { +impl<'py> FromPyObject<'_, 'py> for Span { + fn extract(ob: Borrowed<'_, 'py, PyAny>) -> PyResult { let duration = ob.extract::()?; Ok(duration.try_into()?) } diff --git a/src/conversions/num_bigint.rs b/src/conversions/num_bigint.rs index ebc4058b1a6..a718eec502b 100644 --- a/src/conversions/num_bigint.rs +++ b/src/conversions/num_bigint.rs @@ -52,9 +52,8 @@ use crate::types::{bytes::PyBytesMethods, PyBytes}; use crate::{ conversion::IntoPyObject, ffi, - instance::Bound, types::{any::PyAnyMethods, PyInt}, - FromPyObject, Py, PyAny, PyErr, PyResult, Python, + Borrowed, Bound, FromPyObject, Py, PyAny, PyErr, PyResult, Python, }; use num_bigint::{BigInt, BigUint}; @@ -125,8 +124,8 @@ bigint_conversion!(BigUint, false, BigUint::to_bytes_le); bigint_conversion!(BigInt, true, BigInt::to_signed_bytes_le); #[cfg_attr(docsrs, doc(cfg(feature = "num-bigint")))] -impl<'py> FromPyObject<'py> for BigInt { - fn extract_bound(ob: &Bound<'py, PyAny>) -> PyResult { +impl<'py> FromPyObject<'_, 'py> for BigInt { + fn extract(ob: Borrowed<'_, 'py, PyAny>) -> PyResult { let py = ob.py(); // fast path - checking for subclass of `int` just checks a bit in the type object let num_owned: Py; @@ -134,11 +133,11 @@ impl<'py> FromPyObject<'py> for BigInt { long } else { num_owned = unsafe { Py::from_owned_ptr_or_err(py, ffi::PyNumber_Index(ob.as_ptr()))? }; - num_owned.bind(py) + num_owned.bind_borrowed(py) }; #[cfg(not(Py_LIMITED_API))] { - let mut buffer = int_to_u32_vec::(num)?; + let mut buffer = int_to_u32_vec::(&num)?; let sign = if buffer.last().copied().map_or(false, |last| last >> 31 != 0) { // BigInt::new takes an unsigned array, so need to convert from two's complement // flip all bits, 'subtract' 1 (by adding one to the unsigned array) @@ -162,19 +161,19 @@ impl<'py> FromPyObject<'py> for BigInt { } #[cfg(Py_LIMITED_API)] { - let n_bits = int_n_bits(num)?; + let n_bits = int_n_bits(&num)?; if n_bits == 0 { return Ok(BigInt::from(0isize)); } - let bytes = int_to_py_bytes(num, (n_bits + 8) / 8, true)?; + let bytes = int_to_py_bytes(&num, (n_bits + 8) / 8, true)?; Ok(BigInt::from_signed_bytes_le(bytes.as_bytes())) } } } #[cfg_attr(docsrs, doc(cfg(feature = "num-bigint")))] -impl<'py> FromPyObject<'py> for BigUint { - fn extract_bound(ob: &Bound<'py, PyAny>) -> PyResult { +impl<'py> FromPyObject<'_, 'py> for BigUint { + fn extract(ob: Borrowed<'_, 'py, PyAny>) -> PyResult { let py = ob.py(); // fast path - checking for subclass of `int` just checks a bit in the type object let num_owned: Py; @@ -182,20 +181,20 @@ impl<'py> FromPyObject<'py> for BigUint { long } else { num_owned = unsafe { Py::from_owned_ptr_or_err(py, ffi::PyNumber_Index(ob.as_ptr()))? }; - num_owned.bind(py) + num_owned.bind_borrowed(py) }; #[cfg(not(Py_LIMITED_API))] { - let buffer = int_to_u32_vec::(num)?; + let buffer = int_to_u32_vec::(&num)?; Ok(BigUint::new(buffer)) } #[cfg(Py_LIMITED_API)] { - let n_bits = int_n_bits(num)?; + let n_bits = int_n_bits(&num)?; if n_bits == 0 { return Ok(BigUint::from(0usize)); } - let bytes = int_to_py_bytes(num, (n_bits + 7) / 8, false)?; + let bytes = int_to_py_bytes(&num, (n_bits + 7) / 8, false)?; Ok(BigUint::from_bytes_le(bytes.as_bytes())) } } @@ -280,6 +279,7 @@ fn int_to_py_bytes<'py>( is_signed: bool, ) -> PyResult> { use crate::intern; + use crate::types::{PyAnyMethods, PyDictMethods}; let py = long.py(); let kwargs = if is_signed { let kwargs = crate::types::PyDict::new(py); @@ -313,6 +313,7 @@ fn int_n_bits(long: &Bound<'_, PyInt>) -> PyResult { #[cfg(Py_LIMITED_API)] { // slow path + use crate::types::PyAnyMethods; long.call_method0(crate::intern!(py, "bit_length")) .and_then(|any| any.extract()) } @@ -322,7 +323,7 @@ fn int_n_bits(long: &Bound<'_, PyInt>) -> PyResult { mod tests { use super::*; use crate::tests::common::generate_unique_module_name; - use crate::types::{PyDict, PyModule}; + use crate::types::{PyAnyMethods, PyDict, PyModule}; use indoc::indoc; use pyo3_ffi::c_str; diff --git a/src/conversions/num_complex.rs b/src/conversions/num_complex.rs index e9fb1096ff5..ace581fbdba 100644 --- a/src/conversions/num_complex.rs +++ b/src/conversions/num_complex.rs @@ -97,7 +97,7 @@ use crate::{ ffi, ffi_ptr_ext::FfiPtrExt, types::{any::PyAnyMethods, PyComplex}, - Bound, FromPyObject, PyAny, PyErr, PyResult, Python, + Borrowed, Bound, FromPyObject, PyAny, PyErr, PyResult, Python, }; use num_complex::Complex; use std::os::raw::c_double; @@ -148,8 +148,8 @@ macro_rules! complex_conversion { } #[cfg_attr(docsrs, doc(cfg(feature = "num-complex")))] - impl FromPyObject<'_> for Complex<$float> { - fn extract_bound(obj: &Bound<'_, PyAny>) -> PyResult> { + impl FromPyObject<'_, '_> for Complex<$float> { + fn extract(obj: Borrowed<'_, '_, PyAny>) -> PyResult> { #[cfg(not(any(Py_LIMITED_API, PyPy)))] unsafe { let val = ffi::PyComplex_AsCComplex(obj.as_ptr()); @@ -170,7 +170,7 @@ macro_rules! complex_conversion { obj.lookup_special(crate::intern!(obj.py(), "__complex__"))? { complex = method.call0()?; - &complex + complex.as_borrowed() } else { // `obj` might still implement `__float__` or `__index__`, which will be // handled by `PyComplex_{Real,Imag}AsDouble`, including propagating any diff --git a/src/conversions/num_rational.rs b/src/conversions/num_rational.rs index dfdd80cf54f..a1994d08c94 100644 --- a/src/conversions/num_rational.rs +++ b/src/conversions/num_rational.rs @@ -48,7 +48,7 @@ use crate::ffi; use crate::sync::GILOnceCell; use crate::types::any::PyAnyMethods; use crate::types::PyType; -use crate::{Bound, FromPyObject, Py, PyAny, PyErr, PyResult, Python}; +use crate::{Borrowed, Bound, FromPyObject, Py, PyAny, PyErr, PyResult, Python}; #[cfg(feature = "num-bigint")] use num_bigint::BigInt; @@ -62,8 +62,8 @@ fn get_fraction_cls(py: Python<'_>) -> PyResult<&Bound<'_, PyType>> { macro_rules! rational_conversion { ($int: ty) => { - impl<'py> FromPyObject<'py> for Ratio<$int> { - fn extract_bound(obj: &Bound<'py, PyAny>) -> PyResult { + impl<'py> FromPyObject<'_, 'py> for Ratio<$int> { + fn extract(obj: Borrowed<'_, 'py, PyAny>) -> PyResult { let py = obj.py(); let py_numerator_obj = obj.getattr(crate::intern!(py, "numerator"))?; let py_denominator_obj = obj.getattr(crate::intern!(py, "denominator"))?; diff --git a/src/conversions/rust_decimal.rs b/src/conversions/rust_decimal.rs index 392971a0b4b..405ba09bff1 100644 --- a/src/conversions/rust_decimal.rs +++ b/src/conversions/rust_decimal.rs @@ -55,12 +55,12 @@ use crate::sync::GILOnceCell; use crate::types::any::PyAnyMethods; use crate::types::string::PyStringMethods; use crate::types::PyType; -use crate::{Bound, FromPyObject, Py, PyAny, PyErr, PyResult, Python}; +use crate::{Borrowed, Bound, FromPyObject, Py, PyAny, PyErr, PyResult, Python}; use rust_decimal::Decimal; use std::str::FromStr; -impl FromPyObject<'_> for Decimal { - fn extract_bound(obj: &Bound<'_, PyAny>) -> PyResult { +impl FromPyObject<'_, '_> for Decimal { + fn extract(obj: Borrowed<'_, '_, PyAny>) -> PyResult { // use the string representation to not be lossy if let Ok(val) = obj.extract() { Ok(Decimal::new(val, 0)) diff --git a/src/conversions/smallvec.rs b/src/conversions/smallvec.rs index d4229bc2865..ccebbc1353d 100644 --- a/src/conversions/smallvec.rs +++ b/src/conversions/smallvec.rs @@ -15,14 +15,15 @@ //! //! Note that you must use compatible versions of smallvec and PyO3. //! The required smallvec version may vary based on the version of PyO3. -use crate::conversion::IntoPyObject; +use crate::conversion::{FromPyObjectOwned, IntoPyObject}; use crate::exceptions::PyTypeError; #[cfg(feature = "experimental-inspect")] use crate::inspect::types::TypeInfo; use crate::types::any::PyAnyMethods; use crate::types::{PySequence, PyString}; -use crate::PyErr; -use crate::{err::DowncastError, ffi, Bound, FromPyObject, PyAny, PyResult, Python}; +use crate::{ + err::DowncastError, ffi, Borrowed, Bound, FromPyObject, PyAny, PyErr, PyResult, Python, +}; use smallvec::{Array, SmallVec}; impl<'py, A> IntoPyObject<'py> for SmallVec @@ -70,12 +71,12 @@ where } } -impl<'py, A> FromPyObject<'py> for SmallVec +impl<'py, A> FromPyObject<'_, 'py> for SmallVec where A: Array, - A::Item: FromPyObject<'py>, + A::Item: FromPyObjectOwned<'py>, { - fn extract_bound(obj: &Bound<'py, PyAny>) -> PyResult { + fn extract(obj: Borrowed<'_, 'py, PyAny>) -> PyResult { if obj.is_instance_of::() { return Err(PyTypeError::new_err("Can't extract `str` to `SmallVec`")); } @@ -88,10 +89,10 @@ where } } -fn extract_sequence<'py, A>(obj: &Bound<'py, PyAny>) -> PyResult> +fn extract_sequence<'py, A>(obj: Borrowed<'_, 'py, PyAny>) -> PyResult> where A: Array, - A::Item: FromPyObject<'py>, + A::Item: FromPyObjectOwned<'py>, { // Types that pass `PySequence_Check` usually implement enough of the sequence protocol // to support this function and if not, we will only fail extraction safely. @@ -99,7 +100,7 @@ where if ffi::PySequence_Check(obj.as_ptr()) != 0 { obj.downcast_unchecked::() } else { - return Err(DowncastError::new(obj, "Sequence").into()); + return Err(DowncastError::new_from_borrowed(obj, "Sequence").into()); } }; diff --git a/src/conversions/std/array.rs b/src/conversions/std/array.rs index 36db5ec640f..452e8d75caa 100644 --- a/src/conversions/std/array.rs +++ b/src/conversions/std/array.rs @@ -1,9 +1,8 @@ -use crate::conversion::IntoPyObject; -use crate::instance::Bound; +use crate::conversion::{FromPyObjectOwned, IntoPyObject}; use crate::types::any::PyAnyMethods; use crate::types::PySequence; use crate::{err::DowncastError, ffi, FromPyObject, PyAny, PyResult, Python}; -use crate::{exceptions, PyErr}; +use crate::{exceptions, Borrowed, Bound, PyErr}; impl<'py, T, const N: usize> IntoPyObject<'py> for [T; N] where @@ -37,18 +36,18 @@ where } } -impl<'py, T, const N: usize> FromPyObject<'py> for [T; N] +impl<'py, T, const N: usize> FromPyObject<'_, 'py> for [T; N] where - T: FromPyObject<'py>, + T: FromPyObjectOwned<'py>, { - fn extract_bound(obj: &Bound<'py, PyAny>) -> PyResult { + fn extract(obj: Borrowed<'_, 'py, PyAny>) -> PyResult { create_array_from_obj(obj) } } -fn create_array_from_obj<'py, T, const N: usize>(obj: &Bound<'py, PyAny>) -> PyResult<[T; N]> +fn create_array_from_obj<'py, T, const N: usize>(obj: Borrowed<'_, 'py, PyAny>) -> PyResult<[T; N]> where - T: FromPyObject<'py>, + T: FromPyObjectOwned<'py>, { // Types that pass `PySequence_Check` usually implement enough of the sequence protocol // to support this function and if not, we will only fail extraction safely. @@ -56,7 +55,7 @@ where if ffi::PySequence_Check(obj.as_ptr()) != 0 { obj.downcast_unchecked::() } else { - return Err(DowncastError::new(obj, "Sequence").into()); + return Err(DowncastError::new_from_borrowed(obj, "Sequence").into()); } }; let seq_len = seq.len()?; diff --git a/src/conversions/std/cell.rs b/src/conversions/std/cell.rs index 983a0350228..12c08918e76 100644 --- a/src/conversions/std/cell.rs +++ b/src/conversions/std/cell.rs @@ -1,9 +1,6 @@ use std::cell::Cell; -use crate::{ - conversion::IntoPyObject, types::any::PyAnyMethods, Bound, FromPyObject, PyAny, PyResult, - Python, -}; +use crate::{conversion::IntoPyObject, Borrowed, FromPyObject, PyAny, PyResult, Python}; impl<'py, T: Copy + IntoPyObject<'py>> IntoPyObject<'py> for Cell { type Target = T::Target; @@ -27,8 +24,8 @@ impl<'py, T: Copy + IntoPyObject<'py>> IntoPyObject<'py> for &Cell { } } -impl<'py, T: FromPyObject<'py>> FromPyObject<'py> for Cell { - fn extract_bound(ob: &Bound<'py, PyAny>) -> PyResult { +impl<'a, 'py, T: FromPyObject<'a, 'py>> FromPyObject<'a, 'py> for Cell { + fn extract(ob: Borrowed<'a, 'py, PyAny>) -> PyResult { ob.extract().map(Cell::new) } } diff --git a/src/conversions/std/ipaddr.rs b/src/conversions/std/ipaddr.rs index 76f6a6927c2..16aceaefa6d 100644 --- a/src/conversions/std/ipaddr.rs +++ b/src/conversions/std/ipaddr.rs @@ -2,15 +2,14 @@ use std::net::{IpAddr, Ipv4Addr, Ipv6Addr}; use crate::conversion::IntoPyObject; use crate::exceptions::PyValueError; -use crate::instance::Bound; use crate::sync::GILOnceCell; use crate::types::any::PyAnyMethods; use crate::types::string::PyStringMethods; use crate::types::PyType; -use crate::{intern, FromPyObject, Py, PyAny, PyErr, PyResult, Python}; +use crate::{intern, Borrowed, Bound, FromPyObject, Py, PyAny, PyErr, PyResult, Python}; -impl FromPyObject<'_> for IpAddr { - fn extract_bound(obj: &Bound<'_, PyAny>) -> PyResult { +impl FromPyObject<'_, '_> for IpAddr { + fn extract(obj: Borrowed<'_, '_, PyAny>) -> PyResult { match obj.getattr(intern!(obj.py(), "packed")) { Ok(packed) => { if let Ok(packed) = packed.extract::<[u8; 4]>() { diff --git a/src/conversions/std/map.rs b/src/conversions/std/map.rs index b472957d1b1..01e7f29fb39 100644 --- a/src/conversions/std/map.rs +++ b/src/conversions/std/map.rs @@ -3,10 +3,10 @@ use std::{cmp, collections, hash}; #[cfg(feature = "experimental-inspect")] use crate::inspect::types::TypeInfo; use crate::{ - conversion::IntoPyObject, + conversion::{FromPyObjectOwned, IntoPyObject}, instance::Bound, types::{any::PyAnyMethods, dict::PyDictMethods, PyDict}, - FromPyObject, PyAny, PyErr, Python, + Borrowed, FromPyObject, PyAny, PyErr, Python, }; impl<'py, K, V, H> IntoPyObject<'py> for collections::HashMap @@ -107,16 +107,16 @@ where } } -impl<'py, K, V, S> FromPyObject<'py> for collections::HashMap +impl<'py, K, V, S> FromPyObject<'_, 'py> for collections::HashMap where - K: FromPyObject<'py> + cmp::Eq + hash::Hash, - V: FromPyObject<'py>, + K: FromPyObjectOwned<'py> + cmp::Eq + hash::Hash, + V: FromPyObjectOwned<'py>, S: hash::BuildHasher + Default, { - fn extract_bound(ob: &Bound<'py, PyAny>) -> Result { + fn extract(ob: Borrowed<'_, 'py, PyAny>) -> Result { let dict = ob.downcast::()?; let mut ret = collections::HashMap::with_capacity_and_hasher(dict.len(), S::default()); - for (k, v) in dict { + for (k, v) in dict.iter() { ret.insert(k.extract()?, v.extract()?); } Ok(ret) @@ -128,15 +128,15 @@ where } } -impl<'py, K, V> FromPyObject<'py> for collections::BTreeMap +impl<'py, K, V> FromPyObject<'_, 'py> for collections::BTreeMap where - K: FromPyObject<'py> + cmp::Ord, - V: FromPyObject<'py>, + K: FromPyObjectOwned<'py> + cmp::Ord, + V: FromPyObjectOwned<'py>, { - fn extract_bound(ob: &Bound<'py, PyAny>) -> Result { + fn extract(ob: Borrowed<'_, 'py, PyAny>) -> Result { let dict = ob.downcast::()?; let mut ret = collections::BTreeMap::new(); - for (k, v) in dict { + for (k, v) in dict.iter() { ret.insert(k.extract()?, v.extract()?); } Ok(ret) diff --git a/src/conversions/std/num.rs b/src/conversions/std/num.rs index 40073d8af69..af53870bd4c 100644 --- a/src/conversions/std/num.rs +++ b/src/conversions/std/num.rs @@ -5,7 +5,7 @@ use crate::ffi_ptr_ext::FfiPtrExt; use crate::inspect::types::TypeInfo; use crate::types::any::PyAnyMethods; use crate::types::{PyBytes, PyInt}; -use crate::{exceptions, ffi, Bound, FromPyObject, PyAny, PyErr, PyResult, Python}; +use crate::{exceptions, ffi, Borrowed, Bound, FromPyObject, PyAny, PyErr, PyResult, Python}; use std::convert::Infallible; use std::num::{ NonZeroI128, NonZeroI16, NonZeroI32, NonZeroI64, NonZeroI8, NonZeroIsize, NonZeroU128, @@ -45,8 +45,8 @@ macro_rules! int_fits_larger_int { } } - impl FromPyObject<'_> for $rust_type { - fn extract_bound(obj: &Bound<'_, PyAny>) -> PyResult { + impl FromPyObject<'_, '_> for $rust_type { + fn extract(obj: Borrowed<'_, '_, PyAny>) -> PyResult { let val: $larger_type = obj.extract()?; <$rust_type>::try_from(val) .map_err(|e| exceptions::PyOverflowError::new_err(e.to_string())) @@ -114,14 +114,9 @@ macro_rules! int_convert_u64_or_i64 { fn into_pyobject(self, py: Python<'py>) -> Result { (*self).into_pyobject(py) } - - #[cfg(feature = "experimental-inspect")] - fn type_output() -> TypeInfo { - TypeInfo::builtin("int") - } } - impl FromPyObject<'_> for $rust_type { - fn extract_bound(obj: &Bound<'_, PyAny>) -> PyResult<$rust_type> { + impl FromPyObject<'_, '_> for $rust_type { + fn extract(obj: Borrowed<'_, '_, PyAny>) -> PyResult<$rust_type> { extract_int!(obj, !0, $pylong_as_ll_or_ull, $force_index_call) } @@ -170,8 +165,8 @@ macro_rules! int_fits_c_long { } } - impl<'py> FromPyObject<'py> for $rust_type { - fn extract_bound(obj: &Bound<'_, PyAny>) -> PyResult { + impl<'py> FromPyObject<'_, 'py> for $rust_type { + fn extract(obj: Borrowed<'_, 'py, PyAny>) -> PyResult { let val: c_long = extract_int!(obj, -1, ffi::PyLong_AsLong)?; <$rust_type>::try_from(val) .map_err(|e| exceptions::PyOverflowError::new_err(e.to_string())) @@ -244,8 +239,8 @@ impl<'py> IntoPyObject<'py> for &'_ u8 { } } -impl FromPyObject<'_> for u8 { - fn extract_bound(obj: &Bound<'_, PyAny>) -> PyResult { +impl<'py> FromPyObject<'_, 'py> for u8 { + fn extract(obj: Borrowed<'_, 'py, PyAny>) -> PyResult { let val: c_long = extract_int!(obj, -1, ffi::PyLong_AsLong)?; u8::try_from(val).map_err(|e| exceptions::PyOverflowError::new_err(e.to_string())) } @@ -366,8 +361,8 @@ mod fast_128bit_int_conversion { } } - impl FromPyObject<'_> for $rust_type { - fn extract_bound(ob: &Bound<'_, PyAny>) -> PyResult<$rust_type> { + impl FromPyObject<'_, '_> for $rust_type { + fn extract(ob: Borrowed<'_, '_, PyAny>) -> PyResult<$rust_type> { let num = unsafe { ffi::PyNumber_Index(ob.as_ptr()).assume_owned_or_err(ob.py())? }; let mut buffer = [0u8; std::mem::size_of::<$rust_type>()]; @@ -475,8 +470,8 @@ mod slow_128bit_int_conversion { } } - impl FromPyObject<'_> for $rust_type { - fn extract_bound(ob: &Bound<'_, PyAny>) -> PyResult<$rust_type> { + impl FromPyObject<'_, '_> for $rust_type { + fn extract(ob: Borrowed<'_, '_, PyAny>) -> PyResult<$rust_type> { let py = ob.py(); unsafe { let lower = err_if_invalid_value( @@ -554,8 +549,8 @@ macro_rules! nonzero_int_impl { } } - impl FromPyObject<'_> for $nonzero_type { - fn extract_bound(obj: &Bound<'_, PyAny>) -> PyResult { + impl FromPyObject<'_, '_> for $nonzero_type { + fn extract(obj: Borrowed<'_, '_, PyAny>) -> PyResult { let val: $primitive_type = obj.extract()?; <$nonzero_type>::try_from(val) .map_err(|_| exceptions::PyValueError::new_err("invalid zero value")) @@ -585,6 +580,7 @@ nonzero_int_impl!(NonZeroUsize, usize); #[cfg(test)] mod test_128bit_integers { use super::*; + use crate::types::PyAnyMethods; #[cfg(not(target_arch = "wasm32"))] use crate::types::PyDict; diff --git a/src/conversions/std/option.rs b/src/conversions/std/option.rs index ae0ec441c61..04369bf79f9 100644 --- a/src/conversions/std/option.rs +++ b/src/conversions/std/option.rs @@ -1,7 +1,8 @@ use crate::{ - conversion::IntoPyObject, types::any::PyAnyMethods, Bound, BoundObject, FromPyObject, PyAny, - PyResult, Python, + conversion::IntoPyObject, types::any::PyAnyMethods, BoundObject, FromPyObject, PyAny, PyResult, + Python, }; +use crate::{Borrowed, Bound}; impl<'py, T> IntoPyObject<'py> for Option where @@ -37,11 +38,11 @@ where } } -impl<'py, T> FromPyObject<'py> for Option +impl<'a, 'py, T> FromPyObject<'a, 'py> for Option where - T: FromPyObject<'py>, + T: FromPyObject<'a, 'py>, { - fn extract_bound(obj: &Bound<'py, PyAny>) -> PyResult { + fn extract(obj: Borrowed<'a, 'py, PyAny>) -> PyResult { if obj.is_none() { Ok(None) } else { diff --git a/src/conversions/std/osstr.rs b/src/conversions/std/osstr.rs index 39fa56f373d..54585d20dcb 100644 --- a/src/conversions/std/osstr.rs +++ b/src/conversions/std/osstr.rs @@ -3,7 +3,7 @@ use crate::ffi_ptr_ext::FfiPtrExt; use crate::instance::Bound; use crate::types::any::PyAnyMethods; use crate::types::PyString; -use crate::{ffi, FromPyObject, PyAny, PyResult, Python}; +use crate::{ffi, Borrowed, FromPyObject, PyAny, PyResult, Python}; use std::borrow::Cow; use std::convert::Infallible; use std::ffi::{OsStr, OsString}; @@ -71,8 +71,8 @@ impl<'py> IntoPyObject<'py> for &&OsStr { // There's no FromPyObject implementation for &OsStr because albeit possible on Unix, this would // be impossible to implement on Windows. Hence it's omitted entirely -impl FromPyObject<'_> for OsString { - fn extract_bound(ob: &Bound<'_, PyAny>) -> PyResult { +impl FromPyObject<'_, '_> for OsString { + fn extract(ob: Borrowed<'_, '_, PyAny>) -> PyResult { let pystring = ob.downcast::()?; #[cfg(not(windows))] @@ -98,8 +98,6 @@ impl FromPyObject<'_> for OsString { #[cfg(windows)] { - use crate::types::string::PyStringMethods; - // Take the quick and easy shortcut if UTF-8 if let Ok(utf8_string) = pystring.to_cow() { return Ok(utf8_string.into_owned().into()); @@ -170,7 +168,7 @@ impl<'py> IntoPyObject<'py> for &OsString { #[cfg(test)] mod tests { - use crate::types::{PyAnyMethods, PyString, PyStringMethods}; + use crate::types::{PyString, PyStringMethods}; use crate::{BoundObject, IntoPyObject, Python}; use std::fmt::Debug; use std::{ @@ -182,6 +180,7 @@ mod tests { #[cfg(not(windows))] fn test_non_utf8_conversion() { Python::with_gil(|py| { + use crate::types::PyAnyMethods; #[cfg(not(target_os = "wasi"))] use std::os::unix::ffi::OsStrExt; #[cfg(target_os = "wasi")] diff --git a/src/conversions/std/path.rs b/src/conversions/std/path.rs index 3eb074d04d2..8131ff75734 100644 --- a/src/conversions/std/path.rs +++ b/src/conversions/std/path.rs @@ -1,17 +1,16 @@ use crate::conversion::IntoPyObject; use crate::ffi_ptr_ext::FfiPtrExt; -use crate::instance::Bound; use crate::sync::GILOnceCell; use crate::types::any::PyAnyMethods; -use crate::{ffi, FromPyObject, PyAny, PyErr, PyObject, PyResult, Python}; +use crate::{ffi, Borrowed, Bound, FromPyObject, PyAny, PyErr, PyObject, PyResult, Python}; use std::borrow::Cow; use std::ffi::OsString; use std::path::{Path, PathBuf}; // See osstr.rs for why there's no FromPyObject impl for &Path -impl FromPyObject<'_> for PathBuf { - fn extract_bound(ob: &Bound<'_, PyAny>) -> PyResult { +impl FromPyObject<'_, '_> for PathBuf { + fn extract(ob: Borrowed<'_, '_, PyAny>) -> PyResult { // We use os.fspath to get the underlying path as bytes or str let path = unsafe { ffi::PyOS_FSPath(ob.as_ptr()).assume_owned_or_err(ob.py())? }; Ok(path.extract::()?.into()) @@ -99,6 +98,7 @@ mod tests { #[cfg(not(windows))] fn test_non_utf8_conversion() { Python::with_gil(|py| { + use crate::types::PyAnyMethods; use std::ffi::OsStr; #[cfg(not(target_os = "wasi"))] use std::os::unix::ffi::OsStrExt; diff --git a/src/conversions/std/set.rs b/src/conversions/std/set.rs index ebb737f72c6..8c5cde6b6cc 100644 --- a/src/conversions/std/set.rs +++ b/src/conversions/std/set.rs @@ -3,15 +3,14 @@ use std::{cmp, collections, hash}; #[cfg(feature = "experimental-inspect")] use crate::inspect::types::TypeInfo; use crate::{ - conversion::IntoPyObject, - instance::Bound, + conversion::{FromPyObjectOwned, IntoPyObject}, types::{ any::PyAnyMethods, frozenset::PyFrozenSetMethods, set::{try_new_from_iter, PySetMethods}, PyFrozenSet, PySet, }, - FromPyObject, PyAny, PyErr, PyResult, Python, + Borrowed, Bound, FromPyObject, PyAny, PyErr, PyResult, Python, }; impl<'py, K, S> IntoPyObject<'py> for collections::HashSet @@ -53,12 +52,12 @@ where } } -impl<'py, K, S> FromPyObject<'py> for collections::HashSet +impl<'py, K, S> FromPyObject<'_, 'py> for collections::HashSet where - K: FromPyObject<'py> + cmp::Eq + hash::Hash, + K: FromPyObjectOwned<'py> + cmp::Eq + hash::Hash, S: hash::BuildHasher + Default, { - fn extract_bound(ob: &Bound<'py, PyAny>) -> PyResult { + fn extract(ob: Borrowed<'_, 'py, PyAny>) -> PyResult { match ob.downcast::() { Ok(set) => set.iter().map(|any| any.extract()).collect(), Err(err) => { @@ -114,11 +113,11 @@ where } } -impl<'py, K> FromPyObject<'py> for collections::BTreeSet +impl<'py, K> FromPyObject<'_, 'py> for collections::BTreeSet where - K: FromPyObject<'py> + cmp::Ord, + K: FromPyObjectOwned<'py> + cmp::Ord, { - fn extract_bound(ob: &Bound<'py, PyAny>) -> PyResult { + fn extract(ob: Borrowed<'_, 'py, PyAny>) -> PyResult { match ob.downcast::() { Ok(set) => set.iter().map(|any| any.extract()).collect(), Err(err) => { diff --git a/src/conversions/std/slice.rs b/src/conversions/std/slice.rs index 5bfa5a3d48d..6ed11dd8bf9 100644 --- a/src/conversions/std/slice.rs +++ b/src/conversions/std/slice.rs @@ -35,8 +35,8 @@ where } } -impl<'a> crate::conversion::FromPyObjectBound<'a, '_> for &'a [u8] { - fn from_py_object_bound(obj: crate::Borrowed<'a, '_, PyAny>) -> PyResult { +impl<'a> crate::conversion::FromPyObject<'a, '_> for &'a [u8] { + fn extract(obj: crate::Borrowed<'a, '_, PyAny>) -> PyResult { Ok(obj.downcast::()?.as_bytes()) } @@ -51,8 +51,8 @@ impl<'a> crate::conversion::FromPyObjectBound<'a, '_> for &'a [u8] { /// If the source object is a `bytes` object, the `Cow` will be borrowed and /// pointing into the source object, and no copying or heap allocations will happen. /// If it is a `bytearray`, its contents will be copied to an owned `Cow`. -impl<'a> crate::conversion::FromPyObjectBound<'a, '_> for Cow<'a, [u8]> { - fn from_py_object_bound(ob: crate::Borrowed<'a, '_, PyAny>) -> PyResult { +impl<'a> crate::conversion::FromPyObject<'a, '_> for Cow<'a, [u8]> { + fn extract(ob: crate::Borrowed<'a, '_, PyAny>) -> PyResult { if let Ok(bytes) = ob.downcast::() { return Ok(Cow::Borrowed(bytes.as_bytes())); } diff --git a/src/conversions/std/string.rs b/src/conversions/std/string.rs index 8936bd35000..8a817418122 100644 --- a/src/conversions/std/string.rs +++ b/src/conversions/std/string.rs @@ -3,10 +3,8 @@ use std::{borrow::Cow, convert::Infallible}; #[cfg(feature = "experimental-inspect")] use crate::inspect::types::TypeInfo; use crate::{ - conversion::IntoPyObject, - instance::Bound, - types::{any::PyAnyMethods, string::PyStringMethods, PyString}, - FromPyObject, PyAny, PyResult, Python, + conversion::IntoPyObject, instance::Bound, types::PyString, Borrowed, FromPyObject, PyAny, + PyResult, Python, }; impl<'py> IntoPyObject<'py> for &str { @@ -137,8 +135,8 @@ impl<'py> IntoPyObject<'py> for &String { } #[cfg(any(Py_3_10, not(Py_LIMITED_API)))] -impl<'a> crate::conversion::FromPyObjectBound<'a, '_> for &'a str { - fn from_py_object_bound(ob: crate::Borrowed<'a, '_, PyAny>) -> PyResult { +impl<'a> crate::conversion::FromPyObject<'a, '_> for &'a str { + fn extract(ob: crate::Borrowed<'a, '_, PyAny>) -> PyResult { ob.downcast::()?.to_str() } @@ -148,8 +146,8 @@ impl<'a> crate::conversion::FromPyObjectBound<'a, '_> for &'a str { } } -impl<'a> crate::conversion::FromPyObjectBound<'a, '_> for Cow<'a, str> { - fn from_py_object_bound(ob: crate::Borrowed<'a, '_, PyAny>) -> PyResult { +impl<'a> crate::conversion::FromPyObject<'a, '_> for Cow<'a, str> { + fn extract(ob: crate::Borrowed<'a, '_, PyAny>) -> PyResult { ob.downcast::()?.to_cow() } @@ -161,8 +159,8 @@ impl<'a> crate::conversion::FromPyObjectBound<'a, '_> for Cow<'a, str> { /// Allows extracting strings from Python objects. /// Accepts Python `str` and `unicode` objects. -impl FromPyObject<'_> for String { - fn extract_bound(obj: &Bound<'_, PyAny>) -> PyResult { +impl FromPyObject<'_, '_> for String { + fn extract(obj: Borrowed<'_, '_, PyAny>) -> PyResult { obj.downcast::()?.to_cow().map(Cow::into_owned) } @@ -172,8 +170,8 @@ impl FromPyObject<'_> for String { } } -impl FromPyObject<'_> for char { - fn extract_bound(obj: &Bound<'_, PyAny>) -> PyResult { +impl FromPyObject<'_, '_> for char { + fn extract(obj: Borrowed<'_, '_, PyAny>) -> PyResult { let s = obj.downcast::()?.to_cow()?; let mut iter = s.chars(); if let (Some(ch), None) = (iter.next(), iter.next()) { diff --git a/src/conversions/std/time.rs b/src/conversions/std/time.rs index 53e3bf1a641..ab0f49523db 100644 --- a/src/conversions/std/time.rs +++ b/src/conversions/std/time.rs @@ -12,8 +12,8 @@ use std::time::{Duration, SystemTime, UNIX_EPOCH}; const SECONDS_PER_DAY: u64 = 24 * 60 * 60; -impl FromPyObject<'_> for Duration { - fn extract_bound(obj: &Bound<'_, PyAny>) -> PyResult { +impl FromPyObject<'_, '_> for Duration { + fn extract(obj: Borrowed<'_, '_, PyAny>) -> PyResult { let delta = obj.downcast::()?; #[cfg(not(Py_LIMITED_API))] let (days, seconds, microseconds) = { @@ -87,8 +87,8 @@ impl<'py> IntoPyObject<'py> for &Duration { // // TODO: it might be nice to investigate using timestamps anyway, at least when the datetime is a safe range. -impl FromPyObject<'_> for SystemTime { - fn extract_bound(obj: &Bound<'_, PyAny>) -> PyResult { +impl FromPyObject<'_, '_> for SystemTime { + fn extract(obj: Borrowed<'_, '_, PyAny>) -> PyResult { let duration_since_unix_epoch: Duration = obj.sub(unix_epoch_py(obj.py())?)?.extract()?; UNIX_EPOCH .checked_add(duration_since_unix_epoch) diff --git a/src/conversions/time.rs b/src/conversions/time.rs index 351fbb637d2..a73ff717f2b 100644 --- a/src/conversions/time.rs +++ b/src/conversions/time.rs @@ -58,7 +58,7 @@ use crate::types::datetime::{PyDateAccess, PyDeltaAccess}; use crate::types::{PyAnyMethods, PyDate, PyDateTime, PyDelta, PyNone, PyTime, PyTzInfo}; #[cfg(not(Py_LIMITED_API))] use crate::types::{PyTimeAccess, PyTzInfoAccess}; -use crate::{Bound, FromPyObject, IntoPyObject, PyAny, PyErr, PyResult, Python}; +use crate::{Borrowed, Bound, FromPyObject, IntoPyObject, PyAny, PyErr, PyResult, Python}; use time::{ Date, Duration, Month, OffsetDateTime, PrimitiveDateTime, Time, UtcDateTime, UtcOffset, }; @@ -180,8 +180,8 @@ impl<'py> IntoPyObject<'py> for Duration { } } -impl FromPyObject<'_> for Duration { - fn extract_bound(ob: &Bound<'_, PyAny>) -> PyResult { +impl FromPyObject<'_, '_> for Duration { + fn extract(ob: Borrowed<'_, '_, PyAny>) -> PyResult { #[cfg(not(Py_LIMITED_API))] let (days, seconds, microseconds) = { let delta = ob.downcast::()?; @@ -223,8 +223,8 @@ impl<'py> IntoPyObject<'py> for Date { } } -impl FromPyObject<'_> for Date { - fn extract_bound(ob: &Bound<'_, PyAny>) -> PyResult { +impl FromPyObject<'_, '_> for Date { + fn extract(ob: Borrowed<'_, '_, PyAny>) -> PyResult { let (year, month, day) = { #[cfg(not(Py_LIMITED_API))] { @@ -264,8 +264,8 @@ impl<'py> IntoPyObject<'py> for Time { } } -impl FromPyObject<'_> for Time { - fn extract_bound(ob: &Bound<'_, PyAny>) -> PyResult