PyErr does not implement std::error::Error

[nagisa]

PyErr is a very odd structure. It pretends to emulate the 3-tuple returned by PyErr_Fetch, but then its pvalue is a 3-variant that also allows lazy construction of exception objects.

As thus it fails to implement a couple of traits typically expected of the error type (std::error::Error, std::fmt::Display, Sync and Send) and does not integrate very well with the error handling story in Python.

There are a couple of ways to go forward here, but one of the simpler things that could be done I think is:

1. Add a method along the lines of fn(Python, PyErr) -> Py<BaseException>;
   • I don’t believe this can be done easily right now without unsafe code.
2. Implement the typical error traits for Py<ExceptionT> types.
   • Specifically for Display and friends I think it is fine to just do a Python::acquire_gil() inside if that is necessary to format the data.

Willing to contribute the necessary code to implement this. Please let me know what you think about this. Perhaps other approaches seem better?

[kngwyu]

Thank you for the proposal.
Error handling is indeed a major pain around PyO3 and thus should be enhanced.

Perhaps other approaches seem better?

I'm not sure they are better, but possible alternatives are
3. Get error string when fetching error (difficult, but I don't think it's impossible)
4. Hold some 'intermediate' error message (see #652)

Implement the typical error traits for Py types.

PyErr has already the same memory-management semantics as Py<T> so I don't think Py<ExceptionT> is appropriate.
But now it is really unclear how PyErr and exceptions are related and I think we need better semantics.

[nagisa]

PyErr has already the same memory-management semantics as Py so I don't think Py is appropriate.

To make PyErr Send + Sync it would require a breaking change to the lazy-initialized variants of the pvalue field. If those variants did not exist, this problem would be significantly easier and consist of basically making a Display implementation, something I already have.

FWIW I feel it is fine to have multiple ways to do the same thing in pyo3. It is a necessity, in fact, due to the fact that some conversions are not easily possible (such as going from PyErr to BaseException…)

---

Unless I misunderstand them, options 3&4 are insufficient as they lose information: the type, traceback and causation chain (raise exc from otherexc) are not preserved. BaseException and derived objects store all that information in the object (type(), __traceback__, etc.)

[kngwyu]

So how about separating Rust-originated errors into another struct?
Then we would have(roughly)

struct PyErrRust(Box<dyn IntoPyErrorTrait>);
struct PyErr {
    ptype: Py<PyType>,
    pvalue: PyObject,
    ptracebacj: Option<PyObject>,
}

[nagisa]

Yeah, that’s definitely something that can work, the suggested PyErr is almost exactly what I have used for my PoC.

I do think that it is a waste of space to store ptype and ptraceback, given that they can also be stored within pvalue, but I can live with that.

[flaper87]

I don't have a better proposal right now but I wanted to voice my concern about the current error handling in pyo3 and my interest in this issue. Right now, I can't seem to be able to fetch the actual python error string in a nice way that I can integrate with my application.

Thanks for working on this!

P.S: Is there an example that shows how to deal with errors right now? How should python tracebacks be extracted, etc?

[nagisa]

P.S: Is there an example that shows how to deal with errors right now? How should python tracebacks be extracted, etc?

There isn’t an example right now, but if you consider that you need to essentially write the python to extract the information there might be some relevant python tutorials.

Extracting tracebacks will never integrate well, sadly, but we could at least add a method to obtain it and implement nice printing.

[flaper87]

P.S: Is there an example that shows how to deal with errors right now? How should python tracebacks be extracted, etc?

There isn’t an example right now, but if you consider that you need to essentially write the python to extract the information there might be some relevant python tutorials.

Extracting tracebacks will never integrate well, sadly, but we could at least add a method to obtain it and implement nice printing.

I've written some python to extract the traceback but it doesn't seem to work. Basically, call the traceback module when an error happens. I might be doing something wrong, tho. Thanks for the tip

[davidhewitt]

I have a PR for this almost ready which I'd love get done for 0.12.

[vitorenesduarte]

Hi. I'm currently using the pytry macro below in order to get meaningful errors like ModuleNotFoundError: No module named 'matplotli'. Ignoring the fact that it's not very ergonomic, I found this approach to work really well for me.

Is the plan to allow these nice errors but only with PyModule::import(py, "matplotli")? instead of something like pytry!(py, PyModule::import(py, "matplotli"))?

use pyo3::prelude::*;

#[macro_export]
macro_rules! pytry {
    ($py:expr, $e:expr) => {{
        match $e {
            Ok(v) => v,
            Err(e) => color_eyre::eyre::bail!("{:?}", e.print($py)),
        }
    }};
}

fn main() -> Result<(), color_eyre::Report> {
    let gil = Python::acquire_gil();
    let py = gil.python();

    let _ = pytry!(py, PyModule::import(py, "matplotli"));
    Ok(())
}

My first attempt at this macro was pytry0 below, but the output was something like PyErr { type: Py(0x7fc111eeb780, PhantomData) }. Is still possible to get the nice output of pytry without having access to Python? Thanks.

use pyo3::prelude::*;

#[macro_export]
macro_rules! pytry0 {
    ($e:expr) => {{
        match $e {
            Ok(v) => v,
            Err(e) => color_eyre::eyre::bail!("{:?}", e),
        }
    }};
}

fn main() -> Result<(), color_eyre::Report> {
    let gil = Python::acquire_gil();
    let py = gil.python();

    let _ = pytry0!(PyModule::import(py, "matplotli"));
    Ok(())
}