feat: add coroutine `__name__`/`__qualname__` by wyfo · Pull Request #3588 · PyO3/pyo3

wyfo · 2023-11-20T08:36:41Z

Relates to #1632

codspeed-hq · 2023-11-20T10:29:52Z

CodSpeed Performance Report

Merging #3588 will not alter performance

_{Comparing wyfo:coroutine_name (781b9e3) with main (1203921)}

Summary

✅ 78 untouched benchmarks

davidhewitt · 2023-11-23T00:50:14Z

@wyfo if you can rebase this please I'll get around to reviewing as soon as possible after 👍

src/impl_/coroutine.rs

src/impl_/wrap.rs

adamreichold · 2023-11-25T09:28:44Z

src/coroutine.rs

+    #[getter]
+    fn __name__(&self) -> PyResult<Py<PyString>> {
+        self.name
+            .clone()


Not sure if it is worth it, but using clone_ref would avoid the implicit check for the GIL.

adamreichold · 2023-11-25T09:29:33Z

src/coroutine.rs

+                    .as_ref()
+                    .map_or(Ok("<coroutine>"), |n| n.as_ref(gil).to_str())
+                    .unwrap();
+                let message = format!("coroutine {} was never awaited", qualname);


Does this only happen if it was never awaited or also if it was not polled to completion?

It seemed to me that the warning was always raised when the coroutine was not run to completion (even partially), but I've checked again, and I was wrong.
I will fix that behavior.

In fact, instead of testing self.future.is_some(), I should test self.waker.is_none(), as it's always Some(_) when the coroutine has been polled at least once.

adamreichold · 2023-11-25T09:31:15Z

src/coroutine.rs

+                    .map_or(Ok("<coroutine>"), |n| n.as_ref(gil).to_str())
+                    .unwrap();


Suggested change

.map_or(Ok("<coroutine>"), |n| n.as_ref(gil).to_str())

.unwrap();

.and_then(|n| n.as_ref(gil).to_str().ok())

.unwrap_or("<coroutine>");

Your suggestion change the effect of the code. I wanted to panic in case of to_str error, but panicking may not be the best for a warning, you're right.

davidhewitt

Ok, this diff is now nicely focussed just on the async code so I've given it a full review 👍

I agree with @adamreichold's points and also added a couple of brief thoughts of my own.

davidhewitt · 2023-11-25T10:10:26Z

pyo3-macros-backend/src/method.rs

+                call = quote! {{
+                    let future = #call;
+                    _pyo3::impl_::coroutine::new_coroutine(
+                        _pyo3::types::PyString::new(py, stringify!(#python_name)).into(),


It would probably be worth interning here using our intern! macro, and changing new_coroutine to take N: IntoPy<Py<PyString>>.

davidhewitt · 2023-11-25T10:12:15Z

src/impl_/coroutine.rs

+pub fn coroutine_qualname<'py>(
+    py: Python<'py>,
+    module: Option<&PyModule>,
+    name: &str,
+) -> &'py PyString {
+    match module.and_then(|m| m.name().ok()) {
+        Some(module) => PyString::new(py, &format!("{}.{}", module, name)),
+        None => PyString::new(py, name),
+    }
+}
+
+pub fn method_coroutine_qualname<'py, T: PyClass>(py: Python<'py>, name: &str) -> &'py PyString {
+    let class = T::NAME;
+    let qualname = match T::MODULE {
+        Some(module) => format!("{}.{}.{}", module, class, name),
+        None => format!("{}.{}", class, name),
+    };
+    PyString::new(py, &qualname)
+}


I wonder, to avoid Python string creation with every coroutine, can this be done lazily (by e.g. storing a reference to T::type_object, or taking a function pointer, or storing a reference to the module etc).

You're right. Actually, methods __qualname__ could be interned too, as class name and module name are static. The issue is for free functions, because it would requires to store the module.

I've finally used a qualname_prefix instead of passing the __qualname__ directly.
The constructor parameter may be modified when it will be exposed (maybe using Option<Cow<'static, str>> instead of Option<&'static str> for example), but for now, it's adapted to the macro and that's enough.

davidhewitt · 2023-11-25T10:14:22Z

src/coroutine.rs

+        if self.future.is_some() {
+            Python::with_gil(|gil| {


This structure reminds me of #3386 (comment).

I'm not entirely sure if it's a blocker to not have try_with_gil or some other guard to avoid potential nasty crashes here. We should have a little think about that before merge.

I don't really understand how could with_gil crash here, but I know that Drop::dorp is not the best place to put this warning. In fact, it should go in __del__ (the same as CPython coroutines), but it's not supported yet in PyO3.
Maybe we can postpone the warning implementation until #2479 is resolved, as this feature is quite nonessential? We can also already write the __del__ method, as it has no effect for now, and it will work as a side effect of the __del__ support.

Ahh that's a good suggestion, I should really pick up __del__ support again soon. Are you suggesting split this PR in two; merge the names now and keep the warning as a draft pr for the moment?

I am indeed.

I've opened an issue for the warning.

davidhewitt

This looks good to me, thanks! Will follow up with discussion on the issue.

wyfo force-pushed the coroutine_name branch from d29b9cd to 5b77de9 Compare November 20, 2023 09:41

wyfo force-pushed the coroutine_name branch 5 times, most recently from 9502795 to 5cafd8b Compare November 23, 2023 07:47

wyfo commented Nov 23, 2023

View reviewed changes

src/impl_/coroutine.rs Outdated Show resolved Hide resolved

wyfo force-pushed the coroutine_name branch from 5cafd8b to 6b95d0d Compare November 23, 2023 07:57

davidhewitt reviewed Nov 24, 2023

View reviewed changes

src/impl_/wrap.rs Outdated Show resolved Hide resolved

davidhewitt mentioned this pull request Nov 24, 2023

refactor OkWrap to not call .into_py(py) #3595

Merged

wyfo force-pushed the coroutine_name branch from 6b95d0d to 1bc1640 Compare November 25, 2023 07:36

adamreichold reviewed Nov 25, 2023

View reviewed changes

davidhewitt reviewed Nov 25, 2023

View reviewed changes

wyfo force-pushed the coroutine_name branch from 1bc1640 to 871efcb Compare November 25, 2023 20:17

wyfo mentioned this pull request Nov 25, 2023

Issue a warning when coroutine is never awaited #3597

Open

wyfo changed the title ~~feat: add coroutine __name__/__qualname__ and not-awaited warning~~ feat: add coroutine __name__/__qualname__ Nov 25, 2023

feat: add coroutine __name__/__qualname__ and not-awaited warning

781b9e3

wyfo force-pushed the coroutine_name branch from 871efcb to 781b9e3 Compare November 25, 2023 20:51

wyfo changed the title ~~feat: add coroutine __name__/__qualname__~~ feat: add coroutine __name__/__qualname__ Nov 25, 2023

davidhewitt approved these changes Nov 26, 2023

View reviewed changes

davidhewitt added this pull request to the merge queue Nov 26, 2023

Merged via the queue into PyO3:main with commit d8002c4 Nov 26, 2023

		.map_or(Ok("<coroutine>"), \|n\| n.as_ref(gil).to_str())
		.unwrap();

Conversation

wyfo commented Nov 20, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codspeed-hq bot commented Nov 20, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging #3588 will not alter performance

Summary

Uh oh!

davidhewitt commented Nov 23, 2023

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

davidhewitt left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

davidhewitt left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

wyfo commented Nov 20, 2023 •

edited

Loading

codspeed-hq bot commented Nov 20, 2023 •

edited

Loading