Figure out how to stop fighting with pytest-asyncio over fixtures

[njsmith]

As noted in python-trio/trio#27 (comment):

speaking of pytest-asyncio module, it considers any async fixture to be run with asyncio. So if it pytest pytest_fixture_setup is run before pytest-trio one (not sure how pytest decide which module execute first, but that's what happened during my tests), this cause a crash. So right now pytest-asyncio is not compatible with this pytest-trio

Here's my suggestion: since fixtures already have to be explicitly decorated, it's not too onerous to ask the user to also explicitly say which kind of async fixture they want. So instead of writing

from pytest import fixture

@fixture
async def whatever():
    ...

you'd write

from pytest_trio import trio_fixture

@trio_fixture
async def whatever():
   ...

We should probably also continue to install a fixture hook, but make it just detect when someone has accidentally written @fixture async def ... so we can give an error message saying to use @trio_fixture instead.

We should also try to convince the pytest-asyncio folks to switch to this system (CC @Tinche). Especially if we are going to install a fixture hook that will randomly cause their thing to break and tell their users to use @trio_fixture instead :-)

[njsmith]

(accidentally posted this too soon; if you got an empty notification then click through to the issue to see my actual text.)

[njsmith]

Oh heh, looking further into python-trio/trio#27 (comment) I see that you already proposed something similar. I guess my innovation above is that if people are using a @trio_fixture decorator anyway then we don't need to make them use @pytest.fixture too, we can make the @trio_fixture decorator imply that.

Small tweak we could do: make our special decorator accept non-async functions and silently treat them like regular @pytest.fixture, so you can do from pytest_trio import fixture at the top of your file and then use @fixture everywhere, and it magically works. I'm not sure whether this would be really handy, or way too magical.

[njsmith]

Coming back around to this again, I'd like to make a more refined proposal:

• A fixture is a "trio fixture" if any of the following are true:

  • created with @trio_fixture (required for projects that want to use multiple pytest-(.*)io packages)

  • it's an async fixture and the "everything async is trio" configuration flag is enabled (Provide a way to automatically treat all async tests as trio tests #9)

  • it's a fixture that depends on a "trio fixture"

• It's an error to use a "trio fixture" in any situation except (a) test scope, (b) the test is marked @pytest.mark.trio

• All "trio fixtures" get the special delayed handling when combined with the @pytest.mark.trio runner

I think this provides a good balance of functionality for both values of the "everything is trio" config flag, should provide useful error messages when misused, and has at least the potential to play nicely with packages like pytest-asyncio and pytest-curio.

Of course, right now, any system that allows for plain @pytest.fixture to work on regular async functions will be broken if pytest-asyncio is installed, because that's how it works, and that includes this proposal for when "everything async is trio" mode is enabled. But there would be path forward at least, since in this proposal that brokenness would be entirely on pytest-asyncio's side, and we could hopefully convince them to implement a similar setup. (@Tinche: sound plausible?)