Add nursery fixture #25
Description
I'm currently seeing a big limitation to trio async fixtures: there is no nursery available when they are run.
Considering a usecase when you want to test the connection to a server, you have to:
- start the server
- connect to the server
- send a request to the server and check the result
- close the server
Without nursery, there is no way to achieve 1) from a fixture, so this means the test would look like this:
async def test_server_connection():
async def actual_test(listerner):
connection = await connect_to_server(listerner) # step 2)
ret = await connection.ping() # step 3)
assert ret == 'ok'
nursery.cancel_scope.cancel() # step 4)
async with trio.open_nursery() as nursery:
listerners = nursery.start(trio.serve_tcp, my_server_handle_client, 0) # step 1)
nursery.start_soon(actual_test, listerners[0])This is pretty cumbersome (especially with multiple tests, each one needing to start the server).
A solution to this trouble could be to write a decorator for the test:
async def with_server(fn):
@wraps(fn)
async def wrapper(*args, **kwargs):
async def runtest_then_cancel_scope(listerner):
await fn(listerner, *args, **kwargs)
nursery.cancel_scope.cancel() # step 4)
async with trio.open_nursery() as nursery:
listeners = nursery.start(trio.serve_tcp, my_server_handle_client, 0) # step 1)
nursery.start_soon(runtest_then_cancel_scope, listerners[0])
@with_server
async def test_server_connection(server):
connection = await connect_to_server(server) # step 2)
ret = await connection.ping() # step 3)
assert ret == 'ok'While the test looks much better, boilerplates are still pretty cumbersome. On the top of that this code is wrong because @wraps will pass all the functions parameters to the parent caller which in turn will consider them as fixtures to find. However the first parameter server is in fact injected by the decorator itself ! The work around is to create a custom wraps function that discard the first argument before doing the decoration... This is getting out of hand pretty quickly !
To solve all this, I'm thinking the user test function could be run into a nursery, this way we could provide a nursery fixture to access it, allowing to easily create fixture responsible to start a server.
@pytest.fixture
async def server(nursery):
listeners = await nursery.start(trio.serve_tcp, my_server_handle_client, 0) # step 1)
return listeners[0]
async def test_server_connection(server):
connection = await connect_to_server(server) # step 2)
ret = await connection.ping() # step 3)
assert ret == 'ok'Finally we would call nursery.cancel_scope.cancel() once the teardown of the fixture have been done (which would achieve step 4 automatically).
This seems much more elegant and straightforward, however I'm still unsure if canceling this scope automatically is such a good idea (this could lead to normally infinite loop that would end up hidden).
Another solution would be to take advantage of the async yield fixture to do stop the server during teardown of the fixture:
@pytest.fixture
async def server(nursery):
with trio.open_cancel_scope():
listeners = await nursery.start(trio.serve_tcp, handle_client, 0)
yield listeners[0]However this doesn't seems to work:
../../.local/share/virtualenvs/pytest-trio-fHSyI99q/lib/python3.6/site-packages/trio/_core/_run.py:1502: in run_impl
msg = task.coro.send(next_send)
pytest_trio/plugin.py:96: in teardown
await self._teardown()
pytest_trio/plugin.py:119: in _teardown
await self.agen.asend(None)
pytest_trio/_tests/test_try.py:14: in server
yield listeners[0]
/usr/local/lib/python3.6/contextlib.py:89: in __exit__
next(self.gen)
../../.local/share/virtualenvs/pytest-trio-fHSyI99q/lib/python3.6/site-packages/trio/_core/_run.py:206: in open_cancel_scope
scope._remove_task(task)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <cancel scope object at 0x7f1ee62c41d0>, task = <Task 'pytest_trio.plugin._setup_async_fixtures_in.<locals>._resolve_and_update_deps' at 0x7f1ee62c4208>
def _remove_task(self, task):
with self._might_change_effective_deadline():
> self._tasks.remove(task)
E KeyError: <Task 'pytest_trio.plugin._setup_async_fixtures_in.<locals>._resolve_and_update_deps' at 0x7f1ee62c4208>
../../.local/share/virtualenvs/pytest-trio-fHSyI99q/lib/python3.6/site-packages/trio/_core/_run.py:165: KeyError
I'm not sure if this bug is due to a conflict between open_nursery and the open_cancel_scope over the server_tcp coroutine or a more generic bug caused by how an async yield fixture is created inside the setup coroutine but finished consumed into a teardown one...
@njsmith I would be really interested to have your point of view on this
I've created a branch implementing this feature, you can have a look at the test crashing