Support usefixtures with parametrize

[ShurikMen]

closes #4112

[bluetech]

Thanks @ShurikMen. This seems like a very nice feature to have.

The implementation doing set(fixtureinfo_.names_closure) != set(fixtureinfo.names_closure): is too hacky for my tastes; feels like a thing that will cause unexpected issues. I will try to look into what it would take to this in a more robust way. #11412 is relevant.

[bluetech]

Can you explain this change?

[ShurikMen]

Parameterized tests are bound to an instance of the test class. And in their signature there is no longer a parameter for an instance of this class ('self').
A check on bound is needed so as not to delete the first fixture or parameter that is passed through arguments.

[bluetech]

An example would be cool here I think. Also mentioning this technique in the docs somewhere would be good.

[bluetech]

Another problem that I forgot to mention -- what if the fixture is itself again parametrized? Adapting the example from the PR:

import pytest

@pytest.fixture(params=[21, 22])
def some_fixture(request):
    return request.param

@pytest.mark.parametrize("val", [
    1,
    pytest.param(2, marks=pytest.mark.usefixtures('some_fixture')),
    3,
])
def test_foo(request, val):
    assert val

With the current PR it fails:

The requested fixture has no parameter defined for test:
    a.py::test_foo[2]

Requested fixture 'some_fixture' defined in:
a.py:4

This happens because expand parametrization once, in Metafunc, in a cartesian product sort of way, while this feature requires Metafunc to be smarter.

[ShurikMen]

Thanks @ShurikMen. This seems like a very nice feature to have.

The implementation doing set(fixtureinfo_.names_closure) != set(fixtureinfo.names_closure): is too hacky for my tastes; feels like a thing that will cause unexpected issues. I will try to look into what it would take to this in a more robust way. #11412 is relevant.

Perhaps a hacky, but sufficient check for the fact of changing the list of fixtures for this instance of the test with this parameter in comparison with the parent set.

[ShurikMen]

In theory, we can try to change the order of calling pytest_generate_tests.
Now fixes.py::FixtureManager::pytest_generate_test (parameterization in fixtures) is performed first, then python.py::pytest_generate_test (parameterization via the corresponding marker). But it can break the order of execution of parameterized tests. And I can't say yet what the consequences will be.

[bluetech]

In theory, we can try to change the order of calling pytest_generate_tests.

It's worth a try to see what happens. I expect it wouldn't work, because of indirect fixtures stuff. Though maybe it's already handled at this point of execution, I don't recall ATM...

[obestwalter]

@ShurikMen would you like to pick that up and try the suggested approach? Otherwise we might want to close this to be picked up later.

[ShurikMen]

Not relevant after changes in #11416. Now usefixtures works with parameters.