Cartesian product while treating some parameters as tuples?

[aomader]

Hej,

I am currently experimenting with TUnit and how to best solve the following:

There is an unary data source that yields different results based on certain runtime parameters. This should be used to build the Cartesian product with the final set of arguments for all other parameters. This is very similar to your nested data sources, except for the fact that the first parameter is not a singular value.

Here a bit pseudo code to help get my point across:

public Config(string Name);

public class Tests
{
    [Test]
    [WhatEverToLoadSomeConfigsAndBuildTheCartesianProductWithTheOthers]
    [Arguments("one", 1)]
    [Arguments("two", 2)]
    public async Task Test_it(
        Config app,
        string foo,
        int bar)
    {
        // Assuming there are the configs Config("hey") and Config("you"),
        // it should be called with:
        // - Config("hey"), "one", 1
        // - Config("hey"), "two", 2
        // - Config("you"), "one", 1
        // - Config("you"), "two", 2
    }
}

So it seems trivial to implement the WhatEverToLoadSomeConfigsAndBuildTheCartesianProductWithTheOthers attribute to provide the unary data source, which can then also be used with Matrix and the Combined.. version to generate Cartesian product over all parameters, but what about the middle ground, when I want to treat certain parameters as tuples to form the product with?

[thomhurst]

I actually think the docs you linked are wrong. I'll need to fix that.

I think CombinedDataSources might be able to solve this for you?

https://tunit.dev/docs/test-authoring/combined-data-source-summary/

[aomader]

Hej @thomhurst, thank you for coming back to me this quick.

I probably did not explain my issue well enough.

So let me try again: I want the Cartesian product, but not across all parameters. I still want to be able to provide data for a subset of parameters via e.g. Arguments or the TestDataRow<>, and combine those with a subset of others (to phrase it a bit general).

Again, some pseudo code which obviously does not work like this, I am just trying to get my point across. Its really about how to make an IDataSourceAttribute provide data for only a subset of parameters and build the combinatorial product between them, but not independently for all parameters.

[Test]
// something for a subset of the parameters, here for only the first one
[InstanceMethodDataSource(For: nameof(param1), nameof(TestData))]
// something for a subset of the parameters, here the last two
[Arguments(For: nameof(param2), 123, 456)]
[Arguments(For: nameof(param3), true, false)]
public async Task Test(string param1, int param2, bool param3)
{
    // Should be called with 3 x 2 = 6 different data sets:
    // - "one", 123, true
    // - "one", 456, false
    // - "two", 123, true
    // - "two", 456, false
    // - "three", 123, true
    // - "three", 456, false
}

private IEnumerable<string> TestData => new[] { "one", "two", "three" };

[thomhurst]

This isn't currently possible I'm afraid.

I'll have a think though 🤔

[aomader]

To give a bit more background information, we are evaluating whether how easy it is to replace xUnit v2 in our somewhat complex E2E testing setup. Those are tests are agnostic to the underlying browser, i.e., we operate with Selenium, Playwright, AngleSharp, etc., and run different setups (e.g., different viewports, cookies settings, etc.) depending on the environment. Hence, we model the browser abstraction as a parameter.

Obviously, there are tests that are further parameterized, which leads us to this issue.

Coming back to the above example, a more concrete example we are interested in getting to work looks something like that:

[Test]
[Browsers] // custom IDataSourceAttribute that provides the abstract Browser
[Arguments(".action-a", "Hello")] // values for the other two parameters
[Arguments(".action-b", "Bonjour")]
public async Task Test_click_in_browser(
    Browser browser,
    string buttonSelector,
    string expectedGreeting,
)
{
    await browser.GotoAsync("page");
    await browser.ClickAsync(buttonSelector);
    await browser.WaitForAsync(expectedGreeting);

    // Should be called with |browsers| x 2, i.e., running the 2 parametersets in all browsers
}

Note that this is purely made up just to better illustrate what we are after.