[BUG] multiple-pipeline-playback may start recording!!!!

[plbossart]

Describe the bug
While using the sof-test scripts on a platform with limited capabilities, I came across a "feature" of the multiple-pipeline-playback/capture scripts.

Because of limited capabilities, I only allowed for 2 PCM playback devices by skipping all others with
TPLG_IGNORE_LST['pcm']='HDA Digital,Port2'

This results in Port0 and Port1 being used.

However this also results in Port0 and Port1 being used for capture in parallel with playback. This seems to be a 'feature' where capture streams are added if there are fewer than 4 playback devices.

That's not supposed to happen, mixing playback and capture is a different test.

To Reproduce
TPLG=/lib/firmware/intel/sof-ace-tplg/sof-lnl-nocodec.tplg /root/sof-test-main/test-case/multiple-pipeline-playback.sh -l 50
TPLG=/lib/firmware/intel/sof-ace-tplg/sof-lnl-nocodec.tplg /root/sof-test-main/test-case/multiple-pipeline-capture.sh -l 50

Expected behavior
playback should only playback...

@mengdonglin @keqiaozhang @marc-hb @fredoh9 FYI, this problem needs to be fixed to allow the use of scripts even with limited capabilities.

[mengdonglin]

@plbossart Probably we can look at this issue next week? This week @keqiaozhang is crazy busy with deployment updates for the new building service, this is a task we must finish with help of Yongan before Apr 30. We hope to make basic flow work this week and complete debugging/code review in the next 2 weeks. Not sure if @fredoh9 has bandwidth this week.

[keqiaozhang]

@plbossart and @mengdonglin
This is intentional in our test script. If playback pipeline count is less than requested, capture pipelines will be started to ensure requested number of pipelines are running in parallel.

Please refer to:
https://github.com/thesofproject/sof-test/pull/570/files

[keqiaozhang]

The default value of pipeline count is 4, so if you only want to test 2 playback pipelines in parallel, you can specify it with -f p -c=2.

TPLG=/lib/firmware/intel/sof-ace-tplg/sof-lnl-nocodec.tplg /root/sof-test-main/test-case/multiple-pipeline-playback.sh -l 50 -f p -c 2

[plbossart]

@keqiaozhang I insist that it makes no sense to mix capture and playback for tests. This is difficult to understand and root-cause

What you are describing is a separate test where up to N pipelines are tested in parallel, starting with playback or capture optionally. This test is valuable, but it's a different test.

For now we can get by with manual tests with the -c2 option, but long-term we want to change the CI settings to ONLY test playback when we say we test playback, and add a separate test to allow for mixed playback/capture.

[marc-hb]

@plbossart Probably we can look at this issue next week? This week @keqiaozhang is crazy busy with deployment updates for the new building service,

This looks like a relatively small, logical change that anyone could do. I'd be delighted to review the corresponding PR.

There seems to be an assumption that any change that is not in C code has to be designed, implemented, tested and maintained by a "bottleneck", non-C team of extremely few people. I think this is too inflexible and a wrong estimation of the efforts required. I mean the non-C efforts in total, not for this apparently small change.

[plbossart]

There seems to be an assumption that any change that is not in C code has to be designed, implemented, tested and maintained by a "bottleneck", non-C team of extremely few people.

If I could do these changes I would, but I just don't have any cycle to spend on reverse-engineering that the 'non-C team" came up with during a multi-year set of incremental changes.

It's not just a problem with the language, though those scripts are notoriously not easy to work with. It's also that the assumptions are rather unclear.

In other words, it's complicated from concept to implementation, so there should be no assumption that world+dog can change this easily. Put differently, the test infrastructure has become so complicated it's not a surprise that only the happy few who took part in the development can modify it.

[marc-hb]

If I could do these changes I would,

To be very clear "anyone else" was not pointing at you specifically.

Put differently, the test infrastructure has become so complicated it's not a surprise that only the happy few who took part in the development can modify it.

That's very true in general, I really regret that and I've done a LOT over the last couple years to mitigate and sometimes even correct it, see for instance #532 and before where I de-duplicated multiple-pipeline-playback and multiple-pipeline-capture. They were 95% copy/paste/diverge of each other!

However:

• It does not apply for this particular issue, that's why I made this comment. This particular test script is far from role-modelling shell scripting but it's very short (200 lines!) and really not that complex.
• It's much less true for sof-test than for Jenkins and co.
• Unlike Jenkins, developers are supposedly using sof-test on their system every day. How would they testing their C code otherwise? Totally different areas yet the two are typically put together in the same non-C bucket.
• Unlike the "C team", Jenkins, build, CI and other "non-C" people are NOT routinely interacting with sof-test directly.
• Unlike the "C team", non-C people have a much inferior knowledge of test coverage priorities
• It's not going to get any better by sticking to the status quo. If anything the status quo will keep making things worse. In other words it would be really good to (slowly) start leaving the concept of a "non-C team" behind. sof-test is by far the best place to start.

My 6 cents.

[marc-hb]

Back to the main topic

long-term we want to change the CI settings to ONLY test playback when we say we test playback, and add a separate test to allow for mixed playback/capture.

I have an idea of how we could move test plans to a git repo, out of some mongoDB where they should have never been stored in the first place (technical debt). Time permitting (= probably not), I think I could have something working in just a couple weeks. Then the CI side would have to simplify and adjust and I don't know how much effort that would require.

[plbossart]

well let's agree then that nothing is going to happen. I've seen that movie before.

[fredoh9]

For multi pipeline test cases, we have made some changes over years after having discussions/suggestions.

I can take a look at again. My understanding is that,

• multiple-pipeline-playback => use only playback pipelines
• multiple-pipeline-capture => use only capture pipelines
• multiple-pipeline-all => use all playback/capture pipelines

[plbossart]

@fredoh9 your definition is precisely what makes sense, but it's not what we have:

multiple-pipeline-playback => use playback pipelines and then ADD capture pipelines to reach the expected count.