Allow tests to extend endTestScope timeout

[haoxli]

Added a per-test endTestScope timeout function so heavy cases can extend the time without slowing down the rest of the suite, and used it to give the 64K timestamp-query stress test run extra headroom, avoiding flakes while leaving the default 5000ms limit untouched elsewhere.

Issue: #

---

Requirements for PR author:

• All missing test coverage is tracked with "TODO" or .unimplemented().
• New helpers are /** documented */ and new helper files are found in helper_index.txt.
• Test behaves as expected in a WebGPU implementation. (If not passing, explain above.)
• Test have be tested with compatibility mode validation enabled and behave as expected. (If not passing, explain above.)

Requirements for reviewer sign-off:

• Tests are properly located.
• Test descriptions are accurate and complete.
• Tests provide complete coverage (including validation control cases). Missing coverage MUST be covered by TODOs.
• Tests avoid over-parameterization (see case count report).

When landing this PR, be sure to make any necessary issue status updates.

[github-actions]

Results for build job (at 52edcaa):

 Test case/subcase counts did not change.

[kainino0x]

I'm not sure if I want to make this part of the "official" interface of the test fixture. It seems a bit fragile or easy to misuse because it's very specific to the test why this might need to be done.

Would it work to wait inside the test by just doing await t.queue.onSubmittedWorkDone() at the end before returning from the test function?

[haoxli]

I try it not work for this timeout case. This failure was not because the test ended before the commands were completed, but rather because it didn't have enough time to finish the tests (includes waiting submitted commands done + error scop popping + celanup). I think what we need here is a mechanism to increase the execution time of cases that require long execution times.

To avoid exposing it as a public interface and thus causing misuse, how about make the framework to auto-detect slow tests and extend timeouts automatically, if the default timeout in endTestScope is hit, run the case again with twice the default timeout. However, this approach increases the overall runtime of the test suite if a network-related timeout occurs.

Another idea is whether we can move this case into the stress test src/stress/queries/timestamps.spec.ts. I see there are some stress tests for render/compute pass are unimplemented, It would be reasonable to set a separate timeout for them.

[kainino0x]

Ah... it occurs to me that the test fixture already does onSubmittedWorkDone at the end of tests, so I should have realized that wouldn't work.

I was thinking that if the popErrorScope timeout worked, onSubmittedWorkDone would also work. But maybe not anymore because of work in Dawn to make error scopes asynchronous?

Anyway, what about just wrapping the test body in t.device.pushErrorScope('validation')/await t.device.popErrorScope()? If error scopes are able to capture the slowness at the test-fixture level, they should work at the test level too.

To avoid exposing it as a public interface and thus causing misuse, how about make the framework to auto-detect slow tests and extend timeouts automatically, if the default timeout in endTestScope is hit, run the case again with twice the default timeout.

IMO that would add too much complexity to the CTS and would probably interact poorly with the browsers' test harnesses which have their own timeout systems. My objection to putting it in the public interface is not that strong :)

Another idea is whether we can move this case into the stress test src/stress/queries/timestamps.spec.ts. I see there are some stress tests for render/compute pass are unimplemented, It would be reasonable to set a separate timeout for them.

I thought about suggesting moving this to stress, but right now I don't think anyone is running those tests so I don't want to lose the coverage. If we can make it run reliably then I'd like to keep it in the conformance section.

[haoxli]

I'm OOO this week and will update it next week.