[GoSDK Infra] Limit simultaneous tests binaries to 3. #15321
Conversation
Executing all test binaries at the same time seems to be overloading the mini-local Flink runner, setting a limit to avoid overwhelming it.
|
Run Go Flink ValidatesRunner |
|
Run Go PostCommit |
|
R: @ibzib |
| TIMEOUT=1h | ||
|
|
||
| # Default limit on simultaneous test binaries/packages being executed. | ||
| PARALLEL=3 |
There was a problem hiding this comment.
I saw in the docs that the default is the number of cores, so it should be fine to leave it unset for the Dataflow runner, which doesn't need to do a lot of heavy-duty computation locally. But for Flink I think it should be 1. If nothing else, it makes reading the logs easier -- otherwise the logs for different jobs get interleaved and it becomes very difficult to tell which logs came from which test.
There was a problem hiding this comment.
I'd say that's overthinking the current issue.
Dataflow does have a per-project simultaneous job limit though, and they all run on apache-beam-testing.
TBH most of the output available from the Jenkins tests, due to gradle aren't actually useful in debugging anything in specific because gradle+ jenkins swallow the logs. It's more important at the moment that the tests can run to completion. We can always tool it downwards later.
There was a problem hiding this comment.
TBH most of the output available from the Jenkins tests, due to gradle aren't actually useful in debugging anything in specific because gradle+ jenkins swallow the logs.
Flink logs don't get swallowed, so they'd be useful if only we could tell which job they came from.
There was a problem hiding this comment.
I think the main source of conflict here is that I didn't see that Spark and Samza were already being hard limited to 1 test binary at a time anyway. I'm keeping the 3 limit for Dataflow and Portable, and have made it flaggable. Also changed things so they're only specifying the flag once, what I had would override it.
A later PR might change this kind of hacky overrides to happen at the script invocation level, but I'm currently making these changes via the web UI, and would rather keep it simple.
|
Run Go Flink ValidatesRunner |
|
Run Go Flink ValidatesRunner |
|
Run Go PostCommit |
|
Run Go Spark ValidatesRunner |
|
Run Go Samza ValidatesRunner |
2 participants
Executing all test binaries at the same time seems to be overloading the mini-local Flink runner,
setting a limit to avoid overwhelming it.
Only applies to the portable/validates runner tests.
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
R: @username).[BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replaceBEAM-XXXwith the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.CHANGES.mdwith noteworthy changes.See the Contributor Guide for more tips on how to make review process smoother.
ValidatesRunnercompliance status (on master branch)Examples testing status on various runners
Post-Commit SDK/Transform Integration Tests Status (on master branch)
Pre-Commit Tests Status (on master branch)
See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI.