Skip to content

[serve] add cmdline options to control which microbenchmarks to run#46798

Merged
edoakes merged 2 commits intoray-project:masterfrom
zcin:microbenchmarks-script
Jul 29, 2024
Merged

[serve] add cmdline options to control which microbenchmarks to run#46798
edoakes merged 2 commits intoray-project:masterfrom
zcin:microbenchmarks-script

Conversation

@zcin
Copy link
Contributor

@zcin zcin commented Jul 25, 2024

Why are these changes needed?

Add command line flags that can be used to control which microbenchmarks to run. The release test pipeline will use --run-all, but when running the script for debugging, these flags make it a lot easier to run only the tests you want.

Related issue number

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
    • I've added any new APIs to the API Reference. For example, if I added a
      method in Tune, I've added it in doc/source/tune/api/ under the
      corresponding .rst file.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

Signed-off-by: Cindy Zhang <cindyzyx9@gmail.com>
@zcin zcin requested a review from edoakes July 25, 2024 18:48
Comment on lines +289 to +294
@click.option("--run-all", is_flag=True)
@click.option("--run-http", is_flag=True)
@click.option("--run-grpc", is_flag=True)
@click.option("--run-handle", is_flag=True)
@click.option("--run-latency", is_flag=True)
@click.option("--run-throughput", is_flag=True)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the default?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

defaults are all False

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems confusing; would suggest default runs everything (or the most common one in isolation)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added logic that defaults to run everything if no other flags are set!

@edoakes edoakes added the go add ONLY when ready to merge, run all tests label Jul 26, 2024
Signed-off-by: Cindy Zhang <cindyzyx9@gmail.com>
@zcin
Copy link
Contributor Author

zcin commented Jul 29, 2024

@edoakes tests are passing, ready to merge

@edoakes edoakes merged commit e079a90 into ray-project:master Jul 29, 2024
@zcin zcin deleted the microbenchmarks-script branch August 21, 2024 15:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

go add ONLY when ready to merge, run all tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants