-
-
Notifications
You must be signed in to change notification settings - Fork 748
Some updates to scheduling policies docs #5911
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
I was reading through the [scheduling policy docs](http://distributed.dask.org/en/latest/scheduling-policies.html) and noticed some things were outdated or missing important details. I think it could still benefit from a bigger revamp, but makes it a bit more accurate.
Unit Test Results 12 files ± 0 12 suites ±0 5h 50m 30s ⏱️ - 1h 29m 1s For more details on these failures, see this check. Results for commit 641f3b7. ± Comparison against base commit 39c5e88. ♻️ This comment has been updated with latest results. |
docs/source/scheduling-policies.rst
Outdated
| before starting new work. This often conflicts with the | ||
| first-come-first-served objective but often results in shorter total runtimes | ||
| first-come-first-served objective, but often results in shorter total runtimes | ||
| and significantly reduced memory footprints. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
short of avoiding disk spillage, why would it result in a shorter runtime?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't write this, really not sure. Maybe just that depth-first traversal is faster than breadth-first traversal (simply because it has a reduced memory footprint)? I feel like it might be referring to something about when multiple clients are submitting work, completing the work of one, rather than fully interleaving them, results in better overall runtimes (again because of reduced memory footprint)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It says that it does in fact go against first-come-first-served expectations.
Maybe change to:
but often results in significantly reduced memory footprints and, due to avoiding data spillage to disk, often in better overall runtimes.
Co-authored-by: crusaderky <crusaderky@gmail.com>
Co-authored-by: crusaderky <crusaderky@gmail.com>
|
@crusaderky I believe this is ready |
|
Thank you! |
I was reading through the scheduling policy docs and noticed some things were outdated or missing important details. I think it could still benefit from a bigger revamp, but makes it a bit more accurate.
cc @fjetter @crusaderky
pre-commit run --all-files