Skip to content

Conversation

@jakirkham
Copy link
Member

@jakirkham jakirkham commented Jan 23, 2021

Moves task_groups and task_prefixes to SchedulerState where they are type annotated. Then uses them through parent within Scheduler. Allows Cython to recognize these are Python dicts and optimize calls and operations on them.

Note: This is needed as part of optimizing transitions ( #4451 )

Moves `task_groups` and `task_prefixes` to `SchedulerState` where they
are type annotated. Then uses them through `parent` within `Scheduler`.
Allows Cython to recognize these are Python `dict`s and optimize calls
and operations on them.
Instead of using `try...except...` to catch and handle `KeyError`s, just
use `.get(...), which gets the key needed or returns `None`. This has
less overhead. Also the following `None` check is a quick pointer
comparison. Otherwise the code is unchanged.
Since it is used in the check as well, go ahead and assign it beforehand
for simplicity.
@jakirkham jakirkham mentioned this pull request Jan 23, 2021
@mrocklin mrocklin merged commit 2089e79 into dask:master Jan 23, 2021
@jakirkham jakirkham deleted the ref_task_grp_pre branch January 23, 2021 18:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants