Active Memory Manager to use bulk comms #5357
Conversation
distributed/active_memory_manager.py
Outdated
| { | ||
| "op": "acquire-replicas", | ||
| "keys": [ts.key for ts in ts_to_who_has], | ||
| "stimulus_id": "acquire-replicas-" + stimulus_id, |
There was a problem hiding this comment.
@fjetter unsure if, when a function generates as burst of messages, the stimulus ID is supposed to be unique to each message or not?
There was a problem hiding this comment.
TLDR Your code change aligns with my idea
We're up to define this how we want since this system is new. I'd like think of this as "one decision creates one ID". If one decision triggers multiple messages, I would probably put the same ID into the messages. My idea about this ID is to trace the impact of a decision. In this example, a run_once call is the stimulus/trigger and makes the decision to change something about the cluster state (this breaks into many smaller decisions but they have the same origin/root cause/base state). This specific run_once call will have an ID and we'll assign this ID to all messages generated by it.
|
All failures unrelated; ready for review and merge. |
|
|
||
|
|
||
| @pytest.mark.xfail(reason="distributed#5046, distributed#5265") | ||
| @pytest.mark.xfail(reason="distributed#5265") |
|
Happy to have yours first. This one will then need to change to remove the scheduler-side replica. |
|
Added scheduler-side instant drop of the replica post #5356. |
|
test_drop_stess fails. Need to investigate... |
crusaderky
force-pushed
the
AMM_WSMR
branch
from
23a5dc7 to
5d61001
Compare
|
This is ready to be reviewed again |
|
@fjetter are you happy to merge this? |
2 participants
CC @fjetter @mrocklin @jrbourbeau