Skip to content

Workers submit a reply to the scheduler if replica removal was rejected #5356

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
crusaderky merged 8 commits into from
Sep 28, 2021
Merged

Workers submit a reply to the scheduler if replica removal was rejected #5356

crusaderky merged 8 commits into from
Sep 28, 2021

Conversation

@fjetter
Copy link
Member

@fjetter fjetter commented Sep 27, 2021

Closes #5265

@fjetter fjetter requested a review from crusaderky September 27, 2021 12:07
@fjetter fjetter mentioned this pull request Sep 28, 2021
Merged
@crusaderky
Copy link
Collaborator

crusaderky commented Sep 28, 2021 •
edited
Loading

Could you do the following changes to test_worker.py:

  1. change _remove_replicas to immediately delete the server side key
  2. change test_remove_replica_while_computing to test that the server side key disappears immediately and then reappears after a while
  3. change test_remove_replica_simple to test that the key does not reappear after a while

crusaderky
crusaderky reviewed Sep 28, 2021
View reviewed changes
@fjetter @crusaderky
Update distributed/tests/test_worker.py
ac9b7f4 
Co-authored-by: crusaderky <crusaderky@gmail.com>
fjetter
fjetter commented Sep 28, 2021
View reviewed changes
crusaderky
crusaderky reviewed Sep 28, 2021
View reviewed changes
crusaderky
crusaderky reviewed Sep 28, 2021
View reviewed changes
fjetter and others added 3 commits September 28, 2021 16:51
@fjetter @crusaderky
Update distributed/tests/test_worker.py
846d7f8 
Co-authored-by: crusaderky <crusaderky@gmail.com>
@fjetter
Merge remote-tracking branch 'origin/main' into gh5265/race_condition…
2ffcd98 
…_remove_replica
@fjetter
Merge branch 'gh5265/race_condition_remove_replica' of github.com:fje…
1b88870 
…tter/distributed into gh5265/race_condition_remove_replica
@crusaderky crusaderky merged commit 2701d4a into dask:main Sep 28, 2021
@fjetter fjetter deleted the gh5265/race_condition_remove_replica branch September 30, 2021 11:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@crusaderky crusaderky crusaderky approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Race condition in remove-replicas could make you lose the last replica of a key

2 participants

@fjetter @crusaderky