Skip to content

Missing test: duplicates in Worker.ready #6096

@crusaderky

Description

@crusaderky

Follow-up from #6062

  1. A task is assigned to a Worker
  2. The task transitions to ready, thus is pushed into the Worker.ready heap, or to constrained, thus is pushed into the Worker.constrained heap
  3. Before it reaches computing state, it is stolen. As there's no way to pop from the middle of a heap, the task stays there.
  4. The task is assigned to the worker again before Worker._ensure_computing could reach its position in the heap
  5. You end up with a duplicate key in the heap.

There's nothing wrong with having a duplicate per se, and Worker._ensure_computing gracefully deals with it; however the use case is far from trivial. At the moment of writing, replacing the graceful handling with a hard assertion of no duplicates only causes very mild flakiness in the most demanding stress tests.

Write a unit test that deterministically triggers the use case both with and without resource constraints.

CC @fjetter

Metadata

Metadata

Assignees

No one assigned

    Labels

    testsUnit tests and/or continuous integration

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions