Skip to content

Potential race condition in scheduler? #2554

@nishanttotla

Description

@nishanttotla

I noticed a few instances in manager/scheduler/scheduler.go where some checks seem suspect. In particular, we're doing something like

// Ignore all tasks that have not reached PENDING
		// state and tasks that no longer consume resources.
		if t.Status.State < api.TaskStatePending || t.Status.State > api.TaskStateRunning {
			continue
		}

in three places:

  1. https://github.com/docker/swarmkit/blob/master/manager/scheduler/scheduler.go#L71

  2. https://github.com/docker/swarmkit/blob/master/manager/scheduler/scheduler.go#L206

  3. https://github.com/docker/swarmkit/blob/master/manager/scheduler/scheduler.go#L243

From the comment, it seems like we're assuming that tasks with current state > RUNNING do not consume resources, which is no longer true with #2461. For the brief period where the task reaper hasn't deleted some tasks, I'm trying to figure out if the task reaper could race with the scheduler here.

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions