Garbage Collect Scheduler TaskState

[sjperkins]

Based on #6246

Closes #xxxx

• Tests added / passed
• Passes pre-commit run --all-files

[fjetter]

I suggest to focus first on Scheduler and Worker instances before you tackle TaskState. That's a required step but much easier and we can get it merged incrementally

[sjperkins]

I suggest to focus first on Scheduler and Worker instances before you tackle TaskState. That's a required step but much easier and we can get it merged incrementally

Thanks. By this I think you mean deal with things in the following way in utils.check_instances?

# Fix leaks here first
if Scheduler._instances:
    scheduler = Scheduler._instances.pop()
    import objgraph

    objgraph.show_backrefs[worker], filename="scheduler.png"
assert not Scheduler._instances    

# Fix leaks here second
if Worker._instances:
    worker = Worker._instances.pop()
    import objgraph

    objgraph.show_backrefs[worker], filename="worker.png"
assert not Worker._instances    

# Fix leaks here last
if TaskState._instances:
    ts = TaskState._instances.pop()
    import objgraph

    objgraph.show_backrefs([ts], filename="taskstate.png")```

[sjperkins]

I'll start a new PR to deal with Scheduler instances and then work to Worker and TaskState PR's incrementally.

[sjperkins]

Closed in favour of #6281