Transmit Layer annotations to scheduler

[sjperkins]

No description provided.

[sjperkins]

Depends on dask/dask#6889

[mrocklin]

I have an alternative here: #4288

It's not quite as comprehensive (annotations other than priority/retires/workers don't get transmitted) but it's simple and this approach covers what we can do today I think.

[mrocklin]

A few comments. Some are somewhat pedantic. I hope that that is ok.

[sjperkins]

A few comments. Some are somewhat pedantic. I hope that that is ok.

No complaints here. The above makes sense to me and I think I understand where you're coming from, as I've been labelled "fussy about how things are done" upon occasion. ;-)

[mrocklin]

This all seems good to me. I'm happy to merge.

For future work (or now, if you feel like you're on a roll) we might consider assigning the annotations to the TaskState objects, such that the following might work:

>>> ts.annotations
{"foo": 1}

The test failure here seems unrelated. I'll push up a separate PR with a fix.

[sjperkins]

For future work (or now, if you feel like you're on a roll) we might consider assigning the annotations to the TaskState objects, such that the following might work:

>>> ts.annotations
{"foo": 1}

Done.

One of the future additions I'm considering is transmitting annotations to the workers, but I haven't looked into any of the details yet. I think it would be good to leave that to the scope of a new PR.

[jakirkham]

It looks like this checked an empty file, distributed/tests/test_highgraph.py. Is that needed for some reason or should we drop that?

[jrbourbeau]

Thanks for catching that, I believe it can be dropped