Skip to content

Conversation

@mrocklin
Copy link
Member

@mrocklin mrocklin commented Nov 11, 2016

The core.write coroutine waits on IOStream.write futures. However, there are
some cases where IOStream.write futures don't complete. Notably if something
else writes to the same stream before we're done waiting then the first future
is forgotten and hangs forever.

If `write` is called again before that `.Future` has resolved, the
previous future will be orphaned and will never resolve.

This solution wraps the wait in a timeout and checks the stream's
write_buffer as a secondary check for completion. There are probably better
solutions that don't rely on the timeout but this should work fine.

cc @pitrou

The core.write coroutine waits on IOStream.write futures.  However, there are
some cases where IOStream.write futures don't complete.  Notably if something
else writes to the same stream before we're done waiting then the first future
is forgotten and hangs forever.

    If `write` is called again before that `.Future` has resolved, the
    previous future will be orphaned and will never resolve.

This solution wraps the wait in a timeout and checks the stream's
write_buffer as a secondary check for completion.  There are probably better
solutions but this seems to work fine.
This was recently moved to core.write
@mrocklin mrocklin merged commit dbb5c79 into dask:master Nov 11, 2016
@mrocklin mrocklin deleted the timeout-on-write branch November 11, 2016 15:32
@mrocklin mrocklin mentioned this pull request Nov 12, 2016
pitrou added a commit to pitrou/distributed that referenced this pull request Nov 14, 2016
Does away with the timeout and looking up a private attribute on IOStream.
Refs PR dask#653.
pitrou added a commit to pitrou/distributed that referenced this pull request Nov 14, 2016
Does away with the timeout and looking up a private attribute on IOStream.
Refs PR dask#653.
pitrou added a commit to pitrou/distributed that referenced this pull request Nov 14, 2016
Does away with the timeout and looking up a private attribute on IOStream.
Refs PR dask#653.
@pitrou pitrou mentioned this pull request Nov 14, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant