KAFKA-3418: add javadoc section describing consumer failure detection#1129
KAFKA-3418: add javadoc section describing consumer failure detection#1129hachikuji wants to merge 1 commit into
Conversation
|
cc @jkreps |
|
LGTM |
| * </ol> | ||
| * <p> | ||
| * For use cases where message processing time varies unpredictably, neither of these options may be viable. | ||
| * The recommended way to handle these cases is to move message processing to another thread, which allows |
There was a problem hiding this comment.
Is it correct that another downside of this approach is that the foreground thread could end up buffering too much data as we have no way to apply backpressure?
There was a problem hiding this comment.
@ijuma I think this question is addressed below -- pausing partitions in the network thread effectively is backpressure and turns the network thread into a pure heartbeating thread while processing is being performed. You can also, of course, choose to buffer as much or as little as you want by adjusting when you decide to pause the collection.
I'd say the current docs explain this well enough, though I think a few code examples (in the somewhat defunct examples jar) would be the most helpful way to show how to make this work in practice.
There was a problem hiding this comment.
Makes sense @ewencp, not sure how I missed that sentence when I read it originally.
|
LGTM. We may want to move some of this into the normal docs given how large this part of the javadocs is getting, but better to get this in somewhere for the next release. |
Author: Jason Gustafson <jason@confluent.io> Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>, Ismael Juma <ismael@juma.me.uk>, Ewen Cheslack-Postava <ewen@confluent.io> Closes apache#1129 from hachikuji/KAFKA-3418
4 participants
No description provided.