KAFKA-18026: KIP-1112, clean up StatefulProcessorNode#18195
KAFKA-18026: KIP-1112, clean up StatefulProcessorNode#18195ableegoldman merged 3 commits intoapache:trunkfrom
Conversation
ableegoldman
force-pushed
the
KIP-1112-StatefulProcessorNode-cleanup
branch
from
c46f9a4 to
af0931d
Compare
|
@ableegoldman #17881 adds a "triage" label to PRs from non-committers. Turns out this also affect committers if their membership visibility in the ASF GitHub org is not public. I added instructions for setting your membership visibility to public https://github.com/apache/kafka/blob/trunk/.github/workflows/README.md#pr-triage |
ableegoldman
force-pushed
the
KIP-1112-StatefulProcessorNode-cleanup
branch
from
af0931d to
8c061b7
Compare
This comment was marked as outdated.
This comment was marked as outdated.
ableegoldman
force-pushed
the
KIP-1112-StatefulProcessorNode-cleanup
branch
from
35c459a to
a3238e3
Compare
guozhangwang
left a comment
guozhangwang
left a comment
There was a problem hiding this comment.
Thanks! Made a pass. Please feel free to merge after incorporating the unit test comment.
| new ProcessorGraphNode<>( | ||
| aggFunctionName, | ||
| new ProcessorParameters<>(aggregateSupplier, aggFunctionName), | ||
| new String[] {storeFactory.storeName()} |
There was a problem hiding this comment.
This is for line 57 above and independent from the PR: just a thought, could we pass in storeFactory.storeName() in the future?
There was a problem hiding this comment.
As a matter of fact, I do exactly this in one of the followup PRs: https://github.com/apache/kafka/pull/18342/files#diff-f3bff5bc56202b353e1715217ebe7fa0c59b3622e3c7c4703f47068cd0803165R70
| final KTableValueGetterSupplier<KO, VO> foreignKeyValueGetter = ((KTableImpl<KO, VO, VO>) foreignKeyTable).valueGetterSupplier(); | ||
| final StatefulProcessorNode<CombinedKey<KO, K>, Change<ValueAndTimestamp<SubscriptionWrapper<K>>>> subscriptionJoinNode = | ||
| new StatefulProcessorNode<>( | ||
| final ProcessorToStateConnectorNode<CombinedKey<KO, K>, Change<ValueAndTimestamp<SubscriptionWrapper<K>>>> subscriptionJoinNode = |
There was a problem hiding this comment.
Originally I thought I do not need to use the newly introduced ProcessorToStateConnectorNode for this case but only needed for those process/transform with a store list, but thinking that again we probably do not have another way around at the moment..
There was a problem hiding this comment.
I had the same thought actually. I did take a quick look at it but ultimately decided that whether or not it was possible, it would be too much for this one PR. So I'm going to revisit this in a followup PR if it does indeed make sense to do
Final cleanup of StatefulProcessorNode after converting all stateful operators to adding state stores via implementing the #stores method.
|
Merged to trunk and cherrypicked to 4.0 |
…18342) Minor followup to #18195 that I split out into a separate PR since that one was getting a bit long. Should be rebased & reviewed after that one is merged. Introduces a new class for windowed graph nodes with a grace period defined to improve (slightly) the type safety Reviewers: Guozhang Wang <guozhang.wang.us@gmail.com>, Almog Gavra <almog@responsive.dev>
Final cleanup of StatefulProcessorNode after converting all stateful operators to adding state stores via implementing the #stores method.
…pache#18342) Minor followup to apache#18195 that I split out into a separate PR since that one was getting a bit long. Should be rebased & reviewed after that one is merged. Introduces a new class for windowed graph nodes with a grace period defined to improve (slightly) the type safety Reviewers: Guozhang Wang <guozhang.wang.us@gmail.com>, Almog Gavra <almog@responsive.dev>
Final cleanup of StatefulProcessorNode after converting all stateful operators to adding state stores via implementing the #stores method.
…pache#18342) Minor followup to apache#18195 that I split out into a separate PR since that one was getting a bit long. Should be rebased & reviewed after that one is merged. Introduces a new class for windowed graph nodes with a grace period defined to improve (slightly) the type safety Reviewers: Guozhang Wang <guozhang.wang.us@gmail.com>, Almog Gavra <almog@responsive.dev>
Final cleanup of StatefulProcessorNode after converting all stateful operators to adding state stores via implementing the #stores method.
…pache#18342) Minor followup to apache#18195 that I split out into a separate PR since that one was getting a bit long. Should be rebased & reviewed after that one is merged. Introduces a new class for windowed graph nodes with a grace period defined to improve (slightly) the type safety Reviewers: Guozhang Wang <guozhang.wang.us@gmail.com>, Almog Gavra <almog@responsive.dev>
3 participants
Final cleanup of StatefulProcessorNode after converting all stateful operators to adding state stores via implementing the
#storesmethod. This PR can't be merged until we merge these two open PRs: