Skip to content

KAFKA-6894: Improve err msg when explicitly connecting processor to global store#5000

Merged
guozhangwang merged 1 commit intoapache:trunkfrom
rayokota:kafka-6894-global-store-in-transform
May 10, 2018
Merged

KAFKA-6894: Improve err msg when explicitly connecting processor to global store#5000
guozhangwang merged 1 commit intoapache:trunkfrom
rayokota:kafka-6894-global-store-in-transform

Conversation

@rayokota
Copy link
Copy Markdown
Contributor

@rayokota rayokota commented May 10, 2018 •
edited
Loading

I've improved the error message when explicitly trying to connect a processor to a global store as in this example:

      builder.globalTable("supersteps", Consumed.with(Serdes.Integer(), Serdes.Long()),
                        Materialized.<Integer, Long, KeyValueStore<Bytes, byte[]>>as("superstepStore")
                                .withKeySerde(Serdes.Integer()).withValueSerde(Serdes.Long()));

      stream.transform((TransformerSupplier<K, Tuple3<Integer, VV, Map<K, Message>>, KeyValue<Integer, Long>>)
                        InitialTransformer::new, "superstepStore");

@rayokota rayokota changed the title KAFKA-6894: Fix InternalTopologyBuilder.connectProcessAndState for global stores KAFKA-6894: Fix InternalTopologyBuilder.connectProcessorAndState for global stores May 10, 2018
@rayokota rayokota force-pushed the kafka-6894-global-store-in-transform branch 3 times, most recently from 8616cba to edc652e Compare May 10, 2018 19:37
guozhangwang
guozhangwang reviewed May 10, 2018
View reviewed changes
Comment thread streams/src/test/java/org/apache/kafka/test/NoOpKeyValueStore.java Outdated
Copy link
Copy Markdown
Contributor

@guozhangwang guozhangwang May 10, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we enhance MockStateStore instead?

Comment thread streams/src/test/java/org/apache/kafka/test/MockGlobalStateStoreSupplier.java Outdated
Copy link
Copy Markdown
Contributor

@guozhangwang guozhangwang May 10, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are removing the deprecated state store supplier in #4991, could you use MockStoreBuilder instead?

@mjsax mjsax added the streams label May 10, 2018
@rayokota rayokota force-pushed the kafka-6894-global-store-in-transform branch from edc652e to 3b37ab5 Compare May 10, 2018 21:02
@rayokota rayokota changed the title KAFKA-6894: Fix InternalTopologyBuilder.connectProcessorAndState for global stores KAFKA-6894: Improve err msg when explicitly connecting processor to global store May 10, 2018
mjsax
mjsax approved these changes May 10, 2018
View reviewed changes
Copy link
Copy Markdown
Member

@mjsax mjsax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@guozhangwang guozhangwang merged commit f69900c into apache:trunk May 10, 2018
@rayokota rayokota deleted the kafka-6894-global-store-in-transform branch May 11, 2018 00:17
ijuma added a commit to ijuma/kafka that referenced this pull request May 11, 2018
@ijuma
Merge remote-tracking branch 'apache-github/trunk' into record-format…
99f469b 
…-record-version

* apache-github/trunk:
  KAFKA-6894: Improve err msg when connecting processor with global store (apache#5000)
  KAFKA-6893; Create processors before starting acceptor in SocketServer (apache#4999)
  MINOR: Fix typo in ConsumerRebalanceListener JavaDoc (apache#4996)
  MINOR: Remove deprecated valueTransformer.punctuate (apache#4993)
  MINOR: Update dynamic broker configuration doc for truststore update (apache#4954)
  KAFKA-6870 Concurrency conflicts in SampledStat (apache#4985)
  KAFKA-6361: Fix log divergence between leader and follower after fast leader fail over (apache#4882)
  KAFKA-6813: Remove deprecated APIs in KIP-182, Part II (apache#4976)
  KAFKA-6878 Switch the order of underlying.init and initInternal (apache#4988)
  KAFKA-6299; Fix AdminClient error handling when metadata changes (apache#4295)
  KAFKA-6878: NPE when querying global state store not in READY state (apache#4978)
  KAFKA 6673: Implemented missing override equals method (apache#4745)
  KAFKA-6834: Handle compaction with batches bigger than max.message.bytes (apache#4953)
ying-zheng pushed a commit to ying-zheng/kafka that referenced this pull request Jul 6, 2018
@rayokota @ying-zheng
KAFKA-6894: Improve err msg when connecting processor with global sto…
ffece8e 
…re (apache#5000)

Reviewers: Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@guozhangwang guozhangwang guozhangwang left review comments

@mjsax mjsax mjsax approved these changes

Assignees

No one assigned

Labels

streams

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@rayokota @mjsax @guozhangwang