Fix lz4 library incompatibility in kafka-indexing-service extension#4115
Fix lz4 library incompatibility in kafka-indexing-service extension#4115gianm merged 3 commits intoapache:masterfrom
Conversation
satishbhor
changed the title
satishbhor
changed the title
gianm
left a comment
gianm
left a comment
There was a problem hiding this comment.
This will break compatibility with 0.9 brokers, but I suppose I'm okay with that. Kafka makes it tough to support old versions since they keep breaking wire compatibility, so I guess there's not much we can do.
this extension is still experimental so it's ok to make the change in a point release.
@satishbhor could you please update the Kafka indexing docs to reflect that users' brokers must be 0.10.x or higher?
LGTM after that and the version bump to 0.10.2.0 (assuming that version works too)
| <groupId>org.apache.kafka</groupId> | ||
| <artifactId>kafka_2.11</artifactId> | ||
| <version>0.9.0.0</version> | ||
| <version>0.10.0.0</version> |
There was a problem hiding this comment.
0.10.2.0 is the latest, please use that if it works.
There was a problem hiding this comment.
@gianm Thanks! I will update Kafka indexing service document and will also check with Kafka 0.10.2.0
|
@satishbhor Btw, if you haven't already filled out a CLA, could you please do so at: http://druid.io/community/cla.html |
…y in kafka-indexing-service extension apache#3266
|
@gianm with minor changes in unit test cases it is working fine with Kafka 0.10.2.0 and also updated Kafka indexing service 0.10.x note in docs. |
| final long nextOffset = outOfRangePartition.getValue(); | ||
| // seek to the beginning to get the least available offset | ||
| consumer.seekToBeginning(topicPartition); | ||
| consumer.seekToBeginning(Lists.newArrayList(topicPartition)); |
There was a problem hiding this comment.
Could use Collections.singletonList()?
There was a problem hiding this comment.
This file contains Lists.newArrayList() other places as well, so shall I replace those as well ?
There was a problem hiding this comment.
The other place uses Iterable as argument, it's a different case
There was a problem hiding this comment.
I mean to say statements like,
consumer.assign(Lists.newArrayList(topicPartition));There was a problem hiding this comment.
Also in #4135 you mentioned prefer Collections.emptyList() over Arrays.asList() then shall i also replace Lists.newArrayList() with Collections.emptyList() ?
There was a problem hiding this comment.
@leventov my apologize Collections.emptyList() can't be used as lists are being manipulated in code.
|
|
||
| if (useEarliestOffset) { | ||
| consumer.seekToBeginning(topicPartition); | ||
| consumer.seekToBeginning(Lists.newArrayList(topicPartition)); |
…x lz4 library incompatibility in kafka-indexing-service extension apache#4115
gianm
left a comment
gianm
left a comment
There was a problem hiding this comment.
👍 thanks for the contribution!
|
Thanks @gianm for accepting changes. |
…gress (fixes #4214) (#4240) * Fix lz4 library incompatibility in kafka-indexing-service extension #3266 * Bumped Kafka version to 0.10.2.0 for : Fix lz4 library incompatibility in kafka-indexing-service extension #3266 * Replaced Lists.newArrayList() with Collections.singletonList() For Fix lz4 library incompatibility in kafka-indexing-service extension #4115 * Fixed: Average Server Percent Used: NaN% Error when server startup is in progress #4214
…exing-service extension) * Fix lz4 library incompatibility in kafka-indexing-service extension apache#3266 * Bumped Kafka version to 0.10.2.0 for : Fix lz4 library incompatibility in kafka-indexing-service extension apache#3266 * Replaced Lists.newArrayList() with Collections.singletonList() For Fix lz4 library incompatibility in kafka-indexing-service extension apache#4115
3 participants
Added support for Kafka 0.10.0.0 lz4 so that #3266 issue can be resolved.