KAFKA-3355 by mgarmes · Pull Request #2289 · apache/kafka

mgarmes · 2016-12-22T11:11:27Z

KAFKA-3355
GetOffsetShell command doesn't work with SASL enabled Kafka

GetOffsetShell command doesn't work with SASL enabled Kafka

asfbot · 2016-12-22T11:13:57Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.10/362/
Test FAILed (JDK 7 and Scala 2.10).

asfbot · 2016-12-22T11:14:05Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/363/
Test FAILed (JDK 8 and Scala 2.12).

asfbot · 2016-12-22T11:15:05Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.11/364/
Test FAILed (JDK 8 and Scala 2.11).

ijuma · 2016-12-22T11:26:11Z

Thanks for the PR. Is there a reason to keep the old implementation?

mgarmes · 2016-12-30T15:01:12Z

Hello,

Can someone check this pull request ?

best regard

ijuma · 2016-12-30T15:47:39Z

@garmes-gdev, did you see my question?

mgarmes · 2016-12-31T13:04:59Z

Hello @ijuma,
Thank you very much for your reply.
regarding you question, for my personal opinion I don't want to keep the old version, but I noticed in the others PR, contributors keep the both version so I did the same. if you think is it not useful I can delete it !!

best regards.

ijuma · 2017-01-02T22:05:29Z

@garmes-gdev, the old version only needs to be kept if there's a reason to do so (maybe the new version is missing some functionality or is not as stable). If there is no reason to keep the old version, then we can simply replace it.

mgarmes · 2017-01-05T12:01:14Z

Hello @ijuma,
I eliminated the old version and I updated the tests.

asfbot · 2017-01-05T12:38:46Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/524/
Test PASSed (JDK 8 and Scala 2.12).

asfbot · 2017-01-05T12:41:36Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.11/525/
Test PASSed (JDK 8 and Scala 2.11).

asfbot · 2017-01-05T13:55:45Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.10/523/
Test PASSed (JDK 7 and Scala 2.10).

…me of the offending field. https://issues.apache.org/jira/browse/KAFKA-4709 Author: Aegeaner <xihuke@gmail.com> Reviewers: Dong Lin, Guozhang Wang Closes #2521 from Aegeaner/KAFKA-4709

Author: Jason Gustafson <jason@confluent.io> Reviewers: Ismael Juma <ismael@juma.me.uk> Closes #2536 from hachikuji/minor-move-compression-io-construction

…y usage Lowered the default RocksDB settings for the block cache and write buffers Author: Damian Guy <damian.guy@gmail.com> Reviewers: Eno Thereska, Guozhang Wang Closes #2525 from dguy/kafka-4484

Author: Colin P. Mccabe <cmccabe@confluent.io> Reviewers: Ismael Juma <ismael@juma.me.uk> Closes #2559 from cmccabe/KAFKA-4775

…fault log.retention.ms Author: Jiangjie Qin <becket.qin@gmail.com> Reviewers: Jun Rao <junrao@gmail.com>, Ismael Juma <ismael@juma.me.uk> Closes #2544 from becketqin/KAFKA-4340_follow_up

More details: * Replaced `struct` field in Request/Response with a `toStruct` method. This makes the performance model (including memory usage) easier to understand. Note that requests have `toStruct()` while responses have `toStruct(version)`. * Replaced mutable `version` field in `Request.Builder` with an immutable field `desiredVersion` and a `version` parameter passed to the `build` method. * Optimised `handleFetchRequest` to avoid unnecessary creation of `Struct` instances (from 4 to 2 in the worst case and 2 to 1 in the best case). * Various clean-ups in request/response classes and their test. In particular, it is now clear what we are testing. Previously, it looked like we were testing more than we really were. With this in place, we could remove `AbstractRequest.Builder` in the future by doing the following: * Change `AbstractRequest.toStruct` to accept a version (like responses). * Change `AbstractRequest.version` to be `desiredVersion` (like `Builder`). * Change `ClientRequest` to take `AbstractRequest`. * Move validation from the `build` methods to the request constructors or static factory methods. * Anything else required for the code to compile again. Author: Ismael Juma <ismael@juma.me.uk> Reviewers: Apurva Mehta <apurva.1618@gmail.com>, Jason Gustafson <jason@confluent.io> Closes #2513 from ijuma/separate-struct

Currently the checkpoint file is deleted at state store initialization and it is only ever written again during a clean shutdown. This can result in significant delays during restarts as the entire store needs to be loaded from the changelog. We can mitigate against this by frequently checkpointing the offsets. The checkpointing happens only during the commit phase, i.e, after we have manually flushed the store and the producer. So we guarantee that the checkpointed offsets are never greater than what has been flushed. In the event of hard failure we can recover by reading the checkpoints and consuming from the stored offsets. Author: Damian Guy <damian.guy@gmail.com> Reviewers: Eno Thereska, Matthias J. Sax, Guozhang Wang Closes #2471 from dguy/kafka-4317

Author: Jason Gustafson <jason@confluent.io> Reviewers: Colin P. Mccabe <cmccabe@confluent.io>, Ewen Cheslack-Postava <me@ewencp.org> Closes #2566 from hachikuji/hotfix-request-logging

Author: Prabhat Kashyap <prabhat.kashyap@knoldus.in> Reviewers: Ismael Juma <ismael@juma.me.uk> Closes #2530 from PKOfficial/refactored-code

The throttling system test sometimes fail because it takes longer than the current 10 second time out for partitions to get assigned to the consumer. Author: Apurva Mehta <apurva@confluent.io> Reviewers: Ismael Juma <ismael@juma.me.uk> Closes #2567 from apurvam/increase-timeout-for-partitions-assigned

…lass should be static Author: Colin P. Mccabe <cmccabe@confluent.io> Reviewers: Jason Gustafson <jason@confluent.io>, Ismael Juma <ismael@juma.me.uk> Closes #2558 from cmccabe/KAFKA-4774

…rs are available Author: Allen Xiang <allen.xiang@monsanto.com> Reviewers: Jason Gustafson <jason@confluent.io> Closes #2564 from allenxiang/client-heartbeat-fix

Fixed ClassCastException resulting from missing type hint in request logging. Author: Armin Braun <me@obrown.io> Reviewers: Jason Gustafson <jason@confluent.io> Closes #2571 from original-brownbear/fix-logging-err-response

* Turned 3 inner classes that weren't static but could be into `static` ones. * Turned one `public` inner class that wasn't used publicly into a `private`. Trivial but imo worthwhile to explicitly keep visibility and "staticness" correct in syntax (if only to be nice to the GC) :) Author: Armin Braun <me@obrown.io> Reviewers: Matthias J. Sax <matthias@confluent.io>, Jason Gustafson <jason@confluent.io> Closes #2574 from original-brownbear/cleanup-inner-nonstatic

This commmit brings improved test coverage for window store fetch method and WindowStoreIterator Author: Andrey Dyachkov <andrey.dyachkov@zalando.de> Reviewers: Damian Guy, Guozhang Wang Closes #2672 from adyach/trunk

Both the headers and requests have regressed to just show object ids instead of their contents from their underlying structs. I'm guessing this regression came from commit [fc1cfe4](fc1cfe4) Author: Onur Karaman <okaraman@linkedin.com> Reviewers: Jiangjie Qin <becket.qin@gmail.com> Closes #2678 from onurkaraman/KAFKA-4891

hachikuji Author: Guozhang Wang <wangguoz@gmail.com> Reviewers: Jason Gustafson <jason@confluent.io> Closes #2679 from guozhangwang/KHotfix-serde-headers

Make sure that the iterator returned from `WindowStore.fetch(..)` only returns matching keys, rather than all keys that are a prefix match. Author: Damian Guy <damian.guy@gmail.com> Reviewers: Eno Thereska, Guozhang Wang Closes #2662 from dguy/kafka-4863

Author: Ewen Cheslack-Postava <me@ewencp.org> Reviewers: Guozhang Wang <wangguoz@gmail.com> Closes #2599 from ewencp/kafka-4783-byte-array-converter

asfbot · 2017-03-15T13:46:30Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.10/2191/
Test FAILed (JDK 7 and Scala 2.10).

asfbot · 2017-03-15T13:46:37Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/2192/
Test FAILed (JDK 8 and Scala 2.12).

asfbot · 2017-03-15T13:46:49Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.11/2194/
Test FAILed (JDK 8 and Scala 2.11).

Author: Ismael Juma <ismael@juma.me.uk> Reviewers: Manikumar reddy O <manikumar.reddy@gmail.com> Closes #2690 from ijuma/fix-header-in-byte-array-converter

Author: Guozhang Wang <wangguoz@gmail.com> Reviewers: Matthias J. Sax <matthias@confluent.io>, Eno Thereska <eno@confluent.io>, Damian Guy <damian.guy@gmail.com>, Jason Gustafson <jason@confluent.io> Closes #2685 from guozhangwang/KMinor-improve-log4j

Author: Vahid Hashemian <vahidhashemian@us.ibm.com> Reviewers: Sriharsha Chintalapani <harsha@hortonworks.com> Closes #2674 from vahidhashemian/minor/fix_typos_1703

asfbot · 2017-03-16T08:33:00Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.11/2214/
Test FAILed (JDK 8 and Scala 2.11).

asfbot · 2017-03-16T08:33:24Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/2212/
Test FAILed (JDK 8 and Scala 2.12).

asfbot · 2017-03-16T08:33:32Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.10/2211/
Test FAILed (JDK 7 and Scala 2.10).

asfbot · 2017-03-16T11:22:24Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.11/2217/
Test FAILed (JDK 8 and Scala 2.11).

asfbot · 2017-03-16T11:22:31Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/2215/
Test FAILed (JDK 8 and Scala 2.12).

asfbot · 2017-03-16T11:22:40Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.10/2214/
Test FAILed (JDK 7 and Scala 2.10).

asfbot · 2017-03-16T13:59:28Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.11/2220/
Test PASSed (JDK 8 and Scala 2.11).

asfbot · 2017-03-16T14:03:46Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.10/2217/
Test PASSed (JDK 7 and Scala 2.10).

asfbot · 2017-03-16T14:28:53Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/2218/
Test PASSed (JDK 8 and Scala 2.12).

ppanero · 2017-04-18T10:02:05Z

Hello, is there any intention to merge/work more on this? @ijuma
Thanks in advance

mgarmes · 2017-04-21T09:34:04Z

Hello @ppanero,
a provide the fix for kafka 10 and is not merged so is no more compatible with 10.2 ! so I will try to provide the fix for this new version and I hop this time that someone merge it

Bests

ppanero · 2017-04-21T09:36:16Z

Thanks @garmes-gdev let me know if u need any help and I could contribute :D

mgarmes · 2017-04-21T21:37:02Z

@ppanero #2891

mgarmes added 2 commits December 20, 2016 15:42

Merge remote-tracking branch 'refs/remotes/apache/trunk'

7544b53

KAFKA-3355

d38df2a

GetOffsetShell command doesn't work with SASL enabled Kafka

mgarmes changed the title ~~Master~~ KAFKA-3355 Dec 26, 2016

KAFKA-3355 GetOffsetShell command doesn't work with SASL enabled.

a36aa06

Aegeaner and others added 14 commits February 16, 2017 13:44

KAFKA-4709：Error message from Struct.validate() should include the na…

6c83939

…me of the offending field. https://issues.apache.org/jira/browse/KAFKA-4709 Author: Aegeaner <xihuke@gmail.com> Reviewers: Dong Lin, Guozhang Wang Closes #2521 from Aegeaner/KAFKA-4709

MINOR: Move compression stream construction into CompressionType

022d201

Author: Jason Gustafson <jason@confluent.io> Reviewers: Ismael Juma <ismael@juma.me.uk> Closes #2536 from hachikuji/minor-move-compression-io-construction

KAFKA-4484: Set more conservative default values on RocksDB for memor…

505af6b

…y usage Lowered the default RocksDB settings for the block cache and write buffers Author: Damian Guy <damian.guy@gmail.com> Reviewers: Eno Thereska, Guozhang Wang Closes #2525 from dguy/kafka-4484

KAFKA-4775; Fix findbugs warnings in kafka-tools

a887379

Author: Colin P. Mccabe <cmccabe@confluent.io> Reviewers: Ismael Juma <ismael@juma.me.uk> Closes #2559 from cmccabe/KAFKA-4775

KAFKA-4340; Follow-up fixing system test failures and handling non de…

1f2ee5f

…fault log.retention.ms Author: Jiangjie Qin <becket.qin@gmail.com> Reviewers: Jun Rao <junrao@gmail.com>, Ismael Juma <ismael@juma.me.uk> Closes #2544 from becketqin/KAFKA-4340_follow_up

HOTFIX: ClassCastException in request logging

a515e93

Author: Jason Gustafson <jason@confluent.io> Reviewers: Colin P. Mccabe <cmccabe@confluent.io>, Ewen Cheslack-Postava <me@ewencp.org> Closes #2566 from hachikuji/hotfix-request-logging

MINOR: Replace for with foreach loop in common module

d734f4e

Author: Prabhat Kashyap <prabhat.kashyap@knoldus.in> Reviewers: Ismael Juma <ismael@juma.me.uk> Closes #2530 from PKOfficial/refactored-code

KAFKA-4774; Inner classes which don't need a reference to the outer c…

916edc3

…lass should be static Author: Colin P. Mccabe <cmccabe@confluent.io> Reviewers: Jason Gustafson <jason@confluent.io>, Ismael Juma <ismael@juma.me.uk> Closes #2558 from cmccabe/KAFKA-4774

KAFKA-4777; Backoff properly in consumer heartbeat thread if no broke…

dcea498

…rs are available Author: Allen Xiang <allen.xiang@monsanto.com> Reviewers: Jason Gustafson <jason@confluent.io> Closes #2564 from allenxiang/client-heartbeat-fix

HOTFIX: ClassCastException in Request error logging

d24d932

Fixed ClassCastException resulting from missing type hint in request logging. Author: Armin Braun <me@obrown.io> Reviewers: Jason Gustafson <jason@confluent.io> Closes #2571 from original-brownbear/fix-logging-err-response

adyach and others added 6 commits March 13, 2017 12:05

KAFKA-4657: Improve test coverage of CompositeReadOnlyWindowStore

85e7a0a

This commmit brings improved test coverage for window store fetch method and WindowStoreIterator Author: Andrey Dyachkov <andrey.dyachkov@zalando.de> Reviewers: Damian Guy, Guozhang Wang Closes #2672 from adyach/trunk

HOTFIX: Fix apache headers in float serde class files

d3b8ff0

hachikuji Author: Guozhang Wang <wangguoz@gmail.com> Reviewers: Jason Gustafson <jason@confluent.io> Closes #2679 from guozhangwang/KHotfix-serde-headers

KAFKA-4783: Add ByteArrayConverter (KIP-128)

52a15d7

Author: Ewen Cheslack-Postava <me@ewencp.org> Reviewers: Guozhang Wang <wangguoz@gmail.com> Closes #2599 from ewencp/kafka-4783-byte-array-converter

KAFKA 3355 update

7a29ed3

ijuma and others added 4 commits March 15, 2017 14:46

HOTFIX: Fix header in ByteArrayConverter

962c378

Author: Ismael Juma <ismael@juma.me.uk> Reviewers: Manikumar reddy O <manikumar.reddy@gmail.com> Closes #2690 from ijuma/fix-header-in-byte-array-converter

MINOR: Fix a documentation typo

be11272

Author: Vahid Hashemian <vahidhashemian@us.ibm.com> Reviewers: Sriharsha Chintalapani <harsha@hortonworks.com> Closes #2674 from vahidhashemian/minor/fix_typos_1703

fix and tests (status: PASS)

7c38fc0

mgarmes added 2 commits March 16, 2017 12:10

rebase

c659e8e

Merge branch 'master' of https://github.com/garmes-gdev/kafka

d1bc11f

fix

3e8bfca

mgarmes closed this Mar 16, 2017

Conversation

mgarmes commented Dec 22, 2016

Uh oh!

asfbot commented Dec 22, 2016

Uh oh!

asfbot commented Dec 22, 2016

Uh oh!

asfbot commented Dec 22, 2016

Uh oh!

ijuma commented Dec 22, 2016

Uh oh!

mgarmes commented Dec 30, 2016

Uh oh!

ijuma commented Dec 30, 2016

Uh oh!

mgarmes commented Dec 31, 2016

Uh oh!

ijuma commented Jan 2, 2017

Uh oh!

mgarmes commented Jan 5, 2017

Uh oh!

asfbot commented Jan 5, 2017

Uh oh!

asfbot commented Jan 5, 2017

Uh oh!

asfbot commented Jan 5, 2017

Uh oh!

asfbot commented Mar 15, 2017

Uh oh!

asfbot commented Mar 15, 2017

Uh oh!

asfbot commented Mar 15, 2017

Uh oh!

asfbot commented Mar 16, 2017

Uh oh!

asfbot commented Mar 16, 2017

Uh oh!

asfbot commented Mar 16, 2017

Uh oh!

asfbot commented Mar 16, 2017

Uh oh!

asfbot commented Mar 16, 2017

Uh oh!

asfbot commented Mar 16, 2017

Uh oh!

asfbot commented Mar 16, 2017

Uh oh!

asfbot commented Mar 16, 2017

Uh oh!

asfbot commented Mar 16, 2017

Uh oh!

ppanero commented Apr 18, 2017

Uh oh!

mgarmes commented Apr 21, 2017

Uh oh!

ppanero commented Apr 21, 2017

Uh oh!

mgarmes commented Apr 21, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants