KAFKA-1686: Implement SASL/Kerberos.#191
Conversation
|
Anyone interested running this on vagrant , build kafka with this patch in and checkout https://github.com/harshach/kafka-vagrant . Drop the .tgz under kafka-vagrant. Vagrant up. |
|
kafka-trunk-git-pr #340 FAILURE |
|
kafka-trunk-git-pr #344 FAILURE |
|
kafka-trunk-git-pr #345 FAILURE |
|
Thanks Harsha, good to see this. In order to review this, it would be good to have a summary of the status of the PR, what areas (if any) you are still working on? |
|
@ijuma cleaning up the code and adding sasl configs part, more unit tests. |
|
Thanks. |
There was a problem hiding this comment.
We should probably use 2.7.1, right?
|
I pushed the inter-broker tests for SASL_PLAINTEXT and SASL_SSL to harshach#5 Working on SASL_SSL tests for producer and consumer now. |
Introduce LoginType and refactor KerberosName in order to make `authToLocal` rules configurable
There was a problem hiding this comment.
Shouldn't this be a daemon thread? Otherwise it would prevent client applications from terminating.
There was a problem hiding this comment.
@junrao suggested that it shouldn't be because we wait for it to terminate on shutdown. And if consumers are closed, then it won't prevent client applications from terminating. But it may cause this problem when consumers are not closed, so I am tempted to change it back to a daemon thread. What are your thoughts Jun?
* apache/trunk: (21 commits) KAFKA-2665: Add images to code github KAFKA-2397: add leave group request to force coordinator trigger rebalance KAFKA-2515: Handle oversized messages properly in new consumer KAFKA-2484: Add schema projection utilities KAFKA-2594: Add InMemoryLRUCacheStore as a preliminary method for bounding in-memory stores KAFKA-2419 - Fix to prevent background thread from getting created when not required KAFKA-2120: cleaning follow-up KAFKA-2487: change kafka.examples.Consumer to use the new java consumer MINOR: set up temp directories properly in StreamTaskTest KAFKA-2295; Support dynamically loaded classes from context class loader KAFKA-2654: optimize unnecessary poll(0) away in StreamTask TRIVIAL: add @throws ConsumerWakeupException in KafkaConsumer KAFKA-2536: topics tool should allow users to alter topic configuration KAFKA-2650: Change ConfigCommand --deleted-config option to align wit… KAFKA-2593: Key value stores can use specified serializers and deserializers KAFKA-2603: Add timeout arg to ConsoleConsumer for new consumer MINOR: ignore wakeups when committing offsets on consumer close KAFKA-2632: move fetchable check ahead in handleFetchResponse MINOR: flush record collector after local state flush KAFKA-2527; System Test for Quotas in Ducktape ...
…d fix bug in `ClientUtils.createChannelBuilder`
…ocols Include improvements from `SaslIntegrationTest` and remove duplicated code.
As pointed out by Jun
* apache/trunk: MINOR: Fixed README examples on running specific tests. KAFKA-2639: Refactoring of ZkUtils
|
@harshach, PR 6 is ready harshach#6:
We need to move fast in order to get this into 0.9.0.0, so I'd appreciate it if you could merge this into your branch so that Jun can review it. |
There was a problem hiding this comment.
@harshach, this is not actually used at the moment. Can you please point me to where it should be used and I can quickly address it?
* apache/trunk: HOTFIX: check logic of KAFKA-2515 should be on buffer.limit() MINOR: Capture stderr in ConsumerPerformanceService. KAFKA-2669; Fix LogCleaner.awaitCleaned for LogCleanerIntegrationTest KAFKA-2656; Remove hardcoded default key and truststores MINOR: Reduce logging level for controller connection failures from `error` to `warn`
This seems to be necessary for our tests to pass consistently.
`Sasl*ConsumerTest` already tests similar scenarios.
This PR implements SASL/Kerberos which was originally submitted by harshach as #191. I've been submitting PRs to Harsha's branch with fixes and improvements and he has integrated all, but the most recent one. I'm creating this PR so that the Jenkins can run the tests on the branch (they pass locally). Author: Ismael Juma <ismael@juma.me.uk> Author: Sriharsha Chintalapani <harsha@hortonworks.com> Author: Harsha <harshach@users.noreply.github.com> Reviewers: Ismael Juma <ismael@juma.me.uk>, Rajini Sivaram <rajinisivaram@googlemail.com>, Parth Brahmbhatt <brahmbhatt.parth@gmail.com>, Jun Rao <junrao@gmail.com> Closes #334 from ijuma/KAFKA-1686-V1
|
@ijuma can you open a PR against this branch so that I can merge it in |
Merge trunk, refactor consumer/producer tests and other fixes and improvements
|
Thanks for merging the PR to your branch Harsha. Since Jun merged the SASL PR via #334 (which is exactly the same as this one now), would you mind closing this one please? Thanks for your work on this. We'll probably need a follow-up to tweak some things and some more tests (particularly ducktape ones) but we're most of the way there. |
…fault was changed (apache#191) The default is now to disable ACLs. Updated system test to override. Reviewers: Stanislav Kozlovski <stanislav_kozlovski@outlook.com>
A file related to a single batch might be split into several blocks by the selected KeyAlignmentStrategy. In case of FixedBlockAlignment, if the file is bigger than the block size, it will be split into two or more files. With these changes it's now possible to re-assemble all the records related to that batch.
7 participants
No description provided.