Skip to content

KAFKA-8860: Let SslPrincipalMapper split SSL principal mapping rules#7140

Closed
teebee wants to merge 1 commit intoapache:trunkfrom
teebee:teebee/ssl-principal-mapping-rules-handling
Closed

KAFKA-8860: Let SslPrincipalMapper split SSL principal mapping rules#7140
teebee wants to merge 1 commit intoapache:trunkfrom
teebee:teebee/ssl-principal-mapping-rules-handling

Conversation

@teebee
Copy link
Copy Markdown

@teebee teebee commented Jul 30, 2019

PR #6099 tried to undo the splitting of the ssl.principal.mapper.rules list on comma with whitespace by sophisticated rejoining of the split list using a comma as separator. However, since possibly surrounding whitespace is not reconstructed this approach fails in general. Consider the following test case:

@test
public void testCommaWithWhitespace() throws Exception {
String value = "RULE:^CN=((\\, |\w)+)(,.|$)/$1/,DEFAULT";

@SuppressWarnings("unchecked")
List<String> rules = (List<String>) ConfigDef.parseType("ssl.principal.mapper.rules", value, Type.LIST);

SslPrincipalMapper mapper = SslPrincipalMapper.fromRules(rules);
assertEquals("Tkac\\, Adam", mapper.getName("CN=Tkac\\, Adam,OU=ITZ,DC=geodis,DC=cz"));

}
The space after the escaped comma is essential. Unfortunately, it has disappeared after splitting and rejoining.

Moreover, in joinSplitRules the decision to rejoin list elements is based on local information only which might not be sufficient. It works for "RULE:^CN=([^,ADEFLTU,]+)(,.|$)/$1/" but fails for the equivalent regular expression "RULE:^CN=([^,DEFAULT,]+)(,.|$)/$1/".

The approach of the current PR is to change the type of the ssl.principal.mapper.rules attribute from LIST to STRING and to delegate the splitting of the rules to the SslPrincipalMapper. It knows about the structure of the rules and can perform the splitting context-based.

@teebee
Copy link
Copy Markdown
Author

teebee commented Aug 9, 2019

The changes should not have affected the failed test:

Error details

java.lang.AssertionError

Stacktrace

java.lang.AssertionError
	at org.junit.Assert.fail(Assert.java:87)
	at org.junit.Assert.assertTrue(Assert.java:42)
	at org.junit.Assert.assertTrue(Assert.java:53)
	at kafka.server.DescribeLogDirsRequestTest.testDescribeLogDirsRequest(DescribeLogDirsRequestTest.scala:60)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:305)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:365)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:330)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:78)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:328)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:65)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:292)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:305)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:412)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
	at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
	at jdk.internal.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
	at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:118)
	at jdk.internal.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:175)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:157)
	at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:404)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
	at java.base/java.lang.Thread.run(Thread.java:834)

Standard Output (STDOUT)

[2019-08-09 00:34:24,743] INFO [ControllerEventThread controllerId=0] Starting (kafka.controller.ControllerEventManager$ControllerEventThread:66)
[2019-08-09 00:34:24,751] INFO [Controller id=0] 0 successfully elected as the controller. Epoch incremented to 1 and epoch zk version is now 1 (kafka.controller.KafkaController:66)
[2019-08-09 00:34:24,751] INFO [Controller id=0] Registering handlers (kafka.controller.KafkaController:66)
[2019-08-09 00:34:24,755] INFO [Controller id=0] Deleting log dir event notifications (kafka.controller.KafkaController:66)
[2019-08-09 00:34:24,756] INFO [Controller id=0] Deleting isr change notifications (kafka.controller.KafkaController:66)
[2019-08-09 00:34:24,757] INFO [Controller id=0] Initializing controller context (kafka.controller.KafkaController:66)
[2019-08-09 00:34:24,759] INFO [Controller id=0] Initialized broker epochs cache: Map(0 -> 25) (kafka.controller.KafkaController:66)
[2019-08-09 00:34:24,764] INFO [Controller id=0] Partitions being reassigned: Map() (kafka.controller.KafkaController:66)
[2019-08-09 00:34:24,764] INFO [Controller id=0] Currently active brokers in the cluster: Set(0) (kafka.controller.KafkaController:66)
[2019-08-09 00:34:24,764] INFO [Controller id=0] Currently shutting brokers in the cluster: Set() (kafka.controller.KafkaController:66)
[2019-08-09 00:34:24,765] INFO [Controller id=0] Current list of topics in the cluster: Set() (kafka.controller.KafkaController:66)
[2019-08-09 00:34:24,765] INFO [Controller id=0] Fetching topic deletions in progress (kafka.controller.KafkaController:66)
[2019-08-09 00:34:24,766] INFO [Controller id=0] List of topics to be deleted:  (kafka.controller.KafkaController:66)
[2019-08-09 00:34:24,766] INFO [Controller id=0] List of topics ineligible for deletion:  (kafka.controller.KafkaController:66)
[2019-08-09 00:34:24,766] INFO [Controller id=0] Initializing topic deletion manager (kafka.controller.KafkaController:66)
[2019-08-09 00:34:24,766] INFO [Controller id=0] Sending update metadata request (kafka.controller.KafkaController:66)
[2019-08-09 00:34:24,767] INFO [Controller id=0] Ready to serve as the new controller with epoch 1 (kafka.controller.KafkaController:66)
[2019-08-09 00:34:24,767] INFO [Controller id=0] Removing partitions Set() from the list of reassigned partitions in zookeeper (kafka.controller.KafkaController:66)
[2019-08-09 00:34:24,767] INFO [Controller id=0] No more partitions need to be reassigned. Deleting zk path /admin/reassign_partitions (kafka.controller.KafkaController:66)
[2019-08-09 00:34:24,770] INFO [Controller id=0] Partitions undergoing preferred replica election:  (kafka.controller.KafkaController:66)
[2019-08-09 00:34:24,771] INFO [Controller id=0] Partitions that completed preferred replica election:  (kafka.controller.KafkaController:66)
[2019-08-09 00:34:24,772] INFO [Controller id=0] Skipping preferred replica election for partitions due to topic deletion:  (kafka.controller.KafkaController:66)
[2019-08-09 00:34:24,772] INFO [Controller id=0] Resuming preferred replica election for partitions:  (kafka.controller.KafkaController:66)
[2019-08-09 00:34:24,773] INFO [Controller id=0] Starting replica leader election (PREFERRED) for partitions  triggerd by ZkTriggered (kafka.controller.KafkaController:66)
[2019-08-09 00:34:24,775] INFO [Controller id=0] Starting the controller scheduler (kafka.controller.KafkaController:66)
[2019-08-09 00:34:24,785] INFO [Controller id=0] New topics: [Set(__consumer_offsets)], deleted topics: [Set()], new partition replica assignment [Map(__consumer_offsets-4 -> Vector(0), __consumer_offsets-3 -> Vector(0), __consumer_offsets-2 -> Vector(0), __consumer_offsets-0 -> Vector(0), __consumer_offsets-1 -> Vector(0))] (kafka.controller.KafkaController:66)
[2019-08-09 00:34:24,786] INFO [Controller id=0] New partition creation callback for __consumer_offsets-4,__consumer_offsets-3,__consumer_offsets-2,__consumer_offsets-0,__consumer_offsets-1 (kafka.controller.KafkaController:66)
[2019-08-09 00:34:25,408] INFO [Controller id=0] Handling log directory failure for brokers 0 (kafka.controller.KafkaController:66)
[2019-08-09 00:34:25,411] INFO [Controller id=0] Handling log directory failure for brokers  (kafka.controller.KafkaController:66)
[2019-08-09 00:34:25,413] INFO [Controller id=0] New topics: [Set(topic)], deleted topics: [Set()], new partition replica assignment [Map(topic-1 -> Vector(0), topic-0 -> Vector(0))] (kafka.controller.KafkaController:66)
[2019-08-09 00:34:25,413] INFO [Controller id=0] New partition creation callback for topic-1,topic-0 (kafka.controller.KafkaController:66)
[2019-08-09 00:34:26,344] INFO [ControllerEventThread controllerId=0] Shutting down (kafka.controller.ControllerEventManager$ControllerEventThread:66)
[2019-08-09 00:34:26,344] INFO [ControllerEventThread controllerId=0] Stopped (kafka.controller.ControllerEventManager$ControllerEventThread:66)
[2019-08-09 00:34:26,344] INFO [ControllerEventThread controllerId=0] Shutdown completed (kafka.controller.ControllerEventManager$ControllerEventThread:66)
[2019-08-09 00:34:26,347] INFO [Controller id=0] Resigned (kafka.controller.KafkaController:66)

Copy link
Copy Markdown
Contributor

@omkreddy omkreddy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@teebee Thanks for the contribution. LGTM. Can you please file a JIRA? So that it will be easy to track the issue for others.
https://issues.apache.org/jira/projects/KAFKA/

@omkreddy omkreddy changed the title MINOR: Let SslPrincipalMapper split SSL principal mapping rules KAFKA-8860: Let SslPrincipalMapper split SSL principal mapping rules Sep 2, 2019
@omkreddy omkreddy closed this in 88d1b6d Sep 2, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants