Skip to content

KAFKA-9980: Fix bug where alterClientQuotas could not set default client quotas#8658

Merged
cmccabe merged 2 commits intoapache:trunkfrom
bdbyrne:KAFKA-9980
May 22, 2020
Merged

KAFKA-9980: Fix bug where alterClientQuotas could not set default client quotas#8658
cmccabe merged 2 commits intoapache:trunkfrom
bdbyrne:KAFKA-9980

Conversation

@bdbyrne
Copy link
Copy Markdown
Contributor

@bdbyrne bdbyrne commented May 12, 2020

Fixes an issue where the default entity name "<default>" was erroneously
being sanitized, which prevented proper inheritance of default configs.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@bdbyrne bdbyrne marked this pull request as ready for review May 16, 2020 22:25
@bdbyrne
Copy link
Copy Markdown
Contributor Author

bdbyrne commented May 16, 2020

I've applied @d8tltanc's updates to quota_test.py and verified their success:

docker exec ducker01 bash -c "cd /opt/kafka-dev && ducktape --cluster-file /opt/kafka-dev/tests/docker/build/cluster.json  ./tests/kafkatest/tests/client/quota_test.py"
[INFO:2020-05-16 14:54:39,531]: starting test run with session id 2020-05-16--002...
[INFO:2020-05-16 14:54:39,531]: running 9 tests...
[INFO:2020-05-16 14:54:39,531]: Triggering test 1 of 9...
[INFO:2020-05-16 14:54:39,536]: RunnerClient: Loading test {'directory': '/opt/kafka-dev/tests/kafkatest/tests/client', 'file_name': 'quota_test.py', 'method_name': 'test_quota', 'cls_name': 'QuotaTest', 'injected_args': {'consumer_num': 2, 'quota_type': 'client-id'}}
[INFO:2020-05-16 14:54:39,539]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.consumer_num=2.quota_type=client-id: Setting up...
/usr/local/lib/python2.7/dist-packages/paramiko/kex_ecdh_nist.py:39: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding.
  m.add_string(self.Q_C.public_numbers().encode_point())
/usr/local/lib/python2.7/dist-packages/paramiko/kex_ecdh_nist.py:94: CryptographyDeprecationWarning: Support for unsafe construction of public numbers from encoded data will be removed in a future version. Please use EllipticCurvePublicKey.from_encoded_point
  self.curve, Q_S_bytes
/usr/local/lib/python2.7/dist-packages/paramiko/kex_ecdh_nist.py:109: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding.
  hm.add_string(self.Q_C.public_numbers().encode_point())
[INFO:2020-05-16 14:54:41,606]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.consumer_num=2.quota_type=client-id: Running...
[INFO:2020-05-16 14:57:18,680]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.consumer_num=2.quota_type=client-id: PASS
[INFO:2020-05-16 14:57:18,681]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.consumer_num=2.quota_type=client-id: Tearing down...
[WARNING - 2020-05-16 14:57:18,981 - service_registry - stop_all - lineno:55]: Error stopping service <ProducerPerformanceService-0-140504546534544: num_nodes: 1, nodes: [u'ducker04']>: 1
[INFO:2020-05-16 14:57:25,809]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.consumer_num=2.quota_type=client-id: Summary: 
[INFO:2020-05-16 14:57:25,810]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.consumer_num=2.quota_type=client-id: Data: None
[INFO:2020-05-16 14:57:25,866]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[INFO:2020-05-16 14:57:25,867]: Triggering test 2 of 9...
[INFO:2020-05-16 14:57:25,871]: RunnerClient: Loading test {'directory': '/opt/kafka-dev/tests/kafkatest/tests/client', 'file_name': 'quota_test.py', 'method_name': 'test_quota', 'cls_name': 'QuotaTest', 'injected_args': {'old_broker_throttling_behavior': True, 'quota_type': 'client-id'}}
[INFO:2020-05-16 14:57:25,873]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.old_broker_throttling_behavior=True.quota_type=client-id: Setting up...
/usr/local/lib/python2.7/dist-packages/paramiko/kex_ecdh_nist.py:39: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding.
  m.add_string(self.Q_C.public_numbers().encode_point())
/usr/local/lib/python2.7/dist-packages/paramiko/kex_ecdh_nist.py:94: CryptographyDeprecationWarning: Support for unsafe construction of public numbers from encoded data will be removed in a future version. Please use EllipticCurvePublicKey.from_encoded_point
  self.curve, Q_S_bytes
/usr/local/lib/python2.7/dist-packages/paramiko/kex_ecdh_nist.py:109: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding.
  hm.add_string(self.Q_C.public_numbers().encode_point())
[INFO:2020-05-16 14:57:27,557]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.old_broker_throttling_behavior=True.quota_type=client-id: Running...
[INFO:2020-05-16 14:59:58,427]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.old_broker_throttling_behavior=True.quota_type=client-id: PASS
[INFO:2020-05-16 14:59:58,428]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.old_broker_throttling_behavior=True.quota_type=client-id: Tearing down...
[WARNING - 2020-05-16 14:59:58,770 - service_registry - stop_all - lineno:55]: Error stopping service <ProducerPerformanceService-0-140504546777232: num_nodes: 1, nodes: [u'ducker09']>: 1
[INFO:2020-05-16 15:00:04,564]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.old_broker_throttling_behavior=True.quota_type=client-id: Summary: 
[INFO:2020-05-16 15:00:04,565]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.old_broker_throttling_behavior=True.quota_type=client-id: Data: None
[INFO:2020-05-16 15:00:04,635]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[INFO:2020-05-16 15:00:04,635]: Triggering test 3 of 9...
[INFO:2020-05-16 15:00:04,641]: RunnerClient: Loading test {'directory': '/opt/kafka-dev/tests/kafkatest/tests/client', 'file_name': 'quota_test.py', 'method_name': 'test_quota', 'cls_name': 'QuotaTest', 'injected_args': {'quota_type': '(user, client-id)', 'override_quota': False}}
[INFO:2020-05-16 15:00:04,645]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=.user.client-id.override_quota=False: Setting up...
/usr/local/lib/python2.7/dist-packages/paramiko/kex_ecdh_nist.py:39: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding.
  m.add_string(self.Q_C.public_numbers().encode_point())
/usr/local/lib/python2.7/dist-packages/paramiko/kex_ecdh_nist.py:94: CryptographyDeprecationWarning: Support for unsafe construction of public numbers from encoded data will be removed in a future version. Please use EllipticCurvePublicKey.from_encoded_point
  self.curve, Q_S_bytes
/usr/local/lib/python2.7/dist-packages/paramiko/kex_ecdh_nist.py:109: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding.
  hm.add_string(self.Q_C.public_numbers().encode_point())
[INFO:2020-05-16 15:00:06,418]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=.user.client-id.override_quota=False: Running...
[INFO:2020-05-16 15:03:45,093]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=.user.client-id.override_quota=False: PASS
[INFO:2020-05-16 15:03:45,093]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=.user.client-id.override_quota=False: Tearing down...
[WARNING - 2020-05-16 15:03:45,489 - service_registry - stop_all - lineno:55]: Error stopping service <ProducerPerformanceService-0-140504546806864: num_nodes: 1, nodes: [u'ducker14']>: 1
[INFO:2020-05-16 15:03:51,822]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=.user.client-id.override_quota=False: Summary: 
[INFO:2020-05-16 15:03:51,823]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=.user.client-id.override_quota=False: Data: None
[INFO:2020-05-16 15:03:51,844]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[INFO:2020-05-16 15:03:51,844]: Triggering test 4 of 9...
[INFO:2020-05-16 15:03:51,848]: RunnerClient: Loading test {'directory': '/opt/kafka-dev/tests/kafkatest/tests/client', 'file_name': 'quota_test.py', 'method_name': 'test_quota', 'cls_name': 'QuotaTest', 'injected_args': {'quota_type': '(user, client-id)', 'override_quota': True}}
[INFO:2020-05-16 15:03:51,851]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=.user.client-id.override_quota=True: Setting up...
/usr/local/lib/python2.7/dist-packages/paramiko/kex_ecdh_nist.py:39: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding.
  m.add_string(self.Q_C.public_numbers().encode_point())
/usr/local/lib/python2.7/dist-packages/paramiko/kex_ecdh_nist.py:94: CryptographyDeprecationWarning: Support for unsafe construction of public numbers from encoded data will be removed in a future version. Please use EllipticCurvePublicKey.from_encoded_point
  self.curve, Q_S_bytes
/usr/local/lib/python2.7/dist-packages/paramiko/kex_ecdh_nist.py:109: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding.
  hm.add_string(self.Q_C.public_numbers().encode_point())
[INFO:2020-05-16 15:03:53,924]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=.user.client-id.override_quota=True: Running...
[INFO:2020-05-16 15:06:44,312]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=.user.client-id.override_quota=True: PASS
[INFO:2020-05-16 15:06:44,313]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=.user.client-id.override_quota=True: Tearing down...
[WARNING - 2020-05-16 15:06:44,637 - service_registry - stop_all - lineno:55]: Error stopping service <ProducerPerformanceService-0-140504772528720: num_nodes: 1, nodes: [u'ducker06']>: 1
[INFO:2020-05-16 15:06:52,462]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=.user.client-id.override_quota=True: Summary: 
[INFO:2020-05-16 15:06:52,462]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=.user.client-id.override_quota=True: Data: None
[INFO:2020-05-16 15:06:52,483]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[INFO:2020-05-16 15:06:52,483]: Triggering test 5 of 9...
[INFO:2020-05-16 15:06:52,487]: RunnerClient: Loading test {'directory': '/opt/kafka-dev/tests/kafkatest/tests/client', 'file_name': 'quota_test.py', 'method_name': 'test_quota', 'cls_name': 'QuotaTest', 'injected_args': {'quota_type': 'client-id', 'old_client_throttling_behavior': True}}
[INFO:2020-05-16 15:06:52,490]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=client-id.old_client_throttling_behavior=True: Setting up...
/usr/local/lib/python2.7/dist-packages/paramiko/kex_ecdh_nist.py:39: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding.
  m.add_string(self.Q_C.public_numbers().encode_point())
/usr/local/lib/python2.7/dist-packages/paramiko/kex_ecdh_nist.py:94: CryptographyDeprecationWarning: Support for unsafe construction of public numbers from encoded data will be removed in a future version. Please use EllipticCurvePublicKey.from_encoded_point
  self.curve, Q_S_bytes
/usr/local/lib/python2.7/dist-packages/paramiko/kex_ecdh_nist.py:109: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding.
  hm.add_string(self.Q_C.public_numbers().encode_point())
[INFO:2020-05-16 15:06:53,987]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=client-id.old_client_throttling_behavior=True: Running...
[INFO:2020-05-16 15:09:34,121]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=client-id.old_client_throttling_behavior=True: PASS
[INFO:2020-05-16 15:09:34,122]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=client-id.old_client_throttling_behavior=True: Tearing down...
[WARNING - 2020-05-16 15:09:34,390 - service_registry - stop_all - lineno:55]: Error stopping service <ProducerPerformanceService-0-140504546937104: num_nodes: 1, nodes: [u'ducker11']>: 1
[INFO:2020-05-16 15:09:41,085]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=client-id.old_client_throttling_behavior=True: Summary: 
[INFO:2020-05-16 15:09:41,085]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=client-id.old_client_throttling_behavior=True: Data: None
[INFO:2020-05-16 15:09:41,110]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[INFO:2020-05-16 15:09:41,110]: Triggering test 6 of 9...
[INFO:2020-05-16 15:09:41,115]: RunnerClient: Loading test {'directory': '/opt/kafka-dev/tests/kafkatest/tests/client', 'file_name': 'quota_test.py', 'method_name': 'test_quota', 'cls_name': 'QuotaTest', 'injected_args': {'quota_type': 'client-id', 'override_quota': False}}
[INFO:2020-05-16 15:09:41,118]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=client-id.override_quota=False: Setting up...
/usr/local/lib/python2.7/dist-packages/paramiko/kex_ecdh_nist.py:39: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding.
  m.add_string(self.Q_C.public_numbers().encode_point())
/usr/local/lib/python2.7/dist-packages/paramiko/kex_ecdh_nist.py:94: CryptographyDeprecationWarning: Support for unsafe construction of public numbers from encoded data will be removed in a future version. Please use EllipticCurvePublicKey.from_encoded_point
  self.curve, Q_S_bytes
/usr/local/lib/python2.7/dist-packages/paramiko/kex_ecdh_nist.py:109: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding.
  hm.add_string(self.Q_C.public_numbers().encode_point())
[INFO:2020-05-16 15:09:42,955]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=client-id.override_quota=False: Running...
[INFO:2020-05-16 15:13:20,158]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=client-id.override_quota=False: PASS
[INFO:2020-05-16 15:13:20,159]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=client-id.override_quota=False: Tearing down...
[WARNING - 2020-05-16 15:13:20,311 - service_registry - stop_all - lineno:55]: Error stopping service <ProducerPerformanceService-0-140504772477456: num_nodes: 1, nodes: [u'ducker03']>: 1
[INFO:2020-05-16 15:13:27,474]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=client-id.override_quota=False: Summary: 
[INFO:2020-05-16 15:13:27,474]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=client-id.override_quota=False: Data: None
[INFO:2020-05-16 15:13:27,499]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[INFO:2020-05-16 15:13:27,499]: Triggering test 7 of 9...
[INFO:2020-05-16 15:13:27,503]: RunnerClient: Loading test {'directory': '/opt/kafka-dev/tests/kafkatest/tests/client', 'file_name': 'quota_test.py', 'method_name': 'test_quota', 'cls_name': 'QuotaTest', 'injected_args': {'quota_type': 'client-id', 'override_quota': True}}
[INFO:2020-05-16 15:13:27,506]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=client-id.override_quota=True: Setting up...
/usr/local/lib/python2.7/dist-packages/paramiko/kex_ecdh_nist.py:39: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding.
  m.add_string(self.Q_C.public_numbers().encode_point())
/usr/local/lib/python2.7/dist-packages/paramiko/kex_ecdh_nist.py:94: CryptographyDeprecationWarning: Support for unsafe construction of public numbers from encoded data will be removed in a future version. Please use EllipticCurvePublicKey.from_encoded_point
  self.curve, Q_S_bytes
/usr/local/lib/python2.7/dist-packages/paramiko/kex_ecdh_nist.py:109: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding.
  hm.add_string(self.Q_C.public_numbers().encode_point())
[INFO:2020-05-16 15:13:29,704]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=client-id.override_quota=True: Running...
[INFO:2020-05-16 15:16:10,339]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=client-id.override_quota=True: PASS
[INFO:2020-05-16 15:16:10,340]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=client-id.override_quota=True: Tearing down...
[WARNING - 2020-05-16 15:16:10,622 - service_registry - stop_all - lineno:55]: Error stopping service <ProducerPerformanceService-0-140504546885008: num_nodes: 1, nodes: [u'ducker08']>: 1
[INFO:2020-05-16 15:16:17,281]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=client-id.override_quota=True: Summary: 
[INFO:2020-05-16 15:16:17,282]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=client-id.override_quota=True: Data: None
[INFO:2020-05-16 15:16:17,308]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[INFO:2020-05-16 15:16:17,308]: Triggering test 8 of 9...
[INFO:2020-05-16 15:16:17,312]: RunnerClient: Loading test {'directory': '/opt/kafka-dev/tests/kafkatest/tests/client', 'file_name': 'quota_test.py', 'method_name': 'test_quota', 'cls_name': 'QuotaTest', 'injected_args': {'quota_type': 'user', 'override_quota': False}}
[INFO:2020-05-16 15:16:17,315]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=user.override_quota=False: Setting up...
/usr/local/lib/python2.7/dist-packages/paramiko/kex_ecdh_nist.py:39: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding.
  m.add_string(self.Q_C.public_numbers().encode_point())
/usr/local/lib/python2.7/dist-packages/paramiko/kex_ecdh_nist.py:94: CryptographyDeprecationWarning: Support for unsafe construction of public numbers from encoded data will be removed in a future version. Please use EllipticCurvePublicKey.from_encoded_point
  self.curve, Q_S_bytes
/usr/local/lib/python2.7/dist-packages/paramiko/kex_ecdh_nist.py:109: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding.
  hm.add_string(self.Q_C.public_numbers().encode_point())
[INFO:2020-05-16 15:16:19,454]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=user.override_quota=False: Running...
[INFO:2020-05-16 15:19:58,103]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=user.override_quota=False: PASS
[INFO:2020-05-16 15:19:58,104]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=user.override_quota=False: Tearing down...
[WARNING - 2020-05-16 15:19:58,385 - service_registry - stop_all - lineno:55]: Error stopping service <ProducerPerformanceService-0-140504546864592: num_nodes: 1, nodes: [u'ducker13']>: 1
[INFO:2020-05-16 15:20:04,791]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=user.override_quota=False: Summary: 
[INFO:2020-05-16 15:20:04,791]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=user.override_quota=False: Data: None
[INFO:2020-05-16 15:20:04,815]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[INFO:2020-05-16 15:20:04,815]: Triggering test 9 of 9...
[INFO:2020-05-16 15:20:04,820]: RunnerClient: Loading test {'directory': '/opt/kafka-dev/tests/kafkatest/tests/client', 'file_name': 'quota_test.py', 'method_name': 'test_quota', 'cls_name': 'QuotaTest', 'injected_args': {'quota_type': 'user', 'override_quota': True}}
[INFO:2020-05-16 15:20:04,822]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=user.override_quota=True: Setting up...
/usr/local/lib/python2.7/dist-packages/paramiko/kex_ecdh_nist.py:39: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding.
  m.add_string(self.Q_C.public_numbers().encode_point())
/usr/local/lib/python2.7/dist-packages/paramiko/kex_ecdh_nist.py:94: CryptographyDeprecationWarning: Support for unsafe construction of public numbers from encoded data will be removed in a future version. Please use EllipticCurvePublicKey.from_encoded_point
  self.curve, Q_S_bytes
/usr/local/lib/python2.7/dist-packages/paramiko/kex_ecdh_nist.py:109: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding.
  hm.add_string(self.Q_C.public_numbers().encode_point())
[INFO:2020-05-16 15:20:06,965]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=user.override_quota=True: Running...
[INFO:2020-05-16 15:22:58,922]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=user.override_quota=True: PASS
[INFO:2020-05-16 15:22:58,923]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=user.override_quota=True: Tearing down...
[WARNING - 2020-05-16 15:22:59,232 - service_registry - stop_all - lineno:55]: Error stopping service <ProducerPerformanceService-0-140504546865104: num_nodes: 1, nodes: [u'ducker05']>: 1
[INFO:2020-05-16 15:23:06,326]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=user.override_quota=True: Summary: 
[INFO:2020-05-16 15:23:06,326]: RunnerClient: kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=user.override_quota=True: Data: None
================================================================================
SESSION REPORT (ALL TESTS)
ducktape version: 0.7.7
session_id:       2020-05-16--002
run time:         28 minutes 26.823 seconds
tests run:        9
passed:           9
failed:           0
ignored:          0
================================================================================
test_id:    kafkatest.tests.client.quota_test.QuotaTest.test_quota.consumer_num=2.quota_type=client-id
status:     PASS
run time:   2 minutes 46.270 seconds
--------------------------------------------------------------------------------
test_id:    kafkatest.tests.client.quota_test.QuotaTest.test_quota.old_broker_throttling_behavior=True.quota_type=client-id
status:     PASS
run time:   2 minutes 38.690 seconds
--------------------------------------------------------------------------------
test_id:    kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=.user.client-id.override_quota=False
status:     PASS
run time:   3 minutes 47.177 seconds
--------------------------------------------------------------------------------
test_id:    kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=.user.client-id.override_quota=True
status:     PASS
run time:   3 minutes 0.611 seconds
--------------------------------------------------------------------------------
test_id:    kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=client-id.old_client_throttling_behavior=True
status:     PASS
run time:   2 minutes 48.595 seconds
--------------------------------------------------------------------------------
test_id:    kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=client-id.override_quota=False
status:     PASS
run time:   3 minutes 46.356 seconds
--------------------------------------------------------------------------------
test_id:    kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=client-id.override_quota=True
status:     PASS
run time:   2 minutes 49.775 seconds
--------------------------------------------------------------------------------
test_id:    kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=user.override_quota=False
status:     PASS
run time:   3 minutes 47.476 seconds
--------------------------------------------------------------------------------
test_id:    kafkatest.tests.client.quota_test.QuotaTest.test_quota.quota_type=user.override_quota=True
status:     PASS
run time:   3 minutes 1.504 seconds
--------------------------------------------------------------------------------

@bdbyrne
Copy link
Copy Markdown
Contributor Author

bdbyrne commented May 16, 2020

@cmccabe this is ready for review. Thanks!

@cmccabe
Copy link
Copy Markdown
Contributor

cmccabe commented May 21, 2020

ok to test

@bdbyrne
Copy link
Copy Markdown
Contributor Author

bdbyrne commented May 21, 2020

Both test failures in org.apache.kafka.streams.integration.EosBetaUpgradeIntegrationTest.shouldUpgradeFromEosAlphaToEosBeta[true] appear unrelated:

Failing for the past 1 build (Since Failed#6410 )
Took 2 min 32 sec.
Error Message
java.lang.AssertionError: Did not receive all 20 records from topic multiPartitionOutputTopic within 60000 ms
Expected: is a value equal to or greater than <20>
     but: <0> was less than <20>

@cmccabe cmccabe changed the title KAFKA-9980: Fix client quotas default entity name handling in broker. KAFKA-9980: Fix bug where alterClientQuotas could not set default client quotas May 22, 2020
@cmccabe
Copy link
Copy Markdown
Contributor

cmccabe commented May 22, 2020

LGTM. Thanks, @bdbyrne .

@cmccabe cmccabe merged commit d9e9a18 into apache:trunk May 22, 2020
Kvicii pushed a commit to Kvicii/kafka that referenced this pull request May 22, 2020
* 'trunk' of github.com:apache/kafka:
  KAFKA-9980: Fix bug where alterClientQuotas could not set default client quotas (apache#8658)
  KAFKA-9780: Deprecate commit records without record metadata (apache#8379)
  MINOR: Deploy VerifiableClient in constructor to avoid test timeouts (apache#8651)
  MINOR: Added unit tests for ConnectionQuotas (apache#8650)
  MINOR: Correct MirrorMaker2 integration test configs for Connect internal topics (apache#8653)
  KAFKA-9855 - return cached Structs for Schemas with no fields (apache#8472)
  KAFKA-9950: Construct new ConfigDef for MirrorTaskConfig before defining new properties (apache#8608)
  KAFKA-8869: Remove task configs for deleted connectors from config snapshot (apache#8444)
  KAFKA-9409: Supplement immutability of ClusterConfigState class in Connect (apache#7942)
@bdbyrne bdbyrne deleted the KAFKA-9980 branch May 22, 2020 17:35
gitlw pushed a commit to linkedin/kafka that referenced this pull request Sep 9, 2021
…s could not set default client quotas (apache#8658)

Reviewers: Colin P. McCabe <cmccabe@apache.org>
@afireinside09
Copy link
Copy Markdown

Sorry for resurrecting an old thread but I'm still experiencing this in Kafka 2.6.2 with Java kafka-clients 2.6.2.

        ClientQuotaEntity clientEntity;
        if (clientId.equals(ConfigEntityName.Default())) {
            clientEntity = new ClientQuotaEntity(Map.of(ClientQuotaEntity.CLIENT_ID, null));
        } else {
            clientEntity = new ClientQuotaEntity(Map.of(ClientQuotaEntity.CLIENT_ID, clientId));
        }

What am I missing here so that the <default> string doesn't get sanitized when the entity name gets written to Zookeeper?

@afireinside09
Copy link
Copy Markdown

From the comments on ClientQuotaEntity:

Constructs a quota entity for the given types and names. If a name is null, then it is mapped to the built-in default entity name.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants