Skip to content

MINOR: improve test coverage for dynamic LogConfig(s)#7616

Merged
ijuma merged 6 commits intoapache:trunkfrom
lbradstreet:explicitly-check-dynamic-log-configs
Apr 17, 2020
Merged

MINOR: improve test coverage for dynamic LogConfig(s)#7616
ijuma merged 6 commits intoapache:trunkfrom
lbradstreet:explicitly-check-dynamic-log-configs

Conversation

@lbradstreet
Copy link
Copy Markdown
Contributor

@lbradstreet lbradstreet commented Oct 30, 2019 •
edited
Loading

Adding a dynamically updatable log config is currently error prone, as it is easy to set them up as a val not a def and this would result in a dynamically updated broker default not applying to a LogConfig after broker restart.

This PR adds a guard against introducing these issues by ensuring that all log configs are exhaustively checked via a test.

For example, if the following line was a val and not a def, there would be a problem with dynamically updating broker defaults for the config.

kafka/core/src/main/scala/kafka/server/KafkaConfig.scala

Line 1216 in 4bde9bb

def logFlushIntervalMessages = getLong(KafkaConfig.LogFlushIntervalMessagesProp)

@lbradstreet
Copy link
Copy Markdown
Contributor Author

lbradstreet commented Oct 30, 2019

Scala 2.11 compile issues are present.

dhruvilshah3
dhruvilshah3 reviewed Nov 1, 2019
View reviewed changes
Comment thread core/src/test/scala/unit/kafka/server/KafkaConfigTest.scala Outdated
dhruvilshah3
dhruvilshah3 approved these changes Nov 1, 2019
View reviewed changes
Copy link
Copy Markdown
Contributor

@dhruvilshah3 dhruvilshah3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR, LGTM. Left one minor comment.

@lbradstreet
Copy link
Copy Markdown
Contributor Author

lbradstreet commented Nov 4, 2019 •
edited
Loading

@dhruvilshah3 thanks, your suggestion was great and picked up a couple of issues in the checks. PR is ready to merge when ready.

@lbradstreet
Copy link
Copy Markdown
Contributor Author

lbradstreet commented Feb 13, 2020

This needs a rebuild by a committer.

@ijuma
Copy link
Copy Markdown
Member

ijuma commented Feb 13, 2020

ok to test

@mumrah
Copy link
Copy Markdown
Member

mumrah commented Mar 10, 2020

retest this please

@lbradstreet
Copy link
Copy Markdown
Contributor Author

lbradstreet commented Mar 10, 2020

One flake in JDK 11 and scala 2.13: kafka.api.ConsumerBounceTest.testRollingBrokerRestartsWithSmallerMaxGroupSizeConfigDisruptsBigGroup.

No failures in scala 2.12.

Looks like we're good to merge @ijuma or @mumrah

ijuma
ijuma approved these changes Apr 17, 2020
View reviewed changes
Copy link
Copy Markdown
Member

@ijuma ijuma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks.

@ijuma ijuma merged commit 00a59b3 into apache:trunk Apr 17, 2020
Kvicii pushed a commit to Kvicii/kafka that referenced this pull request Apr 19, 2020
Merge branch 'trunk' of github.com:apache/kafka into trunk-read
6de951a 
* 'trunk' of github.com:apache/kafka: (28 commits)
  MINOR: cleanup RocksDBStore tests  (apache#8510)
  KAFKA-9818: Fix flaky test in RecordCollectorTest (apache#8507)
  MINOR: reduce impact of trace logging in replica hot path (apache#8468)
  KAFKA-6145: KIP-441: Add test scenarios to ensure rebalance convergence (apache#8475)
  KAFKA-9881: Convert integration test to verify measurements from RocksDB to unit test (apache#8501)
  MINOR: improve test coverage for dynamic LogConfig(s) (apache#7616)
  MINOR: Switch order of sections on tumbling and hopping windows in streams doc. Tumbling windows are defined as "special case of hopping time windows" - but hopping windows currently only explained later in the docs. (apache#8505)
  KAFKA-9819: Fix flaky test in StoreChangelogReaderTest (apache#8488)
  HOTFIX: fix active task process ratio metric recording
  KAFKA-9796; Ensure broker shutdown is not stuck when Acceptor is waiting on connection queue (apache#8448)
  MINOR: Use streaming iterator with decompression buffer when building offset map (apache#8494)
  Add log message in release.py (apache#8461)
  KAFKA-9854 Re-authenticating causes mismatched parse of response (apache#8471)
  KAFKA-9838; Add log concurrency test and fix minor race condition (apache#8476)
  KAFKA-9703; Free up compression buffer after splitting a large batch
  KAFKA-9779: Add Stream system test for 2.5 release (apache#8378)
  KAFKA-7885: TopologyDescription violates equals-hashCode contract. (apache#6210)
  MINOR: KafkaApis#handleOffsetDeleteRequest does not group result correctly (apache#8485)
  HOTFIX: don't close or wipe out someone else's state (apache#8478)
  MINOR: add process(Test)Messages to the README (apache#8480)
  ...
ijuma added a commit to confluentinc/kafka that referenced this pull request Apr 21, 2020
@ijuma
Merge remote-tracking branch 'apache-github/trunk' into master
48158e5 
* apache-github/trunk:
  MINOR: Fix grammar in error message for InvalidRecordException (apache#8465)
  KAFKA-9868: Reduce number of transaction log partitions for embed broker (apache#8522)
  MINOR: Adding github whitelist (apache#8523)
  MINOR: Upgrade gradle plugins and test libraries for Java 14 support (apache#8519)
  MINOR: Further reduce runtime for metrics integration tests (apache#8514)
  MINOR: Use .asf.yaml to direct github notifications to JIRA and mailing lists (apache#8521)
  MINOR: Update to Gradle 6.3 (apache#7677)
  HOTFIX: fix checkstyle error of RocksDBStoreTest and flaky RocksDBTimestampedStoreTest.shouldOpenExistingStoreInRegularMode (apache#8515)
  MINOR: cleanup RocksDBStore tests  (apache#8510)
  KAFKA-9818: Fix flaky test in RecordCollectorTest (apache#8507)
  MINOR: reduce impact of trace logging in replica hot path (apache#8468)
  KAFKA-6145: KIP-441: Add test scenarios to ensure rebalance convergence (apache#8475)
  KAFKA-9881: Convert integration test to verify measurements from RocksDB to unit test (apache#8501)
  MINOR: improve test coverage for dynamic LogConfig(s) (apache#7616)
  MINOR: Switch order of sections on tumbling and hopping windows in streams doc. Tumbling windows are defined as "special case of hopping time windows" - but hopping windows currently only explained later in the docs. (apache#8505)
  KAFKA-9819: Fix flaky test in StoreChangelogReaderTest (apache#8488)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ijuma ijuma ijuma approved these changes

+1 more reviewer

@dhruvilshah3 dhruvilshah3 dhruvilshah3 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@lbradstreet @ijuma @mumrah @dhruvilshah3