KAFKA-6569: Move OffsetIndex/TimeIndex logger to companion object#4586
Merged
ijuma merged 3 commits intoapache:trunkfrom Feb 18, 2019
Merged
KAFKA-6569: Move OffsetIndex/TimeIndex logger to companion object#4586ijuma merged 3 commits intoapache:trunkfrom
ijuma merged 3 commits intoapache:trunkfrom
Conversation
ijuma
reviewed
Feb 20, 2018
Member
ijuma
left a comment
ijuma
left a comment
There was a problem hiding this comment.
Thanks for the PR. Would it be worth it to maintain the existing behaviour by having each AbstractIndex subclass provide the Logger? Something like:
class OffsetIndex {
def logger = OffsetIndex.logger
}
Member
|
@ambroff what do you think of the feedback above? |
We identified that we spend a lot of time in reflection when creating OffsetIndex, TimeIndex, or other implementations of AbstractIndex[K, V], because of the Logging mixin. When the broker is bootstrapping it's just doing this in a tight loop, so this time adds up. This patch moves the logging to a companion objects, statically initializing the logger.
omkreddy
reviewed
Feb 18, 2019
| // Avoid shadowing mutable file in AbstractIndex | ||
| class TimeIndex(_file: File, baseOffset: Long, maxIndexSize: Int = -1, writable: Boolean = true) | ||
| extends AbstractIndex[Long, Long](_file, baseOffset, maxIndexSize, writable) with Logging { | ||
| extends AbstractIndex[Long, Long](_file, baseOffset, maxIndexSize, writable) { |
Contributor
There was a problem hiding this comment.
This class also should provide the Logger, Right?
Member
There was a problem hiding this comment.
There are no log statements in this class.
Contributor
There was a problem hiding this comment.
There is one statement at https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/log/TimeIndex.scala#L129
Member
There was a problem hiding this comment.
Good catch! CoreUtils was exposing its logger, so I fixed both in a new commit.
omkreddy
approved these changes
Feb 18, 2019
Member
|
Tests passed, merging to trunk. |
ijuma
changed the title
jarekr
pushed a commit
to confluentinc/kafka
that referenced
this pull request
Apr 18, 2019
* ak/trunk: (45 commits) KAFKA-7487: DumpLogSegments misreports offset mismatches (apache#5756) MINOR: improve JavaDocs about auto-repartitioning in Streams DSL (apache#6269) KAFKA-7935: UNSUPPORTED_COMPRESSION_TYPE if ReplicaManager.getLogConfig returns None (apache#6274) KAFKA-7895: Fix stream-time reckoning for suppress (apache#6278) KAFKA-6569: Move OffsetIndex/TimeIndex logger to companion object (apache#4586) MINOR: add log indicating the suppression time (apache#6260) MINOR: Make info logs for KafkaConsumer a bit more verbose (apache#6279) KAFKA-7758: Reuse KGroupedStream/KGroupedTable with named repartition topics (apache#6265) KAFKA-7884; Docs for message.format.version should display valid values (apache#6209) MINOR: Save failed test output to build output directory MINOR: add test for StreamsSmokeTestDriver (apache#6231) MINOR: Fix bugs identified by compiler warnings (apache#6258) KAFKA-6474: Rewrite tests to use new public TopologyTestDriver [part 4] (apache#5433) MINOR: fix bypasses in ChangeLogging stores (apache#6266) MINOR: Make MockClient#poll() more thread-safe (apache#5942) MINOR: drop dbAccessor reference on close (apache#6254) KAFKA-7811: Avoid unnecessary lock acquire when KafkaConsumer commits offsets (apache#6119) KAFKA-7916: Unify store wrapping code for clarity (apache#6255) MINOR: Add missing Alter Operation to Topic supported operations list in AclCommand KAFKA-7921: log at error level for missing source topic (apache#6262) ...
pengxiaolong
pushed a commit
to pengxiaolong/kafka
that referenced
this pull request
Jun 14, 2019
…ache#4586) We identified that we spend a lot of time in the creation of Logger instances when creating OffsetIndex/TimeIndex due to the Logging mixin. When the broker is bootstrapping it's just doing this in a tight loop, so the time adds up. This patch moves the logger to the companion objects of OffsetIndex, TimeIndex and AbstractIndex resolving this issue. Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com> Co-authored-by: Kyle Ambroff <kyle@ambroff.com> Co-authored-by: Ismael Juma <ismael@juma.me.uk>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We identified that we spend a lot of time in reflection when creating
OffsetIndex, TimeIndex, or other implementations of
AbstractIndex[K, V], because of the Logging mixin. When the broker is
bootstrapping it's just doing this in a tight loop, so this time adds
up.
This patch moves the logging to a companion objects, statically
initializing the logger.
More detailed description of your change,
if necessary. The PR title and PR message become
the squashed commit message, so use a separate
comment to ping reviewers.
Summary of testing strategy (including rationale)
for the feature or bug fix. Unit and/or integration
tests are expected for any behaviour change and
system tests should be considered for larger changes.
Committer Checklist (excluded from commit message)