Skip to content

KAFKA-9730 add tag "partition" to BrokerTopicMetrics so as to observe…#8309

Closed
chia7712 wants to merge 1 commit intoapache:trunkfrom
chia7712:fix_9730
Closed

KAFKA-9730 add tag "partition" to BrokerTopicMetrics so as to observe…#8309
chia7712 wants to merge 1 commit intoapache:trunkfrom
chia7712:fix_9730

Conversation

@chia7712
Copy link
Copy Markdown
Member

@chia7712 chia7712 commented Mar 17, 2020

Partitioner enable us to dispatch the data to the specify partition as we wish. However, we can't observe the partition metrics if they are on the same broker. The root cause is that the key of topic metrics doesn't contain "partition" so all partitions on the same broker are merged into single metrics (see attachment).

before
trunk

after
patch

KAFKA-9730

KIP-581

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@chia7712
Copy link
Copy Markdown
Member Author

the flaky StateDirectoryTest.shouldReturnEmptyArrayIfListFilesReturnsNull is traced by #8310

val tags: scala.collection.Map[String, String] = name match {
case None => Map.empty
case Some(topic) => Map("topic" -> topic)
case Some(topic) => Map("topic" -> topic.topic(), "partition" -> topic.partition().toString)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that this causes an explosion in the number of metrics and it will break JMX users. As such, a KIP would be required.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ijuma Got it

@chia7712 chia7712 closed this May 29, 2020
@chia7712 chia7712 deleted the fix_9730 branch March 25, 2024 15:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants