KAFKA-10473: Add docs on partition size-on-disk, and other log-related metrics#9276
Conversation
kafka.log,type=Log,name=Size kafka.log,type=Log,name=NumLogSegments kafka.log,type=Log,name=LogStartOffset kafka.log,type=Log,name=LogEndOffset
| <tr> | ||
| <td>Number of log segments in a partition</td> | ||
| <td>kafka.log:type=Log,name=NumLogSegments,topic=([-.\w]+),partition=([0-9]+)</td> | ||
| <td>The number of log segments in a partition.</td> |
There was a problem hiding this comment.
I wasn't sure what information to add for this column of "normal value".
- For date-based retention topics, it should roughly be sizeOfTopic/segmentSize
- But for compacted topics, that isn't the case
- If this is too high, it will mean more file handles.
- If it is too high, it might inefficient (is this true?)
Anyway, I wasn't sure what to put, so I left it simple.
There was a problem hiding this comment.
I think this is fine :)
| <tr> | ||
| <td>Last offset in a partition</td> | ||
| <td>kafka.log:type=Log,name=LogEndOffset,topic=([-.\w]+),partition=([0-9]+)</td> | ||
| <td>The last offset in a partition.</td> |
There was a problem hiding this comment.
Is this the last committed offset? Or, is it uncommitted offsets? Meaning, if a producer appends a record there but the followers haven't yet acked it, will this value get incremented?
Does that level of detail matter, in the docs?
There was a problem hiding this comment.
This is the last uncomitted offset: after the local append it would be incremented, not waiting for other replicas to ack.
|
@guozhangwang Can I get a review? |
|
The pull request lists lots of failed checks. However, this pull request only changed an HTML file, and it seems unrelated to the test failures in those checks. |
|
@wushujames Could you make some updates? I think it would be good for this PR to be included in the 2.7.0 release. |
|
@dongjinleekr : I've pulled in the latest changes from trunk. I'll check back in after the builds have completed. Aside from pulling the latest changes from trunk, were there any other updates/changes you think are needed? |
|
@dongjinleekr All builds failed, but none of the failures seem related to my change. |
| <tr> | ||
| <td>Last offset in a partition</td> | ||
| <td>kafka.log:type=Log,name=LogEndOffset,topic=([-.\w]+),partition=([0-9]+)</td> | ||
| <td>The last offset in a partition.</td> |
There was a problem hiding this comment.
This is the last uncomitted offset: after the local append it would be incremented, not waiting for other replicas to ack.
| <tr> | ||
| <td>Number of log segments in a partition</td> | ||
| <td>kafka.log:type=Log,name=NumLogSegments,topic=([-.\w]+),partition=([0-9]+)</td> | ||
| <td>The number of log segments in a partition.</td> |
There was a problem hiding this comment.
I think this is fine :)
…t-for-generated-requests * apache-github/trunk: MINOR: Fix flaky test shouldQueryOnlyActivePartitionStoresByDefault (apache#9681) KAFKA-10799 AlterIsr utilizes ReplicaManager ISR metrics (apache#9677) MINOR: Fix KTable-KTable foreign-key join example (apache#9683) KAFKA-10473: Add docs on partition size-on-disk, and other log-related metrics (apache#9276) KAFKA-10739; Replace EpochEndOffset with automated protocol (apache#9630) KAFKA-10460: ReplicaListValidator format checking is incomplete (apache#9326) KAFKA-10554; Perform follower truncation based on diverging epochs in Fetch response (apache#9382) MINOR: Align the UID inside/outside container (apache#9652) KAFKA-10794 Replica leader election is too slow in the case of too many partitions (apache#9675) KAFKA-10090 Misleading warnings: The configuration was supplied but i… (apache#8826) clients/src/main/java/org/apache/kafka/common/requests/OffsetsForLeaderEpochResponse.java clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java core/src/test/scala/unit/kafka/server/epoch/util/ReplicaFetcherMockBlockingSend.scala
3 participants
Add docs on the following JMX metrics
kafka.log,type=Log,name=Size
kafka.log,type=Log,name=NumLogSegments
kafka.log,type=Log,name=LogStartOffset
kafka.log,type=Log,name=LogEndOffset
To test this, I opened the docs in a web browser
Committer Checklist (excluded from commit message)