Log scheduler stats for Pulsar Functions #7474
Conversation
| } | ||
| log.info("Rebalance - Total number of new assignments computed: {}", rebalancedAssignments.size()); | ||
|
|
||
| log.info("Rebalance summary - execution time: {} sec | stats: {}\n{}", |
There was a problem hiding this comment.
Can we also report this stats to prometheus?
There was a problem hiding this comment.
Yes I was going to work on that in the near future. Stats like scheduler execution time can be in prometheus. However, a breakdown of what how many instances moved grouped by worker is not suitable to be put into prometheus
| } | ||
| if (triggerScheduler) { | ||
| log.info("Functions that need scheduling/rescheduling: {}", needSchedule); | ||
| log.info("Failure check - Total number of instances that need to be scheduled/rescheduled: {} | Number of unassigned instances that need to be scheduled: {} | Number of instances on dead workers that need to be reassigned {}", |
There was a problem hiding this comment.
Maybe a better wording here?
There was a problem hiding this comment.
Do you have a suggestion?
There was a problem hiding this comment.
I actually liked the old message. What was the reason for the change?
There was a problem hiding this comment.
Added "Failure check" in the beginning so it will be easier to search for
| public static class RebalanceInProgressException extends RuntimeException { | ||
| } | ||
|
|
||
| @Data |
There was a problem hiding this comment.
Is this exported somewhere?
Co-authored-by: Jerry Peng <jerryp@splunk.com>
3 participants
Motivation
Add stats to be logged for schedule, rebalance, and check failure routines for debugging and performance details