Emit event when a datasource has been dropped from the coordinator#18497
Emit event when a datasource has been dropped from the coordinator#18497cryptoe merged 16 commits intoapache:masterfrom
Conversation
kfaraz
left a comment
kfaraz
left a comment
There was a problem hiding this comment.
Since we already have this PR, do we still need the one disabling the test?
|
@kfaraz yeah let me do that. |
| .hasDimension(DruidMetrics.DATASOURCE, dataSource), | ||
| agg -> agg.hasSumAtLeast(numSegments) | ||
| ); | ||
| broker.latchableEmitter().waitForEvent( |
There was a problem hiding this comment.
Maybe we should we look for aggregate where the sum is equal to numSegments.
After this change, do we even need to wait to wait for the load success.
@uds5501 , could you try to push a change with the coordinator part commented out?
If not needed, we can remove that check completely.
There was a problem hiding this comment.
Tested locally, the tests seems to be working fine without the coordinator wait, removed in the latest comment.
Also updated the EmbeddedBroker as part of the numSegments change.
kfaraz
left a comment
kfaraz
left a comment
There was a problem hiding this comment.
Looks good, left some minor suggestions.
This reverts commit 805e255.
|
One input source IT was running. The patch does not change anything there hence merged the PR. |
4 participants
Follow up PR for #18495
It introduces a new event
segment/schemaCache/datasource/dropped/countthat is emitted when the data source is dropped from the coordinator and the cache refresh in broker acknowledges it.