Check if supervisor could be idle on startup#16844
Conversation
ac9817
changed the title
ac9817
force-pushed
the
adithyac/supervisor-idle
branch
from
e0a044c to
c34ad09
Compare
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| ).anyTimes(); | ||
| EasyMock.reset(spec); | ||
| EasyMock.expect(spec.isSuspended()).andReturn(false).anyTimes(); | ||
| EasyMock.expect(spec.getDataSchema()).andReturn(getDataSchema()).anyTimes(); |
Check notice
Code scanning / CodeQL
Deprecated method or constructor invocation
kfaraz
left a comment
kfaraz
left a comment
There was a problem hiding this comment.
Minor comments, rest looks good to me.
georgew5656
left a comment
georgew5656
left a comment
There was a problem hiding this comment.
thought through all the possible cases and i think this will work. i think some comments in the logic would be helpful though, like explaining each scenario
- running supervisor with data, overlord restarts
- new supervisor with data on running overlord
- running supervisor that stops getting data
for future changes to the code
|
|
||
| int taskCountAfterScale = supervisor.getIoConfig().getTaskCount(); | ||
| Assert.assertEquals(2, taskCountAfterScale); | ||
| Assert.assertEquals(SupervisorStateManager.BasicState.IDLE, supervisor.getState()); |
There was a problem hiding this comment.
This is an invalid test. There are no actuals tasks running to consume the lag and go to idle. It used to work previously because reporting thread is yet to be executed before the completion of test and computeLag implied null as 0 lag. But with us now updating the lag proactively this fails.
4 participants
Fixes #13936.
Description
In cases where a supervisor is idle and the overlord is restarted for some reason, the supervisor would start spinning tasks again. In clusters where there are many low throughput stream this spike the task count unnecessarily. This PR checks if the latest offset with that of the ones committed in metadata during the startup of supervisor and sets it idle state if they match.
Release note
Supervisor would not go back into running state from the idle if the overlord restart happens for some reason.
Key changed/added classes in this PR
SeekableStreamSupervisorThis PR has: