-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Closed
Labels
enhancementNew feature or requestNew feature or request
Description
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Currently the ballista scheduler state mod messes many things up in the mod file, which is not good for code reviewing.
Describe the solution you'd like
It's better to reorganize the scheduler contents as following:
- backend, for persisting states in some storage
- in_memory_state, for managing part of the scheduler state which is volatile and only be stored in memory
- persistent_state, for managing part of the scheduler state which is relatively stable and be stored in one of the backend storage and cached in memory for fast reading.
- scheduler_state, for providing a unified interface for scheduler state
- task_scheduler, for fetching tasks to be scheduled
- query_stage_scheduler, for dealing with stage status changes (TODO)
Additional context
#1704
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request