For some documents a pipeline configuration might lead to a stale state where no stage wants to fetch the document.
Detecting such edge cases are time consuming and hinders the ability to write complex pipeline configuration.
Hydra core should:
- ... be extended with the ability to detect timeouts
- ... allow configuration of the timeout value
- ... log on error whenever a document times out
- ... track the state of the document as STALE
In addition, test cases should be added where stale pipeline configurations are demonstrated.
For some documents a pipeline configuration might lead to a stale state where no stage wants to fetch the document.
Detecting such edge cases are time consuming and hinders the ability to write complex pipeline configuration.
Hydra core should:
In addition, test cases should be added where stale pipeline configurations are demonstrated.