Kafka indexing service has been being actively developed since it was first introduced. As requirements increase, a ton of new cool features have also been added for Kafka indexing service like #4815. However, the active development led frequent code changes and introduced more bugs than before. (See this discussion for 0.12.2 release. The release has postponed to fix more bugs in kafka indexing service.)
I think one of the reasons causing more bugs is its code complexity. It is very complicated so that people should read the entire class (1700 lines) again and again to exactly understand how it works, at least for me. This complex code make it difficult to make kafka index task more robust by handling all possible exceptional cases. So, I think it wouldn't be easy to make kafka indexing service more stable with the current implementation.
KafkaIndexTask changes its state based on the events from the kafka supervisor, so basically I think we can do better by adopting the event-driven architecture to it.
I have been working on fixing some bugs as well as some simple refactoring issues for kafka indexing service (like #5854), and am also available for this issue. Actually, I really want to do this work.
Kafka indexing service has been being actively developed since it was first introduced. As requirements increase, a ton of new cool features have also been added for Kafka indexing service like #4815. However, the active development led frequent code changes and introduced more bugs than before. (See this discussion for 0.12.2 release. The release has postponed to fix more bugs in kafka indexing service.)
I think one of the reasons causing more bugs is its code complexity. It is very complicated so that people should read the entire class (1700 lines) again and again to exactly understand how it works, at least for me. This complex code make it difficult to make kafka index task more robust by handling all possible exceptional cases. So, I think it wouldn't be easy to make kafka indexing service more stable with the current implementation.
KafkaIndexTask changes its state based on the events from the kafka supervisor, so basically I think we can do better by adopting the event-driven architecture to it.
I have been working on fixing some bugs as well as some simple refactoring issues for kafka indexing service (like #5854), and am also available for this issue. Actually, I really want to do this work.