Description
Though we could make kafka index work by using avro and kafka extensions, the implementation of the avro part should support Schema Registry with authentications, since normally, if you use Confluent Cloud, the Schema Registry part would have a basic auth mechanism for security reason.
The implementation would be quite straightforward, the current implementation is in the data/input/avro/SchemaRegistryBasedAvroBytesDecoder.java. We could instantiate the CachedSchemaRegistryClient with additional configuration, then authentication and other configs of schema registry could be easily added.
We could add something like this
"avroBytesDecoder": { "type": "schema_registry", "url": "http://schema-registry:8081", "config": { }, }
and replace this line https://github.com/apache/incubator-druid/blob/49bd16766fb347e1aad222b8da7f83bb9e951cde/extensions-core/avro-extensions/src/main/java/org/apache/druid/data/input/avro/SchemaRegistryBasedAvroBytesDecoder.java#L48
Motivation
Support Confluent Schema Registry with authentication
Description
Though we could make kafka index work by using avro and kafka extensions, the implementation of the avro part should support Schema Registry with authentications, since normally, if you use Confluent Cloud, the Schema Registry part would have a basic auth mechanism for security reason.
The implementation would be quite straightforward, the current implementation is in the data/input/avro/SchemaRegistryBasedAvroBytesDecoder.java. We could instantiate the CachedSchemaRegistryClient with additional configuration, then authentication and other configs of schema registry could be easily added.
We could add something like this
"avroBytesDecoder": { "type": "schema_registry", "url": "http://schema-registry:8081", "config": { }, }and replace this line https://github.com/apache/incubator-druid/blob/49bd16766fb347e1aad222b8da7f83bb9e951cde/extensions-core/avro-extensions/src/main/java/org/apache/druid/data/input/avro/SchemaRegistryBasedAvroBytesDecoder.java#L48
Motivation
Support Confluent Schema Registry with authentication