-
Notifications
You must be signed in to change notification settings - Fork 14
Description
Currently, the configuration property keys are hard-coded in the ingestor (or any custom component jar). This makes it impossible to infer the required configuration properties, given just the jars. It should be possible that an external program (e.g. the hyperdrive-trigger) can read the available configuration properties of a component (through reflection) if the jars are given.
To that end, every ComponentFactory should implement a method that returns a list of configuration properties. Furthermore, this list should also contain information whether a configuration property is required or optional, and some validation rules.
Some components may depend on each other, e.g. the CheckpointOffsetManager requires the KafkaStreamReader to be configured (or at least the property reader.kafka.topic must be defined). This should be taken into account as well.
Possible breakdown of this issue:
- List of config properties with required/optional info.
- Add validation rules to list
- Provide e.g. another method to define dependencies on other components (multiple components, either component A or component B,...)
Migration from older versions
Classes that implemented StreamReaderFactory, OffsetManagerFactory, StreamDecoderFactory, StreamTransformerFactory or StreamWriterFactory must now implement the trait HasComponentAttributes