Skip to content

[QTL][Lookup] LookupExtractor configuration management. #2328

@b-slim

Description

@b-slim

In order to bring the query time lookup [QTL] to a production ready state, Druid need to have a central configuration and management layer.
This management layer is suppose to provide:
1 Static (via property file) and Dynamic (via Coordinator at runtime) registration/unregistration of LookupExtractor implementations.
2 Periodic checkpointing of LookupExtractor instances, in order to be able to restart after failures or manual restart of the druid process.
This layer of management can be split to XX pieces:
-1 LookupRefManager that exposes listing/adding/deleting of LookupExtractor references PR 2291.
-2 LookupHttpEndPointResource that exposes listing/adding/deleting via HTTP endpoint (this resource will depend on point 1 LookupRefManager).
-3 LookupConfigurationLoader This piece will be responsible to load configuration for runtime property file, checkpoint periodically the current lookupExtracor object and reload both files after restart.
-4 LookupCoordinator This piece will be running on the coordinators and will perform distributed configuration of lookupExtractor.
All those pieces will be part of the druid core in order to guaranty homogeneous and clear way to manage LookupExtractor, then every one if free to actually implement the LookupExtractor interface.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions