Add listening-announcer extension

[drcrallen]

This extension is proposed as a means for "capability" discovery. For this discussion any specific capability is a group of resources which can be defined by IDs and are essentially cluster-wide resources (resource with unique id across all of cluster)

The most notable example of things which would fit under "capability" are

• Serving deep storage Segments for querying
• Lookup namespaces used in querying
• Tasks (maybe?)

This PR proposes a different paradigm for such items such that all services which can host capacity based on IDs can guarantee a very basic post/get/delete. Any extra functionality beyond a simple post/get/delete is not guaranteed by this paradigm, but any implementation may wish to expose a SEPARATE endpoint with the expanded functionality. An such an implementation is encouraged to use the mechanisms in this PR for capability discovery.

This PR uses ZK/Curator ServiceDiscovery to find the instances with a capability, and relies on http REST calls to manage the post/get/deletes to the instances.

This PR does not have any concept of state, and any particular implementation is expected to manage its own state and maintain its own reconciliation expectations on cluster state conflicts.

An example of a use case can be seen in https://github.com/metamx/druid/blob/queryTimeLookup_announce/extensions/namespace-lookup/src/main/java/io/druid/server/namespace/NamespacedExtractionModule.java#L262

TODO:

• Reconcile this PR with [WIP]Use Announcer for internal announcements + ClusterInforResource #2242 ( see [WIP]Use Announcer for internal announcements + ClusterInforResource #2242 (comment) )
• Annotate the curator stuff so that a different ZK can be used at some point. Will use [WIP]Use Announcer for internal announcements + ClusterInforResource #2242 instead when it is ready
• Ensure accommodation of Separate internal node discovery from external service discovery #2040

[drcrallen]

@nishantmonu51 this is the core part of what I was speaking to you about the other day

[drcrallen]

@cheddar / @b-slim I was planning on using this to communicate namespace lookups around to the cluster. Please let me know if you have any comments.

[drcrallen]

If accepted as part of mainline druid this would probably merge into ZkPathsConfig

[guobingkun]

@drcrallen I wonder if this PR overlaps #2242, where I've been working on separating internal/external announcement and Druid service discovery.

[drcrallen]

@guobingkun partly. which is why I put it up as quickly as I could. I was using the approach from this PR in the metadata propagation for QTL, but thought it might be able to be used for stuff at a wider capacity.

There are separate parts of your PR compared to this one. It would be a good idea for you, myself, and @xvrl to sync up to come up with a unified approach and distinction of responsibilities for the micro-components in the PRs.

[drcrallen]

As per @cheddar 's comments in #2040 (comment) this probably should start off with an Announcer-like interface rather than using ServiceDiscovery.

[drcrallen]

Spoke with @cheddar a bit yesterday and will be implementing some of the suggestions. namely: move the AbstractGET/POST/DELETE to a single class.

[drcrallen]

This may or may not be desired in general. I find it handy for the lookup stuff.

[drcrallen]

I went with including this with the lookup stuff instead of separately. Closing down for now.