[Proposal] Support multiple dimensions in Dimension Spec

[sirpkt]

Currently, Druid supports value transformation of only one dimension because DimensionSpec accepts single input dimension.
I want to extend DimensionSpec to accept inputs from multiple dimensions especially in order to enable lookup based join with a small table on multiple columns as mentioned https://groups.google.com/forum/#!searchin/druid-development/multiple/druid-development/OUnXaRgEZG0/PnOYxmgpBwAJ.
This is different from #2374 which targets generating multiple output values from one input value.

As DimensionSpec is not limited for lookup, there can be other benefits from multi-dimensional support.
For example, with #2292, query time arithmetic calculation with multiple dimensions will be possible, which is currently supported only for post aggregated metric values.

Approach to support multiple dimensions that I think is as follows:

1. changing DimensionSpec to accept list of dimensions
2. enabling StorageAdapters to provide dimensionSelector for multi-dimensional DimensionSpec
   I think makeDimensionSelectorUndecorated() is target method to change and
   IncrementalIndex, IncrementalIndexStorageAdapter, and QueryableIndexStorageAdapter are target classes to change.
3. forcing Cursor-based processing for queries with multi-dimensional DimensionSpec
   Currently, only search query uses bitmap processing. Just changing to use cursor-based processing when multi-dimensional spec is used.
4. enabling ExtractionFn to accept list of String as input
   ExtractionFn of multi-dimensional inputs implements already existing interface apply(Object) with check routine that Object is List<String>.

And I'm thinking about generalization of lookup to support not only String to String mapping but also List<String> to String mapping.
My approach to support that is as follows:

1. changing lookup map from Map<String, String> to Map<Object, String>
   I think this is the way to minimize the side effect on existing code and easily supporting multi-dimensional key
2. multi-dimensional lookup uses Map<MultiKey, String>
   I'm try to use org.apache.commons.collections.keyvalue.MultiKey to represent multi-dimensional lookup key

[b-slim]

@sirpkt which lookup map you are talking about ?

changing lookup map from Map<String, String> to Map<Object, String>
I think this is the way to minimize the side effect on existing code and easily supporting multi-dimensional key

[sirpkt]

@b-slim I mean the Map<String, String> structure of MapLookupExtractor which is given as a parameter map.
In the case of NamespacedExtractor, Function<String, String> is used for lookup transformation.
I want to change that Map or Function to support transforming not only String to String but also list of String to String.
I modified above script as < String, String > was recognized as Markdown and not shown.

[stale]

This issue has been marked as stale due to 280 days of inactivity. It will be closed in 2 weeks if no further activity occurs. If this issue is still relevant, please simply write any comment. Even if closed, you can still revive the issue at any time or discuss it on the dev@druid.apache.org list. Thank you for your contributions.

[stale]

This issue has been closed due to lack of activity. If you think that is incorrect, or the issue requires additional review, you can revive the issue at any time.