The "regex" extractionFn needs a "retainMissingValue" to be useful

[vogievetsky]

Ref: http://druid.io/docs/latest/querying/dimensionspecs.html

The current Regular Expression Extraction Function is almost useful but it needs the extra trimmings that can be found on the lookup extraction function. Specifically the fact that it "If there is no match, it returns the dimension value as is." is not useful. Ideally I want it to send anything that does not match the regexp to "null".

I believe that this functionality can be achieved (without breaking backwards compatibility) by adding the "retainMissingValue", "injective", and "replaceMissingValueWith" properties that can be found on the lookup (retainMissingValue should be true by default to preserve backwards computability).

This is my use case:

Say I have a dimensions of files the were downloaded from my web server:

Files:

• index.html
• the_end_is_near_2.html
• kafka-0.6.2.tar.gz
• kafka-0.6.1.tar.gz
• kafka-0.5.9.tar.gz

I would like to extract the version number (make a derived dimension) at query time.
I want to run this regexp: (\d+\.\d+\.\d+) and I want index.html and the_end_is_near_2.html to be transformed to null (not kept as is).

[gianm]

Usually we have things be false by default, so maybe we should call retainMissingValue whatever the opposite of retainMissingValue is. Or make a new extraction function and deprecate "regex"…

[vogievetsky]

Yep, I was aware of that when I suggested and honestly 'false' is a more useful default IMO but it would break backwards compatibility. I have a crazy alt. idea: leave "regex" like it is and make a new "regexp" function. Or call it "match".

I think being consistent about retainMissingValue should be top priority.

[jon-wei]

I'll take a look at this

[gianm]

Fixed by #2075