math expression support

[himanshug]

this PR adds a math expression implementation to be used in #1965, possibly as a post aggregator and some other places to query. I will send/update those PR once this one is merged.

pls see math-expr.md for the quick overview/documentation.

also, experimental DruidSQL.g4 is removed because it was failing with newer version of ANTLR, can bring it back and fix if/when needed.

[xvrl]

@himanshug any reason we are rolling our own parser / lexer instead of levering antlr or something similar?

[himanshug]

@xvrl mostly because parser/lexer is simple/small for this grammar (see the number of lines with real code in Lexer/Parser class), having it implemented here means less magic. All the rest of the code would be needed whether you used ANTLR or not.

[xvrl]

@himanshug we already have a build-time dependency for antlr that's used for the toy sql grammar I wrote a while back. We could use antlr to remove all the boilerplate code and reduce the chance of bugs. I think it would make it easier to extend / understand for other folks, and remove a lot of boilerplate code.
Here's what the sql one looked like:
https://github.com/druid-io/druid/blob/master/server/src/main/antlr4/io/druid/sql/antlr4/DruidSQL.g4

[rasahner]

Precedence is generally shown in decreasing order.
There's a missing "the" in front of "following operators".

[himanshug]

@xvrl i agree with "understandability and extensibility" point, I will try to generate lexer/parser with antlr.

[himanshug]

@xvrl generated stuff with ANTLR .
@rasahner comments addressed

[rasahner]

needs "the" before "following" - sorry I didn't say that before.

[himanshug]

changed

[fjy]

this should be in query docs

[fjy]

@himanshug we need docs on how to use this in queries

[himanshug]

@fjy language introduced in this PR is not directly usable, it will be made availble to druid queries via #1965 and the doc for same will also appear there.

[jon-wei]

👍

This can be for later, but any thoughts on whether it's useful to expand this with more supported math expressions? e.g. trigonometric functions, ln, e^x?

[himanshug]

@jon-wei yes, I will be adding more math functions, this PR sets up the framework really... adding more math functions would be trivial.

[navis]

@himanshug Is it necessary to remove our toy-sql runner? :)

[himanshug]

@navis had to remove that because it was written with an older version of antlr and was broken with antlr4. older version required writing grammar and implementation details in the same file which was messy.

[navis]

👍 wanna use this.

[navis]

@himanshug Could you rebase this? Let get this in.

[navis]

@fjy The sole conflict is from DruidSql.g4, which was removed in this PR. I've asked @himanshug on it before but can we remove it and proceed?

[himanshug]

@navis @fjy rebased to fix conflicts, will update #1965 once this one is merged to use this.

[fjy]

@himanshug This needs docs

[himanshug]

@fjy pls see #2090 (comment)

[fjy]

👍

[fjy]

@xvrl I can't not commit and squash?

[fjy]

@xvrl nvm updated settings

[xvrl]

@fjy from now on you just hit the "confirm squash and merge" button, I disabled the old merge as indicated in the mailing list.

[giaosudau]

This PR delete ExprLexer and ExprParser and those classes still a dependency of alot of components.
For example in: package io.druid.math.expr.Parser

import com.google.common.collect.ImmutableMap;
import com.google.common.collect.Maps;
import com.metamx.common.logger.Logger;
import io.druid.math.expr.antlr.ExprLexer;
import io.druid.math.expr.antlr.ExprParser;
import org.antlr.v4.runtime.ANTLRInputStream;
import org.antlr.v4.runtime.CommonTokenStream;
import org.antlr.v4.runtime.tree.ParseTree;
import org.antlr.v4.runtime.tree.ParseTreeWalker;