Improve the DruidRexExecutor w.r.t handling of numeric arrays#11968
Improve the DruidRexExecutor w.r.t handling of numeric arrays#11968abhishekagarwal87 merged 4 commits intoapache:masterfrom
Conversation
clintropolis
left a comment
There was a problem hiding this comment.
👍 could you add a test for this in https://github.com/apache/druid/blob/master/sql/src/test/java/org/apache/druid/sql/calcite/planner/DruidRexExecutorTest.java
|
Added the testcases, and fixed the older incorrect ones which were known. |
| .map(BigDecimal::valueOf) | ||
| .collect(Collectors.toList()); | ||
| literal = rexBuilder.makeLiteral(resultAsBigDecimalList, constExp.getType(), true); | ||
| } else { |
There was a problem hiding this comment.
would be better if you can add a check for double type here
There was a problem hiding this comment.
An else case is required to stop the "Variable might not have been initialized" exception from cropping up, and I didn't want to provide a failsafe else block. In case when it's not an array (https://github.com/LakshSingla/druid/blob/4dcdb2d68563302a9b4d387cdbb220ec4f3a59b3/sql/src/main/java/org/apache/druid/sql/calcite/planner/DruidRexExecutor.java#L130), it was using the else case as well, that's why I omitted the else if.
One approach can be to initialize the literal to null and then including your suggestion. Let me know if that seems better, and I can add that in a separate commit.
Description
Issue:
DruidRexExecutorwhile reducing Arrays, specially numeric arrays, doesn't convert the value from ExprResult's type to BigDecimal, which causesmakeLiteralto cast the values. Also, if NaN or Infinite values are present in the array, the error is a genericNumberFormatException. For example:SELECT ARRAY[1.11, 2.22]returns [1, 2]SELECT SQRT(-1)throws a genericNumberFormatExceptioninstead ofIAEThis PR introduces change to cast the numeric values to
BigDecimalsince Calcite's library understands that easily, and doesn't perform casts.Key changed/added classes in this PR
DruidRexExecutor#reduce()This PR has: