Skip to content

SQL compatible Null Handling Part 2 - Processing Layer and Druid-SQL changes#5452

Closed
nishantmonu51 wants to merge 50 commits intoapache:masterfrom
nishantmonu51:null-handling-pr-2
Closed

SQL compatible Null Handling Part 2 - Processing Layer and Druid-SQL changes#5452
nishantmonu51 wants to merge 50 commits intoapache:masterfrom
nishantmonu51:null-handling-pr-2

Conversation

@nishantmonu51
Copy link
Copy Markdown
Member

@nishantmonu51 nishantmonu51 commented Mar 1, 2018

Part 2 of changes for SQL Compatible Null Handling -
This PR contains changes to improve handling of null values in druid by treating nulls as missing values which don’t actually exist. This will make it more compatible to SQL standard and help with integration with other existing BI systems which support ODBC/JDBC. (Detailed description in proposal - #4349)

Nullability for Aggregators/Metrics

As per the current implementation, most aggregators are coded to replace null values with default e.g. sum treats them as zeroes, min treats them as positive infinity, etc.
To match the new semantics and make aggregators nullable, where if an aggregator encounters only the null values the result will be null value following changes are made -

  • Aggrgator/BufferAggregator - Added a new method boolean isNull() which returns false by default. aggregators that support nullability can choose to override this and return true if the aggregated result is null.
  • Added a NullableAggregator/NullableBufferAggregator which is be used as a decorator for existing aggregator implementations to make them nullable. This will return null values if all the values aggregated are null, If any of the aggregated value is non-null, it will have same behavior as the delegate aggregator.
  • If new null behavior is enabled, aggregator factories will decorate the aggregators with NullableAggregator/NullableBufferAggregator.
  • Added a new abstract class NullableAggregatorFactory which helps in reducing duplicate code and decide to switch between nullable and non-nullable aggregators.
  • Cardinality and HyperUnique aggregators ignore null values and only count non-null values, If all the encountered values are null, the result will be 0. This is different from current behavior where null values are also counted.
  • Count aggregator - since count aggregator is not associated with any column and just a count of the number of rows, it has the same behavior as before.

Math Expressions Null Handling

  • For general calculations like sum, full expression is be considered as null if any of the components is null.
  • StringLiteral now supports null and and expressions containing non-quoted null is parsed as StringLiteral with null
  • Specifying a default value for null is supported by the use of NVL or IF clause to assign default values at query time.

Filtering on Null values

  • SelectorDimFilter currently specifies filtering on null values but the implementation assumes null and empty strings as equivalent. The implementation is changed to consider null and empty string differently. Generation of cache key for selectorDimFilter is also modified to support null.
  • InFilter/ExpressionFilter/LikeFilter are also modified to not treat nulls as empty strings.

Changes to Druid build-in SQL layer

  • NULL and empty strings are treated differently.
  • IS NULL and IS NOT NULL filters against null values instead of empty strings.
  • NVL, IFNULL and COALESCE work as per SQL standard.
  • count now skips null values and works as per SQL standard.

Backwards Compatibility

  • "druid.generic.useDefaultValueForNull" is added as a new config. The default value is set to true for backwards compatibility.
  • code places in query/ingestion where we converted emptyToNull or nullToEmpty now use NullHandlingHelper which only does the conversion if backwards compatibility is enabled.
  • Storage layer changes are backwards compatible for GenericIndexed/String columns.
  • For Numeric columns new V2 serde implementations are introduced which also supports serde of nullRowsBitmaps. Previous implementations are kept unchanged to read any existing segments.

Testing

  • Most of the unit tests that tested nulls are modified to have two branches of expected results to test new/old behavior both.
  • For CI have added new runs to test matrix to test new behavior for nulls.
  • To run the tests with new behavior run "-Ddruid.generic.useDefaultValueForNull=false"

This change is Reviewable

fix more tests

more test fixes

fix more tests

more test fixes

Fix more tests
fix test failures

more test fixes
fix format

Fix test

Fix tests

more test fix

more test failures

fix formatting
remove unused import

Uniformly set Calcite systemProperties in all tests

formatting fix

fix checkstyle

extract CalciteSetSysteProperties at a common place
@b-slim
Copy link
Copy Markdown
Contributor

b-slim commented Mar 2, 2018

Reviewed 45 of 167 files at r1.
Review status: 45 of 167 files reviewed at latest revision, 1 unresolved discussion.


processing/src/main/java/io/druid/guice/GuiceInjectors.java, line 45 at r1 (raw file):

        new PropertiesModule(Arrays.asList("common.runtime.properties", "runtime.properties")),
        new ConfigModule(),
        new NullHandlingModule(),

I thought this was done part of the previous patch?


Comments from Reviewable

@nishantmonu51
Copy link
Copy Markdown
Member Author

@jon-wei @gianm : Can one of you please review this ?

// Result of any Binary expressions is null if any of the argument is null.
// e.g "select null * 2 as c;" or "select null + 1 as c;" will return null as per Standard SQL spec.
if (NullHandling.sqlCompatible() && (leftVal.isNull() || rightVal.isNull())) {
if (NullHandling.sqlCompatible() && (leftVal.value() == null || rightVal.value() == null)) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this change?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reason is that for String ExprEval isNull method now checks that the expression can be parsed as a valid long/double/float or not instead of checking the nullability.
Have added a comment on the ExprEval isNull method that explains this change.

public static boolean asBoolean(String x)
{
return !Strings.isNullOrEmpty(x) && Boolean.valueOf(x);
return !NullHandling.isNullOrEquivalent(x) && Boolean.valueOf(x);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you implement this thing: #5278 (comment) in this PR?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, will add a checkstyle rule.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added.

@@ -127,7 +128,7 @@ private abstract static class NumericExprEval extends ExprEval<Number>

private NumericExprEval(Number value)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Nullable

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

@@ -99,6 +97,9 @@ public Object value()
return value;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This field should be annotated @Nullable.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

@@ -153,7 +154,7 @@ private static class DoubleExprEval extends NumericExprEval
{
private DoubleExprEval(Number value)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Nullable. @nishantmonu51 could you please self-review this PR and fix all missing @Nullable annotations in this PR?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added @nullable to these places, I already did one pass and added @nullable at places, but its easy to miss the @nullable annotations, Do you have any easy way that i can use to check the missing @nullable annotations ?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IntelliJ usually highlights such places with yellow (e. g. when you return null from a method not annotated @Nullable, or check for non-null an argument not annotated @Nullable)

return NullHandling.replaceWithDefault() ? aggregator : new NullableBufferAggregator(aggregator, selector);
}

protected abstract BufferAggregator factorizeBuffered(
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move below ABSTRACT METHODS BELOW

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

* Implementations of {@link AggregatorFactory} which needs to Support Nullable Aggregations are encouraged
* to extend this class.
*/
public abstract class NullableAggregatorFactory extends AggregatorFactory
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ExtensionPoint

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added.

return getMaxIntermediateSize2() + (NullHandling.replaceWithDefault() ? 0 : Byte.BYTES);
}

// ---- ABSTRACT METHODS BELOW ------
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add docs to those methods, for extension writers.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

*/
public class NullableBufferAggregator implements BufferAggregator
{
private static final byte IS_NULL_BYTE = (byte) 1;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would switch 1 and 0 values. NULL = 0, NOT_NULL = 1 is more natural, plus comparisons with 0 (done below) are cheaper.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

switched.

/**
* The result of a NullableBufferAggregator will be null if all the values to be aggregated are null values or no values are aggregated at all.
* If any of the value is non-null, the result would be the aggregated value of the delegate aggregator.
* Note that the delegate aggregator is not required to perform check for isNull on the columnValueSelector as only non-null values
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please make a javadoc ref

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

@b-slim
Copy link
Copy Markdown
Contributor

b-slim commented Mar 13, 2018

Reviewed 123 of 167 files at r1, 1 of 1 files at r2.
Review status: all files reviewed at latest revision, 29 unresolved discussions, some commit checks failed.


processing/src/main/java/io/druid/query/aggregation/last/DoubleLastAggregatorFactory.java, line 154 at r2 (raw file):

            if (pair.lhs >= lastTime) {
              buf.putLong(position, pair.lhs);
              buf.putDouble(position + Long.BYTES, pair.rhs);

why are we changing this? I know it is the same size but not really needed.


processing/src/main/java/io/druid/query/filter/SelectorDimFilter.java, line 79 at r2 (raw file):

    byte[] extractionFnBytes = extractionFn == null ? new byte[0] : extractionFn.getCacheKey();

    return ByteBuffer.allocate(6 + dimensionBytes.length + valueBytes.length + extractionFnBytes.length)

5 ?


sql/src/main/java/io/druid/sql/calcite/planner/Calcites.java, line 108 at r2 (raw file):

  public static String escapeStringLiteral(final String s)
  {
    Preconditions.checkNotNull(s);

this will throw an exception, is that what should be done? the old code seems to be fine with nulls?


sql/src/main/java/io/druid/sql/calcite/planner/DruidOperatorTable.java, line 121 at r2 (raw file):

          .add(new UnaryPrefixOperatorConversion(SqlStdOperatorTable.NOT, "!"))
          .add(new UnaryPrefixOperatorConversion(SqlStdOperatorTable.UNARY_MINUS, "-"))
          .add(new UnaryFunctionOperatorConversion(SqlStdOperatorTable.IS_NULL, "isnull"))

why not == null or != null ?


Comments from Reviewable

@nishantmonu51
Copy link
Copy Markdown
Member Author

Review status: 128 of 194 files reviewed at latest revision, 29 unresolved discussions.


processing/src/main/java/io/druid/query/aggregation/last/DoubleLastAggregatorFactory.java, line 154 at r2 (raw file):

Previously, b-slim (Slim) wrote…

why are we changing this? I know it is the same size but not really needed.

fixed.


processing/src/main/java/io/druid/query/filter/SelectorDimFilter.java, line 79 at r2 (raw file):

Previously, b-slim (Slim) wrote…

5 ?

fixed.


sql/src/main/java/io/druid/sql/calcite/planner/Calcites.java, line 108 at r2 (raw file):

Previously, b-slim (Slim) wrote…

this will throw an exception, is that what should be done? the old code seems to be fine with nulls?

yes, we never expect null here now.


sql/src/main/java/io/druid/sql/calcite/planner/DruidOperatorTable.java, line 121 at r2 (raw file):

Previously, b-slim (Slim) wrote…

why not == null or != null ?

== and != are binary operators in SQL which when comparing nulls return false as per sql standard.
e.g. Select null == null; will return false
and Select isNull(null) will return true;


Comments from Reviewable

@nishantmonu51
Copy link
Copy Markdown
Member Author

@leventov @b-slim fixed review comments.

@nishantmonu51
Copy link
Copy Markdown
Member Author

@leventov: Updated PR based on your comments, please check.

… UTs

* Extract NullableGroupBycolumnSelectorStrategy - only used when null handling is enabled, fix failing UTs

* fix test after making extracting groupby changes

* Set AWS region in travis
@nishantmonu51
Copy link
Copy Markdown
Member Author

@leventov: Any more comments here ?
Can we get this merged if there are no more blockers ?

@leventov
Copy link
Copy Markdown
Member

@nishantmonu51 I haven't finished my review. I will continue the review soon

final DelimitedParser delegate = new DelimitedParser(
Strings.emptyToNull(delimiter),
Strings.emptyToNull(listDelimiter),
StringUtils.emptyToNullNonDruidDataString(delimiter),
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wrong indentation, and on the next line

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed.

@Override
public String call()
{
// When SQL based null handling is disabled,
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

based -> compatible

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed.

// Do some queries.
Assert.assertEquals(2000, sumMetric(task, null, "rows"));
Assert.assertEquals(2000, sumMetric(task, null, "met1"));
Assert.assertEquals(2000, sumMetric(task, null, "met1").longValue());
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why rows -> met1?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed.

// Do some queries.
Assert.assertEquals(3, sumMetric(task, null, "rows"));
Assert.assertEquals(3, sumMetric(task, null, "met1"));
Assert.assertEquals(3, sumMetric(task, null, "met1").longValue());
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why rows -> met1?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed.

* See io.druid.guice.NullHandlingModule for details.
* It does not take effect in all unit tests since we don't use Guice Injection.
* For tests default system property is supposed to be used only in tests
* default system property is supposed to be used only in tests
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doc comment has proper sentences (starting with capital and ending with dot), please follow this within a comment.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed.

@Override
public long getLong(ByteBuffer buf, int position)
{
if (isNull(buf, position)) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed.

@Override
public double getDouble(ByteBuffer buf, int position)
{
if (isNull(buf, position)) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed.

import java.util.Objects;

public abstract class SimpleDoubleAggregatorFactory extends AggregatorFactory
public abstract class SimpleDoubleAggregatorFactory extends NullableAggregatorFactory
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If NullableAggregatorFactory is generified, it's going to be extends NullableAggregatorFactory<BaseDoubleColumnValueSelector>

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed.

{
Assert.assertEquals(null, loadingLookup.apply(null));
Assert.assertEquals(null, loadingLookup.apply(""));
if (!NullHandling.replaceWithDefault()) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is NullHandling.sqlCompatible()

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed.

// SQL standard spec does not count null values,
// Skip counting null values when we are not replacing null with default value.
// A special value for null in case null handling is configured to use empty string for null.
if (!NullHandling.replaceWithDefault() && !hasNonNullValue && value != null) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NullHandling.sqlCompatible()

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed.

@nishantmonu51
Copy link
Copy Markdown
Member Author

@leventov: Please review. made NullableAggregatorFactory and NullableAggregatorCombiner generic and handled other comments.

import java.util.Objects;

public class FloatFirstAggregatorFactory extends NullableAggregatorFactory
public class FloatFirstAggregatorFactory extends NullableAggregatorFactory<ColumnValueSelector>
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why ColumnValueSelector rather than BaseFloatColumnValueSelector?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It returns a SerializablePair<Long, Float> instead of simple Float

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The parameter of NullableAggregatorFactory is not the output selector type, it's the input selector type. So it should be BaseFloatColumnValueSelector for FloatFirstAggregatorFactory. The reason why this refactoring couldn't be done right now is that getCombiningFactory() returns an anonymous subclass of FloatFirstAggregatorFactory, that IMO wrong. It seems to me that it should be a separate class FloatFirstCombiningAggregatorFactory extends NullableAggregatorFactory<ColumnValueSelector<SerializablePair<Long, Float>>>

import java.util.Objects;

public class DoubleFirstAggregatorFactory extends NullableAggregatorFactory
public class DoubleFirstAggregatorFactory extends NullableAggregatorFactory<ColumnValueSelector>
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why ColumnValueSelector rather than BaseDoubleColumnValueSelector?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It returns a SerializablePair<Long, Double> instead of simple Double

import java.util.Objects;

public class DoubleLastAggregatorFactory extends NullableAggregatorFactory
public class DoubleLastAggregatorFactory extends NullableAggregatorFactory<ColumnValueSelector>
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It returns a Pair instead of primitive object.

import java.util.Objects;

public class LongLastAggregatorFactory extends NullableAggregatorFactory
public class LongLastAggregatorFactory extends NullableAggregatorFactory<ColumnValueSelector>
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It returns a Pair instead of primitive object

import java.util.Objects;

public class FloatLastAggregatorFactory extends NullableAggregatorFactory
public class FloatLastAggregatorFactory extends NullableAggregatorFactory<ColumnValueSelector>
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It returns a Pair instead of primitive object


@Override
public void processValueFromGroupingKey(
GroupByColumnSelectorPlus selectorPlus, ByteBuffer key, Map<String, Object> resultMap, int keyBufferPosition
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Separate lines, and below in this class

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

if (metricVal == null || value == null) {
return metricVal == null && value == null;
}
return HavingSpecMetricComparator.compare(row, aggregationName, value, aggregators) == 0;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could it optimize to not perform row.getRaw() twice?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

if (metricVal == null || value == null) {
return false;
}
return HavingSpecMetricComparator.compare(row, aggregationName, value, aggregators) > 0;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

if (metricVal == null || value == null) {
return false;
}
return HavingSpecMetricComparator.compare(row, aggregationName, value, aggregators) < 0;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

private Ordering<Row> metricOrdering(final String column, final Comparator comparator)
{
return Ordering.from(Comparator.comparing((Row row) -> row.getRaw(column), Comparator.nullsLast(comparator)));
return Ordering.from(Comparator.comparing((Row row) -> row.getRaw(column), Comparator.nullsFirst(comparator)));
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in case of SQL compatibility we need nullsFirst, have modified to not change the behavior for non-sql compatible case to older one.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explain this in a code comment

@nishantmonu51
Copy link
Copy Markdown
Member Author

@leventov : handled comments, do you have any more comments ?

@nishantmonu51
Copy link
Copy Markdown
Member Author

@leventov : have you finished your review ? can this be merged ?

return ((Double) o).compareTo((Double) o1);
}
};
private static final Comparator<Number> COMPARATOR = Comparator.nullsFirst(Comparator.comparingDouble(Number::doubleValue));
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line longer than 120 cols. Might also be in other similar classes, I didn't check.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

import javax.annotation.Nullable;

/**
* The result of a NullableAggregator will be null if all the values to be aggregated are null values or no values are aggregated at all.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

120 cols

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

/**
* The result of a NullableAggregator will be null if all the values to be aggregated are null values or no values are aggregated at all.
* If any of the value is non-null, the result would be the aggregated value of the delegate aggregator.
* Note that the delegate aggregator is not required to perform check for {@link BaseNullableColumnValueSelector#isNull()} on the selector as only non-null values
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

120 cols

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

);

/**
* Creates an {@link AggregateCombiner} to fold rollup aggregation results from serveral "rows" of different indexes during
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

120 cols

@leventov
Copy link
Copy Markdown
Member

leventov commented Jul 5, 2018

@nishantmonu51 after addressing latest comments please squash all commits and create a new PR, it's impossible to work with this one anymore.

@nishantmonu51
Copy link
Copy Markdown
Member Author

Closing in favor of #5958

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants