groupBy v2: Ignore timestamp completely when granularity = all, except for the final merge.

[gianm]

~30% improvement on query benchmarks that include serde time (queryMultiQueryableIndexWithSerde).

Specifically:

• Remove timestamp from RowBasedKey when not needed.
• Set timestamp to null in MapBasedRows that are not part of the final merge.
• Added two new benchmarks, queryMultiQueryableIndexWithSerde (simulates serde between historical and broker) and queryMultiQueryableIndexWithSpilling (includes spilling to disk) that show the improvement here, which is mostly from having less serde work to do.

groupby-improvements

Benchmark                                              (defaultStrategy)  (initialBuckets)  (numProcessingThreads)  (numSegments)  (queryGranularity)  (rowsPerSegment)  (schemaAndQuery)  Mode  Cnt        Score       Error  Units
GroupByBenchmark.queryMultiQueryableIndex                             v2                -1                       2              4                 all            100000           basic.A  avgt   30   373040.126 ±  6687.778  us/op
GroupByBenchmark.queryMultiQueryableIndex                             v2                -1                       2              4                 day            100000           basic.A  avgt   30   704732.206 ±  9292.572  us/op
GroupByBenchmark.queryMultiQueryableIndexWithSerde                    v2                -1                       2              4                 all            100000           basic.A  avgt   30   486083.016 ±  4252.756  us/op
GroupByBenchmark.queryMultiQueryableIndexWithSerde                    v2                -1                       2              4                 day            100000           basic.A  avgt   30  1028039.357 ± 11060.639  us/op
GroupByBenchmark.queryMultiQueryableIndexWithSpilling                 v2                -1                       2              4                 all            100000           basic.A  avgt   30   444659.485 ±  5380.572  us/op
GroupByBenchmark.queryMultiQueryableIndexWithSpilling                 v2                -1                       2              4                 day            100000           basic.A  avgt   30   532730.064 ±  6565.590  us/op
GroupByBenchmark.querySingleIncrementalIndex                          v2                -1                       2              4                 all            100000           basic.A  avgt   30    75440.164 ±  1382.679  us/op
GroupByBenchmark.querySingleIncrementalIndex                          v2                -1                       2              4                 day            100000           basic.A  avgt   30    76651.784 ±  1288.932  us/op
GroupByBenchmark.querySingleQueryableIndex                            v2                -1                       2              4                 all            100000           basic.A  avgt   30    37673.145 ±   689.344  us/op
GroupByBenchmark.querySingleQueryableIndex                            v2                -1                       2              4                 day            100000           basic.A  avgt   30    40981.706 ±  1276.147  us/op

master

Benchmark                                              (defaultStrategy)  (initialBuckets)  (numProcessingThreads)  (numSegments)  (queryGranularity)  (rowsPerSegment)  (schemaAndQuery)  Mode  Cnt        Score       Error  Units
GroupByBenchmark.queryMultiQueryableIndex                             v2                -1                       2              4                 all            100000           basic.A  avgt   30   366566.054 ±  5324.826  us/op
GroupByBenchmark.queryMultiQueryableIndex                             v2                -1                       2              4                 day            100000           basic.A  avgt   30   704091.741 ±  7608.095  us/op
GroupByBenchmark.queryMultiQueryableIndexWithSerde                    v2                -1                       2              4                 all            100000           basic.A  avgt   30   676987.839 ± 10675.159  us/op
GroupByBenchmark.queryMultiQueryableIndexWithSerde                    v2                -1                       2              4                 day            100000           basic.A  avgt   30  1015616.319 ± 12242.756  us/op
GroupByBenchmark.queryMultiQueryableIndexWithSpilling                 v2                -1                       2              4                 all            100000           basic.A  avgt   30   487088.000 ±  6308.555  us/op
GroupByBenchmark.queryMultiQueryableIndexWithSpilling                 v2                -1                       2              4                 day            100000           basic.A  avgt   30   536788.568 ± 10157.553  us/op
GroupByBenchmark.querySingleIncrementalIndex                          v2                -1                       2              4                 all            100000           basic.A  avgt   30    75694.447 ±  1499.463  us/op
GroupByBenchmark.querySingleIncrementalIndex                          v2                -1                       2              4                 day            100000           basic.A  avgt   30    76950.858 ±  1223.638  us/op
GroupByBenchmark.querySingleQueryableIndex                            v2                -1                       2              4                 all            100000           basic.A  avgt   30    37716.549 ±   938.930  us/op
GroupByBenchmark.querySingleQueryableIndex                            v2                -1                       2              4                 day            100000           basic.A  avgt   30    38152.708 ±   548.103  us/op

[fjy]

👍

[fjy]

@gianm there's some conflicts

[gianm]

@fjy @jon-wei updated

[jon-wei]

👍 after travis