Remove deprecated maxRowsPerSegment from batch task tuning configs

docs/api-reference/tasks-api.md

@@ -756,16 +756,13 @@ Host: http://ROUTER_IP:ROUTER_PORT
             },
             "tuningConfig": {
                 "type": "index_parallel",
-                "maxRowsPerSegment": 5000000,
                 "appendableIndexSpec": {
                     "type": "onheap",
                     "preserveExistingMetrics": false
                 },
                 "maxRowsInMemory": 25000,
                 "maxBytesInMemory": 0,
                 "skipBytesInMemoryOverheadCheck": false,
-                "maxTotalRows": null,
-                "numShards": null,
                 "splitHintSpec": null,
                 "partitionsSpec": {
                     "type": "dynamic",
@@ -1282,7 +1279,6 @@ curl "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/task" \
     },
     "tuningConfig" : {
       "type" : "index_parallel",
-      "maxRowsPerSegment" : 5000000,
       "maxRowsInMemory" : 25000
     }
   }
@@ -1334,7 +1330,6 @@ Content-Length: 952
     },
     "tuningConfig" : {
       "type" : "index_parallel",
-      "maxRowsPerSegment" : 5000000,
       "maxRowsInMemory" : 25000
     }
   }

docs/ingestion/native-batch-simple-task.md

@@ -138,8 +138,6 @@ The tuningConfig is optional and default parameters will be used if no tuningCon
 |type|The task type, this should always be "index".|none|yes|
 |maxRowsInMemory|Used in determining when intermediate persists to disk should occur. Normally user does not need to set this, but depending on the nature of data, if rows are short in terms of bytes, user may not want to store a million rows in memory and this value should be set.|1000000|no|
 |maxBytesInMemory|Used in determining when intermediate persists to disk should occur. Normally this is computed internally and user does not need to set it. This value represents number of bytes to aggregate in heap memory before persisting. This is based on a rough estimate of memory usage and not actual usage. The maximum heap memory usage for indexing is maxBytesInMemory * (2 + maxPendingPersists). Note that `maxBytesInMemory` also includes heap usage of artifacts created from intermediary persists. This means that after every persist, the amount of `maxBytesInMemory` until next persist will decreases, and task will fail when the sum of bytes of all intermediary persisted artifacts exceeds `maxBytesInMemory`.|1/6 of max JVM memory|no|
-|maxTotalRows|Deprecated. Use `partitionsSpec` instead. Total number of rows in segments waiting for being pushed. Used in determining when intermediate pushing should occur.|20000000|no|
-|numShards|Deprecated. Use `partitionsSpec` instead. Directly specify the number of shards to create. If this is specified and `intervals` is specified in the `granularitySpec`, the index task can skip the determine intervals/partitions pass through the data.|null|no|
 |partitionDimensions|Deprecated. Use `partitionsSpec` instead. The dimensions to partition on. Leave blank to select all dimensions. Only used with `forceGuaranteedRollup` = true, will be ignored otherwise.|null|no|
 |partitionsSpec|Defines how to partition data in each timeChunk, see [PartitionsSpec](#partitionsspec)|`dynamic` if `forceGuaranteedRollup` = false, `hashed` if `forceGuaranteedRollup` = true|no|
 |indexSpec|Defines segment storage format options to be used at indexing time, see [IndexSpec](ingestion-spec.md#indexspec)|null|no|

docs/ingestion/native-batch.md

@@ -261,8 +261,6 @@ The following table lists the properties of a `tuningConfig` object:
 |`maxRowsInMemory`|Determines when Druid should perform intermediate persists to disk. Normally you don't need to set this. Depending on the nature of your data, if rows are short in terms of bytes. For example, you may not want to store a million rows in memory. In this case, set this value.|1000000|no|
 |`maxBytesInMemory`|Use to determine when Druid should perform intermediate persists to disk. Normally Druid computes this internally and you don't need to set it. This value represents number of bytes to aggregate in heap memory before persisting. This is based on a rough estimate of memory usage and not actual usage. The maximum heap memory usage for indexing is `maxBytesInMemory * (2 + maxPendingPersists)`. Note that `maxBytesInMemory` also includes heap usage of artifacts created from intermediary persists. This means that after every persist, the amount of `maxBytesInMemory` until next persist will decrease. Tasks fail when the sum of bytes of all intermediary persisted artifacts exceeds `maxBytesInMemory`.|1/6 of max JVM memory|no|
 |`maxColumnsToMerge`|Limit of the number of segments to merge in a single phase when merging segments for publishing. This limit affects the total number of columns present in a set of segments to merge. If the limit is exceeded, segment merging occurs in multiple phases. Druid merges at least 2 segments per phase, regardless of this setting.|-1 (unlimited)|no|
-|`maxTotalRows`|Deprecated. Use `partitionsSpec` instead. Total number of rows in segments waiting to be pushed. Used to determine when intermediate pushing should occur.|20000000|no|
-|`numShards`|Deprecated. Use `partitionsSpec` instead. Directly specify the number of shards to create when using a `hashed` `partitionsSpec`. If this is specified and `intervals` is specified in the `granularitySpec`, the index task can skip the determine intervals/partitions pass through the data.|null|no|
 |`splitHintSpec`|Hint to control the amount of data that each first phase task reads. Druid may ignore the hint depending on the implementation of the input source. See [Split hint spec](#split-hint-spec) for more details.|size-based split hint spec|no|
 |`partitionsSpec`|Defines how to partition data in each timeChunk, see [PartitionsSpec](#partitionsspec).|`dynamic` if `forceGuaranteedRollup` = false, `hashed` or `single_dim` if `forceGuaranteedRollup` = true|no|
 |`indexSpec`|Defines segment storage format options to be used at indexing time, see [IndexSpec](ingestion-spec.md#indexspec).|null|no|
@@ -659,8 +657,6 @@ The returned result contains the worker task spec, a current task status if exis
           "type": "dynamic"
         },
         "maxRowsInMemory": 1000000,
-        "maxTotalRows": 20000000,
-        "numShards": null,
         "indexSpec": {
           "bitmap": {
             "type": "roaring"

extensions-contrib/kubernetes-overlord-extensions/src/test/java/org/apache/druid/k8s/overlord/common/K8sTestUtils.java

@@ -85,16 +85,10 @@ public static Task getTask()
                 false
             ),
             new IndexTask.IndexTuningConfig(
-                null,
-                null,
                 null,
                 10,
                 null,
                 null,
-                null,
-                9999,
-                null,
-                null,
                 new DynamicPartitionsSpec(10000, null),
                 INDEX_SPEC,
                 null,
@@ -107,7 +101,6 @@ public static Task getTask()
                 null,
                 null,
                 null,
-                null,
                 1L,
                 null
             )

extensions-core/multi-stage-query/src/test/java/org/apache/druid/msq/indexing/MSQCompactionRunnerTest.java

@@ -431,7 +431,6 @@ private static CompactionTask.CompactionTuningConfig createTuningConfig(
         .forCompactionTask()
         .withMaxRowsInMemory(500000)
         .withMaxBytesInMemory(1000000L)
-        .withMaxTotalRows(Long.MAX_VALUE)
         .withPartitionsSpec(partitionsSpec)
         .withIndexSpec(indexSpec)
         .withForceGuaranteedRollup(!(partitionsSpec instanceof DynamicPartitionsSpec))

indexing-service/src/main/java/org/apache/druid/indexing/common/task/CompactionTask.java

@@ -253,14 +253,10 @@ static CompactionTuningConfig getTuningConfig(TuningConfig tuningConfig)
     } else if (tuningConfig instanceof ParallelIndexTuningConfig) {
       final ParallelIndexTuningConfig parallelIndexTuningConfig = (ParallelIndexTuningConfig) tuningConfig;
       return new CompactionTuningConfig(
-          null,
-          parallelIndexTuningConfig.getMaxRowsPerSegment(),
           parallelIndexTuningConfig.getAppendableIndexSpec(),
           parallelIndexTuningConfig.getMaxRowsInMemory(),
           parallelIndexTuningConfig.getMaxBytesInMemory(),
           parallelIndexTuningConfig.isSkipBytesInMemoryOverheadCheck(),
-          parallelIndexTuningConfig.getMaxTotalRows(),
-          parallelIndexTuningConfig.getNumShards(),
           parallelIndexTuningConfig.getSplitHintSpec(),
           parallelIndexTuningConfig.getPartitionsSpec(),
           parallelIndexTuningConfig.getIndexSpec(),
@@ -288,14 +284,10 @@ static CompactionTuningConfig getTuningConfig(TuningConfig tuningConfig)
     } else if (tuningConfig instanceof IndexTuningConfig) {
       final IndexTuningConfig indexTuningConfig = (IndexTuningConfig) tuningConfig;
       return new CompactionTuningConfig(
-          null,
-          indexTuningConfig.getMaxRowsPerSegment(),
           indexTuningConfig.getAppendableIndexSpec(),
           indexTuningConfig.getMaxRowsInMemory(),
           indexTuningConfig.getMaxBytesInMemory(),
           indexTuningConfig.isSkipBytesInMemoryOverheadCheck(),
-          indexTuningConfig.getMaxTotalRows(),
-          indexTuningConfig.getNumShards(),
           null,
           indexTuningConfig.getPartitionsSpec(),
           indexTuningConfig.getIndexSpec(),
@@ -1202,25 +1194,17 @@ public static CompactionTuningConfig defaultConfig()
           null,
           null,
           null,
-          null,
-          null,
-          null,
-          null,
           0L,
           null
       );
     }
 
     @JsonCreator
     public CompactionTuningConfig(
-        @JsonProperty("targetPartitionSize") @Deprecated @Nullable Integer targetPartitionSize,
-        @JsonProperty("maxRowsPerSegment") @Deprecated @Nullable Integer maxRowsPerSegment,
         @JsonProperty("appendableIndexSpec") @Nullable AppendableIndexSpec appendableIndexSpec,
         @JsonProperty("maxRowsInMemory") @Nullable Integer maxRowsInMemory,
         @JsonProperty("maxBytesInMemory") @Nullable Long maxBytesInMemory,
         @JsonProperty("skipBytesInMemoryOverheadCheck") @Nullable Boolean skipBytesInMemoryOverheadCheck,
-        @JsonProperty("maxTotalRows") @Deprecated @Nullable Long maxTotalRows,
-        @JsonProperty("numShards") @Deprecated @Nullable Integer numShards,
         @JsonProperty("splitHintSpec") @Nullable SplitHintSpec splitHintSpec,
         @JsonProperty("partitionsSpec") @Nullable PartitionsSpec partitionsSpec,
         @JsonProperty("indexSpec") @Nullable IndexSpec indexSpec,
@@ -1248,14 +1232,10 @@ public CompactionTuningConfig(
     )
     {
       super(
-          targetPartitionSize,
-          maxRowsPerSegment,
           appendableIndexSpec,
           maxRowsInMemory,
           maxBytesInMemory,
           skipBytesInMemoryOverheadCheck,
-          maxTotalRows,
-          numShards,
           splitHintSpec,
           partitionsSpec,
           indexSpec,
@@ -1294,14 +1274,10 @@ public CompactionTuningConfig(
     public CompactionTuningConfig withPartitionsSpec(PartitionsSpec partitionsSpec)
     {
       return new CompactionTuningConfig(
-          null,
-          null,
           getAppendableIndexSpec(),
           getMaxRowsInMemory(),
           getMaxBytesInMemory(),
           isSkipBytesInMemoryOverheadCheck(),
-          null,
-          null,
           getSplitHintSpec(),
           partitionsSpec,
           getIndexSpec(),