Skip to content

hard-code compaction tasks to use ARRAY for multi-value handling to preserve order#17110

Merged
clintropolis merged 3 commits intoapache:masterfrom
clintropolis:hardcode-compaction-mvd-mode-array
Sep 19, 2024
Merged

hard-code compaction tasks to use ARRAY for multi-value handling to preserve order#17110
clintropolis merged 3 commits intoapache:masterfrom
clintropolis:hardcode-compaction-mvd-mode-array

Conversation

@clintropolis
Copy link
Copy Markdown
Member

Changes compaction tasks to always use MultiValueHandling.ARRAY if the column schema is not explicitly specified. This should be safe because it will just preserve whatever order the values in the row were stored in on the initial ingestion, and not re-order the values if initially ingested as unordered.

final ColumnHolder columnHolder = index.getColumnHolder(dimension);
if (columnHolder != null) {
DimensionSchema schema = columnHolder.getColumnFormat().getColumnSchema(dimension);
if (schema instanceof StringDimensionSchema) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A comment about why this is happening would be useful. It probably won't be obvious to a reader.

@clintropolis clintropolis merged commit 4f137d2 into apache:master Sep 19, 2024
@clintropolis clintropolis deleted the hardcode-compaction-mvd-mode-array branch September 19, 2024 18:56
@adarshsanjeev adarshsanjeev added this to the 32.0.0 milestone Jan 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants