Skip to content

[fix](Index)Make column unique ids in index dynamically computed (#48988) #49283

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 1 commit into from
Mar 20, 2025
Merged

[fix](Index)Make column unique ids in index dynamically computed (#48988) #49283

dataroaring merged 1 commit into from
Mar 20, 2025

Conversation

@qidaye
Copy link
Contributor

@qidaye qidaye commented Mar 20, 2025

bp #48988

[fix](Index)Make column unique ids in index dynamically computed (apa…
4e40a4d 
…che#48988)

Issue Number: close apache#48989

Related PR: apache#46648 apache#42882

Problem Summary:

We introduced `Index.columnUniqueIds` in 3.0.3, and make sure
`Index.columnUniqueIds` is not empty in 3.0.4.
But when we upgrade Doris from 3.0.2 and before to 3.0.4, the
`Index.columnUniqueIds` will be `null`.

Versions 302 and earlier do not have this variable, and gson
deserialization will assign it to null.
We need to initialize the column unique ids associated with the index
correctly.

Key Changes

Removed stored `columnUniqueIds` from `Index` class:

Previously, column unique IDs were stored as a field in the `Index`
class
Now they're dynamically computed when needed rather than stored
redundantly
Added dynamic computation method:

New method `getColumnUniqueIds(List<Column> schema)` computes IDs at
runtime
Column IDs are looked up by matching index column names with schema
columns

Updated method signatures:

Changed `toThrift()` to `toThrift(List<Integer> indexColumnUniqueIds)`
Changed `toPb()` to `toPb(Map<Integer, Column> columnMap, List<Integer>
indexColumnUniqueIds)`
Updated all callers to pass column IDs as parameters
@qidaye qidaye requested a review from dataroaring as a code owner March 20, 2025 03:08
@hello-stephen
Copy link
Contributor

hello-stephen commented Mar 20, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@qidaye
Copy link
Contributor Author

qidaye commented Mar 20, 2025

run buildall

dataroaring
dataroaring approved these changes Mar 20, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit 6ad6d66 into apache:branch-3.0 Mar 20, 2025
22 of 24 checks passed
@qidaye qidaye deleted the pick_48988_3.0 branch March 20, 2025 07:11
dataroaring pushed a commit that referenced this pull request Mar 20, 2025
[fix](Index)Make column unique ids in index dynamically computed (#48988
c896e97 
) (#49283)

bp #48988
qidaye pushed a commit to qidaye/incubator-doris that referenced this pull request Mar 20, 2025
[fix](compile)Fix FE compile for apache#49283
5e0b5ac
dataroaring pushed a commit that referenced this pull request Mar 25, 2025
[fix](compile)Fix FE compile for #49283 (#49327)
c79c6c1
@gavinchou gavinchou mentioned this pull request Apr 23, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@qidaye @hello-stephen @dataroaring