Skip to content

[chore] Reduce range of nextId when calling advanceNextId() #40160

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dataroaring merged 4 commits into from
Sep 3, 2024
Merged

[chore] Reduce range of nextId when calling advanceNextId() #40160

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
c7836b8
[chore] Reduce range of nextId when calling advanceNextId()
gavinchou Aug 29, 2024
16a6b2a
Update fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
gavinchou Aug 30, 2024
b66748c
Update fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
gavinchou Aug 30, 2024
e4942f8
Merge branch 'master' into gavin-reduce-advance-next-id
gavinchou Aug 31, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1679,10 +1679,15 @@ private void transferToMaster() {
*/
void advanceNextId() {
long currentId = idGenerator.getBatchEndId();
long currentNanos = System.nanoTime();
long currentMill = System.currentTimeMillis();
long nextId = currentId + 1;
if (nextId < currentNanos) {
nextId = currentNanos;
// Reserve ~1 trillion for use in case of bugs or frequent reboots (~2 billion reboots)
if ((1L << 63) - nextId < (1L << 40)) {
LOG.warn("nextId is too large: {}, it may be a bug and consider backup and migration", nextId);
} else {
// Keep compatible with previous impl, the previous impl may result in extreme large nextId,
// and guess there are no more than 1L<<32 (~4e9) ids used since last reboot
nextId = (currentId + 1) < currentMill ? currentMill : currentId + (1L << 32);
}

// ATTN: Because MetaIdGenerator has guaranteed that each id it returns must have
Expand Down