perf: various btree performance improvements#5446
perf: various btree performance improvements#5446westonpace merged 9 commits intolance-format:mainfrom
Conversation
|
Other potential improvements:
|
|
This drops the |
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
BubbleCal
left a comment
BubbleCal
left a comment
There was a problem hiding this comment.
Great work!
Just some questions
| fn pages_null(&self) -> Vec<u32> { | ||
| self.null_pages.clone() | ||
| fn pages_null(&self) -> Vec<Matches> { | ||
| // TODO: We could keep track of all-null pages and return Matches::All for those. |
There was a problem hiding this comment.
why we can't do this for now?
There was a problem hiding this comment.
We can, I just didn't want to 😆. Let me add that.
| impl FlatIndex { | ||
| pub fn try_new(data: RecordBatch) -> Result<Self> { | ||
| // Sort by row id to make bitmap construction more efficient | ||
| let data = data.sort_by_column(1, None)?; |
There was a problem hiding this comment.
isn't this already sorted?
There was a problem hiding this comment.
It's sorted by value. This sorts it by row id.
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
westonpace
force-pushed
the
perf/improve-btree-perf
branch
from
70d409e to
ea59263
Compare
westonpace
force-pushed
the
perf/improve-btree-perf
branch
from
ea59263 to
8b4c11e
Compare
The main improvements are: * Sort page by row ids when loading * Don't search page if the entire page matches, instead use a precomputed "all_ids" bitmap
2 participants
The main improvements are: