-
Notifications
You must be signed in to change notification settings - Fork 3.7k
[fix](cloud) retry read_at when corruption using file cache #48786
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: zhengyu <zhangzhengyu@selectdb.com>
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
| PageFooterPB footer; | ||
| RETURN_IF_ERROR( | ||
| PageIO::read_and_decompress_page(opts, &_sk_index_handle, &body, &footer)); | ||
| RETURN_IF_ERROR(PageIO::read_and_decompress_page_with_file_cache_retry( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it's better that we expand read_and_decompress_page_with_file_cache_retry() in segment.cpp
instead of hacking page_io.h/cpp.
the low level IO should not notice "file cache".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这个可能做不到,因为 read_and_decompress 用到的地方太多了,除非要多增加一层抽象。另外,Page IO 本身是 read data from file,引入 file cache 是比较自然的。相反,向更高层次的 segment 暴露反而不好,但是 segment 有一些 open file 的操作,需要做一些 check 所以也没法避免
|
run buildall |
TPC-H: Total hot run time: 32610 ms |
TPC-DS: Total hot run time: 186876 ms |
ClickBench: Total hot run time: 30.95 s |
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
|
run cloud_p0 |
Signed-off-by: zhengyu <zhangzhengyu@selectdb.com>
|
run buildall |
TPC-H: Total hot run time: 32651 ms |
TPC-DS: Total hot run time: 186342 ms |
ClickBench: Total hot run time: 31.21 s |
|
run beut |
Signed-off-by: zhengyu <zhangzhengyu@selectdb.com>
|
run buildall |
|
run buildall |
TPC-H: Total hot run time: 34297 ms |
TPC-DS: Total hot run time: 193252 ms |
ClickBench: Total hot run time: 31.7 s |
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
|
PR approved by at least one committer and no changes requested. |
|
PR approved by anyone and no changes requested. |
|
run cloud_p0 |
What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)