[enhancement](cloud) optimize block cache lock #41818
Conversation
1. async deletion when do stale rowsets reclycle 2. minimize lock critical size 3. add cache lock held & wait time info for debug Signed-off-by: freemandealer <freeman.zhang1992@gmail.com>
|
Thank you for your contribution to Apache Doris. Since 2024-03-18, the Document has been moved to doris-website. |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
|
|
||
| #pragma once | ||
|
|
||
| #include <bvar/bvar.h> |
There was a problem hiding this comment.
warning: 'bvar/bvar.h' file not found [clang-diagnostic-error]
#include <bvar/bvar.h>
^
be/src/io/cache/block_file_cache.h
Outdated
|
|
||
| class LockScopedTimer { | ||
| public: | ||
| LockScopedTimer() : start_(std::chrono::high_resolution_clock::now()) {} |
There was a problem hiding this comment.
warning: use '= default' to define a trivial default constructor [modernize-use-equals-default]
| LockScopedTimer() : start_(std::chrono::high_resolution_clock::now()) {} | |
| LockScopedTimer() : start_(std::chrono::high_resolution_clock::now()) = default; |
|
run buildall |
1 similar comment
|
run buildall |
freemandealer
force-pushed
the
opt-cache-lock
branch
from
9e44e1e to
1981d93
Compare
freemandealer
force-pushed
the
opt-cache-lock
branch
from
1981d93 to
384b29d
Compare
|
run buildall |
Signed-off-by: freemandealer <freeman.zhang1992@gmail.com>
|
|
||
| class LockScopedTimer { | ||
| public: | ||
| LockScopedTimer() : start_(std::chrono::steady_clock::now()) {} |
There was a problem hiding this comment.
warning: use '= default' to define a trivial default constructor [modernize-use-equals-default]
| LockScopedTimer() : start_(std::chrono::steady_clock::now()) {} | |
| LockScopedTimer() : start_(std::chrono::steady_clock::now()) = default; |
|
run buildall |
|
TeamCity be ut coverage result: |
| std::chrono::duration_cast<std::chrono::milliseconds>(acq_time - start_time).count(); \ | ||
| if (duration > config::cache_lock_long_tail_threshold) \ | ||
| LOG(WARNING) << "Lock wait time " << std::to_string(duration) << "ms. " \ | ||
| << get_stack_trace_by_boost() << std::endl; \ |
There was a problem hiding this comment.
We should limit frequency of the log.
There was a problem hiding this comment.
@dataroaring It is limited already: only > config::cache_lock_long_tail_threshold (default:1s) will print the log. If not enough, enlarge the threshold.
|
run cloud_p0 |
|
run buildall |
|
TeamCity be ut coverage result: |
| while (_recycle_keys->pop(key)) { | ||
| Status st = _storage->remove(key); | ||
| if (!st.ok()) { | ||
| LOG_WARNING("").error(st); |
There was a problem hiding this comment.
put some key word in the log e.g.
"failed to remove when recycle_stale_rowset_async_bottom_half, key = {}" ...
| LOG_WARNING("Failed to push recycle key to queue, do it synchronously"); | ||
| Status st = _storage->remove(key); | ||
| if (!st.ok()) { | ||
| LOG_WARNING("").error(st); |
There was a problem hiding this comment.
put some key word in the log e.g.
"failed to remove async, key = {}" ...
github-actions
bot
added
the
approved
|
PR approved by at least one committer and no changes requested. |
|
PR approved by anyone and no changes requested. |
1. async deletion when do stale rowsets reclycle 2. minimize lock critical size 3. add cache lock held & wait time info for debug
pick apache#41818 from master 1. async deletion when do stale rowsets reclycle 2. minimize lock critical size 3. add cache lock held & wait time info for debug
…#43401) pick apache#41818 from master 1. async deletion when do stale rowsets reclycle 2. minimize lock critical size 3. add cache lock held & wait time info for debug
5 participants
Uh oh!
There was an error while loading. Please reload this page.