[Optimize][Cache]Implementation of Separated Page Cache #5008

Skysheepwang · 2020-12-02T10:26:39Z

Proposed changes

#4995
Implementation of Separated Page Cache

Add config "index_page_cache_ratio" to set the ratio of capacity of index page cache
Change the member of StoragePageCache to maintain two type of cache
Change the interface of StoragePageCache for selecting type of cache
Change the usage of page cache in read_and_decompress_page in page_io.cpp
- add page type as argument
- check if current page type is available in StoragePageCache (cover the situation of ratio == 0 or 1)
Add type as argument in superior call of read_and_decompress_page
Change Unit Test

Types of changes

What types of changes does your code introduce to Doris?
Put an x in the boxes that apply

[] Bugfix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
[] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[] Documentation Update (if none of the other choices apply)
Code refactor (Modify the code structure, format the code, etc...)

Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

I have create an issue on (Fix [Proposal] Separate Page Cache by page type #4995), and have described the bug/feature there in detail
Compiling and unit tests pass locally with my changes
I have added tests that prove my fix is effective or that my feature works
[] If this change need a document change, I have updated the document
Any dependent changes have been merged

be/src/common/config.h

be/src/olap/page_cache.h

be/src/olap/page_cache.cpp

docs/en/administrator-guide/config/be_config.md

docs/zh-CN/administrator-guide/config/be_config.md

be/src/olap/page_cache.cpp

acelyc111

LGTM

be/src/olap/page_cache.cpp

be/src/olap/rowset/segment_v2/page_io.cpp

morningman

LGTM

be/src/olap/page_cache.cpp

morningman · 2020-12-16T09:33:11Z

hi @Skysheepwang Could you please rebase the trunk to make UT the NewDorisTest happy?

Skysheepwang · 2020-12-18T09:43:21Z

@morningman I 've already rebased but there is still something wrong in UT (Incubator Doris), but my local UT was passed so could you please help me check the problem?

morningman · 2020-12-18T13:34:54Z

@morningman I 've already rebased but there is still something wrong in UT (Incubator Doris), but my local UT was passed so could you please help me check the problem?

It seems our proxy of github has some problem. I checked the UT, it works.

morningman

LGTM

acelyc111

LGTM

Skysheepwang · 2020-12-22T03:42:12Z

@morningman @acelyc111 Sorry I just made a mistake to push my code to this branch, which dismissed your approving review, please add again.

acelyc111

LGTM

morningman

LGTM

morningman · 2020-12-24T01:41:04Z

Hi @Skysheepwang, Sorry for late review. Please resolve the conflict~

acelyc111 · 2021-01-04T03:26:40Z

Hi @Skysheepwang, Sorry for late review. Please resolve the conflict~

@morningman conflicts have been resolved.

morningman

LGTM

apache#4995 **Implementation of Separated Page Cache** - Add config "index_page_cache_ratio" to set the ratio of capacity of index page cache - Change the member of StoragePageCache to maintain two type of cache - Change the interface of StoragePageCache for selecting type of cache - Change the usage of page cache in read_and_decompress_page in page_io.cpp - add page type as argument - check if current page type is available in StoragePageCache (cover the situation of ratio == 0 or 1) - Add type as argument in superior call of read_and_decompress_page - Change Unit Test

…or selectdb and velodb (apache#5008)

acelyc111 reviewed Dec 8, 2020

View reviewed changes

Skysheepwang force-pushed the master branch from 5e2cfa8 to 0086e4e Compare December 9, 2020 07:45

acelyc111 reviewed Dec 9, 2020

View reviewed changes

be/src/olap/page_cache.cpp Outdated Show resolved Hide resolved

be/src/olap/page_cache.cpp Outdated Show resolved Hide resolved

be/src/olap/page_cache.cpp Outdated Show resolved Hide resolved

acelyc111 reviewed Dec 10, 2020

View reviewed changes

be/src/olap/page_cache.cpp Outdated Show resolved Hide resolved

acelyc111 approved these changes Dec 10, 2020

View reviewed changes

acelyc111 previously approved these changes Dec 10, 2020

View reviewed changes

morningman added the kind/improvement label Dec 11, 2020

morningman reviewed Dec 14, 2020

View reviewed changes

be/src/olap/page_cache.cpp Show resolved Hide resolved

be/src/olap/page_cache.cpp Show resolved Hide resolved

be/src/olap/rowset/segment_v2/page_io.cpp Show resolved Hide resolved

morningman previously approved these changes Dec 16, 2020

View reviewed changes

be/src/olap/page_cache.cpp Show resolved Hide resolved

Skysheepwang dismissed stale reviews from morningman and acelyc111 via adb95fc December 16, 2020 09:39

Skysheepwang force-pushed the master branch from e0248a3 to adb95fc Compare December 16, 2020 09:39

Implementation of Separated Page Cache

2149cfd

Skysheepwang force-pushed the master branch from adb95fc to 2149cfd Compare December 16, 2020 09:53

morningman previously approved these changes Dec 18, 2020

View reviewed changes

morningman added the approved Indicates a PR has been approved by one committer. label Dec 18, 2020

acelyc111 previously approved these changes Dec 22, 2020

View reviewed changes

Skysheepwang dismissed stale reviews from acelyc111 and morningman via d7c2aa5 December 22, 2020 03:27

Skysheepwang force-pushed the master branch from d7c2aa5 to 2149cfd Compare December 22, 2020 03:37

acelyc111 previously approved these changes Dec 22, 2020

View reviewed changes

morningman previously approved these changes Dec 24, 2020

View reviewed changes

Merge branch 'master' into master

140a8e5

Skysheepwang dismissed stale reviews from morningman and acelyc111 via 140a8e5 December 24, 2020 02:07

morningman approved these changes Jan 4, 2021

View reviewed changes

morningman merged commit 6c098e4 into apache:master Jan 4, 2021

yangzhg mentioned this pull request Feb 9, 2021

Release Notes 0.14.0 #5374

Closed

Hastyshell pushed a commit to Hastyshell/doris that referenced this pull request Oct 23, 2025

[opt](jdbc catalog)Add JdbcMySQLClient to recognize version_comment f…

dad4f2a

…or selectdb and velodb (apache#5008)

[Optimize][Cache]Implementation of Separated Page Cache #5008

[Optimize][Cache]Implementation of Separated Page Cache #5008

Uh oh!

Conversation

Skysheepwang commented Dec 2, 2020

Proposed changes

Types of changes

Checklist

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

acelyc111 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

morningman left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

morningman commented Dec 16, 2020

Uh oh!

Skysheepwang commented Dec 18, 2020

Uh oh!

morningman commented Dec 18, 2020

Uh oh!

morningman left a comment

Choose a reason for hiding this comment

Uh oh!

acelyc111 left a comment

Choose a reason for hiding this comment

Uh oh!

Skysheepwang commented Dec 22, 2020

Uh oh!

acelyc111 left a comment

Choose a reason for hiding this comment

Uh oh!

morningman left a comment

Choose a reason for hiding this comment

Uh oh!

morningman commented Dec 24, 2020

Uh oh!

acelyc111 commented Jan 4, 2021

Uh oh!

morningman left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants