testing: pageworkload HeavySkewWriteRead support deleting data

[JaySon-Huang]

What problem does this PR solve?

Issue Number: ref #6163

Problem Summary:
We do not delete the page and update existing pages in the previous pageworkload HeavySkewWriteRead. It always appends new pages. So it can not simulate the real workload of delta-tree and cost lots of disk space.

We want to use this workload for verifying the bugs reported in #6163

What is changed and how it works?

• Keep track of page id committed to PageStorage
• Update pages for about
• Remove the pages if that is unlikely to be updated or read anymore
• Previously, PSWriter::updatedRandomData generate random data for updating and reading with a bytes length range of [0, buffer_size_max]. Now it is replaced by PSWriter::getRandomData and the bytes length range is limited in [buffer_size_min, buffer_size_max]. The random data has a smaller range and more stable reading/writing speed.

The read/write performance:

W: 633028 pages, 618.1914 GB, 0.6182 GB/s
R: 31338450 pages, 30603.9551 GB, 30.6035 GB/s

W: 641408 pages, 626.3750 GB, 0.6264 GB/s
R: 30543630 pages, 29827.7637 GB, 29.8274 GB/s

W: 648852 pages, 633.6445 GB, 0.6336 GB/s
R: 29991155 pages, 29288.2373 GB, 29.2879 GB/s

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No code

I run the workload for 20 minutes. After the workload ends, the remaining page num is 757 rather than a large number of pages. And the disk space used for this testing is more reasonable.

> ./dbms/src/Server/tiflash pageworkload -M 16 --clean_before_run 1 --gc_interval 5 --avg_page_size 4096 --timeout 1200 &> workload.log
> cat workload.log
....
[2022/10/25 02:30:32.151 +08:00] [INFO] [PageDirectory.cpp:1461] ["After MVCC gc in memory [lowest_seq=4164205] clean [invalid_snapshot_nums=0] [invalid_page_nums=16336] [total_deref_counter=0] [all_del_entries=101659]. Still exist [snapshot_nums=0], [page_nums=757]. Longest alive snapshot: [longest_alive_snapshot_time=0] [longest_alive_snapshot_seq=0] [stale_snapshot_nums=0]"] [source=stress_test] [thread_id=23]
>  du -sh stress/HeavySkewWriteRead/*
145M    stress/HeavySkewWriteRead/blobfile_65
13M     stress/HeavySkewWriteRead/blobfile_66
3.0M    stress/HeavySkewWriteRead/blobfile_67
93M     stress/HeavySkewWriteRead/wal
>  du -sh stress/HeavySkewWriteRead/*/*
88M     stress/HeavySkewWriteRead/wal/log_280_1
2.1M    stress/HeavySkewWriteRead/wal/log_281_0
2.1M    stress/HeavySkewWriteRead/wal/log_282_0
944K    stress/HeavySkewWriteRead/wal/log_283_0

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

None

[ti-chi-bot]

[REVIEW NOTIFICATION]

This pull request has been approved by:

• hehechen
• lidezhu

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Details

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

[JaySon-Huang]

/run-all-tests

[flowbehappy]

Is it possible to detect the bug automatically? Like by checking the wal file size.

[JaySon-Huang]

Is it possible to detect the bug automatically? Like by checking the wal file size.

I add some metrics about the WAL log file size in this PR #6175. We can add some checks after that.

[JaySon-Huang]

/run-all-tests

[JaySon-Huang]

/merge

[ti-chi-bot]

@JaySon-Huang: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

[ti-chi-bot]

This pull request has been accepted and is ready to merge.

Details

Commit hash: 3a709cb

[JaySon-Huang]

/merge

[ti-chi-bot]

@JaySon-Huang: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

[ti-chi-bot]

In response to a cherrypick label: new pull request created: #6286.