[Enhancement] support vertical compaction

[yixiutt]

Search before asking

• I had searched in the issues and found no similar issues.

Description

Compaction in some case such as wide table will cost a lot memory, every row will be loaded in compaction, so memory cost will be rowset_num * segment_num * column_num * page_size when every column load first page, default page size is 64K, so if a table has 1000 columns it will cost rowset_num * segment_num * 64M。

Vertical compaction can handle compaction in column groups so not cost to much memory.

I'll accomplish this work in few weeks.

Solution

Vertical Compaction, steps to do:

1. framework, support duplicate key vertical compaction, including all basic functions and can run correctly.
2. unique key
3. agg key
4. some optimize such as RowSourceBuffer write to tmp file to support large compaction.

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct