-
Notifications
You must be signed in to change notification settings - Fork 3.7k
Description
Is your feature request related to a problem? Please describe.
I have developed a Benchmark tool that is used to test some read and write performance tests related to the storage layer.
Describe the solution you'd like
It is currently testing segment and partial Page.
Dataset can be randomly generated or imported by data files.
It also supports testing functions written by users, used to get some effects of some third-party libraries at doris.
Describe alternatives you've considered
It will be compiled when the user executes ./run-be-ut.sh and placed in ut_build_ASAN/test/tools/benchmark_tool.
But it doesn't be executed when Run All Ut (- Run) .
Additional context
Here are some usage examples:
./benchmark_tool --operation=Custom
Execute the benchmark of the custum function (need to implement the init() and run() functions)
./benchmark_tool --operation=BinaryDictPageEncode --rows_number=10000 --iterations=40
Execute the benchmark of BinaryDictPage encode 1000 randomly generated data,Iterate forty times to calculate the average time consumption.
./benchmark_tool --operation=SegmentScan --column_type=int,varchar --rows_number=10000 --iterations=0
Execute the benchmark of Segment scan data from file(data is randomly generated), iterations is set to 0 means the number of iterations is automatically set.
./benchmark_tool --operation=SegmentWrite --column_type=int --rows_number=10000 --iterations=10
Execute the benchmark of Segment write data to file(data is randomly generated).
./benchmark_tool --operation=SegmentScanByFile --input_file=./sample.dat --iterations=10
Execute the benchmark of Segment scan data from file(data from input files).
Data file format:
The first line defines Shcema
The rest of the content is DataSet
For example:
int,char,varchar
123,hello,world
321,good,bye
./benchmark_tool --operation=SegmentWriteByFile --input_file=./sample.dat --iterations=10
Execute the benchmark of Segment write data from file(data from input files).