Skip to content

Conversation

@github-actions
Copy link
Contributor

Cherry-picked from #51649

…lection strategy as cumulative compaction (#51649)

1. Make base compaction use the same tablet selection strategy as
cumulative compaction.
2. Modify some parameters to make them clearer
@github-actions github-actions bot requested a review from morrySnow as a code owner June 26, 2025 14:40
@morrySnow morrySnow closed this Jun 27, 2025
@morrySnow morrySnow reopened this Jun 27, 2025
@morrySnow
Copy link
Contributor

run buildall

1 similar comment
@morrySnow
Copy link
Contributor

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40506 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e5ab90d0a062200fca0be644d04f1983f8b47e16, data reload: false

------ Round 1 ----------------------------------
q1	17936	7615	6700	6700
q2	2068	169	164	164
q3	10671	1170	1203	1170
q4	10559	704	793	704
q5	7739	2931	2911	2911
q6	215	135	136	135
q7	1001	642	623	623
q8	9402	1987	2043	1987
q9	6679	6481	6460	6460
q10	7030	2264	2312	2264
q11	469	273	274	273
q12	403	222	219	219
q13	17795	3008	3026	3008
q14	245	222	219	219
q15	508	469	479	469
q16	465	398	389	389
q17	1008	621	631	621
q18	7467	6910	6829	6829
q19	1329	976	1064	976
q20	509	210	215	210
q21	3999	3206	3181	3181
q22	1113	994	1034	994
Total cold run time: 108610 ms
Total hot run time: 40506 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6652	6622	6660	6622
q2	333	234	239	234
q3	3152	2940	3009	2940
q4	2137	1885	1867	1867
q5	5782	5830	5785	5785
q6	209	130	129	129
q7	2264	1857	1825	1825
q8	3448	3474	3574	3474
q9	9014	8992	8997	8992
q10	3644	3574	3554	3554
q11	605	504	505	504
q12	841	633	627	627
q13	6636	3193	3198	3193
q14	306	277	272	272
q15	521	469	466	466
q16	508	450	472	450
q17	1871	1667	1632	1632
q18	8389	7956	7729	7729
q19	1694	1608	1570	1570
q20	2123	1918	1907	1907
q21	5124	4971	5133	4971
q22	1133	1071	1079	1071
Total cold run time: 66386 ms
Total hot run time: 59814 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 199563 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e5ab90d0a062200fca0be644d04f1983f8b47e16, data reload: false

query1	1332	932	919	919
query2	6255	1944	1926	1926
query3	10979	4522	4697	4522
query4	33048	23498	23581	23498
query5	3412	480	464	464
query6	280	201	192	192
query7	4006	337	339	337
query8	303	236	255	236
query9	9543	2732	2733	2732
query10	503	275	265	265
query11	18274	15267	15282	15267
query12	158	108	110	108
query13	1586	466	449	449
query14	8673	7805	6823	6823
query15	274	198	193	193
query16	8091	538	562	538
query17	1645	608	639	608
query18	2204	347	331	331
query19	391	170	195	170
query20	132	122	122	122
query21	210	112	120	112
query22	4859	4527	4467	4467
query23	35640	34287	34508	34287
query24	11710	2994	2980	2980
query25	714	451	430	430
query26	1769	186	181	181
query27	2778	367	369	367
query28	7806	2320	2313	2313
query29	1067	483	493	483
query30	271	164	163	163
query31	1084	842	841	841
query32	110	60	56	56
query33	791	296	312	296
query34	984	524	566	524
query35	885	733	735	733
query36	1137	969	987	969
query37	263	75	73	73
query38	4080	3956	4090	3956
query39	1515	1668	1467	1467
query40	260	104	107	104
query41	49	49	50	49
query42	126	116	107	107
query43	534	509	506	506
query44	1260	852	857	852
query45	185	173	175	173
query46	1196	749	762	749
query47	2022	1959	2012	1959
query48	464	356	358	356
query49	1065	403	426	403
query50	858	446	437	437
query51	7576	7323	7456	7323
query52	109	101	91	91
query53	272	187	184	184
query54	1318	494	490	490
query55	84	84	83	83
query56	277	254	266	254
query57	1360	1218	1236	1218
query58	248	223	220	220
query59	3237	3162	3093	3093
query60	300	281	277	277
query61	117	117	115	115
query62	857	690	695	690
query63	231	192	196	192
query64	5343	686	664	664
query65	3347	3203	3211	3203
query66	1329	295	309	295
query67	16218	15758	15783	15758
query68	5584	647	610	610
query69	457	269	273	269
query70	1207	1149	1165	1149
query71	347	264	266	264
query72	6162	4042	4008	4008
query73	776	371	392	371
query74	10774	9411	9332	9332
query75	3412	2723	2672	2672
query76	3255	1040	1160	1040
query77	380	287	284	284
query78	10608	9873	9634	9634
query79	2375	624	652	624
query80	1118	446	440	440
query81	545	235	230	230
query82	659	91	92	91
query83	236	151	145	145
query84	238	78	86	78
query85	1452	300	304	300
query86	462	306	300	300
query87	4375	4244	4237	4237
query88	4134	2519	2502	2502
query89	418	299	309	299
query90	2005	185	187	185
query91	139	115	112	112
query92	73	52	51	51
query93	2036	587	585	585
query94	892	314	292	292
query95	359	267	269	267
query96	631	287	290	287
query97	3313	3187	3197	3187
query98	232	206	210	206
query99	1520	1302	1269	1269
Total cold run time: 309178 ms
Total hot run time: 199563 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.73 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e5ab90d0a062200fca0be644d04f1983f8b47e16, data reload: false

query1	0.03	0.04	0.04
query2	0.06	0.04	0.04
query3	0.23	0.07	0.07
query4	1.61	0.11	0.10
query5	0.51	0.53	0.52
query6	1.12	0.74	0.73
query7	0.02	0.02	0.01
query8	0.04	0.04	0.04
query9	0.56	0.52	0.54
query10	0.57	0.56	0.58
query11	0.14	0.10	0.11
query12	0.15	0.12	0.11
query13	0.61	0.61	0.60
query14	0.78	0.81	0.79
query15	0.85	0.82	0.85
query16	0.38	0.38	0.39
query17	1.02	1.11	1.06
query18	0.24	0.23	0.20
query19	1.96	1.82	1.78
query20	0.01	0.01	0.01
query21	15.40	0.61	0.57
query22	2.55	2.46	1.70
query23	16.74	0.95	0.87
query24	3.25	1.26	1.92
query25	0.22	0.10	0.05
query26	0.72	0.14	0.13
query27	0.04	0.05	0.04
query28	9.10	0.55	0.52
query29	12.58	3.31	3.27
query30	0.25	0.07	0.07
query31	2.84	0.39	0.39
query32	3.23	0.46	0.45
query33	3.01	3.08	3.07
query34	17.27	4.55	4.51
query35	4.58	4.55	4.58
query36	0.67	0.49	0.49
query37	0.09	0.06	0.06
query38	0.05	0.03	0.03
query39	0.04	0.02	0.02
query40	0.18	0.13	0.12
query41	0.08	0.03	0.03
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 103.85 s
Total hot run time: 30.73 s

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 100.00% (12/12) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 44.98% (12350/27459)
Line Coverage 36.01% (109403/303840)
Region Coverage 35.06% (56683/161676)
Branch Coverage 32.19% (30758/95540)

@morrySnow morrySnow merged commit a58635e into branch-3.1 Jul 1, 2025
21 of 23 checks passed
@github-actions github-actions bot deleted the auto-pick-51649-branch-3.1 branch July 1, 2025 11:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants