Skip to content

Conversation

@github-actions
Copy link
Contributor

Cherry-picked from #48329

The cold compaction did not set the compaction policy in the initial phase, causing a null pointer exception when logging later. This issue occurs infrequently; if the tablet has previously done any compaction or calculated a compaction score, the compaction policy will be set automatically, allowing cold compaction to run normally. However, if a newly created tablet undergoes cold compaction immediately, this problem will arise. This PR fixes the issue.
@Thearas
Copy link
Contributor

Thearas commented Feb 26, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Feb 26, 2025
@Thearas
Copy link
Contributor

Thearas commented Feb 26, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40061 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a0fbd4a542daa4ea9091fff9ac206288109d60a0, data reload: false

------ Round 1 ----------------------------------
q1	17578	6764	6542	6542
q2	2060	178	164	164
q3	10555	1076	1128	1076
q4	10498	709	858	709
q5	7767	2868	2855	2855
q6	217	135	130	130
q7	977	623	617	617
q8	9373	1948	2015	1948
q9	6594	6454	6403	6403
q10	7056	2268	2296	2268
q11	463	253	253	253
q12	396	215	211	211
q13	18127	2986	3073	2986
q14	248	210	217	210
q15	498	476	463	463
q16	663	590	574	574
q17	966	542	508	508
q18	7241	6730	6774	6730
q19	1410	1075	981	981
q20	459	208	194	194
q21	4097	3200	3176	3176
q22	1175	1063	1080	1063
Total cold run time: 108418 ms
Total hot run time: 40061 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6676	6733	6619	6619
q2	336	260	235	235
q3	3171	3027	3150	3027
q4	2139	1896	1857	1857
q5	6030	5972	5975	5972
q6	216	127	135	127
q7	2274	1843	1869	1843
q8	3421	3652	3580	3580
q9	9146	8725	8869	8725
q10	3573	3502	3497	3497
q11	575	492	512	492
q12	821	628	598	598
q13	16705	3107	3196	3107
q14	303	270	278	270
q15	505	474	470	470
q16	701	652	655	652
q17	1834	1629	1596	1596
q18	8269	7743	7549	7549
q19	1675	1559	1569	1559
q20	2155	1826	1807	1807
q21	5453	5402	5225	5225
q22	1126	1055	1022	1022
Total cold run time: 77104 ms
Total hot run time: 59829 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197311 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a0fbd4a542daa4ea9091fff9ac206288109d60a0, data reload: false

query1	1278	900	904	900
query2	6380	2130	2081	2081
query3	10845	4442	4347	4347
query4	60536	29441	23469	23469
query5	5167	451	458	451
query6	401	196	175	175
query7	5437	311	300	300
query8	302	219	214	214
query9	8368	2577	2573	2573
query10	472	285	255	255
query11	17538	14949	15549	14949
query12	165	105	104	104
query13	1470	453	428	428
query14	10421	7529	6758	6758
query15	199	178	172	172
query16	7143	493	462	462
query17	1104	548	574	548
query18	1789	322	304	304
query19	209	161	148	148
query20	112	108	108	108
query21	210	102	99	99
query22	4841	4584	4749	4584
query23	35495	34317	33875	33875
query24	6332	2960	2887	2887
query25	521	419	437	419
query26	655	171	174	171
query27	1740	369	364	364
query28	4073	2502	2437	2437
query29	685	469	463	463
query30	251	168	166	166
query31	969	865	800	800
query32	68	62	58	58
query33	434	303	292	292
query34	906	507	503	503
query35	863	730	730	730
query36	1095	976	995	976
query37	123	67	70	67
query38	4089	3998	4026	3998
query39	1553	1462	1496	1462
query40	201	108	106	106
query41	54	54	50	50
query42	117	102	96	96
query43	529	507	495	495
query44	1152	845	824	824
query45	186	174	177	174
query46	1153	736	727	727
query47	1977	1943	1958	1943
query48	497	401	389	389
query49	755	413	432	413
query50	839	441	417	417
query51	7464	7159	7263	7159
query52	105	93	102	93
query53	270	191	182	182
query54	582	460	464	460
query55	81	77	77	77
query56	267	260	256	256
query57	1311	1141	1171	1141
query58	221	212	213	212
query59	3359	2888	2942	2888
query60	277	245	257	245
query61	112	113	111	111
query62	763	657	664	657
query63	215	182	188	182
query64	1373	691	639	639
query65	3252	3194	3198	3194
query66	709	295	295	295
query67	16047	15766	15832	15766
query68	4356	567	568	567
query69	450	260	260	260
query70	1155	1118	1133	1118
query71	317	256	258	256
query72	6045	4156	4200	4156
query73	743	346	361	346
query74	10597	9245	9219	9219
query75	3310	2682	2644	2644
query76	2091	1115	1111	1111
query77	488	274	270	270
query78	10649	9659	9689	9659
query79	1447	591	589	589
query80	877	417	422	417
query81	529	237	234	234
query82	1290	86	81	81
query83	244	141	140	140
query84	291	78	77	77
query85	874	296	298	296
query86	330	305	282	282
query87	4436	4314	4333	4314
query88	3925	2386	2353	2353
query89	411	287	285	285
query90	2063	185	185	185
query91	180	148	149	148
query92	62	50	53	50
query93	1958	543	557	543
query94	783	293	297	293
query95	357	247	248	247
query96	608	280	274	274
query97	3317	3135	3145	3135
query98	216	208	202	202
query99	1541	1272	1298	1272
Total cold run time: 314047 ms
Total hot run time: 197311 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.14 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a0fbd4a542daa4ea9091fff9ac206288109d60a0, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.03
query3	0.23	0.06	0.06
query4	1.63	0.10	0.11
query5	0.52	0.50	0.52
query6	1.15	0.72	0.72
query7	0.02	0.02	0.02
query8	0.03	0.03	0.03
query9	0.56	0.51	0.52
query10	0.57	0.55	0.56
query11	0.13	0.10	0.10
query12	0.14	0.11	0.12
query13	0.61	0.60	0.59
query14	2.72	2.86	2.73
query15	0.89	0.82	0.81
query16	0.40	0.38	0.38
query17	1.00	1.00	1.09
query18	0.23	0.21	0.22
query19	1.96	1.77	2.04
query20	0.02	0.01	0.01
query21	15.37	0.58	0.60
query22	3.24	2.72	2.01
query23	16.92	0.85	0.74
query24	3.25	1.19	0.92
query25	0.29	0.08	0.05
query26	0.49	0.14	0.14
query27	0.04	0.05	0.05
query28	10.20	0.53	0.44
query29	12.61	3.25	3.28
query30	0.24	0.06	0.06
query31	2.86	0.38	0.38
query32	3.25	0.46	0.45
query33	2.98	3.03	3.07
query34	16.93	4.47	4.50
query35	4.50	4.54	4.52
query36	0.68	0.48	0.48
query37	0.10	0.06	0.07
query38	0.04	0.04	0.04
query39	0.03	0.02	0.03
query40	0.17	0.12	0.13
query41	0.08	0.02	0.03
query42	0.04	0.03	0.02
query43	0.03	0.04	0.03
Total cold run time: 107.26 s
Total hot run time: 32.14 s

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit c406aa9 into branch-3.0 Feb 27, 2025
22 of 24 checks passed
@github-actions github-actions bot deleted the auto-pick-48329-branch-3.0 branch February 27, 2025 11:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants