Skip to content

Conversation

@github-actions
Copy link
Contributor

Cherry-picked from #47295

#47295)

after pr #46365, commit mow table
may cost 1800s at most (rpc timeout*retry times), so the delete bitmap
cache expiration time should be set to 1800s.This cache will be
automatically deleted after the loading task is completed. Only when
task fails, this cache will be clean in background.
@github-actions github-actions bot requested a review from dataroaring as a code owner January 29, 2025 23:13
@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Jan 29, 2025
@hello-stephen
Copy link
Contributor

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40968 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3b246acb5f8e467bf61bd49c2b4b2fef48c8c0e0, data reload: false

------ Round 1 ----------------------------------
q1	17563	7810	7281	7281
q2	2056	172	166	166
q3	10593	1066	1162	1066
q4	10571	751	747	747
q5	7749	2847	2761	2761
q6	233	149	147	147
q7	983	625	619	619
q8	9353	1895	2020	1895
q9	6466	6399	6362	6362
q10	6991	2314	2331	2314
q11	460	269	264	264
q12	398	208	204	204
q13	17796	3006	3050	3006
q14	246	218	220	218
q15	581	525	518	518
q16	679	596	600	596
q17	987	642	571	571
q18	7387	6867	6705	6705
q19	1381	1021	1125	1021
q20	474	212	199	199
q21	4068	3294	3360	3294
q22	1140	1015	1014	1014
Total cold run time: 108155 ms
Total hot run time: 40968 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7243	7205	7210	7205
q2	326	236	233	233
q3	2914	2897	2972	2897
q4	2078	1828	1826	1826
q5	5736	5770	5699	5699
q6	227	147	148	147
q7	2243	1835	1823	1823
q8	3330	3586	3487	3487
q9	8845	8897	8958	8897
q10	3633	3613	3574	3574
q11	601	536	508	508
q12	787	604	610	604
q13	9408	3202	3159	3159
q14	317	284	294	284
q15	577	525	513	513
q16	678	672	663	663
q17	1830	1622	1604	1604
q18	8307	7739	7733	7733
q19	1645	1564	1611	1564
q20	2130	1844	1870	1844
q21	5518	5323	5447	5323
q22	1172	1041	1022	1022
Total cold run time: 69545 ms
Total hot run time: 60609 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197788 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3b246acb5f8e467bf61bd49c2b4b2fef48c8c0e0, data reload: false

query1	1361	919	927	919
query2	6226	2072	2074	2072
query3	10932	4394	4635	4394
query4	66582	28533	23467	23467
query5	5063	447	441	441
query6	414	176	169	169
query7	5659	301	310	301
query8	297	208	207	207
query9	9343	2678	2699	2678
query10	455	260	260	260
query11	18105	15313	16122	15313
query12	159	104	100	100
query13	1566	458	416	416
query14	10506	7044	7319	7044
query15	212	173	181	173
query16	7063	519	503	503
query17	1083	578	560	560
query18	2012	310	319	310
query19	204	157	162	157
query20	116	103	106	103
query21	206	102	99	99
query22	4617	4495	4639	4495
query23	34329	34056	33976	33976
query24	6387	2963	2946	2946
query25	535	428	410	410
query26	658	167	170	167
query27	1927	362	353	353
query28	4352	2514	2468	2468
query29	718	466	453	453
query30	248	167	168	167
query31	1012	796	842	796
query32	66	61	58	58
query33	410	287	303	287
query34	904	499	528	499
query35	860	756	747	747
query36	1098	978	945	945
query37	122	79	74	74
query38	4161	4056	4113	4056
query39	1523	1469	1490	1469
query40	203	103	104	103
query41	51	53	50	50
query42	119	110	109	109
query43	546	494	497	494
query44	1203	843	835	835
query45	186	173	173	173
query46	1162	739	735	735
query47	2020	1899	1943	1899
query48	506	387	400	387
query49	758	422	409	409
query50	832	428	414	414
query51	7350	7123	7266	7123
query52	103	99	89	89
query53	251	186	181	181
query54	545	449	445	445
query55	75	78	75	75
query56	259	251	243	243
query57	1249	1093	1113	1093
query58	213	204	208	204
query59	3030	2937	3042	2937
query60	280	256	269	256
query61	114	105	109	105
query62	819	746	724	724
query63	220	186	187	186
query64	1351	677	630	630
query65	3227	3166	3177	3166
query66	695	315	297	297
query67	15859	15638	15658	15638
query68	3960	570	568	568
query69	430	265	270	265
query70	1146	1137	1126	1126
query71	368	259	254	254
query72	6403	4104	4002	4002
query73	752	346	349	346
query74	10627	8997	9074	8997
query75	3338	2644	2641	2641
query76	1821	1043	1109	1043
query77	494	269	271	269
query78	10420	9570	9658	9570
query79	2148	604	606	604
query80	1405	426	432	426
query81	534	244	242	242
query82	1272	122	115	115
query83	163	149	140	140
query84	279	86	78	78
query85	1028	307	295	295
query86	418	308	303	303
query87	4424	4376	4236	4236
query88	3725	2394	2358	2358
query89	419	287	293	287
query90	1960	187	182	182
query91	180	151	152	151
query92	69	50	52	50
query93	2935	554	545	545
query94	870	293	294	293
query95	370	252	249	249
query96	614	276	277	276
query97	3382	3195	3185	3185
query98	236	221	216	216
query99	1701	1426	1424	1424
Total cold run time: 322608 ms
Total hot run time: 197788 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.18 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3b246acb5f8e467bf61bd49c2b4b2fef48c8c0e0, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.23	0.07	0.06
query4	1.63	0.10	0.11
query5	0.52	0.52	0.52
query6	1.13	0.72	0.73
query7	0.02	0.02	0.01
query8	0.05	0.03	0.03
query9	0.56	0.51	0.50
query10	0.56	0.54	0.57
query11	0.14	0.10	0.10
query12	0.15	0.12	0.11
query13	0.62	0.62	0.60
query14	2.84	2.78	2.78
query15	0.91	0.83	0.81
query16	0.37	0.37	0.36
query17	1.08	1.04	1.06
query18	0.25	0.22	0.22
query19	1.98	1.91	2.08
query20	0.01	0.02	0.01
query21	15.36	0.60	0.57
query22	2.42	1.72	1.74
query23	17.06	1.04	0.88
query24	3.32	0.84	0.28
query25	0.37	0.24	0.04
query26	0.26	0.12	0.14
query27	0.05	0.04	0.04
query28	11.04	1.10	1.08
query29	12.62	3.30	3.28
query30	0.25	0.06	0.07
query31	2.87	0.39	0.38
query32	3.24	0.46	0.46
query33	3.01	3.00	3.06
query34	17.06	4.50	4.49
query35	4.62	4.52	4.54
query36	0.65	0.48	0.51
query37	0.10	0.06	0.07
query38	0.04	0.03	0.03
query39	0.04	0.03	0.02
query40	0.16	0.13	0.12
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.04
Total cold run time: 107.85 s
Total hot run time: 32.18 s

@dataroaring dataroaring merged commit a2827a2 into branch-3.0 Jan 30, 2025
21 of 22 checks passed
@github-actions github-actions bot deleted the auto-pick-47295-branch-3.0 branch January 30, 2025 02:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants