Skip to content

Conversation

@zddr
Copy link
Contributor

@zddr zddr commented May 29, 2024

Proposed changes

Issue Number: close #xxx

This is a temporary solution. In order to avoid affecting the existing backup function, the backup MTMV will be allowed after detailed design

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zddr
Copy link
Contributor Author

zddr commented May 29, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41202 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit caa732046f53c5939f53d18dd212c799c3568402, data reload: false

------ Round 1 ----------------------------------
q1	17595	4245	4295	4245
q2	2021	204	199	199
q3	10463	1285	1157	1157
q4	10200	815	867	815
q5	7484	2715	2718	2715
q6	218	131	136	131
q7	968	622	612	612
q8	9220	2167	2107	2107
q9	9195	6701	6748	6701
q10	9503	3912	3904	3904
q11	449	247	289	247
q12	455	223	228	223
q13	18168	3193	3287	3193
q14	269	214	234	214
q15	511	475	469	469
q16	514	421	397	397
q17	990	658	687	658
q18	8526	7810	7773	7773
q19	6416	1604	1511	1511
q20	671	320	314	314
q21	5230	4062	3335	3335
q22	353	285	282	282
Total cold run time: 119419 ms
Total hot run time: 41202 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4540	4407	4396	4396
q2	367	286	275	275
q3	3214	2928	2924	2924
q4	1916	1620	1605	1605
q5	5491	5513	5526	5513
q6	212	127	126	126
q7	2187	1850	1818	1818
q8	3220	3416	3393	3393
q9	8558	8672	8698	8672
q10	4052	3766	3790	3766
q11	585	487	500	487
q12	800	642	616	616
q13	17138	3163	3155	3155
q14	298	283	274	274
q15	516	478	486	478
q16	499	444	447	444
q17	1810	1492	1472	1472
q18	7719	7680	7339	7339
q19	1664	1571	1545	1545
q20	2006	1793	1788	1788
q21	10488	4680	4714	4680
q22	574	482	503	482
Total cold run time: 77854 ms
Total hot run time: 55248 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 169341 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit caa732046f53c5939f53d18dd212c799c3568402, data reload: false

query1	893	374	365	365
query2	6440	2470	2302	2302
query3	6642	201	205	201
query4	19356	17371	17342	17342
query5	4108	413	415	413
query6	250	156	152	152
query7	4590	309	300	300
query8	238	185	176	176
query9	8342	2470	2442	2442
query10	452	294	269	269
query11	10625	10086	10207	10086
query12	135	90	87	87
query13	1641	375	375	375
query14	9394	6779	7585	6779
query15	212	168	167	167
query16	7711	273	262	262
query17	1695	538	527	527
query18	1866	273	266	266
query19	207	155	156	155
query20	94	84	84	84
query21	196	133	127	127
query22	4519	4280	4008	4008
query23	33539	33004	33093	33004
query24	10577	2865	2832	2832
query25	590	348	362	348
query26	712	154	152	152
query27	2216	320	316	316
query28	5497	2072	2074	2072
query29	873	604	591	591
query30	251	153	153	153
query31	981	775	748	748
query32	91	51	51	51
query33	658	272	255	255
query34	870	475	466	466
query35	738	619	600	600
query36	1113	921	875	875
query37	100	67	65	65
query38	2893	2770	2780	2770
query39	836	796	801	796
query40	200	120	123	120
query41	45	42	41	41
query42	131	94	95	94
query43	583	588	535	535
query44	1082	737	736	736
query45	177	165	163	163
query46	1063	719	743	719
query47	1866	1776	1853	1776
query48	362	303	296	296
query49	831	383	382	382
query50	769	397	387	387
query51	6743	6640	6741	6640
query52	108	93	89	89
query53	353	281	283	281
query54	871	437	429	429
query55	71	70	71	70
query56	270	245	247	245
query57	1115	1050	1035	1035
query58	227	222	211	211
query59	3354	3175	3196	3175
query60	272	250	254	250
query61	88	88	85	85
query62	617	456	454	454
query63	306	277	283	277
query64	8504	2253	1705	1705
query65	3179	3102	3135	3102
query66	794	331	319	319
query67	15141	14840	15014	14840
query68	4541	550	558	550
query69	446	272	265	265
query70	1079	1157	1083	1083
query71	390	268	268	268
query72	7570	2701	2516	2516
query73	712	327	326	326
query74	6061	5634	5623	5623
query75	3300	2611	2657	2611
query76	2625	1012	983	983
query77	395	261	262	261
query78	10165	9974	9806	9806
query79	2410	515	515	515
query80	1130	444	423	423
query81	537	218	219	218
query82	952	91	92	91
query83	249	172	168	168
query84	251	85	87	85
query85	1236	259	255	255
query86	512	327	291	291
query87	3291	3063	3160	3063
query88	4014	2429	2415	2415
query89	477	390	381	381
query90	1945	182	190	182
query91	134	108	107	107
query92	59	50	49	49
query93	1865	529	516	516
query94	1191	195	255	195
query95	396	313	307	307
query96	595	275	265	265
query97	3168	3047	3022	3022
query98	261	222	219	219
query99	1164	875	847	847
Total cold run time: 264384 ms
Total hot run time: 169341 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.28 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit caa732046f53c5939f53d18dd212c799c3568402, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.24	0.05	0.05
query4	1.68	0.07	0.06
query5	0.54	0.48	0.50
query6	1.12	0.73	0.74
query7	0.02	0.02	0.02
query8	0.05	0.04	0.04
query9	0.54	0.49	0.51
query10	0.54	0.55	0.53
query11	0.17	0.11	0.12
query12	0.14	0.12	0.12
query13	0.59	0.58	0.59
query14	0.78	0.80	0.76
query15	0.82	0.80	0.80
query16	0.36	0.36	0.36
query17	0.98	0.94	0.95
query18	0.20	0.25	0.22
query19	1.75	1.70	1.80
query20	0.02	0.01	0.01
query21	15.76	0.65	0.63
query22	4.18	7.54	1.70
query23	18.30	1.39	1.32
query24	1.71	0.27	0.20
query25	0.14	0.09	0.08
query26	0.27	0.17	0.16
query27	0.08	0.08	0.08
query28	13.41	1.00	0.99
query29	13.22	3.34	3.29
query30	0.25	0.06	0.06
query31	2.86	0.38	0.38
query32	3.30	0.46	0.46
query33	2.90	2.92	2.90
query34	17.11	4.43	4.42
query35	4.55	4.48	4.53
query36	0.70	0.49	0.47
query37	0.18	0.14	0.15
query38	0.15	0.15	0.15
query39	0.04	0.04	0.04
query40	0.15	0.14	0.15
query41	0.09	0.04	0.05
query42	0.05	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 110.1 s
Total hot run time: 30.28 s

@zddr
Copy link
Contributor Author

zddr commented May 29, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41708 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c04f8f16338390389575c91cbd2f233583538393, data reload: false

------ Round 1 ----------------------------------
q1	17621	4307	4209	4209
q2	2028	190	200	190
q3	10463	1248	1231	1231
q4	10205	782	803	782
q5	7473	2704	2658	2658
q6	218	132	132	132
q7	975	608	608	608
q8	9221	2102	2095	2095
q9	9678	6654	6708	6654
q10	9062	3916	3886	3886
q11	475	247	250	247
q12	462	221	223	221
q13	17368	3292	3210	3210
q14	267	218	229	218
q15	507	477	479	477
q16	516	395	401	395
q17	979	615	737	615
q18	8302	7806	7753	7753
q19	8256	1568	1505	1505
q20	639	305	315	305
q21	5098	4124	4040	4040
q22	349	280	277	277
Total cold run time: 120162 ms
Total hot run time: 41708 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4476	4417	4393	4393
q2	374	265	267	265
q3	3127	2901	2868	2868
q4	2038	1658	1558	1558
q5	5318	5485	5426	5426
q6	210	123	124	123
q7	2204	1851	1805	1805
q8	3209	3383	3353	3353
q9	8474	8576	8657	8576
q10	4096	3741	3774	3741
q11	595	480	491	480
q12	810	615	648	615
q13	15975	3123	3173	3123
q14	291	272	281	272
q15	522	496	490	490
q16	492	429	419	419
q17	1790	1514	1533	1514
q18	7805	7720	7321	7321
q19	1648	1567	1606	1567
q20	1964	1783	1797	1783
q21	8168	4793	4766	4766
q22	559	492	492	492
Total cold run time: 74145 ms
Total hot run time: 54950 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 171203 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c04f8f16338390389575c91cbd2f233583538393, data reload: false

query1	918	377	370	370
query2	6412	2360	2412	2360
query3	6659	204	201	201
query4	19510	17387	17345	17345
query5	4141	418	414	414
query6	251	160	157	157
query7	4576	292	291	291
query8	241	185	177	177
query9	8440	2344	2365	2344
query10	454	294	263	263
query11	10414	9988	10101	9988
query12	133	86	86	86
query13	1635	389	368	368
query14	9753	6733	7514	6733
query15	214	167	175	167
query16	7719	263	272	263
query17	1389	520	511	511
query18	1946	270	273	270
query19	195	150	145	145
query20	90	88	83	83
query21	195	131	128	128
query22	4159	3965	3890	3890
query23	33576	33098	32997	32997
query24	7100	2776	2846	2776
query25	567	361	368	361
query26	714	153	159	153
query27	2070	317	321	317
query28	4089	2092	2085	2085
query29	845	595	601	595
query30	244	150	148	148
query31	948	765	734	734
query32	83	54	53	53
query33	486	265	265	265
query34	871	464	476	464
query35	695	591	590	590
query36	1035	902	893	893
query37	104	65	67	65
query38	2875	2810	2754	2754
query39	835	815	781	781
query40	206	130	128	128
query41	50	45	78	45
query42	98	97	91	91
query43	552	549	569	549
query44	1067	736	752	736
query45	177	155	154	154
query46	1041	690	727	690
query47	1826	1717	1752	1717
query48	378	303	316	303
query49	836	370	379	370
query50	766	384	379	379
query51	6944	6830	6698	6698
query52	95	89	85	85
query53	346	278	290	278
query54	525	429	426	426
query55	71	69	70	69
query56	252	237	243	237
query57	1117	1020	1058	1020
query58	249	208	205	205
query59	3346	3268	3066	3066
query60	265	251	249	249
query61	86	84	94	84
query62	607	464	438	438
query63	303	284	282	282
query64	8408	2229	1716	1716
query65	3188	3170	3123	3123
query66	792	318	332	318
query67	15179	14591	15036	14591
query68	4592	554	533	533
query69	487	265	269	265
query70	1098	1135	1147	1135
query71	391	260	271	260
query72	8317	5372	5575	5372
query73	736	335	320	320
query74	6118	5546	5600	5546
query75	3377	2598	2646	2598
query76	2930	1008	992	992
query77	570	266	261	261
query78	10137	9853	9643	9643
query79	2009	510	512	510
query80	1051	454	429	429
query81	517	225	218	218
query82	625	89	86	86
query83	233	169	169	169
query84	243	89	90	89
query85	1751	275	323	275
query86	457	325	296	296
query87	3285	3153	3127	3127
query88	4237	2443	2430	2430
query89	469	385	382	382
query90	2019	189	185	185
query91	120	97	93	93
query92	59	48	48	48
query93	2624	515	500	500
query94	1283	183	178	178
query95	405	303	306	303
query96	585	273	267	267
query97	3169	2998	3023	2998
query98	236	234	217	217
query99	1170	857	846	846
Total cold run time: 260651 ms
Total hot run time: 171203 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.67 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c04f8f16338390389575c91cbd2f233583538393, data reload: false

query1	0.04	0.03	0.03
query2	0.09	0.03	0.03
query3	0.22	0.06	0.05
query4	1.67	0.08	0.08
query5	0.50	0.48	0.49
query6	1.12	0.73	0.72
query7	0.01	0.01	0.01
query8	0.05	0.04	0.04
query9	0.54	0.49	0.50
query10	0.54	0.56	0.53
query11	0.15	0.10	0.11
query12	0.14	0.11	0.12
query13	0.59	0.59	0.60
query14	0.80	0.79	0.76
query15	0.82	0.80	0.82
query16	0.36	0.36	0.36
query17	1.00	1.01	0.99
query18	0.22	0.25	0.22
query19	1.75	1.70	1.73
query20	0.02	0.01	0.02
query21	15.52	0.69	0.65
query22	4.60	6.51	2.26
query23	18.22	1.33	1.16
query24	1.35	0.41	0.23
query25	0.15	0.08	0.08
query26	0.26	0.16	0.17
query27	0.08	0.07	0.07
query28	13.32	1.02	0.99
query29	13.31	3.38	3.31
query30	0.25	0.06	0.06
query31	2.89	0.38	0.38
query32	3.30	0.46	0.47
query33	2.85	2.90	2.86
query34	17.22	4.38	4.45
query35	4.50	4.46	4.55
query36	0.69	0.49	0.47
query37	0.17	0.15	0.15
query38	0.15	0.15	0.14
query39	0.05	0.03	0.03
query40	0.16	0.15	0.15
query41	0.08	0.05	0.05
query42	0.04	0.04	0.04
query43	0.04	0.03	0.03
Total cold run time: 109.83 s
Total hot run time: 30.67 s

dataroaring
dataroaring previously approved these changes May 30, 2024
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update doc of ba

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label May 30, 2024
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label May 30, 2024
@zddr
Copy link
Contributor Author

zddr commented May 30, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39740 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1a0b0c3ca375d7636344bf8167a03ee90f5ece56, data reload: false

------ Round 1 ----------------------------------
q1	18056	4441	4313	4313
q2	2547	200	195	195
q3	11591	1239	1304	1239
q4	11553	805	907	805
q5	7519	2712	2623	2623
q6	220	135	135	135
q7	959	611	586	586
q8	9222	2037	2041	2037
q9	8990	6471	6436	6436
q10	8883	3646	3686	3646
q11	457	241	234	234
q12	392	223	220	220
q13	17764	2969	2956	2956
q14	270	218	214	214
q15	514	459	472	459
q16	483	394	380	380
q17	955	635	705	635
q18	8073	7468	7293	7293
q19	5569	1584	1583	1583
q20	647	304	312	304
q21	4892	3114	3200	3114
q22	371	333	333	333
Total cold run time: 119927 ms
Total hot run time: 39740 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4324	4239	4212	4212
q2	379	271	278	271
q3	2943	2753	2708	2708
q4	1889	1564	1548	1548
q5	5286	5260	5282	5260
q6	212	125	127	125
q7	2094	1756	1758	1756
q8	3191	3291	3273	3273
q9	8281	8276	8273	8273
q10	3834	3691	3652	3652
q11	589	529	472	472
q12	755	593	580	580
q13	16307	2985	2985	2985
q14	300	244	265	244
q15	517	475	469	469
q16	475	412	412	412
q17	1780	1483	1492	1483
q18	7648	7559	7327	7327
q19	1684	1551	1545	1545
q20	1968	1785	1791	1785
q21	4737	4723	4682	4682
q22	578	526	537	526
Total cold run time: 69771 ms
Total hot run time: 53588 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 168902 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1a0b0c3ca375d7636344bf8167a03ee90f5ece56, data reload: false

query1	934	377	365	365
query2	6442	2557	2281	2281
query3	6653	204	205	204
query4	21748	17281	17248	17248
query5	4228	427	424	424
query6	246	164	154	154
query7	4589	305	291	291
query8	331	288	281	281
query9	8597	2367	2364	2364
query10	456	285	276	276
query11	10454	10061	10080	10061
query12	134	93	88	88
query13	1681	367	360	360
query14	9453	7523	6508	6508
query15	277	196	192	192
query16	8137	277	255	255
query17	1837	524	514	514
query18	2114	276	269	269
query19	196	160	152	152
query20	94	83	86	83
query21	201	136	127	127
query22	4251	3941	3822	3822
query23	33492	33184	33108	33108
query24	11923	2840	2900	2840
query25	639	359	371	359
query26	1758	160	168	160
query27	2973	328	320	320
query28	7536	2034	2037	2034
query29	994	609	599	599
query30	300	151	152	151
query31	964	764	744	744
query32	90	53	54	53
query33	768	273	264	264
query34	995	479	484	479
query35	749	611	601	601
query36	1061	932	938	932
query37	145	65	67	65
query38	2904	2772	2752	2752
query39	853	827	797	797
query40	276	124	128	124
query41	54	56	58	56
query42	103	99	127	99
query43	558	568	567	567
query44	1193	721	748	721
query45	190	172	174	172
query46	1070	742	731	731
query47	1844	1751	1756	1751
query48	371	295	298	295
query49	1180	385	396	385
query50	780	385	390	385
query51	6763	6724	6662	6662
query52	110	90	95	90
query53	359	295	293	293
query54	946	424	419	419
query55	74	74	70	70
query56	269	263	246	246
query57	1149	1043	1033	1033
query58	243	218	219	218
query59	3469	3227	3240	3227
query60	311	263	262	262
query61	93	88	88	88
query62	663	432	452	432
query63	312	294	295	294
query64	9723	2227	1770	1770
query65	3212	3118	3179	3118
query66	1375	330	325	325
query67	15045	14759	14768	14759
query68	4613	527	525	525
query69	424	268	277	268
query70	1166	1117	1068	1068
query71	409	269	273	269
query72	7692	2751	2545	2545
query73	709	324	318	318
query74	6026	5579	5654	5579
query75	3318	2688	2642	2642
query76	2556	1019	929	929
query77	409	270	270	270
query78	10193	9732	9782	9732
query79	2317	519	508	508
query80	885	443	452	443
query81	511	217	222	217
query82	663	92	93	92
query83	269	174	172	172
query84	233	83	92	83
query85	1560	280	264	264
query86	471	315	306	306
query87	3323	3120	3113	3113
query88	4275	2338	2320	2320
query89	503	398	384	384
query90	1998	196	196	196
query91	126	95	98	95
query92	59	52	50	50
query93	2081	515	496	496
query94	1240	188	185	185
query95	404	313	321	313
query96	586	266	265	265
query97	3163	3032	3039	3032
query98	233	221	217	217
query99	1186	837	854	837
Total cold run time: 276533 ms
Total hot run time: 168902 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.38 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1a0b0c3ca375d7636344bf8167a03ee90f5ece56, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.22	0.06	0.05
query4	1.67	0.07	0.07
query5	0.50	0.48	0.50
query6	1.12	0.72	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.53	0.48	0.49
query10	0.53	0.54	0.54
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.59	0.59	0.60
query14	0.80	0.77	0.78
query15	0.83	0.81	0.82
query16	0.37	0.35	0.35
query17	0.96	0.95	0.96
query18	0.24	0.25	0.23
query19	1.78	1.68	1.65
query20	0.01	0.00	0.01
query21	15.71	0.66	0.66
query22	3.98	7.29	1.86
query23	18.29	1.34	1.21
query24	1.46	0.43	0.19
query25	0.13	0.09	0.09
query26	0.26	0.17	0.17
query27	0.08	0.09	0.08
query28	13.26	1.02	1.07
query29	12.78	3.33	3.34
query30	0.24	0.05	0.06
query31	2.87	0.38	0.39
query32	3.31	0.48	0.46
query33	2.92	2.91	2.91
query34	17.29	4.45	4.44
query35	4.59	4.49	4.44
query36	0.65	0.46	0.46
query37	0.17	0.16	0.16
query38	0.16	0.15	0.14
query39	0.04	0.03	0.04
query40	0.16	0.14	0.14
query41	0.08	0.04	0.05
query42	0.06	0.04	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.18 s
Total hot run time: 30.38 s

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label May 30, 2024
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@zddr
Copy link
Contributor Author

zddr commented May 30, 2024

run cloud_p1

@morrySnow morrySnow merged commit 861ae81 into apache:master May 30, 2024
zddr added a commit to zddr/incubator-doris that referenced this pull request May 31, 2024
This is a temporary solution. In order to avoid affecting the existing
backup function, the backup MTMV will be allowed after detailed design
morrySnow pushed a commit that referenced this pull request May 31, 2024
pick from master #35586​

This is a temporary solution. In order to avoid affecting the existing
backup function, the backup MTMV will be allowed after detailed design
dataroaring pushed a commit that referenced this pull request May 31, 2024
This is a temporary solution. In order to avoid affecting the existing
backup function, the backup MTMV will be allowed after detailed design
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.4-merged dev/3.0.0-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants