Skip to content

Conversation

@nextdreamblue
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

cherry pick from #27027

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

Signed-off-by: nextdreamblue <zxw520blue1@163.com>
@nextdreamblue
Copy link
Contributor Author

run buildall

@github-actions github-actions bot added the area/planner Issues or PRs related to the query planner label Jan 19, 2024
@doris-robot
Copy link

TPC-H: Total hot run time: 49918 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ad1a1ff898e98ea01db499ea87860d83aca6a764, data reload: false

------ Round 1 ----------------------------------
q1	17685	4430	4387	4387
q2	2094	162	140	140
q3	10365	1890	1957	1890
q4	10099	1221	1324	1221
q5	8302	3940	3968	3940
q6	223	122	124	122
q7	2042	1592	1615	1592
q8	9312	2713	2721	2713
q9	11192	10681	10490	10490
q10	8643	3530	3491	3491
q11	409	230	242	230
q12	460	301	296	296
q13	18332	3975	4019	3975
q14	355	322	329	322
q15	511	455	456	455
q16	694	579	583	579
q17	1134	943	932	932
q18	7343	6820	6847	6820
q19	1682	1559	1567	1559
q20	536	313	305	305
q21	4470	4178	4055	4055
q22	492	414	404	404
Total cold run time: 116375 ms
Total hot run time: 49918 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4326	4307	4283	4283
q2	317	229	223	223
q3	4167	4119	4169	4119
q4	2747	2725	2720	2720
q5	7292	7223	7204	7204
q6	235	117	116	116
q7	3236	2784	2834	2784
q8	4294	4432	4448	4432
q9	17551	17374	17161	17161
q10	4182	4255	4237	4237
q11	759	731	671	671
q12	1020	860	843	843
q13	6699	3721	3723	3721
q14	446	427	435	427
q15	498	450	464	450
q16	758	696	693	693
q17	3824	3812	3911	3812
q18	8762	8829	8774	8774
q19	1689	1698	1648	1648
q20	2427	2181	2097	2097
q21	8438	8412	8514	8412
q22	1007	929	983	929
Total cold run time: 84674 ms
Total hot run time: 79756 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 240692 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ad1a1ff898e98ea01db499ea87860d83aca6a764, data reload: false

query1	910	396	384	384
query2	6554	2149	1974	1974
query3	6918	203	194	194
query4	21417	17906	17990	17906
query5	19735	6218	6253	6218
query6	293	216	214	214
query7	4145	296	304	296
query8	255	234	215	215
query9	3090	2680	2588	2588
query10	441	289	295	289
query11	11269	10606	10709	10606
query12	124	77	67	67
query13	5642	637	624	624
query14	17586	13072	13370	13072
query15	379	222	230	222
query16	6433	266	279	266
query17	3159	873	867	867
query18	2273	431	402	402
query19	201	144	146	144
query20	78	75	75	75
query21	185	94	102	94
query22	5185	5070	5029	5029
query23	32550	31694	31757	31694
query24	6917	6509	6465	6465
query25	533	411	425	411
query26	522	168	153	153
query27	1865	298	293	293
query28	6108	2238	2209	2209
query29	2997	2705	2781	2705
query30	251	160	157	157
query31	895	723	725	723
query32	60	61	58	58
query33	396	259	237	237
query34	831	462	467	462
query35	1099	951	886	886
query36	1740	1579	1391	1391
query37	88	58	59	58
query38	3113	2937	2930	2930
query39	1361	1328	1312	1312
query40	203	98	101	98
query41	34	33	36	33
query42	86	84	78	78
query43	668	536	580	536
query44	1144	708	705	705
query45	241	228	219	219
query46	1242	949	952	949
query47	1966	1717	1911	1717
query48	949	687	686	686
query49	611	364	356	356
query50	876	580	598	580
query51	5616	5491	5534	5491
query52	101	86	87	86
query53	439	323	321	321
query54	2452	2249	2245	2245
query55	83	72	84	72
query56	213	209	204	204
query57	1168	1100	1087	1087
query58	202	191	214	191
query59	3395	3143	3412	3143
query60	210	191	199	191
query61	89	90	92	90
query62	797	501	525	501
query63	478	331	333	331
query64	2008	1331	1292	1292
query65	3671	3625	3601	3601
query66	829	369	371	369
query67	15531	15145	15205	15145
query68	9515	670	649	649
query69	571	334	342	334
query70	1995	1516	1891	1516
query71	416	300	305	300
query72	4436	3408	3375	3375
query73	729	322	314	314
query74	6308	5865	5862	5862
query75	4892	3559	3569	3559
query76	6018	1206	1206	1206
query77	999	244	250	244
query78	31985	51413	56669	51413
query79	15472	670	633	633
query80	4501	389	400	389
query81	579	228	229	228
query82	2118	92	91	91
query83	337	142	132	132
query84	264	77	78	77
query85	1661	297	283	283
query86	449	366	363	363
query87	3236	3018	2997	2997
query88	7295	2316	2290	2290
query89	475	275	282	275
query90	2517	207	208	207
query91	154	118	118	118
query92	64	51	49	49
query93	6641	566	611	566
query94	1663	212	206	206
query95	1158	1061	1051	1051
query96	640	323	327	323
query97	6472	6424	6377	6377
query98	179	168	166	166
query99	3679	903	892	892
Total cold run time: 350318 ms
Total hot run time: 240692 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.37 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ad1a1ff898e98ea01db499ea87860d83aca6a764, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.02	0.02
query3	0.25	0.06	0.05
query4	1.79	0.07	0.07
query5	0.53	0.52	0.52
query6	1.29	0.63	0.62
query7	0.02	0.01	0.01
query8	0.04	0.02	0.02
query9	0.54	0.49	0.47
query10	0.53	0.52	0.53
query11	0.12	0.09	0.08
query12	0.12	0.09	0.09
query13	0.61	0.62	0.61
query14	0.77	0.78	0.78
query15	0.77	0.75	0.74
query16	0.36	0.38	0.37
query17	0.99	0.99	1.02
query18	0.20	0.26	0.26
query19	1.93	1.84	1.83
query20	0.02	0.02	0.02
query21	15.46	0.58	0.56
query22	1.87	1.82	1.52
query23	17.56	0.97	0.88
query24	5.21	1.52	2.92
query25	2.40	0.16	0.14
query26	0.15	0.13	0.14
query27	0.15	0.15	0.15
query28	5.51	0.73	0.71
query29	12.74	2.37	2.27
query30	0.56	0.54	0.53
query31	2.81	0.40	0.37
query32	3.37	0.49	0.49
query33	3.08	3.08	3.06
query34	15.30	4.80	4.81
query35	4.86	4.87	4.87
query36	1.03	1.01	1.01
query37	0.06	0.04	0.05
query38	0.03	0.02	0.02
query39	0.02	0.02	0.02
query40	0.16	0.14	0.14
query41	0.06	0.02	0.02
query42	0.02	0.01	0.01
query43	0.02	0.02	0.02
Total cold run time: 103.4 s
Total hot run time: 31.37 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit ad1a1ff898e98ea01db499ea87860d83aca6a764 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.4 seconds inserted 10000000 Rows, about 467K ops/s

@xiaokang xiaokang requested a review from dataroaring January 31, 2024 08:25
@xiaokang xiaokang marked this pull request as draft January 31, 2024 10:02
@xiaokang
Copy link
Contributor

xiaokang commented Feb 4, 2024

@nextdreamblue This PR changed OperationType and will cause rollback problem that old fe can not start with new fe image. So it can not be merged.

@github-actions
Copy link
Contributor

github-actions bot commented Aug 3, 2024

We're closing this PR because it hasn't been updated in a while.
This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and feel free a maintainer to remove the Stale tag!

@github-actions github-actions bot added the Stale label Aug 3, 2024
@github-actions github-actions bot closed this Aug 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/planner Issues or PRs related to the query planner Stale

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants