Skip to content

[fix](Nereids) fix fold constant by be return type mismatched (#39723)(#41164) #41537

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morrySnow merged 2 commits into from
Oct 14, 2024
Merged

[fix](Nereids) fix fold constant by be return type mismatched (#39723)(#41164) #41537

morrySnow merged 2 commits into from
Oct 14, 2024

Conversation

@LiBinfeng-01
Copy link
Contributor

@LiBinfeng-01 LiBinfeng-01 commented Oct 8, 2024

cherry-pick: #39723 #41164 because later problem is intro by prev one, so put them together
when using fold constant by be, the return type of substring('123456',1, 3) would changed to be text, which we want it to be 3
remove windowframe in window expression to avoid folding constant on be

@LiBinfeng-01
[fix](Nereids) fix fold constant by be return type mismatched (apache…
6dcdb03 
…#39723)

when using fold constant by be, the return type of substring('123456',
1, 3) would changed to be text, which we want it to be 3
@LiBinfeng-01
[Fix](Nereids) fix fold const be return type (apache#41164)
dc5036a 
remove windowframe in window expression to avoid folding constant on be
@doris-robot
Copy link

doris-robot commented Oct 8, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@LiBinfeng-01
Copy link
Contributor Author

LiBinfeng-01 commented Oct 8, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Oct 8, 2024

TPC-H: Total hot run time: 48957 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit dc5036aacc54dcaa5b143c71426ee1eb618da21a, data reload: false

------ Round 1 ----------------------------------
q1	17724	4385	4329	4329
q2	2057	158	139	139
q3	10262	1898	1935	1898
q4	10345	1268	1297	1268
q5	8414	3925	3900	3900
q6	229	122	123	122
q7	2041	1608	1614	1608
q8	9321	2709	2694	2694
q9	10230	9737	9774	9737
q10	8662	3548	3488	3488
q11	409	250	253	250
q12	474	292	297	292
q13	18349	3984	4039	3984
q14	360	325	331	325
q15	522	469	468	468
q16	543	456	459	456
q17	1126	964	949	949
q18	7267	6877	6776	6776
q19	1699	1560	1554	1554
q20	516	334	305	305
q21	4394	4119	4023	4023
q22	495	392	399	392
Total cold run time: 115439 ms
Total hot run time: 48957 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4334	4299	4299	4299
q2	316	232	227	227
q3	4118	4150	4130	4130
q4	2761	2741	2760	2741
q5	7099	7114	7056	7056
q6	238	118	118	118
q7	3200	2895	2849	2849
q8	4377	4493	4497	4493
q9	13571	13580	13527	13527
q10	4269	4258	4221	4221
q11	757	671	723	671
q12	998	840	853	840
q13	6615	3750	3673	3673
q14	443	429	424	424
q15	499	464	463	463
q16	643	578	591	578
q17	3900	3829	3841	3829
q18	8798	8912	8834	8834
q19	1725	1713	1705	1705
q20	2370	2158	2126	2126
q21	8585	8447	8491	8447
q22	1069	955	938	938
Total cold run time: 80685 ms
Total hot run time: 76189 ms

@doris-robot
Copy link

doris-robot commented Oct 8, 2024

TPC-DS: Total hot run time: 213130 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit dc5036aacc54dcaa5b143c71426ee1eb618da21a, data reload: false

query1	925	395	418	395
query2	7036	2417	2098	2098
query3	7002	210	208	208
query4	23704	22313	22082	22082
query5	21017	6616	6645	6616
query6	351	216	237	216
query7	4846	308	312	308
query8	269	270	265	265
query9	3100	2625	2570	2570
query10	481	311	304	304
query11	15945	15585	15005	15005
query12	122	74	75	74
query13	1032	455	432	432
query14	18929	13651	13906	13651
query15	388	223	232	223
query16	6470	291	254	254
query17	1597	898	900	898
query18	899	316	317	316
query19	211	157	151	151
query20	101	91	97	91
query21	189	98	95	95
query22	5191	4860	5014	4860
query23	34406	33671	33399	33399
query24	7283	6329	6320	6320
query25	535	436	409	409
query26	1277	167	163	163
query27	2353	297	290	290
query28	6104	2247	2212	2212
query29	2887	2824	2787	2787
query30	240	166	169	166
query31	976	726	734	726
query32	71	61	59	59
query33	455	267	263	263
query34	869	470	478	470
query35	1146	922	898	898
query36	1438	1265	1105	1105
query37	173	58	61	58
query38	3101	2972	2946	2946
query39	1370	1313	1325	1313
query40	307	98	96	96
query41	47	36	49	36
query42	88	78	81	78
query43	596	637	581	581
query44	1176	700	708	700
query45	242	230	227	227
query46	1224	974	967	967
query47	2120	1707	1762	1707
query48	504	411	410	410
query49	637	381	369	369
query50	856	623	586	586
query51	4733	4700	4735	4700
query52	96	85	72	72
query53	238	188	189	188
query54	2679	2496	2496	2496
query55	87	89	85	85
query56	230	210	192	192
query57	1294	1178	1079	1079
query58	213	211	206	206
query59	3526	3163	3248	3163
query60	223	197	206	197
query61	96	95	93	93
query62	894	457	460	457
query63	204	172	180	172
query64	3484	1610	1501	1501
query65	3624	3535	3563	3535
query66	765	408	434	408
query67	17857	15594	15367	15367
query68	6414	654	659	654
query69	472	270	273	270
query70	1496	1323	1500	1323
query71	376	316	310	310
query72	6558	4768	4807	4768
query73	742	321	317	317
query74	6309	5880	5827	5827
query75	4344	3669	3765	3669
query76	3106	1185	1182	1182
query77	515	251	251	251
query78	12548	11622	11504	11504
query79	5685	635	627	627
query80	3088	384	393	384
query81	535	238	243	238
query82	1684	106	97	97
query83	300	130	133	130
query84	257	69	71	69
query85	1556	323	315	315
query86	402	285	285	285
query87	3234	3047	3058	3047
query88	4823	2298	2287	2287
query89	365	323	280	280
query90	1790	211	215	211
query91	172	125	125	125
query92	62	52	50	50
query93	2662	535	544	535
query94	1039	213	207	207
query95	2047	2057	2019	2019
query96	636	328	314	314
query97	6461	6383	6348	6348
query98	238	216	219	216
query99	3074	907	924	907
Total cold run time: 315519 ms
Total hot run time: 213130 ms

@doris-robot
Copy link

doris-robot commented Oct 8, 2024

ClickBench: Total hot run time: 31.55 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit dc5036aacc54dcaa5b143c71426ee1eb618da21a, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.02	0.02
query3	0.25	0.05	0.04
query4	1.79	0.07	0.07
query5	0.53	0.52	0.51
query6	1.23	0.66	0.62
query7	0.01	0.01	0.01
query8	0.03	0.02	0.02
query9	0.52	0.50	0.48
query10	0.54	0.54	0.53
query11	0.12	0.08	0.08
query12	0.11	0.10	0.09
query13	0.61	0.61	0.61
query14	0.80	0.80	0.79
query15	0.79	0.76	0.76
query16	0.40	0.37	0.38
query17	0.98	0.98	1.02
query18	0.24	0.25	0.27
query19	1.93	1.82	1.83
query20	0.01	0.01	0.01
query21	15.47	0.57	0.56
query22	1.97	2.54	1.89
query23	17.53	1.11	0.84
query24	4.99	3.13	1.65
query25	0.35	0.10	0.07
query26	0.81	0.15	0.16
query27	0.05	0.03	0.04
query28	5.59	0.75	0.72
query29	12.64	2.36	2.23
query30	0.53	0.54	0.52
query31	2.78	0.39	0.37
query32	3.42	0.49	0.50
query33	3.09	3.06	3.09
query34	15.24	4.80	4.80
query35	4.84	4.84	4.87
query36	1.05	1.01	1.02
query37	0.05	0.04	0.05
query38	0.04	0.02	0.02
query39	0.02	0.02	0.01
query40	0.15	0.13	0.15
query41	0.07	0.01	0.01
query42	0.02	0.01	0.01
query43	0.02	0.02	0.02
Total cold run time: 101.7 s
Total hot run time: 31.55 s

@doris-robot
Copy link

doris-robot commented Oct 8, 2024

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit dc5036aacc54dcaa5b143c71426ee1eb618da21a with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       22.0 seconds inserted 10000000 Rows, about 454K ops/s

@LiBinfeng-01
Copy link
Contributor Author

LiBinfeng-01 commented Oct 9, 2024

run p0

2 similar comments
@LiBinfeng-01
Copy link
Contributor Author

LiBinfeng-01 commented Oct 9, 2024

run p0

@LiBinfeng-01
Copy link
Contributor Author

LiBinfeng-01 commented Oct 11, 2024

run p0

@morrySnow morrySnow merged commit 2df49fb into apache:branch-2.0 Oct 14, 2024
LiBinfeng-01 added a commit to LiBinfeng-01/doris that referenced this pull request Oct 15, 2024
@LiBinfeng-01
[fix](Nereids) fix fold constant by be return type mismatched (apache…
14fcd46 
…#39723)(apache#41164) (apache#41537)

cherry-pick: apache#39723 apache#41164 because later problem is intro by prev one,
so put them together
when using fold constant by be,
the return type of substring('123456',1, 3) would changed to be text,
which we want it to be 3 remove windowframe in window expression
to avoid folding constant on be
LiBinfeng-01 added a commit to LiBinfeng-01/doris that referenced this pull request Oct 17, 2024
@LiBinfeng-01
[fix](Nereids) fix fold constant by be return type mismatched (apache…
7b7e114 
…#39723)(apache#41164) (apache#41537)

cherry-pick: apache#39723 apache#41164 because later problem is intro by prev one,
so put them together
when using fold constant by be,
the return type of substring('123456',1, 3) would changed to be text,
which we want it to be 3 remove windowframe in window expression
to avoid folding constant on be
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

area/nereids kind/test

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@LiBinfeng-01 @doris-robot @morrySnow