Skip to content

Conversation

@LiBinfeng-01
Copy link
Contributor

@LiBinfeng-01 LiBinfeng-01 commented May 13, 2024

Proposed changes

pick from master #34761

Problem:
When using current_date as input of functions like date_sub, fold constant would failed cause of missing of function signature
in Planner
Solved:
Add complete function signature of functions like date_sub

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@LiBinfeng-01
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49695 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1a6b90c3b7973ed74837d7e533e381dc990db4b3, data reload: false

------ Round 1 ----------------------------------
q1	18271	4310	4293	4293
q2	2031	151	144	144
q3	10449	1872	1963	1872
q4	10354	1205	1337	1205
q5	8468	3922	3920	3920
q6	234	123	122	122
q7	2048	1610	1592	1592
q8	9495	2729	2710	2710
q9	14251	10506	10674	10506
q10	8644	3511	3495	3495
q11	405	244	248	244
q12	468	300	305	300
q13	18392	3982	4025	3982
q14	347	328	335	328
q15	511	475	464	464
q16	647	578	579	578
q17	1136	921	891	891
q18	7346	6844	6900	6844
q19	1713	1514	1533	1514
q20	539	305	302	302
q21	4377	4104	3997	3997
q22	495	396	392	392
Total cold run time: 120621 ms
Total hot run time: 49695 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4297	4267	4303	4267
q2	319	219	221	219
q3	4161	4111	4094	4094
q4	2736	2720	2729	2720
q5	7140	7132	7140	7132
q6	237	118	121	118
q7	3209	2766	2803	2766
q8	4326	4428	4444	4428
q9	17418	17082	17178	17082
q10	4225	4247	4237	4237
q11	761	670	650	650
q12	1045	868	864	864
q13	6618	3786	3743	3743
q14	470	426	431	426
q15	512	465	453	453
q16	733	668	675	668
q17	3757	3798	3817	3798
q18	8774	8684	8835	8684
q19	1724	1698	1657	1657
q20	2372	2163	2134	2134
q21	8470	8380	8372	8372
q22	1006	937	914	914
Total cold run time: 84310 ms
Total hot run time: 79426 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 201865 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1a6b90c3b7973ed74837d7e533e381dc990db4b3, data reload: false

query1	930	388	380	380
query2	6529	2767	2425	2425
query3	6921	211	205	205
query4	21347	17783	17836	17783
query5	19734	6533	6410	6410
query6	275	215	226	215
query7	4153	303	299	299
query8	250	276	225	225
query9	3124	2678	2621	2621
query10	413	301	292	292
query11	11322	10564	10705	10564
query12	124	79	71	71
query13	5588	672	675	672
query14	17236	13270	13490	13270
query15	358	220	221	220
query16	6452	284	257	257
query17	1728	1438	862	862
query18	2337	409	402	402
query19	199	155	150	150
query20	76	79	78	78
query21	182	90	95	90
query22	5093	5005	4985	4985
query23	32467	31931	31694	31694
query24	7018	6481	6491	6481
query25	519	436	424	424
query26	609	161	160	160
query27	2030	293	296	293
query28	6159	2371	2314	2314
query29	2943	2714	2776	2714
query30	247	164	167	164
query31	881	729	748	729
query32	68	64	61	61
query33	391	256	236	236
query34	845	484	476	476
query35	1119	915	918	915
query36	1241	1236	1186	1186
query37	88	57	60	57
query38	3045	2894	2925	2894
query39	1384	1337	1306	1306
query40	202	97	93	93
query41	38	35	35	35
query42	88	80	83	80
query43	688	692	692	692
query44	1121	709	707	707
query45	238	236	229	229
query46	1225	983	1000	983
query47	1788	1752	1690	1690
query48	995	712	688	688
query49	639	364	373	364
query50	865	617	634	617
query51	4788	4700	4674	4674
query52	95	83	91	83
query53	448	315	313	313
query54	2644	2454	2486	2454
query55	90	78	85	78
query56	224	226	223	223
query57	1083	1067	1041	1041
query58	192	212	209	209
query59	4105	3988	3863	3863
query60	216	214	194	194
query61	88	90	111	90
query62	867	446	448	446
query63	474	337	342	337
query64	2495	1499	1389	1389
query65	3582	3531	3572	3531
query66	816	361	376	361
query67	15735	15071	15267	15071
query68	9779	653	688	653
query69	562	348	350	348
query70	1631	1662	1521	1521
query71	411	303	302	302
query72	6400	3407	3411	3407
query73	739	348	326	326
query74	6286	5883	5828	5828
query75	5317	3684	3665	3665
query76	6265	1119	1186	1119
query77	1061	255	248	248
query78	12732	11649	13017	11649
query79	11484	654	655	654
query80	1339	391	384	384
query81	494	235	229	229
query82	526	99	92	92
query83	189	140	133	133
query84	262	68	71	68
query85	940	299	295	295
query86	344	291	319	291
query87	3260	3031	3002	3002
query88	5338	2371	2348	2348
query89	379	270	272	270
query90	2710	203	211	203
query91	166	133	134	133
query92	56	51	51	51
query93	3761	549	540	540
query94	1300	213	214	213
query95	1107	1069	1053	1053
query96	661	325	341	325
query97	6465	6384	6480	6384
query98	201	177	172	172
query99	3167	844	861	844
Total cold run time: 316586 ms
Total hot run time: 201865 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.66 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1a6b90c3b7973ed74837d7e533e381dc990db4b3, data reload: false

query1	0.02	0.02	0.03
query2	0.08	0.03	0.02
query3	0.26	0.04	0.05
query4	1.78	0.06	0.06
query5	0.53	0.53	0.52
query6	1.25	0.61	0.62
query7	0.02	0.01	0.01
query8	0.02	0.02	0.02
query9	0.54	0.49	0.49
query10	0.54	0.52	0.53
query11	0.12	0.08	0.09
query12	0.12	0.08	0.08
query13	0.63	0.62	0.60
query14	0.78	0.80	0.80
query15	0.78	0.77	0.76
query16	0.37	0.37	0.37
query17	1.03	0.99	1.01
query18	0.22	0.26	0.25
query19	1.93	1.84	1.85
query20	0.02	0.01	0.01
query21	15.48	0.56	0.55
query22	1.99	2.82	1.63
query23	17.06	0.99	1.00
query24	5.86	0.86	1.30
query25	0.42	0.07	0.07
query26	0.65	0.16	0.15
query27	0.04	0.04	0.04
query28	7.60	0.71	0.71
query29	12.65	2.35	2.29
query30	0.52	0.51	0.52
query31	2.82	0.38	0.37
query32	3.40	0.50	0.50
query33	3.13	3.05	3.03
query34	15.27	4.84	4.79
query35	4.86	4.87	4.85
query36	1.06	1.00	1.02
query37	0.06	0.04	0.05
query38	0.03	0.02	0.02
query39	0.02	0.01	0.01
query40	0.15	0.13	0.14
query41	0.06	0.01	0.02
query42	0.02	0.01	0.01
query43	0.02	0.02	0.01
Total cold run time: 104.21 s
Total hot run time: 30.66 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 1a6b90c3b7973ed74837d7e533e381dc990db4b3 with default session variables
Stream load json:         21 seconds loaded 2358488459 Bytes, about 107 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       21.9 seconds inserted 10000000 Rows, about 456K ops/s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label May 15, 2024
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@morrySnow
Copy link
Contributor

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49338 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 03e7ef98b43128dc6915e4e395fbb50b8ced5b23, data reload: false

------ Round 1 ----------------------------------
q1	17639	4384	4348	4348
q2	2038	153	149	149
q3	10427	1839	1912	1839
q4	10404	1249	1335	1249
q5	8889	3842	3904	3842
q6	235	125	123	123
q7	1999	1622	1560	1560
q8	9259	2706	2694	2694
q9	10614	10299	10139	10139
q10	8596	3487	3488	3487
q11	430	242	250	242
q12	469	306	306	306
q13	18361	3950	4007	3950
q14	346	334	318	318
q15	512	465	448	448
q16	675	577	579	577
q17	1134	970	975	970
q18	7353	6902	6796	6796
q19	1712	1608	1575	1575
q20	538	309	295	295
q21	4410	4088	4043	4043
q22	506	389	388	388
Total cold run time: 116546 ms
Total hot run time: 49338 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4292	4325	4355	4325
q2	320	220	225	220
q3	4163	4117	4136	4117
q4	2742	2731	2730	2730
q5	7151	7117	7083	7083
q6	233	121	120	120
q7	3218	2832	2853	2832
q8	4338	4425	4471	4425
q9	16872	16772	16716	16716
q10	4200	4211	4268	4211
q11	773	670	677	670
q12	1035	851	838	838
q13	6738	3739	3709	3709
q14	435	429	409	409
q15	502	467	449	449
q16	718	684	675	675
q17	3833	3861	3804	3804
q18	8743	8814	8810	8810
q19	1710	1701	1627	1627
q20	2386	2146	2119	2119
q21	8441	8456	8489	8456
q22	1018	918	924	918
Total cold run time: 83861 ms
Total hot run time: 79263 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 203860 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 03e7ef98b43128dc6915e4e395fbb50b8ced5b23, data reload: false

query1	953	388	379	379
query2	6523	2854	2697	2697
query3	6919	211	208	208
query4	21462	17959	18007	17959
query5	19717	6501	6497	6497
query6	273	222	222	222
query7	4164	314	314	314
query8	259	257	243	243
query9	3143	2701	2598	2598
query10	405	309	287	287
query11	11334	10692	10733	10692
query12	129	77	66	66
query13	5582	688	689	688
query14	17227	13357	13227	13227
query15	383	229	222	222
query16	6452	280	258	258
query17	1743	1469	874	874
query18	2319	409	402	402
query19	203	148	149	148
query20	75	73	81	73
query21	189	93	96	93
query22	5182	5044	5013	5013
query23	32523	31859	31998	31859
query24	7063	6624	6718	6624
query25	524	420	417	417
query26	526	160	165	160
query27	1874	305	294	294
query28	6184	2358	2294	2294
query29	2887	2823	2755	2755
query30	242	166	164	164
query31	900	745	714	714
query32	68	59	60	59
query33	394	239	269	239
query34	840	477	475	475
query35	1095	908	927	908
query36	1456	1336	1255	1255
query37	90	66	60	60
query38	3064	2927	2925	2925
query39	1368	1322	1325	1322
query40	205	91	92	91
query41	38	37	36	36
query42	81	88	89	88
query43	660	706	723	706
query44	1125	723	713	713
query45	239	225	225	225
query46	1222	982	981	981
query47	1790	1732	1654	1654
query48	1002	730	698	698
query49	614	366	356	356
query50	854	584	645	584
query51	4727	4649	4697	4649
query52	94	78	80	78
query53	448	323	321	321
query54	2669	2441	2421	2421
query55	88	82	90	82
query56	227	205	205	205
query57	1227	1226	1151	1151
query58	213	215	187	187
query59	4216	4148	4230	4148
query60	200	212	210	210
query61	95	83	82	82
query62	851	536	500	500
query63	474	345	346	345
query64	2394	1527	1514	1514
query65	3616	3587	3569	3569
query66	818	382	375	375
query67	16740	16297	15491	15491
query68	8797	648	640	640
query69	568	342	354	342
query70	1647	1363	1482	1363
query71	423	319	317	317
query72	6440	3432	3442	3432
query73	730	338	316	316
query74	6277	5843	5778	5778
query75	5214	3687	3690	3687
query76	5232	1162	1158	1158
query77	892	259	256	256
query78	12637	11668	12850	11668
query79	11084	663	670	663
query80	1420	392	393	392
query81	490	234	229	229
query82	641	99	96	96
query83	168	130	127	127
query84	253	67	69	67
query85	911	303	291	291
query86	327	292	290	290
query87	3257	2992	3005	2992
query88	4561	2357	2364	2357
query89	381	275	289	275
query90	1965	212	206	206
query91	166	143	134	134
query92	54	53	55	53
query93	4175	595	586	586
query94	751	206	207	206
query95	1127	1087	1037	1037
query96	648	333	321	321
query97	6477	6363	6374	6363
query98	191	175	169	169
query99	3058	905	916	905
Total cold run time: 313623 ms
Total hot run time: 203860 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.08 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 03e7ef98b43128dc6915e4e395fbb50b8ced5b23, data reload: false

query1	0.03	0.02	0.02
query2	0.07	0.02	0.02
query3	0.25	0.05	0.05
query4	1.79	0.07	0.06
query5	0.53	0.53	0.52
query6	1.24	0.62	0.61
query7	0.02	0.01	0.01
query8	0.03	0.02	0.02
query9	0.53	0.50	0.49
query10	0.55	0.54	0.53
query11	0.11	0.09	0.09
query12	0.11	0.09	0.09
query13	0.62	0.61	0.61
query14	0.79	0.78	0.81
query15	0.77	0.75	0.76
query16	0.37	0.38	0.38
query17	1.03	1.02	1.03
query18	0.24	0.24	0.24
query19	1.92	1.80	1.88
query20	0.01	0.02	0.01
query21	15.46	0.57	0.55
query22	2.02	3.12	1.41
query23	16.48	0.93	0.81
query24	6.09	0.81	0.66
query25	0.37	0.11	0.05
query26	0.61	0.15	0.14
query27	0.05	0.04	0.03
query28	7.56	0.80	0.76
query29	12.61	2.27	2.34
query30	0.60	0.57	0.55
query31	2.81	0.38	0.37
query32	3.41	0.50	0.49
query33	3.07	3.05	3.05
query34	15.25	4.80	4.77
query35	4.86	4.83	4.80
query36	1.05	1.01	1.03
query37	0.06	0.06	0.05
query38	0.03	0.02	0.02
query39	0.02	0.02	0.02
query40	0.17	0.13	0.14
query41	0.06	0.02	0.01
query42	0.02	0.02	0.01
query43	0.02	0.02	0.02
Total cold run time: 103.69 s
Total hot run time: 30.08 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 03e7ef98b43128dc6915e4e395fbb50b8ced5b23 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       22.0 seconds inserted 10000000 Rows, about 454K ops/s

@morrySnow morrySnow merged commit ba1c3da into apache:branch-2.0 May 15, 2024
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
…ture (apache#34760)

pick from master apache#34761

Problem:
When using current_date as input of functions like date_sub, fold constant would failed cause of missing of function signature in Planner

Solved:
Add complete function signature of functions like date_sub
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. kind/test reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants