Skip to content

Conversation

@amorynan
Copy link
Contributor

@amorynan amorynan commented Dec 3, 2024

What problem does this PR solve?

nereids funcs signature for some nested type func's param type defined not right which will make result wrong such as

mysql> SELECT array_position([1,258],257),array_position([2],258);
+-------------------------------+-------------------------------------------+
| array_position([1, 258], 257) | array_position([2], cast(258 as TINYINT)) |
+-------------------------------+-------------------------------------------+
|                             0 |                                         1 |
+-------------------------------+-------------------------------------------+
1 row in set (0.14 sec)

mysql> select array_apply([258], '>' , 257), array_apply([1,2,3], '>', 258);
+------------------------------+---------------------------------------------------+
| array_apply([258], '>', 257) | array_apply([1, 2, 3], '>', cast(258 as TINYINT)) |
+------------------------------+---------------------------------------------------+
| [258]                        | [3]                                               |
+------------------------------+---------------------------------------------------+
1 row in set (0.11 sec)

mysql>  select array_contains([258], 257), array_contains([1,2,3], 258);
+----------------------------+-------------------------------------------------+
| array_contains([258], 257) | array_contains([1, 2, 3], cast(258 as TINYINT)) |
+----------------------------+-------------------------------------------------+
|                          0 |                                               1 |
+----------------------------+-------------------------------------------------+
1 row in set (0.12 sec)

mysql>
mysql> select array_pushfront([258], 257), array_pushfront([1,2,3], 258);
+-----------------------------+--------------------------------------------------+
| array_pushfront([258], 257) | array_pushfront([1, 2, 3], cast(258 as TINYINT)) |
+-----------------------------+--------------------------------------------------+
| [257, 258]                  | [2, 1, 2, 3]                                     |
+-----------------------------+--------------------------------------------------+
1 row in set (0.12 sec)

mysql> select array_pushback([1,258], 257), array_pushback([1,2,3], 258);
+-------------------------------+-------------------------------------------------+
| array_pushback([1, 258], 257) | array_pushback([1, 2, 3], cast(258 as TINYINT)) |
+-------------------------------+-------------------------------------------------+
| [1, 258, 257]                 | [1, 2, 3, 2]                                    |
+-------------------------------+-------------------------------------------------+
1 row in set (0.10 sec)

mysql> select array_remove([1,258], 257), array_remove([1,2,3], 258);
+-----------------------------+-----------------------------------------------+
| array_remove([1, 258], 257) | array_remove([1, 2, 3], cast(258 as TINYINT)) |
+-----------------------------+-----------------------------------------------+
| [1, 258]                    | [1, 3]                                        |
+-----------------------------+-----------------------------------------------+
1 row in set (0.12 sec)
mysql> select map_contains_key(map(1,258), 257), map_contains_key(map(1,2), 258);
+-----------------------------------------------------+---------------------------------------------------+
| map_contains_key(map(1, 258), cast(257 as TINYINT)) | map_contains_key(map(1, 2), cast(258 as TINYINT)) |
+-----------------------------------------------------+---------------------------------------------------+
|                                                   1 |                                                 0 |
+-----------------------------------------------------+---------------------------------------------------+
1 row in set (0.12 sec)

mysql> select map_contains_key(map(1,258), 257), map_contains_key(map(1,3), 258);
+-----------------------------------------------------+---------------------------------------------------+
| map_contains_key(map(1, 258), cast(257 as TINYINT)) | map_contains_key(map(1, 3), cast(258 as TINYINT)) |
+-----------------------------------------------------+---------------------------------------------------+
|                                                   1 |                                                 0 |
+-----------------------------------------------------+---------------------------------------------------+
1 row in set (0.11 sec)

mysql> select map_contains_key(map(1,258), 257), map_contains_key(map(3,1), 258);
+-----------------------------------------------------+---------------------------------------------------+
| map_contains_key(map(1, 258), cast(257 as TINYINT)) | map_contains_key(map(3, 1), cast(258 as TINYINT)) |
+-----------------------------------------------------+---------------------------------------------------+
|                                                   1 |                                                 0 |
+-----------------------------------------------------+---------------------------------------------------+

but true result is

mysql> SELECT array_position([1,258],257),array_position([2],258);
+-------------------------------+---------------------------------------------------+
| array_position([1, 258], 257) | array_position(cast([2] as ARRAY<SMALLINT>), 258) |
+-------------------------------+---------------------------------------------------+
|                             0 |                                                 0 |
+-------------------------------+---------------------------------------------------+
1 row in set (0.73 sec)

mysql> select array_apply([258], '>' , 257), array_apply([1,2,3], '>', 258);
+------------------------------+-----------------------------------------------------------+
| array_apply([258], '>', 257) | array_apply(cast([1, 2, 3] as ARRAY<SMALLINT>), '>', 258) |
+------------------------------+-----------------------------------------------------------+
| [258]                        | []                                                        |
+------------------------------+-----------------------------------------------------------+
1 row in set (0.10 sec)

mysql> select array_contains([258], 257), array_contains([1,2,3], 258);
+----------------------------+---------------------------------------------------------+
| array_contains([258], 257) | array_contains(cast([1, 2, 3] as ARRAY<SMALLINT>), 258) |
+----------------------------+---------------------------------------------------------+
|                          0 |                                                       0 |
+----------------------------+---------------------------------------------------------+
1 row in set (0.09 sec)

mysql> select array_pushfront([258], 257), array_pushfront([1,2,3], 258);
+-----------------------------+----------------------------------------------------------+
| array_pushfront([258], 257) | array_pushfront(cast([1, 2, 3] as ARRAY<SMALLINT>), 258) |
+-----------------------------+----------------------------------------------------------+
| [257, 258]                  | [258, 1, 2, 3]                                           |
+-----------------------------+----------------------------------------------------------+
1 row in set (0.11 sec)

mysql> select array_remove([1,258], 257), array_remove([1,2,3], 258);
+-----------------------------+-------------------------------------------------------+
| array_remove([1, 258], 257) | array_remove(cast([1, 2, 3] as ARRAY<SMALLINT>), 258) |
+-----------------------------+-------------------------------------------------------+
| [1, 258]                    | [1, 2, 3]                                             |
+-----------------------------+-------------------------------------------------------+
1 row in set (0.09 sec)

mysql> select countequal([1,258], 257), array_count_equal([1,2,3], 258);
ERROR 1105 (HY000): errCode = 2, detailMessage = Can not found function 'array_count_equal'
mysql> select countequal([1,258], 257), countequal([1,2,3], 258);
+---------------------------+-----------------------------------------------------+
| countequal([1, 258], 257) | countequal(cast([1, 2, 3] as ARRAY<SMALLINT>), 258) |
+---------------------------+-----------------------------------------------------+
|                         0 |                                                   0 |
+---------------------------+-----------------------------------------------------+
1 row in set (0.08 sec)

mysql> select map_contains_key(map(1,258), 257), map_contains_key(map(2,1), 258);
+--------------------------------------------------------------------+-----------------------------------------------------------------+
| map_contains_key(cast(map(1, 258) as MAP<SMALLINT,SMALLINT>), 257) | map_contains_key(cast(map(2, 1) as MAP<SMALLINT,TINYINT>), 258) |
+--------------------------------------------------------------------+-----------------------------------------------------------------+
|                                                                  0 |                                                               0 |
+--------------------------------------------------------------------+-----------------------------------------------------------------+
1 row in set (0.09 sec)

mysql> select map_contains_value(map(1,1), 257), map_contains_value(map(1,2), 258);
+-------------------------------------------------------------------+-------------------------------------------------------------------+
| map_contains_value(cast(map(1, 1) as MAP<TINYINT,SMALLINT>), 257) | map_contains_value(cast(map(1, 2) as MAP<TINYINT,SMALLINT>), 258) |
+-------------------------------------------------------------------+-------------------------------------------------------------------+
|                                                                 0 |                                                                 0 |
+--------------------------------------------------

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

Copy link
Contributor

@morrySnow morrySnow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some function return the first arg's type, so just change FollowToAnyDataType to AnyDataType is not ok

@amorynan
Copy link
Contributor Author

amorynan commented Dec 3, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40344 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f394a3572d47eaf0c30f4dc2726b9e16f801f2ff, data reload: false

------ Round 1 ----------------------------------
q1	17768	7483	7397	7397
q2	2040	173	174	173
q3	11076	1118	1211	1118
q4	10227	729	734	729
q5	7579	2755	2690	2690
q6	234	148	148	148
q7	1009	637	597	597
q8	9873	1842	1944	1842
q9	6869	6656	6676	6656
q10	7174	2378	2355	2355
q11	480	268	266	266
q12	492	230	229	229
q13	18072	3100	3079	3079
q14	248	222	208	208
q15	606	535	521	521
q16	1630	576	604	576
q17	1069	597	496	496
q18	8119	6664	6697	6664
q19	1346	985	994	985
q20	464	182	182	182
q21	3985	3211	3122	3122
q22	375	311	327	311
Total cold run time: 110735 ms
Total hot run time: 40344 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7297	7241	7254	7241
q2	329	228	227	227
q3	3011	2972	2936	2936
q4	2137	1820	1812	1812
q5	5699	5626	5634	5626
q6	237	142	146	142
q7	2224	1842	1804	1804
q8	3401	3531	3451	3451
q9	9040	8980	8991	8980
q10	3634	3527	3516	3516
q11	593	512	505	505
q12	828	631	626	626
q13	14487	3254	3218	3218
q14	311	279	271	271
q15	563	532	546	532
q16	694	651	678	651
q17	1845	1651	1603	1603
q18	8291	7793	7651	7651
q19	1703	1583	1562	1562
q20	2132	1899	1883	1883
q21	5559	5406	5510	5406
q22	634	604	573	573
Total cold run time: 74649 ms
Total hot run time: 60216 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197342 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f394a3572d47eaf0c30f4dc2726b9e16f801f2ff, data reload: false

query1	1213	757	763	757
query2	6267	2081	2057	2057
query3	10994	4354	4460	4354
query4	66715	28876	23582	23582
query5	4908	469	478	469
query6	414	191	175	175
query7	5497	300	291	291
query8	324	242	239	239
query9	8577	2683	2687	2683
query10	441	249	246	246
query11	17152	15421	16019	15421
query12	160	115	110	110
query13	1493	440	425	425
query14	10717	7140	7242	7140
query15	220	184	169	169
query16	7195	473	461	461
query17	1467	577	582	577
query18	1964	287	297	287
query19	213	155	148	148
query20	121	119	111	111
query21	203	101	99	99
query22	4719	4495	4585	4495
query23	34974	34421	34550	34421
query24	5594	2485	2545	2485
query25	494	379	391	379
query26	644	153	150	150
query27	1756	291	302	291
query28	4242	2472	2453	2453
query29	675	412	402	402
query30	209	151	151	151
query31	1006	855	837	837
query32	64	55	58	55
query33	409	303	333	303
query34	925	538	552	538
query35	881	786	774	774
query36	1109	949	951	949
query37	119	66	78	66
query38	4502	4447	4474	4447
query39	1494	1474	1493	1474
query40	197	102	101	101
query41	48	45	45	45
query42	115	103	97	97
query43	534	502	495	495
query44	1244	828	821	821
query45	198	179	171	171
query46	1175	738	720	720
query47	2036	1914	1917	1914
query48	406	316	315	315
query49	716	387	397	387
query50	836	389	402	389
query51	7471	7058	7095	7058
query52	97	89	87	87
query53	252	177	175	175
query54	500	405	382	382
query55	75	75	72	72
query56	248	222	235	222
query57	1244	1085	1113	1085
query58	210	214	210	210
query59	3266	3175	3061	3061
query60	309	262	257	257
query61	131	129	133	129
query62	809	689	669	669
query63	212	189	183	183
query64	1407	652	622	622
query65	3310	3252	3164	3164
query66	702	304	305	304
query67	15831	15678	15613	15613
query68	4220	563	542	542
query69	422	253	253	253
query70	1200	1106	1120	1106
query71	339	242	244	242
query72	6374	4055	4138	4055
query73	745	363	361	361
query74	10503	9005	9043	9005
query75	3414	2647	2656	2647
query76	1839	1152	1039	1039
query77	448	261	259	259
query78	10531	9504	9521	9504
query79	1608	588	587	587
query80	869	417	440	417
query81	485	243	227	227
query82	1284	115	114	114
query83	182	140	148	140
query84	284	70	73	70
query85	864	291	296	291
query86	336	298	299	298
query87	4748	4515	4645	4515
query88	3476	2230	2205	2205
query89	417	290	286	286
query90	2003	187	191	187
query91	142	107	103	103
query92	68	50	48	48
query93	1977	563	552	552
query94	800	294	295	294
query95	341	252	252	252
query96	624	271	269	269
query97	2867	2741	2720	2720
query98	222	199	197	197
query99	1595	1348	1338	1338
Total cold run time: 318877 ms
Total hot run time: 197342 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.87 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f394a3572d47eaf0c30f4dc2726b9e16f801f2ff, data reload: false

query1	0.03	0.04	0.03
query2	0.07	0.03	0.03
query3	0.23	0.08	0.06
query4	1.63	0.10	0.11
query5	0.43	0.41	0.42
query6	1.14	0.66	0.65
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.59	0.50	0.51
query10	0.56	0.56	0.56
query11	0.15	0.10	0.10
query12	0.15	0.12	0.10
query13	0.61	0.60	0.61
query14	2.82	2.79	2.83
query15	0.90	0.83	0.82
query16	0.38	0.37	0.38
query17	1.07	1.07	1.01
query18	0.22	0.20	0.21
query19	1.96	1.86	2.03
query20	0.01	0.01	0.01
query21	15.36	0.61	0.60
query22	2.75	2.33	1.42
query23	16.91	0.99	0.88
query24	3.02	1.80	1.47
query25	0.22	0.20	0.11
query26	0.58	0.14	0.14
query27	0.04	0.06	0.04
query28	9.78	1.12	1.08
query29	12.53	3.26	3.22
query30	0.25	0.07	0.07
query31	2.86	0.39	0.37
query32	3.29	0.47	0.48
query33	3.13	3.07	3.00
query34	17.11	4.44	4.47
query35	4.54	4.52	4.55
query36	0.67	0.49	0.50
query37	0.10	0.06	0.06
query38	0.04	0.03	0.03
query39	0.03	0.02	0.02
query40	0.17	0.13	0.13
query41	0.07	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.54 s
Total hot run time: 32.87 s

@amorynan amorynan requested a review from morrySnow December 4, 2024 09:33
@amorynan
Copy link
Contributor Author

amorynan commented Dec 4, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39634 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c5c5c205f361733d13feeb765059020dfce6c7c5, data reload: false

------ Round 1 ----------------------------------
q1	17583	7497	7199	7199
q2	2045	174	169	169
q3	10684	1075	1113	1075
q4	10539	714	661	661
q5	7581	2661	2663	2661
q6	240	147	150	147
q7	984	629	595	595
q8	9231	1844	1927	1844
q9	6576	6398	6439	6398
q10	7037	2321	2282	2282
q11	468	263	268	263
q12	420	232	221	221
q13	17786	3045	3010	3010
q14	245	224	212	212
q15	566	527	519	519
q16	658	579	585	579
q17	973	503	639	503
q18	7225	6793	6729	6729
q19	1358	993	1019	993
q20	473	178	185	178
q21	4327	3271	3086	3086
q22	380	333	310	310
Total cold run time: 107379 ms
Total hot run time: 39634 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7231	7199	7191	7191
q2	325	226	223	223
q3	2896	2818	2938	2818
q4	2063	1823	1834	1823
q5	5653	5664	5661	5661
q6	224	142	143	142
q7	2187	1838	1823	1823
q8	3314	3529	3462	3462
q9	8968	9014	8953	8953
q10	3589	3528	3538	3528
q11	597	521	515	515
q12	809	617	588	588
q13	10958	3289	3214	3214
q14	302	291	274	274
q15	568	550	537	537
q16	703	633	672	633
q17	1846	1621	1598	1598
q18	8208	7934	7793	7793
q19	1645	1456	1620	1456
q20	2118	1966	1885	1885
q21	5587	5534	5549	5534
q22	653	552	579	552
Total cold run time: 70444 ms
Total hot run time: 60203 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197439 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c5c5c205f361733d13feeb765059020dfce6c7c5, data reload: false

query1	1551	972	957	957
query2	6229	2001	1961	1961
query3	10961	4240	4295	4240
query4	66782	28893	23484	23484
query5	4884	471	456	456
query6	413	181	176	176
query7	5512	312	297	297
query8	319	250	237	237
query9	8744	2733	2723	2723
query10	454	250	290	250
query11	17199	15282	16086	15282
query12	157	115	103	103
query13	1479	441	412	412
query14	9842	6802	7371	6802
query15	214	194	203	194
query16	6931	467	528	467
query17	1106	630	568	568
query18	1599	307	312	307
query19	216	153	146	146
query20	118	121	112	112
query21	219	131	103	103
query22	4630	4617	4616	4616
query23	34961	35032	34961	34961
query24	5393	2663	2443	2443
query25	507	399	400	399
query26	724	162	161	161
query27	2003	293	293	293
query28	4335	2499	2475	2475
query29	671	418	420	418
query30	215	151	161	151
query31	1047	859	851	851
query32	68	54	54	54
query33	453	290	310	290
query34	933	527	522	522
query35	915	778	766	766
query36	1114	940	964	940
query37	125	77	77	77
query38	4519	4446	4519	4446
query39	1549	1493	1440	1440
query40	202	99	99	99
query41	45	43	44	43
query42	113	105	101	101
query43	558	499	514	499
query44	1216	878	823	823
query45	195	165	165	165
query46	1207	740	728	728
query47	1987	1947	1905	1905
query48	436	332	325	325
query49	759	412	391	391
query50	859	396	399	396
query51	7354	7138	7068	7068
query52	96	89	90	89
query53	257	181	183	181
query54	508	398	397	397
query55	79	76	75	75
query56	255	226	257	226
query57	1275	1127	1124	1124
query58	222	216	214	214
query59	3323	2902	3055	2902
query60	273	266	247	247
query61	110	127	105	105
query62	774	674	680	674
query63	214	193	202	193
query64	2104	703	655	655
query65	3282	3196	3243	3196
query66	668	296	308	296
query67	15843	15941	15676	15676
query68	3981	576	562	562
query69	414	260	251	251
query70	1218	1154	1159	1154
query71	331	259	265	259
query72	6296	4064	4057	4057
query73	760	364	364	364
query74	10115	8983	9046	8983
query75	3388	2689	2676	2676
query76	2123	1085	1048	1048
query77	547	299	282	282
query78	10542	9366	9412	9366
query79	2080	612	608	608
query80	1379	436	431	431
query81	502	274	226	226
query82	1290	118	118	118
query83	258	142	145	142
query84	279	67	68	67
query85	1024	310	304	304
query86	425	303	303	303
query87	4679	4525	4532	4525
query88	3813	2261	2198	2198
query89	424	300	293	293
query90	1860	197	192	192
query91	136	103	102	102
query92	63	52	51	51
query93	2791	545	540	540
query94	744	298	292	292
query95	340	249	247	247
query96	642	276	281	276
query97	2876	2687	2651	2651
query98	225	203	193	193
query99	1617	1315	1305	1305
Total cold run time: 320467 ms
Total hot run time: 197439 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.19 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c5c5c205f361733d13feeb765059020dfce6c7c5, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.06	0.08
query4	1.62	0.10	0.12
query5	0.42	0.39	0.41
query6	1.17	0.66	0.66
query7	0.03	0.02	0.02
query8	0.04	0.03	0.04
query9	0.59	0.51	0.50
query10	0.55	0.56	0.56
query11	0.14	0.10	0.10
query12	0.14	0.11	0.10
query13	0.61	0.61	0.59
query14	2.80	2.83	2.82
query15	0.89	0.82	0.83
query16	0.38	0.40	0.41
query17	1.04	0.97	1.07
query18	0.23	0.20	0.21
query19	1.85	1.79	2.00
query20	0.01	0.02	0.01
query21	15.37	0.56	0.58
query22	2.56	2.82	2.00
query23	16.92	0.91	0.83
query24	3.05	1.79	0.38
query25	0.26	0.12	0.24
query26	0.31	0.14	0.14
query27	0.04	0.05	0.04
query28	10.78	1.10	1.07
query29	12.52	3.27	3.25
query30	0.24	0.06	0.07
query31	2.86	0.39	0.37
query32	3.25	0.46	0.45
query33	2.99	3.04	3.00
query34	16.75	4.48	4.46
query35	4.52	4.53	4.49
query36	0.67	0.50	0.49
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.16	0.12	0.12
query41	0.07	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.02	0.02
Total cold run time: 106.41 s
Total hot run time: 32.19 s

@amorynan
Copy link
Contributor Author

amorynan commented Dec 5, 2024

run cloud_p0

@amorynan amorynan requested a review from morrySnow December 6, 2024 10:10
@amorynan
Copy link
Contributor Author

amorynan commented Dec 9, 2024

run cloud_p0

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 9, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Dec 9, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Dec 9, 2024

PR approved by anyone and no changes requested.

Copy link
Member

@eldenmoon eldenmoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@amorynan
Copy link
Contributor Author

amorynan commented Dec 9, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39935 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 829f542d1199afc4ab5f420b87128a72de993058, data reload: false

------ Round 1 ----------------------------------
q1	17639	7539	7247	7247
q2	2051	177	178	177
q3	10713	1196	1213	1196
q4	10512	692	753	692
q5	7600	2692	2644	2644
q6	238	147	150	147
q7	1009	627	607	607
q8	9254	1869	1899	1869
q9	6749	6451	6524	6451
q10	7032	2320	2258	2258
q11	470	255	261	255
q12	420	220	217	217
q13	17780	3012	3038	3012
q14	243	213	213	213
q15	572	525	516	516
q16	673	600	583	583
q17	979	538	582	538
q18	7455	6763	6802	6763
q19	1354	985	1074	985
q20	465	179	180	179
q21	3944	3240	3066	3066
q22	369	320	324	320
Total cold run time: 107521 ms
Total hot run time: 39935 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7267	7242	7214	7214
q2	328	235	228	228
q3	2920	2763	3119	2763
q4	2138	1801	1802	1801
q5	5554	5688	5650	5650
q6	227	143	137	137
q7	2221	1761	1773	1761
q8	3426	3552	3486	3486
q9	9025	8936	9074	8936
q10	3611	3540	3554	3540
q11	601	515	506	506
q12	814	622	659	622
q13	12865	3208	3173	3173
q14	301	261	268	261
q15	579	517	522	517
q16	677	615	639	615
q17	1797	1601	1564	1564
q18	7838	7587	7476	7476
q19	1645	1427	1558	1427
q20	2046	1789	1809	1789
q21	5388	5198	5227	5198
q22	651	539	570	539
Total cold run time: 71919 ms
Total hot run time: 59203 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191048 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 829f542d1199afc4ab5f420b87128a72de993058, data reload: false

query1	958	377	363	363
query2	6517	2088	2021	2021
query3	6706	219	213	213
query4	34148	23533	23603	23533
query5	4369	459	450	450
query6	288	202	184	184
query7	4635	303	321	303
query8	291	249	235	235
query9	9501	2693	2710	2693
query10	487	269	259	259
query11	18691	15257	15454	15257
query12	153	103	100	100
query13	1650	424	411	411
query14	9723	7555	7243	7243
query15	297	181	187	181
query16	8216	473	471	471
query17	1809	606	536	536
query18	2133	289	288	288
query19	355	154	149	149
query20	118	107	121	107
query21	198	102	106	102
query22	4364	4335	4146	4146
query23	34922	34148	34501	34148
query24	11444	2487	2390	2390
query25	655	363	383	363
query26	1779	153	150	150
query27	2786	289	284	284
query28	8009	2457	2433	2433
query29	1033	397	402	397
query30	300	148	148	148
query31	1030	822	809	809
query32	98	57	60	57
query33	813	284	280	280
query34	994	511	518	511
query35	891	739	742	739
query36	1103	946	939	939
query37	272	76	75	75
query38	4540	4262	4332	4262
query39	1492	1469	1442	1442
query40	279	98	98	98
query41	47	45	43	43
query42	109	99	96	96
query43	536	488	488	488
query44	1312	813	816	813
query45	182	168	168	168
query46	1166	714	693	693
query47	1929	1862	1835	1835
query48	413	324	321	321
query49	1285	392	390	390
query50	792	393	399	393
query51	7221	7171	7027	7027
query52	100	94	90	90
query53	262	193	184	184
query54	1242	410	405	405
query55	79	79	80	79
query56	259	238	233	233
query57	1265	1125	1093	1093
query58	237	211	218	211
query59	3227	3230	2988	2988
query60	289	247	258	247
query61	154	106	106	106
query62	882	659	664	659
query63	225	190	184	184
query64	5121	655	616	616
query65	3305	3218	3221	3218
query66	1426	307	318	307
query67	16121	15549	15613	15549
query68	4555	565	554	554
query69	421	261	268	261
query70	1236	1082	1109	1082
query71	345	255	249	249
query72	6301	3859	4027	3859
query73	770	355	355	355
query74	10643	9004	8990	8990
query75	3441	2689	2664	2664
query76	2709	1057	999	999
query77	528	281	275	275
query78	10384	9464	9501	9464
query79	1704	596	655	596
query80	1170	436	435	435
query81	538	225	230	225
query82	922	129	116	116
query83	240	155	144	144
query84	242	66	65	65
query85	1377	297	294	294
query86	408	307	302	302
query87	4685	4651	4576	4576
query88	3398	2192	2174	2174
query89	405	298	349	298
query90	2131	192	190	190
query91	138	100	100	100
query92	61	53	51	51
query93	1305	545	542	542
query94	1120	299	270	270
query95	361	259	251	251
query96	609	274	295	274
query97	2815	2688	2673	2673
query98	221	196	194	194
query99	1533	1350	1320	1320
Total cold run time: 303949 ms
Total hot run time: 191048 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.2 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 829f542d1199afc4ab5f420b87128a72de993058, data reload: false

query1	0.03	0.04	0.03
query2	0.07	0.03	0.03
query3	0.23	0.08	0.07
query4	1.61	0.11	0.10
query5	0.42	0.42	0.41
query6	1.15	0.66	0.65
query7	0.03	0.01	0.01
query8	0.04	0.03	0.03
query9	0.56	0.51	0.49
query10	0.55	0.56	0.54
query11	0.14	0.11	0.11
query12	0.13	0.12	0.11
query13	0.62	0.60	0.61
query14	2.72	2.74	2.70
query15	0.91	0.84	0.83
query16	0.38	0.38	0.38
query17	1.08	1.02	1.06
query18	0.22	0.22	0.22
query19	1.98	1.90	1.95
query20	0.02	0.01	0.01
query21	15.38	0.58	0.58
query22	2.55	2.16	1.63
query23	16.94	0.87	0.78
query24	3.44	1.73	0.78
query25	0.25	0.08	0.04
query26	0.59	0.14	0.13
query27	0.06	0.07	0.05
query28	10.34	1.10	1.07
query29	12.52	3.25	3.23
query30	0.24	0.06	0.06
query31	2.86	0.39	0.38
query32	3.29	0.47	0.46
query33	3.08	3.09	3.13
query34	16.84	4.45	4.45
query35	4.49	4.46	4.46
query36	0.66	0.48	0.48
query37	0.09	0.06	0.06
query38	0.04	0.03	0.03
query39	0.03	0.02	0.02
query40	0.16	0.12	0.11
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.89 s
Total hot run time: 32.2 s

@amorynan
Copy link
Contributor Author

amorynan commented Dec 9, 2024

run compile

@amorynan
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39785 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8a56be8d9144a5a482634cd4771e7b7b104b8663, data reload: false

------ Round 1 ----------------------------------
q1	17597	7512	7278	7278
q2	2040	179	176	176
q3	10683	1103	1151	1103
q4	10552	727	685	685
q5	7604	2726	2629	2629
q6	239	151	150	150
q7	997	632	594	594
q8	9239	1884	1900	1884
q9	6633	6492	6419	6419
q10	6994	2273	2330	2273
q11	468	262	270	262
q12	432	224	234	224
q13	17783	3007	3012	3007
q14	247	210	238	210
q15	558	540	526	526
q16	663	580	610	580
q17	988	624	534	534
q18	7283	6706	6649	6649
q19	1355	1062	962	962
q20	472	177	182	177
q21	4175	3268	3146	3146
q22	384	317	321	317
Total cold run time: 107386 ms
Total hot run time: 39785 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7248	7286	7212	7212
q2	349	226	221	221
q3	2904	2773	2993	2773
q4	2050	1789	1794	1789
q5	5725	5632	5681	5632
q6	227	141	145	141
q7	2246	1782	1836	1782
q8	3343	3521	3452	3452
q9	9095	9027	9029	9027
q10	3590	3576	3537	3537
q11	614	510	494	494
q12	828	607	625	607
q13	11015	3355	3302	3302
q14	302	263	279	263
q15	577	527	534	527
q16	707	673	654	654
q17	1853	1625	1618	1618
q18	8399	7780	7470	7470
q19	1697	1624	1528	1528
q20	2135	1884	1847	1847
q21	5637	5517	5469	5469
q22	673	611	563	563
Total cold run time: 71214 ms
Total hot run time: 59908 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196774 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8a56be8d9144a5a482634cd4771e7b7b104b8663, data reload: false

query1	1323	972	966	966
query2	6244	2037	2010	2010
query3	10983	4356	4339	4339
query4	66442	28779	23483	23483
query5	4958	473	448	448
query6	409	185	175	175
query7	5442	297	297	297
query8	333	240	239	239
query9	8617	2705	2694	2694
query10	421	242	244	242
query11	17057	15279	15874	15279
query12	160	98	97	97
query13	1404	405	408	405
query14	10472	6796	6850	6796
query15	222	202	198	198
query16	7163	473	460	460
query17	1108	617	638	617
query18	2009	318	307	307
query19	219	159	160	159
query20	122	117	122	117
query21	203	112	109	109
query22	4726	4648	4437	4437
query23	35021	34616	34868	34616
query24	5482	2491	2523	2491
query25	492	383	391	383
query26	640	156	161	156
query27	1943	280	285	280
query28	4732	2476	2447	2447
query29	712	411	422	411
query30	208	152	152	152
query31	1020	853	855	853
query32	66	55	54	54
query33	405	288	298	288
query34	942	517	536	517
query35	891	764	757	757
query36	1109	961	978	961
query37	123	86	82	82
query38	4595	4423	4551	4423
query39	1529	1501	1472	1472
query40	204	110	100	100
query41	46	43	45	43
query42	106	93	110	93
query43	544	502	494	494
query44	1232	850	823	823
query45	194	171	169	169
query46	1202	720	732	720
query47	2016	1970	1923	1923
query48	433	326	327	326
query49	759	386	375	375
query50	838	407	394	394
query51	7392	7223	7092	7092
query52	97	84	88	84
query53	252	181	174	174
query54	510	396	383	383
query55	79	78	77	77
query56	254	248	235	235
query57	1234	1126	1139	1126
query58	213	206	224	206
query59	3319	3265	2867	2867
query60	266	243	240	240
query61	111	102	108	102
query62	766	692	677	677
query63	221	189	186	186
query64	1401	644	672	644
query65	3273	3226	3172	3172
query66	694	299	304	299
query67	16002	15731	15558	15558
query68	3848	552	550	550
query69	427	270	247	247
query70	1165	1150	1138	1138
query71	362	251	243	243
query72	6459	4132	3999	3999
query73	776	351	347	347
query74	10242	9026	9113	9026
query75	3411	2680	2680	2680
query76	2126	1195	971	971
query77	509	276	270	270
query78	10519	9513	9460	9460
query79	1506	597	594	594
query80	889	438	438	438
query81	500	230	234	230
query82	1293	127	119	119
query83	248	150	144	144
query84	277	73	69	69
query85	959	311	292	292
query86	345	301	313	301
query87	4807	4564	4706	4564
query88	3660	2264	2170	2170
query89	423	301	289	289
query90	2026	188	193	188
query91	136	102	109	102
query92	70	48	51	48
query93	1854	530	531	530
query94	791	279	288	279
query95	347	243	243	243
query96	623	275	273	273
query97	2862	2675	2674	2674
query98	222	192	196	192
query99	1738	1318	1305	1305
Total cold run time: 319025 ms
Total hot run time: 196774 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.22 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8a56be8d9144a5a482634cd4771e7b7b104b8663, data reload: false

query1	0.03	0.04	0.03
query2	0.07	0.03	0.03
query3	0.24	0.07	0.07
query4	1.62	0.10	0.10
query5	0.44	0.41	0.42
query6	1.16	0.66	0.65
query7	0.03	0.01	0.02
query8	0.04	0.03	0.03
query9	0.56	0.52	0.49
query10	0.56	0.55	0.56
query11	0.13	0.10	0.10
query12	0.14	0.11	0.11
query13	0.61	0.60	0.61
query14	2.70	2.82	2.75
query15	0.91	0.82	0.83
query16	0.37	0.37	0.38
query17	1.06	1.04	1.05
query18	0.23	0.21	0.20
query19	1.80	1.76	1.98
query20	0.02	0.02	0.02
query21	15.36	0.58	0.58
query22	2.83	2.68	1.70
query23	16.82	1.12	0.88
query24	2.93	0.78	1.19
query25	0.16	0.06	0.06
query26	0.55	0.13	0.13
query27	0.05	0.04	0.04
query28	10.72	1.10	1.08
query29	12.55	3.23	3.20
query30	0.25	0.06	0.05
query31	2.87	0.38	0.38
query32	3.27	0.46	0.46
query33	2.99	3.06	3.04
query34	17.00	4.46	4.45
query35	4.52	4.51	4.47
query36	0.65	0.48	0.49
query37	0.09	0.06	0.06
query38	0.04	0.03	0.03
query39	0.03	0.02	0.02
query40	0.16	0.12	0.12
query41	0.08	0.03	0.02
query42	0.04	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.72 s
Total hot run time: 32.22 s

@eldenmoon eldenmoon closed this Dec 11, 2024
@eldenmoon eldenmoon reopened this Dec 11, 2024
Copy link
Member

@eldenmoon eldenmoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@eldenmoon eldenmoon merged commit f772baa into apache:master Dec 11, 2024
41 of 42 checks passed
github-actions bot pushed a commit that referenced this pull request Dec 11, 2024
… suitable and make result wrong (#44923)

nereids funcs signature for some nested type func's param type defined
not right which will make result wrong such as
```
mysql> SELECT array_position([1,258],257),array_position([2],258);
+-------------------------------+-------------------------------------------+
| array_position([1, 258], 257) | array_position([2], cast(258 as TINYINT)) |
+-------------------------------+-------------------------------------------+
|                             0 |                                         1 |
+-------------------------------+-------------------------------------------+
1 row in set (0.14 sec)

mysql> select array_apply([258], '>' , 257), array_apply([1,2,3], '>', 258);
+------------------------------+---------------------------------------------------+
| array_apply([258], '>', 257) | array_apply([1, 2, 3], '>', cast(258 as TINYINT)) |
+------------------------------+---------------------------------------------------+
| [258]                        | [3]                                               |
+------------------------------+---------------------------------------------------+
1 row in set (0.11 sec)

mysql>  select array_contains([258], 257), array_contains([1,2,3], 258);
+----------------------------+-------------------------------------------------+
| array_contains([258], 257) | array_contains([1, 2, 3], cast(258 as TINYINT)) |
+----------------------------+-------------------------------------------------+
|                          0 |                                               1 |
+----------------------------+-------------------------------------------------+
1 row in set (0.12 sec)

mysql>
mysql> select array_pushfront([258], 257), array_pushfront([1,2,3], 258);
+-----------------------------+--------------------------------------------------+
| array_pushfront([258], 257) | array_pushfront([1, 2, 3], cast(258 as TINYINT)) |
+-----------------------------+--------------------------------------------------+
| [257, 258]                  | [2, 1, 2, 3]                                     |
+-----------------------------+--------------------------------------------------+
1 row in set (0.12 sec)

mysql> select array_pushback([1,258], 257), array_pushback([1,2,3], 258);
+-------------------------------+-------------------------------------------------+
| array_pushback([1, 258], 257) | array_pushback([1, 2, 3], cast(258 as TINYINT)) |
+-------------------------------+-------------------------------------------------+
| [1, 258, 257]                 | [1, 2, 3, 2]                                    |
+-------------------------------+-------------------------------------------------+
1 row in set (0.10 sec)

mysql> select array_remove([1,258], 257), array_remove([1,2,3], 258);
+-----------------------------+-----------------------------------------------+
| array_remove([1, 258], 257) | array_remove([1, 2, 3], cast(258 as TINYINT)) |
+-----------------------------+-----------------------------------------------+
| [1, 258]                    | [1, 3]                                        |
+-----------------------------+-----------------------------------------------+
1 row in set (0.12 sec)
mysql> select map_contains_key(map(1,258), 257), map_contains_key(map(1,2), 258);
+-----------------------------------------------------+---------------------------------------------------+
| map_contains_key(map(1, 258), cast(257 as TINYINT)) | map_contains_key(map(1, 2), cast(258 as TINYINT)) |
+-----------------------------------------------------+---------------------------------------------------+
|                                                   1 |                                                 0 |
+-----------------------------------------------------+---------------------------------------------------+
1 row in set (0.12 sec)

mysql> select map_contains_key(map(1,258), 257), map_contains_key(map(1,3), 258);
+-----------------------------------------------------+---------------------------------------------------+
| map_contains_key(map(1, 258), cast(257 as TINYINT)) | map_contains_key(map(1, 3), cast(258 as TINYINT)) |
+-----------------------------------------------------+---------------------------------------------------+
|                                                   1 |                                                 0 |
+-----------------------------------------------------+---------------------------------------------------+
1 row in set (0.11 sec)

mysql> select map_contains_key(map(1,258), 257), map_contains_key(map(3,1), 258);
+-----------------------------------------------------+---------------------------------------------------+
| map_contains_key(map(1, 258), cast(257 as TINYINT)) | map_contains_key(map(3, 1), cast(258 as TINYINT)) |
+-----------------------------------------------------+---------------------------------------------------+
|                                                   1 |                                                 0 |
+-----------------------------------------------------+---------------------------------------------------+
```
but true result is 
```
mysql> SELECT array_position([1,258],257),array_position([2],258);
+-------------------------------+---------------------------------------------------+
| array_position([1, 258], 257) | array_position(cast([2] as ARRAY<SMALLINT>), 258) |
+-------------------------------+---------------------------------------------------+
|                             0 |                                                 0 |
+-------------------------------+---------------------------------------------------+
1 row in set (0.73 sec)

mysql> select array_apply([258], '>' , 257), array_apply([1,2,3], '>', 258);
+------------------------------+-----------------------------------------------------------+
| array_apply([258], '>', 257) | array_apply(cast([1, 2, 3] as ARRAY<SMALLINT>), '>', 258) |
+------------------------------+-----------------------------------------------------------+
| [258]                        | []                                                        |
+------------------------------+-----------------------------------------------------------+
1 row in set (0.10 sec)

mysql> select array_contains([258], 257), array_contains([1,2,3], 258);
+----------------------------+---------------------------------------------------------+
| array_contains([258], 257) | array_contains(cast([1, 2, 3] as ARRAY<SMALLINT>), 258) |
+----------------------------+---------------------------------------------------------+
|                          0 |                                                       0 |
+----------------------------+---------------------------------------------------------+
1 row in set (0.09 sec)

mysql> select array_pushfront([258], 257), array_pushfront([1,2,3], 258);
+-----------------------------+----------------------------------------------------------+
| array_pushfront([258], 257) | array_pushfront(cast([1, 2, 3] as ARRAY<SMALLINT>), 258) |
+-----------------------------+----------------------------------------------------------+
| [257, 258]                  | [258, 1, 2, 3]                                           |
+-----------------------------+----------------------------------------------------------+
1 row in set (0.11 sec)

mysql> select array_remove([1,258], 257), array_remove([1,2,3], 258);
+-----------------------------+-------------------------------------------------------+
| array_remove([1, 258], 257) | array_remove(cast([1, 2, 3] as ARRAY<SMALLINT>), 258) |
+-----------------------------+-------------------------------------------------------+
| [1, 258]                    | [1, 2, 3]                                             |
+-----------------------------+-------------------------------------------------------+
1 row in set (0.09 sec)

mysql> select countequal([1,258], 257), array_count_equal([1,2,3], 258);
ERROR 1105 (HY000): errCode = 2, detailMessage = Can not found function 'array_count_equal'
mysql> select countequal([1,258], 257), countequal([1,2,3], 258);
+---------------------------+-----------------------------------------------------+
| countequal([1, 258], 257) | countequal(cast([1, 2, 3] as ARRAY<SMALLINT>), 258) |
+---------------------------+-----------------------------------------------------+
|                         0 |                                                   0 |
+---------------------------+-----------------------------------------------------+
1 row in set (0.08 sec)

mysql> select map_contains_key(map(1,258), 257), map_contains_key(map(2,1), 258);
+--------------------------------------------------------------------+-----------------------------------------------------------------+
| map_contains_key(cast(map(1, 258) as MAP<SMALLINT,SMALLINT>), 257) | map_contains_key(cast(map(2, 1) as MAP<SMALLINT,TINYINT>), 258) |
+--------------------------------------------------------------------+-----------------------------------------------------------------+
|                                                                  0 |                                                               0 |
+--------------------------------------------------------------------+-----------------------------------------------------------------+
1 row in set (0.09 sec)

mysql> select map_contains_value(map(1,1), 257), map_contains_value(map(1,2), 258);
+-------------------------------------------------------------------+-------------------------------------------------------------------+
| map_contains_value(cast(map(1, 1) as MAP<TINYINT,SMALLINT>), 257) | map_contains_value(cast(map(1, 2) as MAP<TINYINT,SMALLINT>), 258) |
+-------------------------------------------------------------------+-------------------------------------------------------------------+
|                                                                 0 |                                                                 0 |
+--------------------------------------------------
```
morrySnow pushed a commit that referenced this pull request Dec 23, 2024
…which is not suitable and make result wrong #44923 (#45296)

Cherry-picked from #44923

Co-authored-by: amory <wangqiannan@selectdb.com>
morrySnow pushed a commit that referenced this pull request Dec 24, 2024
…which is not suitable and make result wrong #44923 (#45798)

Cherry-picked from #44923
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.8-merged dev/3.0.4-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants