Skip to content

Conversation

@924060929
Copy link
Contributor

@924060929 924060929 commented Apr 1, 2024

Proposed changes

This sql will failed because

  1. 2 in the group by will bind to 1 as col2 in BindExpression
  2. ResolveOrdinalInOrderByAndGroupBy will replace 1 to MIN (LENGTH (cast(age as varchar)))
  3. CheckAnalysis will throw an exception because group by can not contains aggregate function
select MIN (LENGTH (cast(age as varchar))), 1 AS col2
from test_bind_groupby_slots
group by 2

we should move ResolveOrdinalInOrderByAndGroupBy into BindExpression

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@924060929
Copy link
Contributor Author

run buildall

@924060929
Copy link
Contributor Author

run buildall

@924060929
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38613 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 32724926867f1b91c98877c2bc8733f75c2ba29d, data reload: false

------ Round 1 ----------------------------------
q1	17634	4097	4078	4078
q2	2005	187	178	178
q3	10486	1307	1378	1307
q4	10212	868	1018	868
q5	7506	2941	2892	2892
q6	217	136	136	136
q7	1104	615	618	615
q8	9425	2054	2010	2010
q9	6681	6209	6129	6129
q10	8461	3502	3502	3502
q11	419	240	236	236
q12	374	214	215	214
q13	17773	2886	2924	2886
q14	266	243	249	243
q15	545	484	479	479
q16	486	399	385	385
q17	960	900	892	892
q18	7153	6437	6364	6364
q19	1574	1535	1524	1524
q20	610	337	305	305
q21	3514	3091	3065	3065
q22	361	312	305	305
Total cold run time: 107766 ms
Total hot run time: 38613 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4117	4048	4030	4030
q2	330	217	220	217
q3	2975	2940	2979	2940
q4	1883	1832	1861	1832
q5	5218	5210	5240	5210
q6	208	125	123	123
q7	2237	1836	1799	1799
q8	3197	3264	3270	3264
q9	8413	8442	8446	8442
q10	3723	3962	3985	3962
q11	583	484	473	473
q12	755	611	579	579
q13	16040	3049	3091	3049
q14	315	273	270	270
q15	539	486	495	486
q16	518	446	467	446
q17	1767	1737	1760	1737
q18	8248	7913	7756	7756
q19	1823	1669	1670	1669
q20	2078	1832	1763	1763
q21	5102	4861	4942	4861
q22	501	449	463	449
Total cold run time: 70570 ms
Total hot run time: 55357 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 183667 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 32724926867f1b91c98877c2bc8733f75c2ba29d, data reload: false

query1	1212	1129	1101	1101
query2	6279	2066	2068	2066
query3	6666	225	211	211
query4	24706	21560	21586	21560
query5	4211	401	417	401
query6	284	194	190	190
query7	4597	301	301	301
query8	233	181	171	171
query9	8451	2237	2236	2236
query10	446	250	258	250
query11	15037	15045	14637	14637
query12	141	100	98	98
query13	1642	383	395	383
query14	8465	7054	6780	6780
query15	214	181	176	176
query16	6808	276	277	276
query17	978	602	579	579
query18	1877	288	286	286
query19	211	168	169	168
query20	100	92	92	92
query21	194	144	128	128
query22	4948	4789	4786	4786
query23	33624	32773	32863	32773
query24	11720	3245	3190	3190
query25	620	406	408	406
query26	726	166	173	166
query27	2948	382	386	382
query28	5490	1880	1850	1850
query29	1027	583	604	583
query30	301	160	150	150
query31	1086	748	771	748
query32	98	66	61	61
query33	621	247	241	241
query34	918	505	514	505
query35	855	737	738	737
query36	1011	876	879	876
query37	124	84	92	84
query38	3730	3556	3643	3556
query39	1675	1611	1601	1601
query40	202	148	163	148
query41	52	49	50	49
query42	115	110	114	110
query43	461	435	423	423
query44	1076	750	737	737
query45	300	269	279	269
query46	1114	840	801	801
query47	1975	1861	1914	1861
query48	409	329	325	325
query49	880	384	378	378
query50	843	415	393	393
query51	6840	6804	6855	6804
query52	107	109	102	102
query53	387	308	310	308
query54	324	257	271	257
query55	93	84	82	82
query56	261	245	248	245
query57	1261	1189	1179	1179
query58	260	247	245	245
query59	2909	2727	2512	2512
query60	262	257	258	257
query61	129	108	107	107
query62	643	453	461	453
query63	310	284	287	284
query64	4438	3080	3084	3080
query65	3037	2997	2998	2997
query66	842	318	311	311
query67	15694	14945	14868	14868
query68	9260	563	572	563
query69	591	332	324	324
query70	1402	1097	1095	1095
query71	455	298	274	274
query72	6405	2577	2421	2421
query73	1603	320	320	320
query74	6677	6430	6434	6430
query75	3321	2327	2268	2268
query76	5031	1078	1219	1078
query77	583	250	251	250
query78	10899	10287	10153	10153
query79	10491	531	522	522
query80	1981	427	428	427
query81	515	1071	220	220
query82	336	103	103	103
query83	201	168	159	159
query84	255	89	86	86
query85	856	284	280	280
query86	354	278	312	278
query87	3693	3497	3467	3467
query88	3697	2267	2273	2267
query89	545	362	373	362
query90	1878	180	178	178
query91	130	103	111	103
query92	62	52	53	52
query93	6970	531	527	527
query94	1180	196	197	196
query95	1096	1099	1094	1094
query96	631	280	270	270
query97	2705	2478	2539	2478
query98	230	211	213	211
query99	1212	854	841	841
Total cold run time: 294743 ms
Total hot run time: 183667 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.46 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 32724926867f1b91c98877c2bc8733f75c2ba29d, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.24	0.06	0.05
query4	1.67	0.08	0.07
query5	0.50	0.49	0.49
query6	1.15	0.66	0.66
query7	0.02	0.01	0.01
query8	0.06	0.04	0.04
query9	0.56	0.51	0.51
query10	0.56	0.58	0.56
query11	0.15	0.11	0.11
query12	0.14	0.12	0.12
query13	0.61	0.59	0.60
query14	0.76	0.79	0.79
query15	0.86	0.85	0.84
query16	0.36	0.34	0.35
query17	0.99	1.00	0.97
query18	0.26	0.25	0.24
query19	1.79	1.72	1.73
query20	0.01	0.01	0.01
query21	15.41	0.75	0.71
query22	3.62	5.58	2.17
query23	17.53	1.23	1.09
query24	1.42	0.31	0.24
query25	0.13	0.10	0.09
query26	0.28	0.18	0.18
query27	0.09	0.09	0.09
query28	13.59	0.98	0.97
query29	12.73	3.24	3.46
query30	0.25	0.06	0.05
query31	2.84	0.39	0.38
query32	3.28	0.46	0.46
query33	2.90	2.87	2.92
query34	15.51	4.29	4.32
query35	4.36	4.38	4.35
query36	0.66	0.47	0.47
query37	0.20	0.18	0.16
query38	0.19	0.14	0.17
query39	0.04	0.04	0.04
query40	0.17	0.15	0.15
query41	0.09	0.05	0.04
query42	0.07	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 106.21 s
Total hot run time: 30.46 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 32724926867f1b91c98877c2bc8733f75c2ba29d with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       15.8 seconds inserted 10000000 Rows, about 632K ops/s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 1, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Apr 1, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Apr 1, 2024

PR approved by anyone and no changes requested.

@starocean999 starocean999 merged commit 3fab449 into apache:master Apr 1, 2024
@924060929 924060929 deleted the fix_groupby_constant branch April 7, 2024 02:26
924060929 added a commit to 924060929/incubator-doris that referenced this pull request Apr 10, 2024
This sql will failed because

    2 in the group by will bind to 1 as col2 in BindExpression
    ResolveOrdinalInOrderByAndGroupBy will replace 1 to MIN (LENGTH (cast(age as varchar)))
    CheckAnalysis will throw an exception because group by can not contains aggregate function

select MIN (LENGTH (cast(age as varchar))), 1 AS col2
from test_bind_groupby_slots
group by 2

we should move ResolveOrdinalInOrderByAndGroupBy into BindExpression

(cherry picked from commit 3fab449)
924060929 added a commit to 924060929/incubator-doris that referenced this pull request Apr 10, 2024
This sql will failed because

    2 in the group by will bind to 1 as col2 in BindExpression
    ResolveOrdinalInOrderByAndGroupBy will replace 1 to MIN (LENGTH (cast(age as varchar)))
    CheckAnalysis will throw an exception because group by can not contains aggregate function

select MIN (LENGTH (cast(age as varchar))), 1 AS col2
from test_bind_groupby_slots
group by 2

we should move ResolveOrdinalInOrderByAndGroupBy into BindExpression

(cherry picked from commit 3fab449)
yiguolei pushed a commit that referenced this pull request Apr 10, 2024
This sql will failed because

    2 in the group by will bind to 1 as col2 in BindExpression
    ResolveOrdinalInOrderByAndGroupBy will replace 1 to MIN (LENGTH (cast(age as varchar)))
    CheckAnalysis will throw an exception because group by can not contains aggregate function

select MIN (LENGTH (cast(age as varchar))), 1 AS col2
from test_bind_groupby_slots
group by 2

we should move ResolveOrdinalInOrderByAndGroupBy into BindExpression

(cherry picked from commit 3fab449)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.3-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants