Skip to content

Conversation

@morrySnow
Copy link
Contributor

pick from master #39640

otherwise:

push limit through union could generate plan:

limit
+-- union
|-- limit
| +-- empty relation
+-- limit
+-- project

and then eliminate union will generate plan:

+-- limit
+- project
+-- limit
+-- project

it could not be processed by tranlator correctly

Proposed changes

Issue Number: close #xxx

…n node (apache#39640)

pick from master apache#39640

otherwise:

push limit through union could generate plan:

limit
+-- union
    |-- limit
    |   +-- empty relation
    +-- limit
        +-- project

and then eliminate union will generate plan:

+-- limit
    +-  project
        +-- limit
            +-- project

it could not be processed by tranlator correctly
@morrySnow
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@doris-robot
Copy link

TPC-H: Total hot run time: 50802 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8b6388fae9257c4e2c469b4ae13f816a27f1cd2e, data reload: false

------ Round 1 ----------------------------------
q1	17810	4434	4374	4374
q2	2118	211	193	193
q3	10364	1951	1944	1944
q4	10381	1288	1337	1288
q5	8600	3966	3981	3966
q6	273	146	143	143
q7	2086	1661	1675	1661
q8	9572	2779	2770	2770
q9	13963	10534	10495	10495
q10	8669	3517	3519	3517
q11	441	277	275	275
q12	540	334	354	334
q13	18351	3985	4044	3985
q14	383	355	351	351
q15	548	500	499	499
q16	688	596	600	596
q17	1171	943	916	916
q18	7258	6954	7006	6954
q19	1738	1603	1616	1603
q20	555	353	349	349
q21	4470	4159	4152	4152
q22	550	438	437	437
Total cold run time: 120529 ms
Total hot run time: 50802 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4362	4365	4320	4320
q2	368	281	283	281
q3	4194	4144	4153	4144
q4	2764	2773	2780	2773
q5	7235	7159	7241	7159
q6	258	138	138	138
q7	3332	2880	2930	2880
q8	4396	4527	4492	4492
q9	17472	17214	16925	16925
q10	4300	4300	4296	4296
q11	840	720	721	720
q12	1058	892	869	869
q13	7121	3756	3763	3756
q14	483	448	449	448
q15	548	512	515	512
q16	778	725	718	718
q17	3825	3942	3885	3885
q18	8718	8699	8843	8699
q19	1778	1740	1713	1713
q20	2431	2149	2132	2132
q21	8525	8498	8470	8470
q22	1066	996	933	933
Total cold run time: 85852 ms
Total hot run time: 80263 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 211353 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8b6388fae9257c4e2c469b4ae13f816a27f1cd2e, data reload: false

query1	971	424	418	418
query2	6756	2275	2195	2195
query3	6947	231	226	226
query4	20449	18235	18166	18166
query5	19953	6738	6732	6732
query6	337	256	269	256
query7	4185	334	340	334
query8	489	459	454	454
query9	3256	2822	2765	2765
query10	501	377	377	377
query11	11338	10826	10783	10783
query12	165	116	110	110
query13	5673	745	725	725
query14	17683	13418	13623	13418
query15	385	264	247	247
query16	6497	323	305	305
query17	1648	1478	944	944
query18	2313	481	488	481
query19	232	195	183	183
query20	118	115	116	115
query21	247	154	151	151
query22	5159	4941	4947	4941
query23	32754	32214	32171	32171
query24	7125	6595	6577	6577
query25	567	483	479	479
query26	559	197	192	192
query27	1837	342	336	336
query28	5971	2427	2365	2365
query29	3066	2680	2853	2680
query30	291	217	204	204
query31	1020	825	826	825
query32	101	92	92	92
query33	484	356	335	335
query34	906	540	545	540
query35	1199	1001	1042	1001
query36	1279	1120	1249	1120
query37	123	88	96	88
query38	3137	2978	3007	2978
query39	1516	1437	1451	1437
query40	256	152	151	151
query41	163	157	155	155
query42	110	116	110	110
query43	737	709	798	709
query44	1210	780	796	780
query45	281	274	272	272
query46	1294	1016	1018	1016
query47	1939	1770	1828	1770
query48	1046	742	745	742
query49	854	592	601	592
query50	943	695	675	675
query51	4877	4697	4789	4697
query52	131	112	102	102
query53	516	383	383	383
query54	2743	2546	2535	2535
query55	121	106	100	100
query56	317	305	288	288
query57	1298	1244	1202	1202
query58	325	312	307	307
query59	3553	3381	3471	3381
query60	290	277	300	277
query61	163	162	157	157
query62	888	531	539	531
query63	534	403	418	403
query64	2703	1712	1678	1678
query65	3678	3627	3623	3623
query66	1272	815	820	815
query67	15486	16889	17000	16889
query68	6916	727	707	707
query69	624	437	418	418
query70	1670	1478	1461	1461
query71	454	363	381	363
query72	6603	3613	3561	3561
query73	793	382	370	370
query74	6388	5988	6008	5988
query75	4618	3773	3835	3773
query76	4419	1193	1219	1193
query77	720	435	426	426
query78	12823	11949	11886	11886
query79	8203	695	698	695
query80	1523	564	565	564
query81	546	278	277	277
query82	1590	133	132	132
query83	266	220	218	218
query84	288	99	98	98
query85	1087	419	421	419
query86	378	326	327	326
query87	3237	3088	3095	3088
query88	5410	2538	2529	2529
query89	439	392	342	342
query90	1943	263	268	263
query91	196	173	159	159
query92	92	85	87	85
query93	5091	623	593	593
query94	819	252	262	252
query95	1171	1107	1101	1101
query96	673	344	343	343
query97	6664	6540	6537	6537
query98	230	216	196	196
query99	3046	904	952	904
Total cold run time: 313885 ms
Total hot run time: 211353 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.1 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8b6388fae9257c4e2c469b4ae13f816a27f1cd2e, data reload: false

query1	0.02	0.02	0.03
query2	0.08	0.03	0.03
query3	0.25	0.05	0.05
query4	1.77	0.07	0.08
query5	0.55	0.53	0.52
query6	1.31	0.61	0.68
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.52	0.50	0.49
query10	0.55	0.55	0.55
query11	0.14	0.09	0.10
query12	0.13	0.11	0.11
query13	0.65	0.62	0.61
query14	0.79	0.79	0.79
query15	0.78	0.76	0.78
query16	0.37	0.37	0.37
query17	0.97	1.04	1.04
query18	0.24	0.25	0.24
query19	1.86	1.80	1.84
query20	0.02	0.02	0.01
query21	15.48	0.57	0.56
query22	1.90	1.55	2.08
query23	17.48	1.07	0.98
query24	5.89	0.98	0.86
query25	0.33	0.15	0.05
query26	0.63	0.16	0.16
query27	0.05	0.05	0.05
query28	7.42	0.85	0.75
query29	12.78	2.19	2.28
query30	0.77	0.71	0.79
query31	2.82	0.40	0.38
query32	3.35	0.51	0.52
query33	3.10	3.12	3.06
query34	15.28	4.86	4.81
query35	4.87	4.90	4.87
query36	1.06	1.03	1.06
query37	0.07	0.07	0.06
query38	0.05	0.03	0.05
query39	0.04	0.03	0.03
query40	0.18	0.16	0.17
query41	0.08	0.04	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.04
Total cold run time: 104.77 s
Total hot run time: 31.1 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 8b6388fae9257c4e2c469b4ae13f816a27f1cd2e with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.4 seconds inserted 10000000 Rows, about 467K ops/s

@morrySnow morrySnow merged commit 3befbe2 into apache:branch-2.0 Aug 22, 2024
@morrySnow morrySnow deleted the 2.0_39640 branch September 18, 2024 03:02
mongo360 pushed a commit to mongo360/doris that referenced this pull request Dec 11, 2024
…n node (apache#39640) (apache#39761)

pick from master apache#39640

otherwise:

push limit through union could generate plan:

limit
+-- union
    |-- limit
    |   +-- empty relation
    +-- limit
        +-- project

and then eliminate union will generate plan:

+-- limit
    +-  project
        +-- limit
            +-- project

it could not be processed by tranlator correctly
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants