Skip to content

[refactor](nereids)unify outputTupleDesc and projection #32093

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
englefly merged 2 commits into from
Mar 22, 2024
Merged

[refactor](nereids)unify outputTupleDesc and projection #32093

englefly merged 2 commits into from
Mar 22, 2024

Conversation

@englefly
Copy link
Contributor

@englefly englefly commented Mar 12, 2024 •
edited
Loading

Proposed changes

be part #32673

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

doris-robot commented Mar 12, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

@englefly englefly marked this pull request as draft March 12, 2024 05:26
@Mryange Mryange mentioned this pull request Mar 12, 2024
Closed
@englefly englefly force-pushed the unify-join branch 5 times, most recently from d87144e to 69a7338 Compare March 19, 2024 05:55
@github-actions
Copy link
Contributor

github-actions bot commented Mar 19, 2024

clang-tidy review says "All clean, LGTM! 👍"

@github-actions
Copy link
Contributor

github-actions bot commented Mar 19, 2024

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions
Copy link
Contributor

github-actions bot commented Mar 19, 2024

clang-tidy review says "All clean, LGTM! 👍"

@englefly
Copy link
Contributor Author

englefly commented Mar 19, 2024

run feut

@englefly
Copy link
Contributor Author

englefly commented Mar 19, 2024

run buildall

@englefly
Copy link
Contributor Author

englefly commented Mar 22, 2024

run buildall

@englefly englefly marked this pull request as ready for review March 22, 2024 02:03
@doris-robot
Copy link

doris-robot commented Mar 22, 2024

TPC-H: Total hot run time: 38006 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b72dac073c3838113e9a0f222ea356df119b4740, data reload: false

------ Round 1 ----------------------------------
q1	17629	4272	4177	4177
q2	2034	153	147	147
q3	10593	1153	1211	1153
q4	10234	791	741	741
q5	7474	3004	2969	2969
q6	203	126	126	126
q7	1053	617	567	567
q8	9341	2023	1975	1975
q9	7182	6627	6572	6572
q10	8423	3455	3536	3455
q11	434	235	222	222
q12	430	197	196	196
q13	17811	2870	2862	2862
q14	227	201	205	201
q15	519	462	466	462
q16	493	362	367	362
q17	961	556	594	556
q18	7074	6595	6483	6483
q19	4912	1400	1477	1400
q20	548	258	261	258
q21	3502	2938	2834	2834
q22	351	303	288	288
Total cold run time: 111428 ms
Total hot run time: 38006 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4161	4097	4088	4088
q2	318	220	220	220
q3	2947	2864	2831	2831
q4	1858	1600	1578	1578
q5	5292	5310	5336	5310
q6	196	113	117	113
q7	2269	1847	1835	1835
q8	3141	3268	3288	3268
q9	8708	8655	8717	8655
q10	3823	3802	3777	3777
q11	545	439	445	439
q12	723	559	527	527
q13	16920	2835	2851	2835
q14	283	249	253	249
q15	493	467	458	458
q16	455	427	409	409
q17	1744	1519	1483	1483
q18	7602	7148	7027	7027
q19	1653	1568	1509	1509
q20	1943	1718	1712	1712
q21	4692	4661	4673	4661
q22	527	436	454	436
Total cold run time: 70293 ms
Total hot run time: 53420 ms

@doris-robot
Copy link

doris-robot commented Mar 22, 2024

TPC-DS: Total hot run time: 187413 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b72dac073c3838113e9a0f222ea356df119b4740, data reload: false

query1	942	370	352	352
query2	7469	2062	1966	1966
query3	6715	216	227	216
query4	31850	21251	21350	21251
query5	4324	429	489	429
query6	266	174	187	174
query7	4636	292	296	292
query8	224	175	180	175
query9	9414	2278	2291	2278
query10	458	240	256	240
query11	15660	14616	14602	14602
query12	135	91	89	89
query13	1609	419	405	405
query14	12633	11529	11283	11283
query15	257	204	198	198
query16	8212	267	263	263
query17	1972	595	552	552
query18	2128	297	284	284
query19	345	162	164	162
query20	93	97	90	90
query21	206	132	127	127
query22	4880	4783	4745	4745
query23	33569	32715	32733	32715
query24	10780	2889	2874	2874
query25	610	388	388	388
query26	1189	155	163	155
query27	2505	351	361	351
query28	6999	1888	1902	1888
query29	892	675	644	644
query30	304	152	151	151
query31	1007	743	720	720
query32	104	61	57	57
query33	778	268	257	257
query34	1057	520	497	497
query35	836	618	615	615
query36	1058	904	895	895
query37	125	80	78	78
query38	3577	3474	3458	3458
query39	1468	1407	1424	1407
query40	214	120	118	118
query41	52	48	52	48
query42	107	100	98	98
query43	471	445	451	445
query44	1202	731	737	731
query45	294	260	252	252
query46	1113	688	720	688
query47	1906	1824	1838	1824
query48	455	365	352	352
query49	1102	346	339	339
query50	767	386	377	377
query51	6716	6633	6589	6589
query52	103	95	86	86
query53	347	277	277	277
query54	301	248	249	248
query55	91	76	77	76
query56	254	227	229	227
query57	1188	1121	1127	1121
query58	227	207	203	203
query59	2823	2653	2546	2546
query60	260	250	244	244
query61	94	95	91	91
query62	673	438	471	438
query63	306	282	280	280
query64	5226	4033	3881	3881
query65	3126	3057	3021	3021
query66	850	363	357	357
query67	14972	15095	14864	14864
query68	5467	519	537	519
query69	582	381	373	373
query70	1190	1194	1165	1165
query71	434	282	276	276
query72	6761	2839	2704	2704
query73	707	322	319	319
query74	7019	6681	6669	6669
query75	3766	2864	2866	2864
query76	4025	870	902	870
query77	622	267	266	266
query78	11014	10275	10257	10257
query79	6099	525	528	525
query80	1708	443	399	399
query81	563	216	216	216
query82	1016	216	215	215
query83	328	151	157	151
query84	295	88	83	83
query85	1629	387	311	311
query86	498	289	294	289
query87	3672	3577	3528	3528
query88	4995	2329	2311	2311
query89	490	384	375	375
query90	1978	177	176	176
query91	168	134	131	131
query92	64	51	47	47
query93	5078	503	491	491
query94	1200	182	175	175
query95	440	332	337	332
query96	604	278	266	266
query97	3070	2907	2873	2873
query98	226	215	212	212
query99	1246	905	881	881
Total cold run time: 303480 ms
Total hot run time: 187413 ms

@doris-robot
Copy link

doris-robot commented Mar 22, 2024

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit b72dac073c3838113e9a0f222ea356df119b4740 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      33 seconds loaded 861443392 Bytes, about 24 MB/s
Insert into select:       21.7 seconds inserted 10000000 Rows, about 460K ops/s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 22, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Mar 22, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Mar 22, 2024

PR approved by anyone and no changes requested.

@Mryange Mryange mentioned this pull request Mar 22, 2024
Merged
@englefly englefly merged commit 2e0e3c4 into apache:master Mar 22, 2024
yiguolei pushed a commit that referenced this pull request Mar 24, 2024
@englefly @Doris-Extras
[refactor](nereids)unify outputTupleDesc and projection (#32093)
8b960be 
* unify join node output project
Mryange added a commit to Mryange/doris that referenced this pull request Mar 28, 2024
@Mryange
Revert "[refactor](nereids)unify outputTupleDesc and projection (apac…
023077a 
…he#32093)"

This reverts commit 8b960be.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@starocean999 starocean999 starocean999 approved these changes

+1 more reviewer

@Mryange Mryange Mryange approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. meta-change reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@englefly @doris-robot @starocean999 @Mryange