Skip to content

[opt](nereids) disable infer column name when query #27450

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
starocean999 merged 2 commits into from
Nov 27, 2023
Merged

[opt](nereids) disable infer column name when query #27450

starocean999 merged 2 commits into from
Nov 27, 2023

Conversation

@seawinde
Copy link
Contributor

@seawinde seawinde commented Nov 23, 2023

Proposed changes

Disable infer column name when query, because it cause some errors when using BI tools
This feature is firstly developed by #26055

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@seawinde
Copy link
Contributor Author

seawinde commented Nov 23, 2023

run buildall

@wm1581066 wm1581066 requested a review from morrySnow November 23, 2023 03:59
@doris-robot
Copy link

doris-robot commented Nov 23, 2023

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 43.45 seconds
stream load tsv: 564 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.8 seconds inserted 10000000 Rows, about 347K ops/s
storage size: 17098825604 Bytes

@doris-robot
Copy link

doris-robot commented Nov 23, 2023

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' 
Tpch sf100 test result on commit 4b0d023e778dd7574ff608b140241c3ae9b07c06, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4911	4640	4724	4640
q2	354	164	159	159
q3	2043	1955	1928	1928
q4	1384	1272	1250	1250
q5	3987	3971	4040	3971
q6	254	127	131	127
q7	1451	887	875	875
q8	2787	2784	2789	2784
q9	9787	9580	9497	9497
q10	3491	3526	3559	3526
q11	384	236	254	236
q12	435	289	296	289
q13	4718	3813	3805	3805
q14	314	287	294	287
q15	582	527	520	520
q16	664	583	586	583
q17	1156	988	951	951
q18	7836	7485	7473	7473
q19	1692	1677	1669	1669
q20	580	338	311	311
q21	4440	3947	4047	3947
q22	486	378	374	374
Total cold run time: 53736 ms
Total hot run time: 49202 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4595	4598	4584	4584
q2	338	238	259	238
q3	4011	3991	4001	3991
q4	2704	2690	2686	2686
q5	9644	9700	9649	9649
q6	245	125	124	124
q7	3022	2487	2455	2455
q8	4481	4480	4457	4457
q9	12925	12848	12740	12740
q10	4107	4175	4173	4173
q11	801	697	624	624
q12	967	802	815	802
q13	4427	3533	3589	3533
q14	379	345	359	345
q15	577	516	528	516
q16	749	655	666	655
q17	3917	3878	3850	3850
q18	9554	8981	9146	8981
q19	1805	1771	1761	1761
q20	2401	2064	2046	2046
q21	8860	8578	8621	8578
q22	927	813	777	777
Total cold run time: 81436 ms
Total hot run time: 77565 ms

morrySnow
morrySnow previously approved these changes Nov 23, 2023
View reviewed changes
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 23, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Nov 23, 2023

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Nov 23, 2023

PR approved by anyone and no changes requested.

@seawinde
Copy link
Contributor Author

seawinde commented Nov 23, 2023

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Nov 23, 2023
@doris-robot
Copy link

doris-robot commented Nov 23, 2023

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' 
Tpch sf100 test result on commit f4e780bbb988bfc24fc0699ca3910dc3ce2dce88, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4909	4692	4632	4632
q2	359	164	159	159
q3	2084	1890	1938	1890
q4	1393	1254	1260	1254
q5	3973	3955	4029	3955
q6	248	128	131	128
q7	1435	887	880	880
q8	2775	2794	2781	2781
q9	9925	9681	9441	9441
q10	3448	3551	3538	3538
q11	375	245	237	237
q12	440	291	297	291
q13	4593	3779	3796	3779
q14	318	290	286	286
q15	590	534	518	518
q16	666	587	587	587
q17	1153	979	937	937
q18	7829	7398	7341	7341
q19	1680	1686	1682	1682
q20	542	327	306	306
q21	4449	4001	3985	3985
q22	476	377	383	377
Total cold run time: 53660 ms
Total hot run time: 48984 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4643	4574	4565	4565
q2	333	234	258	234
q3	4010	3994	4010	3994
q4	2707	2706	2700	2700
q5	9805	9741	9807	9741
q6	244	123	129	123
q7	3031	2487	2456	2456
q8	4495	4452	4453	4452
q9	13029	12942	13015	12942
q10	4076	4171	4176	4171
q11	850	707	646	646
q12	980	805	812	805
q13	4301	3571	3569	3569
q14	367	345	351	345
q15	572	517	522	517
q16	738	694	667	667
q17	3841	4008	3847	3847
q18	9727	8994	9130	8994
q19	1819	1795	1837	1795
q20	2414	2056	2054	2054
q21	8870	8710	8540	8540
q22	947	813	827	813
Total cold run time: 81799 ms
Total hot run time: 77970 ms

@doris-robot
Copy link

doris-robot commented Nov 23, 2023

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 43.56 seconds
stream load tsv: 569 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.6 seconds inserted 10000000 Rows, about 349K ops/s
storage size: 17097579746 Bytes

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 27, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Nov 27, 2023

PR approved by at least one committer and no changes requested.

@starocean999 starocean999 merged commit 0e1e4c8 into apache:master Nov 27, 2023
seawinde added a commit to seawinde/doris that referenced this pull request Nov 28, 2023
@seawinde
[opt](nereids) disable infer column name when query (apache#27450)
40ef670 
Disable infer column name when query, because it cause some errors when using BI tools
This feature is firstly developed by apache#26055
XuJianxu pushed a commit to XuJianxu/doris that referenced this pull request Dec 14, 2023
@seawinde
[opt](nereids) disable infer column name when query (apache#27450)
19d96dc 
Disable infer column name when query, because it cause some errors when using BI tools
This feature is firstly developed by apache#26055
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@starocean999 starocean999 starocean999 approved these changes

@morrySnow morrySnow morrySnow approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@seawinde @doris-robot @starocean999 @morrySnow