Skip to content

Conversation

@BePPPower
Copy link
Contributor

Problem Summary:

If the query is an outfile statement, we execute it only once to avoid exporting redundant data.

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Feb 19, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@BePPPower
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 31304 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b6e1b2644ee31860ffe6d5db035fe82f6ded53e1, data reload: false

------ Round 1 ----------------------------------
q1	17639	5332	5104	5104
q2	2041	309	168	168
q3	10400	1200	750	750
q4	10195	1001	552	552
q5	7530	2405	2279	2279
q6	181	168	132	132
q7	908	774	591	591
q8	9294	1276	1155	1155
q9	4844	4758	4578	4578
q10	6819	2304	1874	1874
q11	479	279	250	250
q12	352	350	225	225
q13	17759	3654	3076	3076
q14	223	225	205	205
q15	513	476	455	455
q16	623	620	578	578
q17	549	843	317	317
q18	6940	6104	6154	6104
q19	1209	939	536	536
q20	308	317	186	186
q21	2699	2101	1884	1884
q22	365	326	305	305
Total cold run time: 101870 ms
Total hot run time: 31304 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5187	5129	5146	5129
q2	239	327	227	227
q3	2166	2696	2280	2280
q4	1415	1862	1486	1486
q5	4540	4218	4214	4214
q6	218	167	129	129
q7	1922	1880	1724	1724
q8	2649	2652	2518	2518
q9	7191	7185	7085	7085
q10	3041	3199	2764	2764
q11	577	527	496	496
q12	697	792	627	627
q13	3483	3846	3360	3360
q14	273	295	272	272
q15	503	465	476	465
q16	633	682	637	637
q17	1160	1590	1329	1329
q18	7657	7399	7344	7344
q19	787	795	901	795
q20	1955	2013	1853	1853
q21	5368	4950	4913	4913
q22	579	584	547	547
Total cold run time: 52240 ms
Total hot run time: 50194 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190900 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b6e1b2644ee31860ffe6d5db035fe82f6ded53e1, data reload: false

query1	1300	963	932	932
query2	6245	1859	1863	1859
query3	11148	4567	4557	4557
query4	56795	25362	23450	23450
query5	5245	524	517	517
query6	364	186	185	185
query7	5214	498	294	294
query8	330	248	237	237
query9	6731	2548	2561	2548
query10	435	309	248	248
query11	15345	15117	14827	14827
query12	163	119	104	104
query13	1162	535	392	392
query14	10696	6383	6872	6383
query15	219	205	185	185
query16	7090	665	511	511
query17	1044	706	576	576
query18	1436	419	300	300
query19	194	203	171	171
query20	133	137	113	113
query21	206	121	102	102
query22	4551	4638	4420	4420
query23	34009	33198	33318	33198
query24	5798	2474	2469	2469
query25	490	487	439	439
query26	675	307	164	164
query27	1757	535	352	352
query28	2960	2482	2459	2459
query29	582	577	467	467
query30	219	194	158	158
query31	881	896	830	830
query32	70	65	60	60
query33	465	384	324	324
query34	760	853	539	539
query35	824	902	767	767
query36	939	999	896	896
query37	111	97	78	78
query38	4299	4233	4244	4233
query39	1488	1431	1424	1424
query40	199	110	100	100
query41	51	50	50	50
query42	126	116	117	116
query43	487	491	494	491
query44	1372	825	823	823
query45	186	175	172	172
query46	893	1062	678	678
query47	1802	1874	1799	1799
query48	384	421	315	315
query49	696	526	417	417
query50	717	762	447	447
query51	4278	4270	4261	4261
query52	106	110	95	95
query53	235	271	193	193
query54	484	507	422	422
query55	83	83	80	80
query56	256	269	269	269
query57	1190	1171	1112	1112
query58	262	241	234	234
query59	2849	2714	2589	2589
query60	294	295	273	273
query61	138	115	114	114
query62	731	725	688	688
query63	233	193	189	189
query64	1417	997	657	657
query65	3226	3157	3128	3128
query66	703	396	297	297
query67	15999	15488	15627	15488
query68	5474	796	533	533
query69	543	294	270	270
query70	1215	1082	1056	1056
query71	439	300	280	280
query72	5882	3674	3687	3674
query73	1000	737	366	366
query74	9191	9154	9025	9025
query75	3408	3150	2710	2710
query76	3855	1164	748	748
query77	542	358	282	282
query78	9846	10017	9403	9403
query79	2244	836	623	623
query80	581	533	454	454
query81	474	273	243	243
query82	638	127	97	97
query83	182	173	152	152
query84	341	92	74	74
query85	810	349	298	298
query86	410	304	289	289
query87	4500	4540	4463	4463
query88	3466	2230	2218	2218
query89	432	327	298	298
query90	1976	195	199	195
query91	130	133	108	108
query92	70	59	56	56
query93	1174	1023	594	594
query94	659	414	281	281
query95	346	273	264	264
query96	506	548	268	268
query97	2872	2913	2803	2803
query98	229	204	201	201
query99	1335	1416	1309	1309
Total cold run time: 296540 ms
Total hot run time: 190900 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.3 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b6e1b2644ee31860ffe6d5db035fe82f6ded53e1, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.03	0.03
query3	0.23	0.07	0.08
query4	1.61	0.10	0.10
query5	0.42	0.41	0.39
query6	1.18	0.66	0.68
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.57	0.52	0.52
query10	0.58	0.58	0.56
query11	0.15	0.10	0.11
query12	0.15	0.12	0.11
query13	0.62	0.59	0.60
query14	2.69	2.72	2.70
query15	0.92	0.87	0.84
query16	0.39	0.38	0.38
query17	1.02	1.02	1.03
query18	0.22	0.20	0.20
query19	1.87	1.84	1.78
query20	0.01	0.01	0.01
query21	15.36	0.91	0.54
query22	0.73	1.09	0.74
query23	14.95	1.38	0.60
query24	12.09	0.95	0.38
query25	0.33	0.11	0.10
query26	0.62	0.18	0.13
query27	0.06	0.05	0.04
query28	6.06	0.78	0.45
query29	12.56	3.94	3.32
query30	0.25	0.09	0.06
query31	2.84	0.59	0.39
query32	3.23	0.55	0.47
query33	2.99	3.01	2.98
query34	15.71	5.20	4.58
query35	4.59	4.58	4.57
query36	0.66	0.49	0.49
query37	0.09	0.07	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.02
query40	0.17	0.13	0.13
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.03	0.04	0.03
Total cold run time: 106.32 s
Total hot run time: 30.3 s

Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 21, 2025
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@morningman morningman added usercase Important user case type label dev/2.1.x dev/3.0.x labels Feb 21, 2025
@morningman morningman merged commit 2b01053 into apache:master Feb 24, 2025
34 of 37 checks passed
github-actions bot pushed a commit that referenced this pull request Feb 24, 2025
…ment. (#48095)

Problem Summary:

If the query is an `outfile` statement, we execute it only once to avoid
exporting redundant data.
dataroaring pushed a commit that referenced this pull request Feb 25, 2025
…tfile` statement. #48095 (#48216)

Cherry-picked from #48095

Co-authored-by: Tiewei Fang <fangtiewei@selectdb.com>
seawinde pushed a commit to seawinde/doris that referenced this pull request Feb 28, 2025
…ment. (apache#48095)

Problem Summary:

If the query is an `outfile` statement, we execute it only once to avoid
exporting redundant data.
morningman pushed a commit that referenced this pull request Feb 28, 2025
@yiguolei yiguolei mentioned this pull request Mar 25, 2025
@gavinchou gavinchou mentioned this pull request Apr 23, 2025
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
…ment. (apache#48095)

Problem Summary:

If the query is an `outfile` statement, we execute it only once to avoid
exporting redundant data.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.9-merged dev/3.0.5-merged kind/behavior-changed reviewed usercase Important user case type label

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants