Skip to content

Conversation

@morningman
Copy link
Contributor

…0463)

Issue Number: apache#50238

Problem Summary:

Previously, we refactored the code of the `fileFormat` attribute
(apache#50225). However, we only added the relevant code without modifying the
business code. This pull request (PR) modifies the code of the
table-valued function (tvf) that is related to the `fileformat` format.
…he#50471)

Issue Number: apache#50238

Problem Summary:

Previously, we refactored the code of the fileFormat attribute (apache#50225).
However, we only added the relevant code without modifying the business
code. This pull request modifies the code of the `SELECT INTO OUTFILE`
feature that is related to the fileformat.
@morningman morningman requested a review from morrySnow as a code owner June 21, 2025 07:43
@Thearas
Copy link
Contributor

Thearas commented Jun 21, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morningman
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39265 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 613d908944ee87510d271e9f543fe4f54f66785f, data reload: false

------ Round 1 ----------------------------------
q1	17566	6700	6597	6597
q2	2071	163	171	163
q3	10624	1073	1143	1073
q4	10567	758	804	758
q5	7747	2846	2782	2782
q6	214	130	133	130
q7	985	608	593	593
q8	9355	1944	2001	1944
q9	6582	6337	6423	6337
q10	6988	2262	2304	2262
q11	462	260	255	255
q12	391	213	209	209
q13	17774	2955	2993	2955
q14	238	209	212	209
q15	511	458	461	458
q16	472	365	370	365
q17	965	544	541	541
q18	7211	6625	6519	6519
q19	1386	1085	1005	1005
q20	468	198	193	193
q21	3854	3206	2933	2933
q22	1110	1000	984	984
Total cold run time: 107541 ms
Total hot run time: 39265 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6641	6549	6539	6539
q2	329	236	235	235
q3	2870	2720	2753	2720
q4	2021	1854	1790	1790
q5	5708	5660	5710	5660
q6	210	126	129	126
q7	2184	1836	1760	1760
q8	3371	3526	3549	3526
q9	8920	8739	8853	8739
q10	3588	3478	3513	3478
q11	607	517	494	494
q12	812	580	614	580
q13	9437	3144	3097	3097
q14	312	289	280	280
q15	507	464	455	455
q16	487	436	438	436
q17	1839	1629	1621	1621
q18	8109	7740	7717	7717
q19	1668	1573	1654	1573
q20	2099	1906	1844	1844
q21	5143	5031	5106	5031
q22	1096	1039	1040	1039
Total cold run time: 67958 ms
Total hot run time: 58740 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196425 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 613d908944ee87510d271e9f543fe4f54f66785f, data reload: false

query1	1288	900	901	900
query2	6360	1921	1873	1873
query3	10913	4370	4273	4273
query4	61529	29687	23534	23534
query5	5222	462	449	449
query6	411	176	196	176
query7	5441	301	300	300
query8	296	223	219	219
query9	8319	2546	2540	2540
query10	466	282	256	256
query11	17476	15290	15665	15290
query12	170	107	111	107
query13	1410	440	433	433
query14	10438	7166	7109	7109
query15	198	186	173	173
query16	7159	484	489	484
query17	1098	559	560	559
query18	1861	310	301	301
query19	207	155	164	155
query20	115	108	108	108
query21	206	102	101	101
query22	4689	4543	4688	4543
query23	34793	33714	33878	33714
query24	6121	3005	2897	2897
query25	549	429	424	424
query26	657	174	176	174
query27	2102	361	357	357
query28	4045	2148	2093	2093
query29	713	447	454	447
query30	234	152	158	152
query31	998	835	833	833
query32	70	57	63	57
query33	474	311	307	307
query34	963	499	527	499
query35	867	753	738	738
query36	1052	960	964	960
query37	110	69	74	69
query38	4033	3963	4007	3963
query39	1649	1489	1458	1458
query40	200	103	103	103
query41	50	50	50	50
query42	118	109	108	108
query43	523	485	490	485
query44	1198	809	815	809
query45	203	178	184	178
query46	1159	761	777	761
query47	2029	1906	1919	1906
query48	439	337	339	337
query49	737	394	380	380
query50	849	438	438	438
query51	7430	7167	7264	7167
query52	104	91	92	91
query53	257	181	182	181
query54	575	462	467	462
query55	85	77	81	77
query56	282	261	247	247
query57	1334	1225	1200	1200
query58	213	206	207	206
query59	3125	3057	3037	3037
query60	289	278	273	273
query61	107	109	110	109
query62	820	717	712	712
query63	216	190	190	190
query64	1440	658	629	629
query65	3483	3189	3179	3179
query66	708	292	306	292
query67	15777	15475	15448	15448
query68	4142	569	582	569
query69	434	253	257	253
query70	1182	1077	1129	1077
query71	348	270	259	259
query72	6314	4073	4024	4024
query73	760	353	354	353
query74	10285	8962	9352	8962
query75	3400	2663	2664	2663
query76	1994	1066	1058	1058
query77	482	275	277	275
query78	10680	9583	9542	9542
query79	1507	595	591	591
query80	879	425	413	413
query81	484	218	225	218
query82	1257	86	88	86
query83	242	146	151	146
query84	287	83	75	75
query85	884	332	297	297
query86	334	284	300	284
query87	4479	4241	4239	4239
query88	3709	2392	2382	2382
query89	423	300	290	290
query90	2025	187	188	187
query91	141	109	116	109
query92	64	52	55	52
query93	1989	547	540	540
query94	735	298	305	298
query95	340	255	258	255
query96	610	280	281	280
query97	3267	3118	3096	3096
query98	209	203	199	199
query99	1618	1314	1273	1273
Total cold run time: 313956 ms
Total hot run time: 196425 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.94 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 613d908944ee87510d271e9f543fe4f54f66785f, data reload: false

query1	0.04	0.03	0.02
query2	0.07	0.03	0.03
query3	0.23	0.06	0.06
query4	1.62	0.10	0.10
query5	0.52	0.51	0.52
query6	1.13	0.73	0.73
query7	0.02	0.01	0.01
query8	0.04	0.03	0.04
query9	0.57	0.48	0.50
query10	0.55	0.56	0.55
query11	0.14	0.13	0.11
query12	0.14	0.11	0.10
query13	0.60	0.59	0.59
query14	0.78	0.80	0.80
query15	0.84	0.81	0.82
query16	0.36	0.39	0.38
query17	1.02	0.99	1.06
query18	0.22	0.22	0.22
query19	1.97	1.85	1.85
query20	0.02	0.01	0.01
query21	15.40	0.57	0.56
query22	2.23	1.59	1.54
query23	17.27	0.86	0.79
query24	3.60	1.03	2.05
query25	0.33	0.20	0.12
query26	0.52	0.13	0.15
query27	0.05	0.05	0.04
query28	8.99	0.55	0.53
query29	12.58	3.24	3.17
query30	0.24	0.06	0.05
query31	2.87	0.40	0.39
query32	3.23	0.45	0.46
query33	2.95	3.01	3.01
query34	17.21	4.48	4.50
query35	4.55	4.59	4.49
query36	0.64	0.47	0.48
query37	0.08	0.06	0.06
query38	0.05	0.03	0.03
query39	0.03	0.02	0.02
query40	0.16	0.13	0.13
query41	0.07	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 104.01 s
Total hot run time: 29.94 s

@morrySnow morrySnow merged commit 8b59672 into apache:branch-3.1 Jun 23, 2025
21 of 22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants