Skip to content

[fix](csv) fix failed to get prop from connection context #53159

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morningman merged 1 commit into from
Jul 15, 2025
Merged

[fix](csv) fix failed to get prop from connection context #53159

morningman merged 1 commit into from
Jul 15, 2025

Conversation

@morningman
Copy link
Contributor

@morningman morningman commented Jul 13, 2025 •
edited
Loading

What problem does this PR solve?

Related PR: #50882

Problem Summary:

The enable_text_validate_utf8 is got from session variable, but sometimes the connection context
does not exist, so NPE will throw if we try to get session variable from connection context.
This PR fix it.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Jul 13, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morningman morningman mentioned this pull request Jul 13, 2025
Merged
@morningman
Copy link
Contributor Author

morningman commented Jul 13, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Jul 13, 2025

TPC-H: Total hot run time: 33014 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9dab3055e57359bec26da3e9416c8fc11c70e758, data reload: false

------ Round 1 ----------------------------------
q1	17580	5240	5009	5009
q2	1918	284	186	186
q3	10392	1259	740	740
q4	10305	1015	510	510
q5	8583	2421	2290	2290
q6	230	162	125	125
q7	927	731	603	603
q8	9306	1290	1097	1097
q9	7181	5120	5092	5092
q10	6933	2367	1952	1952
q11	478	291	271	271
q12	355	344	225	225
q13	17756	3731	3049	3049
q14	230	232	216	216
q15	547	492	486	486
q16	421	419	370	370
q17	602	868	357	357
q18	7420	7174	7051	7051
q19	1373	944	539	539
q20	341	333	209	209
q21	3702	3177	2340	2340
q22	356	332	297	297
Total cold run time: 106936 ms
Total hot run time: 33014 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5229	5345	5112	5112
q2	242	326	226	226
q3	2200	2716	2325	2325
q4	1343	1774	1330	1330
q5	4243	4427	4494	4427
q6	221	164	122	122
q7	2077	2032	1822	1822
q8	2747	2492	2575	2492
q9	7350	7057	7295	7057
q10	3133	3259	2893	2893
q11	594	531	495	495
q12	691	763	612	612
q13	3647	4166	3262	3262
q14	285	328	282	282
q15	511	460	464	460
q16	431	508	457	457
q17	1219	1595	1387	1387
q18	7938	7564	7757	7564
q19	810	835	981	835
q20	1939	1950	1811	1811
q21	4772	4396	4330	4330
q22	651	596	539	539
Total cold run time: 52273 ms
Total hot run time: 49840 ms

@doris-robot
Copy link

doris-robot commented Jul 13, 2025

TPC-DS: Total hot run time: 187408 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9dab3055e57359bec26da3e9416c8fc11c70e758, data reload: false

query1	977	401	405	401
query2	6512	1712	1699	1699
query3	6764	218	225	218
query4	26192	23652	23445	23445
query5	4355	564	420	420
query6	323	216	201	201
query7	4620	498	300	300
query8	261	223	212	212
query9	8595	2624	2605	2605
query10	496	314	253	253
query11	15665	15016	15009	15009
query12	169	107	107	107
query13	1646	517	391	391
query14	9309	6030	6044	6030
query15	202	192	167	167
query16	7666	429	246	246
query17	1342	717	550	550
query18	2003	405	292	292
query19	207	190	155	155
query20	123	113	111	111
query21	209	120	100	100
query22	4125	4166	4059	4059
query23	33988	32931	32987	32931
query24	8459	2397	2346	2346
query25	525	466	378	378
query26	1235	266	141	141
query27	2771	507	333	333
query28	4326	2125	2110	2110
query29	739	545	441	441
query30	284	231	230	230
query31	925	843	769	769
query32	73	60	62	60
query33	542	332	279	279
query34	788	840	513	513
query35	611	653	550	550
query36	933	991	915	915
query37	123	109	85	85
query38	4120	4146	4025	4025
query39	1503	1483	1448	1448
query40	214	112	104	104
query41	55	54	52	52
query42	116	110	112	110
query43	503	514	471	471
query44	1295	818	821	818
query45	174	169	162	162
query46	869	1024	635	635
query47	1792	1790	1713	1713
query48	382	409	301	301
query49	734	468	394	394
query50	654	682	428	428
query51	5569	5632	5405	5405
query52	104	103	97	97
query53	229	252	180	180
query54	584	568	501	501
query55	86	91	81	81
query56	297	303	274	274
query57	1169	1209	1145	1145
query58	259	258	259	258
query59	2546	2618	2531	2531
query60	318	333	317	317
query61	155	145	146	145
query62	773	731	652	652
query63	225	190	190	190
query64	4581	1170	824	824
query65	4291	4223	4191	4191
query66	1048	405	324	324
query67	16022	15372	15384	15372
query68	7914	892	535	535
query69	499	387	261	261
query70	1205	1115	1136	1115
query71	462	318	301	301
query72	5570	4746	4837	4746
query73	700	615	347	347
query74	8853	9193	8878	8878
query75	3881	3199	2693	2693
query76	3647	1154	756	756
query77	785	398	295	295
query78	11095	10913	10260	10260
query79	2268	802	567	567
query80	605	517	490	490
query81	500	261	227	227
query82	460	133	100	100
query83	252	249	236	236
query84	245	107	93	93
query85	795	375	313	313
query86	413	333	281	281
query87	4376	4499	4305	4305
query88	3943	2309	2317	2309
query89	381	325	287	287
query90	1893	209	210	209
query91	145	138	112	112
query92	74	60	54	54
query93	1851	993	596	596
query94	685	317	199	199
query95	373	299	284	284
query96	500	565	281	281
query97	2684	2796	2617	2617
query98	239	222	204	204
query99	1652	1425	1257	1257
Total cold run time: 277695 ms
Total hot run time: 187408 ms

@doris-robot
Copy link

doris-robot commented Jul 13, 2025

ClickBench: Total hot run time: 29.42 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9dab3055e57359bec26da3e9416c8fc11c70e758, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.05
query3	0.24	0.07	0.08
query4	1.62	0.11	0.11
query5	0.46	0.42	0.41
query6	1.18	0.67	0.66
query7	0.03	0.02	0.01
query8	0.05	0.03	0.03
query9	0.61	0.50	0.52
query10	0.57	0.57	0.55
query11	0.16	0.11	0.11
query12	0.15	0.12	0.11
query13	0.63	0.61	0.62
query14	0.81	0.84	0.80
query15	0.92	0.87	0.92
query16	0.38	0.38	0.41
query17	1.04	1.05	1.07
query18	0.23	0.22	0.21
query19	2.08	1.91	1.90
query20	0.02	0.01	0.02
query21	15.38	0.89	0.56
query22	0.76	1.07	0.65
query23	15.08	1.40	0.63
query24	7.05	1.66	0.59
query25	0.47	0.18	0.14
query26	0.61	0.17	0.14
query27	0.07	0.05	0.05
query28	9.40	0.89	0.44
query29	12.55	4.03	3.32
query30	0.25	0.10	0.07
query31	2.82	0.62	0.39
query32	3.24	0.56	0.49
query33	3.08	3.08	3.13
query34	16.07	5.49	4.78
query35	4.85	4.82	4.81
query36	0.70	0.51	0.49
query37	0.09	0.06	0.06
query38	0.04	0.04	0.04
query39	0.04	0.03	0.03
query40	0.17	0.15	0.13
query41	0.08	0.03	0.02
query42	0.03	0.03	0.02
query43	0.04	0.03	0.04
Total cold run time: 104.17 s
Total hot run time: 29.42 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Jul 13, 2025

FE UT Coverage Report

Increment line coverage 50.00% (4/8) 🎉
Increment coverage report
Complete coverage report

morrySnow pushed a commit that referenced this pull request Jul 14, 2025
@morningman @BePPPower
branch-3.1: [feat](refactor-param) refactor brokerLoad's code about f…
b0f0573 
…ileformat #50882 #53159 (#53145)

bp #50882 #53159

---------

Co-authored-by: Tiewei Fang <fangtiewei@selectdb.com>
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 14, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Jul 14, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Jul 14, 2025

PR approved by anyone and no changes requested.

@morningman morningman merged commit 8bce836 into apache:master Jul 15, 2025
31 of 33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@CalvinKirs CalvinKirs CalvinKirs approved these changes

+1 more reviewer

@suxiaogang223 suxiaogang223 suxiaogang223 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.1.0-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@morningman @Thearas @doris-robot @hello-stephen @CalvinKirs @suxiaogang223 @morrySnow