Skip to content

[fix](json) Replace invalid JSONB with default JSONB null value #59007

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
eldenmoon merged 2 commits into from
Dec 15, 2025
Merged

[fix](json) Replace invalid JSONB with default JSONB null value #59007

eldenmoon merged 2 commits into from
Dec 15, 2025

Conversation

@csun5285
Copy link
Contributor

@csun5285 csun5285 commented Dec 12, 2025 •
edited
Loading

What problem does this PR solve?

This PR fixes the issue that was supposed to be resolved by #58656
. We need to address it as soon as possible, so I am submitting this PR.

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

hello-stephen commented Dec 12, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@csun5285
Copy link
Contributor Author

csun5285 commented Dec 12, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Dec 12, 2025

TPC-H: Total hot run time: 35302 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e4395a67a0d8a486311ab35ad01ee1e8fa29f968, data reload: false

------ Round 1 ----------------------------------
q1	17591	4189	4057	4057
q2	2024	365	242	242
q3	10177	1357	730	730
q4	10327	908	325	325
q5	8810	2168	1973	1973
q6	227	165	135	135
q7	1009	849	707	707
q8	9358	1468	1155	1155
q9	7197	5363	5391	5363
q10	6870	2429	1970	1970
q11	506	333	296	296
q12	696	724	577	577
q13	17798	3681	3043	3043
q14	286	304	280	280
q15	597	519	513	513
q16	722	687	645	645
q17	700	796	561	561
q18	7612	7423	7147	7147
q19	1102	985	612	612
q20	403	366	246	246
q21	4283	4009	3737	3737
q22	1059	1005	988	988
Total cold run time: 109354 ms
Total hot run time: 35302 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4346	4182	4080	4080
q2	323	399	318	318
q3	2182	2696	2338	2338
q4	1334	1774	1327	1327
q5	4499	4864	4586	4586
q6	228	174	134	134
q7	2053	1986	1720	1720
q8	2693	2571	2516	2516
q9	7693	7522	7587	7522
q10	3134	3303	2845	2845
q11	753	513	496	496
q12	703	741	604	604
q13	3642	4075	3345	3345
q14	299	291	320	291
q15	551	536	508	508
q16	640	679	626	626
q17	1203	1684	1484	1484
q18	7914	7691	7434	7434
q19	835	809	832	809
q20	1891	2010	1840	1840
q21	4693	4244	4143	4143
q22	1111	1032	981	981
Total cold run time: 52720 ms
Total hot run time: 49947 ms

@doris-robot
Copy link

doris-robot commented Dec 12, 2025

TPC-DS: Total hot run time: 176935 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e4395a67a0d8a486311ab35ad01ee1e8fa29f968, data reload: false

query5	4796	620	459	459
query6	330	248	219	219
query7	4237	476	270	270
query8	305	265	251	251
query9	8796	2573	2575	2573
query10	503	383	335	335
query11	15305	14726	15040	14726
query12	176	119	117	117
query13	1271	527	405	405
query14	6269	3231	2996	2996
query14_1	2853	2929	2910	2910
query15	208	201	179	179
query16	835	490	462	462
query17	1091	686	587	587
query18	2485	447	343	343
query19	225	224	203	203
query20	117	113	109	109
query21	226	139	117	117
query22	4089	4050	3871	3871
query23	16542	16488	16060	16060
query23_1	16166	16214	16139	16139
query24	7389	1669	1244	1244
query24_1	1232	1254	1253	1253
query25	594	500	461	461
query26	1244	270	165	165
query27	2756	464	313	313
query28	4505	2152	2125	2125
query29	825	569	485	485
query30	317	256	218	218
query31	866	715	628	628
query32	80	73	72	72
query33	571	349	308	308
query34	986	887	537	537
query35	804	824	730	730
query36	856	919	827	827
query37	133	98	78	78
query38	2884	2861	2822	2822
query39	758	739	718	718
query39_1	703	721	703	703
query40	231	139	124	124
query41	74	69	66	66
query42	111	105	109	105
query43	435	435	412	412
query44	1332	756	741	741
query45	199	195	186	186
query46	882	989	628	628
query47	1652	1728	1649	1649
query48	331	342	253	253
query49	642	456	382	382
query50	677	302	224	224
query51	3849	4019	3868	3868
query52	112	111	101	101
query53	327	351	295	295
query54	301	290	275	275
query55	79	80	74	74
query56	312	327	330	327
query57	1135	1144	1081	1081
query58	279	275	259	259
query59	2394	2570	2416	2416
query60	326	338	313	313
query61	213	158	162	158
query62	685	686	611	611
query63	327	294	297	294
query64	4930	1309	1028	1028
query65	4031	3952	3939	3939
query66	1441	460	311	311
query67	15046	14894	14854	14854
query68	8378	995	733	733
query69	504	352	316	316
query70	1077	1008	990	990
query71	383	309	282	282
query72	6105	2776	5336	2776
query73	773	747	308	308
query74	8878	8808	8746	8746
query75	3212	3160	2798	2798
query76	3987	1138	757	757
query77	626	393	303	303
query78	9545	9703	8889	8889
query79	1540	856	618	618
query80	735	634	564	564
query81	525	270	239	239
query82	201	149	108	108
query83	261	253	236	236
query84	257	123	100	100
query85	892	524	474	474
query86	384	308	287	287
query87	3029	3134	2950	2950
query88	3290	2268	2238	2238
query89	468	428	388	388
query90	2191	164	151	151
query91	176	168	146	146
query92	83	66	66	66
query93	2209	922	556	556
query94	474	298	283	283
query95	586	377	305	305
query96	584	476	204	204
query97	2294	2292	2257	2257
query98	211	200	185	185
query99	1341	1313	1226	1226
Total cold run time: 261556 ms
Total hot run time: 176935 ms

@doris-robot
Copy link

doris-robot commented Dec 12, 2025

ClickBench: Total hot run time: 27.35 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e4395a67a0d8a486311ab35ad01ee1e8fa29f968, data reload: false

query1	0.05	0.04	0.04
query2	0.10	0.05	0.04
query3	0.26	0.09	0.09
query4	1.61	0.11	0.11
query5	0.28	0.26	0.25
query6	1.17	0.64	0.63
query7	0.03	0.02	0.03
query8	0.05	0.04	0.04
query9	0.57	0.50	0.50
query10	0.54	0.56	0.56
query11	0.15	0.10	0.11
query12	0.16	0.11	0.12
query13	0.62	0.60	0.61
query14	0.99	0.98	0.96
query15	0.82	0.78	0.79
query16	0.39	0.39	0.39
query17	1.00	1.06	1.06
query18	0.24	0.21	0.22
query19	1.94	1.87	1.88
query20	0.02	0.02	0.01
query21	15.44	0.29	0.14
query22	4.78	0.05	0.04
query23	16.06	0.28	0.10
query24	0.97	0.29	0.18
query25	0.10	0.05	0.05
query26	0.14	0.13	0.14
query27	0.06	0.04	0.04
query28	2.98	1.22	1.03
query29	12.61	3.93	3.27
query30	0.28	0.14	0.12
query31	2.82	0.64	0.39
query32	3.24	0.55	0.48
query33	2.98	3.03	3.05
query34	16.83	5.19	4.57
query35	4.52	4.54	4.60
query36	0.65	0.50	0.49
query37	0.10	0.06	0.07
query38	0.08	0.04	0.04
query39	0.04	0.04	0.04
query40	0.17	0.14	0.14
query41	0.08	0.03	0.03
query42	0.05	0.03	0.03
query43	0.04	0.04	0.04
Total cold run time: 96.01 s
Total hot run time: 27.35 s

@csun5285
Copy link
Contributor Author

csun5285 commented Dec 12, 2025

run buildall

@yiguolei
Copy link
Contributor

yiguolei commented Dec 12, 2025

run buildall

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 12, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Dec 12, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Dec 12, 2025

PR approved by anyone and no changes requested.

@hello-stephen
Copy link
Contributor

hello-stephen commented Dec 12, 2025

BE UT Coverage Report

Increment line coverage 80.00% (24/30) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 53.42% (18823/35237)
Line Coverage 39.16% (174005/444388)
Region Coverage 33.80% (134736/398609)
Branch Coverage 34.74% (57976/166886)

@hello-stephen
Copy link
Contributor

hello-stephen commented Dec 12, 2025

BE Regression && UT Coverage Report

Increment line coverage 80.00% (24/30) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 73.19% (25277/34534)
Line Coverage 60.36% (267928/443883)
Region Coverage 55.96% (225697/403325)
Branch Coverage 57.16% (95892/167759)

mrhhsg
mrhhsg approved these changes Dec 13, 2025
View reviewed changes
Copy link
Member

@mrhhsg mrhhsg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hello-stephen
Copy link
Contributor

hello-stephen commented Dec 13, 2025

BE Regression && UT Coverage Report

Increment line coverage 80.00% (24/30) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 73.19% (25277/34534)
Line Coverage 60.36% (267928/443883)
Region Coverage 55.96% (225697/403325)
Branch Coverage 57.16% (95892/167759)

eldenmoon
eldenmoon approved these changes Dec 15, 2025
View reviewed changes
Copy link
Member

@eldenmoon eldenmoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@eldenmoon eldenmoon merged commit d63f8db into apache:master Dec 15, 2025
32 of 34 checks passed
github-actions bot pushed a commit that referenced this pull request Dec 15, 2025
@csun5285
[fix](json) Replace invalid JSONB with default JSONB null value (#59007)
2a23951 
This PR fixes the issue that was supposed to be resolved by
#58656
. We need to address it as soon as possible, so I am submitting this PR.
@github-actions github-actions bot mentioned this pull request Dec 15, 2025
Merged
csun5285 added a commit to csun5285/doris that referenced this pull request Dec 16, 2025
@csun5285
[fix](json) Replace invalid JSONB with default JSONB null value (apac…
4b3d1b5 
…he#59007)

This PR fixes the issue that was supposed to be resolved by
apache#58656
. We need to address it as soon as possible, so I am submitting this PR.
@csun5285 csun5285 mentioned this pull request Dec 16, 2025
Merged
yiguolei pushed a commit that referenced this pull request Dec 16, 2025
@github-actions @csun5285
branch-4.0: [fix](json) Replace invalid JSONB with default JSONB null…
6518277 
… value #59007 (#59022)

Cherry-picked from #59007

Co-authored-by: Sun Chenyang <sunchenyang@selectdb.com>
morrySnow pushed a commit that referenced this pull request Dec 17, 2025
@csun5285
branch-3.1: [fix](json) Replace invalid JSONB with default JSONB null…
1922fab 
… value #59007 (#59067)

pick from master #59007
@yiguolei yiguolei mentioned this pull request Jan 23, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mrhhsg mrhhsg mrhhsg approved these changes

@yiguolei yiguolei yiguolei approved these changes

@eldenmoon eldenmoon eldenmoon approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.x dev/3.0.x dev/3.1.4-merged dev/4.0.3-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@csun5285 @hello-stephen @doris-robot @yiguolei @mrhhsg @eldenmoon @morrySnow