Skip to content

Conversation

@eldenmoon
Copy link
Member

@eldenmoon eldenmoon commented Jun 24, 2025

The null bitmap of the related json path should be inited

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@eldenmoon eldenmoon changed the title [Fix](JsonReader) Fix the issue where the null bitmap of the JSON rea… [Fix](JsonReader) Fix the issue where the null bitmap of the JSON reader was not initialized when the JSON path is specified as '$.’ Jun 24, 2025
@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

…der was not initialized when the JSON path is specified as '$.’.
@eldenmoon
Copy link
Member Author

run buildall

@eldenmoon eldenmoon added dev/2.1.x dev/3.0.x usercase Important user case type label dev/3.1.x labels Jun 24, 2025
@doris-robot
Copy link

TPC-H: Total hot run time: 34038 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7fe7c7ab1ce1520aab69e6c3cf6da0ec72e2cb4e, data reload: false

------ Round 1 ----------------------------------
q1	17597	5302	5008	5008
q2	1941	307	192	192
q3	10292	1301	728	728
q4	10226	1020	550	550
q5	7530	2270	2473	2270
q6	183	161	134	134
q7	899	759	617	617
q8	9342	1340	1191	1191
q9	6812	5079	5028	5028
q10	6895	2394	1960	1960
q11	488	297	285	285
q12	347	351	220	220
q13	17770	3686	3153	3153
q14	235	222	217	217
q15	552	475	476	475
q16	431	425	379	379
q17	597	880	370	370
q18	8000	7346	7193	7193
q19	1478	959	543	543
q20	349	336	230	230
q21	3687	3256	2341	2341
q22	1032	1029	954	954
Total cold run time: 106683 ms
Total hot run time: 34038 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5118	5097	5135	5097
q2	247	322	222	222
q3	2214	2845	2351	2351
q4	1342	1813	1331	1331
q5	4265	4424	4488	4424
q6	220	173	128	128
q7	1981	1900	1791	1791
q8	2582	2581	2569	2569
q9	7170	7179	7110	7110
q10	3046	3257	2818	2818
q11	581	507	490	490
q12	661	768	631	631
q13	3542	3938	3329	3329
q14	280	298	282	282
q15	533	495	491	491
q16	444	513	450	450
q17	1187	1554	1332	1332
q18	7310	7231	7126	7126
q19	760	761	805	761
q20	1913	1998	1802	1802
q21	4834	4393	4362	4362
q22	1062	1056	988	988
Total cold run time: 51292 ms
Total hot run time: 49885 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 185787 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7fe7c7ab1ce1520aab69e6c3cf6da0ec72e2cb4e, data reload: false

query1	1020	391	412	391
query2	6530	1896	1865	1865
query3	6739	229	221	221
query4	26223	23492	23066	23066
query5	4325	637	467	467
query6	291	209	191	191
query7	4621	510	287	287
query8	264	223	204	204
query9	8613	2581	2594	2581
query10	472	333	259	259
query11	15379	15132	14886	14886
query12	160	107	105	105
query13	1654	541	393	393
query14	9507	6172	6345	6172
query15	204	195	171	171
query16	7453	621	493	493
query17	1236	734	638	638
query18	2012	407	290	290
query19	189	192	159	159
query20	120	115	112	112
query21	212	124	103	103
query22	4176	4276	4086	4086
query23	34155	33155	33071	33071
query24	8464	2369	2431	2369
query25	538	458	381	381
query26	1226	265	149	149
query27	2754	519	342	342
query28	4282	2125	2104	2104
query29	754	540	434	434
query30	296	226	218	218
query31	942	845	775	775
query32	73	66	61	61
query33	564	369	316	316
query34	820	863	524	524
query35	765	834	747	747
query36	947	974	883	883
query37	113	98	82	82
query38	4202	4141	4023	4023
query39	1494	1496	1413	1413
query40	219	116	105	105
query41	63	58	66	58
query42	125	110	110	110
query43	515	530	497	497
query44	1316	824	832	824
query45	178	176	166	166
query46	844	1024	626	626
query47	1767	1819	1733	1733
query48	400	436	329	329
query49	736	491	392	392
query50	638	678	409	409
query51	4132	4149	4167	4149
query52	117	106	102	102
query53	227	251	189	189
query54	582	572	499	499
query55	93	97	83	83
query56	313	321	294	294
query57	1218	1185	1118	1118
query58	276	268	270	268
query59	2676	2752	2652	2652
query60	351	343	331	331
query61	162	147	149	147
query62	814	724	682	682
query63	223	218	185	185
query64	4309	1012	679	679
query65	4261	4195	4183	4183
query66	1106	418	322	322
query67	15688	15589	15323	15323
query68	7853	896	540	540
query69	465	307	271	271
query70	1178	1123	1113	1113
query71	457	330	295	295
query72	5324	4707	4619	4619
query73	679	569	358	358
query74	8887	8861	8893	8861
query75	3864	3196	2671	2671
query76	3710	1196	753	753
query77	788	385	296	296
query78	10047	10168	9432	9432
query79	2337	872	583	583
query80	645	522	452	452
query81	502	262	227	227
query82	187	126	101	101
query83	256	269	234	234
query84	248	103	85	85
query85	759	351	314	314
query86	378	309	297	297
query87	4410	4544	4273	4273
query88	3619	2297	2283	2283
query89	411	319	289	289
query90	1960	215	221	215
query91	141	139	111	111
query92	85	62	61	61
query93	2182	946	589	589
query94	633	411	295	295
query95	373	296	287	287
query96	487	570	284	284
query97	2641	2740	2632	2632
query98	231	214	204	204
query99	1324	1431	1275	1275
Total cold run time: 274199 ms
Total hot run time: 185787 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.62 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7fe7c7ab1ce1520aab69e6c3cf6da0ec72e2cb4e, data reload: false

query1	0.03	0.04	0.02
query2	0.08	0.03	0.04
query3	0.25	0.06	0.07
query4	1.64	0.10	0.10
query5	0.44	0.44	0.41
query6	1.16	0.66	0.68
query7	0.03	0.02	0.02
query8	0.05	0.03	0.04
query9	0.57	0.51	0.52
query10	0.56	0.58	0.57
query11	0.15	0.12	0.11
query12	0.15	0.12	0.11
query13	0.63	0.61	0.61
query14	0.80	0.79	0.82
query15	0.91	0.87	0.89
query16	0.41	0.39	0.39
query17	1.07	1.10	1.09
query18	0.22	0.21	0.22
query19	2.02	1.94	1.92
query20	0.02	0.02	0.01
query21	15.36	0.89	0.53
query22	0.75	1.21	0.68
query23	14.91	1.38	0.66
query24	6.86	1.11	0.73
query25	0.47	0.29	0.06
query26	0.54	0.17	0.15
query27	0.06	0.06	0.06
query28	10.22	0.94	0.45
query29	12.62	4.06	3.41
query30	0.26	0.10	0.06
query31	2.83	0.61	0.39
query32	3.25	0.56	0.48
query33	3.07	3.11	3.16
query34	16.07	5.39	4.72
query35	4.83	4.88	4.81
query36	0.66	0.50	0.47
query37	0.09	0.07	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.02
query40	0.17	0.14	0.15
query41	0.08	0.03	0.02
query42	0.03	0.02	0.03
query43	0.03	0.03	0.04
Total cold run time: 104.43 s
Total hot run time: 29.62 s

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 0.00% (0/1) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 56.93% (15338/26941)
Line Coverage 46.01% (139031/302175)
Region Coverage 45.36% (70432/155260)
Branch Coverage 40.15% (37213/92678)

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/1) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 62.30% (16512/26503)
Line Coverage 52.11% (157364/301987)
Region Coverage 49.63% (90435/182226)
Branch Coverage 43.25% (45607/105450)

Copy link
Contributor

@csun5285 csun5285 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

Copy link
Member

@airborne12 airborne12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 25, 2025
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@eldenmoon eldenmoon merged commit f9e6543 into apache:master Jun 25, 2025
28 of 31 checks passed
github-actions bot pushed a commit that referenced this pull request Jun 25, 2025
…der was not initialized when the JSON path is specified as '$.’ (#52211)

The null bitmap of the related json path should be inited
github-actions bot pushed a commit that referenced this pull request Jun 25, 2025
…der was not initialized when the JSON path is specified as '$.’ (#52211)

The null bitmap of the related json path should be inited
github-actions bot pushed a commit that referenced this pull request Jun 25, 2025
…der was not initialized when the JSON path is specified as '$.’ (#52211)

The null bitmap of the related json path should be inited
@eldenmoon eldenmoon added the p0_b label Jun 25, 2025
@eldenmoon eldenmoon deleted the fix-jr-dollar branch June 25, 2025 08:23
dataroaring pushed a commit that referenced this pull request Jun 26, 2025
…the JSON reader was not initialized when the JSON path is specified as '$.’ #52211 (#52267)

Cherry-picked from #52211

Co-authored-by: lihangyu <lihangyu@selectdb.com>
morrySnow pushed a commit that referenced this pull request Jun 26, 2025
…the JSON reader was not initialized when the JSON path is specified as '$.’ #52211 (#52269)

Cherry-picked from #52211

Co-authored-by: lihangyu <lihangyu@selectdb.com>
yiguolei pushed a commit that referenced this pull request Jun 28, 2025
…the JSON reader was not initialized when the JSON path is specified as '$.’ #52211 (#52268)

Cherry-picked from #52211

Co-authored-by: lihangyu <lihangyu@selectdb.com>
koarz pushed a commit to koarz/doris that referenced this pull request Jul 3, 2025
…the JSON reader was not initialized when the JSON path is specified as '$.’ apache#52211 (apache#52267)

Cherry-picked from apache#52211

Co-authored-by: lihangyu <lihangyu@selectdb.com>
eldenmoon added a commit to eldenmoon/incubator-doris that referenced this pull request Aug 8, 2025
…the JSON reader was not initialized when the JSON path is specified as '$.’ apache#52211 (apache#52267)

Cherry-picked from apache#52211

Co-authored-by: lihangyu <lihangyu@selectdb.com>
BiteTheDDDDt pushed a commit to BiteTheDDDDt/incubator-doris that referenced this pull request Sep 1, 2025
…the JSON reader was not initialized when the JSON path is specified as '$.’ apache#52211 apache#52267 (apache#4255)

Cherry-picked from apache#52267
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.11-merged dev/3.0.7-merged dev/3.1.0-merged p0_b reviewed usercase Important user case type label

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants