[Fix](Variant) fix serialize with json key contains `.` as name #51857

eldenmoon · 2025-06-18T06:27:21Z

What problem does this PR solve?

get_path with lost object nesting level information when calling ColumnObject::get when VariantMap is std::map<std::string, Field>, so change VariantMap to std::<PathInData, field> to maintain nesting level
serialize/deserialize should also serialize PathInData to ColumnPathInfo to maintain nesting level

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

1. get_path with lost object nesting level information when calling ColumnObject::get when VariantMap is std::map<std::string, Field>, so change VariantMap to std::<PathInData, field> to maintain nesting level 2. serialize/deserialize should also serialize PathInData to ColumnPathInfo to maintain nesting level

hello-stephen · 2025-06-18T06:27:25Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

eldenmoon · 2025-06-18T06:27:53Z

run buildall

eldenmoon · 2025-06-18T06:44:01Z

run buildall

doris-robot · 2025-06-18T06:53:33Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	83.33% (1120/1344)
Line Coverage	66.81% (19313/28909)
Region Coverage	66.56% (9578/14390)
Branch Coverage	56.46% (5203/9216)

hello-stephen · 2025-06-18T08:22:30Z

FE UT Coverage Report

Increment line coverage `` 🎉
Increment coverage report
Complete coverage report

doris-robot · 2025-06-18T08:27:50Z

TPC-H: Total hot run time: 34223 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5b532ce0993cc17d48e2f3391b93b6b701860257, data reload: false

------ Round 1 ----------------------------------
q1	17605	5222	4936	4936
q2	1932	293	179	179
q3	10338	1330	757	757
q4	10216	1030	543	543
q5	7563	2390	2311	2311
q6	182	163	131	131
q7	917	743	645	645
q8	9335	1302	1170	1170
q9	6742	5163	5210	5163
q10	6907	2375	1989	1989
q11	486	293	276	276
q12	347	356	226	226
q13	17769	3634	3097	3097
q14	232	227	214	214
q15	562	502	509	502
q16	430	434	387	387
q17	599	863	353	353
q18	7794	7163	7162	7162
q19	2023	955	580	580
q20	333	337	229	229
q21	3783	3200	2421	2421
q22	1038	1039	952	952
Total cold run time: 107133 ms
Total hot run time: 34223 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6034	5393	5105	5105
q2	246	327	225	225
q3	2153	2708	2309	2309
q4	1352	1806	1413	1413
q5	4209	4268	4534	4268
q6	210	170	128	128
q7	2003	1931	1765	1765
q8	2615	2545	2512	2512
q9	7095	7201	7133	7133
q10	3124	3333	2869	2869
q11	587	504	493	493
q12	647	761	618	618
q13	3490	3915	3290	3290
q14	293	305	268	268
q15	549	499	490	490
q16	440	477	437	437
q17	1163	1536	1395	1395
q18	7801	7654	7369	7369
q19	858	950	1161	950
q20	2023	2046	1961	1961
q21	4977	4629	4334	4334
q22	1098	1053	981	981
Total cold run time: 52967 ms
Total hot run time: 50313 ms

doris-robot · 2025-06-18T08:39:23Z

TPC-DS: Total hot run time: 185558 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5b532ce0993cc17d48e2f3391b93b6b701860257, data reload: false

query1	1012	392	392	392
query2	6544	1840	1811	1811
query3	6748	217	222	217
query4	26233	23525	22874	22874
query5	4312	631	472	472
query6	314	208	196	196
query7	4619	484	295	295
query8	270	229	216	216
query9	8598	2672	2695	2672
query10	456	339	261	261
query11	15612	14950	15023	14950
query12	154	111	116	111
query13	1687	522	432	432
query14	9411	6343	6112	6112
query15	207	195	169	169
query16	7257	639	477	477
query17	1198	715	581	581
query18	1990	428	311	311
query19	193	193	178	178
query20	122	122	117	117
query21	215	130	115	115
query22	4115	4268	4142	4142
query23	34055	33087	32941	32941
query24	8425	2372	2401	2372
query25	549	457	398	398
query26	1234	271	153	153
query27	2751	512	346	346
query28	4268	2162	2136	2136
query29	774	531	429	429
query30	285	217	191	191
query31	952	854	774	774
query32	69	69	68	68
query33	557	365	318	318
query34	801	847	541	541
query35	779	816	721	721
query36	924	952	840	840
query37	107	100	81	81
query38	4150	4239	4054	4054
query39	1477	1436	1398	1398
query40	216	124	101	101
query41	64	62	57	57
query42	127	116	109	109
query43	487	499	481	481
query44	1360	838	843	838
query45	184	176	162	162
query46	836	1016	627	627
query47	1766	1794	1708	1708
query48	384	414	316	316
query49	735	474	417	417
query50	641	677	412	412
query51	4151	4245	4142	4142
query52	112	107	101	101
query53	221	258	188	188
query54	584	577	509	509
query55	88	92	84	84
query56	316	310	305	305
query57	1206	1204	1104	1104
query58	265	258	264	258
query59	2517	2682	2558	2558
query60	331	349	303	303
query61	129	129	130	129
query62	785	722	665	665
query63	224	185	189	185
query64	4338	992	674	674
query65	4256	4141	4223	4141
query66	1146	414	315	315
query67	15570	15721	15328	15328
query68	8060	901	540	540
query69	467	343	315	315
query70	1225	1070	1053	1053
query71	464	321	298	298
query72	5605	4738	4782	4738
query73	722	622	360	360
query74	9134	8872	8897	8872
query75	3803	3190	2707	2707
query76	3671	1205	763	763
query77	786	375	306	306
query78	10012	10268	9423	9423
query79	1982	875	595	595
query80	604	507	451	451
query81	480	266	235	235
query82	458	123	97	97
query83	260	251	231	231
query84	250	108	94	94
query85	766	360	309	309
query86	380	300	281	281
query87	4370	4352	4298	4298
query88	3574	2300	2262	2262
query89	392	362	287	287
query90	1932	213	213	213
query91	142	146	114	114
query92	78	59	56	56
query93	1509	981	593	593
query94	654	410	307	307
query95	385	296	279	279
query96	494	570	278	278
query97	2685	2764	2611	2611
query98	227	215	202	202
query99	1419	1401	1311	1311
Total cold run time: 273453 ms
Total hot run time: 185558 ms

doris-robot · 2025-06-18T10:57:50Z

BE UT Coverage Report

Increment line coverage 22.22% (2/9) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	56.35% (15051/26712)
Line Coverage	45.11% (134582/298370)
Region Coverage	44.24% (67677/152983)
Branch Coverage	38.82% (34720/89440)

hello-stephen · 2025-06-18T11:42:47Z

BE Regression && UT Coverage Report

Increment line coverage 22.22% (2/9) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	61.07% (16055/26289)
Line Coverage	50.59% (150872/298212)
Region Coverage	47.91% (86217/179952)
Branch Coverage	41.44% (42372/102244)

csun5285

LGTM

github-actions · 2025-06-19T06:27:56Z

PR approved by anyone and no changes requested.

airborne12

LGTM

github-actions · 2025-06-19T06:43:30Z

PR approved by at least one committer and no changes requested.

…ains `.` as name (#51864) cherry-pick from #51857

…ains `.` as name (#51930) cherry-pick from #51857

…ains `.` as name (apache#51930) cherry-pick from apache#51857

eldenmoon added 2 commits June 18, 2025 14:25

fix

5899f17

eldenmoon force-pushed the fix-dot-json-v branch from e762ad2 to 807173a Compare June 18, 2025 06:29

fix

5b532ce

eldenmoon force-pushed the fix-dot-json-v branch from 807173a to 5b532ce Compare June 18, 2025 06:43

eldenmoon added dev/2.1.x dev/3.0.x labels Jun 18, 2025

eldenmoon mentioned this pull request Jun 18, 2025

brach-2.1 cherry-pick [Fix](Variant) fix serialize with json key contains . as name #51864

Merged

csun5285 approved these changes Jun 19, 2025

View reviewed changes

github-actions bot added the reviewed label Jun 19, 2025

eldenmoon added the usercase Important user case type label label Jun 19, 2025

airborne12 approved these changes Jun 19, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 19, 2025

eldenmoon mentioned this pull request Jun 19, 2025

brach-3.0 cherry-pick [Fix](Variant) fix serialize with json key contains . as name #51930

Merged

eldenmoon merged commit 40ddeab into apache:master Jun 19, 2025
28 of 32 checks passed

github-actions bot added dev/3.0.x-conflict dev/2.1.x-conflict labels Jun 19, 2025

yiguolei pushed a commit that referenced this pull request Jun 20, 2025

brach-2.1 cherry-pick [Fix](Variant) fix serialize with json key cont…

a75760d

…ains `.` as name (#51864) cherry-pick from #51857

yiguolei added dev/2.1.11-merged and removed dev/2.1.x dev/2.1.x-conflict labels Jun 20, 2025

morrySnow added the not-merge/3.1 label Jun 24, 2025

dataroaring pushed a commit that referenced this pull request Jun 26, 2025

brach-3.0 cherry-pick [Fix](Variant) fix serialize with json key cont…

1c2ba48

…ains `.` as name (#51930) cherry-pick from #51857

dataroaring added dev/3.0.7-merged and removed dev/3.0.x dev/3.0.x-conflict labels Jun 26, 2025

koarz pushed a commit to koarz/doris that referenced this pull request Jul 3, 2025

brach-3.0 cherry-pick [Fix](Variant) fix serialize with json key cont…

c3db552

…ains `.` as name (apache#51930) cherry-pick from apache#51857

[Fix](Variant) fix serialize with json key contains . as name #51857

[Fix](Variant) fix serialize with json key contains . as name #51857

Uh oh!

Conversation

eldenmoon commented Jun 18, 2025

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Jun 18, 2025

Uh oh!

eldenmoon commented Jun 18, 2025

Uh oh!

eldenmoon commented Jun 18, 2025

Uh oh!

doris-robot commented Jun 18, 2025

Cloud UT Coverage Report

Uh oh!

hello-stephen commented Jun 18, 2025

FE UT Coverage Report

Uh oh!

doris-robot commented Jun 18, 2025

Uh oh!

doris-robot commented Jun 18, 2025

Uh oh!

doris-robot commented Jun 18, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented Jun 18, 2025

BE Regression && UT Coverage Report

Uh oh!

csun5285 left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jun 19, 2025

Uh oh!

airborne12 left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jun 19, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

[Fix](Variant) fix serialize with json key contains `.` as name #51857

[Fix](Variant) fix serialize with json key contains `.` as name #51857