Skip to content

branch-3.0: [Fix](field) Fix potential memory leak and wrong binary reading about JsonbField (#50174) #52542

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 1 commit into from
Jul 2, 2025
Merged

branch-3.0: [Fix](field) Fix potential memory leak and wrong binary reading about JsonbField (#50174) #52542

dataroaring merged 1 commit into from
Jul 2, 2025

Conversation

@zclllyybb
Copy link
Contributor

@zclllyybb zclllyybb commented Jun 30, 2025

pick #50174

@zclllyybb zclllyybb requested a review from dataroaring as a code owner June 30, 2025 10:01
@zclllyybb
Copy link
Contributor Author

zclllyybb commented Jun 30, 2025

run buildall

@hello-stephen
Copy link
Contributor

hello-stephen commented Jun 30, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@doris-robot
Copy link

doris-robot commented Jun 30, 2025

TPC-H: Total hot run time: 40237 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3fdd2c71a8bbae49d8252e9f380d16fcdcbdf275, data reload: false

------ Round 1 ----------------------------------
q1	17802	6873	6726	6726
q2	2076	170	165	165
q3	10700	1089	1203	1089
q4	10436	829	715	715
q5	7737	2896	2860	2860
q6	226	141	137	137
q7	981	625	609	609
q8	9351	2005	2035	2005
q9	6701	6470	6506	6470
q10	7030	2312	2340	2312
q11	462	262	261	261
q12	393	215	215	215
q13	17795	3037	3012	3012
q14	233	208	215	208
q15	518	459	471	459
q16	468	368	371	368
q17	1010	570	558	558
q18	7492	6744	6801	6744
q19	1403	1063	1015	1015
q20	460	203	197	197
q21	3902	3211	3126	3126
q22	1055	986	992	986
Total cold run time: 108231 ms
Total hot run time: 40237 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6667	6639	6645	6639
q2	330	233	238	233
q3	2957	2805	2914	2805
q4	2007	1800	1761	1761
q5	5820	5777	5802	5777
q6	217	133	134	133
q7	2231	1821	1800	1800
q8	3423	3574	3566	3566
q9	9032	8841	9006	8841
q10	3584	3575	3536	3536
q11	603	511	506	506
q12	795	618	590	590
q13	9422	3187	3201	3187
q14	299	269	257	257
q15	511	479	465	465
q16	491	441	457	441
q17	1895	1623	1631	1623
q18	8325	7851	7735	7735
q19	1720	1627	1516	1516
q20	2138	1828	1831	1828
q21	5249	4999	4923	4923
q22	1132	1034	1040	1034
Total cold run time: 68848 ms
Total hot run time: 59196 ms

@doris-robot
Copy link

doris-robot commented Jun 30, 2025

TPC-DS: Total hot run time: 197421 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3fdd2c71a8bbae49d8252e9f380d16fcdcbdf275, data reload: false

query1	1295	912	898	898
query2	6354	2006	1926	1926
query3	10824	4257	4347	4257
query4	61632	29201	23522	23522
query5	5192	451	456	451
query6	404	167	172	167
query7	5425	306	316	306
query8	321	237	223	223
query9	8510	2584	2568	2568
query10	464	269	270	269
query11	17555	15356	16058	15356
query12	164	107	106	106
query13	1500	436	441	436
query14	10390	7420	7079	7079
query15	206	186	186	186
query16	7227	485	454	454
query17	1164	594	627	594
query18	1891	328	310	310
query19	206	164	165	164
query20	121	120	122	120
query21	213	108	107	107
query22	4765	4505	4713	4505
query23	34806	33974	34259	33974
query24	6202	3015	2985	2985
query25	545	428	436	428
query26	658	178	170	170
query27	1879	386	374	374
query28	4059	2186	2138	2138
query29	728	466	437	437
query30	242	159	159	159
query31	993	815	810	810
query32	69	55	57	55
query33	435	300	291	291
query34	919	521	527	521
query35	842	735	752	735
query36	1083	959	938	938
query37	112	71	77	71
query38	4120	3966	4003	3966
query39	1532	1483	1505	1483
query40	201	100	101	100
query41	50	50	48	48
query42	107	106	101	101
query43	524	513	500	500
query44	1175	814	825	814
query45	192	172	175	172
query46	1142	733	730	730
query47	2014	1921	1924	1921
query48	481	401	387	387
query49	737	395	398	395
query50	827	432	435	432
query51	7559	7320	7268	7268
query52	102	87	88	87
query53	262	191	195	191
query54	570	483	464	464
query55	80	79	81	79
query56	267	239	260	239
query57	1324	1201	1237	1201
query58	230	214	222	214
query59	3362	3034	3067	3034
query60	301	252	258	252
query61	112	107	160	107
query62	768	686	643	643
query63	212	185	197	185
query64	1390	665	657	657
query65	3302	3224	3214	3214
query66	720	294	300	294
query67	16087	15699	15548	15548
query68	4226	579	581	579
query69	437	264	271	264
query70	1168	1093	1116	1093
query71	353	262	263	262
query72	6303	4044	4031	4031
query73	750	352	365	352
query74	9844	8985	9269	8985
query75	3386	2622	2657	2622
query76	2263	1058	1071	1058
query77	474	295	279	279
query78	10688	9656	9670	9656
query79	1544	601	607	601
query80	887	439	438	438
query81	496	226	222	222
query82	1232	90	86	86
query83	248	153	150	150
query84	289	76	81	76
query85	895	315	295	295
query86	336	295	297	295
query87	4432	4273	4232	4232
query88	3851	2402	2369	2369
query89	426	286	294	286
query90	2014	190	187	187
query91	184	151	155	151
query92	64	51	52	51
query93	2019	547	554	547
query94	754	299	306	299
query95	350	258	259	258
query96	611	288	280	280
query97	3331	3155	3119	3119
query98	211	211	191	191
query99	1600	1291	1335	1291
Total cold run time: 314811 ms
Total hot run time: 197421 ms

@doris-robot
Copy link

doris-robot commented Jun 30, 2025

ClickBench: Total hot run time: 30 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3fdd2c71a8bbae49d8252e9f380d16fcdcbdf275, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.03	0.03
query3	0.23	0.07	0.06
query4	1.63	0.10	0.10
query5	0.51	0.52	0.51
query6	1.13	0.74	0.72
query7	0.02	0.02	0.02
query8	0.04	0.03	0.02
query9	0.56	0.49	0.51
query10	0.55	0.55	0.57
query11	0.13	0.10	0.12
query12	0.14	0.11	0.11
query13	0.62	0.59	0.60
query14	0.81	0.79	0.81
query15	0.84	0.82	0.82
query16	0.39	0.37	0.37
query17	1.06	1.06	1.04
query18	0.22	0.22	0.20
query19	1.84	1.78	1.87
query20	0.01	0.01	0.01
query21	15.39	0.60	0.57
query22	2.57	2.03	1.56
query23	16.98	0.93	0.79
query24	2.51	1.20	1.08
query25	0.20	0.05	0.05
query26	0.62	0.13	0.13
query27	0.05	0.05	0.04
query28	10.65	0.51	0.44
query29	12.60	3.24	3.24
query30	0.24	0.06	0.06
query31	2.85	0.38	0.38
query32	3.25	0.47	0.47
query33	3.00	3.05	3.00
query34	17.13	4.53	4.52
query35	4.54	4.54	4.54
query36	0.67	0.49	0.47
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.02	0.02
query40	0.16	0.13	0.13
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 104.57 s
Total hot run time: 30 s

@zclllyybb
Copy link
Contributor Author

zclllyybb commented Jun 30, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Jun 30, 2025

TPC-H: Total hot run time: 39982 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ea1250325bd893781e6d9e64b68e71dff61afb54, data reload: false

------ Round 1 ----------------------------------
q1	17806	6915	6630	6630
q2	2058	172	195	172
q3	10707	1085	1201	1085
q4	10528	765	725	725
q5	7747	2841	2842	2841
q6	218	135	136	135
q7	995	639	616	616
q8	9349	1960	2020	1960
q9	6684	6445	6448	6445
q10	7012	2241	2281	2241
q11	484	263	260	260
q12	399	220	207	207
q13	17765	3005	2975	2975
q14	240	216	207	207
q15	504	470	482	470
q16	500	380	379	379
q17	998	593	616	593
q18	7335	6739	6696	6696
q19	1391	1098	1142	1098
q20	504	211	213	211
q21	3862	3295	3033	3033
q22	1093	1003	1027	1003
Total cold run time: 108179 ms
Total hot run time: 39982 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6609	6566	6563	6563
q2	332	233	229	229
q3	2895	2766	2971	2766
q4	2066	1799	1719	1719
q5	5801	5740	5762	5740
q6	225	130	130	130
q7	2181	1841	1828	1828
q8	3411	3592	3540	3540
q9	8895	8716	8924	8716
q10	3576	3525	3539	3525
q11	592	506	497	497
q12	799	580	627	580
q13	9596	3167	3156	3156
q14	290	261	267	261
q15	520	458	476	458
q16	493	439	429	429
q17	1856	1646	1612	1612
q18	8275	7892	7648	7648
q19	1697	1560	1653	1560
q20	2111	1810	1823	1810
q21	5095	4971	5000	4971
q22	1136	1033	1045	1033
Total cold run time: 68451 ms
Total hot run time: 58771 ms

@doris-robot
Copy link

doris-robot commented Jun 30, 2025

TPC-DS: Total hot run time: 196559 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ea1250325bd893781e6d9e64b68e71dff61afb54, data reload: false

query1	1281	909	900	900
query2	6254	2002	1967	1967
query3	10820	4416	4315	4315
query4	60942	28929	23284	23284
query5	5202	472	460	460
query6	407	190	182	182
query7	5446	323	326	323
query8	318	235	225	225
query9	8475	2591	2612	2591
query10	460	286	267	267
query11	17553	15314	15620	15314
query12	164	109	107	107
query13	1459	456	416	416
query14	10515	6693	6689	6689
query15	199	173	177	173
query16	7142	458	505	458
query17	1144	566	571	566
query18	1909	302	298	298
query19	212	152	151	151
query20	117	107	107	107
query21	199	101	102	101
query22	4828	4395	4761	4395
query23	34394	34021	33986	33986
query24	6423	2927	2928	2927
query25	536	453	415	415
query26	659	166	172	166
query27	1801	371	361	361
query28	4320	2159	2134	2134
query29	718	495	460	460
query30	234	151	168	151
query31	987	816	812	812
query32	65	57	61	57
query33	441	310	303	303
query34	917	546	534	534
query35	859	729	721	721
query36	1074	966	939	939
query37	115	69	69	69
query38	4023	3992	4001	3992
query39	1545	1494	1468	1468
query40	229	106	107	106
query41	52	51	48	48
query42	119	109	103	103
query43	543	502	481	481
query44	1179	831	818	818
query45	187	175	181	175
query46	1183	763	733	733
query47	2005	1875	1940	1875
query48	482	385	377	377
query49	732	394	407	394
query50	846	431	438	431
query51	7341	7316	7218	7218
query52	104	92	93	92
query53	259	186	188	186
query54	606	466	464	464
query55	79	80	83	80
query56	267	251	243	243
query57	1299	1191	1186	1186
query58	213	206	207	206
query59	3222	3093	3059	3059
query60	280	263	265	263
query61	136	152	120	120
query62	808	729	683	683
query63	226	205	208	205
query64	1477	687	661	661
query65	3353	3253	3181	3181
query66	719	298	296	296
query67	16140	15627	15635	15627
query68	4175	585	575	575
query69	442	261	270	261
query70	1143	1107	1068	1068
query71	354	259	264	259
query72	6417	4080	3988	3988
query73	756	359	364	359
query74	10505	8999	9217	8999
query75	3356	2660	2669	2660
query76	2016	1081	1056	1056
query77	470	268	286	268
query78	10584	9618	9603	9603
query79	1758	605	600	600
query80	1140	436	420	420
query81	526	216	217	216
query82	1260	89	88	88
query83	267	149	145	145
query84	279	77	78	77
query85	1019	309	295	295
query86	390	310	310	310
query87	4347	4193	4182	4182
query88	3919	2371	2369	2369
query89	419	290	296	290
query90	1942	186	187	186
query91	179	147	150	147
query92	60	49	50	49
query93	2003	544	551	544
query94	759	292	298	292
query95	362	264	260	260
query96	613	279	278	278
query97	3286	3140	3137	3137
query98	211	201	193	193
query99	1593	1294	1290	1290
Total cold run time: 314793 ms
Total hot run time: 196559 ms

@doris-robot
Copy link

doris-robot commented Jun 30, 2025

ClickBench: Total hot run time: 30.26 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ea1250325bd893781e6d9e64b68e71dff61afb54, data reload: false

query1	0.04	0.03	0.03
query2	0.06	0.03	0.03
query3	0.23	0.07	0.07
query4	1.60	0.10	0.10
query5	0.54	0.50	0.53
query6	1.13	0.72	0.73
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.56	0.49	0.51
query10	0.55	0.56	0.56
query11	0.16	0.10	0.10
query12	0.14	0.11	0.11
query13	0.61	0.59	0.60
query14	0.77	0.80	0.80
query15	0.84	0.83	0.81
query16	0.38	0.38	0.39
query17	1.03	1.06	1.05
query18	0.23	0.22	0.21
query19	1.91	1.77	1.87
query20	0.01	0.01	0.00
query21	15.41	0.59	0.56
query22	2.55	1.62	1.83
query23	16.94	0.90	0.84
query24	3.08	1.35	1.81
query25	0.23	0.27	0.11
query26	0.47	0.15	0.13
query27	0.04	0.04	0.03
query28	9.38	0.50	0.46
query29	12.57	3.19	3.19
query30	0.25	0.06	0.07
query31	2.86	0.38	0.39
query32	3.25	0.46	0.45
query33	2.96	3.01	3.00
query34	17.23	4.44	4.51
query35	4.53	4.51	4.54
query36	0.66	0.47	0.48
query37	0.08	0.06	0.06
query38	0.04	0.04	0.03
query39	0.03	0.02	0.03
query40	0.16	0.12	0.12
query41	0.08	0.02	0.03
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 103.71 s
Total hot run time: 30.26 s

dataroaring
dataroaring approved these changes Jul 2, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit 32ac14f into apache:branch-3.0 Jul 2, 2025
21 of 24 checks passed
@zclllyybb zclllyybb deleted the pick_jsonb_30 branch July 2, 2025 02:44
zclllyybb added a commit to zclllyybb/doris that referenced this pull request Jul 2, 2025
@zclllyybb
branch-3.0: [Fix](field) Fix potential memory leak and wrong binary r…
3a54c82 
…eading about JsonbField (apache#50174) (apache#52542)

pick apache#50174
zclllyybb added a commit to zclllyybb/doris that referenced this pull request Jul 2, 2025
@zclllyybb
branch-3.0: [Fix](field) Fix potential memory leak and wrong binary r…
c7ed3be 
…eading about JsonbField (apache#50174) (apache#52542)

pick apache#50174
koarz pushed a commit to koarz/doris that referenced this pull request Jul 3, 2025
@zclllyybb @koarz
branch-3.0: [Fix](field) Fix potential memory leak and wrong binary r…
c5d22b7 
…eading about JsonbField (apache#50174) (apache#52542)

pick apache#50174
@gavinchou gavinchou mentioned this pull request Aug 15, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@zclllyybb @hello-stephen @doris-robot @dataroaring