[Fix](memory) Add try catch block for `Segment::load_pk_index_and_bf` #47715

bobhan1 · 2025-02-10T12:12:33Z

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

Thearas · 2025-02-10T12:12:38Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

bobhan1 · 2025-02-10T12:12:41Z

run buildall

zhannngchen

LGTM

github-actions · 2025-02-10T12:14:10Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-02-10T12:14:12Z

PR approved by anyone and no changes requested.

doris-robot · 2025-02-10T13:19:27Z

TPC-H: Total hot run time: 31244 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0b4815a9805e2013f2c06c11a9d2dca866283b03, data reload: false

------ Round 1 ----------------------------------
q1	17621	5212	5020	5020
q2	2058	305	183	183
q3	10594	1255	748	748
q4	10212	1022	509	509
q5	7541	2449	2294	2294
q6	190	167	136	136
q7	903	745	609	609
q8	9325	1299	1062	1062
q9	4889	4537	4667	4537
q10	6799	2302	1893	1893
q11	488	274	251	251
q12	349	356	226	226
q13	17778	3704	3057	3057
q14	234	222	219	219
q15	512	463	467	463
q16	620	605	583	583
q17	588	875	350	350
q18	6801	6354	6147	6147
q19	1493	954	564	564
q20	319	321	195	195
q21	2822	2229	1902	1902
q22	362	323	296	296
Total cold run time: 102498 ms
Total hot run time: 31244 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5152	5147	5102	5102
q2	236	325	232	232
q3	2198	2719	2290	2290
q4	1428	1825	1373	1373
q5	4228	4139	4207	4139
q6	208	162	123	123
q7	1863	1790	1643	1643
q8	2606	2694	2620	2620
q9	7297	7269	7129	7129
q10	3019	3224	2792	2792
q11	580	506	483	483
q12	681	777	628	628
q13	3472	3873	3335	3335
q14	300	295	280	280
q15	521	469	470	469
q16	637	683	612	612
q17	1151	1571	1333	1333
q18	7581	7401	7299	7299
q19	790	780	820	780
q20	1952	2061	1854	1854
q21	5467	5025	5039	5025
q22	625	587	514	514
Total cold run time: 51992 ms
Total hot run time: 50055 ms

doris-robot · 2025-02-10T13:31:32Z

TPC-DS: Total hot run time: 190994 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0b4815a9805e2013f2c06c11a9d2dca866283b03, data reload: false

query1	1291	959	940	940
query2	6230	1869	1864	1864
query3	11121	4697	4497	4497
query4	54048	25457	23224	23224
query5	5147	594	496	496
query6	348	209	185	185
query7	4969	516	305	305
query8	307	243	237	237
query9	5852	2547	2539	2539
query10	417	316	255	255
query11	15110	15067	14961	14961
query12	155	120	105	105
query13	1099	516	387	387
query14	10575	6701	6755	6701
query15	212	215	188	188
query16	7079	688	524	524
query17	1106	739	604	604
query18	1450	440	326	326
query19	202	204	177	177
query20	125	121	117	117
query21	208	134	105	105
query22	4631	4551	4596	4551
query23	33877	33447	33350	33350
query24	5607	2421	2419	2419
query25	453	459	414	414
query26	721	278	161	161
query27	1873	508	331	331
query28	2768	2438	2409	2409
query29	566	570	431	431
query30	210	194	157	157
query31	876	874	837	837
query32	71	60	60	60
query33	467	354	298	298
query34	761	878	513	513
query35	824	822	773	773
query36	943	1002	927	927
query37	126	97	79	79
query38	4204	4367	4284	4284
query39	1509	1424	1419	1419
query40	216	120	103	103
query41	49	48	53	48
query42	136	104	99	99
query43	510	525	471	471
query44	1317	820	831	820
query45	185	177	180	177
query46	886	1081	663	663
query47	1835	1887	1832	1832
query48	391	428	310	310
query49	707	503	420	420
query50	710	756	415	415
query51	4292	4254	4224	4224
query52	114	107	101	101
query53	232	257	198	198
query54	471	482	417	417
query55	94	83	89	83
query56	250	282	276	276
query57	1140	1203	1128	1128
query58	244	244	240	240
query59	2799	2787	2753	2753
query60	286	276	266	266
query61	131	113	138	113
query62	777	714	664	664
query63	225	194	189	189
query64	2050	1057	666	666
query65	3194	3139	3145	3139
query66	750	411	295	295
query67	15895	15842	15326	15326
query68	5496	771	503	503
query69	519	303	253	253
query70	1216	1143	1125	1125
query71	435	308	267	267
query72	6242	3733	3753	3733
query73	1320	752	351	351
query74	9198	9144	9062	9062
query75	3137	3137	2698	2698
query76	4207	1170	739	739
query77	510	362	281	281
query78	9973	9936	9280	9280
query79	2828	807	594	594
query80	683	526	438	438
query81	497	284	234	234
query82	633	150	166	150
query83	178	167	153	153
query84	285	104	69	69
query85	750	337	301	301
query86	363	313	299	299
query87	4535	4568	4356	4356
query88	3516	2240	2212	2212
query89	425	318	282	282
query90	1661	194	194	194
query91	137	137	107	107
query92	72	59	64	59
query93	2536	1021	574	574
query94	718	411	310	310
query95	335	269	256	256
query96	480	560	334	334
query97	2818	2859	2719	2719
query98	234	216	208	208
query99	1326	1409	1267	1267
Total cold run time: 294687 ms
Total hot run time: 190994 ms

doris-robot · 2025-02-10T13:36:50Z

ClickBench: Total hot run time: 30.99 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0b4815a9805e2013f2c06c11a9d2dca866283b03, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.04
query3	0.23	0.06	0.07
query4	1.61	0.10	0.11
query5	0.40	0.41	0.40
query6	1.17	0.66	0.66
query7	0.03	0.02	0.01
query8	0.04	0.03	0.03
query9	0.60	0.52	0.52
query10	0.58	0.59	0.58
query11	0.15	0.10	0.10
query12	0.15	0.11	0.10
query13	0.62	0.60	0.60
query14	2.70	2.68	2.70
query15	0.94	0.86	0.84
query16	0.37	0.37	0.38
query17	1.03	1.08	1.04
query18	0.22	0.21	0.19
query19	1.94	1.81	2.02
query20	0.01	0.01	0.01
query21	15.35	0.89	0.55
query22	0.78	1.20	0.64
query23	14.95	1.37	0.62
query24	6.91	2.03	1.08
query25	0.48	0.22	0.18
query26	0.69	0.16	0.15
query27	0.05	0.06	0.05
query28	9.52	0.79	0.44
query29	12.58	3.99	3.30
query30	0.25	0.09	0.06
query31	2.82	0.59	0.38
query32	3.23	0.55	0.47
query33	2.98	3.09	3.01
query34	15.80	5.19	4.53
query35	4.53	4.57	4.58
query36	0.66	0.49	0.49
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.02
query40	0.17	0.14	0.13
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.03	0.03	0.02
Total cold run time: 104.96 s
Total hot run time: 30.99 s

bobhan1 · 2025-02-11T02:16:03Z

run buildall

doris-robot · 2025-02-11T02:54:54Z

TPC-H: Total hot run time: 31447 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 51eccbf05b94b6407c3d6e847ea0c0338c37084e, data reload: false

------ Round 1 ----------------------------------
q1	17577	5178	5085	5085
q2	2043	312	178	178
q3	10383	1240	765	765
q4	10202	1017	548	548
q5	7546	2381	2345	2345
q6	192	168	133	133
q7	891	745	597	597
q8	9290	1275	1084	1084
q9	4936	4885	4552	4552
q10	6889	2312	1905	1905
q11	499	307	274	274
q12	370	386	241	241
q13	18039	3709	3106	3106
q14	225	227	208	208
q15	516	463	455	455
q16	626	609	590	590
q17	585	865	344	344
q18	6667	6339	6082	6082
q19	1207	951	535	535
q20	316	327	195	195
q21	2876	2176	1919	1919
q22	368	336	306	306
Total cold run time: 102243 ms
Total hot run time: 31447 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5182	5120	5242	5120
q2	242	322	233	233
q3	2148	2689	2340	2340
q4	1439	1829	1360	1360
q5	4239	4156	4178	4156
q6	212	166	129	129
q7	1872	1834	1685	1685
q8	2619	2695	2584	2584
q9	7241	7189	7116	7116
q10	3017	3260	2789	2789
q11	589	525	506	506
q12	710	775	646	646
q13	3539	3946	3297	3297
q14	278	304	274	274
q15	505	477	465	465
q16	629	682	628	628
q17	1157	1591	1374	1374
q18	7647	7387	7389	7387
q19	844	877	934	877
q20	2005	2004	1877	1877
q21	5534	5033	4650	4650
q22	671	575	548	548
Total cold run time: 52319 ms
Total hot run time: 50041 ms

zhannngchen

LGTM

github-actions · 2025-02-11T03:06:42Z

PR approved by at least one committer and no changes requested.

doris-robot · 2025-02-11T03:06:51Z

TPC-DS: Total hot run time: 190653 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 51eccbf05b94b6407c3d6e847ea0c0338c37084e, data reload: false

query1	1329	962	965	962
query2	6135	1899	1851	1851
query3	11090	4588	4478	4478
query4	55652	25790	23431	23431
query5	4880	582	463	463
query6	316	194	192	192
query7	4869	522	301	301
query8	310	241	258	241
query9	5366	2516	2534	2516
query10	418	297	241	241
query11	15509	14995	14935	14935
query12	149	104	102	102
query13	1033	494	390	390
query14	10638	6336	6721	6336
query15	206	202	179	179
query16	7093	654	523	523
query17	1076	760	629	629
query18	1568	429	342	342
query19	207	183	163	163
query20	124	123	126	123
query21	205	125	104	104
query22	4258	4425	4433	4425
query23	34132	33360	33349	33349
query24	6070	2477	2456	2456
query25	478	464	406	406
query26	678	277	149	149
query27	1692	494	340	340
query28	2965	2437	2405	2405
query29	559	566	445	445
query30	215	198	154	154
query31	888	930	836	836
query32	72	66	59	59
query33	440	351	306	306
query34	805	852	508	508
query35	799	843	754	754
query36	976	1026	947	947
query37	120	102	76	76
query38	4276	4319	4199	4199
query39	1513	1440	1429	1429
query40	207	116	102	102
query41	53	47	52	47
query42	125	113	106	106
query43	507	531	482	482
query44	1322	816	805	805
query45	183	171	165	165
query46	869	1066	670	670
query47	1828	1919	1827	1827
query48	397	427	320	320
query49	701	514	433	433
query50	727	751	432	432
query51	4343	4365	4245	4245
query52	115	108	94	94
query53	239	272	191	191
query54	498	496	422	422
query55	91	88	83	83
query56	280	284	266	266
query57	1141	1185	1100	1100
query58	244	250	244	244
query59	2683	2803	2630	2630
query60	291	295	281	281
query61	117	118	123	118
query62	764	743	661	661
query63	239	198	199	198
query64	1449	1020	664	664
query65	3288	3160	3146	3146
query66	689	395	300	300
query67	15711	15621	15358	15358
query68	2601	783	541	541
query69	438	304	280	280
query70	1234	1153	1134	1134
query71	331	303	269	269
query72	5213	3541	3711	3541
query73	635	739	350	350
query74	9173	9029	8982	8982
query75	3111	3186	2723	2723
query76	1724	1173	745	745
query77	405	380	281	281
query78	10095	10060	9296	9296
query79	2609	802	683	683
query80	956	519	448	448
query81	531	270	243	243
query82	422	149	120	120
query83	208	179	155	155
query84	283	95	72	72
query85	756	368	356	356
query86	364	300	294	294
query87	4501	4535	4497	4497
query88	2935	2236	2179	2179
query89	397	324	285	285
query90	1685	192	188	188
query91	129	139	110	110
query92	59	61	56	56
query93	1673	1032	574	574
query94	641	418	297	297
query95	350	265	269	265
query96	478	545	265	265
query97	2784	2879	2775	2775
query98	223	204	211	204
query99	1314	1400	1310	1310
Total cold run time: 286468 ms
Total hot run time: 190653 ms

doris-robot · 2025-02-11T03:12:09Z

ClickBench: Total hot run time: 31.2 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 51eccbf05b94b6407c3d6e847ea0c0338c37084e, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.03	0.04
query3	0.24	0.07	0.06
query4	1.64	0.10	0.11
query5	0.42	0.41	0.39
query6	1.18	0.66	0.67
query7	0.02	0.02	0.01
query8	0.04	0.04	0.03
query9	0.59	0.52	0.51
query10	0.56	0.58	0.57
query11	0.15	0.10	0.11
query12	0.15	0.11	0.12
query13	0.62	0.59	0.59
query14	2.67	2.68	2.72
query15	0.94	0.85	0.83
query16	0.37	0.36	0.38
query17	1.04	1.04	1.02
query18	0.21	0.20	0.19
query19	1.95	1.84	1.92
query20	0.02	0.00	0.01
query21	15.36	0.92	0.56
query22	0.75	1.14	0.82
query23	14.82	1.38	0.63
query24	6.46	1.83	1.15
query25	0.53	0.26	0.13
query26	0.58	0.15	0.14
query27	0.05	0.06	0.05
query28	10.50	0.86	0.43
query29	12.54	3.92	3.31
query30	0.25	0.08	0.06
query31	2.83	0.58	0.39
query32	3.22	0.56	0.47
query33	2.99	3.02	3.12
query34	15.82	5.14	4.56
query35	4.56	4.56	4.56
query36	0.66	0.48	0.50
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.17	0.14	0.13
query41	0.08	0.03	0.03
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 105.34 s
Total hot run time: 31.2 s

doris-robot · 2025-02-11T03:17:57Z

TeamCity be ut coverage result:
Function Coverage: 42.81% (11263/26309)
Line Coverage: 32.79% (94608/288570)
Region Coverage: 31.94% (48504/151839)
Branch Coverage: 27.81% (24459/87944)
Coverage Report: http://coverage.selectdb-in.cc/coverage/51eccbf05b94b6407c3d6e847ea0c0338c37084e_51eccbf05b94b6407c3d6e847ea0c0338c37084e/report/index.html

dataroaring

LGTM

…#47715)

…apache#47715)

…ndex_and_bf`… (#47742) pick #47715

…apache#47715)

…ndex_and_bf` #47715 (#47736) Cherry-picked from #47715 Co-authored-by: bobhan1 <baohan@selectdb.com>

…apache#47715)

…ndex_and_bf`… (apache#47742) pick apache#47715

zhannngchen previously approved these changes Feb 10, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 10, 2025

github-actions bot added the reviewed label Feb 10, 2025

bobhan1 dismissed zhannngchen’s stale review via f78c9a4 February 10, 2025 14:07

bobhan1 force-pushed the catch-for-load_pk_index_and_bf-2 branch from 0b4815a to f78c9a4 Compare February 10, 2025 14:07

github-actions bot removed the approved Indicates a PR has been approved by one committer. label Feb 10, 2025

bobhan1 force-pushed the catch-for-load_pk_index_and_bf-2 branch from f78c9a4 to 73f817f Compare February 10, 2025 14:08

fix

51eccbf

bobhan1 force-pushed the catch-for-load_pk_index_and_bf-2 branch from 73f817f to 51eccbf Compare February 10, 2025 14:09

bobhan1 changed the title ~~[Fix](memory) Add try catch block for call to dst->insert_many_continuous_binary_data~~ [Fix](memory) Add try catch block for call to Segment::load_pk_index_and_bf Feb 10, 2025

bobhan1 requested a review from zhannngchen February 10, 2025 14:55

bobhan1 changed the title ~~[Fix](memory) Add try catch block for call to Segment::load_pk_index_and_bf~~ [Fix](memory) Add try catch block for Segment::load_pk_index_and_bf Feb 11, 2025

zhannngchen approved these changes Feb 11, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 11, 2025

dataroaring approved these changes Feb 11, 2025

View reviewed changes

dataroaring added dev/2.1.x dev/3.0.x labels Feb 11, 2025

dataroaring merged commit b0e5e09 into apache:master Feb 11, 2025
29 of 30 checks passed

github-actions bot pushed a commit that referenced this pull request Feb 11, 2025

[Fix](memory) Add try catch block for Segment::load_pk_index_and_bf (…

993064f

…#47715)

github-actions bot mentioned this pull request Feb 11, 2025

branch-3.0: [Fix](memory) Add try catch block for Segment::load_pk_index_and_bf #47715 #47736

Merged

github-actions bot added the dev/2.1.x-conflict label Feb 11, 2025

bobhan1 added a commit to bobhan1/doris that referenced this pull request Feb 11, 2025

[Fix](memory) Add try catch block for Segment::load_pk_index_and_bf (…

98caf02

…apache#47715)

bobhan1 mentioned this pull request Feb 11, 2025

branch-2.1: [Fix](memory) Add try catch block for Segment::load_pk_index_and_bf… #47742

Merged

bobhan1 added a commit to bobhan1/doris that referenced this pull request Feb 11, 2025

[Fix](memory) Add try catch block for Segment::load_pk_index_and_bf (…

91cb60c

…apache#47715)

yiguolei pushed a commit that referenced this pull request Feb 11, 2025

branch-2.1: [Fix](memory) Add try catch block for `Segment::load_pk_i…

c8ccada

…ndex_and_bf`… (#47742) pick #47715

yiguolei added dev/2.1.9-merged and removed dev/2.1.x dev/2.1.x-conflict labels Feb 11, 2025

lzyy2024 pushed a commit to lzyy2024/doris that referenced this pull request Feb 21, 2025

[Fix](memory) Add try catch block for Segment::load_pk_index_and_bf (…

fc26f13

…apache#47715)

dataroaring added the p0_b label Feb 24, 2025

dataroaring pushed a commit that referenced this pull request Feb 24, 2025

branch-3.0: [Fix](memory) Add try catch block for `Segment::load_pk_i…

098c13c

…ndex_and_bf` #47715 (#47736) Cherry-picked from #47715 Co-authored-by: bobhan1 <baohan@selectdb.com>

dataroaring added dev/3.0.5-merged and removed dev/3.0.x labels Feb 24, 2025

yiguolei mentioned this pull request Mar 25, 2025

2.1.9 Release Note #49443

Closed

koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025

[Fix](memory) Add try catch block for Segment::load_pk_index_and_bf (…

2cb815b

…apache#47715)

deardeng pushed a commit to deardeng/incubator-doris that referenced this pull request Dec 19, 2025

branch-2.1: [Fix](memory) Add try catch block for `Segment::load_pk_i…

ef3088c

…ndex_and_bf`… (apache#47742) pick apache#47715

[Fix](memory) Add try catch block for Segment::load_pk_index_and_bf #47715

[Fix](memory) Add try catch block for Segment::load_pk_index_and_bf #47715

Uh oh!

Conversation

bobhan1 commented Feb 10, 2025

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

Thearas commented Feb 10, 2025

Uh oh!

bobhan1 commented Feb 10, 2025

Uh oh!

zhannngchen left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Feb 10, 2025

Uh oh!

github-actions bot commented Feb 10, 2025

Uh oh!

doris-robot commented Feb 10, 2025

Uh oh!

doris-robot commented Feb 10, 2025

Uh oh!

doris-robot commented Feb 10, 2025

Uh oh!

bobhan1 commented Feb 11, 2025

Uh oh!

doris-robot commented Feb 11, 2025

Uh oh!

zhannngchen left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Feb 11, 2025

Uh oh!

doris-robot commented Feb 11, 2025

Uh oh!

doris-robot commented Feb 11, 2025

Uh oh!

doris-robot commented Feb 11, 2025

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

[Fix](memory) Add try catch block for `Segment::load_pk_index_and_bf` #47715

[Fix](memory) Add try catch block for `Segment::load_pk_index_and_bf` #47715