Skip to content

[fix](paimon) fix crash when enable cache with paimon deletion vector #39877

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morningman merged 1 commit into from
Aug 26, 2024
Merged

[fix](paimon) fix crash when enable cache with paimon deletion vector #39877

morningman merged 1 commit into from
Aug 26, 2024

Conversation

@morningman
Copy link
Contributor

@morningman morningman commented Aug 24, 2024

missing IOContext param, cause NPE.
When init row filter for paimon reader, it will read deletion file, which need io context

@doris-robot
Copy link

doris-robot commented Aug 24, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@morningman morningman mentioned this pull request Aug 24, 2024
Merged
@morningman
Copy link
Contributor Author

morningman commented Aug 24, 2024

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Aug 24, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

doris-robot commented Aug 24, 2024

TPC-H: Total hot run time: 38262 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 763401c42a697fcf966bcb2f6d831f1206fc47ba, data reload: false

------ Round 1 ----------------------------------
q1	17627	4514	4364	4364
q2	2027	183	180	180
q3	11890	980	1246	980
q4	10516	695	690	690
q5	7740	2912	2788	2788
q6	228	141	144	141
q7	977	619	598	598
q8	9329	2108	2116	2108
q9	7206	6555	6580	6555
q10	7008	2202	2230	2202
q11	450	248	247	247
q12	399	227	233	227
q13	18952	3073	3061	3061
q14	283	235	236	235
q15	519	487	507	487
q16	517	395	388	388
q17	1012	682	717	682
q18	7465	6886	7011	6886
q19	1387	1077	1038	1038
q20	692	332	332	332
q21	4539	3396	3057	3057
q22	1121	1038	1016	1016
Total cold run time: 111884 ms
Total hot run time: 38262 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4415	4363	4306	4306
q2	379	266	288	266
q3	2867	2654	2667	2654
q4	2041	1728	1750	1728
q5	5752	5651	5771	5651
q6	228	138	148	138
q7	2240	1809	1840	1809
q8	3298	3472	3528	3472
q9	8907	8940	8890	8890
q10	3689	3377	3401	3377
q11	604	527	515	515
q12	847	650	654	650
q13	14954	3206	3278	3206
q14	343	291	299	291
q15	526	475	507	475
q16	498	455	447	447
q17	1877	1548	1558	1548
q18	8258	7831	7916	7831
q19	1766	1661	1621	1621
q20	2191	1966	1907	1907
q21	5746	5646	5626	5626
q22	1130	1068	1045	1045
Total cold run time: 72556 ms
Total hot run time: 57453 ms

@doris-robot
Copy link

doris-robot commented Aug 24, 2024

TPC-DS: Total hot run time: 192201 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 763401c42a697fcf966bcb2f6d831f1206fc47ba, data reload: false

query1	1268	900	882	882
query2	6350	1994	1986	1986
query3	10733	4090	4128	4090
query4	60053	24967	23319	23319
query5	5421	485	475	475
query6	404	161	173	161
query7	5770	302	291	291
query8	288	199	208	199
query9	8855	2454	2437	2437
query10	507	276	258	258
query11	18229	15136	15407	15136
query12	162	106	102	102
query13	1555	409	384	384
query14	11266	7241	7688	7241
query15	225	184	174	174
query16	7068	482	498	482
query17	1140	572	555	555
query18	1437	299	292	292
query19	299	151	148	148
query20	119	109	193	109
query21	204	103	101	101
query22	4630	4303	4434	4303
query23	34349	33864	33486	33486
query24	5964	2868	2809	2809
query25	520	377	380	377
query26	687	162	161	161
query27	1785	289	308	289
query28	3720	2060	2027	2027
query29	644	405	404	404
query30	234	151	152	151
query31	927	762	765	762
query32	83	54	57	54
query33	490	312	287	287
query34	868	494	476	476
query35	828	722	714	714
query36	1037	896	928	896
query37	140	77	82	77
query38	4015	3819	4002	3819
query39	1467	1386	1396	1386
query40	196	115	114	114
query41	44	45	43	43
query42	115	94	95	94
query43	504	485	483	483
query44	1115	755	764	755
query45	200	163	170	163
query46	1088	758	713	713
query47	1881	1764	1798	1764
query48	390	301	304	301
query49	776	441	460	441
query50	809	442	423	423
query51	7150	7096	7060	7060
query52	99	86	89	86
query53	250	179	175	175
query54	556	458	467	458
query55	80	78	78	78
query56	273	263	256	256
query57	1172	1083	1094	1083
query58	220	233	231	231
query59	3066	2794	2898	2794
query60	297	265	272	265
query61	98	97	98	97
query62	760	668	655	655
query63	221	183	183	183
query64	3546	2047	1751	1751
query65	3229	3177	3122	3122
query66	627	338	340	338
query67	15312	15209	15183	15183
query68	2987	588	593	588
query69	402	287	283	283
query70	1197	1121	1126	1121
query71	348	276	277	276
query72	2565	2127	2059	2059
query73	708	336	330	330
query74	9166	8849	8981	8849
query75	3329	2643	2779	2643
query76	1398	1009	985	985
query77	547	345	317	317
query78	9625	8981	9000	8981
query79	1040	552	531	531
query80	674	514	498	498
query81	454	222	225	222
query82	297	135	131	131
query83	178	147	147	147
query84	247	76	74	74
query85	664	290	285	285
query86	324	318	297	297
query87	4276	4267	4263	4263
query88	3135	2377	2394	2377
query89	391	293	288	288
query90	1917	197	193	193
query91	139	121	115	115
query92	131	54	58	54
query93	1056	531	538	531
query94	699	300	306	300
query95	310	267	270	267
query96	587	271	272	271
query97	3217	3051	3061	3051
query98	217	203	198	198
query99	1547	1304	1267	1267
Total cold run time: 301587 ms
Total hot run time: 192201 ms

@doris-robot
Copy link

doris-robot commented Aug 24, 2024

ClickBench: Total hot run time: 31.26 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 763401c42a697fcf966bcb2f6d831f1206fc47ba, data reload: false

query1	0.05	0.04	0.04
query2	0.07	0.03	0.04
query3	0.22	0.05	0.05
query4	1.67	0.07	0.08
query5	0.50	0.50	0.48
query6	1.12	0.72	0.73
query7	0.02	0.02	0.01
query8	0.05	0.04	0.05
query9	0.55	0.49	0.49
query10	0.55	0.53	0.53
query11	0.16	0.12	0.12
query12	0.15	0.12	0.13
query13	0.62	0.59	0.59
query14	0.76	0.81	0.80
query15	0.85	0.81	0.83
query16	0.37	0.39	0.37
query17	1.04	1.07	1.06
query18	0.22	0.21	0.21
query19	1.86	1.83	1.87
query20	0.02	0.01	0.01
query21	15.40	0.67	0.67
query22	4.45	6.61	2.18
query23	18.21	1.48	1.31
query24	2.20	0.23	0.23
query25	0.15	0.08	0.07
query26	0.26	0.17	0.18
query27	0.08	0.08	0.08
query28	13.18	1.02	0.99
query29	12.66	3.42	3.37
query30	0.24	0.05	0.06
query31	2.85	0.41	0.39
query32	3.27	0.49	0.47
query33	3.00	2.99	3.00
query34	16.97	4.36	4.49
query35	4.52	4.47	4.47
query36	0.68	0.47	0.49
query37	0.19	0.16	0.15
query38	0.16	0.15	0.15
query39	0.04	0.04	0.04
query40	0.17	0.13	0.13
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.73 s
Total hot run time: 31.26 s

morningman added a commit that referenced this pull request Aug 24, 2024
@morningman
[fix](paimon) fix crash when enable cache with paimon deletion vector(#…
263746b 
…39877) (#39875)

bp #39877
@github-actions
Copy link
Contributor

github-actions bot commented Aug 26, 2024

PR approved by anyone and no changes requested.

kaka11chen
kaka11chen approved these changes Aug 26, 2024
View reviewed changes
Copy link
Contributor

@kaka11chen kaka11chen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 26, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Aug 26, 2024

PR approved by at least one committer and no changes requested.

@morningman morningman merged commit 4214bed into apache:master Aug 26, 2024
dataroaring pushed a commit that referenced this pull request Aug 26, 2024
@morningman @dataroaring
[fix](paimon) fix crash when enable cache with paimon deletion vector (…
ec51990 
…#39877)

missing IOContext param, cause NPE.
When init row filter for paimon reader, it will read deletion file,
which need io context
@gavinchou gavinchou mentioned this pull request Oct 13, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kaka11chen kaka11chen kaka11chen approved these changes

@wuwenchi wuwenchi wuwenchi approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.6-merged dev/3.0.2-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@morningman @doris-robot @kaka11chen @wuwenchi @dataroaring