Skip to content

branch-3.1: [chore](external catalog) deprecated lakesoul external catalog #56843 #57163

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morrySnow merged 2 commits into from
Oct 30, 2025
Merged

branch-3.1: [chore](external catalog) deprecated lakesoul external catalog #56843 #57163

morrySnow merged 2 commits into from
Oct 30, 2025

Conversation

@morningman
Copy link
Contributor

@morningman morningman commented Oct 20, 2025

bp #56843

@morningman morningman requested a review from morrySnow as a code owner October 20, 2025 08:41
@Thearas
Copy link
Contributor

Thearas commented Oct 20, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morningman
Copy link
Contributor Author

morningman commented Oct 21, 2025

run buildall

zy-kkk and others added 2 commits October 28, 2025 10:22
@zy-kkk @morningman
[chore](external catalog) deprecated lakesoul external catalog (apach…
b4ae585 
…e#56843)

We are deprecating LakeSoul external catalog support in the current
Doris release due to several critical issues:

**Design Flaws** – The current driver management design causes conflicts
with other components, leading to instability and increased maintenance
complexity.

**Security Vulnerabilities** – LakeSoul dependencies contain multiple
CVE-reported vulnerabilities, posing security risks.

**Lack of Maintenance** – The LakeSoul catalog integration in Doris
lacks active maintainers and has not received timely updates.

**No User Adoption** – No user feedback or usage requests have been
received, indicating that the feature has not been used in practice.

Given these factors, maintaining this integration introduces unnecessary
security and maintenance burdens without providing tangible value to
users. Therefore, we have decided to **deprecate** LakeSoul catalog
support in this release:

- All LakeSoul-related code has been marked as `@Deprecated` and will be
kept for backward compatibility
- LakeSoul dependencies (including `lakesoul-io-java`, `postgresql`, and
`scala-library`) have been changed to `provided` scope and will not be
packaged in the release
  - Comment `extensions_modules+=("lakesoul-scanner")` in build.sh

Users who still need LakeSoul support can manually enable it by
modifying Maven configurations. However, we strongly recommend migrating
to alternative catalog solutions. Complete removal of LakeSoul support
is planned for a future release.
@morningman
1
b0a2784
@morningman
Copy link
Contributor Author

morningman commented Oct 28, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Oct 28, 2025

TPC-H: Total hot run time: 32689 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b0a278416b7e9626d353db7d1383ffcf940798fa, data reload: false

------ Round 1 ----------------------------------
q1	17590	5527	5525	5525
q2	2021	389	287	287
q3	12242	1233	764	764
q4	10481	876	446	446
q5	9585	2370	2143	2143
q6	189	168	134	134
q7	896	737	619	619
q8	9327	1396	1123	1123
q9	5243	5028	4886	4886
q10	6770	2294	1828	1828
q11	460	281	261	261
q12	337	356	211	211
q13	17765	3595	3018	3018
q14	223	227	215	215
q15	519	466	475	466
q16	434	430	371	371
q17	591	860	359	359
q18	6865	6406	6347	6347
q19	1527	951	527	527
q20	318	332	202	202
q21	2759	2163	1960	1960
q22	1032	1024	997	997
Total cold run time: 107174 ms
Total hot run time: 32689 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5549	5502	5529	5502
q2	231	334	231	231
q3	2220	2659	2322	2322
q4	1374	1826	1406	1406
q5	4404	4960	4968	4960
q6	172	163	137	137
q7	2075	2007	1801	1801
q8	2652	2832	2724	2724
q9	7330	7238	7211	7211
q10	3039	3274	2709	2709
q11	564	510	479	479
q12	657	782	599	599
q13	3347	3776	3199	3199
q14	274	304	268	268
q15	523	472	478	472
q16	431	483	431	431
q17	1215	1739	1277	1277
q18	7622	7540	7368	7368
q19	773	1187	1096	1096
q20	2028	2036	1907	1907
q21	5318	4852	4504	4504
q22	1070	1084	1038	1038
Total cold run time: 52868 ms
Total hot run time: 51641 ms

@doris-robot
Copy link

doris-robot commented Oct 28, 2025

TPC-DS: Total hot run time: 192414 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b0a278416b7e9626d353db7d1383ffcf940798fa, data reload: false

query1	933	418	400	400
query2	6249	1947	1840	1840
query3	8690	200	196	196
query4	33738	23838	23843	23838
query5	4131	630	469	469
query6	303	203	178	178
query7	4201	488	316	316
query8	301	234	229	229
query9	9493	2588	2578	2578
query10	470	314	261	261
query11	18005	15576	15282	15282
query12	162	110	110	110
query13	1553	544	422	422
query14	9319	6905	6615	6615
query15	255	196	174	174
query16	8100	681	444	444
query17	1566	760	599	599
query18	2145	416	338	338
query19	236	191	169	169
query20	130	133	129	129
query21	211	133	108	108
query22	4610	4580	4513	4513
query23	34957	34047	34283	34047
query24	7361	2711	2701	2701
query25	536	503	450	450
query26	902	289	173	173
query27	2331	464	370	370
query28	5400	2167	2139	2139
query29	730	605	459	459
query30	233	193	170	170
query31	1010	943	858	858
query32	94	61	79	61
query33	486	371	323	323
query34	737	843	521	521
query35	796	782	714	714
query36	1029	1096	982	982
query37	103	96	66	66
query38	4094	4114	3959	3959
query39	1525	1467	1483	1467
query40	211	120	104	104
query41	52	49	46	46
query42	122	106	104	104
query43	502	523	489	489
query44	1318	869	819	819
query45	189	182	170	170
query46	887	1069	678	678
query47	1951	1981	1912	1912
query48	403	430	347	347
query49	734	496	390	390
query50	689	677	421	421
query51	7325	7485	7134	7134
query52	104	100	94	94
query53	228	262	190	190
query54	542	539	463	463
query55	80	83	82	82
query56	270	263	268	263
query57	1277	1301	1199	1199
query58	241	235	223	223
query59	3051	3262	3073	3073
query60	307	295	272	272
query61	116	117	124	117
query62	796	737	678	678
query63	243	197	198	197
query64	3848	1005	665	665
query65	3407	3271	3279	3271
query66	1002	404	308	308
query67	16117	15841	15799	15799
query68	7642	819	528	528
query69	505	314	263	263
query70	1166	1126	1076	1076
query71	361	290	262	262
query72	5624	3837	3807	3807
query73	634	743	349	349
query74	10313	9397	9132	9132
query75	3166	3133	2680	2680
query76	2973	1193	773	773
query77	479	371	292	292
query78	10243	10522	9600	9600
query79	3516	846	589	589
query80	729	543	460	460
query81	507	259	222	222
query82	547	117	93	93
query83	174	170	149	149
query84	287	104	85	85
query85	764	353	301	301
query86	346	298	297	297
query87	4302	4329	4239	4239
query88	4932	2405	2396	2396
query89	413	334	283	283
query90	1868	193	194	193
query91	140	144	110	110
query92	67	57	53	53
query93	1936	909	547	547
query94	701	389	296	296
query95	354	281	275	275
query96	499	603	289	289
query97	3173	3237	3073	3073
query98	215	210	214	210
query99	1516	1411	1316	1316
Total cold run time: 293229 ms
Total hot run time: 192414 ms

@doris-robot
Copy link

doris-robot commented Oct 28, 2025

ClickBench: Total hot run time: 28.36 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b0a278416b7e9626d353db7d1383ffcf940798fa, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.04	0.03
query3	0.23	0.07	0.06
query4	1.61	0.11	0.10
query5	0.52	0.50	0.49
query6	1.12	0.73	0.72
query7	0.02	0.02	0.02
query8	0.06	0.03	0.04
query9	0.57	0.49	0.51
query10	0.55	0.55	0.56
query11	0.14	0.10	0.10
query12	0.14	0.12	0.11
query13	0.62	0.61	0.60
query14	0.78	0.81	0.78
query15	0.84	0.83	0.83
query16	0.37	0.38	0.38
query17	1.01	1.06	1.03
query18	0.25	0.22	0.22
query19	1.98	1.88	1.89
query20	0.01	0.01	0.01
query21	15.40	0.93	0.58
query22	0.73	0.79	0.59
query23	15.19	1.46	0.62
query24	3.37	0.36	1.76
query25	0.26	0.06	0.09
query26	0.35	0.15	0.14
query27	0.07	0.05	0.03
query28	13.35	1.03	0.43
query29	12.56	3.98	3.28
query30	0.24	0.08	0.06
query31	2.83	0.57	0.37
query32	3.23	0.54	0.46
query33	3.03	3.00	3.03
query34	16.65	5.19	4.57
query35	4.56	4.56	4.58
query36	0.62	0.50	0.48
query37	0.08	0.06	0.06
query38	0.04	0.04	0.03
query39	0.03	0.02	0.03
query40	0.17	0.14	0.13
query41	0.07	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 103.8 s
Total hot run time: 28.36 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 28, 2025

FE UT Coverage Report

Increment line coverage 0.00% (0/6) 🎉
Increment coverage report
Complete coverage report

@morrySnow morrySnow changed the title branch-3.1: [chore](external catalog) deprecated lakesoul external catalog (#56843) branch-3.1: [chore](external catalog) deprecated lakesoul external catalog #56843 Oct 30, 2025
@morrySnow morrySnow merged commit ee813c1 into apache:branch-3.1 Oct 30, 2025
21 of 22 checks passed
@morrySnow morrySnow mentioned this pull request Nov 13, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morrySnow morrySnow morrySnow approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@morningman @Thearas @doris-robot @hello-stephen @morrySnow @zy-kkk