Skip to content

Conversation

@zy-kkk
Copy link
Member

@zy-kkk zy-kkk commented Oct 11, 2025

What problem does this PR solve?

We are deprecating LakeSoul external catalog support in the current Doris release due to several critical issues:

Design Flaws – The current driver management design causes conflicts with other components, leading to instability and increased maintenance complexity.

Security Vulnerabilities – LakeSoul dependencies contain multiple CVE-reported vulnerabilities, posing security risks.

Lack of Maintenance – The LakeSoul catalog integration in Doris lacks active maintainers and has not received timely updates.

No User Adoption – No user feedback or usage requests have been received, indicating that the feature has not been used in practice.

Given these factors, maintaining this integration introduces unnecessary security and maintenance burdens without providing tangible value to users. Therefore, we have decided to deprecate LakeSoul catalog support in this release:

  • All LakeSoul-related code has been marked as @Deprecated and will be kept for backward compatibility
  • LakeSoul dependencies (including lakesoul-io-java, postgresql, and scala-library) have been changed to provided scope and will not be packaged in the release
  • Comment extensions_modules+=("lakesoul-scanner") in build.sh

Users who still need LakeSoul support can manually enable it by modifying Maven configurations. However, we strongly recommend migrating to alternative catalog solutions. Complete removal of LakeSoul support is planned for a future release.

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Oct 11, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@zy-kkk
Copy link
Member Author

zy-kkk commented Oct 11, 2025

run buildall

@hello-stephen
Copy link
Contributor

FE UT Coverage Report

Increment line coverage 100% (0/0) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.51% (17730/33768)
Line Coverage 37.68% (160931/427093)
Region Coverage 32.16% (122896/382133)
Branch Coverage 33.56% (53892/160603)

morningman
morningman previously approved these changes Oct 11, 2025
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Oct 11, 2025
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@morningman
Copy link
Contributor

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Oct 16, 2025
@hello-stephen
Copy link
Contributor

FE UT Coverage Report

Increment line coverage 0.00% (0/1) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.54% (17813/33904)
Line Coverage 37.74% (161861/428892)
Region Coverage 32.18% (123328/383236)
Branch Coverage 33.61% (54207/161298)

@morningman
Copy link
Contributor

run buildall

@doris-robot
Copy link

TPC-DS: Total hot run time: 189869 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 296f2b8bbcfc0db4706286ce07178248ddd692ea, data reload: false

query1	1053	479	416	416
query2	6565	1674	1687	1674
query3	6756	235	226	226
query4	26969	23756	23148	23148
query5	5255	628	485	485
query6	346	253	229	229
query7	4666	494	303	303
query8	310	279	279	279
query9	8716	2580	2570	2570
query10	537	350	310	310
query11	15859	15020	14874	14874
query12	188	124	109	109
query13	1692	553	438	438
query14	12537	9147	9155	9147
query15	209	188	186	186
query16	7662	721	530	530
query17	1580	789	623	623
query18	2094	469	343	343
query19	230	211	178	178
query20	145	193	155	155
query21	222	142	122	122
query22	4505	4562	4532	4532
query23	35516	34446	34062	34062
query24	8131	2525	2497	2497
query25	628	534	467	467
query26	1363	284	157	157
query27	2717	520	352	352
query28	4410	2270	2207	2207
query29	821	647	515	515
query30	305	263	210	210
query31	932	907	780	780
query32	88	75	77	75
query33	595	395	317	317
query34	812	831	536	536
query35	793	833	755	755
query36	949	996	868	868
query37	135	111	85	85
query38	3545	3508	3507	3507
query39	1516	1510	1595	1510
query40	216	124	120	120
query41	60	61	58	58
query42	126	118	106	106
query43	471	505	487	487
query44	1390	823	832	823
query45	189	184	187	184
query46	830	1006	646	646
query47	1777	1777	1739	1739
query48	400	433	320	320
query49	763	523	466	466
query50	650	700	407	407
query51	3855	3861	3895	3861
query52	114	107	98	98
query53	228	259	207	207
query54	602	592	530	530
query55	87	87	89	87
query56	342	325	306	306
query57	1179	1189	1105	1105
query58	297	273	267	267
query59	2663	2604	2451	2451
query60	355	344	320	320
query61	148	152	151	151
query62	789	742	668	668
query63	233	195	196	195
query64	4404	1155	826	826
query65	4044	3962	3963	3962
query66	1083	428	318	318
query67	15441	15130	14956	14956
query68	8030	937	594	594
query69	499	322	282	282
query70	1270	1284	1255	1255
query71	468	343	313	313
query72	5841	5113	4916	4916
query73	648	591	363	363
query74	8924	9132	8687	8687
query75	3783	3385	2865	2865
query76	3463	1169	751	751
query77	790	412	317	317
query78	9666	9782	8920	8920
query79	2757	805	592	592
query80	662	556	607	556
query81	518	262	230	230
query82	472	164	138	138
query83	269	271	261	261
query84	263	111	95	95
query85	871	478	414	414
query86	391	304	318	304
query87	3818	3703	3667	3667
query88	3883	2225	2218	2218
query89	385	331	302	302
query90	2042	223	223	223
query91	174	166	132	132
query92	82	77	62	62
query93	2292	993	639	639
query94	731	451	332	332
query95	407	326	318	318
query96	489	590	280	280
query97	2943	2977	2875	2875
query98	246	224	210	210
query99	1395	1424	1317	1317
Total cold run time: 282560 ms
Total hot run time: 189869 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.06 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 296f2b8bbcfc0db4706286ce07178248ddd692ea, data reload: false

query1	0.06	0.05	0.05
query2	0.10	0.06	0.06
query3	0.26	0.09	0.08
query4	1.61	0.12	0.12
query5	0.27	0.26	0.24
query6	1.16	0.67	0.65
query7	0.04	0.03	0.03
query8	0.06	0.05	0.05
query9	0.64	0.54	0.51
query10	0.57	0.58	0.58
query11	0.17	0.12	0.11
query12	0.15	0.12	0.12
query13	0.65	0.61	0.62
query14	1.02	1.04	1.02
query15	0.86	0.86	0.86
query16	0.40	0.40	0.42
query17	1.06	1.11	1.03
query18	0.21	0.20	0.19
query19	1.96	1.86	1.79
query20	0.01	0.01	0.02
query21	15.43	0.93	0.58
query22	0.77	1.24	0.74
query23	14.76	1.40	0.64
query24	7.61	1.14	0.48
query25	0.49	0.32	0.07
query26	0.62	0.15	0.13
query27	0.06	0.07	0.06
query28	9.61	1.40	0.92
query29	12.69	4.05	3.34
query30	0.28	0.13	0.12
query31	2.83	0.58	0.38
query32	3.25	0.56	0.47
query33	3.11	3.06	3.20
query34	16.08	5.48	4.83
query35	4.91	4.87	4.94
query36	0.73	0.53	0.51
query37	0.10	0.07	0.08
query38	0.06	0.04	0.04
query39	0.04	0.04	0.03
query40	0.17	0.14	0.13
query41	0.08	0.03	0.03
query42	0.04	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 105.02 s
Total hot run time: 30.06 s

@morningman
Copy link
Contributor

run check_coverage

@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Oct 17, 2025
@hello-stephen
Copy link
Contributor

FE Regression Coverage Report

Increment line coverage 0.00% (0/6) 🎉
Increment coverage report
Complete coverage report

@morningman morningman merged commit bb121e3 into apache:master Oct 17, 2025
26 of 28 checks passed
github-actions bot pushed a commit that referenced this pull request Oct 17, 2025
### What problem does this PR solve?

We are deprecating LakeSoul external catalog support in the current
Doris release due to several critical issues:

**Design Flaws** – The current driver management design causes conflicts
with other components, leading to instability and increased maintenance
complexity.

**Security Vulnerabilities** – LakeSoul dependencies contain multiple
CVE-reported vulnerabilities, posing security risks.

**Lack of Maintenance** – The LakeSoul catalog integration in Doris
lacks active maintainers and has not received timely updates.

**No User Adoption** – No user feedback or usage requests have been
received, indicating that the feature has not been used in practice.

Given these factors, maintaining this integration introduces unnecessary
security and maintenance burdens without providing tangible value to
users. Therefore, we have decided to **deprecate** LakeSoul catalog
support in this release:

- All LakeSoul-related code has been marked as `@Deprecated` and will be
kept for backward compatibility
- LakeSoul dependencies (including `lakesoul-io-java`, `postgresql`, and
`scala-library`) have been changed to `provided` scope and will not be
packaged in the release
  - Comment `extensions_modules+=("lakesoul-scanner")` in build.sh

Users who still need LakeSoul support can manually enable it by
modifying Maven configurations. However, we strongly recommend migrating
to alternative catalog solutions. Complete removal of LakeSoul support
is planned for a future release.
yiguolei pushed a commit that referenced this pull request Oct 18, 2025
…talog #56843 (#57123)

Cherry-picked from #56843

Co-authored-by: zy-kkk <zhongyk10@gmail.com>
morningman pushed a commit to morningman/doris that referenced this pull request Oct 20, 2025
…e#56843)

We are deprecating LakeSoul external catalog support in the current
Doris release due to several critical issues:

**Design Flaws** – The current driver management design causes conflicts
with other components, leading to instability and increased maintenance
complexity.

**Security Vulnerabilities** – LakeSoul dependencies contain multiple
CVE-reported vulnerabilities, posing security risks.

**Lack of Maintenance** – The LakeSoul catalog integration in Doris
lacks active maintainers and has not received timely updates.

**No User Adoption** – No user feedback or usage requests have been
received, indicating that the feature has not been used in practice.

Given these factors, maintaining this integration introduces unnecessary
security and maintenance burdens without providing tangible value to
users. Therefore, we have decided to **deprecate** LakeSoul catalog
support in this release:

- All LakeSoul-related code has been marked as `@Deprecated` and will be
kept for backward compatibility
- LakeSoul dependencies (including `lakesoul-io-java`, `postgresql`, and
`scala-library`) have been changed to `provided` scope and will not be
packaged in the release
  - Comment `extensions_modules+=("lakesoul-scanner")` in build.sh

Users who still need LakeSoul support can manually enable it by
modifying Maven configurations. However, we strongly recommend migrating
to alternative catalog solutions. Complete removal of LakeSoul support
is planned for a future release.
morningman pushed a commit to morningman/doris that referenced this pull request Oct 28, 2025
…e#56843)

We are deprecating LakeSoul external catalog support in the current
Doris release due to several critical issues:

**Design Flaws** – The current driver management design causes conflicts
with other components, leading to instability and increased maintenance
complexity.

**Security Vulnerabilities** – LakeSoul dependencies contain multiple
CVE-reported vulnerabilities, posing security risks.

**Lack of Maintenance** – The LakeSoul catalog integration in Doris
lacks active maintainers and has not received timely updates.

**No User Adoption** – No user feedback or usage requests have been
received, indicating that the feature has not been used in practice.

Given these factors, maintaining this integration introduces unnecessary
security and maintenance burdens without providing tangible value to
users. Therefore, we have decided to **deprecate** LakeSoul catalog
support in this release:

- All LakeSoul-related code has been marked as `@Deprecated` and will be
kept for backward compatibility
- LakeSoul dependencies (including `lakesoul-io-java`, `postgresql`, and
`scala-library`) have been changed to `provided` scope and will not be
packaged in the release
  - Comment `extensions_modules+=("lakesoul-scanner")` in build.sh

Users who still need LakeSoul support can manually enable it by
modifying Maven configurations. However, we strongly recommend migrating
to alternative catalog solutions. Complete removal of LakeSoul support
is planned for a future release.
morrySnow pushed a commit that referenced this pull request Oct 30, 2025
…talog #56843 (#57163)

bp #56843

---------

Co-authored-by: zy-kkk <zhongyk10@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/3.1.3-merged dev/4.0.1-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants