Skip to content

branch-2.1: [fix](jdbc catalog) Change BE jdbc Driver loading to Java code #48002

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
yiguolei merged 1 commit into from
Feb 21, 2025
Merged

branch-2.1: [fix](jdbc catalog) Change BE jdbc Driver loading to Java code #48002

yiguolei merged 1 commit into from
Feb 21, 2025

Conversation

@zy-kkk
Copy link
Member

@zy-kkk zy-kkk commented Feb 18, 2025

cherry-pick from (#46912)

@zy-kkk
[fix](jdbc catalog) Change BE jdbc Driver loading to Java code (apach…
b0b657a 
…e#46912)

Related PR: apache#45806

Problem Summary:

In the previous BE processing of JDBC Driver, the Driver jar will be downloaded to the local cache directory first, and the cached jar package will be provided to the JVM for loading. This will cause two problems
1. The jar package in the cache may fail to load due to duplication
2. Frequent repeated loading of the driver by the JVM will cause Compressed class space OOM

In order to fix these two problems, this PR has the following changes
1. Remove the logic of BE downloading the Driver Jar to the local cache directory, and directly hand over the original path to Java's Classloader for processing
2. Treat the jar packages with the same name in the same path as one and cache them in the map to avoid repeated loading and cause Compressed class space OOM
@Thearas
Copy link
Contributor

Thearas commented Feb 18, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@zy-kkk
Copy link
Member Author

zy-kkk commented Feb 18, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Feb 18, 2025

TeamCity be ut coverage result:
Function Coverage: 36.50% (9610/26327)
Line Coverage: 28.09% (79670/283594)
Region Coverage: 26.75% (40867/152764)
Branch Coverage: 23.50% (20712/88124)
Coverage Report: http://coverage.selectdb-in.cc/coverage/b0b657a55851701b6f15661a90f4691ca417418d_b0b657a55851701b6f15661a90f4691ca417418d/report/index.html

@zy-kkk zy-kkk changed the title [fix](jdbc catalog) Change BE jdbc Driver loading to Java code branch-2.1: [fix](jdbc catalog) Change BE jdbc Driver loading to Java code Feb 18, 2025
@yiguolei yiguolei merged commit 469bc77 into apache:branch-2.1 Feb 21, 2025
20 of 21 checks passed
@zy-kkk zy-kkk deleted the pick-46912-to-2.1 branch March 17, 2025 08:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@zy-kkk @Thearas @doris-robot @yiguolei