[Fix](jdk) Fix jdk17 crash on some envs. #32865
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This reverts commit 5c7964d.
|
Thank you for your contribution to Apache Doris. Since 2024-03-18, the Document has been moved to doris-website. |
Contributor
|
clang-tidy review says "All clean, LGTM! 👍" |
kaka11chen
force-pushed
the
fix_jdk17_crash
branch
from
92666da to
2444954
Compare
Contributor
|
Contributor
|
clang-tidy review says "All clean, LGTM! 👍" |
Contributor
|
good fix!, jemalloc is used in JVM, use dlopen to dynamically load JVM has a problem, see: |
xinyiZzz
added a commit
that referenced
this pull request
May 10, 2024
…#34578) Now in order to support malloc hook, compiling jemalloc will add tow compilation options --with-jemalloc-prefix=je and --disable-cxx, then overwrite malloc/free as follows to allow BE to use jemalloc: #define ALIAS(doris_fn) __attribute__((alias(#doris_fn), used)) void* malloc(size_t size) __THROW ALIAS(doris_malloc); void free(void* p) __THROW ALIAS(doris_free); void* realloc(void* p, size_t size) __THROW ALIAS(doris_realloc); void* calloc(size_t n, size_t size) __THROW ALIAS(doris_calloc); void cfree(void* ptr) __THROW ALIAS(doris_cfree); but after such overwrite like this, Doris BE on jdk17 cannot be started in some environments. there are three solutions: Modify overwrite malloc/free, fixed in Use jemalloc's experimental features to hook jemalloc #33897 Modify BE dynamically loads JVM, fixed in [Fix](jdk) Fix jdk17 crash on some envs. #32865 Commit of this PR, modify jemalloc compilation options, after remove --with-jemalloc-prefix=je, there is not need to overwrite malloc/free, BE will use Jemalloc by default. So, if need to use memory hook, this PR still cannot solve this problem. Looking forward to modify BE dynamically loads JVM, which will not only solve this problem, also fix other crashes caused by BE use JVM.
xinyiZzz
added a commit
to xinyiZzz/incubator-doris
that referenced
this pull request
May 11, 2024
…apache#34578) Now in order to support malloc hook, compiling jemalloc will add tow compilation options --with-jemalloc-prefix=je and --disable-cxx, then overwrite malloc/free as follows to allow BE to use jemalloc: void* malloc(size_t size) __THROW ALIAS(doris_malloc); void free(void* p) __THROW ALIAS(doris_free); void* realloc(void* p, size_t size) __THROW ALIAS(doris_realloc); void* calloc(size_t n, size_t size) __THROW ALIAS(doris_calloc); void cfree(void* ptr) __THROW ALIAS(doris_cfree); but after such overwrite like this, Doris BE on jdk17 cannot be started in some environments. there are three solutions: Modify overwrite malloc/free, fixed in Use jemalloc's experimental features to hook jemalloc apache#33897 Modify BE dynamically loads JVM, fixed in [Fix](jdk) Fix jdk17 crash on some envs. apache#32865 Commit of this PR, modify jemalloc compilation options, after remove --with-jemalloc-prefix=je, there is not need to overwrite malloc/free, BE will use Jemalloc by default. So, if need to use memory hook, this PR still cannot solve this problem. Looking forward to modify BE dynamically loads JVM, which will not only solve this problem, also fix other crashes caused by BE use JVM.
ByteYue
pushed a commit
to ByteYue/doris
that referenced
this pull request
May 15, 2024
…apache#34578) Now in order to support malloc hook, compiling jemalloc will add tow compilation options --with-jemalloc-prefix=je and --disable-cxx, then overwrite malloc/free as follows to allow BE to use jemalloc: #define ALIAS(doris_fn) __attribute__((alias(#doris_fn), used)) void* malloc(size_t size) __THROW ALIAS(doris_malloc); void free(void* p) __THROW ALIAS(doris_free); void* realloc(void* p, size_t size) __THROW ALIAS(doris_realloc); void* calloc(size_t n, size_t size) __THROW ALIAS(doris_calloc); void cfree(void* ptr) __THROW ALIAS(doris_cfree); but after such overwrite like this, Doris BE on jdk17 cannot be started in some environments. there are three solutions: Modify overwrite malloc/free, fixed in Use jemalloc's experimental features to hook jemalloc apache#33897 Modify BE dynamically loads JVM, fixed in [Fix](jdk) Fix jdk17 crash on some envs. apache#32865 Commit of this PR, modify jemalloc compilation options, after remove --with-jemalloc-prefix=je, there is not need to overwrite malloc/free, BE will use Jemalloc by default. So, if need to use memory hook, this PR still cannot solve this problem. Looking forward to modify BE dynamically loads JVM, which will not only solve this problem, also fix other crashes caused by BE use JVM.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Proposed changes
Issue Number: close #xxx
Further comments
If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...