Skip to content

Conversation

@xinyiZzz
Copy link
Contributor

@xinyiZzz xinyiZzz commented Jan 27, 2022

Problem Summary:

Error details in be.out:

/home/disk3/zxy/baidu/bdg/doris/core/output/be/lib/palo_be: /lib64/libc.so.6: version `GLIBC_2.18' not found (required by /home/disk3/zxy/ baidu/bdg/doris/core/output/be/lib/palo_be)

nm looks at palo_be and finds that the method that depends on GLIBC_2.18 is __cxa_thread_atexit_impl
3be717fca2b8563d447aa1e1c8f1f39a

I found that the __cxa_thread_atexit_impl method was introduced by glibc to support thread_local variables
https://sourceware.org/glibc/wiki/Destructor%20support%20for%20thread_local%20variables

It was no problem to compile through gcc10 before

So, how can I avoid this problem, ask for help.

@xinyiZzz xinyiZzz force-pushed the thread_local_include branch from 1837c81 to 6a252d7 Compare January 27, 2022 10:38
@morningman
Copy link
Contributor

PTAL @amosbird , if you have time, thanks

@amosbird
Copy link
Contributor

Is it for some future PRs with thread local?

@morningman
Copy link
Contributor

morningman commented Jan 27, 2022

Is it for some future PRs with thread local?

Yes, @xinyiZzz is developing memory tracking and management based on thread local.
And this is just a PR for troubleshooting.

@amosbird
Copy link
Contributor

Is it for some future PRs with thread local?

Yes, @xinyiZzz is developing memory tracking and management based on thread local. And this is just a PR for troubleshooting.

Cool. Will take a look.

@amosbird
Copy link
Contributor

amosbird commented Jan 27, 2022

I did some investigation. It's not easy to mask this symbol, which is meant to be added to the libc++abi. It's also not safe to use the polyfill implementation from some libc++api directly. I'd suggest the following:

  1. use clang toolchain
  2. keep thread_local with trivial destructor if gcc is used.

@xinyiZzz
Copy link
Contributor Author

I did some investigation. It's not easy to mask this symbol, which is meant to be added to the libc++abi. It's also not safe to use the polyfill implementation from some libc++api directly. I'd suggest the following:

  1. use clang toolchain
  2. keep thread_local with trivial destructor if gcc is used.

Thanks for the answer~ @amosbird
Compiling with clang is indeed fine.
Regarding the second point, do you mean changing the thread_local variable to a member variable of the class? For example, abstract a parent class to save this variable. or other meanings.

My pr: #7198
I want to track all memory requests during a thread lifetime via thread_local + TCMalloc hook, like Clickhouse does.

@amosbird
Copy link
Contributor

Regarding the second point, do you mean changing the thread_local variable to a member variable of the class? For example, abstract a parent class to save this variable. or other meanings.

You can redesign the implementation of your thread_local structs to have trivial destructor. The definition is in https://en.cppreference.com/w/cpp/language/destructor .

@xinyiZzz
Copy link
Contributor Author

xinyiZzz commented Jan 28, 2022

Regarding the second point, do you mean changing the thread_local variable to a member variable of the class? For example, abstract a parent class to save this variable. or other meanings.

You can redesign the implementation of your thread_local structs to have trivial destructor. The definition is in https://en.cppreference.com/w/cpp/language/destructor .

I think I understand trivial destructor,
For non-trivial variables in thread_local, such as std::string, I plan to store them as pointers.
To ensure that thread_local is trivial, these non-trivial pointers will uniformly call destructors elsewhere.
And try to use POD type.

thanks again, very helpful @amosbird

@xinyiZzz xinyiZzz closed this Mar 7, 2022
morningman pushed a commit that referenced this pull request Mar 10, 2022
The thread context saves some info about a working thread.
1. thread_id:   Current thread id, Auto generated.
2. type:           The type is a enum value indicating which type of task current thread is running.
                        For example: QUERY, LOAD, COMPACTION, ...
3. task id:       A unique id to identify this task. maybe query id, load job id, etc.

Using gcc11 compiles thread_local variable on lower versions of GLIBC will report an error, see #7911
This is very difficult to solve, so kudu Class-scoped static thread local implementation was introduced.
Solve the above problem by Thread-scopedthread local + Class-scoped thread local.
See the comments for ThreadContextPtr for details.
cambyzju pushed a commit to cambyzju/incubator-doris that referenced this pull request Mar 12, 2022
…he#7234)

The thread context saves some info about a working thread.
1. thread_id:   Current thread id, Auto generated.
2. type:           The type is a enum value indicating which type of task current thread is running.
                        For example: QUERY, LOAD, COMPACTION, ...
3. task id:       A unique id to identify this task. maybe query id, load job id, etc.

Using gcc11 compiles thread_local variable on lower versions of GLIBC will report an error, see apache#7911
This is very difficult to solve, so kudu Class-scoped static thread local implementation was introduced.
Solve the above problem by Thread-scopedthread local + Class-scoped thread local.
See the comments for ThreadContextPtr for details.
morningman pushed a commit that referenced this pull request Mar 14, 2022
The thread context saves some info about a working thread.
1. thread_id:   Current thread id, Auto generated.
2. type:           The type is a enum value indicating which type of task current thread is running.
                        For example: QUERY, LOAD, COMPACTION, ...
3. task id:       A unique id to identify this task. maybe query id, load job id, etc.

Using gcc11 compiles thread_local variable on lower versions of GLIBC will report an error, see #7911
This is very difficult to solve, so kudu Class-scoped static thread local implementation was introduced.
Solve the above problem by Thread-scopedthread local + Class-scoped thread local.
See the comments for ThreadContextPtr for details.
zhengte pushed a commit to zhengte/incubator-doris that referenced this pull request Mar 15, 2022
…he#7234)

The thread context saves some info about a working thread.
1. thread_id:   Current thread id, Auto generated.
2. type:           The type is a enum value indicating which type of task current thread is running.
                        For example: QUERY, LOAD, COMPACTION, ...
3. task id:       A unique id to identify this task. maybe query id, load job id, etc.

Using gcc11 compiles thread_local variable on lower versions of GLIBC will report an error, see apache#7911
This is very difficult to solve, so kudu Class-scoped static thread local implementation was introduced.
Solve the above problem by Thread-scopedthread local + Class-scoped thread local.
See the comments for ThreadContextPtr for details.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants