Musl doesn't support init-exec tls

[gbaraldi]

Hi!. We are studying using mimalloc as the malloc for julia. While doing this, we found an error on musl, where if you statically link mimalloc to a shared library, then dlopen it. You get Error relocating /workspace/mimalloc/build/test.so: __errno_location: initial-exec TLS resolves to dynamic definition in /workspace/mimalloc/build/test.so.

While the issue is simple enough to solve in our case, just build it with -DMI_LOCAL_DYNAMIC_TLS=ON on musl, it would be more correct if the build system detected it was targeting musl and did that automatically.

[daanx]

Ah, very cool to try this with Julia! Contact me if you need further help. For language integration, it can help by using mi_heap_malloc_small for example for any allocation under 1k (MI_SMALL_SIZE_MAX) -- really speeds up things for Koka and Lean.

In particular, if the language (runtime) already has a thread-local context, you may keep a thread-local heap pointer there and use the mi_heap_xxx api which avoids needing to get the default heap pointer from the TLS which may be slower if you do not use initial-exec TLS.

Not sure about detecting this automatically as usually we can use initial-exec on MUSL right? This is only an issue when linking mimalloc statically into a shared-object file? (also, I think linking mimalloc dynamically would work as well right (with initial-exec)?)

[Wodann]

This is only an issue when linking mimalloc statically into a shared-object file?

We ran into the same issue with mimalloc on musl and I can confirm that our use case was specifically for statically linking mimalloc into a shared library.

Using MI_LOCAL_DYNAMIC_TLS fixed the problem.

[daanx]

Thanks! Would there be an easy way to detect musl is targeted such that we can set the -DMI_LOCAL_DYNAMIC_TLS=ON flag automatically in cmake ?

[giordano]

Would there be an easy way to detect musl is targeted

No. Musl developers maintain the position that you don't need to do that, so they don't provide any preprocessor macro to identify a Musl target: https://wiki.musl-libc.org/faq. The only thing you can do is to rule out other similar systems you plan to support, e.g. "this is a Linux system, it isn't glibc, I don't support other libc apart from that one and musl, this better be musl then".

[res2k]

FWIW, CMake should be able to detect the problematic case by, well, replicating it and seeing if it links.
May not fit all definitions of "easy" though.

[daanx]

... May not fit all definitions of "easy" though.

:-). Mmm, maybe I'll add a MI_MUSL=ON switch to the cmake that will define and add the right definitions to make it work (for now, that would only be setting -DMI_LOCAL_DYNAMIC_TLS=ON which is otherwise hard to guess)

[jirutka]

What exactly does MI_LOCAL_DYNAMIC_TLS do and when is it needed? We package mimalloc2 in Alpine Linux since v3.17 with MI_LOCAL_DYNAMIC_TLS left in default (disabled) and it works. If it’s something needed only when mimalloc is linked statically, then this new MI_MUSL option is a bit misleading.

[daanx]

Ah, I thought MI_LOCAL_DYNAMIC_TLS was always needed but it may well be it only is needed for static builds. Can anyone confirm this? If so, the MI_LIBC_MUSL option is good since we can then only add the local dynamic tls for the static builds (as the default is more efficient for dynamic libraries)

[Wodann]

Ah, I thought MI_LOCAL_DYNAMIC_TLS was always needed but it may well be it only is needed for static builds. Can anyone confirm this? If so, the MI_LIBC_MUSL option is good since we can then only add the local dynamic tls for the static builds (as the default is more efficient for dynamic libraries)

For my case, it is specifically needed when you're building a shared object/library which depends on a statically linked version of mimalloc.

[nekopsykose]

Not sure about detecting this automatically as usually we can use initial-exec on MUSL right? This is only an issue when linking mimalloc statically into a shared-object file? (also, I think linking mimalloc dynamically would work as well right (with initial-exec)?)

it's not related to static linkage; musl does not support dlopen() of a shared object that uses initial-exec tls. using it in something directly linked (e.g. you run an executable that has DT_NEEDED on a library that then has DT_NEEDED on mimalloc which uses initial-exec) is ok. but if anything is dlopend and the dlopen lib or any DT_NEEDED that the dlopen lib uses initial_exec (so it doesn't matter if mimalloc is statically linked or not to this lib, like in julia) would fail.