ARROW-7605: [C++] Bundle private jemalloc symbols into static library libarrow.a

[wesm]

If something like this is not done, then libarrow.a cannot be used without obtaining the exact libjemalloc_pic.a that we build privately.

Open questions I need help with

• Does this break any packaging builds?
• How does this affect the INSTALL CMake commands (and ArrowTargets.cmake, etc.)? Ideally we want people to be able to use Arrow as a dependency in other CMake projects and for things to Just Work
• Test with ARROW_MIMALLOC=ON on Windows

[nealrichardson]

Should we do this for all bundled dependencies in a static build? See also https://github.com/apache/arrow/blob/master/r/inst/build_arrow_static.sh#L93-L95

[wesm]

@nealrichardson it would indeed make linking for third parties much easier. One problem with doing this for other external dependencies is that they may be part of a larger static library toolchain.

As an example, consider an application that uses libzstd.a in more than one place. If we bundle libzstd.a in libarrow.a then this makes things more difficult for the third party developer.

On the other hand, there's nothing harmful about bundling libjemalloc_pic.a because these symbols are exclusive to Arrow

[wesm]

That being said it might not be a bad idea to have an optional build option to generate a libarrow_all_dependencies.a

[nealrichardson]

Right, but if we're building it in the arrow build process (ARROW_DEPENDENCY_SOURCE=bundled), no one else is using the one we build. And in fact we can't use the static arrow library we build unless we fish those dependencies out of the _ep directories that cmake makes.

[github-actions]

https://issues.apache.org/jira/browse/ARROW-7605

[wesm]

@nealrichardson yes, I agree that when we use the BUNDLED approach for a dependency, we should splice that dependency into libarrow.a rather than install a useless static library

[kou]

Does this break any packaging builds?

RPM build will be broken because we install new files. We need to update install file list: https://github.com/apache/arrow/blob/master/dev/tasks/linux-packages/apache-arrow/yum/arrow.spec.in#L239

How does this affect the INSTALL CMake commands (and ArrowTargets.cmake, etc.)? Ideally we want people to be able to use Arrow as a dependency in other CMake projects and for things to Just Work

It affects nothing. arrow_static in ArrowTargets.cmake still doesn't work.
arrow_static has INTERFACE_LINK_LIBRARIES property and it includes jemalloc::jemalloc and mimalloc::mimalloc. But we don't provide jemalloc::jemalloc and minmalloc::minmalloc yet. We need to provide these targets to ArrowTargets.cmake to support arrow_static.

[kou]

The following will be better because we can use both jemalloc and mimalloc at the same time.

set(ARROW_BUNDLE_STATIC_LIBS)

if(ARROW_JEMALLOC)
  list(APPEND ARROW_BUNDLE_STATIC_LIBS jemalloc::jemalloc)
endif()

if(ARROW_MIMALLOC)
  list(APPEND ARROW_BUNDLE_STATIC_LIBS mimalloc::mimalloc)
endif()

[kou]

We want to lib.exe instead of libtool on Windows, right?
We will be able to use ${CMAKE_LINKER} on MSVC case: set(BUNDLE_TOOL ${CMAKE_LINKER})

[kou]

How about _plain or _raw?

[wesm]

I'll take another turn on this tomorrow (Monday). I will see if I can assemble a list of all bundled libraries (not just jemalloc/mimalloc) to merge into libarrow.a.

[wesm]

We'll have to tackle this more completely after 0.16.0 rather than rush something through right now.

[fsaintjacques]

Could you add a comment saying what this does and why? This is a potential future head scratcher for whoever will debug this.

[wesm]

Yes, I certainly will

[wesm]

I'm going to clean this up as soon as I can and try to include any BUNDLED dependencies in libarrow.a

More work is needed on this

[tobim]

Bundling jemalloc into libarrow.a is probably the right course, but altering the generated archive in a way that CMake doesn't understand complicates the target export quite a bit.

Did you consider directly adding the object files from jemalloc to the list of source files of libarrow? The approach is described on Stackoverflow.

Another alternative is to install the custom jemalloc.a. Here is a patch that outlines that approach (This does not take care of pkgconfig yet).

[wesm]

@tobim you're welcome to propose an alternative solution and demonstrate how third party applications would be expected to statically link. Since we have install both CMake targets and pkg-config there's a number of things to test

[wesm]

Closing this PR for now. Hopefully someone can pick up this project

[wesm]

This work has been superseded by #7696