[C++] Linking error with pre-built Boost binaries on Windows

[pitrou]

Describe the bug, including details regarding any error messages, version, and platform.

Using pre-built Boost binaries on Windows, I get this linker error:

[128/292] Linking CXX shared library debug\arrow_testing.dll
FAILED: debug/arrow_testing.dll debug/arrow_testing.lib
C:\WINDOWS\system32\cmd.exe /C "cd . && "C:\Program Files\CMake\bin\cmake.exe" -E vs_link_dll --msvc-ver=1943 --intdir=src\arrow\CMakeFiles\arrow_testing_shared.dir --rc=C:\PROGRA~2\WI3CF2~1\10\bin\100203~1.0\x64\rc.exe --mt=C:\PROGRA~2\WI3CF2~1\10\bin\100203~1.0\x64\mt.exe --manifests  -- C:\PROGRA~1\MICROS~2\2022\COMMUN~1\VC\Tools\MSVC\1443~1.348\bin\Hostx64\x64\link.exe /nologo src\arrow\CMakeFiles\arrow_testing_shared.dir\io\test_common.cc.obj src\arrow\CMakeFiles\arrow_testing_shared.dir\ipc\test_common.cc.obj src\arrow\CMakeFiles\arrow_testing_shared.dir\testing\fixed_width_test_util.cc.obj src\arrow\CMakeFiles\arrow_testing_shared.dir\testing\generator.cc.obj src\arrow\CMakeFiles\arrow_testing_shared.dir\testing\gtest_util.cc.obj src\arrow\CMakeFiles\arrow_testing_shared.dir\testing\math.cc.obj src\arrow\CMakeFiles\arrow_testing_shared.dir\testing\process.cc.obj src\arrow\CMakeFiles\arrow_testing_shared.dir\testing\random.cc.obj src\arrow\CMakeFiles\arrow_testing_shared.dir\testing\util.cc.obj src\arrow\CMakeFiles\arrow_testing_shared.dir\filesystem\test_util.cc.obj  /out:debug\arrow_testing.dll /implib:debug\arrow_testing.lib /pdb:debug\arrow_testing.pdb /dll /version:2000.0 /machine:x64  /NODEFAULTLIB:LIBCMT /debug /INCREMENTAL  debug\arrow.lib  debug\arrow_gtestd.lib  ws2_32.lib  debug\arrow_gmockd.lib  C:\local\boost_1_87_0\lib64-msvc-14.3\libboost_process-vc143-mt-gd-x64-1_87.lib  kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib && cd ."
LINK Pass 1: command "C:\PROGRA~1\MICROS~2\2022\COMMUN~1\VC\Tools\MSVC\1443~1.348\bin\Hostx64\x64\link.exe /nologo src\arrow\CMakeFiles\arrow_testing_shared.dir\io\test_common.cc.obj src\arrow\CMakeFiles\arrow_testing_shared.dir\ipc\test_common.cc.obj src\arrow\CMakeFiles\arrow_testing_shared.dir\testing\fixed_width_test_util.cc.obj src\arrow\CMakeFiles\arrow_testing_shared.dir\testing\generator.cc.obj src\arrow\CMakeFiles\arrow_testing_shared.dir\testing\gtest_util.cc.obj src\arrow\CMakeFiles\arrow_testing_shared.dir\testing\math.cc.obj src\arrow\CMakeFiles\arrow_testing_shared.dir\testing\process.cc.obj src\arrow\CMakeFiles\arrow_testing_shared.dir\testing\random.cc.obj src\arrow\CMakeFiles\arrow_testing_shared.dir\testing\util.cc.obj src\arrow\CMakeFiles\arrow_testing_shared.dir\filesystem\test_util.cc.obj /out:debug\arrow_testing.dll /implib:debug\arrow_testing.lib /pdb:debug\arrow_testing.pdb /dll /version:2000.0 /machine:x64 /NODEFAULTLIB:LIBCMT /debug /INCREMENTAL debug\arrow.lib debug\arrow_gtestd.lib ws2_32.lib debug\arrow_gmockd.lib C:\local\boost_1_87_0\lib64-msvc-14.3\libboost_process-vc143-mt-gd-x64-1_87.lib kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib /MANIFEST /MANIFESTFILE:src\arrow\CMakeFiles\arrow_testing_shared.dir/intermediate.manifest src\arrow\CMakeFiles\arrow_testing_shared.dir/manifest.res" failed (exit code 1104) with the following output:
LINK : fatal error LNK1104: cannot open file 'libboost_filesystem-vc143-mt-gd-x64-1_87.lib'

This was with the command line:

cmake .. -GNinja -DCMAKE_BUILD_TYPE=Debug -DARROW_BUILD_SHARED=ON -DARROW_BUILD_STATIC=OFF -DBOOST_ROOT=c:/local/boost_1_87_0 -DBOOST_SOURCE=SYSTEM -DARROW_BUILD_TESTS=ON -DARROW_PARQUET=ON -DPARQUET_REQUIRE_ENCRYPTION=ON

Yet the file exists:

dir c:\local\boost_1_87_0\lib64-msvc-14.3\libboost_filesystem-vc143-mt-gd-x64-1_87.lib  Volume in drive C has no label.
 Volume Serial Number is E412-056F

 Directory of c:\local\boost_1_87_0\lib64-msvc-14.3

2024-12-05  21:56         8,844,152 libboost_filesystem-vc143-mt-gd-x64-1_87.lib
               1 File(s)      8,844,152 bytes
               0 Dir(s)  24,412,188,672 bytes free

Component(s)

C++

[pitrou]

cc @kou

[kou]

Where did you download pre-built Boost binaries?

https://sourceforge.net/projects/boost/files/boost-binaries/1.87.0/ ?

It seems that Boost::process CMake target misses Boost::filesystem dependencies.

Could you share full cmake ... log?

[pitrou]

Here is the cmake log: https://gist.github.com/pitrou/0dc9dce94685caf4e9d75d00810da263

[pitrou]

https://sourceforge.net/projects/boost/files/boost-binaries/1.87.0/ ?

Yes!

[pitrou]

I've tried with 1.86.0 instead and got the same error...

[kou]

Thanks. Could you try this?

diff --git a/cpp/CMakeLists.txt b/cpp/CMakeLists.txt
index f2500b3a72..10f54bcd57 100644
--- a/cpp/CMakeLists.txt
+++ b/cpp/CMakeLists.txt
@@ -71,6 +71,13 @@ if(POLICY CMP0135)
   cmake_policy(SET CMP0135 NEW)
 endif()
 
+# https://cmake.org/cmake/help/latest/policy/CMP0167.html
+#
+# Use Boost's CMake packages instead of FindBoost.cmake in CMake.
+if(POLICY CMP0167)
+  cmake_policy(SET CMP0167 NEW)
+endif()
+
 # https://cmake.org/cmake/help/latest/policy/CMP0170.html
 #
 # CMP0170 is for enforcing dependency populations by users with

[pitrou]

Thanks. Could you try this?

I'm trying this out. It seems it may be desirable to also enable https://cmake.org/cmake/help/latest/policy/CMP0144.html ?

[pitrou]

Okay, building seems to work with:

cmake .. -GNinja -DCMAKE_BUILD_TYPE=Debug -DARROW_BUILD_SHARED=ON -DARROW_BUILD_STATIC=OFF -DCMAKE_POLICY_DEFAULT_CMP0167=NEW  -DBoost_ROOT=C:\local\boost_1_87_0\lib64-msvc-14.3\cmake -DBOOST_SOURCE=SYSTEM -DARROW_BOOST_USE_SHARED=ON -DARROW_BUILD_TESTS=ON -DARROW_PARQUET=ON -DPARQUET_REQUIRE_ENCRYPTION=ON

[kou]

Yay!

#45623

CMP0144 may be useful when we work on #26842.

[pitrou]

Trying to execute the tests then fails with the following error if ARROW_BOOST_USE_SHARED=ON. However, if ARROW_BOOST_USE_SHARED=OFF, the tests run fine.