[R] Linux static build should always bundle dependencies

[asfimport]

I'm following the instructions here: https://arrow.apache.org/install/

arrow::install_arrow()

gives error:

./configure: line 132: cd: libarrow/arrow-0.17.1/lib: No such file or directory

This leaves me without a working arrow::read_feather.

Environment: CentOS 7.6.1810
R 4.0.2
Reporter: Nathan TeBlunthuis
Assignee: Neal Richardson / @nealrichardson

Related issues:

• [C++] Create and install static library containing all dependencies built by Arrow (relates to)

Original Issue Attachments:

• build_arrow_2.txt
• build_arrow_3.txt
• build_arrow.txt

PRs and other links:

• GitHub Pull Request #7699

_{Note: This issue was originally created as ARROW-9303. Please see the migration documentation for further details.}

[asfimport]

Neal Richardson / @nealrichardson:
Could be this: https://github.com/apache/arrow/blob/master/r/vignettes/install.Rmd#L261-L263

If not, please provide more information. ARROW-8586 is a good model for the kinds of questions we'll need to answer in order to help.

[asfimport]

Nathan TeBlunthuis:
Thanks for the response.  I double-checked that the issue is not the CC and CCX environment variables.  From the linked issue it seems like the C++ build was failing.  I tried again and saw an error that my cmake version was less than 3.2.   I tried again with an updated cmake and it seems to work.  I suggest that raising this error if the build fails or at least updating the R documentation to indicate dependency on cmake>-3.2 may improve the user experience for those using R on HPCs.

[asfimport]

Nathan TeBlunthuis:
Ok so it turns out that the cmake wasn't my only problem

The version that built didn't have support for any compression codecs, so I tried setting LIBARROW_BINARY=centos-7 and reinstalling to see if the C++ binaries will work. 

R fails to load the package built this way with error: Error: package or namespace load failed for ‘arrow’ in dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object '/.../R_4.0.2_libs/00LOCK-arrow/00new/arrow/libs/arrow.so':
/.../R_4.0.2_libs/00LOCK-arrow/00new/arrow/libs/arrow.so: undefined symbol: _ZN5arrow5fieldENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt10shared_ptrINS_8DataTypeEEbS6_IKNS_16KeyValueMetadataEE

*** Successfully retrieved C++ binaries for centos-7
PKG_CFLAGS=-I/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/include -DARROW_R_WITH_ARROW
PKG_LIBS=-L/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/lib -larrow_dataset -lparquet -larrow -larrow -larrow_dataset -lboost_filesystem -lboost_regex -lboost_system -lbrotlidec-static -lbrotlienc-static -lbrotlizzz-static -lbz2 -ljemalloc_pic -llz4 -lparquet -lsnappy -lthrift -lz -lzstd
** libs
g++ -std=gnu++11 -I"/.../R/4.0.2/lib64/R/include" -DNDEBUG -I/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/include -DARROW_R_WITH_ARROW -I'/.../R/4.0.2/lib64/R/library/Rcpp/include' -I/usr/local/include -fpic -O3 -march=native -mtune=native -fPIC -c array.cpp -o array.o
g++ -std=gnu++11 -I"/.../R/4.0.2/lib64/R/include" -DNDEBUG -I/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/include -DARROW_R_WITH_ARROW -I'/.../R/4.0.2/lib64/R/library/Rcpp/include' -I/usr/local/include -fpic -O3 -march=native -mtune=native -fPIC -c array_from_vector.cpp -o array_from_vector.o
g++ -std=gnu++11 -I"/.../R/4.0.2/lib64/R/include" -DNDEBUG -I/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/include -DARROW_R_WITH_ARROW -I'/.../R/4.0.2/lib64/R/library/Rcpp/include' -I/usr/local/include -fpic -O3 -march=native -mtune=native -fPIC -c array_to_vector.cpp -o array_to_vector.o
g++ -std=gnu++11 -I"/.../R/4.0.2/lib64/R/include" -DNDEBUG -I/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/include -DARROW_R_WITH_ARROW -I'/.../R/4.0.2/lib64/R/library/Rcpp/include' -I/usr/local/include -fpic -O3 -march=native -mtune=native -fPIC -c arraydata.cpp -o arraydata.o
g++ -std=gnu++11 -I"/.../R/4.0.2/lib64/R/include" -DNDEBUG -I/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/include -DARROW_R_WITH_ARROW -I'/.../R/4.0.2/lib64/R/library/Rcpp/include' -I/usr/local/include -fpic -O3 -march=native -mtune=native -fPIC -c arrowExports.cpp -o arrowExports.o
g++ -std=gnu++11 -I"/.../R/4.0.2/lib64/R/include" -DNDEBUG -I/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/include -DARROW_R_WITH_ARROW -I'/.../R/4.0.2/lib64/R/library/Rcpp/include' -I/usr/local/include -fpic -O3 -march=native -mtune=native -fPIC -c buffer.cpp -o buffer.o
g++ -std=gnu++11 -I"/.../R/4.0.2/lib64/R/include" -DNDEBUG -I/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/include -DARROW_R_WITH_ARROW -I'/.../R/4.0.2/lib64/R/library/Rcpp/include' -I/usr/local/include -fpic -O3 -march=native -mtune=native -fPIC -c chunkedarray.cpp -o chunkedarray.o
g++ -std=gnu++11 -I"/.../R/4.0.2/lib64/R/include" -DNDEBUG -I/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/include -DARROW_R_WITH_ARROW -I'/.../R/4.0.2/lib64/R/library/Rcpp/include' -I/usr/local/include -fpic -O3 -march=native -mtune=native -fPIC -c compression.cpp -o compression.o
g++ -std=gnu++11 -I"/.../R/4.0.2/lib64/R/include" -DNDEBUG -I/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/include -DARROW_R_WITH_ARROW -I'/.../R/4.0.2/lib64/R/library/Rcpp/include' -I/usr/local/include -fpic -O3 -march=native -mtune=native -fPIC -c compute.cpp -o compute.o
g++ -std=gnu++11 -I"/.../R/4.0.2/lib64/R/include" -DNDEBUG -I/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/include -DARROW_R_WITH_ARROW -I'/.../R/4.0.2/lib64/R/library/Rcpp/include' -I/usr/local/include -fpic -O3 -march=native -mtune=native -fPIC -c csv.cpp -o csv.o
g++ -std=gnu++11 -I"/.../R/4.0.2/lib64/R/include" -DNDEBUG -I/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/include -DARROW_R_WITH_ARROW -I'/.../R/4.0.2/lib64/R/library/Rcpp/include' -I/usr/local/include -fpic -O3 -march=native -mtune=native -fPIC -c dataset.cpp -o dataset.o
g++ -std=gnu++11 -I"/.../R/4.0.2/lib64/R/include" -DNDEBUG -I/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/include -DARROW_R_WITH_ARROW -I'/.../R/4.0.2/lib64/R/library/Rcpp/include' -I/usr/local/include -fpic -O3 -march=native -mtune=native -fPIC -c datatype.cpp -o datatype.o
g++ -std=gnu++11 -I"/.../R/4.0.2/lib64/R/include" -DNDEBUG -I/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/include -DARROW_R_WITH_ARROW -I'/.../R/4.0.2/lib64/R/library/Rcpp/include' -I/usr/local/include -fpic -O3 -march=native -mtune=native -fPIC -c expression.cpp -o expression.o
g++ -std=gnu++11 -I"/.../R/4.0.2/lib64/R/include" -DNDEBUG -I/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/include -DARROW_R_WITH_ARROW -I'/.../R/4.0.2/lib64/R/library/Rcpp/include' -I/usr/local/include -fpic -O3 -march=native -mtune=native -fPIC -c feather.cpp -o feather.o
g++ -std=gnu++11 -I"/.../R/4.0.2/lib64/R/include" -DNDEBUG -I/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/include -DARROW_R_WITH_ARROW -I'/.../R/4.0.2/lib64/R/library/Rcpp/include' -I/usr/local/include -fpic -O3 -march=native -mtune=native -fPIC -c field.cpp -o field.o
g++ -std=gnu++11 -I"/.../R/4.0.2/lib64/R/include" -DNDEBUG -I/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/include -DARROW_R_WITH_ARROW -I'/.../R/4.0.2/lib64/R/library/Rcpp/include' -I/usr/local/include -fpic -O3 -march=native -mtune=native -fPIC -c filesystem.cpp -o filesystem.o
g++ -std=gnu++11 -I"/.../R/4.0.2/lib64/R/include" -DNDEBUG -I/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/include -DARROW_R_WITH_ARROW -I'/.../R/4.0.2/lib64/R/library/Rcpp/include' -I/usr/local/include -fpic -O3 -march=native -mtune=native -fPIC -c io.cpp -o io.o
g++ -std=gnu++11 -I"/.../R/4.0.2/lib64/R/include" -DNDEBUG -I/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/include -DARROW_R_WITH_ARROW -I'/.../R/4.0.2/lib64/R/library/Rcpp/include' -I/usr/local/include -fpic -O3 -march=native -mtune=native -fPIC -c json.cpp -o json.o
g++ -std=gnu++11 -I"/.../R/4.0.2/lib64/R/include" -DNDEBUG -I/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/include -DARROW_R_WITH_ARROW -I'/.../R/4.0.2/lib64/R/library/Rcpp/include' -I/usr/local/include -fpic -O3 -march=native -mtune=native -fPIC -c memorypool.cpp -o memorypool.o
g++ -std=gnu++11 -I"/.../R/4.0.2/lib64/R/include" -DNDEBUG -I/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/include -DARROW_R_WITH_ARROW -I'/.../R/4.0.2/lib64/R/library/Rcpp/include' -I/usr/local/include -fpic -O3 -march=native -mtune=native -fPIC -c message.cpp -o message.o
g++ -std=gnu++11 -I"/.../R/4.0.2/lib64/R/include" -DNDEBUG -I/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/include -DARROW_R_WITH_ARROW -I'/.../R/4.0.2/lib64/R/library/Rcpp/include' -I/usr/local/include -fpic -O3 -march=native -mtune=native -fPIC -c parquet.cpp -o parquet.o
g++ -std=gnu++11 -I"/.../R/4.0.2/lib64/R/include" -DNDEBUG -I/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/include -DARROW_R_WITH_ARROW -I'/.../R/4.0.2/lib64/R/library/Rcpp/include' -I/usr/local/include -fpic -O3 -march=native -mtune=native -fPIC -c py-to-r.cpp -o py-to-r.o
g++ -std=gnu++11 -I"/.../R/4.0.2/lib64/R/include" -DNDEBUG -I/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/include -DARROW_R_WITH_ARROW -I'/.../R/4.0.2/lib64/R/library/Rcpp/include' -I/usr/local/include -fpic -O3 -march=native -mtune=native -fPIC -c recordbatch.cpp -o recordbatch.o
g++ -std=gnu++11 -I"/.../R/4.0.2/lib64/R/include" -DNDEBUG -I/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/include -DARROW_R_WITH_ARROW -I'/.../R/4.0.2/lib64/R/library/Rcpp/include' -I/usr/local/include -fpic -O3 -march=native -mtune=native -fPIC -c recordbatchreader.cpp -o recordbatchreader.o
g++ -std=gnu++11 -I"/.../R/4.0.2/lib64/R/include" -DNDEBUG -I/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/include -DARROW_R_WITH_ARROW -I'/.../R/4.0.2/lib64/R/library/Rcpp/include' -I/usr/local/include -fpic -O3 -march=native -mtune=native -fPIC -c recordbatchwriter.cpp -o recordbatchwriter.o
g++ -std=gnu++11 -I"/.../R/4.0.2/lib64/R/include" -DNDEBUG -I/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/include -DARROW_R_WITH_ARROW -I'/.../R/4.0.2/lib64/R/library/Rcpp/include' -I/usr/local/include -fpic -O3 -march=native -mtune=native -fPIC -c schema.cpp -o schema.o
g++ -std=gnu++11 -I"/.../R/4.0.2/lib64/R/include" -DNDEBUG -I/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/include -DARROW_R_WITH_ARROW -I'/.../R/4.0.2/lib64/R/library/Rcpp/include' -I/usr/local/include -fpic -O3 -march=native -mtune=native -fPIC -c symbols.cpp -o symbols.o
g++ -std=gnu++11 -I"/.../R/4.0.2/lib64/R/include" -DNDEBUG -I/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/include -DARROW_R_WITH_ARROW -I'/.../R/4.0.2/lib64/R/library/Rcpp/include' -I/usr/local/include -fpic -O3 -march=native -mtune=native -fPIC -c table.cpp -o table.o
g++ -std=gnu++11 -I"/.../R/4.0.2/lib64/R/include" -DNDEBUG -I/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/include -DARROW_R_WITH_ARROW -I'/.../R/4.0.2/lib64/R/library/Rcpp/include' -I/usr/local/include -fpic -O3 -march=native -mtune=native -fPIC -c threadpool.cpp -o threadpool.o
g++ -std=gnu++11 -shared -L/.../R/4.0.2/lib64/R/lib -L/usr/local/lib64 -o arrow.so array.o array_from_vector.o array_to_vector.o arraydata.o arrowExports.o buffer.o chunkedarray.o compression.o compute.o csv.o dataset.o datatype.o expression.o feather.o field.o filesystem.o io.o json.o memorypool.o message.o parquet.o py-to-r.o recordbatch.o recordbatchreader.o recordbatchwriter.o schema.o symbols.o table.o threadpool.o -L/.../tempdir/RtmpWcNNLr/R.INSTALL59cf4fd1c7ac/arrow/libarrow/arrow-0.17.1/lib -larrow_dataset -lparquet -larrow -larrow -larrow_dataset -lboost_filesystem -lboost_regex -lboost_system -lbrotlidec-static -lbrotlienc-static -lbrotlizzz-static -lbz2 -ljemalloc_pic -llz4 -lparquet -lsnappy -lthrift -lz -lzstd -L/.../R/4.0.2/lib64/R/lib -lR
installing to /.../R_4.0.2_libs/00LOCK-arrow/00new/arrow/libs
** R
** inst
** byte-compile and prepare package for lazy loading** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for ‘arrow’ in dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object '/.../R_4.0.2_libs/00LOCK-arrow/00new/arrow/libs/arrow.so':
/.../R_4.0.2_libs/00LOCK-arrow/00new/arrow/libs/arrow.so: undefined symbol: _ZN5arrow5fieldENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt10shared_ptrINS_8DataTypeEEbS6_IKNS_16KeyValueMetadataEE
Error: loading failed
Execution halted
ERROR: loading failed

• removing ‘/.../R_4.0.2_libs/arrow’

[asfimport]

Neal Richardson / @nealrichardson:
We can probably add a minimum cmake version check.

If you're using the prebuilt binary, can you try with setting CC and CXX? The binary was built with https://github.com/ursa-labs/arrow-r-nightly/blob/master/linux/yum.Dockerfile#L18-L19

If you want to build everything from source and you want compression codecs built, you can set LIBARROW_MINIMAL=false, or you can pick and choose the specific ARROW_WITH_X codecs to set, per https://github.com/apache/arrow/blob/master/r/inst/build_arrow_static.sh

[asfimport]

Nathan TeBlunthuis:
Hi Neal,

Thanks for your help so far!

My CC and CXX environment variables were not set to /usr/bin/gcc and /usr/bin/g++ because the system gcc is pretty old so I use a modern one (gcc 8.3.0)  installed in user space.

I tried installing the binaries with CC and CXX both with gcc 8.3.0 and with the system gcc (4.8.5). I get teh same undefined symbol error (as above) in either case.

I also tried building from source with LIBARROW_MINIMAL=FALSE and with LZ4, BZIP,  SNAPPY and LZMA flags set. Compilation succeeds but R cannot load the package, with a somewhat different error.

Error: package or namespace load failed for ‘arrow’ in dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object '/.../R_4.0.2_libs/00LOCK-arrow/00new/arrow/libs/arrow.so':
/.../R_4.0.2_libs/00LOCK-arrow/00new/arrow/libs/arrow.so: undefined symbol: _ZTIN5boost11regex_errorE
Error: loading failed

– Nate

 

 

[asfimport]

Neal Richardson / @nealrichardson:

I tried installing the binaries with CC and CXX both with gcc 8.3.0 and with the system gcc (4.8.5). I get teh same undefined symbol error (as above) in either case.

Odd, I wouldn't expect the same undefined symbol error with those gcc versions.

Error: package or namespace load failed for ‘arrow’ in dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object '/.../R_4.0.2_libs/00LOCK-arrow/00new/arrow/libs/arrow.so':
/.../R_4.0.2_libs/00LOCK-arrow/00new/arrow/libs/arrow.so: undefined symbol: _ZTIN5boost11regex_errorE
Error: loading failed

Could you share the verbose build logs (with ARROW_R_DEV=true set) from that? And is that using gcc 8 or gcc 4.8? Boost regex is only needed when compiling with gcc < 4.9.

[asfimport]

Nathan TeBlunthuis:
I double checked that I'm using gcc 8.3.0 this time. Maybe last time I left CC and CCX set to 4.8.5 ?

My build log is attached. build_arrow.txt

Thanks again

 

[asfimport]

Neal Richardson / @nealrichardson:
Thanks. We're getting closer. Can you unset the various ARROW_WITH_X vars and just use LIBARROW_MINIMAL=false, which will set all of those for you? I'm not immediately seeing why it's looking for snappy but it didn't build it, but my first guess is a confusion between the way you set those WITH env vars, as TRUE, vs. how cmake prefers them (as ON).

[asfimport]

Nathan TeBlunthuis:
Hi Neal,

I set ARROW_WITH_SNAPPY=FALSE (and the rest of the codecs to TRUE) it works fine with support for the rest of the codecs.  So it seems we have traced the issue to snappy.

[asfimport]

Neal Richardson / @nealrichardson:
I'm still curious to see how it goes using ON/OFF instead of TRUE/FALSE. I suspect the (current) problem is somewhere in how the library is determining whether support for a codec has been built, and not in building the codec itself, since I didn't see a compilation error.

[asfimport]

Nathan TeBlunthuis:
Ok I'll try again with all the codecs set to ON.

[asfimport]

Nathan TeBlunthuis:
With all the codecs set to ON, instead of TRUE things are much the same as before.  build_arrow_2.txt

[asfimport]

Neal Richardson / @nealrichardson:
Hmm. Can you try setting EXTRA_CMAKE_FLAGS='-DSnappy_SOURCE=BUNDLED' and see if that forces it to build snappy? Looking at https://github.com/apache/arrow/blob/master/cpp/cmake_modules/ThirdpartyToolchain.cmake#L838-L862, it looks like there's some way it could build "with snappy" and not report that it was using system snappy, which could be a problem somehow?

[asfimport]

Nathan TeBlunthuis:
Now the build completes and I end up with a working package, but snappy is unavailable. build_arrow_3.txt

[asfimport]

Alok Saldanha:
I am not sure if I should be mentioning it here, but I am getting the same "undefined symbol: _ZN6snappy11RawCompressEPKcmPcPm" error on CentOS 7, and I found that I could install an old version of arrow with

PackageUrl <- "https://cran.r-project.org/src/contrib/Archive/arrow/arrow_0.16.0.2.tar.gz" install.packages(PackageUrl, repos=NULL, type="source") 

So the issue may have been introduced into the build process between 0.16.0.2 and 0.17.1 (I haven't tried 0.17.0 yet).