[R] R package fails to install when using clang-16 as the compiler

[paleolimbot]

Describe the bug, including details regarding any error messages, version, and platform.

On the CRAN package check page ( https://cran.r-project.org/web/checks/check_results_arrow.html ) there is a new warning about clang-16. We don't get any diagnostics in the compiler output ( https://www.stats.ox.ac.uk/pub/bdr/clang16/arrow.log ) because we suppress that output by default; however, it seems as though the failure is at the Arrow C++ build step and not the R package build step.

On submission we will need to submit an explanation or we will need to patch the vendored Arrow C++ to ensure that it builds prior to submission. The first step is to reproduce...LLVM provides a Dockerfile that we can probably build from ( https://github.com/llvm/llvm-project/blob/main/llvm/utils/docker/debian10/Dockerfile + apt-get install -y r-base will get us close).

Component(s)

R

[paleolimbot]

@nealrichardson I can't reproduce using Dockerfile:

FROM silkeh/clang:dev-bullseye

RUN apt-get update && apt-get install -y git cmake libxml2-dev libcurl4-openssl-dev libssl-dev
RUN git clone https://github.com/apache/arrow.git /arrow
RUN mkdir /arrow-build && cd /arrow-build && cmake /arrow/cpp -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++  -DARROW_CSV=ON -DARROW_DATASET=ON -DARROW_FILESYSTEM=ON -DARROW_COMPUTE=ON -DARROW_PARQUET=ON -DBoost_SOURCE=BUNDLED && cmake --build . && cmake --install . --prefix /arrow-dist

ENV ARROW_HOME /arrow-dist

CMD /bin/bash

So I'm guessing it's either S3 or GCS.

[paleolimbot]

...and with -DARROW_S3=ON -DARROW_GCS=ON it's OK too. Will try with the R source package on Monday.

[nealrichardson]

Does that image have boost on it? Have you tried with dependency source BUNDLED?

[paleolimbot]

That also works; however, when I check out the 10.0.1 release and try to build that, I don't get any errors either. (Again, will try the R package install from source specifically on Monday).

[paleolimbot]

No errors with the R package (10.0.1) either:

FROM silkeh/clang:dev-bullseye

RUN apt-get update && apt-get install -y git r-base cmake libcurl4-openssl-dev libssl-dev

RUN R -e "install.packages(c('lifecycle', 'purrr', 'rlang', 'tidyselect', 'vctrs', 'dplyr'))"

RUN mkdir ~/.R &&  \
  echo 'CXX11=clang++ -std=c++11' > ~/.R/Makevars &&  \
  echo 'CXX14=clang++ -std=c++14' >> ~/.R/Makevars && \
  echo 'CXX17=clang++ -std=c++17' >> ~/.R/Makevars && \
  echo 'CXX20=clang++ -std=c++20' >> ~/.R/Makevars && \
  echo 'CXX=clang++ -std=c++11' >> ~/.R/Makevars && \
  echo 'CC=clang' >> ~/.R/Makevars

ENV ARROW_R_DEV true
RUN R -e 'install.packages("arrow"); library(arrow)'

CMD /bin/bash

[nealrichardson]

Ok, I can email BDR and ask for more details about his setup. Can you find the clang version/commit from which it was built?

[paleolimbot]

It looks like the image uses https://apt.llvm.org/ , which has a development branch set of packages for Debian. I'm guessing BDR is on Fedora...it would be nice to know how exactly he installed it since it appears that does matter. The image I'm using was built 24 days ago ( https://hub.docker.com/layers/silkeh/clang/dev-bullseye/images/sha256-227546652b0b6d95f8acd5377ec53b32d9f417091f6fbe2190633e0cd30a837a?context=explore ).

[nealrichardson]

Right I saw that, but I don't know what version was there 24 days ago when the image was built, and that might be useful information if his clang version is different.

[paleolimbot]

Good call...here's what I get:

% docker run --rm -it --platform linux/amd64 silkeh/clang:dev-bullseye
root@282cc557891a:/# clang --version
Debian clang version 16.0.0 (++20221230071836+eda2eaabf294-1~exp1~20221230071942.515)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

Probably best if I verify one more time tonight when I'm back at home (the computer that ran the check is in my basement).

[nealrichardson]

Also, if you apt-get install clang in the image, will it pull the latest nightly clang?

[paleolimbot]

I double checked and I have indeed been running the check under the clang from 23 days ago (version above). I got a little worried because the buddy building/uploading that image hasn't uploaded an aarch64 one for a long time and the version of clang on that image is 11 😬

apt-get install clang does indeed pull an image from today (as long as you're on x86)...according to the docs, it gets built twice a day. I have that check running now.

[nealrichardson]

A few details I just noticed in the output on https://www.stats.ox.ac.uk/pub/bdr/clang16/arrow.out:

* using R Under development (unstable) (2023-01-20 r83646)
* using platform: x86_64-pc-linux-gnu (64-bit)
* R was compiled by
    clang version 16.0.0 (https://github.com/llvm/llvm-project.git 2133e8b9f942f91ec54e28c580fccf6d6b26c62e)
    GNU Fortran (GCC) 12.2.1 20221121 (Red Hat 12.2.1-4)

So, (1) we're already using a newer clang build than he is, but (2) we're not testing with R-devel compiled by that clang, for what that's worth.