[R] arrow R package 17.0.0.1 installs without GCS enabled on MacOS

[cklunch]

Describe the bug, including details regarding any error messages, version, and platform.

Installing the current version of arrow from CRAN on Mac is successful, but with arrow_with_gcs() == FALSE. I've tried the options on https://arrow.apache.org/docs/r/articles/install.html for custom installs, but haven't been able to get GCS capability enabled. If I downgrade to arrow 15.0.1, GCS connections are working fine.

I see the same behavior on Sonoma 14.7 with Intel, R version 4.4.0, and on Sequoia 15.1.1 with Apple M2, R version 4.4.2.

Looking at PR #42210 it looks like this may have been a development choice. If so, is there a simple way to install with GCS, or to enable GCS after installation?

Component(s)

R

[amoeba]

Hi @cklunch, sorry for the trouble. This is a bit of a pain point with the package.

Can you please try this and let us know if it works? From an R console:

Sys.setenv("NOT_CRAN" = "true")
install.packages("arrow", type = "source")

If that runs successfully, you should see GCS enabled once you load arrow and run arrow_info().

[cklunch]

@amoeba Thanks for the quick response! I'm getting more informative errors this way, here's the start of the install output:

trying URL '[https://cran.rstudio.com/src/contrib/arrow_17.0.0.1.tar.gz](https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcran.rstudio.com%2Fsrc%2Fcontrib%2Farrow_17.0.0.1.tar.gz&data=05%7C02%7Cclunch%40battelleecology.org%7Cd64564578acc435bbd3c08dd0e64a618%7Cf44d2ab390994d85998610165a8619f5%7C0%7C0%7C638682548174516801%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=ewiZoiPq7PSKt5s89ZaEN6771wyqx1rJTaMZQCFsd1A%3D&reserved=0)'
Content type 'application/x-gzip' length 4492506 bytes (4.3 MB)
==================================================
downloaded 4.3 MB

* installing *source* package ‘arrow’ ...
** package ‘arrow’ successfully unpacked and MD5 sums checked
** using staged installation
*** pkg-config not found.
*** OpenSSL not found
*** Found local C++ source: 'tools/cpp'
*** Building libarrow from source
    For build options and troubleshooting, see the install guide:
    [https://arrow.apache.org/docs/r/articles/install.html](https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Farrow.apache.org%2Fdocs%2Fr%2Farticles%2Finstall.html&data=05%7C02%7Cclunch%40battelleecology.org%7Cd64564578acc435bbd3c08dd0e64a618%7Cf44d2ab390994d85998610165a8619f5%7C0%7C0%7C638682548174533581%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=Z6oY%2FA6d3fa0Cjf%2FPjaD9mxLsw0A01Zj%2BTzGy5toJc4%3D&reserved=0)
**** cmake 3.26.4
**** pkg-config not installed, setting ARROW_DEPENDENCY_SOURCE=BUNDLED
****  S3/GCS  support  requires version >= 1.0.2 of openssl-devel (rpm), libssl-dev (deb), or openssl (brew) ; building with  ARROW_S3=OFF and ARROW_GCS=OFF
**** arrow

Consistent with that message, arrow_info() after install has both GCS and S3 disabled:

Arrow package version: 17.0.0.1

Capabilities:
               
acero      TRUE
dataset    TRUE
substrait FALSE
parquet    TRUE
json       TRUE
s3        FALSE
gcs       FALSE
utf8proc   TRUE
re2        TRUE
snappy     TRUE
gzip       TRUE
brotli     TRUE
zstd       TRUE
lz4        TRUE
lz4_frame  TRUE
lzo       FALSE
bz2        TRUE
jemalloc   TRUE
mimalloc   TRUE

Memory:
                 
Allocator mimalloc
Current    0 bytes
Max        0 bytes

Runtime:
                       
SIMD Level          none
Detected SIMD Level none

Build:
                                   
C++ Library Version           17.0.0
C++ Compiler              AppleClang
C++ Compiler Version 16.0.0.16000026

OpenSSL 1.1.1 is on my machine, so it looks like the arrow install can't find it.

[amoeba]

Thanks. Can you please try install OpenSSL@3 with

brew install openssl@3

And then running a similar command to above (with one tweak):

Sys.setenv("ARROW_R_DEV"=TRUE, "NOT_CRAN" = "true")
install.packages("arrow", type = "source")

[cklunch]

@amoeba Thanks so much for your help! This worked to get arrow running on the M2 chip, and I'm able to successfully access files in GCS.

Still working on getting it set up on the Intel chip, but it's a problem with homebrew, not arrow. Thank you!

[amoeba]

Great, thanks for reporting back. Please feel free to file a new issue if you run into anything with your Intel machine. Hopefully we can enable GCS by default in the future.

[eamcvey]

I had the same problem (M2 chip, Ventura) and was able to fix it following the instructions here, so thank you!

I'll note that the package articles clearly state that the CRAN version will install with GCS on Mac, so if this is more than a brief issue it would be helpful to change that.

[amoeba]

Thanks for the catch @eamcvey, I've filed #45260 and we'll get the docs updated.