-
Notifications
You must be signed in to change notification settings - Fork 4k
ARROW-13398: [R] Update install.Rmd vignette #11521
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from all commits
Commits
Show all changes
48 commits
Select commit
Hold shift + click to select a range
12ee483
Add extra headings, bullet points, and labelling for signposting
thisisnic ca4d396
Fix typo
thisisnic a1bc636
Remove duplicated line!
thisisnic 0a8ba1e
Remove references to compression libraries not being on
thisisnic d7c2cfe
Remove line added back in from previous merge
thisisnic 57765e6
Remove accidentally reintroduced line
thisisnic 331f33b
Add in description of NOT_CRAN env var
thisisnic 74c5ee1
Spacing tweaks
thisisnic 80eb937
Put env vars in table
thisisnic 3881d4d
Add spacing so bullet points work
thisisnic 4dfa450
Remove duplicated content and move developer content to developer sec…
thisisnic 8dbcdf7
Rename file and trim out duplication
thisisnic e2b0639
Move dependency resolution to dev guide
thisisnic 302c510
Add to pkgdown
thisisnic f6a6a97
Add diagram
thisisnic f96d481
Change image size
thisisnic 830d974
Promote heading level of the Build env vars section
thisisnic bd2c904
Update LIBARROW_MINIMAL description
thisisnic c646fdf
Add images, rephrase hugely, reorder everything, divide source build …
thisisnic 232c4bf
Tell users that here be dragons (well, unnecessarily complex stuff)
thisisnic 6dfa05c
Link to distro map and move ARROW_USE_PKG_CONFIG further down
thisisnic 17fef94
Snark free solaris
thisisnic a3bbe34
Whitespace fixes
thisisnic ef01b51
Update r/vignettes/developers/install_details.Rmd
thisisnic 945e435
Update r/vignettes/developers/install_details.Rmd
thisisnic bbf8da0
Remove backticks and call libarrow libarrow; add description of when …
thisisnic 8f2b424
Merge branch 'ARROW-13398_install.Rmd' of github.com:thisisnic/arrow …
thisisnic 8de3933
Rearrange so heading match bullets
thisisnic 95f8c35
Put graphics in the right place
thisisnic e009c02
Add back in "on Linux"
thisisnic f56f96f
Add in why we care
thisisnic f54c4c1
typo
thisisnic c4f07a7
Update r/vignettes/install.Rmd
thisisnic 7b6c332
Add RSPM link
thisisnic c0481c8
Split conda command onto new line
thisisnic e6f22dc
Merge branch 'ARROW-13398_install.Rmd' of github.com:thisisnic/arrow …
thisisnic 89dfe69
Add mention of winlibs and diagram
thisisnic 9d7facb
Update r/vignettes/developers/install_details.Rmd
thisisnic 798f9b6
Rearrange so flows better
thisisnic 988e137
Move dependencies to main install doc, grab an extra top-level headin…
thisisnic 4d08f7d
Move images to directories
thisisnic 7fee133
Make image backgrounds white, remove incorrect dependencies info
thisisnic 39ed38d
Add content about downloading dependencies
thisisnic da109cd
Explicitly mention FALSE
thisisnic 07e2046
Be more explicit about features being disabled
thisisnic 42686cd
Soften recommendation
thisisnic 30a3141
Move dependencies to the top
thisisnic 893825b
A bit of shuffling
jonkeane File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,124 @@ | ||
| --- | ||
| title: "How the R package is installed - advanced" | ||
| --- | ||
|
|
||
| This document is intended specifically for arrow _developers_ who wish to know | ||
| more about these scripts. If you are an arrow _user_ looking for help with | ||
| installing arrow, please see [the installation guide](../install.html) | ||
thisisnic marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| The arrow R package requires that Arrow C++ library (also known as libarrow) to | ||
| be installed in order to work properly. There are a number of different ways | ||
| in which libarrow could be installed: | ||
|
|
||
| * as part of the R package installation process | ||
| * a system package | ||
| * a library you've built yourself outside of the context of installing the R package | ||
|
|
||
| Below, we discuss each of these setups in turn. | ||
|
|
||
| # Installing libarrow during R package installation | ||
|
|
||
| There are a number of scripts that are triggered | ||
| when `R CMD INSTALL .` is run and for Arrow users, these should all just work | ||
| without configuration and pull in the most complete pieces (e.g. official | ||
| binaries that we host). One of the jobs of these scripts is to work out | ||
| if libarrow is installed, and if not, install it. | ||
|
|
||
| An overview of these scripts is shown below: | ||
|
|
||
| * `configure` and `configure.win` - these scripts are triggered during | ||
| `R CMD INSTALL .` on non-Windows and Windows platforms, respectively. They | ||
| handle finding the libarrow, setting up the build variables necessary, and | ||
| writing the package Makevars file that is used to compile the C++ code in the R | ||
| package. | ||
|
|
||
| * `tools/nixlibs.R` - this script is sometimes called by `configure` on Linux | ||
thisisnic marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| (or on any non-windows OS with the environment variable | ||
| `FORCE_BUNDLED_BUILD=true`) if an existing libarrow installation cannot be found. | ||
| This sets up the build process for our bundled builds (which is the default on | ||
| linux) and checks for binaries or downloads libarrow from source depending on | ||
| dependency availability and build configuration. | ||
|
|
||
| * `tools/winlibs.R` - this script is sometimes called by `configure.win` on Windows | ||
| when environment variable `ARROW_HOME` is not set. It looks for an existing libarrow | ||
| installation, and if it can't find one downloads an appropriate libarrow binary. | ||
|
|
||
| * `inst/build_arrow_static.sh` - called by `tools/nixlibs.R` when libarrow | ||
| needs to be built. It builds libarrow for a bundled, static build, and | ||
| mirrors the steps described in the ["Arrow R Developer Guide" vignette](./setup.html) | ||
| This build script is also what is used to generate our prebuilt binaries. | ||
|
|
||
| The actions taken by these scripts to resolve dependencies and install the | ||
| correct components are described below. | ||
|
|
||
| ## How the R package finds libarrow | ||
|
|
||
| ### Windows | ||
|
|
||
| The diagram below shows how the R package finds a libarrow installation on Windows. | ||
|
|
||
| ```{r, echo=FALSE, out.width="70%"} | ||
| knitr::include_graphics("./install_diagram_windows.png") | ||
| ``` | ||
|
|
||
| ### Linux | ||
|
|
||
| The diagram below shows how the R package finds a libarrow installation on non-Windows systems. | ||
|
|
||
| ```{r, echo=FALSE, out.width="70%"} | ||
| knitr::include_graphics("./install_nix.png") | ||
| ``` | ||
thisisnic marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| More information about these steps can be found below. | ||
|
|
||
| #### Using pkg-config | ||
|
|
||
| When you install the arrow R package on Linux, if no environment variables | ||
| relating to the location of an existing libarrow installation have already by | ||
| set, the installation code will attempt to find libarrow on | ||
| your system using the `pkg-config` command. | ||
|
|
||
| This will find either installed system packages or libraries you've built yourself. | ||
| In order for `install.packages("arrow")` to work with these system packages, | ||
| you'll need to install them before installing the R package. | ||
|
|
||
| #### Prebuilt binaries | ||
|
|
||
| If libarrow is not found on the system, the R package installation | ||
| script will next attempt to download prebuilt libarrow binaries | ||
| that match your both your local operating system and arrow R package version. | ||
| The libarrow binaries will only be retrieved if you have set the environment variable | ||
| `LIBARROW_BINARY` or `NOT_CRAN`. | ||
|
|
||
| If found, they will be downloaded and bundled when your R package compiles. | ||
| For a list of supported distributions and versions, | ||
| see the [arrow-r-nightly](https://github.com/ursa-labs/arrow-r-nightly/blob/master/README.md) project. | ||
|
|
||
| #### Building from source | ||
|
|
||
| If no libarrow binary is found, it will attempt to build it locally. | ||
| First, it will also look to see if you are in a checkout of the `apache/arrow` | ||
| git repository and thus have the libarrow source files there. | ||
| Otherwise, it builds from the source files included in the package. | ||
| Depending on your system, building libarrow from source may be slow. If | ||
| libarrow is built from source, `inst/build_arrow_static.sh` is executed. | ||
|
|
||
| # Using the R package with libarrow installed as a system package | ||
|
|
||
| If you are authorized to install system packages and you're installing a CRAN release, | ||
| you may want to use the official Apache Arrow release packages corresponding to | ||
| the R package version via software distribution tools such as `apt` or `yum` | ||
| (though there are some drawbacks: see the | ||
| ["Troubleshooting" section in the main installation docs]("../install.html)). | ||
| See the [Arrow project installation page](https://arrow.apache.org/install/) | ||
| to find pre-compiled binary packages for some common Linux distributions, | ||
| including Debian, Ubuntu, and CentOS. | ||
|
|
||
| Generally, we do not recommend this method of working with libarrow with the R | ||
| package unless you have a specific reason to do so. | ||
|
|
||
| # Using the R package with an existing libarrow build | ||
|
|
||
| This setup is much more common for arrow developers, who may be needing to make | ||
| changes to both the R package and libarrow source code. See | ||
| the [developer setup docs](./setup.html) for more information. | ||
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
❤️