From 75c789002beada394a165fd8e88dd20b49dc48cc Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Fri, 18 Nov 2022 16:28:41 -0400 Subject: [PATCH 01/33] first draft --- r/.Rbuildignore | 1 + r/PACKAGING.md | 124 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 125 insertions(+) create mode 100644 r/PACKAGING.md diff --git a/r/.Rbuildignore b/r/.Rbuildignore index 2259aabe396..756b6faf116 100644 --- a/r/.Rbuildignore +++ b/r/.Rbuildignore @@ -30,3 +30,4 @@ STYLE.md ^cheatsheet$ ^revdep$ ^vignettes$ +^PACKAGING\.md$ diff --git a/r/PACKAGING.md b/r/PACKAGING.md new file mode 100644 index 00000000000..6242ce448fd --- /dev/null +++ b/r/PACKAGING.md @@ -0,0 +1,124 @@ + + + +# Packaging checklist for CRAN release + +For full documentation of the steps in this checklist, see the +[Apache Arrow Release Management Guide](https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-UpdatingRpackages). + +- [ ] When a release candidate is created + [Create a GitHub issue](https://github.com/apache/arrow/issues/new/) + entitled `[R] CRAN release version X.X.X` and copy this checklist to the issue + +- [ ] Confirm that all + [nightly tests and nightly packaging builds](https://lists.apache.org/list.html?builds@arrow.apache.org) are passing. +- [ ] Check [current CRAN check results](https://cran.rstudio.org/web/checks/check_results_arrow.html) +- [ ] Check README +- [ ] `urlchecker::url_check()` +- [ ] [Polish NEWS](https://style.tidyverse.org/news.html#news-release) +- [ ] Prepare tweet thread highlighting new features + +Make draft pull requests into the [autobrew](https://github.com/autobrew) and +[rtools-packages](https://github.com/r-windows/rtools-packages) repositories +used by the configure script on MacOS and Windows: + +- [ ] Open a draft pull request to modify + [the apache-arrow autobrew formula]( https://github.com/autobrew/homebrew-core/blob/master/Formula/apache-arrow.rb) + to update the version, SHA, and any changes to dependencies and build steps that have changed in the + [copy of the formula we have of that formula in the Arrow repo](https://github.com/apache/arrow/blob/master/dev/tasks/homebrew-formulae/autobrew/apache-arrow.rb) +- [ ] Open a draft pull request to modify + [the apache-arrow-static autobrew formula]( https://github.com/autobrew/homebrew-core/blob/master/Formula/apache-arrow-static.rb) + to update the version, SHA, and any changes to dependencies and build steps that have changed in the + [copy of the formula we have of that formula in the Arrow repo](https://github.com/apache/arrow/blob/master/dev/tasks/homebrew-formulae/autobrew/apache-arrow-static.rb) +- [ ] Open a draft pull request to modify the + [autobrew script](https://github.com/autobrew/scripts/blob/master/apache-arrow) + to include any additions made to + [r/tools/autobrew](https://github.com/apache/arrow/blob/master/r/tools/autobrew). +- [ ] Open a draft pull request to modify the + [PKGBUILD script](https://github.com/r-windows/rtools-packages/blob/master/mingw-w64-arrow/PKGBUILD) + to reflect changes in + [ci/PKGBUILD](https://github.com/apache/arrow/blob/master/ci/scripts/PKGBUILD), + uncommenting the line that says "uncomment to test the rc". + +Prepare and check the .tar.gz that will be released to CRAN: + +TODO: The packaging guide says `make release` but isn't there also a step where +we bundle Arrow C++ into tools/cpp and prune it so that the .tar.gz stays under 5 MB? + +- `git fetch upstream && git checkout release-X.X.X-rc0` +- [ ] `devtools::check("arrow_X.X.X.tar.gz")` locally on Linux +- [ ] `devtools::check("arrow_X.X.X.tar.gz")` locally on Windows +- [ ] `devtools::check("arrow_X.X.X.tar.gz")` locally on MacOS +- [ ] `devtools::check("arrow_X.X.X.tar.gz")` locally on MacOS M1 +- [ ] `devtools::check("arrow_X.X.X.tar.gz")` locally on Linux +- [ ] `rhub::check("arrow_X.X.X.tar.gz", platform=c("debian-gcc-patched", "fedora-clang-devel"))` + (TODO: as I understand it, we can't do rhub Windows and Mac checks because those + would require the PRs into autobrew/rtools-packages to be finalized? Or maybe + these already know about/use the nightly binaries?) +- [ ] Run reverse dependency checks locally (TODO: document how to do this + since we can't just use `revdepcheck`) + +After this step, we need an official release: + +- [ ] Wait for the release vote to pass + +TODO is this where we create a CRAN release branch where the minor CRAN-specific +fixes get picked from master? What is the git incantation for that? + +- [ ] Bump the Version in DESCRIPTION to X.X.X +- [ ] Regenerate arrow_X.X.X.tar.tz (via `make release`?) + +Finalize and merge draft PRs/ensure linux binary packages are available: + +- [ ] PR into autobrew/homebrew-core +- [ ] PR into autobrew/homebrew-core +- [ ] PR into autobrew/scripts +- [ ] PR into r-windows/rtools-packages +- [ ] Ensure linux binaries are available (TODO: maybe document where these live? + I don't remember where this is at the top of my head) + +Check binary Arrow C++ distributions specific to the R package: + +- [ ] Upload the .tar.gz to [WinBuilder](https://win-builder.r-project.org/upload.aspx) + and confirm that the check is clean +- [ ] Upload the .tar.gz to [MacBuilder](https://mac.r-project.org/macbuilder/submit.html) + and confirm that the check is clean +- [ ] Check `install.packages("arrow_X.X.X.tar.gz")` on Ubuntu and ensure that the + hosted binaries are used + +Submit! + +- [ ] Upload arrow_X.X.X.tar.gz to the + [CRAN submit page](https://xmpalantir.wu.ac.at/cransubmit/) +- [ ] Confirm the submission email + +Wait for CRAN... + +Post release: + +- [ ] Accepted! +- [ ] Tag the tip of the CRAN-specific release branch? +- [ ] Bump the version number in DESCRIPTION to XXX? +- [ ] Add a new line to the matrix in the [backwards compatability job](https://github.com/apache/arrow/blob/master/dev/tasks/r/github.linux.arrow.version.back.compat.yml) +- [ ] Update the packaging checklist template to reflect any new realities of the + packaging process. +- [ ] Wait for CRAN-hosted binaries on the + [CRAN package page](https://cran.r-project.org/package=arrow) to reflect the + new version From f9aff0105ab907f6eee52886ba9e0caa036eeedd Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Tue, 22 Nov 2022 09:28:29 -0400 Subject: [PATCH 02/33] some clarifications --- r/PACKAGING.md | 38 +++++++++++++++----------------------- 1 file changed, 15 insertions(+), 23 deletions(-) diff --git a/r/PACKAGING.md b/r/PACKAGING.md index 6242ce448fd..3d9ca048b1a 100644 --- a/r/PACKAGING.md +++ b/r/PACKAGING.md @@ -43,8 +43,8 @@ used by the configure script on MacOS and Windows: [the apache-arrow autobrew formula]( https://github.com/autobrew/homebrew-core/blob/master/Formula/apache-arrow.rb) to update the version, SHA, and any changes to dependencies and build steps that have changed in the [copy of the formula we have of that formula in the Arrow repo](https://github.com/apache/arrow/blob/master/dev/tasks/homebrew-formulae/autobrew/apache-arrow.rb) -- [ ] Open a draft pull request to modify - [the apache-arrow-static autobrew formula]( https://github.com/autobrew/homebrew-core/blob/master/Formula/apache-arrow-static.rb) +- [ ] Open a draft pull request to modify + [the apache-arrow-static autobrew formula]( https://github.com/autobrew/homebrew-core/blob/master/Formula/apache-arrow-static.rb) to update the version, SHA, and any changes to dependencies and build steps that have changed in the [copy of the formula we have of that formula in the Arrow repo](https://github.com/apache/arrow/blob/master/dev/tasks/homebrew-formulae/autobrew/apache-arrow-static.rb) - [ ] Open a draft pull request to modify the @@ -57,31 +57,24 @@ used by the configure script on MacOS and Windows: [ci/PKGBUILD](https://github.com/apache/arrow/blob/master/ci/scripts/PKGBUILD), uncommenting the line that says "uncomment to test the rc". -Prepare and check the .tar.gz that will be released to CRAN: - -TODO: The packaging guide says `make release` but isn't there also a step where -we bundle Arrow C++ into tools/cpp and prune it so that the .tar.gz stays under 5 MB? +Prepare and check the .tar.gz that will be released to CRAN. -- `git fetch upstream && git checkout release-X.X.X-rc0` +- [ ] `git fetch upstream && git checkout release-X.X.X-rcXX` +- [ ] `make build` (copies Arrow C++ into tools/cpp, prunes some unnecessary + components, and runs `R CMD build`) - [ ] `devtools::check("arrow_X.X.X.tar.gz")` locally on Linux - [ ] `devtools::check("arrow_X.X.X.tar.gz")` locally on Windows - [ ] `devtools::check("arrow_X.X.X.tar.gz")` locally on MacOS - [ ] `devtools::check("arrow_X.X.X.tar.gz")` locally on MacOS M1 - [ ] `devtools::check("arrow_X.X.X.tar.gz")` locally on Linux - [ ] `rhub::check("arrow_X.X.X.tar.gz", platform=c("debian-gcc-patched", "fedora-clang-devel"))` - (TODO: as I understand it, we can't do rhub Windows and Mac checks because those - would require the PRs into autobrew/rtools-packages to be finalized? Or maybe - these already know about/use the nightly binaries?) -- [ ] Run reverse dependency checks locally (TODO: document how to do this - since we can't just use `revdepcheck`) +- [ ] Run reverse dependency checks -After this step, we need an official release: +Wait for the official release... - [ ] Wait for the release vote to pass - -TODO is this where we create a CRAN release branch where the minor CRAN-specific -fixes get picked from master? What is the git incantation for that? - +- [ ] Create a CRAN-release branch (TODO: previously this branch was on + Neal's fork but we may want to consider a branch elsewhere) - [ ] Bump the Version in DESCRIPTION to X.X.X - [ ] Regenerate arrow_X.X.X.tar.tz (via `make release`?) @@ -91,8 +84,8 @@ Finalize and merge draft PRs/ensure linux binary packages are available: - [ ] PR into autobrew/homebrew-core - [ ] PR into autobrew/scripts - [ ] PR into r-windows/rtools-packages -- [ ] Ensure linux binaries are available (TODO: maybe document where these live? - I don't remember where this is at the top of my head) +- [ ] Ensure linux binaries are available in the artifactory: + https://apache.jfrog.io/ui/repos/tree/General/arrow/r Check binary Arrow C++ distributions specific to the R package: @@ -111,14 +104,13 @@ Submit! Wait for CRAN... -Post release: - - [ ] Accepted! -- [ ] Tag the tip of the CRAN-specific release branch? -- [ ] Bump the version number in DESCRIPTION to XXX? +- [ ] Tag the tip of the CRAN-specific release branch +- [ ] `usethis::use_dev_version()` - [ ] Add a new line to the matrix in the [backwards compatability job](https://github.com/apache/arrow/blob/master/dev/tasks/r/github.linux.arrow.version.back.compat.yml) - [ ] Update the packaging checklist template to reflect any new realities of the packaging process. - [ ] Wait for CRAN-hosted binaries on the [CRAN package page](https://cran.r-project.org/package=arrow) to reflect the new version +- [ ] Tweet! From cd73564977dd98062cd8bb0c86b3105a69213233 Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Tue, 22 Nov 2022 09:47:57 -0400 Subject: [PATCH 03/33] update the R section of the release guide --- docs/source/developers/release.rst | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/docs/source/developers/release.rst b/docs/source/developers/release.rst index 0263ce17a2c..235c0bb689e 100644 --- a/docs/source/developers/release.rst +++ b/docs/source/developers/release.rst @@ -487,7 +487,13 @@ Be sure to go through on the following checklist: :class-title: sd-fs-5 :class-container: sd-shadow-md - TODO: Move from https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-UpdatingRpackages + The R package is distributed on the Comprehensive R Archive Network (CRAN), + which has an involved submission process with a number of automated and + manual checks. The pre- and post- release preparation for this packaging + is documented in the + `Confluence release guide `_ + and the + `packaging checklist `_. .. dropdown:: Update vcpkg port :animate: fade-in-slide-down From 62d4a33d1cb020daf6438e94addedd611d0dd400 Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Tue, 22 Nov 2022 15:12:32 -0400 Subject: [PATCH 04/33] Update r/PACKAGING.md Co-authored-by: Neal Richardson --- r/PACKAGING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/r/PACKAGING.md b/r/PACKAGING.md index 3d9ca048b1a..f0e48371680 100644 --- a/r/PACKAGING.md +++ b/r/PACKAGING.md @@ -76,7 +76,7 @@ Wait for the official release... - [ ] Create a CRAN-release branch (TODO: previously this branch was on Neal's fork but we may want to consider a branch elsewhere) - [ ] Bump the Version in DESCRIPTION to X.X.X -- [ ] Regenerate arrow_X.X.X.tar.tz (via `make release`?) +- [ ] Regenerate arrow_X.X.X.tar.gz (via `make release`?) Finalize and merge draft PRs/ensure linux binary packages are available: From 938627467890a0c4ca909076c413c62fbd25dba8 Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Tue, 22 Nov 2022 15:12:57 -0400 Subject: [PATCH 05/33] Update r/PACKAGING.md Co-authored-by: Neal Richardson --- r/PACKAGING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/r/PACKAGING.md b/r/PACKAGING.md index f0e48371680..9eaa82edc39 100644 --- a/r/PACKAGING.md +++ b/r/PACKAGING.md @@ -89,7 +89,7 @@ Finalize and merge draft PRs/ensure linux binary packages are available: Check binary Arrow C++ distributions specific to the R package: -- [ ] Upload the .tar.gz to [WinBuilder](https://win-builder.r-project.org/upload.aspx) +- [ ] Upload the .tar.gz to [win-builder](https://win-builder.r-project.org/upload.aspx) and confirm that the check is clean - [ ] Upload the .tar.gz to [MacBuilder](https://mac.r-project.org/macbuilder/submit.html) and confirm that the check is clean From 8ae4993221eeb5b72448308a9cee986c6160a434 Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Tue, 22 Nov 2022 15:17:51 -0400 Subject: [PATCH 06/33] apply review suggestions --- docs/source/developers/release.rst | 2 +- r/PACKAGING.md | 31 +++++++++++++++--------------- 2 files changed, 17 insertions(+), 16 deletions(-) diff --git a/docs/source/developers/release.rst b/docs/source/developers/release.rst index 235c0bb689e..d5ac17b5ace 100644 --- a/docs/source/developers/release.rst +++ b/docs/source/developers/release.rst @@ -493,7 +493,7 @@ Be sure to go through on the following checklist: is documented in the `Confluence release guide `_ and the - `packaging checklist `_. + `packaging checklist `_. .. dropdown:: Update vcpkg port :animate: fade-in-slide-down diff --git a/r/PACKAGING.md b/r/PACKAGING.md index 9eaa82edc39..9e7af7ca7f3 100644 --- a/r/PACKAGING.md +++ b/r/PACKAGING.md @@ -30,28 +30,29 @@ For full documentation of the steps in this checklist, see the - [ ] Confirm that all [nightly tests and nightly packaging builds](https://lists.apache.org/list.html?builds@arrow.apache.org) are passing. - [ ] Check [current CRAN check results](https://cran.rstudio.org/web/checks/check_results_arrow.html) -- [ ] Check README +- [ ] Ensure the contents of the README is accurate - [ ] `urlchecker::url_check()` - [ ] [Polish NEWS](https://style.tidyverse.org/news.html#news-release) - [ ] Prepare tweet thread highlighting new features -Make draft pull requests into the [autobrew](https://github.com/autobrew) and +Make pull requests into the [autobrew](https://github.com/autobrew) and [rtools-packages](https://github.com/r-windows/rtools-packages) repositories -used by the configure script on MacOS and Windows: +used by the configure script on MacOS and Windows. These pull requests will +use the release candidate as the source. -- [ ] Open a draft pull request to modify +- [ ] Pull request to modify [the apache-arrow autobrew formula]( https://github.com/autobrew/homebrew-core/blob/master/Formula/apache-arrow.rb) to update the version, SHA, and any changes to dependencies and build steps that have changed in the [copy of the formula we have of that formula in the Arrow repo](https://github.com/apache/arrow/blob/master/dev/tasks/homebrew-formulae/autobrew/apache-arrow.rb) -- [ ] Open a draft pull request to modify +- [ ] Pull request to modify [the apache-arrow-static autobrew formula]( https://github.com/autobrew/homebrew-core/blob/master/Formula/apache-arrow-static.rb) to update the version, SHA, and any changes to dependencies and build steps that have changed in the [copy of the formula we have of that formula in the Arrow repo](https://github.com/apache/arrow/blob/master/dev/tasks/homebrew-formulae/autobrew/apache-arrow-static.rb) -- [ ] Open a draft pull request to modify the +- [ ] Pull request to modify the [autobrew script](https://github.com/autobrew/scripts/blob/master/apache-arrow) to include any additions made to [r/tools/autobrew](https://github.com/apache/arrow/blob/master/r/tools/autobrew). -- [ ] Open a draft pull request to modify the +- [ ] Pull request to modify the [PKGBUILD script](https://github.com/r-windows/rtools-packages/blob/master/mingw-w64-arrow/PKGBUILD) to reflect changes in [ci/PKGBUILD](https://github.com/apache/arrow/blob/master/ci/scripts/PKGBUILD), @@ -66,19 +67,19 @@ Prepare and check the .tar.gz that will be released to CRAN. - [ ] `devtools::check("arrow_X.X.X.tar.gz")` locally on Windows - [ ] `devtools::check("arrow_X.X.X.tar.gz")` locally on MacOS - [ ] `devtools::check("arrow_X.X.X.tar.gz")` locally on MacOS M1 -- [ ] `devtools::check("arrow_X.X.X.tar.gz")` locally on Linux -- [ ] `rhub::check("arrow_X.X.X.tar.gz", platform=c("debian-gcc-patched", "fedora-clang-devel"))` - [ ] Run reverse dependency checks Wait for the official release... -- [ ] Wait for the release vote to pass -- [ ] Create a CRAN-release branch (TODO: previously this branch was on - Neal's fork but we may want to consider a branch elsewhere) +- [ ] Release vote passed! +- [ ] Create a CRAN-release branch from the official release commit +- [ ] Remove badges from README.md +- [ ] `urlchecker::url_check()` - [ ] Bump the Version in DESCRIPTION to X.X.X -- [ ] Regenerate arrow_X.X.X.tar.gz (via `make release`?) +- [ ] Regenerate arrow_X.X.X.tar.gz (i.e., `make build`) -Finalize and merge draft PRs/ensure linux binary packages are available: +Update autobrew and r-windows PRs to use the *release* instead of the +*release candidate*; ensure linux binary packages are available: - [ ] PR into autobrew/homebrew-core - [ ] PR into autobrew/homebrew-core @@ -95,7 +96,7 @@ Check binary Arrow C++ distributions specific to the R package: and confirm that the check is clean - [ ] Check `install.packages("arrow_X.X.X.tar.gz")` on Ubuntu and ensure that the hosted binaries are used - + Submit! - [ ] Upload arrow_X.X.X.tar.gz to the From 14f7af23fb97a59dae86f8ed0675a10d4064a0ba Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Tue, 22 Nov 2022 15:27:30 -0400 Subject: [PATCH 07/33] copy wiki content --- docs/source/developers/release.rst | 38 +++++++++++++++++++++++++----- r/PACKAGING.md | 2 +- 2 files changed, 33 insertions(+), 7 deletions(-) diff --git a/docs/source/developers/release.rst b/docs/source/developers/release.rst index d5ac17b5ace..c8b7e15a140 100644 --- a/docs/source/developers/release.rst +++ b/docs/source/developers/release.rst @@ -487,13 +487,39 @@ Be sure to go through on the following checklist: :class-title: sd-fs-5 :class-container: sd-shadow-md - The R package is distributed on the Comprehensive R Archive Network (CRAN), - which has an involved submission process with a number of automated and - manual checks. The pre- and post- release preparation for this packaging - is documented in the - `Confluence release guide `_ - and the + To publish the R package on CRAN, there are a few steps we need to do first + in order to ensure that binaries for Windows and macOS are available to CRAN. + Jeroen Ooms maintains several projects that build C++ + dependencies for R packages for macOS and Windows. We test copies of these + same build scripts in our CI, and at release time, we need to send any + changes we have and update the versions/hashes upstream. + + When the release candidate is made, make draft pull requests to each + repository using the rc, updating the version and SHA, as well as any cmake + build changes from the corresponding files in apache/arrow. Jeroen may + merge these PRs before the release vote passes, build the binary artifacts, + and publish them in the right places so that we can do pre-submission checks + (see below). After the release candidate vote passes, update these PRs + to point to the official (non-rc) URL and mark them as ready for review. + Jeroen will merge, build the binary artifacts, and publish them in the + right places. See the `packaging checklist `_. + for a precise list of pull requests that must be made prior to submission + to CRAN. + + Once these binary prerequisites have been satisfied, we can submit to CRAN. + Given the vagaries of the process, it is best if the R developers on the + project verify the CRAN-worthiness of the package before submitting. + Our CI systems give us some coverage for the things that CRAN checks, but + there are a couple of final tests we should do to confirm that the release + binaries will work and that everything runs on the same infrastructure that + CRAN has, which is difficult/impossible to emulate fully on Travis or with + Docker. For a precise list of checks, see the + `packaging checklist `_. + + Once all checks are clean, we submit to CRAN, which has a web form for + uploading packages. The release process requires email confirmation + from the R package maintainer, currently Neal Richardson. .. dropdown:: Update vcpkg port :animate: fade-in-slide-down diff --git a/r/PACKAGING.md b/r/PACKAGING.md index 9e7af7ca7f3..4099b9c9e9e 100644 --- a/r/PACKAGING.md +++ b/r/PACKAGING.md @@ -21,7 +21,7 @@ # Packaging checklist for CRAN release For full documentation of the steps in this checklist, see the -[Apache Arrow Release Management Guide](https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-UpdatingRpackages). +[Apache Arrow Release Management Guide](https://arrow.apache.org/docs/developers/release.html#post-release-tasks). - [ ] When a release candidate is created [Create a GitHub issue](https://github.com/apache/arrow/issues/new/) From a58be680ca3521014939f5fdbd2ef1e86cbca1dc Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Tue, 29 Nov 2022 10:56:28 -0400 Subject: [PATCH 08/33] updates --- r/PACKAGING.md | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/r/PACKAGING.md b/r/PACKAGING.md index 4099b9c9e9e..cf03f299569 100644 --- a/r/PACKAGING.md +++ b/r/PACKAGING.md @@ -20,18 +20,22 @@ # Packaging checklist for CRAN release -For full documentation of the steps in this checklist, see the +For a high-level overview of the release process see the [Apache Arrow Release Management Guide](https://arrow.apache.org/docs/developers/release.html#post-release-tasks). - [ ] When a release candidate is created [Create a GitHub issue](https://github.com/apache/arrow/issues/new/) entitled `[R] CRAN release version X.X.X` and copy this checklist to the issue -- [ ] Confirm that all - [nightly tests and nightly packaging builds](https://lists.apache.org/list.html?builds@arrow.apache.org) are passing. +- [ ] Evaluate the status of any failing + [nightly tests and nightly packaging builds](https://lists.apache.org/list.html?builds@arrow.apache.org): these are run + against the master branch and not the release branch, so the results may be + slightly out of date. After the CRAN-specific release branch has been created, + we can run any or all tests again using a GitHub PR and crossbow. - [ ] Check [current CRAN check results](https://cran.rstudio.org/web/checks/check_results_arrow.html) - [ ] Ensure the contents of the README is accurate -- [ ] `urlchecker::url_check()` +- [ ] Run `urlchecker::url_check()` on the R directory at the release candidate + commit. - [ ] [Polish NEWS](https://style.tidyverse.org/news.html#news-release) - [ ] Prepare tweet thread highlighting new features @@ -63,18 +67,17 @@ Prepare and check the .tar.gz that will be released to CRAN. - [ ] `git fetch upstream && git checkout release-X.X.X-rcXX` - [ ] `make build` (copies Arrow C++ into tools/cpp, prunes some unnecessary components, and runs `R CMD build`) -- [ ] `devtools::check("arrow_X.X.X.tar.gz")` locally on Linux -- [ ] `devtools::check("arrow_X.X.X.tar.gz")` locally on Windows -- [ ] `devtools::check("arrow_X.X.X.tar.gz")` locally on MacOS -- [ ] `devtools::check("arrow_X.X.X.tar.gz")` locally on MacOS M1 +- [ ] `devtools::check("arrow_X.X.X.tar.gz")` locally - [ ] Run reverse dependency checks Wait for the official release... - [ ] Release vote passed! - [ ] Create a CRAN-release branch from the official release commit +- [ ] Pick any commits that were made to master since the release commit that + were needed to fix CRAN-related submission issues (e.g., fixed URLs) - [ ] Remove badges from README.md -- [ ] `urlchecker::url_check()` +- [ ] Run `urlchecker::url_check()` on the R directory - [ ] Bump the Version in DESCRIPTION to X.X.X - [ ] Regenerate arrow_X.X.X.tar.gz (i.e., `make build`) @@ -96,6 +99,7 @@ Check binary Arrow C++ distributions specific to the R package: and confirm that the check is clean - [ ] Check `install.packages("arrow_X.X.X.tar.gz")` on Ubuntu and ensure that the hosted binaries are used +- [ ] `devtools::check("arrow_X.X.X.tar.gz")` locally one more time (for luck) Submit! From 6c23b74324b9a1ed275a2f3801beede49ffa946a Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Tue, 29 Nov 2022 10:59:01 -0400 Subject: [PATCH 09/33] ignore badges --- r/PACKAGING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/r/PACKAGING.md b/r/PACKAGING.md index cf03f299569..9b9c670598a 100644 --- a/r/PACKAGING.md +++ b/r/PACKAGING.md @@ -35,7 +35,7 @@ For a high-level overview of the release process see the - [ ] Check [current CRAN check results](https://cran.rstudio.org/web/checks/check_results_arrow.html) - [ ] Ensure the contents of the README is accurate - [ ] Run `urlchecker::url_check()` on the R directory at the release candidate - commit. + commit. Ignore any errors with badges as they will be removed in the CRAN release branch. - [ ] [Polish NEWS](https://style.tidyverse.org/news.html#news-release) - [ ] Prepare tweet thread highlighting new features From 62a65f042a883a140f628ab7ba8f995f6f48a91e Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Tue, 29 Nov 2022 12:22:05 -0400 Subject: [PATCH 10/33] a few more updates --- r/PACKAGING.md | 26 +++++++++++++++++--------- 1 file changed, 17 insertions(+), 9 deletions(-) diff --git a/r/PACKAGING.md b/r/PACKAGING.md index 9b9c670598a..317bb9426be 100644 --- a/r/PACKAGING.md +++ b/r/PACKAGING.md @@ -23,15 +23,15 @@ For a high-level overview of the release process see the [Apache Arrow Release Management Guide](https://arrow.apache.org/docs/developers/release.html#post-release-tasks). -- [ ] When a release candidate is created - [Create a GitHub issue](https://github.com/apache/arrow/issues/new/) - entitled `[R] CRAN release version X.X.X` and copy this checklist to the issue - +Before the release candidate is cut: + +- [ ] [Create a GitHub issue](https://github.com/apache/arrow/issues/new/) + entitled `[R] CRAN packaging checklist for version X.X.X` + and copy this checklist to the issue. - [ ] Evaluate the status of any failing - [nightly tests and nightly packaging builds](https://lists.apache.org/list.html?builds@arrow.apache.org): these are run - against the master branch and not the release branch, so the results may be - slightly out of date. After the CRAN-specific release branch has been created, - we can run any or all tests again using a GitHub PR and crossbow. + [nightly tests and nightly packaging builds](https://lists.apache.org/list.html?builds@arrow.apache.org). These checks + replicate most of the checks that CRAN runs, so we need them all to be passing + or to understand that the failures will not result in a rejection from CRAN. - [ ] Check [current CRAN check results](https://cran.rstudio.org/web/checks/check_results_arrow.html) - [ ] Ensure the contents of the README is accurate - [ ] Run `urlchecker::url_check()` on the R directory at the release candidate @@ -39,6 +39,10 @@ For a high-level overview of the release process see the - [ ] [Polish NEWS](https://style.tidyverse.org/news.html#news-release) - [ ] Prepare tweet thread highlighting new features +Wait for the release candidate to be cut: + +- [ ] Release candidate! + Make pull requests into the [autobrew](https://github.com/autobrew) and [rtools-packages](https://github.com/r-windows/rtools-packages) repositories used by the configure script on MacOS and Windows. These pull requests will @@ -75,10 +79,14 @@ Wait for the official release... - [ ] Release vote passed! - [ ] Create a CRAN-release branch from the official release commit - [ ] Pick any commits that were made to master since the release commit that - were needed to fix CRAN-related submission issues (e.g., fixed URLs) + were needed to fix CRAN-related submission issues (e.g., fixed URLs, fixes + to ) - [ ] Remove badges from README.md - [ ] Run `urlchecker::url_check()` on the R directory - [ ] Bump the Version in DESCRIPTION to X.X.X +- [ ] Create a PR entitled `WIP: [R] Verify CRAN release-10.0.1-rc0`. Add + a comment `@github-actions crossbow submit --group r` to run all R crossbow + jobs against the CRAN-specific release branch. - [ ] Regenerate arrow_X.X.X.tar.gz (i.e., `make build`) Update autobrew and r-windows PRs to use the *release* instead of the From a2a4ac1b1b1ae27521ef98c7132dbf13a8d09f73 Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Tue, 29 Nov 2022 13:08:50 -0400 Subject: [PATCH 11/33] Update r/PACKAGING.md Co-authored-by: Nic Crane --- r/PACKAGING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/r/PACKAGING.md b/r/PACKAGING.md index 317bb9426be..20bbf86d7c6 100644 --- a/r/PACKAGING.md +++ b/r/PACKAGING.md @@ -31,7 +31,7 @@ Before the release candidate is cut: - [ ] Evaluate the status of any failing [nightly tests and nightly packaging builds](https://lists.apache.org/list.html?builds@arrow.apache.org). These checks replicate most of the checks that CRAN runs, so we need them all to be passing - or to understand that the failures will not result in a rejection from CRAN. + or to understand that the failures may (though won't necessarily) result in a rejection from CRAN. - [ ] Check [current CRAN check results](https://cran.rstudio.org/web/checks/check_results_arrow.html) - [ ] Ensure the contents of the README is accurate - [ ] Run `urlchecker::url_check()` on the R directory at the release candidate From f982caac45ecdeb27806e47a1d5222d17ec73ebe Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Tue, 29 Nov 2022 13:18:17 -0400 Subject: [PATCH 12/33] qualify the tweet thread bullet --- r/PACKAGING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/r/PACKAGING.md b/r/PACKAGING.md index 20bbf86d7c6..627f38f8c74 100644 --- a/r/PACKAGING.md +++ b/r/PACKAGING.md @@ -37,7 +37,7 @@ Before the release candidate is cut: - [ ] Run `urlchecker::url_check()` on the R directory at the release candidate commit. Ignore any errors with badges as they will be removed in the CRAN release branch. - [ ] [Polish NEWS](https://style.tidyverse.org/news.html#news-release) -- [ ] Prepare tweet thread highlighting new features +- [ ] For major releases, prepare tweet thread highlighting new features Wait for the release candidate to be cut: From d302439b075e7cfa8fa3a50c213ad937b8280ddb Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Tue, 29 Nov 2022 13:27:03 -0400 Subject: [PATCH 13/33] Update r/PACKAGING.md Co-authored-by: Nic Crane --- r/PACKAGING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/r/PACKAGING.md b/r/PACKAGING.md index 627f38f8c74..e7e031020af 100644 --- a/r/PACKAGING.md +++ b/r/PACKAGING.md @@ -36,7 +36,7 @@ Before the release candidate is cut: - [ ] Ensure the contents of the README is accurate - [ ] Run `urlchecker::url_check()` on the R directory at the release candidate commit. Ignore any errors with badges as they will be removed in the CRAN release branch. -- [ ] [Polish NEWS](https://style.tidyverse.org/news.html#news-release) +- [ ] [Polish NEWS](https://style.tidyverse.org/news.html#news-release) but do **not** update version numbers (this is done automatically later). - [ ] For major releases, prepare tweet thread highlighting new features Wait for the release candidate to be cut: From 5e4fd7a6239edf479948ace71a3a48c486671423 Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Tue, 29 Nov 2022 13:49:08 -0400 Subject: [PATCH 14/33] fix url --- r/PACKAGING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/r/PACKAGING.md b/r/PACKAGING.md index e7e031020af..666262c4407 100644 --- a/r/PACKAGING.md +++ b/r/PACKAGING.md @@ -49,7 +49,7 @@ used by the configure script on MacOS and Windows. These pull requests will use the release candidate as the source. - [ ] Pull request to modify - [the apache-arrow autobrew formula]( https://github.com/autobrew/homebrew-core/blob/master/Formula/apache-arrow.rb) + [the apache-arrow autobrew formula]( https://github.com/autobrew/homebrew-core/blob/high-sierra/Formula/apache-arrow.rb) to update the version, SHA, and any changes to dependencies and build steps that have changed in the [copy of the formula we have of that formula in the Arrow repo](https://github.com/apache/arrow/blob/master/dev/tasks/homebrew-formulae/autobrew/apache-arrow.rb) - [ ] Pull request to modify From 282e78be33d8038d9288357a10f84cffffdf3116 Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Tue, 29 Nov 2022 14:19:35 -0400 Subject: [PATCH 15/33] Update r/PACKAGING.md Co-authored-by: Nic Crane --- r/PACKAGING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/r/PACKAGING.md b/r/PACKAGING.md index 666262c4407..f7858e1d33c 100644 --- a/r/PACKAGING.md +++ b/r/PACKAGING.md @@ -50,7 +50,7 @@ use the release candidate as the source. - [ ] Pull request to modify [the apache-arrow autobrew formula]( https://github.com/autobrew/homebrew-core/blob/high-sierra/Formula/apache-arrow.rb) - to update the version, SHA, and any changes to dependencies and build steps that have changed in the + to update the release version, SHA256 checksum of the release source file, and any changes to dependencies and build steps that have changed in the [copy of the formula we have of that formula in the Arrow repo](https://github.com/apache/arrow/blob/master/dev/tasks/homebrew-formulae/autobrew/apache-arrow.rb) - [ ] Pull request to modify [the apache-arrow-static autobrew formula]( https://github.com/autobrew/homebrew-core/blob/master/Formula/apache-arrow-static.rb) From 780ec18d6572f80cebe76b3ff478310d62166208 Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Wed, 30 Nov 2022 10:20:09 -0400 Subject: [PATCH 16/33] Update r/PACKAGING.md Co-authored-by: Nic Crane --- r/PACKAGING.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/r/PACKAGING.md b/r/PACKAGING.md index f7858e1d33c..6be5d2bc0a9 100644 --- a/r/PACKAGING.md +++ b/r/PACKAGING.md @@ -92,8 +92,8 @@ Wait for the official release... Update autobrew and r-windows PRs to use the *release* instead of the *release candidate*; ensure linux binary packages are available: -- [ ] PR into autobrew/homebrew-core -- [ ] PR into autobrew/homebrew-core +- [ ] PR into autobrew/homebrew-core (apache-arrow autobrew formula) +- [ ] PR into autobrew/homebrew-core (apache-arrow-static autobrew formula) - [ ] PR into autobrew/scripts - [ ] PR into r-windows/rtools-packages - [ ] Ensure linux binaries are available in the artifactory: From ab41e71afd31ffab6c1074a0ef211302cfa46add Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Wed, 30 Nov 2022 10:25:41 -0400 Subject: [PATCH 17/33] Update r/PACKAGING.md Co-authored-by: Nic Crane --- r/PACKAGING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/r/PACKAGING.md b/r/PACKAGING.md index 6be5d2bc0a9..d6e8be4604b 100644 --- a/r/PACKAGING.md +++ b/r/PACKAGING.md @@ -68,7 +68,7 @@ use the release candidate as the source. Prepare and check the .tar.gz that will be released to CRAN. -- [ ] `git fetch upstream && git checkout release-X.X.X-rcXX` +- [ ] `git fetch upstream && git checkout release-X.X.X-rcXX && git clean -f -d` - [ ] `make build` (copies Arrow C++ into tools/cpp, prunes some unnecessary components, and runs `R CMD build`) - [ ] `devtools::check("arrow_X.X.X.tar.gz")` locally From dca9b36aa559a58c3395dab2fe30a3fb8d00c580 Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Wed, 30 Nov 2022 10:43:57 -0400 Subject: [PATCH 18/33] Update r/PACKAGING.md Co-authored-by: Nic Crane --- r/PACKAGING.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/r/PACKAGING.md b/r/PACKAGING.md index d6e8be4604b..d4feb9f5840 100644 --- a/r/PACKAGING.md +++ b/r/PACKAGING.md @@ -69,8 +69,7 @@ use the release candidate as the source. Prepare and check the .tar.gz that will be released to CRAN. - [ ] `git fetch upstream && git checkout release-X.X.X-rcXX && git clean -f -d` -- [ ] `make build` (copies Arrow C++ into tools/cpp, prunes some unnecessary - components, and runs `R CMD build`) +- [ ] Unset the `ARROW_HOME` environment variable and then run `make build` (copies Arrow C++ into tools/cpp, prunes some unnecessary components, and runs `R CMD build`) - [ ] `devtools::check("arrow_X.X.X.tar.gz")` locally - [ ] Run reverse dependency checks From 36bd3e534907060452d665017e7debb2784d018c Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Wed, 30 Nov 2022 11:04:50 -0400 Subject: [PATCH 19/33] Update r/PACKAGING.md Co-authored-by: Nic Crane --- r/PACKAGING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/r/PACKAGING.md b/r/PACKAGING.md index d4feb9f5840..e1ad386f400 100644 --- a/r/PACKAGING.md +++ b/r/PACKAGING.md @@ -70,7 +70,7 @@ Prepare and check the .tar.gz that will be released to CRAN. - [ ] `git fetch upstream && git checkout release-X.X.X-rcXX && git clean -f -d` - [ ] Unset the `ARROW_HOME` environment variable and then run `make build` (copies Arrow C++ into tools/cpp, prunes some unnecessary components, and runs `R CMD build`) -- [ ] `devtools::check("arrow_X.X.X.tar.gz")` locally +- [ ] `devtools::check_built("arrow_X.X.X.tar.gz")` locally - [ ] Run reverse dependency checks Wait for the official release... From 97b350ecda8c2cb47e4c1aa9ffa6a7407a578068 Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Wed, 30 Nov 2022 15:16:12 -0400 Subject: [PATCH 20/33] Update r/PACKAGING.md Co-authored-by: Neal Richardson --- r/PACKAGING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/r/PACKAGING.md b/r/PACKAGING.md index e1ad386f400..ff5fbb0af9b 100644 --- a/r/PACKAGING.md +++ b/r/PACKAGING.md @@ -33,7 +33,7 @@ Before the release candidate is cut: replicate most of the checks that CRAN runs, so we need them all to be passing or to understand that the failures may (though won't necessarily) result in a rejection from CRAN. - [ ] Check [current CRAN check results](https://cran.rstudio.org/web/checks/check_results_arrow.html) -- [ ] Ensure the contents of the README is accurate +- [ ] Ensure the contents of the README are accurate and up to date - [ ] Run `urlchecker::url_check()` on the R directory at the release candidate commit. Ignore any errors with badges as they will be removed in the CRAN release branch. - [ ] [Polish NEWS](https://style.tidyverse.org/news.html#news-release) but do **not** update version numbers (this is done automatically later). From afe23c2646c6181fc637d27a3733f975691c05e7 Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Wed, 30 Nov 2022 15:24:51 -0400 Subject: [PATCH 21/33] Update r/PACKAGING.md Co-authored-by: Neal Richardson --- r/PACKAGING.md | 1 - 1 file changed, 1 deletion(-) diff --git a/r/PACKAGING.md b/r/PACKAGING.md index ff5fbb0af9b..82e0f858d35 100644 --- a/r/PACKAGING.md +++ b/r/PACKAGING.md @@ -82,7 +82,6 @@ Wait for the official release... to ) - [ ] Remove badges from README.md - [ ] Run `urlchecker::url_check()` on the R directory -- [ ] Bump the Version in DESCRIPTION to X.X.X - [ ] Create a PR entitled `WIP: [R] Verify CRAN release-10.0.1-rc0`. Add a comment `@github-actions crossbow submit --group r` to run all R crossbow jobs against the CRAN-specific release branch. From 567ee14d36d1c10f6a20173d6c473bc5034270ec Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Wed, 30 Nov 2022 15:27:21 -0400 Subject: [PATCH 22/33] Update r/PACKAGING.md Co-authored-by: Neal Richardson --- r/PACKAGING.md | 1 - 1 file changed, 1 deletion(-) diff --git a/r/PACKAGING.md b/r/PACKAGING.md index 82e0f858d35..6a64bfc36da 100644 --- a/r/PACKAGING.md +++ b/r/PACKAGING.md @@ -117,7 +117,6 @@ Wait for CRAN... - [ ] Accepted! - [ ] Tag the tip of the CRAN-specific release branch -- [ ] `usethis::use_dev_version()` - [ ] Add a new line to the matrix in the [backwards compatability job](https://github.com/apache/arrow/blob/master/dev/tasks/r/github.linux.arrow.version.back.compat.yml) - [ ] Update the packaging checklist template to reflect any new realities of the packaging process. From 4e9c0b3763ddbf5f5631840d250bfa3b111025bf Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Wed, 30 Nov 2022 15:40:35 -0400 Subject: [PATCH 23/33] edits for Neal's review --- r/PACKAGING.md | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/r/PACKAGING.md b/r/PACKAGING.md index 6a64bfc36da..583f38e437f 100644 --- a/r/PACKAGING.md +++ b/r/PACKAGING.md @@ -29,7 +29,7 @@ Before the release candidate is cut: entitled `[R] CRAN packaging checklist for version X.X.X` and copy this checklist to the issue. - [ ] Evaluate the status of any failing - [nightly tests and nightly packaging builds](https://lists.apache.org/list.html?builds@arrow.apache.org). These checks + [nightly tests and nightly packaging builds](http://crossbow.voltrondata.com). These checks replicate most of the checks that CRAN runs, so we need them all to be passing or to understand that the failures may (though won't necessarily) result in a rejection from CRAN. - [ ] Check [current CRAN check results](https://cran.rstudio.org/web/checks/check_results_arrow.html) @@ -42,6 +42,7 @@ Before the release candidate is cut: Wait for the release candidate to be cut: - [ ] Release candidate! +- [ ] Create a CRAN-release branch from the release candidate commit Make pull requests into the [autobrew](https://github.com/autobrew) and [rtools-packages](https://github.com/r-windows/rtools-packages) repositories @@ -69,17 +70,20 @@ use the release candidate as the source. Prepare and check the .tar.gz that will be released to CRAN. - [ ] `git fetch upstream && git checkout release-X.X.X-rcXX && git clean -f -d` -- [ ] Unset the `ARROW_HOME` environment variable and then run `make build` (copies Arrow C++ into tools/cpp, prunes some unnecessary components, and runs `R CMD build`) +- [ ] Run `make build` (copies Arrow C++ into tools/cpp, prunes some unnecessary components, and runs `R CMD build` to generate the source tarball that includes vignettes) - [ ] `devtools::check_built("arrow_X.X.X.tar.gz")` locally -- [ ] Run reverse dependency checks +- [ ] Run reverse dependency checks. Currently this is a + [manual process](https://gist.github.com/paleolimbot/630fdab1e204d70fea97633d8fa15ccb); + however, in the future it may be a crossbow nightly job. Wait for the official release... - [ ] Release vote passed! -- [ ] Create a CRAN-release branch from the official release commit +- [ ] If the release candidate commit updated, rebase the CRAN release branch + on that commit. - [ ] Pick any commits that were made to master since the release commit that - were needed to fix CRAN-related submission issues (e.g., fixed URLs, fixes - to ) + were needed to fix CRAN-related submission issues identified in the above + steps. - [ ] Remove badges from README.md - [ ] Run `urlchecker::url_check()` on the R directory - [ ] Create a PR entitled `WIP: [R] Verify CRAN release-10.0.1-rc0`. Add @@ -87,8 +91,8 @@ Wait for the official release... jobs against the CRAN-specific release branch. - [ ] Regenerate arrow_X.X.X.tar.gz (i.e., `make build`) -Update autobrew and r-windows PRs to use the *release* instead of the -*release candidate*; ensure linux binary packages are available: +Create new autobrew and r-windows PRs such that they use the *release* +instead of the *release candidate*; ensure linux binary packages are available: - [ ] PR into autobrew/homebrew-core (apache-arrow autobrew formula) - [ ] PR into autobrew/homebrew-core (apache-arrow-static autobrew formula) From ce003a9f14a196aa910df917e40260e32fba5094 Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Wed, 30 Nov 2022 16:12:47 -0400 Subject: [PATCH 24/33] Update packageing --- r/PACKAGING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/r/PACKAGING.md b/r/PACKAGING.md index 583f38e437f..07d291d1f57 100644 --- a/r/PACKAGING.md +++ b/r/PACKAGING.md @@ -49,7 +49,7 @@ Make pull requests into the [autobrew](https://github.com/autobrew) and used by the configure script on MacOS and Windows. These pull requests will use the release candidate as the source. -- [ ] Pull request to modify +- [ ] Pull request to modify [the apache-arrow autobrew formula]( https://github.com/autobrew/homebrew-core/blob/high-sierra/Formula/apache-arrow.rb) to update the release version, SHA256 checksum of the release source file, and any changes to dependencies and build steps that have changed in the [copy of the formula we have of that formula in the Arrow repo](https://github.com/apache/arrow/blob/master/dev/tasks/homebrew-formulae/autobrew/apache-arrow.rb) From 148b47160061fd68009c2819fdabe4054cb967f7 Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Thu, 1 Dec 2022 12:20:09 -0400 Subject: [PATCH 25/33] clarify the `make build` step --- r/PACKAGING.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/r/PACKAGING.md b/r/PACKAGING.md index 07d291d1f57..8aa745e8333 100644 --- a/r/PACKAGING.md +++ b/r/PACKAGING.md @@ -70,7 +70,11 @@ use the release candidate as the source. Prepare and check the .tar.gz that will be released to CRAN. - [ ] `git fetch upstream && git checkout release-X.X.X-rcXX && git clean -f -d` -- [ ] Run `make build` (copies Arrow C++ into tools/cpp, prunes some unnecessary components, and runs `R CMD build` to generate the source tarball that includes vignettes) +- [ ] Run `make build`. This copies Arrow C++ into tools/cpp, prunes some + unnecessary components, and runs `R CMD build` to generate the source tarball. + Because this will install the package, you will need to ensure that the version + of Arrow C++ available to the configure script is the same as the version + that is vendored into the R package (e.g., you may need to unset `ARROW_HOME`). - [ ] `devtools::check_built("arrow_X.X.X.tar.gz")` locally - [ ] Run reverse dependency checks. Currently this is a [manual process](https://gist.github.com/paleolimbot/630fdab1e204d70fea97633d8fa15ccb); From 9cce455e18c6d70eb90b10cccdac2c72ed43bb46 Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Thu, 1 Dec 2022 13:02:09 -0400 Subject: [PATCH 26/33] Update r/PACKAGING.md Co-authored-by: Nic Crane --- r/PACKAGING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/r/PACKAGING.md b/r/PACKAGING.md index 8aa745e8333..329d4c902b4 100644 --- a/r/PACKAGING.md +++ b/r/PACKAGING.md @@ -62,7 +62,7 @@ use the release candidate as the source. to include any additions made to [r/tools/autobrew](https://github.com/apache/arrow/blob/master/r/tools/autobrew). - [ ] Pull request to modify the - [PKGBUILD script](https://github.com/r-windows/rtools-packages/blob/master/mingw-w64-arrow/PKGBUILD) + [RTools PKGBUILD script](https://github.com/r-windows/rtools-packages/blob/master/mingw-w64-arrow/PKGBUILD) to reflect changes in [ci/PKGBUILD](https://github.com/apache/arrow/blob/master/ci/scripts/PKGBUILD), uncommenting the line that says "uncomment to test the rc". From d2edb6e6a2be9104c178cb681ed4159b3ed4d23e Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Thu, 1 Dec 2022 13:22:19 -0400 Subject: [PATCH 27/33] Update r/PACKAGING.md Co-authored-by: Nic Crane --- r/PACKAGING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/r/PACKAGING.md b/r/PACKAGING.md index 329d4c902b4..3997b4bf05f 100644 --- a/r/PACKAGING.md +++ b/r/PACKAGING.md @@ -51,7 +51,7 @@ use the release candidate as the source. - [ ] Pull request to modify [the apache-arrow autobrew formula]( https://github.com/autobrew/homebrew-core/blob/high-sierra/Formula/apache-arrow.rb) - to update the release version, SHA256 checksum of the release source file, and any changes to dependencies and build steps that have changed in the + to update the release version, SHA256 checksum of the release source file (which can be found in the same directory as the release source file), and any changes to dependencies and build steps that have changed in the [copy of the formula we have of that formula in the Arrow repo](https://github.com/apache/arrow/blob/master/dev/tasks/homebrew-formulae/autobrew/apache-arrow.rb) - [ ] Pull request to modify [the apache-arrow-static autobrew formula]( https://github.com/autobrew/homebrew-core/blob/master/Formula/apache-arrow-static.rb) From 528541f0dbc3e01de782b0d08a044139c9db848b Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Thu, 1 Dec 2022 13:52:03 -0400 Subject: [PATCH 28/33] Update r/PACKAGING.md Co-authored-by: Nic Crane --- r/PACKAGING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/r/PACKAGING.md b/r/PACKAGING.md index 3997b4bf05f..8fbdae55702 100644 --- a/r/PACKAGING.md +++ b/r/PACKAGING.md @@ -108,7 +108,7 @@ instead of the *release candidate*; ensure linux binary packages are available: Check binary Arrow C++ distributions specific to the R package: - [ ] Upload the .tar.gz to [win-builder](https://win-builder.r-project.org/upload.aspx) - and confirm that the check is clean + and confirm (with Neal, who will automatically receive an email about the results) that the check is clean - [ ] Upload the .tar.gz to [MacBuilder](https://mac.r-project.org/macbuilder/submit.html) and confirm that the check is clean - [ ] Check `install.packages("arrow_X.X.X.tar.gz")` on Ubuntu and ensure that the From 1b9b843d6c39924987c640cb82d1378b9579c39e Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Thu, 1 Dec 2022 13:52:15 -0400 Subject: [PATCH 29/33] Update r/PACKAGING.md Co-authored-by: Nic Crane --- r/PACKAGING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/r/PACKAGING.md b/r/PACKAGING.md index 8fbdae55702..f7657e35816 100644 --- a/r/PACKAGING.md +++ b/r/PACKAGING.md @@ -113,7 +113,7 @@ Check binary Arrow C++ distributions specific to the R package: and confirm that the check is clean - [ ] Check `install.packages("arrow_X.X.X.tar.gz")` on Ubuntu and ensure that the hosted binaries are used -- [ ] `devtools::check("arrow_X.X.X.tar.gz")` locally one more time (for luck) +- [ ] `devtools::check_built("arrow_X.X.X.tar.gz")` locally one more time (for luck) Submit! From 1ec8b915953ba09b31c8eb068b82a91d0914e28b Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Thu, 1 Dec 2022 14:43:07 -0400 Subject: [PATCH 30/33] Update r/PACKAGING.md Co-authored-by: Nic Crane --- r/PACKAGING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/r/PACKAGING.md b/r/PACKAGING.md index f7657e35816..5431e7c09f4 100644 --- a/r/PACKAGING.md +++ b/r/PACKAGING.md @@ -107,7 +107,7 @@ instead of the *release candidate*; ensure linux binary packages are available: Check binary Arrow C++ distributions specific to the R package: -- [ ] Upload the .tar.gz to [win-builder](https://win-builder.r-project.org/upload.aspx) +- [ ] Upload the .tar.gz to [win-builder](https://win-builder.r-project.org/upload.aspx) (r-devel only) and confirm (with Neal, who will automatically receive an email about the results) that the check is clean - [ ] Upload the .tar.gz to [MacBuilder](https://mac.r-project.org/macbuilder/submit.html) and confirm that the check is clean From 3f8d2b59d187942161b36a88b32e943a11461e98 Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Thu, 1 Dec 2022 15:27:40 -0400 Subject: [PATCH 31/33] Update r/PACKAGING.md Co-authored-by: Nic Crane --- r/PACKAGING.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/r/PACKAGING.md b/r/PACKAGING.md index 5431e7c09f4..8c638c26540 100644 --- a/r/PACKAGING.md +++ b/r/PACKAGING.md @@ -108,7 +108,8 @@ instead of the *release candidate*; ensure linux binary packages are available: Check binary Arrow C++ distributions specific to the R package: - [ ] Upload the .tar.gz to [win-builder](https://win-builder.r-project.org/upload.aspx) (r-devel only) - and confirm (with Neal, who will automatically receive an email about the results) that the check is clean + and confirm (with Neal, who will automatically receive an email about the results) that the check is clean. + This step cannot be completed before Jeroen has put the binaries in the MinGW repository. - [ ] Upload the .tar.gz to [MacBuilder](https://mac.r-project.org/macbuilder/submit.html) and confirm that the check is clean - [ ] Check `install.packages("arrow_X.X.X.tar.gz")` on Ubuntu and ensure that the From a9811904f793d39f4e9d0eefda7de3a18fab99ff Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Thu, 1 Dec 2022 15:56:24 -0400 Subject: [PATCH 32/33] Update r/PACKAGING.md Co-authored-by: Nic Crane --- r/PACKAGING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/r/PACKAGING.md b/r/PACKAGING.md index 8c638c26540..105880df182 100644 --- a/r/PACKAGING.md +++ b/r/PACKAGING.md @@ -109,7 +109,7 @@ Check binary Arrow C++ distributions specific to the R package: - [ ] Upload the .tar.gz to [win-builder](https://win-builder.r-project.org/upload.aspx) (r-devel only) and confirm (with Neal, who will automatically receive an email about the results) that the check is clean. - This step cannot be completed before Jeroen has put the binaries in the MinGW repository. + This step cannot be completed before Jeroen has put the binaries in the MinGW repository, i.e. [here](https://ftp.opencpu.org/rtools/ucrt64/), [here](https://ftp.opencpu.org/rtools/mingw64/), and [here](https://ftp.opencpu.org/rtools/mingw32/). - [ ] Upload the .tar.gz to [MacBuilder](https://mac.r-project.org/macbuilder/submit.html) and confirm that the check is clean - [ ] Check `install.packages("arrow_X.X.X.tar.gz")` on Ubuntu and ensure that the From 6fd9d40b934a3f1b32cfe7db033cc449f0baa447 Mon Sep 17 00:00:00 2001 From: Dewey Dunnington Date: Thu, 8 Dec 2022 11:21:21 -0400 Subject: [PATCH 33/33] Update r/PACKAGING.md Co-authored-by: Nic Crane --- r/PACKAGING.md | 1 + 1 file changed, 1 insertion(+) diff --git a/r/PACKAGING.md b/r/PACKAGING.md index 105880df182..c2f53440652 100644 --- a/r/PACKAGING.md +++ b/r/PACKAGING.md @@ -127,6 +127,7 @@ Wait for CRAN... - [ ] Accepted! - [ ] Tag the tip of the CRAN-specific release branch - [ ] Add a new line to the matrix in the [backwards compatability job](https://github.com/apache/arrow/blob/master/dev/tasks/r/github.linux.arrow.version.back.compat.yml) +- [ ] (patch releases only) Update the package version in `ci/scripts/PKGBUILD`, `dev/tasks/homebrew-formulae/autobrew/apache-arrow.rb`, `r/DESCRIPTION`, and `r/NEWS.md` - [ ] Update the packaging checklist template to reflect any new realities of the packaging process. - [ ] Wait for CRAN-hosted binaries on the