GH-37941: [R][CI][Release] Add checksum verification for pre-compiled binaries #38115
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
assignUser
requested review from
kou,
paleolimbot,
raulcd and
thisisnic
as code owners
Member
Author
|
@github-actions crossbow submit r-binary-packages |
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
Member
Author
|
@github-actions crossbow submit r-binary-packages |
This comment was marked as outdated.
This comment was marked as outdated.
|
Revision: 4dae43c Submitted crossbow builds: ursacomputing/crossbow @ actions-88d6d2b34e
|
paleolimbot
approved these changes
Oct 10, 2023
Member
paleolimbot
left a comment
paleolimbot
left a comment
There was a problem hiding this comment.
Two small comments...thank you for taking this on!
Member
|
Also, it is probably worth rebasing to clear up the CI. |
github-actions
bot
added
awaiting merge
Member
Author
I merged to keep the crossbow job shas valid. I will run another round of validation but then this should be merge ready imo. |
Member
Author
|
@github-actions crossbow submit -g r |
|
Revision: dd292b6 Submitted crossbow builds: ursacomputing/crossbow @ actions-d4c5399398
|
paleolimbot
approved these changes
Oct 11, 2023
Member
paleolimbot
left a comment
paleolimbot
left a comment
There was a problem hiding this comment.
Pending green CI, naturally. Thank you!
github-actions
bot
added
awaiting changes
Member
nealrichardson
left a comment
nealrichardson
left a comment
There was a problem hiding this comment.
A few notes, apologies for the late review here @assignUser
| checksum_cmd <- "shasum" | ||
| checksum_args <- c("--status", "-a", "512", "-c", checksum_file) | ||
|
|
||
| # shasum is not available on all linux versions |
Member
There was a problem hiding this comment.
Use sys.which() to see if it's present?
github-actions
bot
added
awaiting changes
|
After merging your PR, Conbench analyzed the 5 benchmarking runs that have been run so far on merge-commit b20e0ae. There were no benchmark performance regressions. 🎉 The full Conbench report has more details. It also includes information about 2 possible false positives for unstable benchmarks that are known to sometimes produce them. |
loicalleyne
pushed a commit
to loicalleyne/arrow
that referenced
this pull request
Nov 13, 2023
…mpiled binaries (apache#38115) ### Rationale for this change This change is to restore parity with the previous solution on macOS (brew does cs validation) and improve security for windows and linux. This also align with CRAN policy. ### What changes are included in this PR? This PR adds a script that can be run after the arrow release (once all files have been pushed to the artifactory) before the CRAN submission to download the checksum files for the pre-compiled binaries which are already added through the usual release. *libs.R have been extended to use these checksum files to validate the downloaded binaries. ### Are these changes tested? The r-binary-packages nightlies generate checksums and use them when building binary packages, this way the code path is tested. They do not modify the actual src package though. ### Are there any user-facing changes? no (outside of log messages) * Closes: apache#37941 Authored-by: Jacob Wujciak-Jens <jacob@wujciak.de> Signed-off-by: Nic Crane <thisisnic@gmail.com>
dgreiss
pushed a commit
to dgreiss/arrow
that referenced
this pull request
Feb 19, 2024
…mpiled binaries (apache#38115) ### Rationale for this change This change is to restore parity with the previous solution on macOS (brew does cs validation) and improve security for windows and linux. This also align with CRAN policy. ### What changes are included in this PR? This PR adds a script that can be run after the arrow release (once all files have been pushed to the artifactory) before the CRAN submission to download the checksum files for the pre-compiled binaries which are already added through the usual release. *libs.R have been extended to use these checksum files to validate the downloaded binaries. ### Are these changes tested? The r-binary-packages nightlies generate checksums and use them when building binary packages, this way the code path is tested. They do not modify the actual src package though. ### Are there any user-facing changes? no (outside of log messages) * Closes: apache#37941 Authored-by: Jacob Wujciak-Jens <jacob@wujciak.de> Signed-off-by: Nic Crane <thisisnic@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Rationale for this change
This change is to restore parity with the previous solution on macOS (brew does cs validation) and improve security for windows and linux. This also align with CRAN policy.
What changes are included in this PR?
This PR adds a script that can be run after the arrow release (once all files have been pushed to the artifactory) before the CRAN submission to download the checksum files for the pre-compiled binaries which are already added through the usual release. *libs.R have been extended to use these checksum files to validate the downloaded binaries.
Are these changes tested?
The r-binary-packages nightlies generate checksums and use them when building binary packages, this way the code path is tested. They do not modify the actual src package though.
Are there any user-facing changes?
no (outside of log messages)