Introduce test_candidate.sh to automate testing release candidate artifacts

[jaydoane]

No description provided.

[janl]

final bit here could be fetching the lastest git-ish for the release line branch from GitHub and compare it against the one reported on GET /

[jaydoane]

I'm embarrassed to admit I don't know what you mean my the "git-ish for the release line branch" is. I don't think you mean the git sha of 3.4.x like so:

❯ git checkout origin/3.4.x
HEAD is now at cfc66654f Merge pull request #5308 from apache/nouveau-java-23-3.4.x

which would be cfc66654f, and doesn't seem to match e.g.

❯ curl -u adm:pass $D1 | jq .git_sha
"6e5ad2a5c"

[janl]

my version of this also pulls down the hashes and prints them next to the generated ones, so you can spot any difference right away.

Also slight preference to use curl as wget is non-default on BSDs and macOS.

[jaydoane]

my version of this also pulls down the hashes and prints them next to the generated ones, so you can spot any difference right away.

I'm happy to change it if you can show me an example of how your script does it.

Also, I tried to find a nice way to get curl to download an entire directory, but most things I read said that wget was the better tool for the job. Open to suggestions for the equivalent command though.

[pgj]

For what it is worth, on FreeBSD fetch is the default downloader tool, that is part of the base system. Both curl and wget could be installed as third-party packages, I am not sure if there is a generic preference for one or the other.

[janl]

I am realising the script does way more than what I want from it and in a way different way that I does not meet my goals, I’m not sure if I should further mess with it, or just keep things to myself

For example: one very important thing to me is validating checksums, especially the GPG one since that needs manual inspection, but that output is lost in the stdout stream.

I also don’t need the whole “do a test build for me” section as I have the macOS binary build scripts for that, but I can see how this can be useful, but also, I don’t feel like writing a cross-platform (unixy at least) build script, or maybe that is exactly what we should be doing, but that’d duplicate a lot of CI work and needs to keep maintained. IF we were to decide to do this, I’d suggest we put that bit into a separate script.

All that said, I’m not sure if:

• we should merge this as-is as it only helps folks who know what they are doing and if we want folks to test things properly, they better do it by hand
• OR we fix this up a little bit so it is useful for some folks
• OR make this the bells-and-whistles cross-platform script that we officially recommend folks use?

I’m sorry for making a lot of noise about a thing that might in the end not be worth merging after all. wdyt?

[jaydoane]

For example: one very important thing to me is validating checksums, especially the GPG one since that needs manual inspection, but that output is lost in the stdout stream.

Since the test_candidate.sh script begins with #!/bin/sh -ex, the shell should immediately exit if any command in the script returns a non-zero exit status, which means it should stop immediately if there's a gpg --verify failure, and so is maybe less likely to be lost in the stdout stream?

[jaydoane]

I also don’t need the whole “do a test build for me” section

Are you referring to make check and make release? I included those because the RC test instructions explicitly require running those commands.

It does begin to seem like the purpose of this script as it currently works diverges from your needs.

Could it be as simple as adding a --skip-build flag to skip what you don't need?