Fix the pipeline upload --reject-secrets flag not rejecting secrets

[moskyb]

Description

Long, long ago (#1523, #1589), we made it so that customers could opt into rejecting pipeline uploads that contained secret values -- eg, if you accidentally committed a pipeline like:

steps:
  - command: "psql -c 'DROP TABLES *;'"
    env:
      DATABASE_PASSWORD: hunter2

the agent would recognise that there's a cleartext secret in there (DATABASE_PASSWORD) and refuse to upload it to buildkite.

However, much later on we refactored the redactor (#2277), and extracted the secrets detection into a function. That function would return an error if it thought the pipeline shouldn't be uploaded.

... and then we didn't check the error return of the function, meaning that secrets that probably shouldn't get uploaded were uploaded anyway.

This PR adds an error check to that function call, reinstating the behaviour that if a user adds --reject-secrets to their pipeline upload call, it'll refuse to upload sensitive values.

Testing

• Tests have run locally (with go test ./...). Buildkite employees may check this if the pipeline has run automatically.
• Code is formatted (with go fmt ./...)

Disclosures / Credits

I didn't use AI.

[wolfeidau]

just one query

[wolfeidau]

was this another change?

[moskyb]

yeah, although it also got merged in as part of #3579, so it should be gone when i rebase

[moskyb]

(which i have now done)

[zhming0]

LGTM but I am increasingly worried about the lack of integration test like we have in k8s stack 😅 .

[AliSoftware]

@moskyb I've just updated my buildkite-agent to 3.113.0 (and I have BUILDKITE_AGENT_PIPELINE_UPLOAD_REJECT_SECRETS=1 set) to benefit from that, and now my jobs running on those updated agents have a lot of [REDACTED] in their logs in places that are not expected to be redacted:

[2025-11-19T18:44:29Z] 2025-[REDACTED][REDACTED]-[REDACTED]9 [REDACTED]8:44:29 INFO   Reading pipeline configs from [".buildkite/pipeline.yml"]
[2025-11-19T18:44:29Z] 2025-[REDACTED][REDACTED]-[REDACTED]9 [REDACTED]8:44:29 INFO   Updating BUILDKITE_COMMIT to "2fa[REDACTED]2d2c[REDACTED]da[REDACTED][REDACTED]3534ab8aa5a89d3bf5b4f376[REDACTED]cd"
[2025-11-19T18:44:30Z] 2025-[REDACTED][REDACTED]-[REDACTED]9 [REDACTED]8:44:30 INFO   Successfully parsed and uploaded pipeline #[REDACTED] from "pipeline.yml"

I have also added some debug statements to validate the value of $BUILDKITE_REDACTED_VARS was not altered and causing this mis-redaction, but it confirmed its values was the default *_PASSWORD,*_SECRET,*_TOKEN,*_PRIVATE_KEY,*_ACCESS_KEY,*_SECRET_KEY,*_CONNECTION_STRING.

Note

In practice those buildkite agents are EC2 instances so the update I made was to update our CloudFormation stack from the version v6.12.0 we were using on that upload queue of ours to v6.48.0. Which means we made a jump from buildkite-agent version v3.61.0 to v3.113.0 on that queue.
That being said, some of our other queues are using more recent stack (even if still not the latest)—e.g. one is using CF stack v6.41.5 and thus buildkite-agent version 3.107.0)—and don't have any unexpected [REDACTED] in the logs like above.

[AliSoftware]

Ah, ended up finding the issue: that was because I since I set BUILDKITE_AGENT_PIPELINE_UPLOAD_REJECT_SECRETS=1 to enable this feature on our agent… it considered that the value 1 (value of that env var)… was in fact a secret value that needed to be redacted 🤦

I changed it to export BUILDKITE_AGENT_PIPELINE_UPLOAD_REJECT_SECRETS=true instead, and the extraneous redaction of all the 1s in my logs have stopped 🎉

That means though that now every mention of the word true in my logs are redacted… which is still not great… but at least it's not as frequent as having the character 1 redacted in all timestamps and lines 😅

I'm still surprised to see that the log redactor would consider the env var name BUILDKITE_AGENT_PIPELINE_UPLOAD_REJECT_SECRETS one that matches the BUILDKITE_REDACTED_VARS patterns (*_PASSWORD,*_SECRET,*_TOKEN,*_PRIVATE_KEY,*_ACCESS_KEY,*_SECRET_KEY,*_CONNECTION_STRING), given the env var name ends in _SECRETS not _SECRET. Also, I thought the log redactor was not supposed to redact strings less than 6 bytes… 🤔
I've just filed an issue separately about that.