ci(prcheck): use 'azldev component changed' for affected-component detection#17065
Conversation
023bba4 to
427cb41
Compare
There was a problem hiding this comment.
Pull request overview
Updates the Control Tower “sources-upload” ADO PR check to rely on structured azldev outputs (locks + changed-components JSON) instead of ad-hoc git diff | grep '/sources$', enabling scoped spec rendering and a safer upload allow-list.
Changes:
- Add
--changed-components-filesupport torun_prcheck.pyand derive the upload set fromazldev component changedJSON. - Introduce
compute_render_set.pyto union “azldev-changed” components with components whose rendered specs were edited directly. - Reorder/expand the ADO pipeline steps to: validate locks, compute changed-components, enforce a sources/identity tripwire, scoped render + rendered-tree verification, and call the prcheck API.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 9 comments.
| File | Description |
|---|---|
| .github/workflows/scripts/control-tower/run_prcheck.py | Adds --changed-components-file option and filters upload components from azldev’s changed-components JSON. |
| .github/workflows/scripts/control-tower/compute_render_set.py | New helper to compute the render set from changed-components JSON + specs-dir git diff. |
| .github/workflows/ado/templates/sources-upload-stages.yml | Pipeline now validates locks, computes changed-components JSON, enforces a sources/identity consistency check, and does scoped rendering before prcheck submission. |
| .github/instructions/ado-pipeline.instructions.md | Documents AZLDEV_ALLOW_ROOT=1 usage for CI runs. |
427cb41 to
fe7fc1c
Compare
|
ymls have gotten a bit messy, going to break them into scripts |
b463476 to
09756ae
Compare
| echo "##[endgroup]" | ||
|
|
||
| echo "##[group]Changed components (non-unchanged only)" | ||
| jq '(. // []) | [.[] | select(.changeType != "unchanged")]' "$json_file" |
There was a problem hiding this comment.
This looks like we're removing unchanged components. If that's the case, what's the point of --include-unchanged above?
There was a problem hiding this comment.
Useful later in the pipeline for other filtering, it gives us a full list of every expected component.
| json_file="$artifact_dir/changed-components.json" | ||
| mkdir -p "$artifact_dir" | ||
|
|
||
| # 'azldev component changed' compares the source and target commits and emits one |
There was a problem hiding this comment.
Will azldev component changed catch changes if the branches are in an invalid state. As in: someone forgot to update the lock files or render the specs?
There was a problem hiding this comment.
There is a bunch of validatoin logic in azldev, but right now its off by default. Once it turns on bad configurations will generate errors.
09756ae to
310132d
Compare
e511c8e to
5a8c62b
Compare
…tection
Replace the bash 'git diff | grep /sources' step with structured azldev
commands for lock validation, change detection, and scoped rendering.
Pipeline step order:
1. Lock check -- 'azldev component update -a -q -O json', fail if any
component has changed == true. Lock-update JSON published as a
pipeline artifact for triage.
2. Changed-component detection -- 'azldev component changed
--include-unchanged' writes the full per-component JSON to disk,
published via ob_outputDirectory. The --include-unchanged flag
ensures the JSON contains every known component, which is needed
for the renderable-set filter in step 4.
3. Source/identity consistency tripwire -- hard-fail if any component
reports sourcesChange == true with a changeType not in the
allow-list {added, changed, deleted}. Prevents unauthenticated
rewrites of the rendered 'sources' file under an existing
component's identity. Data path is severed (subsequent steps
skip); PR check remains advisory until ADO task 19179 removes
job-level continueOnError.
4. Scoped render -- render set is the union of components flagged by
'azldev component changed' (inputs differ) and components whose
spec tree was touched directly in the PR (git diff under specs/,
mapped back to component names by compute_render_set.py). Deleted
and unknown components are excluded via a renderable-set filter
built from the full --include-unchanged JSON.
5. Prcheck API -- switches from --components <csv> to
--changed-components-file <path>, filtering to entries with
sourcesChange == true and changeType in {added, changed}
(allow-list, mirroring the consistency tripwire).
Also:
* Add --changed-components-file flag (mutually exclusive with
--components) and _load_components_from_file() to run_prcheck.py.
Uses an allow-list of changeType values for defense-in-depth.
* Add compute_render_set.py for render-set computation.
* Document AZLDEV_ALLOW_ROOT in ADO pipeline instructions (OneBranch
containers run as root, azldev refuses by default).
* Mark changedComponentsFile pipeline variable as isreadonly=true.
* Switch API_BASE_URL to $(ApiBaseDirectUrl) (bypasses AFD).
e47001f to
9647ebc
Compare
Replace the bash
git diff … | grep '/sources$'step with structuredazldevcommands for lock validation, change detection, and scoped render verification.Pipeline step order
azldev component update --check-only. Validates without writing; exits nonzero if any lock would change. Output JSON published as a triage artifact.azldev component changed --include-unchanged. Writes the full per-component JSON to disk, published as a triage artifact. azldev hard-fails natively if any component hassourcesChange == truewithout a corresponding identity change (changeType not in{added, changed, deleted}) — supply-chain drift tripwire. Prevents unauthenticated rewrites of the renderedsourcesfile under an existing component's identity.--check-onlymode: azldev renders to a staging area and diffs against on-disk output without writing. Exits nonzero if any component's rendered output would change, catching both stale renders and direct hand-edits.--components <csv>to--changed-components-file <path>, filtering to entries withsourcesChange == trueandchangeTypein{added, changed}(allow-list).PR check remains advisory until ADO task 19179 removes job-level
continueOnError.Upload set filter
New files
verify_locks.sh— runsazldev component update --check-only, publishes the lock-update JSON for triage.compute_changed.sh— runsazldev component changed, publishes the changed-components JSON for triage. azldev's native consistency tripwire (Patch lua CVE-2019-6706, CVE-2020-15888, nopatch CVE-2020-24342 #169) is what hard-fails onsourcesChange + unchanged.render_and_verify.sh— computes the render set (azldev-flagged ∪ specs-diff) and runsazldev component render --check-onlyagainst it.compute_render_set.py— computes the render set as the union of azldev-flagged components and components with hand-edited specs. Deleted and unknown components are filtered out using the full--include-unchangedJSON.Also
--changed-components-fileflag (mutually exclusive with--components) and_load_components_from_file()torun_prcheck.py.AZLDEV_ALLOW_ROOTin ADO pipeline instructions (OneBranch containers run as root, azldev refuses by default). Set inline as a per-command prefix, not at step level.API_BASE_URLto$(ApiBaseDirectUrl)(bypasses Azure Front Door).changedComponentsFilepipeline variable asisreadonly=true.Requires
component update --check-only,component render --check-only, native consistency-check hard-fail incomponent changed, and[](notnull) for empty JSON output. Pinned viaAzldevCommitin the variable group.