Move symbol publishing into a dedicated stage and refactor build artifacts by paulmedynski · Pull Request #4175 · dotnet/SqlClient

paulmedynski · 2026-04-09T19:49:59Z

Summary

Adds a dedicated Publish Symbols stage to the OneBranch pipelines, extracts and refactors build artifact layout, and fixes several signing/pattern issues discovered during validation.

Changes

Symbols Publishing Stage

Extract symbol publishing from build jobs into a new publish_symbols stage (publish-symbols-stage.yml) with per-package jobs (publish-symbols-job.yml).
Extract inline PowerShell into publish-symbols.ps1 script with structured error handling and Pester tests.
Document the two-step process (upload via PublishSymbols@2, then publish to SymWeb/MSDL via REST API) in step template and script.
Guard stage against empty runs when no packages were built (no-op placeholder job).
Append $(System.JobAttempt) to symbol artifact names for retry safety.
Use PPE symbol server endpoints for non-official pipeline, Prod for official.
Disable APIScan and BinSkim in symbols jobs (they only upload PDBs, no assemblies to scan).

Build Artifact Reorganization

Reorganize JOB_OUTPUT subdirectories: assemblies/ (APIScan), packages/ (NuGet), symbols/ (PDBs shared by APIScan and symbol publishing).
Point APIScan at JOB_OUTPUT/assemblies and JOB_OUTPUT/symbols; remove copy-apiscan-files-sqlclient-step.yml.
Rename PACK_OUTPUT → JOB_OUTPUT, PACK_INPUT → JOB_INPUT for clarity.
Remove artifactPath variable from validate job in favor of packagesPath.

File Pattern and Signing Fixes

Tighten searchPattern globs in symbols stage to exact PDB filenames, preventing unrelated PDBs (e.g. native SNI) from being uploaded.
Fix CopyFiles and ESRP DLL signing patterns to exclude native SNI DLLs from SqlClient symbols and signing.
Include resource DLLs (satellite assemblies) in ESRP signing and APIScan scanning.
Add UseMinimatch: true to all EsrpCodeSigning@6 and EsrpMalwareScanning@6 tasks; update patterns to minimatch globs.
Add diagnostic step that logs matched PDB files before upload.

Release Stage Updates

release stage now depends on both mds_package_validation and publish_symbols.
publish_symbols stage always emits when the flag is set.

Documentation and Cleanup

Rename variable group symbols-variables-v2 → Symbols Publishing.
Fix variable name comments and script casing across pipeline and instruction files.
Add comments documenting pattern/glob fields across pipeline files.

Follow-up

Switching symbols jobs to Linux pool: Symbols Publishing Follow-up #4214

Testing

Pester unit tests for publish-symbols.ps1 (success, transient failure, and terminal failure scenarios).
Validated via Non-Official pipeline runs on dev/paul/symbols-publishing branch.

Copilot

Pull request overview

This PR refactors the OneBranch pipeline artifact layout and moves symbol publishing out of individual build jobs into a dedicated publish_symbols stage, aligning APIScan and symbol publishing to consume standardized JOB_OUTPUT subfolders.

Changes:

Introduces a dedicated publish_symbols stage with per-package symbol publishing jobs, driven by a shared step/script.
Renames PACK_INPUT/PACK_OUTPUT to JOB_INPUT/JOB_OUTPUT and reorganizes outputs into assemblies/, packages/, and symbols/.
Updates build/release/validation templates to consume packages from packages/ and symbols from symbols/, removing the old SqlClient APIScan copy step.

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
eng/pipelines/onebranch/variables/onebranch-variables.yml	Switches symbol credentials variable group reference to `Symbols Publishing`.
eng/pipelines/onebranch/variables/common-variables.yml	Renames PACK_* vars to JOB_* and documents new JOB_OUTPUT subfolder conventions.
eng/pipelines/onebranch/steps/tests/README.md	Adds documentation for running Pester tests for symbol publishing script.
eng/pipelines/onebranch/steps/tests/Publish-Symbols.Tests.ps1	Adds Pester tests validating Publish-Symbols behavior via mocks.
eng/pipelines/onebranch/steps/Publish-Symbols.ps1	Extracts symbol publishing logic into a standalone script with structured error handling.
eng/pipelines/onebranch/steps/publish-symbols-step.yml	Refactors publish step to call the extracted script and adds `symbolsFolder` parameter.
eng/pipelines/onebranch/steps/pack-sqlclient-step.yml	Copies SqlClient NuGet outputs into `$(JOB_OUTPUT)/packages`.
eng/pipelines/onebranch/steps/pack-csproj-step.yml	Packs csproj-based packages into `$(JOB_OUTPUT)/packages`.
eng/pipelines/onebranch/steps/copy-apiscan-files-sqlclient-step.yml	Removes legacy APIScan file copy step (deleted).
eng/pipelines/onebranch/stages/release-stages.yml	Updates release stage to push packages from the `packages/` subdirectory.
eng/pipelines/onebranch/stages/publish-symbols-stage.yml	Adds new stage template to publish symbols per package from downloaded build artifacts.
eng/pipelines/onebranch/stages/build-stages.yml	Removes in-job symbol publishing parameters and job wiring (moved to stage).
eng/pipelines/onebranch/sqlclient-official.yml	Wires in the new `publish-symbols-stage.yml` between build and release stages.
eng/pipelines/onebranch/sqlclient-non-official.yml	Wires in the new `publish-symbols-stage.yml` between build and release stages.
eng/pipelines/onebranch/jobs/validate-signed-package-job.yml	Updates validation job to locate packages under `<artifact>/packages`.
eng/pipelines/onebranch/jobs/publish-symbols-job.yml	Adds per-package symbols publishing job template (download artifact + publish step).
eng/pipelines/onebranch/jobs/publish-nuget-package-job.yml	Updates release job to treat packages as living under `packages/` within artifacts.
eng/pipelines/onebranch/jobs/build-signed-sqlclient-package-job.yml	Updates APIScan folder vars and copies DLLs/PDBs into JOB_OUTPUT subfolders; removes symbol publishing.
eng/pipelines/onebranch/jobs/build-signed-csproj-package-job.yml	Updates APIScan folder vars and copies DLLs/PDBs into JOB_OUTPUT subfolders; removes symbol publishing.
.github/instructions/onebranch-pipeline-design.instructions.md	Updates OneBranch pipeline design docs for new symbols stage and artifact layout.

…facts - Extract symbol publishing from build jobs into a new publish_symbols stage (publish-symbols-stage.yml) with per-package jobs (publish-symbols-job.yml) - Reorganize JOB_OUTPUT subdirectories: assemblies/ (APIScan), packages/ (NuGet), symbols/ (PDBs shared by APIScan and symbol publishing) - Point APIScan at JOB_OUTPUT/assemblies and JOB_OUTPUT/symbols instead of separate apiScan/ tree; remove copy-apiscan-files-sqlclient-step.yml - Rename PACK_OUTPUT -> JOB_OUTPUT, PACK_INPUT -> JOB_INPUT for clarity - Extract inline PowerShell into Publish-Symbols.ps1 with structured error handling; add Pester tests - Rename variable group symbols-variables-v2 -> Symbols Publishing - Remove artifactPath variable from validate job in favor of packagesPath - Append $(System.JobAttempt) to symbol artifact names for retry safety - Guard publish_symbols stage against empty runs when no packages were built

Copilot

Pull request overview

Copilot reviewed 20 out of 20 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 20 out of 20 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 20 out of 20 changed files in this pull request and generated 1 comment.

…hProjectName

…ols jobs - Add canonical docs link and two-step process documentation to publish-symbols-step.yml and publish-symbols.ps1 - Rename task display names to Step 1/Step 2 for clarity - Disable APIScan and BinSkim in publish-symbols-job.yml since symbols jobs only upload/publish PDBs with no assemblies to scan

Copilot

Pull request overview

Copilot reviewed 20 out of 20 changed files in this pull request and generated 2 comments.

…ing.

paulmedynski · 2026-04-16T22:01:12Z

Successful run using PPE for symbols publishing is here:

https://sqlclientdrivers.visualstudio.com/ADO.Net/_build/results?buildId=148117&view=results

- Update SymbolsPublishServer/SymbolsPublishTokenUri to their actual suffixed names (Prod/PPE) in onebranch-variables.yml header comment - Fix Publish-Symbols.ps1 -> publish-symbols.ps1 casing in onebranch-pipeline-design.instructions.md

Copilot

Pull request overview

Copilot reviewed 20 out of 20 changed files in this pull request and generated 1 comment.

- Tighten searchPattern globs in publish-symbols-stage.yml to use exact PDB filenames (e.g. '**/X.pdb' instead of '**/X*.pdb'), preventing unrelated PDBs (such as native SNI PDBs) from being uploaded to the symbol server. - Fix CopyFiles pattern in build-signed-sqlclient-package-job.yml to copy only Microsoft.Data.SqlClient.pdb, excluding SNI PDBs from the symbols artifact. - Fix ESRP DLL signing pattern in build-signed-sqlclient-package-job.yml to sign only SqlClient and its resource DLLs, not native SNI DLLs (which are signed in their own package). - Include resource DLLs (satellite assemblies) in ESRP signing and APIScan scanning for both the SqlClient and generic csproj build jobs using brace expansion ({,.resources}.dll) and multi-line Contents. - Add a diagnostic step in publish-symbols-step.yml that logs all PDB files matching the search pattern before upload, converting the glob to a regex for correct cross-platform matching. - Add comments to all pattern/glob fields across pipeline files documenting what each pattern matches, what it excludes, and why resource DLLs are included or not.

…nds on both validation and symbols

Copilot

Pull request overview

Copilot reviewed 20 out of 20 changed files in this pull request and generated 2 comments.

Add UseMinimatch: true to all EsrpCodeSigning@6 and EsrpMalwareScanning@6 task inputs in esrp-dll-signing-step.yml and esrp-nuget-signing-step.yml. Update callers to use minimatch glob patterns: - DLL signing: multi-line patterns with **/ for recursive directory matching, replacing shell-style brace expansion ({,.resources}). - NuGet signing: [0-9]* anchors to the version segment to avoid matching similarly-prefixed packages (e.g. Microsoft.Data.SqlClient.SNI), and ?(s) extglob matches both .nupkg and .snupkg.

To resolve, run: git cherry-pick eacf112

…facts (#4175)

paulmedynski added this to the 7.1.0-preview1 milestone Apr 9, 2026

Copilot AI review requested due to automatic review settings April 9, 2026 19:50

github-project-automation Bot added this to SqlClient Board Apr 9, 2026

github-project-automation Bot moved this to To triage in SqlClient Board Apr 9, 2026

Copilot started reviewing on behalf of paulmedynski April 9, 2026 19:50 View session

Copilot AI reviewed Apr 9, 2026

View reviewed changes

paulmedynski force-pushed the dev/paul/symbols-publishing branch from 0dccecb to bbf6303 Compare April 10, 2026 11:18

Address PR review: fix dependsOn, align comments, fix step count

98aaeef

Copilot AI review requested due to automatic review settings April 10, 2026 11:22

paulmedynski moved this from To triage to In progress in SqlClient Board Apr 10, 2026

Copilot started reviewing on behalf of paulmedynski April 10, 2026 11:23 View session

Copilot AI reviewed Apr 10, 2026

View reviewed changes

Comment thread eng/pipelines/onebranch/scripts/publish-symbols.ps1

Comment thread eng/pipelines/onebranch/scripts/tests/README.md

Fail task on terminal symbol publishing failures, add tests

c387515

paulmedynski moved this from In progress to In review in SqlClient Board Apr 10, 2026

paulmedynski marked this pull request as ready for review April 10, 2026 12:02

paulmedynski requested a review from a team as a code owner April 10, 2026 12:02

Copilot AI review requested due to automatic review settings April 10, 2026 12:02

paulmedynski added the Hotfix Candidate 🚑 Issues/PRs that are candidate for backporting to earlier supported versions. label Apr 10, 2026

Copilot started reviewing on behalf of paulmedynski April 10, 2026 12:03 View session

Copilot AI reviewed Apr 10, 2026

View reviewed changes

Comment thread eng/pipelines/onebranch/jobs/publish-nuget-package-job.yml

paulmedynski added 2 commits April 10, 2026 10:52

Use pwsh for symbol publishing and fix non-ASCII character

8167569

Move and rename publish-symbols script to onebranch/scripts

9ceff58

Copilot AI review requested due to automatic review settings April 10, 2026 13:56

Copilot started reviewing on behalf of paulmedynski April 10, 2026 13:56 View session

Copilot AI reviewed Apr 10, 2026

View reviewed changes

Comment thread eng/pipelines/onebranch/scripts/tests/README.md

paulmedynski added 2 commits April 10, 2026 12:48

Fix status property names to match API response, rename SymbolsPublis…

75cdb27

…hProjectName

Copilot AI review requested due to automatic review settings April 10, 2026 18:26

Copilot started reviewing on behalf of paulmedynski April 10, 2026 18:27 View session

paulmedynski dismissed apoorvdeshmukh’s stale review via 65907ed April 16, 2026 15:19

Copilot started reviewing on behalf of paulmedynski April 16, 2026 15:20 View session

Copilot AI reviewed Apr 16, 2026

View reviewed changes

Comment thread eng/pipelines/onebranch/variables/onebranch-variables.yml Outdated

Comment thread .github/instructions/onebranch-pipeline-design.instructions.md Outdated

Added comments to explain the intention of PPE vs Prod symbol publish…

f04f1fc

…ing.

Copilot AI review requested due to automatic review settings April 16, 2026 22:02

paulmedynski enabled auto-merge (squash) April 16, 2026 22:02

Copilot started reviewing on behalf of paulmedynski April 16, 2026 22:02 View session

Copilot AI reviewed Apr 16, 2026

View reviewed changes

Comment thread eng/pipelines/onebranch/jobs/publish-nuget-package-job.yml

mdaigle reviewed Apr 16, 2026

View reviewed changes

Comment thread eng/pipelines/onebranch/stages/publish-symbols-stage.yml

paulmedynski added 2 commits April 17, 2026 09:11

Make publish_symbols stage always emit when flag is set; release depe…

b5c5762

…nds on both validation and symbols

Copilot AI review requested due to automatic review settings April 17, 2026 12:22

Copilot started reviewing on behalf of paulmedynski April 17, 2026 12:22 View session

Copilot AI reviewed Apr 17, 2026

View reviewed changes

Comment thread eng/pipelines/onebranch/jobs/build-signed-csproj-package-job.yml Outdated

Comment thread eng/pipelines/onebranch/jobs/build-signed-sqlclient-package-job.yml Outdated

mdaigle approved these changes Apr 17, 2026

View reviewed changes

cheenamalhotra approved these changes Apr 20, 2026

View reviewed changes

paulmedynski merged commit eacf112 into main Apr 20, 2026
7 checks passed

paulmedynski deleted the dev/paul/symbols-publishing branch April 20, 2026 05:36

github-project-automation Bot moved this from In review to Done in SqlClient Board Apr 20, 2026

github-actions Bot added a commit that referenced this pull request Apr 20, 2026

Cherry-pick of #4175 requires manual resolution

4c192ad

To resolve, run: git cherry-pick eacf112

github-actions Bot added a commit that referenced this pull request Apr 20, 2026

Cherry-pick of #4175 requires manual resolution

8961e27

To resolve, run: git cherry-pick eacf112

This was referenced Apr 20, 2026

[6.1.5 Cherry-pick - CONFLICTS] Move symbol publishing into a dedicated stage and refactor build artifacts #4210

Closed

[7.0.1 - Cherry-pick] Move symbol publishing into a dedicated stage and refactor build artifacts #4211

Closed

paulmedynski mentioned this pull request Apr 20, 2026

Symbols Publishing Follow-up #4214

Merged

paulmedynski added a commit that referenced this pull request Apr 20, 2026

Move symbol publishing into a dedicated stage and refactor build arti…

7a481d9

…facts (#4175)

paulmedynski mentioned this pull request Apr 20, 2026

Sync release/7.0 with main — cherry-pick batch #4215

Closed

paulmedynski mentioned this pull request Apr 29, 2026

[7.1.0-preview1] ESRP Sign Resources Files, too #4248

Merged

Conversation

paulmedynski commented Apr 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Symbols Publishing Stage

Build Artifact Reorganization

File Pattern and Signing Fixes

Release Stage Updates

Documentation and Cleanup

Follow-up

Testing

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

paulmedynski commented Apr 16, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

paulmedynski commented Apr 9, 2026 •

edited

Loading