diff --git a/.github/instructions/onebranch-pipeline-design.instructions.md b/.github/instructions/onebranch-pipeline-design.instructions.md index f2319ee5b2..0860d5db52 100644 --- a/.github/instructions/onebranch-pipeline-design.instructions.md +++ b/.github/instructions/onebranch-pipeline-design.instructions.md @@ -36,6 +36,8 @@ Defined in `stages/build-stages.yml`. Four build stages plus validation, ordered - **`build_addons`** (Stage 4) — AKV Provider; `dependsOn: build_dependent`; downloads SqlClient + Abstractions + Logging artifacts - **`mds_package_validation`** — Validates signed SqlClient package; `dependsOn: build_dependent`; runs in parallel with Stage 4 +Each build job copies PDB files into `$(JOB_OUTPUT)/symbols/` so they are included in the auto-published pipeline artifact alongside the NuGet packages in `$(JOB_OUTPUT)/packages/`. + Stage conditional rules: - Wrap stages/jobs in `${{ if }}` compile-time conditionals based on build parameters - `buildSqlClient` controls Stages 2, 3, validation, and Logging (when AKV is disabled) @@ -45,10 +47,11 @@ Stage conditional rules: ## Job Templates -- **`build-signed-csproj-package-job.yml`** — Generic job for csproj-based packages (Logging, SqlServer.Server, Abstractions, Azure, AKV Provider). Flow: Build DLLs → ESRP DLL signing → NuGet pack (`NoBuild=true`) → ESRP NuGet signing -- **`build-signed-sqlclient-package-job.yml`** — SqlClient-specific job (nuspec-based). Flow: Build all configurations → ESRP DLL signing (main + resource DLLs) → NuGet pack via nuspec → ESRP NuGet signing +- **`build-signed-csproj-package-job.yml`** — Generic job for csproj-based packages (Logging, SqlServer.Server, Abstractions, Azure, AKV Provider). Flow: Build DLLs → ESRP DLL signing → NuGet pack (`NoBuild=true`) → ESRP NuGet signing → Copy PDBs to artifact +- **`build-signed-sqlclient-package-job.yml`** — SqlClient-specific job (nuspec-based). Flow: Build all configurations → ESRP DLL signing (main + resource DLLs) → NuGet pack via nuspec → ESRP NuGet signing → Copy PDBs to artifact - **`validate-signed-package-job.yml`** — Validates signed MDS package (signature, strong names, folder structure, target frameworks) - **`publish-nuget-package-job.yml`** — Reusable release job using OneBranch `templateContext.type: releaseJob` with `inputs` for artifact download; pushes via `NuGetCommand@2` +- **`publish-symbols-job.yml`** — Reusable symbols job: downloads a build artifact, locates PDBs under `symbols/`, and invokes `publish-symbols-step.yml` When adding a new csproj-based package: - Use `build-signed-csproj-package-job.yml` with appropriate `packageName`, `packageFullName`, `versionProperties`, and `downloadArtifacts` @@ -56,6 +59,18 @@ When adding a new csproj-based package: - Add version variables to `variables/common-variables.yml` - Add artifact name variable to `variables/onebranch-variables.yml` +## Symbols Publishing Stage + +- Defined in `stages/publish-symbols-stage.yml`; produces stage `publish_symbols` +- Entire stage excluded at compile time when `publishSymbols` is false +- `dependsOn` is conditional based on which `build*` parameters are set, mirroring the build stage dependency graph +- One job per package (`publish-symbols-job.yml`), each downloading its build artifact and publishing PDBs from `symbols/` +- Each package's PDBs are published separately with unique artifact names and version information +- Build jobs copy PDBs into `$(JOB_OUTPUT)/symbols/` so they are included in the auto-published artifact +- The `publish-symbols-step.yml` accepts a `symbolsFolder` parameter to point at the downloaded PDB location +- The publish step calls an extracted `publish-symbols.ps1` script with structured error handling and diagnostic logging +- Symbols publishing credentials come from the `Symbols Publishing` variable group + ## Release Stage - Defined in `stages/release-stages.yml`; produces stage `release_production` (official) or `release_test` (non-official) via `stageNameSuffix` parameter @@ -98,8 +113,7 @@ When `isPreview` is true, pipeline resolves `effective*Version` variables to pre - When adding a new package, add GA version, preview version, and assembly file version entries Variable groups: -- `Release Variables` — release configuration (in `common-variables.yml`) -- `Symbols publishing` — symbol publishing credentials (in `common-variables.yml`) +- `Symbols Publishing` — symbol publishing credentials (in `onebranch-variables.yml`) - `ESRP Federated Creds (AME)` — ESRP signing credentials (in `common-variables.yml`) ## Code Signing (ESRP) @@ -115,7 +129,7 @@ Variable groups: - TSA: enabled only in official pipeline; disabled in non-official to avoid spurious alerts - ApiScan: enabled in both; currently `break: false` pending package registration -- Each build job sets `ob_sdl_apiscan_*` variables pointing to `$(Build.SourcesDirectory)/apiScan//` +- Each build job sets `ob_sdl_apiscan_softwareFolder` to `$(JOB_OUTPUT)/assemblies` and `ob_sdl_apiscan_symbolsFolder` to `$(JOB_OUTPUT)/symbols` - CodeQL, SBOM, Policheck (`break: true`): enabled in both pipelines - asyncSdl `enabled: false` in both; individual sub-tools (CredScan, BinSkim, Armory, Roslyn) configured underneath - Policheck exclusions: `$(REPO_ROOT)\.config\PolicheckExclusions.xml` @@ -123,7 +137,11 @@ Variable groups: ## Artifact Conventions -- `ob_outputDirectory` set to `$(PACK_OUTPUT)` (= `$(REPO_ROOT)/output`) — OneBranch auto-publishes this directory +- `ob_outputDirectory` set to `$(JOB_OUTPUT)` (= `$(REPO_ROOT)/output`) — OneBranch auto-publishes this directory +- Each published artifact uses subdirectories to separate file types: + - `assemblies/` — DLL assemblies for APIScan (preserving TFM folder structure) + - `packages/` — NuGet packages (`.nupkg`, `.snupkg`) + - `symbols/` — PDB symbol files (preserving TFM folder structure, shared by APIScan and symbol publishing) - Artifact names follow `drop__` — defined in `variables/onebranch-variables.yml` - Downstream jobs download artifacts via `DownloadPipelineArtifact@2` into `$(Build.SourcesDirectory)/packages` - Downloaded packages serve as a local NuGet source for `dotnet restore` diff --git a/eng/pipelines/onebranch/jobs/build-signed-csproj-package-job.yml b/eng/pipelines/onebranch/jobs/build-signed-csproj-package-job.yml index 6dc72983cc..94e6438ef5 100644 --- a/eng/pipelines/onebranch/jobs/build-signed-csproj-package-job.yml +++ b/eng/pipelines/onebranch/jobs/build-signed-csproj-package-job.yml @@ -36,26 +36,6 @@ parameters: - name: signingEsrpConnectedServiceName type: string - # Symbols Publishing Parameters ------------------------------------------ - - - name: symbolsAzureSubscription - type: string - default: 'Symbols publishing Workload Identity federation service-ADO.Net' - - - name: symbolsPublishProjectName - type: string - default: 'Microsoft.Data.SqlClient.SNI' - - - name: symbolsPublishServer - type: string - - - name: symbolsPublishTokenUri - type: string - - - name: symbolsUploadAccount - type: string - default: 'SqlClientDrivers' - # OTHERS +===================================== # Short package name used in the job name, display strings, filesystem paths, and as a suffix for @@ -74,11 +54,6 @@ parameters: - name: packageFullName type: string - # The version of the package. This is used for symbol publishing. It is not used for the DLL or - # NuGet package versions since those are supplied via the versionProperties parameter. - - name: packageVersion - type: string - # The MSBuild build target in build.proj (e.g. BuildLogging). If not specified, defaults to # Build. - name: buildTarget @@ -106,10 +81,6 @@ parameters: - name: assemblyFileVersion type: string - # True to publish symbols to private and public servers. - - name: publishSymbols - type: boolean - # Optional list of pipeline artifacts to download before building. Each entry is an object # with 'artifactName' (the pipeline artifact name) and 'displayName' (used in the task label). # This replaces hard-coded packageName conditionals so callers declare their own dependencies. @@ -125,12 +96,12 @@ jobs: variables: # Inform OneBranch that files put in this directory should be uploaded as artifacts. - ob_outputDirectory: $(PACK_OUTPUT) + ob_outputDirectory: $(JOB_OUTPUT) # APIScan configuration for this Extension package ob_sdl_apiscan_enabled: true - ob_sdl_apiscan_softwareFolder: $(REPO_ROOT)/apiScan/${{ parameters.packageName }}/dlls - ob_sdl_apiscan_symbolsFolder: $(REPO_ROOT)/apiScan/${{ parameters.packageName }}/pdbs + ob_sdl_apiscan_softwareFolder: $(JOB_OUTPUT)/assemblies + ob_sdl_apiscan_symbolsFolder: $(JOB_OUTPUT)/symbols ob_sdl_apiscan_softwarename: ${{ parameters.packageFullName }} ob_sdl_apiscan_versionNumber: ${{ parameters.assemblyFileVersion }} @@ -149,7 +120,7 @@ jobs: displayName: Download ${{ artifact.displayName }} inputs: artifactName: ${{ artifact.artifactName }} - targetPath: $(REPO_ROOT)/packages + targetPath: $(JOB_INPUT) # Install the .NET SDK. - template: /eng/pipelines/steps/install-dotnet.yml@self @@ -179,26 +150,42 @@ jobs: authSignCertName: '${{ parameters.signingAuthSignCertName }}' esrpClientId: '${{ parameters.signingEsrpClientId }}' esrpConnectedServiceName: '${{ parameters.signingEsrpConnectedServiceName }}' - pattern: '${{ parameters.packageFullName }}.dll' - - # Copy signed/unsigned DLLs and PDBs to APIScan folders. + # Minimatch pattern (multi-line, one pattern per line). + # **/ matches any nested directory (e.g. net8.0/, netstandard2.0/fr/). + # + # Matches: net8.0/.dll + # netstandard2.0/fr/.resources.dll + pattern: | + **/${{ parameters.packageFullName }}.dll + **/${{ parameters.packageFullName }}.resources.dll + + # Copy DLLs to the assemblies/ subdirectory for APIScan. - task: CopyFiles@2 displayName: Copy DLLs for APIScan inputs: SourceFolder: $(BUILD_OUTPUT)/Package/bin - Contents: "**/${{ parameters.packageFullName }}.dll" - TargetFolder: ${{ variables.ob_sdl_apiscan_softwareFolder }} + # Matches: .dll (main assembly) across all TFM subdirs + # .resources.dll (localized satellite assemblies in locale subdirs, if any) + Contents: | + **/${{ parameters.packageFullName }}.dll + **/${{ parameters.packageFullName }}.resources.dll + TargetFolder: $(JOB_OUTPUT)/assemblies # We must preserve the folder structure since our C# projects may produce multiple # identically named DLLs for different target frameworks (e.g. netstandard2.0, net5.0, # etc.), and we need to keep those separate for APIScan to work correctly. flattenFolders: false + # Copy PDBs into the output directory so they are included in the published pipeline + # artifact. The symbols publishing stage will download this artifact and publish PDBs + # for this package using the files under symbols/. - task: CopyFiles@2 - displayName: Copy PDBs for APIScan + displayName: Copy PDBs for APIScan and Symbols Publishing inputs: SourceFolder: $(BUILD_OUTPUT)/Package/bin - Contents: "**/${{ parameters.packageFullName }}.pdb" - TargetFolder: ${{ variables.ob_sdl_apiscan_symbolsFolder }} + # Matches: .pdb across all TFM subdirs. + # Note: Resource DLLs are resource-only satellite assemblies and do not produce PDBs. + Contents: '**/${{ parameters.packageFullName }}.pdb' + TargetFolder: $(JOB_OUTPUT)/symbols flattenFolders: false # Pack the signed DLLs into NuGet package (NoBuild=true). @@ -217,22 +204,13 @@ jobs: authSignCertName: '${{ parameters.signingAuthSignCertName }}' esrpClientId: '${{ parameters.signingEsrpClientId }}' esrpConnectedServiceName: '${{ parameters.signingEsrpConnectedServiceName }}' - searchPath: $(PACK_OUTPUT) - searchPattern: '${{ parameters.packageFullName}}.*nupkg' - - # Publish symbols to servers - # @TODO: Get these parameters from variables/libraries - - ${{ if eq(parameters.publishSymbols, true) }}: - - template: /eng/pipelines/onebranch/steps/publish-symbols-step.yml@self - parameters: - artifactName: '${{ parameters.packageFullName }}_symbols_$(System.TeamProject)_$(Build.Repository.Name)_$(Build.SourceBranchName)_${{ parameters.packageVersion }}_$(System.TimelineId)' - azureSubscription: '${{ parameters.symbolsAzureSubscription }}' - packageName: '${{ parameters.packageFullName }}' - publishProjectName: '${{ parameters.symbolsPublishProjectName }}' - publishServer: '${{ parameters.symbolsPublishServer }}' - publishToInternal: 'true' - publishToPublic: 'true' - publishTokenUri: '${{ parameters.symbolsPublishTokenUri }}' - searchPattern: '**/${{ parameters.packageFullName }}*.pdb' - uploadAccount: '${{ parameters.symbolsUploadAccount }}' - version: '${{ parameters.packageVersion }}' + searchPath: $(JOB_OUTPUT)/packages + # Minimatch pattern with extglob. + # [0-9] matches a single digit, anchoring to the version segment so + # similarly-prefixed package names are not matched. + # ?(s) is an extglob that optionally matches 's'. + # + # Matches: .1.0.0.nupkg + # .1.0.0-preview.1.snupkg + # Excludes: .SomeOther.1.0.0.nupkg + searchPattern: '${{ parameters.packageFullName }}.[0-9]*.?(s)nupkg' diff --git a/eng/pipelines/onebranch/jobs/build-signed-sqlclient-package-job.yml b/eng/pipelines/onebranch/jobs/build-signed-sqlclient-package-job.yml index 752e459b84..5bf33f9c09 100644 --- a/eng/pipelines/onebranch/jobs/build-signed-sqlclient-package-job.yml +++ b/eng/pipelines/onebranch/jobs/build-signed-sqlclient-package-job.yml @@ -7,20 +7,11 @@ # Job that performs a build of Microsoft.Data.SqlClient using the build2.proj file. parameters: - - name: apiScanDllPath - type: string - - - name: apiScanPdbPath - type: string - # Whether this build is for an "official" OneBranch pipeline. This is used to enable ESRP signing # on the artifacts of this job. - name: isOfficial type: boolean - - name: publishSymbols - type: boolean - - name: signingAppRegistrationClientId type: string @@ -39,21 +30,6 @@ parameters: - name: signingEsrpConnectedServiceName type: string - - name: symbolsAzureSubscription - type: string - - - name: symbolsPublishProjectName - type: string - - - name: symbolsPublishServer - type: string - - - name: symbolsPublishTokenUri - type: string - - - name: symbolsUploadAccount - type: string - # Package Parameters - name: abstractionsArtifactName type: string @@ -80,9 +56,9 @@ jobs: type: windows variables: - ob_outputDirectory: '$(PACK_OUTPUT)' - ob_sdl_apiscan_softwareFolder: ${{ parameters.apiScanDllPath }} - ob_sdl_apiscan_symbolsFolder: ${{ parameters.apiScanPdbPath }} + ob_outputDirectory: '$(JOB_OUTPUT)' + ob_sdl_apiscan_softwareFolder: $(JOB_OUTPUT)/assemblies + ob_sdl_apiscan_symbolsFolder: $(JOB_OUTPUT)/symbols ob_sdl_apiscan_softwarename: 'Microsoft.Data.SqlClient' ob_sdl_apiscan_versionNumber: ${{ parameters.sqlClientAssemblyFileVersion }} @@ -102,13 +78,13 @@ jobs: displayName: Download Microsoft.Data.SqlClient.Extensions.Abstractions Artifact inputs: artifactName: '${{ parameters.abstractionsArtifactName }}' - targetPath: '$(REPO_ROOT)/packages' + targetPath: '$(JOB_INPUT)' - task: DownloadPipelineArtifact@2 displayName: Download Microsoft.Data.SqlClient.Extensions.Logging Artifact inputs: artifactName: '${{ parameters.loggingArtifactName }}' - targetPath: '$(REPO_ROOT)/packages' + targetPath: '$(JOB_INPUT)' # Install the .NET SDK - template: /eng/pipelines/steps/install-dotnet.yml@self @@ -127,13 +103,6 @@ jobs: loggingPackageVersion: '${{ parameters.loggingPackageVersion }}' sqlClientPackageVersion: '${{ parameters.sqlClientPackageVersion }}' - # Copy the built DLLs and PDBs to the APIScan output folder for APIScanning post-build - - template: /eng/pipelines/onebranch/steps/copy-apiscan-files-sqlclient-step.yml@self - parameters: - dllPath: '${{ parameters.apiScanDllPath }}' - pdbPath: '${{ parameters.apiScanPdbPath }}' - referenceType: 'Package' - # Sign the DLLs - ${{ if parameters.isOfficial }}: - template: /eng/pipelines/onebranch/steps/esrp-dll-signing-step.yml@self @@ -144,7 +113,43 @@ jobs: authSignCertName: '${{ parameters.signingAuthSignCertName }}' esrpClientId: '${{ parameters.signingEsrpClientId }}' esrpConnectedServiceName: '${{ parameters.signingEsrpConnectedServiceName }}' - pattern: 'Microsoft.Data.SqlClient*.dll' + # Minimatch pattern (multi-line, one pattern per line). + # **/ matches any nested directory (e.g. net8.0/, net462/fr/). + # + # Matches: net8.0/Microsoft.Data.SqlClient.dll + # net462/fr/Microsoft.Data.SqlClient.resources.dll + # Excludes: net8.0/Microsoft.Data.SqlClient.SNI.dll (no ** path matches) + pattern: | + **/Microsoft.Data.SqlClient.dll + **/Microsoft.Data.SqlClient.resources.dll + + # Copy DLLs to the assemblies/ subdirectory for APIScan. + - task: CopyFiles@2 + displayName: Copy DLLs for APIScan + inputs: + SourceFolder: '$(BUILD_OUTPUT)/Microsoft.Data.SqlClient/Package-Release' + # Matches: Microsoft.Data.SqlClient.dll (main assembly) + # Microsoft.Data.SqlClient.resources.dll (localized satellite assemblies in locale subdirs) + # Excludes: Microsoft.Data.SqlClient.SNI*.dll (native SNI — not part of this package) + Contents: | + **/Microsoft.Data.SqlClient.dll + **/Microsoft.Data.SqlClient.resources.dll + TargetFolder: '$(JOB_OUTPUT)/assemblies' + flattenFolders: false + + # Copy PDBs into the output directory so they are included in the published pipeline + # artifact. The symbols publishing stage will download this artifact and publish PDBs + # for this package using the files under symbols/. + - task: CopyFiles@2 + displayName: Copy PDBs for APIScan and Symbols Publishing + inputs: + SourceFolder: '$(BUILD_OUTPUT)/Microsoft.Data.SqlClient/Package-Release' + # Matches: Microsoft.Data.SqlClient.pdb (main assembly symbols) + # Excludes: Microsoft.Data.SqlClient.SNI*.pdb (native SNI — symbols published separately) + # Note: Resource DLLs are resource-only satellite assemblies and do not produce PDBs. + Contents: '**/Microsoft.Data.SqlClient.pdb' + TargetFolder: '$(JOB_OUTPUT)/symbols' + flattenFolders: false # Package the build output into a NuGet package - template: /eng/pipelines/onebranch/steps/pack-sqlclient-step.yml @@ -163,20 +168,13 @@ jobs: authSignCertName: '${{ parameters.signingAuthSignCertName }}' esrpClientId: '${{ parameters.signingEsrpClientId }}' esrpConnectedServiceName: '${{ parameters.signingEsrpConnectedServiceName }}' - searchPath: '$(PACK_OUTPUT)' - searchPattern: 'Microsoft.Data.SqlClient.*nupkg' - - - ${{ if parameters.publishSymbols }}: - - template: /eng/pipelines/onebranch/steps/publish-symbols-step.yml@self - parameters: - artifactName: 'Microsoft.Data.SqlClient_symbols_$(System.TeamProject)_$(Build.Repository.Name)_$(Build.SourceBranchName)_${{ parameters.sqlClientPackageVersion }}_$(System.TimelineId)' - azureSubscription: '${{ parameters.symbolsAzureSubscription }}' - packageName: 'Microsoft.Data.SqlClient' - publishProjectName: '${{ parameters.symbolsPublishProjectName }}' - publishServer: '${{ parameters.symbolsPublishServer }}' - publishToInternal: true - publishToPublic: true - publishTokenUri: '${{ parameters.symbolsPublishTokenUri }}' - searchPattern: '**/Microsoft.Data.SqlClient*.pdb' # @TODO: This seems very heavy - uploadAccount: '${{ parameters.symbolsUploadAccount }}' - version: '${{ parameters.sqlClientPackageVersion }}' + searchPath: '$(JOB_OUTPUT)/packages' + # Minimatch pattern with extglob. + # [0-9] matches a single digit, anchoring to the version segment so + # names like Microsoft.Data.SqlClient.SNI are not matched. + # ?(s) is an extglob that optionally matches 's'. + # + # Matches: Microsoft.Data.SqlClient.6.1.0.nupkg + # Microsoft.Data.SqlClient.6.1.0.snupkg + # Excludes: Microsoft.Data.SqlClient.SNI.6.0.0.nupkg + searchPattern: 'Microsoft.Data.SqlClient.[0-9]*.?(s)nupkg' diff --git a/eng/pipelines/onebranch/jobs/publish-nuget-package-job.yml b/eng/pipelines/onebranch/jobs/publish-nuget-package-job.yml index b7a1d8f62e..21d36b9640 100644 --- a/eng/pipelines/onebranch/jobs/publish-nuget-package-job.yml +++ b/eng/pipelines/onebranch/jobs/publish-nuget-package-job.yml @@ -38,15 +38,18 @@ parameters: # artifact. For example, if we're publishing the SqlClient package, and the build job publishes a # pipeline artifact with the following structure: # - # drop_BuildAndTest_PackageReference/ - # ├── Microsoft.Data.SqlClient.5.0.0.nupkg - # ├── Microsoft.Data.SqlClient.5.0.0.snupkg - # ├── Microsoft.Data.SqlClient.Extensions.Abstractions.1.0.0.nupkg - # └── other-file.txt + # drop_build_independent_build_logging/ + # ├── packages/ + # │ ├── Microsoft.Data.SqlClient.5.0.0.nupkg + # │ └── Microsoft.Data.SqlClient.5.0.0.snupkg + # └── symbols/ + # └── ... # - # Then the packagePath should be 'Microsoft.Data.SqlClient.5.0.0.nupkg'. + # Then the packagePath should be 'packages/Microsoft.Data.SqlClient.5.0.0.nupkg'. # - # Defaults to '${{ parameters.packageName }}.*.nupkg' to match any version of the package. + # Defaults to 'packages/${{ parameters.packageName }}.*.nupkg' to match any version of the + # package. Only .nupkg files are pushed; .snupkg files are included automatically by NuGet + # when they exist alongside the .nupkg in the same directory. # - name: packagePath type: string @@ -65,13 +68,13 @@ jobs: variables: - name: ob_outputDirectory - value: $(PACK_OUTPUT) + value: $(JOB_OUTPUT) - name: artifactPath value: $(Pipeline.Workspace)/${{ parameters.artifactName }} - name: packageToPush - value: $(artifactPath)/${{ coalesce(parameters.packagePath, format('{0}.*.nupkg', parameters.packageName)) }} + value: $(artifactPath)/${{ coalesce(parameters.packagePath, format('packages/{0}.*.nupkg', parameters.packageName)) }} # Template context inputs are used to pass parameters to the deployment job since it doesn't # automatically download pipeline artifacts. diff --git a/eng/pipelines/onebranch/jobs/publish-symbols-job.yml b/eng/pipelines/onebranch/jobs/publish-symbols-job.yml new file mode 100644 index 0000000000..21b20476cd --- /dev/null +++ b/eng/pipelines/onebranch/jobs/publish-symbols-job.yml @@ -0,0 +1,95 @@ +################################################################################# +# Licensed to the .NET Foundation under one or more agreements. # +# The .NET Foundation licenses this file to you under the MIT license. # +# See the LICENSE file in the project root for more information. # +################################################################################# + +# Reusable job template for publishing symbols for a single package. Downloads the pipeline +# artifact produced by a build job, then invokes publish-symbols-step.yml to upload and publish +# the PDB files to the internal and public symbol servers. +# +# Each package's PDBs are published separately to maintain unique naming and versioning. +# PDBs are expected to be located under the 'symbols/' directory within the downloaded +# artifact. + +parameters: + # The pipeline artifact name to download (OneBranch naming: drop__). + - name: artifactName + type: string + + # Short package name used in the job name (e.g. Logging, SqlServer, SqlClient). + - name: packageName + type: string + + # The full NuGet package name (e.g. Microsoft.Data.SqlClient.Internal.Logging). + - name: packageFullName + type: string + + # The package version, used for symbol versioning. + - name: packageVersion + type: string + + # Pattern to use to search for PDB files to upload/publish, relative to the symbols folder. + - name: searchPattern + type: string + + # Symbols Publishing Parameters ------------------------------------------ + + - name: symbolsAzureSubscription + type: string + + - name: symbolsPublishProjectName + type: string + + - name: symbolsPublishServer + type: string + + - name: symbolsPublishTokenUri + type: string + + - name: symbolsUploadAccount + type: string + +jobs: + - job: publish_symbols_${{ parameters.packageName }} + displayName: 'Publish Symbols: ${{ parameters.packageFullName }}' + pool: + type: windows + + variables: + # OneBranch requires ob_outputDirectory to be set, even if this job produces no artifacts. + ob_outputDirectory: $(JOB_OUTPUT) + + # Disable SDL scanning — this job only uploads/publishes PDBs and produces no + # assemblies to scan. APIScan and BinSkim are handled by the build jobs. + ob_sdl_apiscan_enabled: false + ob_sdl_binskim_break: false + ob_sdl_binskim_enabled: false + + # Path where the downloaded artifact will be placed. + artifactPath: $(Pipeline.Workspace)/${{ parameters.artifactName }} + + # Path to the PDB files within the downloaded artifact. + symbolsPath: $(Pipeline.Workspace)/${{ parameters.artifactName }}/symbols + + steps: + - task: DownloadPipelineArtifact@2 + displayName: 'Download ${{ parameters.packageFullName }} Artifact' + inputs: + artifactName: '${{ parameters.artifactName }}' + targetPath: '$(artifactPath)' + + - template: /eng/pipelines/onebranch/steps/publish-symbols-step.yml@self + parameters: + artifactName: '${{ parameters.packageFullName }}_symbols_$(System.TeamProject)_$(Build.Repository.Name)_$(Build.SourceBranchName)_${{ parameters.packageVersion }}_$(System.TimelineId)_$(System.JobAttempt)' + azureSubscription: '${{ parameters.symbolsAzureSubscription }}' + packageName: '${{ parameters.packageFullName }}' + publishProjectName: '${{ parameters.symbolsPublishProjectName }}' + publishServer: '${{ parameters.symbolsPublishServer }}' + publishToInternal: true + publishToPublic: true + publishTokenUri: '${{ parameters.symbolsPublishTokenUri }}' + searchPattern: '${{ parameters.searchPattern }}' + symbolsFolder: '$(symbolsPath)' + uploadAccount: '${{ parameters.symbolsUploadAccount }}' + version: '${{ parameters.packageVersion }}' diff --git a/eng/pipelines/onebranch/jobs/validate-signed-package-job.yml b/eng/pipelines/onebranch/jobs/validate-signed-package-job.yml index 27369eb6cc..a99b46d45f 100644 --- a/eng/pipelines/onebranch/jobs/validate-signed-package-job.yml +++ b/eng/pipelines/onebranch/jobs/validate-signed-package-job.yml @@ -48,8 +48,9 @@ jobs: variables: # More settings at https://aka.ms/obpipelines/yaml/jobs - - name: artifactPath - value: '$(Pipeline.Workspace)\${{ parameters.artifactName }}' + # Path within the downloaded artifact where NuGet packages are located. + - name: packagesPath + value: '$(Pipeline.Workspace)\${{ parameters.artifactName }}\packages' # Path to the SqlClient NuGet package after installation. This path will only exist once the package # been installed. @@ -85,22 +86,22 @@ jobs: - ${{ if eq(parameters.isOfficial, true) }}: - powershell: | # Propagate parameters to PS variables ####################### - $artifactPath = "${{ variables.artifactPath }}" - echo "artifactPath= $artifactPath" + $packagesPath = "${{ variables.packagesPath }}" + echo "packagesPath= $packagesPath" # Verify package signatures ################################## echo "> 1. Verify signature of source package(s)" - nuget verify -All $artifactPath\*.nupkg + nuget verify -All $packagesPath\*.nupkg echo "> 2. Verify signature of symbols package(s)" - nuget verify -All $artifactPath\*.snupkg + nuget verify -All $packagesPath\*.snupkg displayName: "Verify nuget signature" # Install NuGet package to the temporary directory - powershell: | # Propagate pipeline to PS variables ############################# - $artifactPath = "${{ variables.artifactPath }}" - echo "artifactPath= $artifactPath" + $packagesPath = "${{ variables.packagesPath }}" + echo "packagesPath= $packagesPath" $nugetPackageInstallRoot = "${{ variables.nugetPackageInstallRoot }}" echo "nugetPackageInstallRoot= $nugetPackageInstallRoot" @@ -109,7 +110,7 @@ jobs: echo "> 1. Installing Microsoft.Data.SqlClient NuGet package..." Install-Package ` -Name "Microsoft.Data.SqlClient" ` - -Source $artifactPath ` + -Source "$packagesPath" ` -Destination $nugetPackageInstallRoot ` -Force ` -SkipDependencies @@ -117,6 +118,8 @@ jobs: echo "> 2. Listing contents of installed Microsoft.Data.SqlClient NuGet package:" Write-Host $nugetPackageInstallRoot Get-ChildItem $nugetPackageInstallRoot + Write-Host "$packagesPath" + Get-ChildItem "$packagesPath" displayName: "Install NuGet Package" # Find all DLL files in the installed NuGet package, verify each is signed with a strong name diff --git a/eng/pipelines/onebranch/scripts/publish-symbols.ps1 b/eng/pipelines/onebranch/scripts/publish-symbols.ps1 new file mode 100644 index 0000000000..aeaa665a92 --- /dev/null +++ b/eng/pipelines/onebranch/scripts/publish-symbols.ps1 @@ -0,0 +1,236 @@ +<# +.SYNOPSIS + Publishes symbols to the Microsoft symbol publishing service (SymWeb/MSDL). + +.DESCRIPTION + This script is Step 2 of the two-step symbols publishing process. It requests + the Symbols Publishing Pipeline to publish previously uploaded PDB files to + internal (SymWeb) and/or public (MSDL) Microsoft symbol servers. + + The two-step process: + Step 1 (PublishSymbols@2 task in publish-symbols-step.yml): + Uploads PDB files to the Azure DevOps symbol store under a unique + artifact name (SymbolsArtifactName). This stores the symbols but does + NOT make them available on SymWeb or MSDL. + + Step 2 (this script): + Calls the Symbols Publishing Pipeline REST API to request that the + uploaded symbols be published to the symbol servers. + + Step 2 depends on Step 1: the -ArtifactName parameter MUST match the + SymbolsArtifactName used by the PublishSymbols@2 upload task so that + both steps reference the same uploaded artifact. + + This script performs four sub-steps: + 1. Acquires a bearer token from Azure CLI for the symbol publishing service. + 2. Registers a unique request name with the publishing service. + 3. Submits the request to publish symbols to the specified servers. + 4. Queries the publishing status for confirmation. + + For more details on the Symbols Publishing Pipeline, see: + https://www.osgwiki.com/wiki/Symbols_Publishing_Pipeline_to_SymWeb_and_MSDL + +.PARAMETER PublishServer + The hostname prefix of the symbol publishing service. This value is prepended to + '.trafficmanager.net' to construct the service base URL. + +.PARAMETER PublishTokenUri + The resource URI used to acquire a bearer token from Azure CLI + (via 'az account get-access-token --resource '). + +.PARAMETER PublishProjectName + The project name registered with the symbol publishing service (decided during onboarding). + +.PARAMETER ArtifactName + The name of the publishing request. This must match the SymbolsArtifactName used by + the PublishSymbols@2 upload task so that upload and publish reference the same artifact. + +.PARAMETER PublishToInternal + Whether to publish symbols to the internal symbol server. Defaults to $true. + +.PARAMETER PublishToPublic + Whether to publish symbols to the public symbol server. Defaults to $true. + +.EXAMPLE + .\publish-symbols.ps1 ` + -PublishServer "mysymbolserver" ` + -PublishTokenUri "https://login.microsoftonline.com/..." ` + -PublishProjectName "Microsoft.Data.SqlClient.SNI" ` + -ArtifactName "mds_symbols_MyProject_dotnet-sqlclient_main_7.0.0_abc123_1" + + Publishes symbols to both internal and public servers using the specified parameters. + +.EXAMPLE + .\publish-symbols.ps1 ` + -PublishServer "mysymbolserver" ` + -PublishTokenUri "https://login.microsoftonline.com/..." ` + -PublishProjectName "Microsoft.Data.SqlClient.SNI" ` + -ArtifactName "mds_symbols_MyProject_dotnet-sqlclient_main_7.0.0_abc123_2" ` + -PublishToPublic $false + + Publishes symbols to the internal server only (retry attempt 2). + +.NOTES + File Name : publish-symbols.ps1 + Requires : Azure CLI (az) must be installed and authenticated. + Called by : publish-symbols-step.yml (Azure Pipelines template) + + Publishing status codes returned by the service: + + PublishingStatus: + 0 - NotRequested: The request has not been requested to publish. + 1 - Submitted: The request is submitted to be published. + 2 - Processing: The request is still being processed. + 3 - Completed: Processing finished. Check PublishingResult for details. + + PublishingResult: + 0 - Pending: The request has not completed or has not been requested. + 1 - Succeeded: The request published successfully. + 2 - Failed: The request failed to publish. + 3 - Cancelled: The request was cancelled. +#> + +[CmdletBinding()] +param( + [Parameter(Mandatory = $true, HelpMessage = "Hostname prefix of the symbol publishing service (prepended to .trafficmanager.net).")] + [ValidateNotNullOrEmpty()] + [string]$PublishServer, + + [Parameter(Mandatory = $true, HelpMessage = "Resource URI for acquiring a bearer token via Azure CLI.")] + [ValidateNotNullOrEmpty()] + [string]$PublishTokenUri, + + [Parameter(Mandatory = $true, HelpMessage = "Project name registered with the symbol publishing service.")] + [ValidateNotNullOrEmpty()] + [string]$PublishProjectName, + + [Parameter(Mandatory = $true, HelpMessage = "Artifact name for the publishing request (must match PublishSymbols@2 SymbolsArtifactName).")] + [ValidateNotNullOrEmpty()] + [string]$ArtifactName, + + [Parameter(Mandatory = $false, HelpMessage = "Publish symbols to the internal symbol server.")] + [bool]$PublishToInternal = $true, + + [Parameter(Mandatory = $false, HelpMessage = "Publish symbols to the public symbol server.")] + [bool]$PublishToPublic = $true +) + +Set-StrictMode -Version Latest +$ErrorActionPreference = "Stop" + +# --- Log input parameters --- +Write-Host "=== Publish Symbols Parameters ===" +Write-Host "PublishServer: ${PublishServer}" +Write-Host "PublishTokenUri: ${PublishTokenUri}" +Write-Host "PublishProjectName: ${PublishProjectName}" +Write-Host "ArtifactName: ${ArtifactName}" +Write-Host "PublishToInternal: ${PublishToInternal}" +Write-Host "PublishToPublic: ${PublishToPublic}" +Write-Host "==================================" + +# --- Build request name and URLs --- +$requestName = ${ArtifactName} +$baseUrl = "https://${PublishServer}.trafficmanager.net/projects/${PublishProjectName}" +$registerUrl = "${baseUrl}/requests" +$requestUrl = "${baseUrl}/requests/${requestName}" + +Write-Host "=== Constructed URLs ===" +Write-Host "Request Name: ${requestName}" +Write-Host "Base URL: ${baseUrl}" +Write-Host "Register URL: ${registerUrl}" +Write-Host "Request URL: ${requestUrl}" +Write-Host "========================" + +# --- Step 1: Acquire token --- +Write-Host "> 1. Acquiring symbol publishing token..." +$symbolPublishingToken = az account get-access-token --resource ${PublishTokenUri} --query accessToken -o tsv +if ($LASTEXITCODE -ne 0) { + throw "Failed to acquire symbol publishing token via Azure CLI (exit code: ${LASTEXITCODE})." +} +if ($null -ne $symbolPublishingToken) { + $symbolPublishingToken = $symbolPublishingToken.Trim() +} +if ([string]::IsNullOrWhiteSpace($symbolPublishingToken)) { + throw "Failed to acquire symbol publishing token via Azure CLI: received an empty or whitespace-only access token." +} +Write-Host "> 1. Symbol publishing token acquired." + +$authHeaders = @{ Authorization = "Bearer ${symbolPublishingToken}" } + +# --- Step 2: Register request name --- +Write-Host "> 2. Registering request name..." +$requestNameRegistrationBody = @{ requestName = $requestName } | ConvertTo-Json -Compress +try { + Invoke-RestMethod -Method POST -Uri ${registerUrl} -Headers ${authHeaders} -ContentType "application/json" -Body ${requestNameRegistrationBody} +} catch { + throw "Failed to register request name. URI: ${registerUrl} | Body: ${requestNameRegistrationBody} | Error: $_" +} +Write-Host "> 2. Request name registered successfully." + +# --- Step 3: Publish symbols --- +Write-Host "> 3. Submitting request to publish symbols..." +$publishSymbolsBody = @{ + publishToInternalServer = $PublishToInternal + publishToPublicServer = $PublishToPublic +} | ConvertTo-Json -Compress +Write-Host "Publishing symbols request body: ${publishSymbolsBody}" +try { + Invoke-RestMethod -Method POST -Uri ${requestUrl} -Headers ${authHeaders} -ContentType "application/json" -Body ${publishSymbolsBody} +} catch { + throw "Failed to publish symbols. URI: ${requestUrl} | Body: ${publishSymbolsBody} | Error: $_" +} +Write-Host "> 3. Request to publish symbols submitted successfully." + +# --- Step 4: Check status --- +Write-Host "> 4. Checking the status of the request..." +try { + $status = Invoke-RestMethod -Method GET -Uri ${requestUrl} -Headers ${authHeaders} -ContentType "application/json" + $status +} catch { + throw "Failed to check request status. URI: ${requestUrl} | Error: $_" +} + +# Validate publishing results — fail the task when the service reports a terminal failure. +# PublishingResult: 0=Pending, 1=Succeeded, 2=Failed, 3=Cancelled +$resultLabels = @{ 0 = 'Pending'; 1 = 'Succeeded'; 2 = 'Failed'; 3 = 'Cancelled' } +$failures = @() + +if ($PublishToInternal) { + $internalResult = $status.publishToInternalServerResult + if ($null -ne $internalResult -and $internalResult -ge 2) { + $label = if ($resultLabels.ContainsKey([int]$internalResult)) { $resultLabels[[int]$internalResult] } else { "Unknown($internalResult)" } + $failures += "Internal server publishing result: ${label} (${internalResult})" + } +} + +if ($PublishToPublic) { + $publicResult = $status.publishToPublicServerResult + if ($null -ne $publicResult -and $publicResult -ge 2) { + $label = if ($resultLabels.ContainsKey([int]$publicResult)) { $resultLabels[[int]$publicResult] } else { "Unknown($publicResult)" } + $failures += "Public server publishing result: ${label} (${publicResult})" + } +} + +if ($failures.Count -gt 0) { + $failureMessage = $failures -join '; ' + throw "Symbol publishing reported a terminal failure. ${failureMessage}. URI: ${requestUrl}" +} + +Write-Host "> 4. Status check completed - no terminal failures detected." + +Write-Host "" +Write-Host "Use below tables to interpret the xxxServerStatus and xxxServerResult fields from the response." +Write-Host "" +Write-Host "PublishingStatus" +Write-Host "-----------------" +Write-Host "0 NotRequested - The request has not been requested to publish." +Write-Host "1 Submitted - The request is submitted to be published." +Write-Host "2 Processing - The request is still being processed." +Write-Host "3 Completed - Processing finished. Check PublishingResult for details." +Write-Host "" +Write-Host "PublishingResult" +Write-Host "-----------------" +Write-Host "0 Pending - The request has not completed or has not been requested." +Write-Host "1 Succeeded - The request published successfully." +Write-Host "2 Failed - The request failed to publish." +Write-Host "3 Cancelled - The request was cancelled." diff --git a/eng/pipelines/onebranch/scripts/tests/README.md b/eng/pipelines/onebranch/scripts/tests/README.md new file mode 100644 index 0000000000..447dda5d32 --- /dev/null +++ b/eng/pipelines/onebranch/scripts/tests/README.md @@ -0,0 +1,43 @@ +# Publish-Symbols Tests + +Pester tests for the `publish-symbols.ps1` script used by the symbol publishing pipeline step. + +## Prerequisites + +- PowerShell 5.1+ or PowerShell 7+ +- [Pester v5](https://pester.dev/) (`Install-Module Pester -MinimumVersion 5.0 -Scope CurrentUser`) + +## Running the Tests + +From this directory: + +```powershell +Invoke-Pester ./publish-symbols.Tests.ps1 +``` + +Or from the repository root: + +```powershell +Invoke-Pester ./eng/pipelines/onebranch/scripts/tests/ +``` + +For detailed output: + +```powershell +Invoke-Pester ./publish-symbols.Tests.ps1 -Output Detailed +``` + +## Test Coverage + +| Area | What's tested | +| --------------------- | ---------------------------------------------------------------- | +| Parameter validation | Empty strings rejected for all mandatory parameters | +| URL construction | Base URL, register URL, request URL built from parameters | +| Request bodies | Registration body, default publish flags, flag overrides | +| Error handling | Token failure, registration failure, publish failure, status failure — all verify expanded URI in error message | +| Status validation | Detects Failed/Cancelled results, respects PublishToInternal/PublishToPublic flags, passes on Succeeded/Pending | + +## Notes + +- All external calls (`az`, `Invoke-RestMethod`) are mocked — no network access or Azure credentials are required. +- Tests validate the script at `../publish-symbols.ps1` relative to this directory. diff --git a/eng/pipelines/onebranch/scripts/tests/publish-symbols.Tests.ps1 b/eng/pipelines/onebranch/scripts/tests/publish-symbols.Tests.ps1 new file mode 100644 index 0000000000..4d16d54e96 --- /dev/null +++ b/eng/pipelines/onebranch/scripts/tests/publish-symbols.Tests.ps1 @@ -0,0 +1,356 @@ +<# +.SYNOPSIS + Pester tests for publish-symbols.ps1 +#> + +BeforeAll { + $scriptPath = Join-Path $PSScriptRoot '..' 'publish-symbols.ps1' +} + +AfterAll { + # Clean up global variables used by mocks + Remove-Variable -Name 'restCalls' -Scope Global -ErrorAction SilentlyContinue + Remove-Variable -Name 'mockCallCount' -Scope Global -ErrorAction SilentlyContinue +} + +Describe 'publish-symbols.ps1 Parameter Validation' { + + It 'Should reject an empty PublishServer' { + { & $scriptPath -PublishServer '' -PublishTokenUri 'https://token' -PublishProjectName 'proj' -ArtifactName 'art' } | + Should -Throw + } + + It 'Should reject an empty PublishTokenUri' { + { & $scriptPath -PublishServer 'server' -PublishTokenUri '' -PublishProjectName 'proj' -ArtifactName 'art' } | + Should -Throw + } + + It 'Should reject an empty PublishProjectName' { + { & $scriptPath -PublishServer 'server' -PublishTokenUri 'https://token' -PublishProjectName '' -ArtifactName 'art' } | + Should -Throw + } + + It 'Should reject an empty ArtifactName' { + { & $scriptPath -PublishServer 'server' -PublishTokenUri 'https://token' -PublishProjectName 'proj' -ArtifactName '' } | + Should -Throw + } +} + +Describe 'publish-symbols.ps1 URL Construction' { + + BeforeAll { + Mock -CommandName 'az' -MockWith { $global:LASTEXITCODE = 0; return 'fake-token-12345' } + + $global:restCalls = @() + Mock -CommandName 'Invoke-RestMethod' -MockWith { + $global:restCalls += @{ + Method = $Method + Uri = $Uri + Body = $Body + } + return @{ publishToInternalServerStatus = 0; publishToPublicServerStatus = 0; publishToInternalServerResult = 0; publishToPublicServerResult = 0 } + } + } + + BeforeEach { + $global:restCalls = @() + } + + It 'Should construct the correct base URL from PublishServer and PublishProjectName' { + & $scriptPath ` + -PublishServer 'myserver' ` + -PublishTokenUri 'https://token-uri' ` + -PublishProjectName 'My.Project' ` + -ArtifactName 'test_artifact' + + $global:restCalls.Count | Should -Be 3 + + # Registration call + $global:restCalls[0].Uri | Should -Be 'https://myserver.trafficmanager.net/projects/My.Project/requests' + $global:restCalls[0].Method | Should -Be 'POST' + + # Publish call + $global:restCalls[1].Uri | Should -Be 'https://myserver.trafficmanager.net/projects/My.Project/requests/test_artifact' + $global:restCalls[1].Method | Should -Be 'POST' + + # Status call + $global:restCalls[2].Uri | Should -Be 'https://myserver.trafficmanager.net/projects/My.Project/requests/test_artifact' + $global:restCalls[2].Method | Should -Be 'GET' + } + + It 'Should use ArtifactName directly as the request name' { + & $scriptPath ` + -PublishServer 'srv' ` + -PublishTokenUri 'https://token-uri' ` + -PublishProjectName 'proj' ` + -ArtifactName 'myartifact_3' + + $global:restCalls[1].Uri | Should -BeLike '*myartifact_3' + $global:restCalls[2].Uri | Should -BeLike '*myartifact_3' + } +} + +Describe 'publish-symbols.ps1 Request Bodies' { + + BeforeAll { + Mock -CommandName 'az' -MockWith { $global:LASTEXITCODE = 0; return 'fake-token-12345' } + + $global:restCalls = @() + Mock -CommandName 'Invoke-RestMethod' -MockWith { + $global:restCalls += @{ + Method = $Method + Uri = $Uri + Body = $Body + } + return @{ publishToInternalServerStatus = 0; publishToPublicServerStatus = 0; publishToInternalServerResult = 0; publishToPublicServerResult = 0 } + } + } + + BeforeEach { + $global:restCalls = @() + } + + It 'Should send the correct request name in the registration body' { + & $scriptPath ` + -PublishServer 'srv' ` + -PublishTokenUri 'https://token-uri' ` + -PublishProjectName 'proj' ` + -ArtifactName 'my_artifact_1' + + $body = $global:restCalls[0].Body | ConvertFrom-Json + $body.requestName | Should -Be 'my_artifact_1' + } + + It 'Should default to publishing to both internal and public servers' { + & $scriptPath ` + -PublishServer 'srv' ` + -PublishTokenUri 'https://token-uri' ` + -PublishProjectName 'proj' ` + -ArtifactName 'art' + + $body = $global:restCalls[1].Body | ConvertFrom-Json + $body.publishToInternalServer | Should -Be $true + $body.publishToPublicServer | Should -Be $true + } + + It 'Should respect PublishToInternal=false' { + & $scriptPath ` + -PublishServer 'srv' ` + -PublishTokenUri 'https://token-uri' ` + -PublishProjectName 'proj' ` + -ArtifactName 'art' ` + -PublishToInternal $false + + $body = $global:restCalls[1].Body | ConvertFrom-Json + $body.publishToInternalServer | Should -Be $false + $body.publishToPublicServer | Should -Be $true + } + + It 'Should respect PublishToPublic=false' { + & $scriptPath ` + -PublishServer 'srv' ` + -PublishTokenUri 'https://token-uri' ` + -PublishProjectName 'proj' ` + -ArtifactName 'art' ` + -PublishToPublic $false + + $body = $global:restCalls[1].Body | ConvertFrom-Json + $body.publishToInternalServer | Should -Be $true + $body.publishToPublicServer | Should -Be $false + } +} + +Describe 'publish-symbols.ps1 Error Handling' { + + It 'Should throw when token acquisition fails' { + Mock -CommandName 'az' -MockWith { $global:LASTEXITCODE = 1; return '' } + + { & $scriptPath ` + -PublishServer 'srv' ` + -PublishTokenUri 'https://token-uri' ` + -PublishProjectName 'proj' ` + -ArtifactName 'art' } | + Should -Throw '*token*' + } + + It 'Should throw when token is empty' { + Mock -CommandName 'az' -MockWith { $global:LASTEXITCODE = 0; return ' ' } + + { & $scriptPath ` + -PublishServer 'srv' ` + -PublishTokenUri 'https://token-uri' ` + -PublishProjectName 'proj' ` + -ArtifactName 'art' } | + Should -Throw '*empty*' + } + + It 'Should throw with URI details when registration fails' { + Mock -CommandName 'az' -MockWith { $global:LASTEXITCODE = 0; return 'fake-token' } + Mock -CommandName 'Invoke-RestMethod' -MockWith { throw "Connection refused" } + + { & $scriptPath ` + -PublishServer 'srv' ` + -PublishTokenUri 'https://token-uri' ` + -PublishProjectName 'proj' ` + -ArtifactName 'art' } | + Should -Throw '*Failed to register*' + } + + It 'Should throw with URI details when publish fails' { + Mock -CommandName 'az' -MockWith { $global:LASTEXITCODE = 0; return 'fake-token' } + $global:mockCallCount = 0 + Mock -CommandName 'Invoke-RestMethod' -MockWith { + $global:mockCallCount++ + if ($global:mockCallCount -eq 1) { return @{} } + throw "Service unavailable" + } + + { & $scriptPath ` + -PublishServer 'srv' ` + -PublishTokenUri 'https://token-uri' ` + -PublishProjectName 'proj' ` + -ArtifactName 'art' } | + Should -Throw '*Failed to publish*' + } + + It 'Should throw with URI details when status check fails' { + Mock -CommandName 'az' -MockWith { $global:LASTEXITCODE = 0; return 'fake-token' } + $global:mockCallCount = 0 + Mock -CommandName 'Invoke-RestMethod' -MockWith { + $global:mockCallCount++ + if ($global:mockCallCount -le 2) { return @{} } + throw "Timeout" + } + + { & $scriptPath ` + -PublishServer 'srv' ` + -PublishTokenUri 'https://token-uri' ` + -PublishProjectName 'proj' ` + -ArtifactName 'art' } | + Should -Throw '*Failed to check*' + } +} + +Describe 'publish-symbols.ps1 Status Failure Detection' { + + It 'Should throw when internal server result is Failed (2)' { + Mock -CommandName 'az' -MockWith { $global:LASTEXITCODE = 0; return 'fake-token' } + $global:mockCallCount = 0 + Mock -CommandName 'Invoke-RestMethod' -MockWith { + $global:mockCallCount++ + if ($global:mockCallCount -le 2) { return @{} } + return @{ publishToInternalServerResult = 2; publishToPublicServerResult = 0 } + } + + { & $scriptPath ` + -PublishServer 'srv' ` + -PublishTokenUri 'https://token-uri' ` + -PublishProjectName 'proj' ` + -ArtifactName 'art' } | + Should -Throw '*terminal failure*Internal server*Failed*' + } + + It 'Should throw when public server result is Cancelled (3)' { + Mock -CommandName 'az' -MockWith { $global:LASTEXITCODE = 0; return 'fake-token' } + $global:mockCallCount = 0 + Mock -CommandName 'Invoke-RestMethod' -MockWith { + $global:mockCallCount++ + if ($global:mockCallCount -le 2) { return @{} } + return @{ publishToInternalServerResult = 1; publishToPublicServerResult = 3 } + } + + { & $scriptPath ` + -PublishServer 'srv' ` + -PublishTokenUri 'https://token-uri' ` + -PublishProjectName 'proj' ` + -ArtifactName 'art' } | + Should -Throw '*terminal failure*Public server*Cancelled*' + } + + It 'Should throw when both servers report failure' { + Mock -CommandName 'az' -MockWith { $global:LASTEXITCODE = 0; return 'fake-token' } + $global:mockCallCount = 0 + Mock -CommandName 'Invoke-RestMethod' -MockWith { + $global:mockCallCount++ + if ($global:mockCallCount -le 2) { return @{} } + return @{ publishToInternalServerResult = 2; publishToPublicServerResult = 3 } + } + + { & $scriptPath ` + -PublishServer 'srv' ` + -PublishTokenUri 'https://token-uri' ` + -PublishProjectName 'proj' ` + -ArtifactName 'art' } | + Should -Throw '*terminal failure*Internal server*Public server*' + } + + It 'Should not throw when both servers report Succeeded (1)' { + Mock -CommandName 'az' -MockWith { $global:LASTEXITCODE = 0; return 'fake-token' } + $global:mockCallCount = 0 + Mock -CommandName 'Invoke-RestMethod' -MockWith { + $global:mockCallCount++ + if ($global:mockCallCount -le 2) { return @{} } + return @{ publishToInternalServerResult = 1; publishToPublicServerResult = 1 } + } + + { & $scriptPath ` + -PublishServer 'srv' ` + -PublishTokenUri 'https://token-uri' ` + -PublishProjectName 'proj' ` + -ArtifactName 'art' } | + Should -Not -Throw + } + + It 'Should not throw when results are Pending (0)' { + Mock -CommandName 'az' -MockWith { $global:LASTEXITCODE = 0; return 'fake-token' } + $global:mockCallCount = 0 + Mock -CommandName 'Invoke-RestMethod' -MockWith { + $global:mockCallCount++ + if ($global:mockCallCount -le 2) { return @{} } + return @{ publishToInternalServerResult = 0; publishToPublicServerResult = 0 } + } + + { & $scriptPath ` + -PublishServer 'srv' ` + -PublishTokenUri 'https://token-uri' ` + -PublishProjectName 'proj' ` + -ArtifactName 'art' } | + Should -Not -Throw + } + + It 'Should not check internal result when PublishToInternal is false' { + Mock -CommandName 'az' -MockWith { $global:LASTEXITCODE = 0; return 'fake-token' } + $global:mockCallCount = 0 + Mock -CommandName 'Invoke-RestMethod' -MockWith { + $global:mockCallCount++ + if ($global:mockCallCount -le 2) { return @{} } + return @{ publishToInternalServerResult = 2; publishToPublicServerResult = 1 } + } + + { & $scriptPath ` + -PublishServer 'srv' ` + -PublishTokenUri 'https://token-uri' ` + -PublishProjectName 'proj' ` + -ArtifactName 'art' ` + -PublishToInternal $false } | + Should -Not -Throw + } + + It 'Should not check public result when PublishToPublic is false' { + Mock -CommandName 'az' -MockWith { $global:LASTEXITCODE = 0; return 'fake-token' } + $global:mockCallCount = 0 + Mock -CommandName 'Invoke-RestMethod' -MockWith { + $global:mockCallCount++ + if ($global:mockCallCount -le 2) { return @{} } + return @{ publishToInternalServerResult = 1; publishToPublicServerResult = 2 } + } + + { & $scriptPath ` + -PublishServer 'srv' ` + -PublishTokenUri 'https://token-uri' ` + -PublishProjectName 'proj' ` + -ArtifactName 'art' ` + -PublishToPublic $false } | + Should -Not -Throw + } +} diff --git a/eng/pipelines/onebranch/sqlclient-non-official.yml b/eng/pipelines/onebranch/sqlclient-non-official.yml index a908ae4e74..4ae301020f 100644 --- a/eng/pipelines/onebranch/sqlclient-non-official.yml +++ b/eng/pipelines/onebranch/sqlclient-non-official.yml @@ -174,7 +174,6 @@ extends: - template: /eng/pipelines/onebranch/stages/build-stages.yml@self parameters: isOfficial: false # This is a non-official pipeline. - publishSymbols: ${{ parameters.publishSymbols }} buildSqlServerServer: ${{ parameters.buildSqlServerServer }} buildSqlClient: ${{ parameters.buildSqlClient }} buildAkvProvider: ${{ parameters.buildAkvProvider }} @@ -204,10 +203,32 @@ extends: signingAuthSignCertName: '$(SigningAuthSignCertName)' signingEsrpClientId: '$(SigningEsrpClientId)' signingEsrpConnectedServiceName: '$(SigningEsrpConnectedServiceName)' + + - template: /eng/pipelines/onebranch/stages/publish-symbols-stage.yml@self + parameters: + publishSymbols: ${{ parameters.publishSymbols }} + buildSqlServerServer: ${{ parameters.buildSqlServerServer }} + buildSqlClient: ${{ parameters.buildSqlClient }} + buildAkvProvider: ${{ parameters.buildAkvProvider }} + + abstractionsArtifactsName: '${{ variables.abstractionsArtifactsName }}' + abstractionsPackageVersion: '${{ variables.abstractionsPackageVersion }}' + akvProviderArtifactsName: '${{ variables.akvProviderArtifactsName }}' + akvProviderPackageVersion: '${{ variables.akvProviderPackageVersion }}' + azureArtifactsName: '${{ variables.azureArtifactsName }}' + azurePackageVersion: '${{ variables.azurePackageVersion }}' + loggingArtifactsName: '${{ variables.loggingArtifactsName }}' + loggingPackageVersion: '${{ variables.loggingPackageVersion }}' + sqlClientArtifactsName: '${{ variables.sqlClientArtifactsName }}' + sqlClientPackageVersion: '${{ variables.sqlClientPackageVersion }}' + sqlServerArtifactsName: '${{ variables.sqlServerArtifactsName }}' + sqlServerPackageVersion: '${{ variables.sqlServerPackageVersion }}' + symbolsAzureSubscription: '$(SymbolsAzureSubscription)' - symbolsPublishProjectName: '$(SymbolsPublishProjectName)' - symbolsPublishServer: '$(SymbolsPublishServer)' - symbolsPublishTokenUri: '$(SymbolsPublishTokenUri)' + symbolsPublishProjectName: '$(SymbolsPublishProjectNameSqlClient)' + # Non-Official pipelines must publish to the PPE symbol server. + symbolsPublishServer: '$(SymbolsPublishServerPPE)' + symbolsPublishTokenUri: '$(SymbolsPublishTokenUriPPE)' symbolsUploadAccount: '$(SymbolsUploadAccount)' - template: /eng/pipelines/onebranch/stages/release-stages.yml@self @@ -219,6 +240,8 @@ extends: isOfficial: false stageNameSuffix: test + publishSymbols: ${{ parameters.publishSymbols }} + abstractionsArtifactsName: '${{ variables.abstractionsArtifactsName }}' abstractionsPackageVersion: '${{ variables.abstractionsPackageVersion }}' akvProviderArtifactsName: '${{ variables.akvProviderArtifactsName }}' diff --git a/eng/pipelines/onebranch/sqlclient-official.yml b/eng/pipelines/onebranch/sqlclient-official.yml index 7f02ea0226..a2902f1e21 100644 --- a/eng/pipelines/onebranch/sqlclient-official.yml +++ b/eng/pipelines/onebranch/sqlclient-official.yml @@ -202,7 +202,6 @@ extends: - template: /eng/pipelines/onebranch/stages/build-stages.yml@self parameters: isOfficial: true # This is an official pipeline. - publishSymbols: ${{ parameters.publishSymbols }} buildSqlServerServer: ${{ parameters.buildSqlServerServer }} buildSqlClient: ${{ parameters.buildSqlClient }} buildAkvProvider: ${{ parameters.buildAkvProvider }} @@ -232,10 +231,32 @@ extends: signingAuthSignCertName: '$(SigningAuthSignCertName)' signingEsrpClientId: '$(SigningEsrpClientId)' signingEsrpConnectedServiceName: '$(SigningEsrpConnectedServiceName)' + + - template: /eng/pipelines/onebranch/stages/publish-symbols-stage.yml@self + parameters: + publishSymbols: ${{ parameters.publishSymbols }} + buildSqlServerServer: ${{ parameters.buildSqlServerServer }} + buildSqlClient: ${{ parameters.buildSqlClient }} + buildAkvProvider: ${{ parameters.buildAkvProvider }} + + abstractionsArtifactsName: '${{ variables.abstractionsArtifactsName }}' + abstractionsPackageVersion: '${{ variables.abstractionsPackageVersion }}' + akvProviderArtifactsName: '${{ variables.akvProviderArtifactsName }}' + akvProviderPackageVersion: '${{ variables.akvProviderPackageVersion }}' + azureArtifactsName: '${{ variables.azureArtifactsName }}' + azurePackageVersion: '${{ variables.azurePackageVersion }}' + loggingArtifactsName: '${{ variables.loggingArtifactsName }}' + loggingPackageVersion: '${{ variables.loggingPackageVersion }}' + sqlClientArtifactsName: '${{ variables.sqlClientArtifactsName }}' + sqlClientPackageVersion: '${{ variables.sqlClientPackageVersion }}' + sqlServerArtifactsName: '${{ variables.sqlServerArtifactsName }}' + sqlServerPackageVersion: '${{ variables.sqlServerPackageVersion }}' + symbolsAzureSubscription: '$(SymbolsAzureSubscription)' - symbolsPublishProjectName: '$(SymbolsPublishProjectName)' - symbolsPublishServer: '$(SymbolsPublishServer)' - symbolsPublishTokenUri: '$(SymbolsPublishTokenUri)' + symbolsPublishProjectName: '$(SymbolsPublishProjectNameSqlClient)' + # Official pipelines must publish to the Production symbol server. + symbolsPublishServer: '$(SymbolsPublishServerProd)' + symbolsPublishTokenUri: '$(SymbolsPublishTokenUriProd)' symbolsUploadAccount: '$(SymbolsUploadAccount)' - template: /eng/pipelines/onebranch/stages/release-stages.yml@self @@ -246,6 +267,8 @@ extends: isOfficial: true stageNameSuffix: production + publishSymbols: ${{ parameters.publishSymbols }} + abstractionsArtifactsName: '${{ variables.abstractionsArtifactsName }}' abstractionsPackageVersion: '${{ variables.abstractionsPackageVersion }}' akvProviderArtifactsName: '${{ variables.akvProviderArtifactsName }}' diff --git a/eng/pipelines/onebranch/stages/build-stages.yml b/eng/pipelines/onebranch/stages/build-stages.yml index ba24af6309..252f63fb0a 100644 --- a/eng/pipelines/onebranch/stages/build-stages.yml +++ b/eng/pipelines/onebranch/stages/build-stages.yml @@ -22,10 +22,6 @@ parameters: - name: isOfficial type: boolean - # True to publish symbols to public and private servers. - - name: publishSymbols - type: boolean - # ── Build parameters ─────────────────────────────────────────────────── - name: buildSqlServerServer @@ -113,23 +109,6 @@ parameters: - name: signingEsrpConnectedServiceName type: string - # Symbols Publishing Parameters ------------------------------------------ - - - name: symbolsAzureSubscription - type: string - - - name: symbolsPublishProjectName - type: string - - - name: symbolsPublishServer - type: string - - - name: symbolsPublishTokenUri - type: string - - - name: symbolsUploadAccount - type: string - stages: # ==================================================================== # Stage 1: Independent packages (no cross-package dependencies) @@ -148,21 +127,14 @@ stages: signingAuthSignCertName: '${{ parameters.signingAuthSignCertName }}' signingEsrpClientId: '${{ parameters.signingEsrpClientId }}' signingEsrpConnectedServiceName: '${{ parameters.signingEsrpConnectedServiceName }}' - symbolsAzureSubscription: '${{ parameters.symbolsAzureSubscription }}' - symbolsPublishProjectName: '${{ parameters.symbolsPublishProjectName }}' - symbolsPublishServer: '${{ parameters.symbolsPublishServer }}' - symbolsPublishTokenUri: '${{ parameters.symbolsPublishTokenUri }}' - symbolsUploadAccount: '${{ parameters.symbolsUploadAccount }}' isOfficial: ${{ parameters.isOfficial }} packageName: Logging packageFullName: Microsoft.Data.SqlClient.Internal.Logging - packageVersion: '${{ parameters.loggingPackageVersion }}' versionProperties: >- -p:LoggingPackageVersion=${{ parameters.loggingPackageVersion }} -p:LoggingAssemblyFileVersion=${{ parameters.loggingFileVersion }} assemblyFileVersion: '${{ parameters.loggingFileVersion }}' - publishSymbols: ${{ parameters.publishSymbols }} - ${{ if eq(parameters.buildSqlServerServer, true) }}: @@ -174,21 +146,14 @@ stages: signingAuthSignCertName: '${{ parameters.signingAuthSignCertName }}' signingEsrpClientId: '${{ parameters.signingEsrpClientId }}' signingEsrpConnectedServiceName: '${{ parameters.signingEsrpConnectedServiceName }}' - symbolsAzureSubscription: '${{ parameters.symbolsAzureSubscription }}' - symbolsPublishProjectName: '${{ parameters.symbolsPublishProjectName }}' - symbolsPublishServer: '${{ parameters.symbolsPublishServer }}' - symbolsPublishTokenUri: '${{ parameters.symbolsPublishTokenUri }}' - symbolsUploadAccount: '${{ parameters.symbolsUploadAccount }}' isOfficial: ${{ parameters.isOfficial }} packageName: SqlServer packageFullName: Microsoft.SqlServer.Server - packageVersion: '${{ parameters.sqlServerPackageVersion }}' versionProperties: >- -p:SqlServerAssemblyFileVersion=${{ parameters.sqlServerFileVersion }} -p:SqlServerPackageVersion=${{ parameters.sqlServerPackageVersion }} assemblyFileVersion: '${{ parameters.sqlServerFileVersion }}' - publishSymbols: ${{ parameters.publishSymbols }} # ==================================================================== # Stage 2: Abstractions package (depends on Logging from Stage 1) @@ -209,22 +174,15 @@ stages: signingAuthSignCertName: '${{ parameters.signingAuthSignCertName }}' signingEsrpClientId: '${{ parameters.signingEsrpClientId }}' signingEsrpConnectedServiceName: '${{ parameters.signingEsrpConnectedServiceName }}' - symbolsAzureSubscription: '${{ parameters.symbolsAzureSubscription }}' - symbolsPublishProjectName: '${{ parameters.symbolsPublishProjectName }}' - symbolsPublishServer: '${{ parameters.symbolsPublishServer }}' - symbolsPublishTokenUri: '${{ parameters.symbolsPublishTokenUri }}' - symbolsUploadAccount: '${{ parameters.symbolsUploadAccount }}' isOfficial: ${{ parameters.isOfficial }} packageName: Abstractions packageFullName: Microsoft.Data.SqlClient.Extensions.Abstractions - packageVersion: '${{ parameters.abstractionsPackageVersion }}' versionProperties: >- -p:AbstractionsPackageVersion=${{ parameters.abstractionsPackageVersion }} -p:AbstractionsAssemblyFileVersion=${{ parameters.abstractionsFileVersion }} -p:LoggingPackageVersion=${{ parameters.loggingPackageVersion }} assemblyFileVersion: '${{ parameters.abstractionsFileVersion }}' - publishSymbols: ${{ parameters.publishSymbols }} downloadArtifacts: - artifactName: '${{ parameters.loggingArtifactsName }}' displayName: 'Logging Package' @@ -242,21 +200,13 @@ stages: jobs: - template: /eng/pipelines/onebranch/jobs/build-signed-sqlclient-package-job.yml@self parameters: - apiScanDllPath: '$(REPO_ROOT)/apiScan/Microsoft.Data.SqlClient/dlls' - apiScanPdbPath: '$(REPO_ROOT)/apiScan/Microsoft.Data.SqlClient/pdbs' isOfficial: ${{ parameters.isOfficial }} - publishSymbols: ${{ parameters.publishSymbols }} signingAppRegistrationClientId: '${{ parameters.signingAppRegistrationClientId }}' signingAppRegistrationTenantId: '${{ parameters.signingAppRegistrationTenantId }}' signingAuthAkvName: '${{ parameters.signingAuthAkvName }}' signingAuthSignCertName: '${{ parameters.signingAuthSignCertName }}' signingEsrpClientId: '${{ parameters.signingEsrpClientId }}' signingEsrpConnectedServiceName: '${{ parameters.signingEsrpConnectedServiceName }}' - symbolsAzureSubscription: '${{ parameters.symbolsAzureSubscription }}' - symbolsPublishProjectName: '${{ parameters.symbolsPublishProjectName }}' - symbolsPublishServer: '${{ parameters.symbolsPublishServer }}' - symbolsPublishTokenUri: '${{ parameters.symbolsPublishTokenUri }}' - symbolsUploadAccount: '${{ parameters.symbolsUploadAccount }}' abstractionsArtifactName: '${{ parameters.abstractionsArtifactsName }}' abstractionsPackageVersion: '${{ parameters.abstractionsPackageVersion }}' @@ -273,23 +223,16 @@ stages: signingAuthSignCertName: '${{ parameters.signingAuthSignCertName }}' signingEsrpClientId: '${{ parameters.signingEsrpClientId }}' signingEsrpConnectedServiceName: '${{ parameters.signingEsrpConnectedServiceName }}' - symbolsAzureSubscription: '${{ parameters.symbolsAzureSubscription }}' - symbolsPublishProjectName: '${{ parameters.symbolsPublishProjectName }}' - symbolsPublishServer: '${{ parameters.symbolsPublishServer }}' - symbolsPublishTokenUri: '${{ parameters.symbolsPublishTokenUri }}' - symbolsUploadAccount: '${{ parameters.symbolsUploadAccount }}' isOfficial: ${{ parameters.isOfficial }} packageName: Azure packageFullName: Microsoft.Data.SqlClient.Extensions.Azure - packageVersion: '${{ parameters.azurePackageVersion }}' versionProperties: >- -p:AzurePackageVersion=${{ parameters.azurePackageVersion }} -p:AzureAssemblyFileVersion=${{ parameters.azureFileVersion }} -p:AbstractionsPackageVersion=${{ parameters.abstractionsPackageVersion }} -p:LoggingPackageVersion=${{ parameters.loggingPackageVersion }} assemblyFileVersion: '${{ parameters.azureFileVersion }}' - publishSymbols: ${{ parameters.publishSymbols }} downloadArtifacts: - artifactName: '${{ parameters.abstractionsArtifactsName }}' displayName: Abstractions Package @@ -314,16 +257,10 @@ stages: signingAuthSignCertName: '${{ parameters.signingAuthSignCertName }}' signingEsrpClientId: '${{ parameters.signingEsrpClientId }}' signingEsrpConnectedServiceName: '${{ parameters.signingEsrpConnectedServiceName }}' - symbolsAzureSubscription: '${{ parameters.symbolsAzureSubscription }}' - symbolsPublishProjectName: '${{ parameters.symbolsPublishProjectName }}' - symbolsPublishServer: '${{ parameters.symbolsPublishServer }}' - symbolsPublishTokenUri: '${{ parameters.symbolsPublishTokenUri }}' - symbolsUploadAccount: '${{ parameters.symbolsUploadAccount }}' isOfficial: ${{ parameters.isOfficial }} packageName: AkvProvider packageFullName: Microsoft.Data.SqlClient.AlwaysEncrypted.AzureKeyVaultProvider - packageVersion: '${{ parameters.akvProviderPackageVersion }}' versionProperties: >- -p:AkvPackageVersion=${{ parameters.akvProviderPackageVersion }} -p:AkvAssemblyFileVersion=${{ parameters.akvProviderFileVersion }} @@ -331,7 +268,6 @@ stages: -p:LoggingPackageVersion=${{ parameters.loggingPackageVersion }} -p:AbstractionsPackageVersion=${{ parameters.abstractionsPackageVersion }} assemblyFileVersion: '${{ parameters.akvProviderFileVersion }}' - publishSymbols: ${{ parameters.publishSymbols }} downloadArtifacts: - artifactName: '${{ parameters.sqlClientArtifactsName }}' displayName: SqlClient Package diff --git a/eng/pipelines/onebranch/stages/publish-symbols-stage.yml b/eng/pipelines/onebranch/stages/publish-symbols-stage.yml new file mode 100644 index 0000000000..e716d2255d --- /dev/null +++ b/eng/pipelines/onebranch/stages/publish-symbols-stage.yml @@ -0,0 +1,222 @@ +################################################################################# +# Licensed to the .NET Foundation under one or more agreements. # +# The .NET Foundation licenses this file to you under the MIT license. # +# See the LICENSE file in the project root for more information. # +################################################################################# + +# Unified symbols publishing stage template for sqlclient OneBranch pipelines. +# Consumed by both the official and non-official pipeline definitions. +# +# This stage publishes PDB symbol files to the internal and public symbol servers. +# Each package's PDBs are published in a separate job to maintain unique naming and +# versioning information per package. +# +# PDBs are expected to be located under the 'symbols/' directory at the root of +# each build artifact, with target framework subdirectories preserved by the build +# job's CopyFiles step. The publish-symbols job consumes that 'symbols' folder directly. +# +# Note that none of the resource DLLs we build produce PDBs, so there are no patterns below that +# match them. +# +# The stage is excluded at compile time when publishSymbols is false. + +parameters: + # ── General parameters ───────────────────────────────────────────────── + + # True to publish symbols (controls whether this stage is emitted at all). + - name: publishSymbols + type: boolean + + # ── Build parameters (used to conditionally include per-package jobs) ─── + + - name: buildSqlServerServer + type: boolean + + - name: buildSqlClient + type: boolean + + - name: buildAkvProvider + type: boolean + + # ── Package Parameters ───────────────────────────────────────────────── + + - name: abstractionsArtifactsName + type: string + + - name: abstractionsPackageVersion + type: string + + - name: akvProviderArtifactsName + type: string + + - name: akvProviderPackageVersion + type: string + + - name: azureArtifactsName + type: string + + - name: azurePackageVersion + type: string + + - name: loggingArtifactsName + type: string + + - name: loggingPackageVersion + type: string + + - name: sqlClientArtifactsName + type: string + + - name: sqlClientPackageVersion + type: string + + - name: sqlServerArtifactsName + type: string + + - name: sqlServerPackageVersion + type: string + + # ── Symbols Publishing Parameters ────────────────────────────────────── + + - name: symbolsAzureSubscription + type: string + + - name: symbolsPublishProjectName + type: string + + - name: symbolsPublishServer + type: string + + - name: symbolsPublishTokenUri + type: string + + - name: symbolsUploadAccount + type: string + +stages: + # Stage is emitted whenever publishSymbols is true. If no packages were + # built, only the no-op placeholder job runs and the stage succeeds + # immediately — keeping the dependency graph simple for downstream stages. + - ${{ if eq(parameters.publishSymbols, true) }}: + - stage: publish_symbols + displayName: "Publish Symbols" + + # Depend on whichever build stages produced artifacts we need to publish. + dependsOn: + - ${{ if or(parameters.buildSqlServerServer, parameters.buildSqlClient, parameters.buildAkvProvider) }}: + - build_independent + - ${{ if parameters.buildSqlClient }}: + - build_abstractions + - build_dependent + - ${{ if and(parameters.buildAkvProvider, parameters.buildSqlClient) }}: + - build_addons + + jobs: + # ── No-op placeholder ────────────────────────────────────────── + # When no packages were built, this job keeps the stage valid + # (ADO requires at least one job per stage) and succeeds instantly. + - ${{ if not(or(parameters.buildSqlServerServer, parameters.buildSqlClient, parameters.buildAkvProvider)) }}: + - job: publish_symbols_noop + displayName: "No symbols to publish" + pool: + type: linux + steps: + - script: echo "No packages were built — nothing to publish." + displayName: Skip + + # ── Logging ──────────────────────────────────────────────────── + - ${{ if or(parameters.buildAkvProvider, parameters.buildSqlClient) }}: + - template: /eng/pipelines/onebranch/jobs/publish-symbols-job.yml@self + parameters: + artifactName: '${{ parameters.loggingArtifactsName }}' + packageName: Logging + packageFullName: Microsoft.Data.SqlClient.Internal.Logging + packageVersion: '${{ parameters.loggingPackageVersion }}' + # Matches: Microsoft.Data.SqlClient.Internal.Logging.pdb across all TFM subdirs. + searchPattern: '**/Microsoft.Data.SqlClient.Internal.Logging.pdb' + symbolsAzureSubscription: '${{ parameters.symbolsAzureSubscription }}' + symbolsPublishProjectName: '${{ parameters.symbolsPublishProjectName }}' + symbolsPublishServer: '${{ parameters.symbolsPublishServer }}' + symbolsPublishTokenUri: '${{ parameters.symbolsPublishTokenUri }}' + symbolsUploadAccount: '${{ parameters.symbolsUploadAccount }}' + + # ── SqlServer.Server ─────────────────────────────────────────── + - ${{ if eq(parameters.buildSqlServerServer, true) }}: + - template: /eng/pipelines/onebranch/jobs/publish-symbols-job.yml@self + parameters: + artifactName: '${{ parameters.sqlServerArtifactsName }}' + packageName: SqlServer + packageFullName: Microsoft.SqlServer.Server + packageVersion: '${{ parameters.sqlServerPackageVersion }}' + # Matches: Microsoft.SqlServer.Server.pdb across all TFM subdirs. + searchPattern: '**/Microsoft.SqlServer.Server.pdb' + symbolsAzureSubscription: '${{ parameters.symbolsAzureSubscription }}' + symbolsPublishProjectName: '${{ parameters.symbolsPublishProjectName }}' + symbolsPublishServer: '${{ parameters.symbolsPublishServer }}' + symbolsPublishTokenUri: '${{ parameters.symbolsPublishTokenUri }}' + symbolsUploadAccount: '${{ parameters.symbolsUploadAccount }}' + + # ── Abstractions ─────────────────────────────────────────────── + - ${{ if eq(parameters.buildSqlClient, true) }}: + - template: /eng/pipelines/onebranch/jobs/publish-symbols-job.yml@self + parameters: + artifactName: '${{ parameters.abstractionsArtifactsName }}' + packageName: Abstractions + packageFullName: Microsoft.Data.SqlClient.Extensions.Abstractions + packageVersion: '${{ parameters.abstractionsPackageVersion }}' + # Matches: Microsoft.Data.SqlClient.Extensions.Abstractions.pdb across all TFM subdirs. + searchPattern: '**/Microsoft.Data.SqlClient.Extensions.Abstractions.pdb' + symbolsAzureSubscription: '${{ parameters.symbolsAzureSubscription }}' + symbolsPublishProjectName: '${{ parameters.symbolsPublishProjectName }}' + symbolsPublishServer: '${{ parameters.symbolsPublishServer }}' + symbolsPublishTokenUri: '${{ parameters.symbolsPublishTokenUri }}' + symbolsUploadAccount: '${{ parameters.symbolsUploadAccount }}' + + # ── SqlClient ────────────────────────────────────────────────── + - ${{ if eq(parameters.buildSqlClient, true) }}: + - template: /eng/pipelines/onebranch/jobs/publish-symbols-job.yml@self + parameters: + artifactName: '${{ parameters.sqlClientArtifactsName }}' + packageName: SqlClient + packageFullName: Microsoft.Data.SqlClient + packageVersion: '${{ parameters.sqlClientPackageVersion }}' + # Matches: Microsoft.Data.SqlClient.pdb across all OS/TFM subdirs. + # Excludes: Microsoft.Data.SqlClient.SNI*.pdb (native SNI — symbols published separately). + searchPattern: '**/Microsoft.Data.SqlClient.pdb' + symbolsAzureSubscription: '${{ parameters.symbolsAzureSubscription }}' + symbolsPublishProjectName: '${{ parameters.symbolsPublishProjectName }}' + symbolsPublishServer: '${{ parameters.symbolsPublishServer }}' + symbolsPublishTokenUri: '${{ parameters.symbolsPublishTokenUri }}' + symbolsUploadAccount: '${{ parameters.symbolsUploadAccount }}' + + # ── Azure ────────────────────────────────────────────────────── + - ${{ if eq(parameters.buildSqlClient, true) }}: + - template: /eng/pipelines/onebranch/jobs/publish-symbols-job.yml@self + parameters: + artifactName: '${{ parameters.azureArtifactsName }}' + packageName: Azure + packageFullName: Microsoft.Data.SqlClient.Extensions.Azure + packageVersion: '${{ parameters.azurePackageVersion }}' + # Matches: Microsoft.Data.SqlClient.Extensions.Azure.pdb across all TFM subdirs. + searchPattern: '**/Microsoft.Data.SqlClient.Extensions.Azure.pdb' + symbolsAzureSubscription: '${{ parameters.symbolsAzureSubscription }}' + symbolsPublishProjectName: '${{ parameters.symbolsPublishProjectName }}' + symbolsPublishServer: '${{ parameters.symbolsPublishServer }}' + symbolsPublishTokenUri: '${{ parameters.symbolsPublishTokenUri }}' + symbolsUploadAccount: '${{ parameters.symbolsUploadAccount }}' + + # ── AKV Provider ─────────────────────────────────────────────── + - ${{ if and(eq(parameters.buildAkvProvider, true), eq(parameters.buildSqlClient, true)) }}: + - template: /eng/pipelines/onebranch/jobs/publish-symbols-job.yml@self + parameters: + artifactName: '${{ parameters.akvProviderArtifactsName }}' + packageName: AkvProvider + packageFullName: Microsoft.Data.SqlClient.AlwaysEncrypted.AzureKeyVaultProvider + packageVersion: '${{ parameters.akvProviderPackageVersion }}' + # Matches: Microsoft.Data.SqlClient.AlwaysEncrypted.AzureKeyVaultProvider.pdb across all TFM subdirs. + searchPattern: '**/Microsoft.Data.SqlClient.AlwaysEncrypted.AzureKeyVaultProvider.pdb' + symbolsAzureSubscription: '${{ parameters.symbolsAzureSubscription }}' + symbolsPublishProjectName: '${{ parameters.symbolsPublishProjectName }}' + symbolsPublishServer: '${{ parameters.symbolsPublishServer }}' + symbolsPublishTokenUri: '${{ parameters.symbolsPublishTokenUri }}' + symbolsUploadAccount: '${{ parameters.symbolsUploadAccount }}' diff --git a/eng/pipelines/onebranch/stages/release-stages.yml b/eng/pipelines/onebranch/stages/release-stages.yml index 917447a077..6ee3fbc88e 100644 --- a/eng/pipelines/onebranch/stages/release-stages.yml +++ b/eng/pipelines/onebranch/stages/release-stages.yml @@ -89,6 +89,14 @@ parameters: - name: sqlServerPackageVersion type: string + # ── Symbols publishing parameter ─────────────────────────────────────── + # When true, the release stage will depend on publish_symbols so that + # symbol publishing completes before packages are released. + + - name: publishSymbols + type: boolean + default: false + # ── Release parameters ───────────────────────────────────────────────── - name: releaseSqlServerServer @@ -139,6 +147,8 @@ stages: - sqlclient_package_validation - ${{ if parameters.releaseAkvProvider }}: - build_addons + - ${{ if parameters.publishSymbols }}: + - publish_symbols variables: - name: onebranchReleaseEnvironment @@ -183,7 +193,7 @@ stages: parameters: packageName: Microsoft.SqlServer.Server artifactName: '${{ parameters.sqlServerArtifactsName }}' - packagePath: 'Microsoft.SqlServer.Server.${{ parameters.sqlServerPackageVersion }}.nupkg' + packagePath: 'packages/Microsoft.SqlServer.Server.${{ parameters.sqlServerPackageVersion }}.nupkg' nugetServiceConnection: ${{ variables.nugetServiceConnection }} isProduction: ${{ parameters.isOfficial }} displaySuffix: ${{ variables.nugetTargetSuffix }} @@ -193,7 +203,7 @@ stages: parameters: packageName: Microsoft.Data.SqlClient.Internal.Logging artifactName: '${{ parameters.loggingArtifactsName }}' - packagePath: 'Microsoft.Data.SqlClient.Internal.Logging.${{ parameters.loggingPackageVersion }}.nupkg' + packagePath: 'packages/Microsoft.Data.SqlClient.Internal.Logging.${{ parameters.loggingPackageVersion }}.nupkg' nugetServiceConnection: ${{ variables.nugetServiceConnection }} isProduction: ${{ parameters.isOfficial }} displaySuffix: ${{ variables.nugetTargetSuffix }} @@ -203,7 +213,7 @@ stages: parameters: packageName: Microsoft.Data.SqlClient.Extensions.Abstractions artifactName: '${{ parameters.abstractionsArtifactsName }}' - packagePath: 'Microsoft.Data.SqlClient.Extensions.Abstractions.${{ parameters.abstractionsPackageVersion }}.nupkg' + packagePath: 'packages/Microsoft.Data.SqlClient.Extensions.Abstractions.${{ parameters.abstractionsPackageVersion }}.nupkg' nugetServiceConnection: ${{ variables.nugetServiceConnection }} isProduction: ${{ parameters.isOfficial }} displaySuffix: ${{ variables.nugetTargetSuffix }} @@ -213,7 +223,7 @@ stages: parameters: packageName: Microsoft.Data.SqlClient artifactName: '${{ parameters.sqlClientArtifactsName }}' - packagePath: 'Microsoft.Data.SqlClient.${{ parameters.sqlClientPackageVersion }}.nupkg' + packagePath: 'packages/Microsoft.Data.SqlClient.${{ parameters.sqlClientPackageVersion }}.nupkg' nugetServiceConnection: ${{ variables.nugetServiceConnection }} isProduction: ${{ parameters.isOfficial }} displaySuffix: ${{ variables.nugetTargetSuffix }} @@ -223,7 +233,7 @@ stages: parameters: packageName: Microsoft.Data.SqlClient.Extensions.Azure artifactName: '${{ parameters.azureArtifactsName }}' - packagePath: 'Microsoft.Data.SqlClient.Extensions.Azure.${{ parameters.azurePackageVersion }}.nupkg' + packagePath: 'packages/Microsoft.Data.SqlClient.Extensions.Azure.${{ parameters.azurePackageVersion }}.nupkg' nugetServiceConnection: ${{ variables.nugetServiceConnection }} isProduction: ${{ parameters.isOfficial }} displaySuffix: ${{ variables.nugetTargetSuffix }} @@ -233,7 +243,7 @@ stages: parameters: packageName: Microsoft.Data.SqlClient.AlwaysEncrypted.AzureKeyVaultProvider artifactName: '${{ parameters.akvProviderArtifactsName }}' - packagePath: 'Microsoft.Data.SqlClient.AlwaysEncrypted.AzureKeyVaultProvider.${{ parameters.akvProviderPackageVersion }}.nupkg' + packagePath: 'packages/Microsoft.Data.SqlClient.AlwaysEncrypted.AzureKeyVaultProvider.${{ parameters.akvProviderPackageVersion }}.nupkg' nugetServiceConnection: ${{ variables.nugetServiceConnection }} isProduction: ${{ parameters.isOfficial }} displaySuffix: ${{ variables.nugetTargetSuffix }} diff --git a/eng/pipelines/onebranch/steps/copy-apiscan-files-sqlclient-step.yml b/eng/pipelines/onebranch/steps/copy-apiscan-files-sqlclient-step.yml deleted file mode 100644 index 653eb40f5d..0000000000 --- a/eng/pipelines/onebranch/steps/copy-apiscan-files-sqlclient-step.yml +++ /dev/null @@ -1,39 +0,0 @@ -################################################################################# -# Licensed to the .NET Foundation under one or more agreements. # -# The .NET Foundation licenses this file to you under the MIT license. # -# See the LICENSE file in the project root for more information. # -################################################################################# - -# @TODO: This can be made more generic - -parameters: - # Path where dll files should be copied to for APIScan - - name: dllPath - type: string - - # Path where pdb files should be copied to for APIScan - - name: pdbPath - type: string - - - name: referenceType - type: string - values: - - Package - - Project - -steps: - - task: CopyFiles@2 - displayName: 'Copy DLLs for APIScan' - inputs: - contents: '**/Microsoft.Data.SqlClient.dll' - flattenFolders: false # Disabled to keep different frameworks and OS builds separate in the target - sourceFolder: '$(BUILD_OUTPUT)/Microsoft.Data.SqlClient/${{ parameters.referenceType }}-Release/' - targetFolder: '${{ parameters.dllPath }}/' - - - task: CopyFiles@2 - displayName: 'Copy PDBs for APIScan' - inputs: - contents: '**/Microsoft.Data.SqlClient.pdb' - flattenFolders: false # Disabled to keep different frameworks and OS builds separate in the target - sourceFolder: '$(BUILD_OUTPUT)/Microsoft.Data.SqlClient/${{ parameters.referenceType }}-Release/' - targetFolder: '${{ parameters.pdbPath }}/' diff --git a/eng/pipelines/onebranch/steps/esrp-dll-signing-step.yml b/eng/pipelines/onebranch/steps/esrp-dll-signing-step.yml index 073911e5fe..db95ef4f24 100644 --- a/eng/pipelines/onebranch/steps/esrp-dll-signing-step.yml +++ b/eng/pipelines/onebranch/steps/esrp-dll-signing-step.yml @@ -30,8 +30,9 @@ parameters: - name: esrpClientId type: string - # Globbing pattern for the files to sign. All files in $(BUILD_OUTPUT) - # that match this pattern will be scanned and signed. This should end with ".dll". + # Minimatch pattern(s) for the files to sign. All files in $(BUILD_OUTPUT) + # that match will be scanned and signed. Supports multi-line patterns and + # negation (!pattern). See https://aka.ms/esrp.signtask for details. - name: pattern type: string @@ -47,6 +48,7 @@ steps: EsrpClientId: '${{ parameters.esrpClientId }}' FolderPath: '$(BUILD_OUTPUT)' Pattern: '${{ parameters.pattern }}' + UseMinimatch: true UseMSIAuthentication: true VerboseLogin: 1 @@ -62,6 +64,7 @@ steps: EsrpClientId: '${{ parameters.esrpClientId }}' FolderPath: '$(BUILD_OUTPUT)' Pattern: '${{ parameters.pattern }}' + UseMinimatch: true signConfigType: inlineSignParams UseMSIAuthentication: true inlineOperation: | diff --git a/eng/pipelines/onebranch/steps/esrp-nuget-signing-step.yml b/eng/pipelines/onebranch/steps/esrp-nuget-signing-step.yml index bddbac517c..6a6682d40f 100644 --- a/eng/pipelines/onebranch/steps/esrp-nuget-signing-step.yml +++ b/eng/pipelines/onebranch/steps/esrp-nuget-signing-step.yml @@ -34,7 +34,8 @@ parameters: - name: searchPath type: string - # Globbing pattern to use to search for NuGet packages. If not provided, defaults to '*.*nupkg' + # Minimatch pattern(s) to search for NuGet packages. Supports multi-line + # patterns and negation (!pattern). Defaults to '*.*nupkg'. - name: searchPattern type: string default: '*.*nupkg' @@ -51,6 +52,7 @@ steps: EsrpClientId: '${{ parameters.esrpClientId }}' FolderPath: '${{ parameters.searchPath }}' Pattern: '${{ parameters.searchPattern }}' + UseMinimatch: true UseMSIAuthentication: true VerboseLogin: 1 @@ -66,6 +68,7 @@ steps: AuthSignCertName: '${{ parameters.authSignCertName }}' FolderPath: '${{ parameters.searchPath }}' Pattern: '${{ parameters.searchPattern }}' + UseMinimatch: true signConfigType: 'inlineSignParams' UseMSIAuthentication: true inlineOperation: | diff --git a/eng/pipelines/onebranch/steps/pack-csproj-step.yml b/eng/pipelines/onebranch/steps/pack-csproj-step.yml index edbec72c00..535106a9d1 100644 --- a/eng/pipelines/onebranch/steps/pack-csproj-step.yml +++ b/eng/pipelines/onebranch/steps/pack-csproj-step.yml @@ -28,8 +28,8 @@ steps: msbuildArguments: >- -t:${{ parameters.packTarget }} -p:ReferenceType=Package - -p:PackagesDir=$(PACK_OUTPUT)/ + -p:PackagesDir=$(JOB_OUTPUT)/packages/ ${{ parameters.versionProperties }} - - script: tree /a /f $(PACK_OUTPUT) + - script: tree /a /f $(JOB_OUTPUT)/packages displayName: List Pack Output Tree After Pack diff --git a/eng/pipelines/onebranch/steps/pack-sqlclient-step.yml b/eng/pipelines/onebranch/steps/pack-sqlclient-step.yml index 933c37f6a6..73229fe834 100644 --- a/eng/pipelines/onebranch/steps/pack-sqlclient-step.yml +++ b/eng/pipelines/onebranch/steps/pack-sqlclient-step.yml @@ -34,9 +34,9 @@ steps: displayName: Output Build Output Tree - task: CopyFiles@2 - displayName: 'Copy NuGet Packages to PACK_OUTPUT' + displayName: 'Copy NuGet Packages to JOB_OUTPUT' inputs: contents: '**/Microsoft.Data.SqlClient*.*nupkg' flattenFolders: true sourceFolder: '$(BUILD_OUTPUT)/Microsoft.Data.SqlClient/' - targetFolder: '$(PACK_OUTPUT)' + targetFolder: '$(JOB_OUTPUT)/packages' diff --git a/eng/pipelines/onebranch/steps/publish-symbols-step.yml b/eng/pipelines/onebranch/steps/publish-symbols-step.yml index 48ee30b9c2..d06b3c73e5 100644 --- a/eng/pipelines/onebranch/steps/publish-symbols-step.yml +++ b/eng/pipelines/onebranch/steps/publish-symbols-step.yml @@ -4,7 +4,23 @@ # See the LICENSE file in the project root for more information. # ################################################################################# -# For more details, see https://www.osgwiki.com/wiki/Symbols_Publishing_Pipeline_to_SymWeb_and_MSDL +# Symbols Publishing Pipeline +# =========================== +# Canonical docs: https://www.osgwiki.com/wiki/Symbols_Publishing_Pipeline_to_SymWeb_and_MSDL +# +# Publishing symbols to SymWeb (internal) and MSDL (public) is a two-step process: +# +# Step 1 - Upload: The PublishSymbols@2 task uploads PDB files to the Azure DevOps +# symbol store under an artifact name (SymbolsArtifactName). This stores the +# symbols but does NOT make them available on SymWeb or MSDL. +# +# Step 2 - Publish: The publish-symbols.ps1 script calls the Symbols Publishing Pipeline +# REST API to request that the previously uploaded symbols be published to the +# internal (SymWeb) and/or public (MSDL) symbol servers. +# +# IMPORTANT: Step 2 depends on Step 1. The ArtifactName parameter passed to the publish +# script MUST match the SymbolsArtifactName used by PublishSymbols@2 so that both steps +# reference the same uploaded artifact. parameters: # Name of the symbols artifact that will be published @@ -43,6 +59,13 @@ parameters: - name: searchPattern type: string + # Root folder to search for PDB files. When called from a build job this is typically + # $(BUILD_OUTPUT); when called from the dedicated symbols stage it points at the + # downloaded artifact path containing the PDBs. + - name: symbolsFolder + type: string + default: '$(BUILD_OUTPUT)' + # Account/org where the symbols will be uploaded - name: uploadAccount type: string @@ -60,8 +83,46 @@ steps: - script: 'echo ##vso[task.setvariable variable=ArtifactServices.Symbol.AccountName;]${{ parameters.uploadAccount }}' displayName: 'Set ArtifactServices.Symbol.AccountName to ${{ parameters.uploadAccount }}' + # Log the PDB files that match the search pattern so we can verify no + # unexpected files are included in the upload. + - pwsh: | + $folder = '${{ parameters.symbolsFolder }}' + $glob = '${{ parameters.searchPattern }}' + Write-Host "Symbols folder : $folder" + Write-Host "Search pattern : $glob" + + # Convert the glob to a regex that can match against relative paths. + # ** → match any number of path segments (.*?) + # * → match within a single segment ([^/\\]*) + # ? → match a single non-separator char ([^/\\]) + # . → literal dot + $regex = [regex]::Escape($glob) + $regex = $regex -replace '\\\*\\\*[/\\]?', '.*?' # ** or **/ + $regex = $regex -replace '\\\*', '[^/\\]*' # single * + $regex = $regex -replace '\\\?', '[^/\\]' # single ? + $regex = '^' + $regex + '$' + Write-Host "Regex : $regex" + Write-Host "" + + $allFiles = Get-ChildItem -Path $folder -Recurse -File + $matched = $allFiles | Where-Object { + $rel = $_.FullName.Substring($folder.Length).TrimStart('/\') + $rel -match $regex + } + + if (-not $matched) { + Write-Host "##vso[task.logissue type=warning]No PDB files matched pattern '$glob' under '$folder'" + } else { + $count = @($matched).Count + Write-Host "Matched $count PDB file(s):" + $matched | ForEach-Object { Write-Host " $($_.FullName)" } + } + displayName: 'Log PDBs matching ${{ parameters.searchPattern }}' + + # Step 1 - Upload: Push PDB files to the Azure DevOps symbol store. + # The SymbolsArtifactName set here is the key that links this upload to Step 2. - task: PublishSymbols@2 - displayName: 'Upload symbols to ${{ parameters.uploadAccount }} org' + displayName: 'Step 1: Upload symbols to ${{ parameters.uploadAccount }} org' inputs: IndexSources: false Pat: '$(System.AccessToken)' @@ -69,85 +130,25 @@ steps: SymbolExpirationInDays: 1825 # 5 years SymbolServerType: 'TeamServices' SymbolsArtifactName: '${{ parameters.artifactName }}' - SymbolsFolder: '$(BUILD_OUTPUT)' + SymbolsFolder: '${{ parameters.symbolsFolder }}' SymbolsMaximumWaitTime: 60 SymbolsProduct: '${{ parameters.packageName }}' SymbolsVersion: '${{ parameters.version }}' + # Step 2 - Publish: Request the Symbols Publishing Pipeline to publish the uploaded + # symbols to SymWeb and/or MSDL. The -ArtifactName argument must match the + # SymbolsArtifactName from Step 1. - task: AzureCLI@2 - displayName: 'Publish Symbols' + displayName: 'Step 2: Publish symbols to SymWeb/MSDL' inputs: azureSubscription: '${{ parameters.azureSubscription }}' - scriptLocation: inlineScript - scriptType: ps - inlineScript: | - # Propagate parameters to PS variables ################################################ - $artifactName = "${{ parameters.artifactName }}" - echo "artifactName= $artifactName" - - $publishProjectName = "${{ parameters.publishProjectName }}" - echo "publishProjectName= $publishProjectName" - - $publishToInternal = "${{ parameters.publishToInternal }}".ToLower() - echo "publishToInternal= $publishToInternal" - - $publishToPublic = "${{ parameters.publishToPublic }}".ToLower() - echo "publishToPublic= $publishToPublic" - - $publishServer = "${{ parameters.publishServer }}" - echo "publishServer= $publishServer" - - $publishTokenUri = "${{ parameters.publishTokenUri }}" - echo "publishTokenUri= $publishTokenUri" - - # Publish symbols ##################################################################### - # 1) Get the access token for the symbol publishing service - echo "> 1.Acquiring symbol publishing token..." - $symbolPublishingToken = az account get-access-token --resource $publishTokenUri --query accessToken -o tsv - echo "> 1.Symbol publishing token acquired." - - # 2) Register the request name - echo "> 2.Registering request name..." - $requestNameRegistrationBody = "{'requestName': '$artifactName'}" - Invoke-RestMethod ` - -Method POST ` - -Uri "https://$publishServer.trafficmanager.net/projects/$publishProjectName/requests" ` - -Headers @{ Authorization = "Bearer $symbolPublishingToken" } ` - -ContentType "application/json" ` - -Body $requestNameRegistrationBody - echo "> 2.Request name registered successfully." - - # 3) Publish the symbols - echo "> 3.Submitting request to publish symbols..." - $publishSymbolsBody = "{'publishToInternalServer': $publishToInternal, 'publishToPublicServer': $publishToPublic}" - Invoke-RestMethod ` - -Method POST ` - -Uri "https://$publishServer.trafficmanager.net/projects/$publishProjectName/requests/$artifactName" ` - -Headers @{ Authorization = "Bearer $symbolPublishingToken" } ` - -ContentType "application/json" ` - -Body $publishSymbolsBody - echo "> 3.Request to publish symbols submitted successfully." - - # The following REST calls are used to check publishing status. - echo "> 4.Checking the status of the request ..." - Invoke-RestMethod ` - -Method GET ` - -Uri "https://$publishServer.trafficmanager.net/projects/$publishProjectName/requests/$artifactName" ` - -Headers @{ Authorization = "Bearer $symbolPublishingToken" } ` - -ContentType "application/json" - - echo "Use below tables to interpret the values of xxxServerStatus and xxxServerResult fields from the response." - - echo "PublishingStatus" - echo "-----------------" - echo "0 NotRequested; The request has not been requested to publish." - echo "1 Submitted; The request is submitted to be published" - echo "2 Processing; The request is still being processed" - echo "3 Completed; The request has been completed processing. It can be failed or successful. Check PublishingResult to get more details" - - echo "PublishingResult" - echo "-----------------" - echo "0 Pending; The request has not completed or has not been requested." - echo "1 Succeeded; The request has published successfully" - echo "2 Failed; The request has failed to publish" - echo "3 Cancelled; The request was cancelled" + scriptType: pscore + scriptLocation: scriptPath + scriptPath: '$(Build.SourcesDirectory)/eng/pipelines/onebranch/scripts/publish-symbols.ps1' + arguments: >- + -PublishServer "${{ parameters.publishServer }}" + -PublishTokenUri "${{ parameters.publishTokenUri }}" + -PublishProjectName "${{ parameters.publishProjectName }}" + -ArtifactName "${{ parameters.artifactName }}" + -PublishToInternal $${{ parameters.publishToInternal }} + -PublishToPublic $${{ parameters.publishToPublic }} diff --git a/eng/pipelines/onebranch/variables/common-variables.yml b/eng/pipelines/onebranch/variables/common-variables.yml index a2a48ba184..3df1a6264f 100644 --- a/eng/pipelines/onebranch/variables/common-variables.yml +++ b/eng/pipelines/onebranch/variables/common-variables.yml @@ -23,13 +23,21 @@ variables: - name: BUILD_OUTPUT value: $(REPO_ROOT)/artifacts - # This is where downloaded artifacts should be placed. This will enable other build jobs to - # do local NuGet installations with packages built by earlier stages. - - name: PACK_INPUT + # Directory where downloaded pipeline artifacts (NuGet packages from earlier + # stages) are placed. Build jobs use this as a local NuGet package source so + # that downstream packages can resolve dependencies on packages built by + # upstream stages. + - name: JOB_INPUT value: $(REPO_ROOT)/packages - # This is where our C# projects place their NuGet package outputs. This is intentionally - # separated from packages/ (where downloaded pipeline artifacts go) so that ESRP signing and - # OneBranch artifact publishing only operate on packages built by the current job. - - name: PACK_OUTPUT + # Root directory for all job output artifacts (NuGet packages, symbols, etc.). + # OneBranch auto-publishes everything under this directory. It is intentionally + # separated from packages/ (where downloaded pipeline artifacts go) so that ESRP + # signing and OneBranch artifact publishing only operate on the current job's output. + # + # Sub-directory layout: + # assemblies/ - DLL assemblies for APIScan (preserving TFM folder structure) + # packages/ - NuGet packages (.nupkg, .snupkg) + # symbols/ - PDB symbol files (preserving TFM folder structure) + - name: JOB_OUTPUT value: $(REPO_ROOT)/output diff --git a/eng/pipelines/onebranch/variables/onebranch-variables.yml b/eng/pipelines/onebranch/variables/onebranch-variables.yml index 7b7c7597cb..b3bebb0a64 100644 --- a/eng/pipelines/onebranch/variables/onebranch-variables.yml +++ b/eng/pipelines/onebranch/variables/onebranch-variables.yml @@ -24,11 +24,11 @@ variables: # make it clear that these variables are used for symbols (as opposed to other msc tasks). # # SymbolsAzureSubscription - # SymbolsPublishProjectName - # SymbolsPublishServer - # SymbolsPublishTokenUri + # SymbolsPublishProjectNameSqlClient + # SymbolsPublishServerProd / SymbolsPublishServerPPE + # SymbolsPublishTokenUriProd / SymbolsPublishTokenUriPPE # SymbolsUploadAccount - - group: 'symbols-variables-v2' + - group: 'Symbols Publishing' # OneBranch Template Variables ###########################################