Skip to content

Reduce Apple mobile CI matrix #125441

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
kotlarmilos wants to merge 10 commits into
base: main
Choose a base branch
from
Open

Reduce Apple mobile CI matrix #125441

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
ce99125
Reduce Apple mobile CI matrix to alleviate macOS queue congestion
kotlarmilos Mar 11, 2026
25affcc
Rename build parameters for iOS/tvOS simulator and MacCatalyst config…
kotlarmilos Mar 12, 2026
f0439a4
Refactor MacCatalyst CI configurations and remove unnecessary project…
kotlarmilos Mar 12, 2026
0bfee88
Remove project exclusion for System.Formats.Cbor.Tests due to mysteri…
kotlarmilos Mar 12, 2026
8607492
Merge branch 'main' into reduce-apple-ci-matrix
kotlarmilos Mar 12, 2026
31bac4d
Add maccatalyst_x64 platform to MacCatalyst CI jobs
kotlarmilos Mar 12, 2026
0eb6958
Address review feedback: reorder entries, update coverage matrix
kotlarmilos Mar 12, 2026
bedaf5d
Merge remote-tracking branch 'origin/reduce-apple-ci-matrix' into red…
kotlarmilos Mar 12, 2026
caea0f5
Merge branch 'main' into reduce-apple-ci-matrix
kotlarmilos Mar 13, 2026
580f048
Add support for ArchiveTests in maccatalyst with native AOT runtime
kotlarmilos Mar 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 16 additions & 39 deletions eng/pipelines/extra-platforms/runtime-extra-platforms-ioslike.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,6 @@ parameters:

jobs:

#
# iOS/tvOS devices - Full AOT + AggressiveTrimming to reduce size
# Build the whole product using Mono and run libraries tests
#
- template: /eng/pipelines/common/platform-matrix.yml
parameters:
Expand All @@ -23,7 +20,6 @@ jobs:
isExtraPlatformsBuild: ${{ parameters.isExtraPlatformsBuild }}
isiOSLikeOnlyBuild: ${{ parameters.isiOSLikeOnlyBuild }}
platforms:
- ios_arm64
Copy link
Member

@matouskozak matouskozak Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm almost certain that we run more tests in extra-platforms on iOS-arm64 than we do in the runtime pipeline (smoky-only). Do you think that tvOS provides sufficient parity with iOS so we're fine with dropping this?

Copy link
Member Author

@kotlarmilos kotlarmilos Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In extra-platforms ios-arm64, the following test suites run: System.IO.Hashing.Tests, System.IO.MemoryMappedFiles.Tests, and System.Runtime.Numerics.Tests. These tests were previously crashing on tvOS. I assume this was an infrastructure issue rather than something platform-specific, so let's try enabling them on tvOS again.

- tvos_arm64
variables:
# map dependencies variables to local variables
Expand Down Expand Up @@ -51,9 +47,6 @@ jobs:
testRunNamePrefixSuffix: Mono_$(_BuildConfig)
extraHelixArguments: /p:NeedsToBuildAppsOnHelix=true

#
# iOS/tvOS devices
# Build the whole product using Mono and run runtime tests
#
- template: /eng/pipelines/common/platform-matrix.yml
parameters:
Expand All @@ -64,7 +57,6 @@ jobs:
isExtraPlatformsBuild: ${{ parameters.isExtraPlatformsBuild }}
isiOSLikeOnlyBuild: ${{ parameters.isiOSLikeOnlyBuild }}
platforms:
- ios_arm64
- tvos_arm64
variables:
- ${{ if and(eq(variables['System.TeamProject'], 'public'), eq(variables['Build.Reason'], 'PullRequest')) }}:
Expand Down Expand Up @@ -95,9 +87,6 @@ jobs:
testRunNamePrefixSuffix: Mono_$(_BuildConfig)
extraHelixArguments: /p:NeedsToBuildAppsOnHelix=true

#
# iOS/tvOS devices
# Build the whole product using Native AOT and run libraries tests
#
- template: /eng/pipelines/common/platform-matrix.yml
parameters:
Expand All @@ -108,7 +97,6 @@ jobs:
isExtraPlatformsBuild: ${{ parameters.isExtraPlatformsBuild }}
isiOSLikeOnlyBuild: ${{ parameters.isiOSLikeOnlyBuild }}
platforms:
- ios_arm64
- tvos_arm64
variables:
# map dependencies variables to local variables
Expand All @@ -129,46 +117,38 @@ jobs:
testRunNamePrefixSuffix: NativeAOT_$(_BuildConfig)
extraHelixArguments: /p:NeedsToBuildAppsOnHelix=true

#
# Build the whole product using NativeAOT for iOS/tvOS and run runtime tests with iOS/tvOS devices
#
- template: /eng/pipelines/common/platform-matrix.yml
parameters:
jobTemplate: /eng/pipelines/common/global-build-job.yml
helixQueuesTemplate: /eng/pipelines/coreclr/templates/helix-queues-setup.yml
helixQueuesTemplate: /eng/pipelines/libraries/helix-queues-setup.yml
buildConfig: Release
runtimeFlavor: coreclr
isExtraPlatformsBuild: ${{ parameters.isExtraPlatformsBuild }}
isiOSLikeOnlyBuild: ${{ parameters.isiOSLikeOnlyBuild }}
platforms:
- ios_arm64
- tvos_arm64
variables:
- name: timeoutPerTestInMinutes
value: 60
- name: timeoutPerTestCollectionInMinutes
value: 180
# map dependencies variables to local variables
- name: librariesContainsChange
value: $[ stageDependencies.EvaluatePaths.evaluate_paths.outputs['SetPathVars_libraries.containsChange'] ]
- name: coreclrContainsChange
value: $[ stageDependencies.EvaluatePaths.evaluate_paths.outputs['SetPathVars_coreclr.containsChange'] ]
- name: illinkContainsChange
value: $[ stageDependencies.EvaluatePaths.evaluate_paths.outputs['SetPathVars_tools_illink.containsChange'] ]
jobParameters:
testGroup: innerloop
nameSuffix: AllSubsets_NativeAOT_RuntimeTests
nameSuffix: AllSubsets_CoreCLR
buildArgs: -s clr+clr.runtime+libs+packs+libs.tests -c $(_BuildConfig) /p:ArchiveTests=true /p:DevTeamProvisioning=- /p:UseMonoRuntime=false /p:UseNativeAOTRuntime=false $(_runSmokeTestsOnlyArg) /p:BuildTestsOnHelix=true /p:EnableAdditionalTimezoneChecks=true /p:UsePortableRuntimePack=false /p:EnableAggressiveTrimming=true
timeoutInMinutes: 240
buildArgs: --cross -s clr.alljits+clr.tools+clr.nativeaotruntime+clr.nativeaotlibs+libs -c $(_BuildConfig)
# extra steps, run tests
extraVariablesTemplates:
- template: /eng/pipelines/common/templates/runtimes/test-variables.yml
parameters:
testGroup: innerloop
useNativeAOTRuntime: true
postBuildSteps:
- template: /eng/pipelines/common/templates/runtimes/build-runtime-tests-and-send-to-helix.yml
- template: /eng/pipelines/libraries/helix.yml
parameters:
creator: dotnet-bot
testBuildArgs: tree nativeaot/MobileSmokeTest /p:BuildNativeAOTRuntimePack=true
testRunNamePrefixSuffix: NativeAOT_$(_BuildConfig)
buildAllTestsAsStandalone: true
testRunNamePrefixSuffix: CoreCLR_$(_BuildConfig)
extraHelixArguments: /p:NeedsToBuildAppsOnHelix=true

#
# Build the whole product using CoreCLR R2R and run runtime tests
#
- template: /eng/pipelines/common/platform-matrix.yml
parameters:
Expand All @@ -179,7 +159,6 @@ jobs:
isExtraPlatformsBuild: ${{ parameters.isExtraPlatformsBuild }}
isiOSLikeOnlyBuild: ${{ parameters.isiOSLikeOnlyBuild }}
platforms:
- ios_arm64
- tvos_arm64
variables:
- ${{ if and(eq(variables['System.TeamProject'], 'public'), eq(variables['Build.Reason'], 'PullRequest')) }}:
Expand Down Expand Up @@ -210,14 +189,12 @@ jobs:
testRunNamePrefixSuffix: CoreCLR_$(_BuildConfig)
extraHelixArguments: /p:NeedsToBuildAppsOnHelix=true

#
# Build the whole product using CoreCLR R2R and run library tests
#
- template: /eng/pipelines/common/platform-matrix.yml
parameters:
jobTemplate: /eng/pipelines/common/global-build-job.yml
helixQueuesTemplate: /eng/pipelines/libraries/helix-queues-setup.yml
buildConfig: Release
buildConfig: Debug
runtimeFlavor: coreclr
isExtraPlatformsBuild: ${{ parameters.isExtraPlatformsBuild }}
isiOSLikeOnlyBuild: ${{ parameters.isiOSLikeOnlyBuild }}
Expand All @@ -234,8 +211,8 @@ jobs:
value: $[ stageDependencies.EvaluatePaths.evaluate_paths.outputs['SetPathVars_tools_illink.containsChange'] ]
jobParameters:
testGroup: innerloop
nameSuffix: AllSubsets_CoreCLR
buildArgs: -s clr+clr.runtime+libs+packs+libs.tests -c $(_BuildConfig) /p:ArchiveTests=true /p:DevTeamProvisioning=- /p:UseMonoRuntime=false /p:UseNativeAOTRuntime=false $(_runSmokeTestsOnlyArg) /p:BuildTestsOnHelix=true /p:EnableAdditionalTimezoneChecks=true /p:UsePortableRuntimePack=false /p:EnableAggressiveTrimming=true
nameSuffix: AllSubsets_CoreCLR_Smoke
buildArgs: -s clr+clr.runtime+libs+packs+libs.tests -c $(_BuildConfig) /p:ArchiveTests=true /p:DevTeamProvisioning=- /p:UseMonoRuntime=false /p:UseNativeAOTRuntime=false $(_runSmokeTestsOnlyArg) /p:BuildTestsOnHelix=true /p:UsePortableRuntimePack=false
timeoutInMinutes: 240
# extra steps, run tests
postBuildSteps:
Expand Down
96 changes: 49 additions & 47 deletions eng/pipelines/extra-platforms/runtime-extra-platforms-ioslikesimulator.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,22 +10,17 @@ parameters:

jobs:

#
# iOS/tvOS interp - requires AOT Compilation and Interp flags
# Build the whole product using Mono and run libraries tests
#
- template: /eng/pipelines/common/platform-matrix.yml
parameters:
jobTemplate: /eng/pipelines/common/global-build-job.yml
helixQueuesTemplate: /eng/pipelines/libraries/helix-queues-setup.yml
buildConfig: Release
buildConfig: Debug
runtimeFlavor: mono
isExtraPlatformsBuild: ${{ parameters.isExtraPlatformsBuild }}
isiOSLikeSimulatorOnlyBuild: ${{ parameters.isiOSLikeSimulatorOnlyBuild }}
platforms:
- iossimulator_x64
- iossimulator_arm64
- tvossimulator_arm64
variables:
# map dependencies variables to local variables
- name: librariesContainsChange
Expand All @@ -45,21 +40,18 @@ jobs:
interpreter: true
testRunNamePrefixSuffix: Mono_$(_BuildConfig)

#
# Build the whole product using Mono for iOSSimulator/tvOSSimulator and run runtime tests with iOS/tvOS simulators
#
- template: /eng/pipelines/common/platform-matrix.yml
parameters:
jobTemplate: /eng/pipelines/common/global-build-job.yml
helixQueuesTemplate: /eng/pipelines/coreclr/templates/helix-queues-setup.yml
buildConfig: Release
buildConfig: Debug
runtimeFlavor: mono
isExtraPlatformsBuild: ${{ parameters.isExtraPlatformsBuild }}
isiOSLikeOnlyBuild: ${{ parameters.isiOSLikeOnlyBuild }}
isiOSLikeSimulatorOnlyBuild: ${{ parameters.isiOSLikeSimulatorOnlyBuild }}
platforms:
- iossimulator_x64
- iossimulator_arm64
- tvossimulator_arm64
variables:
- ${{ if and(eq(variables['System.TeamProject'], 'public'), eq(variables['Build.Reason'], 'PullRequest')) }}:
- name: _HelixSource
Expand Down Expand Up @@ -89,66 +81,80 @@ jobs:
testRunNamePrefixSuffix: Mono_$(_BuildConfig)
extraHelixArguments: /p:NeedsToBuildAppsOnHelix=true

#
# Build the whole product using Native AOT for iOSSimulator/tvOSSimulator and run runtime tests with iOS/tvOS simulators
#
- template: /eng/pipelines/common/platform-matrix.yml
parameters:
jobTemplate: /eng/pipelines/common/global-build-job.yml
helixQueuesTemplate: /eng/pipelines/coreclr/templates/helix-queues-setup.yml
buildConfig: Release
helixQueuesTemplate: /eng/pipelines/libraries/helix-queues-setup.yml
buildConfig: Debug
runtimeFlavor: coreclr
isExtraPlatformsBuild: ${{ parameters.isExtraPlatformsBuild }}
isiOSLikeOnlyBuild: ${{ parameters.isiOSLikeOnlyBuild }}
isiOSLikeSimulatorOnlyBuild: ${{ parameters.isiOSLikeSimulatorOnlyBuild }}
platforms:
- iossimulator_x64
- iossimulator_arm64
- tvossimulator_arm64
variables:
- ${{ if and(eq(variables['System.TeamProject'], 'public'), eq(variables['Build.Reason'], 'PullRequest')) }}:
- name: _HelixSource
value: pr/dotnet/runtime/$(Build.SourceBranch)
- ${{ if and(eq(variables['System.TeamProject'], 'public'), ne(variables['Build.Reason'], 'PullRequest')) }}:
- name: _HelixSource
value: ci/dotnet/runtime/$(Build.SourceBranch)
- name: timeoutPerTestInMinutes
value: 60
- name: timeoutPerTestCollectionInMinutes
value: 180
# map dependencies variables to local variables
- name: librariesContainsChange
value: $[ stageDependencies.EvaluatePaths.evaluate_paths.outputs['SetPathVars_libraries.containsChange'] ]
- name: coreclrContainsChange
value: $[ stageDependencies.EvaluatePaths.evaluate_paths.outputs['SetPathVars_coreclr.containsChange'] ]
jobParameters:
testGroup: innerloop
nameSuffix: AllSubsets_NativeAOT_RuntimeTests
nameSuffix: AllSubsets_NativeAOT_Smoke
buildArgs: --cross -s clr.alljits+clr.tools+clr.nativeaotruntime+clr.nativeaotlibs+libs+libs.tests -c $(_BuildConfig) /p:ArchiveTests=true /p:RunSmokeTestsOnly=true /p:DevTeamProvisioning=- /p:BuildTestsOnHelix=true /p:UseNativeAOTRuntime=true /p:RunAOTCompilation=false /p:ContinuousIntegrationBuild=true
timeoutInMinutes: 180
buildArgs: --cross -s clr.alljits+clr.tools+clr.nativeaotruntime+clr.nativeaotlibs+libs -c $(_BuildConfig)
# extra steps, run tests
extraVariablesTemplates:
- template: /eng/pipelines/common/templates/runtimes/test-variables.yml
parameters:
testGroup: innerloop
useNativeAOTRuntime: true
postBuildSteps:
- template: /eng/pipelines/common/templates/runtimes/build-runtime-tests-and-send-to-helix.yml
- template: /eng/pipelines/libraries/helix.yml
parameters:
creator: dotnet-bot
testBuildArgs: tree nativeaot/MobileSmokeTest /p:BuildNativeAOTRuntimePack=true
testRunNamePrefixSuffix: NativeAOT_$(_BuildConfig)
buildAllTestsAsStandalone: true
extraHelixArguments: /p:NeedsToBuildAppsOnHelix=true

#
# Build the whole product using CoreCLR R2R and run runtime tests
- template: /eng/pipelines/common/platform-matrix.yml
parameters:
jobTemplate: /eng/pipelines/common/global-build-job.yml
helixQueuesTemplate: /eng/pipelines/libraries/helix-queues-setup.yml
buildConfig: Debug
runtimeFlavor: coreclr
isExtraPlatformsBuild: ${{ parameters.isExtraPlatformsBuild }}
isiOSLikeSimulatorOnlyBuild: ${{ parameters.isiOSLikeSimulatorOnlyBuild }}
platforms:
- iossimulator_x64
variables:
# map dependencies variables to local variables
- name: librariesContainsChange
value: $[ stageDependencies.EvaluatePaths.evaluate_paths.outputs['SetPathVars_libraries.containsChange'] ]
- name: coreclrContainsChange
value: $[ stageDependencies.EvaluatePaths.evaluate_paths.outputs['SetPathVars_coreclr.containsChange'] ]
- name: illinkContainsChange
value: $[ stageDependencies.EvaluatePaths.evaluate_paths.outputs['SetPathVars_tools_illink.containsChange'] ]
jobParameters:
testGroup: innerloop
nameSuffix: AllSubsets_CoreCLR
buildArgs: -s clr+clr.runtime+libs+packs+libs.tests -c $(_BuildConfig) /p:ArchiveTests=true $(_runSmokeTestsOnlyArg) /p:UseMonoRuntime=false /p:UseNativeAOTRuntime=false /p:BuildTestsOnHelix=true
timeoutInMinutes: 240
# extra steps, run tests
postBuildSteps:
- template: /eng/pipelines/libraries/helix.yml
parameters:
creator: dotnet-bot
testRunNamePrefixSuffix: CoreCLR_$(_BuildConfig)
extraHelixArguments: /p:NeedsToBuildAppsOnHelix=true

#
- template: /eng/pipelines/common/platform-matrix.yml
parameters:
jobTemplate: /eng/pipelines/common/global-build-job.yml
helixQueuesTemplate: /eng/pipelines/coreclr/templates/helix-queues-setup.yml
buildConfig: Release
buildConfig: Debug
runtimeFlavor: coreclr
isExtraPlatformsBuild: ${{ parameters.isExtraPlatformsBuild }}
isiOSLikeOnlyBuild: ${{ parameters.isiOSLikeOnlyBuild }}
isiOSLikeSimulatorOnlyBuild: ${{ parameters.isiOSLikeSimulatorOnlyBuild }}
platforms:
- iossimulator_x64
- iossimulator_arm64
- tvossimulator_arm64
variables:
- ${{ if and(eq(variables['System.TeamProject'], 'public'), eq(variables['Build.Reason'], 'PullRequest')) }}:
- name: _HelixSource
Expand Down Expand Up @@ -178,8 +184,6 @@ jobs:
testRunNamePrefixSuffix: CoreCLR_$(_BuildConfig)
extraHelixArguments: /p:NeedsToBuildAppsOnHelix=true

#
# Build the whole product using CoreCLR R2R and run libraries tests
#
- template: /eng/pipelines/common/platform-matrix.yml
parameters:
Expand All @@ -190,9 +194,7 @@ jobs:
isExtraPlatformsBuild: ${{ parameters.isExtraPlatformsBuild }}
isiOSLikeSimulatorOnlyBuild: ${{ parameters.isiOSLikeSimulatorOnlyBuild }}
platforms:
- iossimulator_x64
- iossimulator_arm64
- tvossimulator_arm64
variables:
# map dependencies variables to local variables
- name: librariesContainsChange
Expand All @@ -203,7 +205,7 @@ jobs:
value: $[ stageDependencies.EvaluatePaths.evaluate_paths.outputs['SetPathVars_tools_illink.containsChange'] ]
jobParameters:
testGroup: innerloop
nameSuffix: AllSubsets_CoreCLR
nameSuffix: AllSubsets_CoreCLR_Smoke
buildArgs: -s clr+clr.runtime+libs+packs+libs.tests -c $(_BuildConfig) /p:ArchiveTests=true $(_runSmokeTestsOnlyArg) /p:UseMonoRuntime=false /p:UseNativeAOTRuntime=false /p:BuildTestsOnHelix=true
timeoutInMinutes: 240
# extra steps, run tests
Expand Down
Loading
Loading