[release/10.0.1xx] [tests] Update max simulator OS version.#24885
[release/10.0.1xx] [tests] Update max simulator OS version.#24885rolfbjarne wants to merge 8 commits intorelease/10.0.1xxfrom
Conversation
This fixes the following problem: * We default the max simulator version to the SDK version of an OS (currently 26.2) * However, iOS/tvOS 26.3 exists, and that's the simulator 'xcodebuild -downloadPlatform' installs with Xcode 26.3. * The max simulator version is what we use in xharness to decide which simulator to run tests in. * The result is that xharness will try to use/create a simulator for iOS 26.2, but that might not be possible (unless said simulator was already installed from before the current build), because it wasn't provisioned.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… iOS/tvOS 26.3 Replace xamvideotest.mp4 with a freshly-generated, well-formed H.264 Baseline video. The old video (created in 2015 by Mainconcept encoder) had non-standard dimensions (438x434, not multiples of 16) and produced codecBadDataErr (-8969) on iOS/tvOS 26.4 simulators where the decoder is stricter. The new video uses standard dimensions (320x240), H.264 Constrained Baseline profile, and is generated with ffmpeg to ensure well-formed NAL units. Also add WaitForAsynchronousFrames() after FinishDelayedFrames() in all async decode tests. FinishDelayedFrames() directs the session to emit all delayed frames, but WaitForAsynchronousFrames() is needed to block until all async callbacks have completed before checking assertions. Fixes #24844 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This backport updates simulator OS selection logic so XHarness runs tests on the newest available simulator runtime when Xcode installs a newer simulator than the SDK version (notably with Xcode 26.3).
Changes:
- Bump max iOS simulator version used by XHarness to 26.3 and add Make.config logic to special-case Xcode 26.3 simulator selection.
- Extend Xcode version checks in tests to recognize Xcode 26.3.
- Adjust CoreGraphics
CGFontcreation semantics/docs and update/add tests/resources (incl.xamvideotest.mp4) to support new test coverage.
Reviewed changes
Copilot reviewed 6 out of 7 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/common/SdkVersions.cs | Updates XHarness max iOS simulator version constant to 26.3. |
| Make.config | Special-cases Xcode 26.3 to set max simulator OS versions independently of SDK version. |
| tests/common/TestRuntime.cs | Adds Xcode 26.3 mapping to OS version checks for iOS/tvOS/macOS test gating. |
| src/CoreGraphics/CGFont.cs | Makes font creation return null on native failure and updates XML docs; switches to TransientCFString. |
| tests/monotouch-test/VideoToolbox/VTDecompressionSessionTests.cs | Ensures async decode work completes by waiting for asynchronous frames. |
| tests/monotouch-test/CoreText/FontTest.cs | Makes emoji-font tests resilient to missing font and loosens a bounds assertion. |
| tests/monotouch-test/xamvideotest.mp4 | Adds missing video asset used by multiple AVFoundation/VideoToolbox tests. |
You can also share your feedback on Copilot code review. Take the survey.
| } finally { | ||
| CFString.ReleaseNative (nameHandle); | ||
| } | ||
| var nameHandle = new TransientCFString (name); |
There was a problem hiding this comment.
TransientCFString is a ref struct that allocates a native CFString handle and must be disposed. Here it's created without using, so the native string handle is never released, causing a leak each time CreateWithFontName is called. Use using var nameHandle = new TransientCFString (name); (or otherwise ensure Dispose is called) before invoking CGFontCreateWithFontName.
| var nameHandle = new TransientCFString (name); | |
| using var nameHandle = new TransientCFString (name); |
| // Copyright 2015 Xamarin Inc. All rights reserved. | ||
| // | ||
|
|
||
| using System.Linq; |
There was a problem hiding this comment.
The newly added using System.Linq; doesn't appear to be used anywhere in this file. Please remove it to avoid unnecessary usings (and potential warnings in builds that treat warnings as errors).
| using System.Linq; |
| public const string MaxiOSSimulator = "26.3"; | ||
| public const string MaxTVOSSimulator = "26.2"; |
There was a problem hiding this comment.
The PR description focuses on updating the max simulator OS version for XHarness, but this PR also includes unrelated changes (CoreGraphics CGFont behavior/doc updates, adding a new xamvideotest.mp4 resource, and multiple test changes). Please either update the PR description to reflect the full scope, or split these into separate PRs to make the backport easier to review and risk-assess.
| $(error Check if a max iOS simulator is different than the current SDK version (this happens when there's a new iOS release, with no API changes so no new SDK)) | ||
| else ifneq ($(XCODE_VERSION),$(TVOS_SDK_VERSION)) | ||
| $(error Check if a max tvOS simulator is different than the current SDK version (this happens when there's a new tvOS release, with no API changes so no new SDK)) |
There was a problem hiding this comment.
The new $(error ...) messages are a bit hard to parse/grammatically incorrect ("Check if a max iOS simulator is different..."). Consider rephrasing to explicitly mention "max iOS simulator version" / "max tvOS simulator version" and what action is required (update the hardcoded Xcode version + assign max simulator versions). Clearer error text will make future Xcode bumps easier to handle.
| $(error Check if a max iOS simulator is different than the current SDK version (this happens when there's a new iOS release, with no API changes so no new SDK)) | |
| else ifneq ($(XCODE_VERSION),$(TVOS_SDK_VERSION)) | |
| $(error Check if a max tvOS simulator is different than the current SDK version (this happens when there's a new tvOS release, with no API changes so no new SDK)) | |
| $(error The max iOS simulator version appears to differ from the current iOS SDK version. If this is expected (for example, a new iOS release without API changes), update the hardcoded Xcode version check above and assign MAX_IOS_SIMULATOR_VERSION and MAX_TVOS_SIMULATOR_VERSION accordingly.) | |
| else ifneq ($(XCODE_VERSION),$(TVOS_SDK_VERSION)) | |
| $(error The max tvOS simulator version appears to differ from the current tvOS SDK version. If this is expected (for example, a new tvOS release without API changes), update the hardcoded Xcode version check above and assign MAX_IOS_SIMULATOR_VERSION and MAX_TVOS_SIMULATOR_VERSION accordingly.) |
✅ [CI Build #deb809c] Build passed (Build packages) ✅Pipeline on Agent |
✅ [PR Build #deb809c] Build passed (Detect API changes) ✅Pipeline on Agent |
✅ API diff for current PR / commitNET (empty diffs)✅ API diff vs stableNET (empty diffs)ℹ️ Generator diffGenerator Diff: vsdrops (html) vsdrops (raw diff) gist (raw diff) - Please review changes) Pipeline on Agent |
✅ [CI Build #deb809c] Build passed (Build macOS tests) ✅Pipeline on Agent |
💻 [CI Build #deb809c] Tests on macOS X64 - Mac Sonoma (14) passed 💻✅ All tests on macOS X64 - Mac Sonoma (14) passed. Pipeline on Agent |
💻 [CI Build #deb809c] Tests on macOS M1 - Mac Monterey (12) passed 💻✅ All tests on macOS M1 - Mac Monterey (12) passed. Pipeline on Agent |
💻 [CI Build #deb809c] Tests on macOS M1 - Mac Ventura (13) passed 💻✅ All tests on macOS M1 - Mac Ventura (13) passed. Pipeline on Agent |
💻 [CI Build #deb809c] Tests on macOS arm64 - Mac Sequoia (15) passed 💻✅ All tests on macOS arm64 - Mac Sequoia (15) passed. Pipeline on Agent |
💻 [CI Build #deb809c] Tests on macOS arm64 - Mac Tahoe (26) passed 💻✅ All tests on macOS arm64 - Mac Tahoe (26) passed. Pipeline on Agent |
This comment has been minimized.
This comment has been minimized.
3 similar comments
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
🔥 [CI Build #deb809c] Test results 🔥Test results❌ Tests failed on VSTS: test results 19 tests crashed, 0 tests failed, 3 tests passed. Failures❌ cecil tests🔥 Failed catastrophically on VSTS: test results - cecil (no summary found). Html Report (VSDrops) Download ❌ dotnettests tests (iOS)🔥 Failed catastrophically on VSTS: test results - dotnettests_ios (no summary found). Html Report (VSDrops) Download ❌ dotnettests tests (MacCatalyst)🔥 Failed catastrophically on VSTS: test results - dotnettests_maccatalyst (no summary found). Html Report (VSDrops) Download ❌ dotnettests tests (macOS)🔥 Failed catastrophically on VSTS: test results - dotnettests_macos (no summary found). Html Report (VSDrops) Download ❌ dotnettests tests (Multiple platforms)🔥 Failed catastrophically on VSTS: test results - dotnettests_multiple (no summary found). Html Report (VSDrops) Download ❌ dotnettests tests (tvOS)🔥 Failed catastrophically on VSTS: test results - dotnettests_tvos (no summary found). Html Report (VSDrops) Download ❌ framework tests🔥 Failed catastrophically on VSTS: test results - framework (no summary found). Html Report (VSDrops) Download ❌ fsharp tests🔥 Failed catastrophically on VSTS: test results - fsharp (no summary found). Html Report (VSDrops) Download ❌ generator tests🔥 Failed catastrophically on VSTS: test results - generator (no summary found). Html Report (VSDrops) Download ❌ interdependent-binding-projects tests🔥 Failed catastrophically on VSTS: test results - interdependent-binding-projects (no summary found). Html Report (VSDrops) Download ❌ introspection tests🔥 Failed catastrophically on VSTS: test results - introspection (no summary found). Html Report (VSDrops) Download ❌ linker tests🔥 Failed catastrophically on VSTS: test results - linker (no summary found). Html Report (VSDrops) Download ❌ monotouch tests (iOS)🔥 Failed catastrophically on VSTS: test results - monotouch_ios (no summary found). Html Report (VSDrops) Download ❌ monotouch tests (MacCatalyst)🔥 Failed catastrophically on VSTS: test results - monotouch_maccatalyst (no summary found). Html Report (VSDrops) Download ❌ monotouch tests (macOS)🔥 Failed catastrophically on VSTS: test results - monotouch_macos (no summary found). Html Report (VSDrops) Download ❌ monotouch tests (tvOS)🔥 Failed catastrophically on VSTS: test results - monotouch_tvos (no summary found). Html Report (VSDrops) Download ❌ msbuild tests🔥 Failed catastrophically on VSTS: test results - msbuild (no summary found). Html Report (VSDrops) Download ❌ xcframework tests🔥 Failed catastrophically on VSTS: test results - xcframework (no summary found). Html Report (VSDrops) Download ❌ xtro tests🔥 Failed catastrophically on VSTS: test results - xtro (no summary found). Html Report (VSDrops) Download Successes✅ windows: All 3 tests passed. Html Report (VSDrops) Download Pipeline on Agent |
|
Incorporated into #24896. |
Pull request was closed
This fixes the following problem:
Backport of #24867.