Fix flaky CtorIPAddressPair test. Fixes #25242#25308
Fix flaky CtorIPAddressPair test. Fixes #25242#25308rolfbjarne wants to merge 2 commits intomainfrom
Conversation
Two sources of flakiness:
1. Dns.GetHostAddresses("apple.com") can throw if DNS resolution
fails in CI. Wrap in try-catch and Assert.Ignore on failure.
2. The flags assertion for the remote address was too strict - it only
allowed Reachable + TransientConnection. Different OS versions can
report additional flags (e.g. ConnectionRequired). Use a new
CheckRemoteFlags helper that verifies Reachable is set and no
unexpected flags appear, matching the approach already used by
CheckLoopbackFlags.
Fixes #25242
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add NetworkResources.AppleHost constant instead of hardcoding 'apple.com' in the test. - Use TestRuntime.IgnoreInCIIfBadNetwork to only ignore transient DNS failures on CI (locally the test will still fail if DNS is down). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
✅ [PR Build #7e358c6] Build passed (Detect API changes) ✅Pipeline on Agent |
This comment has been minimized.
This comment has been minimized.
✅ [PR Build #7e358c6] Build passed (Build packages) ✅Pipeline on Agent |
✅ API diff for current PR / commitNET (empty diffs)✅ API diff vs stableNET (empty diffs)ℹ️ Generator diffGenerator Diff: vsdrops (html) vsdrops (raw diff) gist (raw diff) - Please review changes) Pipeline on Agent |
✅ [PR Build #7e358c6] Build passed (Build macOS tests) ✅Pipeline on Agent |
🔥 [CI Build #7e358c6] Test results 🔥Test results❌ Tests failed on VSTS: test results 0 tests crashed, 26 tests failed, 143 tests passed. Failures❌ monotouch tests (iOS)13 tests failed, 0 tests passed.Failed tests
Html Report (VSDrops) Download ❌ monotouch tests (tvOS)13 tests failed, 0 tests passed.Failed tests
Html Report (VSDrops) Download Successes✅ cecil: All 1 tests passed. Html Report (VSDrops) Download macOS tests✅ Tests on macOS Monterey (12): All 5 tests passed. Html Report (VSDrops) Download Linux Build VerificationPipeline on Agent |
2 participants
Fix the flaky
NetworkReachabilityTest.CtorIPAddressPairtest (monotouch-test, macOS).Two sources of flakiness addressed:
DNS resolution failure:
Dns.GetHostAddresses("apple.com")can throw when DNS is unavailable in CI. Wrapped in try-catch withTestRuntime.IgnoreInCIIfBadNetworkso transient DNS failures are only ignored on CI (locally the test still fails).Overly strict flags assertion: The remote-address flags check required exactly
Reachable(after strippingTransientConnection). Different OS versions can report additional flags likeConnectionRequired. Added aCheckRemoteFlagshelper that verifiesReachableis set and no unexpected flags appear — matching the existingCheckLoopbackFlagsphilosophy.Other changes:
NetworkResources.AppleHostconstant instead of hardcoding"apple.com"in the test.Fixes #25242
🤖 Pull request created by Copilot