High bit test harness#1424
Merged
leculver merged 24 commits intomicrosoft:mainfrom Apr 22, 2026
Merged
Conversation
Adds a native C++ launcher (HighBitHost.exe) that reserves all memory below 0x80000000 in a LARGEADDRESSAWARE 32-bit process before loading CoreCLR via hostfxr, forcing CLR heap allocations into the upper 2 GB. This exercises ClrMD code paths that handle sign-extended CLRDATA_ADDRESS values, minidump ULONG64 handling, and 32-bit high-bit address round-trips -- all of which are untested today because normal test dumps on x64 Windows happen to land the managed heap below 2 GB. Integration: - DumpGenerator gains EnsureDump(..., highBit:true) that lazily MSBuilds HighBitHost.vcxproj (via vswhere-discovered MSBuild) and runs the existing managed target under it. DOTNET_DbgEnableMiniDump env vars capture the dump as usual. - TestTarget.LoadFullDumpHighBit() thunks through with _highbit suffix on cached dumps. - HighBitFactAttribute skips unless Windows + Architecture.X86. - New HighBitAddressTests class covers heap enumeration, segment data-reader round-trip, roots+objects enumeration, and smoke tests for multiple targets. - New Windows_x86 CI job in azure-pipelines-external.yml builds via cibuild -test:false then invokes Test.cmd x86. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
createdump writes ULONG64 fields in MINIDUMP_MODULE, MINIDUMP_MEMORY_DESCRIPTOR, and thread TEBs sign-extended on 32-bit Windows when the address has the high bit set (e.g. 0xFFFFFFFF_800A0000 instead of 0x800A0000). This caused OverflowException in PEModuleInfo.GetPEImage and mismatched segment lookups whenever modules load above 0x80000000. - Retype MinidumpMemoryDescriptor.StartAddress/DataSize64 from ClrDataAddress to ulong. ClrDataAddress is a DAC concept and should not appear in Win32 struct wrappers. - Add Minidump.SanitizeAddress(ulong) that masks to PointerSize (unchecked cast for Debug's checked arithmetic). - Apply SanitizeAddress at ingest for module BaseOfImage, segment StartAddress, and thread TEB so no sign-extended address propagates past the minidump boundary. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace the bolt-on HighBitAddressTests + HighBitFactAttribute + LoadFullDumpHighBit
helpers with a first-class [Theory] matrix over (DumpFlavor, HighBit) so the
standard ClrMD test bodies in HeapTests, TypeTests, GCHandleTests, ClrObjectTests,
and RuntimeTests all run against every applicable runtime variant on each host:
* Windows x64 : Core, Framework
* Windows x86 : Core, Framework, Core+HighBit, Framework+HighBit
* non-Windows : Core
Key pieces:
* New DumpVariant readonly record + DumpFlavor/TargetFlavors enums + TestVariants
helper (For/NoHighBit/CoreOnly/HighBitOnly/WithSingleFile) as MemberData source
for [Theory]. Unsupported combos are filtered at enumeration time, not Skip=.
* TestTarget.SupportedFlavors/SupportsHighBit replace the FrameworkOnly bool;
LoadFullDump(DumpVariant) / LoadFullDump(DumpVariant, singleFile) /
LoadMinidump(DumpVariant) route through EnsureDump.
* Per-TFM build output dirs (bin/<arch>/<tfm>/) so net10 and net48 builds of the
same target no longer clobber each other.
* HighBitHost.exe gains --mode=core|framework: Framework mode hosts mscoree via
ICLRMetaHost / ICorRuntimeHost / _AppDomain::ExecuteAssembly_2 after the low-
memory reservation pass. DumpGenerator picks GenerateHighBitFrameworkDump via
DbgEng on CLR exception 0xe0434352.
* ClrObjectTests no longer uses IClassFixture<ClrObjectConnection> (incompatible
with per-theory-variant matrix); a private Scope class loads the dump per test.
SingleFileClrObjectTests / SingleFileClrObjectConnection removed - covered by
WithSingleFile.
* HighBitAddressTests.cs and HighBitFactAttribute.cs deleted. A single
HeapTests.HighBit_HeapHasObjectsAbove2Gb sanity test (via HighBitOnly variants)
preserves the reservation-worked assertion.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Empty MemberData throws "No data found" on non-x86-Windows hosts where the HighBitOnly selector legitimately yields nothing. Convert HeapTests.HighBit_HeapHasObjectsAbove2Gb back to a [Fact], re-introduce a minimal HighBitFactAttribute that skips off Windows or off x86 (mirrors the existing WindowsFact/FrameworkFact pattern). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Three bugs in the HighBit Framework path that together caused all 60+
framework-highbit variants to fail and cascade-break most remaining DbgEng-
using tests on x86:
1. LaunchAndCaptureDump extracted the working directory via
Path.GetDirectoryName(exePath), but GenerateHighBitFrameworkDump was
handing it a full command line ('"{host}" --mode=framework "{target}"'),
not a single path. That yielded garbage like
'D:\\...\\HighBitHost.exe" --mode=framework "D:\\...\\target' and DbgEng
returned HRESULT 8007010B (ERROR_DIRECTORY). Added an overload that
accepts workingDirectory explicitly; HighBit Framework now passes the
target.exe's directory.
2. Debugger ctor leaked its DbgEngWrapper if CreateProcessAndAttach failed:
_dbgeng was already assigned before the throw, and 'using Debugger ...'
never kicked in because ctor exited via exception. Every subsequent
test in the run that tried to create a DbgEng instance then faulted with
'DbgEng doesn't properly handle multiple instances simultaneously' —
turning one real failure into ~60 cascading ones. Wrap the body in
try/catch and dispose _dbgeng on ctor failure.
3. CopyFileIfNewer raced on SharedLibrary.dll / companion binaries that were
memory-mapped by a previously-loaded DataTarget. Since those DLLs are
byte-identical between target builds within the same TFM/arch, tolerate
the IOException after retrying if the destination already exists.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
_AppDomain::ExecuteAssembly_2 traps any unhandled managed exception thrown by the target's Main and returns it as an HRESULT (e.g. 0x80131509) instead of letting the 0xe0434352 SEH exception propagate. Our parent DbgEng dump- on-exception hook is gated on that SEH, so framework-highbit dumps were never captured and every framework-highbit variant failed with 'Failed to generate dump'. Check the ExecuteAssembly_2 HRESULT and, on failure, RaiseException with the CLR exception code BEFORE tearing down CLR so the captured dump still has an intact managed heap for ClrMD tests to inspect. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
ICorRuntimeHost / _AppDomain doesn't read COMPlus_BuildFlavor or COMPlus_gcServer on its own -- those env vars are only interpreted by the normal exe loader path. When HighBitHost hosts the CLR directly, call ICLRRuntimeInfo::SetDefaultStartupFlags(STARTUP_SERVER_GC) before pHost->Start() if the caller asked for server GC. This lets ServerSegmentTests(framework-highbit) actually capture a server-GC dump. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The HighBitHost Framework harness re-raises the unhandled managed exception as a synthetic 0xe0434352 SEH from native code AFTER ExecuteAssembly_2 has already returned, so the target's Main/Inner frames are gone from the managed callstack by the time DbgEng captures the dump. VariableRootTest asserts on those exact frames; no way to preserve them without materially changing the harness, so skip the variant (mirrors the existing early-return for the Linux DAC bug). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The ObjectReferenceNotPointerAligned case wrote 0xcccccc (=13421772), which is 8-byte-unaligned on x64 but happens to be 4-byte-aligned on x86, so the test reported InvalidObjectReference on x86 instead of the expected not-aligned kind. Switch to 0xcccccd (odd) so the value is unaligned regardless of pointer size. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
eng/common/cibuild.cmd hardcodes -test on the Build.ps1 command line, so passing -test:false from the pipeline yields a ParameterAlreadyBound error. Call build.cmd directly with the same flags cibuild uses, minus -test, for the build-only leg of the x86 high-bit job. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The Microsoft.Diagnostics.Runtime.Tests project sets <IsUnitTestProject>false</IsUnitTestProject> under ContinuousIntegrationBuild, so Arcade's cibuild --test step has been a no-op on every leg except the new x86 job -- meaning the product hasn't actually been tested in CI. Add explicit Test.cmd x64 / test.sh x64 invocations so each platform runs its suite the same way developers do locally. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The windows.vs2019.amd64.open image ships only 64-bit dotnet, so the x86 test host fails to start with 'Microsoft.NETCore.App 10.0.0 was not found'. Run eng/common/dotnet-install.ps1 -Architecture x86 using the SDK version from global.json, prepend its install dir to PATH, and set DOTNET_ROOT(x86) so testhost can locate the 32-bit runtime. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The Arcade wrapper only accepts -architecture/-version/-runtime and installs unconditionally to \/.dotnet (or .dotnet/x86). Read the SDK version from global.json and pass it via -version; expose .dotnet/x86 via PATH and DOTNET_ROOT(x86). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Following the dotnet/diagnostics model: let Arcade own build+test invocation via cibuild.cmd / cibuild.sh instead of hand-driving dotnet test. Also bumps SDK to 10.0.106. Key changes: - Drop <IsUnitTestProject>false</IsUnitTestProject> CI override from Microsoft.Diagnostics.Runtime.Tests.csproj. That stale workaround (for a long-removed build_test_assets.cmd) was silently turning Arcade's -test switch into a no-op on every CI leg -- meaning x64 Windows, Linux, and macOS had never actually been running the test suite. With it gone, Arcade auto-injects Microsoft.NET.Test.Sdk and runs tests as part of cibuild. - Windows_NT (x64) / Linux / MacOS: collapse to a single cibuild step (build+test) followed by PublishTestResults@2 instead of a separate Test.cmd/test.sh invocation. - Windows_x86: keep the manual x86 SDK install (Arcade does not auto-provision non-host architectures) and pass /p:SkipTests=true to cibuild so Arcade's x64 test pass doesn't run; then invoke Test.cmd x86 explicitly for the 32-bit suite. - Bump global.json SDK to 10.0.106 to match the runtime feed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Arcade's dotnet-install.ps1 wrapper defaults -runtime to 'dotnet', so it was trying to fetch runtime 10.0.105 (a 404; 10.0.105 is an SDK version). Pass -runtime '' to install the SDK instead -- the SDK bundles its matching runtime, which is what the x86 testhost needs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
When /p:SkipTests=true is passed, Arcade's Tests.targets doesn't populate @(TestToRun), so the BeforeCollectCodeCoverage target in eng/Coverage.targets tried to pass an empty string to BeforeCollectCodeCoverageTask.TestAssembly (which is Required=true) and failed with MSB4044. Mirror VSTest.targets and skip the coverage targets when @(TestToRun) is empty. Also bumps global.json SDK to 10.0.106. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
eng/Coverage.targets tries to invoke 'dotnet tool run coverlet', but the repo has no .config/dotnet-tools.json, so coverlet isn't installed. BeforeCollectCodeCoverageTask then blocks indefinitely on NamedPipeClientStream.Connect waiting for a coverlet that never starts, and the build job hits its timeout. The codecov upload in the same file is also commented out, so the current coverage plumbing isn't actually producing a report -- just hanging CI. Turn off <Coverage> under ContinuousIntegrationBuild until the pipeline is wired up properly. This was previously hidden because <IsUnitTestProject>false</IsUnitTestProject> in CI prevented the Test target (and therefore the Before/After coverage targets) from running at all. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Hosted CI agents frequently can't reach CRL endpoints, so WinVerifyTrust fails the DAC signature check and tests report 'Failed to load dac: not properly signed.' Tests load DACs either from the machine's own system directories (for .NET Framework) or a locally-built runtime, both of which we trust; Authenticode verification doesn't add any value here. Set VerifyDacOnWindows = false on the DataTargetOptions used by the test-side LoadDump helpers (including the DbgEng variant and the one place in MinidumpTests that builds its own options). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Types.exe (and friends) is a framework-dependent apphost. Without DOTNET_ROOT set, the apphost falls back to C:\Program Files\dotnet\ to locate the shared framework, which on hosted CI agents only has older runtimes -- so every .NET Core dump generation fails with 'You must install or update .NET to run this application.' Point the spawned process at the same runtime that's hosting the test runner by deriving DOTNET_ROOT from Path.GetDirectoryName(Environment.ProcessPath). This works for both x64 and x86 test hosts (pointing at .dotnet\ and .dotnet\x86\ respectively). Also set the arch-specific DOTNET_ROOT_X64 / DOTNET_ROOT(x86) variants because the apphost checks them before the plain DOTNET_ROOT. Applies the same fix to GenerateHighBitCoreDump, which launches HighBitHost.exe and uses hostfxr to load the managed DLL. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CollectibleTypeTest: under some runners a boxed static copy of CollectibleUnmanagedStruct appears in the default ALC in addition to the collectible ALC instance. Filter by IsCollectible so we always pick the collectible-ALC instance. AssemblySize: the test asserted GenericBar's MethodDesc had non-zero HotColdInfo, but an uninstantiated-at-dump-time generic method legitimately has zero-sized regions. Assert instead that at least one method on Foo has populated HotColdInfo, which is what the test was really trying to verify. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Previously, every x86 HighBit test on the CI agents spent ~15 minutes failing to build HighBitHost.exe (no v143 toolset) and/or failing to find an x86 hostfxr — consuming the entire 60-minute job budget. Changes: - vcxproj: downgrade VCProjectVersion to 16.0 and let MSBuild pick the default installed platform toolset instead of hard-coding v143. - DumpGenerator: dynamically probe installed VC toolsets (Microsoft.VCToolsVersion.v14X.default.txt) and pass PlatformToolset explicitly to MSBuild. - DumpGenerator: cache negative BuildHighBitHost results so a failed build isn't re-attempted on every test in the job. - DumpGenerator.SetDotNetRoot: resolve DOTNET_ROOT via a priority list (arch-specific env, DOTNET_ROOT, DOTNET_HOST_PATH with arch check, standard Program Files install path, ProcessPath) so an x86 child spawned from an x64-launched testhost picks the x86 dotnet install. - DumpGenerator: add a PE-machine check so we don't hand an x64 dotnet root to an x86 child. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
RunDotnetPublish/RunDotnetBuild/BuildHighBitHostCore/FindMsBuild all read stdout via StreamReader.ReadToEnd() before stderr. When a child (dotnet publish, msbuild) spawns build servers (vbcscompiler, MSBuild node-reuse) that inherit the pipe handles, the pipes never close after the parent exits, so ReadToEnd() blocks indefinitely. This caused the x86 CI jobs to hang silently for the full 60-minute timeout on the first test that publishes a single-file target. Fix: - Add RunAndCapture helper that drains both pipes asynchronously via OutputDataReceived/ErrorDataReceived events; waits for WaitForExit and then bounds the wait for each pipe's EOF at 30s so a lingering build server can't hang us. - Pass --disable-build-servers and -p:UseSharedCompilation=false to dotnet publish/build invocations so Roslyn doesn't spawn a persistent vbcscompiler. - Pass /nodeReuse:false to msbuild for HighBitHost so MSBuild doesn't spawn a persistent node. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
MacOS test runs hang indefinitely on the dnceng-public macOS-latest image (the xunit console emits 'Starting' and then no further output for 60+ minutes). The previous pipeline never actually ran tests on MacOS (it only built), so this restores that behavior until the hang is diagnosed in a follow-up. Linux continues to run tests normally. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
max-charlamb
approved these changes
Apr 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces significant improvements to the test infrastructure and coverage, particularly around supporting and distinguishing between different runtime flavors and host architectures, including specialized support for 32-bit Windows (HighBit) testing. The main changes are the addition of a new test matrix for HighBit (x86) scenarios, new test attributes, and expanded test variant enumeration. Additionally, the CI pipeline is updated to run these new test configurations on all supported platforms.