Implement cDAC memory region APIs (GetHandleTableMemoryRegions, GetGCBookkeepingMemoryRegions, GetGCFreeRegions)#124926
Implement cDAC memory region APIs (GetHandleTableMemoryRegions, GetGCBookkeepingMemoryRegions, GetGCFreeRegions)#124926
Conversation
… for memory region types Co-authored-by: max-charlamb <44248479+max-charlamb@users.noreply.github.com>
…p free region fields, and IGC contract methods Co-authored-by: max-charlamb <44248479+max-charlamb@users.noreply.github.com>
…Impl integration Co-authored-by: max-charlamb <44248479+max-charlamb@users.noreply.github.com>
Add GCMemoryRegionTests with three test methods covering: - GetHandleTableMemoryRegions: single map, bucket, and segment - GetGCBookkeepingMemoryRegions: single card table entry - GetGCFreeRegions: workstation GC with one free region All tests use [Theory] with [ClassData(typeof(MockTarget.StdArch))] to run across all four architecture combinations. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
@copilot , we need to update the cDAC docs and write tests for these new APIs |
Co-authored-by: max-charlamb <44248479+max-charlamb@users.noreply.github.com>
|
@copilot try again |
…eeping entries, empty/multiple free regions Co-authored-by: rcj1 <77995559+rcj1@users.noreply.github.com>
Updated the cDAC docs (GC.md) with new API types, data descriptors, globals, constants, and pseudocode for all three methods. Added 6 additional test methods covering edge cases (empty buckets, multiple linked segments, multiple bookkeeping entries, empty/multiple free regions). Total: 9 test methods × 4 architectures = 36 tests, all passing. See commit |
docs/design/datacontracts/GC.md
Outdated
|
|
||
| int maxSlots = 1; | ||
| if (IsServerGC()) | ||
| maxSlots = (int)GetGCHeapCount(); |
There was a problem hiding this comment.
Fixed in 6320ebd1. Changed maxSlots to read from TotalCpuCount (the processor count) instead of using GetGCHeapCount() for server GC, matching the fix in PR #124875. Added the TotalCpuCount global pointer to datadescriptor.inc (under SERVER_GC), Constants.Globals, and updated the GC.md pseudocode accordingly.
|
@copilot fix the build break |
|
Ah I will merge my changes into main and then this should work better |
…it in server GC Co-authored-by: rcj1 <77995559+rcj1@users.noreply.github.com>
In RhpThrowImpl, before calling INLINE_GETTHREAD we were saving the caller-saved register r0 but we should save r2 as well since it contains the exception kind. Fixes #123979. --------- Co-authored-by: Eduardo Velarde <evelardepola@microsoft.com> Co-authored-by: Michal Strehovský <MichalStrehovsky@users.noreply.github.com>
The new Windows VM image has newer `docker` command, so the dockerfile must be within the build context. And it doesn't have `docker compose` but `docker-compose`.
Fixes #67146 by throwing from `IHost`.`RunAsync`/`StopAsync` with the default host when a `BackgroundService` fails with an exception and `BackgroundServiceExceptionBehavior.StopHost` is set (which is the default).
Add support for reloading configuration to `ChainedConfigurationSource`. Fixes #58683. --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Tarek Mahmoud Sayed <10833894+tarekgh@users.noreply.github.com>
…ingClosedImplementation test for issue #57333 (#124326) ## Description Issue #57333 reported that `GetServices<T>()` inconsistently resolves services when mixing closed generic registrations (`IService<T> -> Impl`) with open generic registrations (`IService<> -> GenericImpl<>`), particularly when `GetService<T>()` is called first. An existing test `CallSiteFactoryResolvesIEnumerableOfOpenGenericServiceAfterResolvingClosedImplementation` was intended to validate this scenario but was marked with `[ActiveIssue("https://github.com/dotnet/runtime/issues/57333")]` because it failed during execution. The failure was due to a test setup issue, not an actual DI bug. ## Changes * **Fixed existing test** in `CallSiteTests`: - `CallSiteFactoryResolvesIEnumerableOfOpenGenericServiceAfterResolvingClosedImplementation` - The test uses `FakeOpenGenericService<TVal>` which requires a `TVal` constructor parameter - The test was failing because no service of type `TVal` was registered in the DI container - Fixed by: - Changed test from `IFakeOpenGenericService<int>` to `IFakeOpenGenericService<string>` (reference type) - Added `FakeStringService` class implementing `IFakeOpenGenericService<string>` - Registered `descriptors.AddSingleton<string>("Foo")` to provide the required dependency - Removed `[ActiveIssue("https://github.com/dotnet/runtime/issues/57333")]` attribute - Test now passes and properly validates the reported behavior Example scenario validated: ```csharp descriptors.Add(ServiceDescriptor.Scoped(typeof(IFakeOpenGenericService<string>), typeof(FakeStringService))); descriptors.Add(ServiceDescriptor.Scoped(typeof(IFakeOpenGenericService<>), typeof(FakeOpenGenericService<>))); descriptors.AddSingleton<string>("Foo"); // Required for FakeOpenGenericService<string> constructor var single = provider.GetService<IFakeOpenGenericService<string>>(); // Returns FakeStringService var enumerable = provider.GetService<IEnumerable<IFakeOpenGenericService<string>>>(); // Returns both FakeStringService and FakeOpenGenericService<string> ``` Test confirms the behavior works correctly - existing implementation handles this scenario properly. No implementation code changes required. <!-- START COPILOT ORIGINAL PROMPT --> <details> <summary>Original prompt</summary> ---- *This section details on the original issue you should resolve* <issue_title>Enumerable dependencies resolving not consistent when open generic is used (area-Extensions-DependencyInjection)</issue_title> <issue_description><!--This is just a template - feel free to delete any and all of it and replace as appropriate.--> ### Description <!-- * Please share a clear and concise description of the problem. * Include minimal steps to reproduce the problem if possible. E.g.: the smallest possible code snippet; or a small repo to clone, with steps to run it. * What behavior are you seeing, and what behavior would you expect? --> Enumerable dependencies are not resolved consistently when open generic services are registered later. Ex: #### Setup ```csharp interface IProcessor<T> {} record AnyProcessor<T> : IProcessor<T>; record IntProcessor : IProcessor<int>; ``` #### Registration ```csharp var services = new ServiceCollection(); services.TryAddEnumerable(ServiceDescriptor.Scoped(typeof(IProcessor<int>), typeof(IntProcessor))); // Fix: ordering open generic above concretes services.TryAddEnumerable(ServiceDescriptor.Scoped(typeof(IProcessor<>), typeof(AnyProcessor<>))); ``` #### Resolving ```csharp var serviceProvider = services.BuildServiceProvider(); using var scope = serviceProvider.CreateAsyncScope(); // bug is reproducible only when below line enabled var processor = scope.ServiceProvider.GetService<IProcessor<int>>(); var processors = scope.ServiceProvider.GetService<IEnumerable<IProcessor<int>>>() ?? Enumerable.Empty<IProcessor<int>>(); // prints "IntProcessor -- IntProcessor" instead of IntProcessor -- AnyProcessor`1 if line 17 commented. Console.WriteLine(string.Join(" -- ", processors.Select(p => p.GetType().Name))); ``` ### To Reproduce sample repo to reproduce the bug https://github.com/skclusive/EnumerableCallSiteBugConsole ### Configuration <!-- * Which version of .NET is the code running on? * What OS and version, and what distro if applicable? * What is the architecture (x64, x86, ARM, ARM64)? * Do you know whether it is specific to that configuration? * If you're using Blazor, which web browser(s) do you see this issue in? --> - `dotnet --info .NET SDK (reflecting any global.json): Version: 6.0.100-preview.7.21379.14 Commit: 22d70b47bc Runtime Environment: OS Name: Windows OS Version: 10.0.22000 OS Platform: Windows RID: win10-x64 Base Path: C:\Program Files\dotnet\sdk\6.0.100-preview.7.21379.14\ Host (useful for support): Version: 6.0.0-preview.7.21377.19 Commit: 91ba017 .NET SDKs installed: 5.0.301 [C:\Program Files\dotnet\sdk] 5.0.400 [C:\Program Files\dotnet\sdk] 6.0.100-preview.7.21379.14 [C:\Program Files\dotnet\sdk] .NET runtimes installed: Microsoft.AspNetCore.All 2.1.28 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All] Microsoft.AspNetCore.App 2.1.28 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 3.1.18 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 5.0.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 5.0.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 6.0.0-preview.7.21378.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.NETCore.App 2.1.28 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 3.1.18 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 5.0.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 5.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 6.0.0-preview.7.21377.19 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.WindowsDesktop.App 3.1.18 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 5.0.7 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 5.0.9 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 6.0.0-preview.7.21378.9 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] - Microsoft Visual Studio Community 2022 Preview (64-bit) Version 17.0.0 Preview 3.0 ### Regression? <!-- * Did this work in a previous build or release of .NET Core, or from .NET Framework? If you can try a previous release or build to find out, that can help us narrow down the problem. If you don't know, that's OK. --> ### Other information <!-- * Please include any relevant stack traces or error messages. If possible please include text as text rather than images (so it shows up in searches). * If you have an idea where the problem might lie, let us know that here. Please include any pointers to code, relevant changes, or related issues you know of. * Do you know ... </details> <!-- START COPILOT CODING AGENT SUFFIX --> - Fixes #57333 <!-- START COPILOT CODING AGENT TIPS --> --- 💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more [Copilot coding agent tips](https://gh.io/copilot-coding-agent-tips) in the docs. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: rosebyte <14963300+rosebyte@users.noreply.github.com> Co-authored-by: Stephen Toub <stoub@microsoft.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com> Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
JIT CI is currently broken because #124033 removed some hardware intrinsics from the JIT that makes the JIT incompatible with previous versions of the BCL/SPMI collections.
…bly names only (#125365) `TrimmerRootAssembly` items and the illink `-a` flag previously accepted file paths, creating an MSBuild incremental build gap: illink could read files not declared as build inputs. This aligns both ILLink and ILC to accept only assembly names. ## Description **ILLink command-line (`-a` flag)** - `Driver.cs`: Remove `File.Exists()` pre-check; `-a` now accepts assembly names only - `RootAssemblyInputStep.cs`: Remove `File.Exists()` loading path ("quirks mode"); rename `LoadAssemblyFile()` → `LoadAssemblyByName()` and `fileName` field/parameter → `assemblyName`, resolve exclusively via `Context.TryResolve()` **MSBuild targets** - `Microsoft.NET.ILLink.targets`: Populate `TrimmerRootAssembly` with `@(IntermediateAssembly->'%(Filename)')` (name, no extension) instead of the full item path - `Microsoft.NETCore.Native.targets`: Remove complex two-step `_Raw` + `System.IO.File::Exists()` pattern that converted paths to names; `TrimmerRootAssembly` is now used directly as assembly names; `ManagedAssemblyToLink` still uses `%(Filename)` since it contains file paths - `BrowserWasmApp.targets`: Update condition to compare `%(Identity)` against `$(AssemblyName)` instead of `@(IntermediateAssembly)` - `AppleBuild.LocalBuild.targets`: Use `%(AppleAssembliesToBundle.FileName)` (no extension) instead of `%(Identity)` (full path) - `eng/illink.targets`: Use explicit `$(ILLinkTrimInputPath)$(TargetName)$(TargetExt)` for `AssemblyPaths` instead of the ambiguous `$(ILLinkTrimInputAssembly)` property that shares a name with the root-name item **Test infrastructure** - `TrimmingArgumentBuilder.cs`: Strip extension via `Path.GetFileNameWithoutExtension()` when building `-a` arguments - `TestCaseMetadataProvider.cs`: Move relative-path expansion from `-a` to `-d` (search directory arguments); `-a` values are now passed through as-is - Update 21 test cases to use assembly names without extensions in `[SetupLinkerArgument("-a", ...)]` - `RootLibraryVisibleForwardersWithoutReference.cs`: Add explicit `[SetupLinkerArgument("-reference", "isolated/library.dll")]` so the subdirectory assembly is resolvable, and use assembly name `library` for the `-a` argument - `Mock.cs`: Update reflection-based field access from `"fileName"` to `"assemblyName"` to match the renamed field in `RootAssemblyInput` ## Customer Impact No direct customer impact; this is an internal build/tooling correctness fix. Users who were passing file paths to `TrimmerRootAssembly` (undocumented usage) will need to switch to assembly names. ## Regression No — this is a proactive correctness fix, not a regression fix. ## Testing All 377 affected ILLink test cases pass (`Mono.Linker.Tests`). All 108 `ILLink.Tasks.Tests` pass. ## Risk Low. The change is contained to ILLink/ILC build integration and test infrastructure. The linker's assembly resolution behavior is unchanged — only the file-path loading shortcut is removed. MSBuild targets now pass assembly names consistently, which is what downstream tools (ILC) already required. <!-- START COPILOT CODING AGENT TIPS --> --- 💬 We'd love your input! Share your thoughts on Copilot coding agent in our [2 minute survey](https://gh.io/copilot-coding-agent-survey). --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: sbomer <787361+sbomer@users.noreply.github.com> Co-authored-by: Sven Boemer <sbomer@gmail.com>
## Summary The Linux `curl` command in `AcquireWasiSdk.targets` has **no retry logic**, unlike the Windows `download-wasi-sdk.ps1` which retries up to 8 times on partial transfers. This causes transient GitHub download failures (rate limits, network blips, truncated responses) to fail the entire build with a cryptic `tar` error: ``` tar: Error is not recoverable: exiting now ``` ## Change Add `--show-error --fail --retry 3 --retry-delay 5` to the Linux curl invocation: ```diff - curl --silent -L -o wasi-sdk-25.0.tar.gz <url> + curl --silent --show-error --fail --retry 3 --retry-delay 5 -L -o wasi-sdk-25.0.tar.gz <url> ``` - `--retry 3`: Retry up to 3 times on failure - `--retry-delay 5`: Wait 5 seconds between retries ## Motivation Build [2924549](https://dev.azure.com/dnceng/internal/_build/results?buildId=2924549) failed on `Linux_Alpine_x64` due to a transient wasi-sdk download failure. The next scheduled build succeeded without any code change — confirming the issue was transient. The Windows download script already handles this case with up to 8 retry attempts. cc @lewing --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: max-charlamb <44248479+max-charlamb@users.noreply.github.com>
…r comparison, null guard, constants Co-authored-by: max-charlamb <44248479+max-charlamb@users.noreply.github.com>
All four feedback items addressed in Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
| int maxRegions = MaxHandleTableRegions; | ||
| TargetPointer handleTableMap = _target.ReadGlobalPointer(Constants.Globals.HandleTableMap); | ||
| while (handleTableMap != TargetPointer.Null && maxRegions >= 0) | ||
| { | ||
| Data.HandleTableMap map = _target.ProcessedData.GetOrAdd<Data.HandleTableMap>(handleTableMap); | ||
| foreach (TargetPointer bucketPtr in map.BucketsPtr) | ||
| { | ||
| if (bucketPtr == TargetPointer.Null) | ||
| continue; | ||
|
|
||
| Data.HandleTableBucket bucket = _target.ProcessedData.GetOrAdd<Data.HandleTableBucket>(bucketPtr); | ||
| for (uint j = 0; j < tableCount; j++) | ||
| { | ||
| TargetPointer handleTablePtr = _target.ReadPointer(bucket.Table + (ulong)(j * _target.PointerSize)); | ||
| if (handleTablePtr == TargetPointer.Null) | ||
| continue; | ||
|
|
||
| Data.HandleTable handleTable = _target.ProcessedData.GetOrAdd<Data.HandleTable>(handleTablePtr); | ||
| if (handleTable.SegmentList == TargetPointer.Null) | ||
| continue; | ||
|
|
||
| TargetPointer segmentPtr = handleTable.SegmentList; | ||
| TargetPointer firstSegment = segmentPtr; | ||
| do | ||
| { | ||
| Data.TableSegment segment = _target.ProcessedData.GetOrAdd<Data.TableSegment>(segmentPtr); | ||
| regions.Add(new GCMemoryRegionData | ||
| { | ||
| Start = segmentPtr, | ||
| Size = handleSegmentSize, | ||
| Heap = (int)j, | ||
| }); | ||
| segmentPtr = segment.NextSegment; | ||
| maxRegions--; | ||
| } while (segmentPtr != TargetPointer.Null && segmentPtr != firstSegment && maxRegions >= 0); | ||
| } | ||
| } | ||
| handleTableMap = map.Next; | ||
| maxRegions--; | ||
| } |
| curr = segment.Next; | ||
| if (curr == start) | ||
| break; | ||
| if (--iterationMax <= 0) |
| // Safety caps matching native DAC | ||
| const int MaxHandleTableRegions = 8192; | ||
| const int MaxBookkeepingRegions = 32; | ||
| const int MaxSegmentListIterations = 2048; | ||
|
|
||
| int maxRegions = MaxHandleTableRegions; | ||
| TargetPointer handleTableMap = target.ReadGlobalPointer("HandleTableMap"); | ||
| while (handleTableMap != null && maxRegions >= 0) | ||
| { | ||
| HandleTableMap map = Read<HandleTableMap>(handleTableMap); | ||
| foreach (TargetPointer bucketPtr in map.BucketsPtr) | ||
| { | ||
| if (bucketPtr == null) continue; | ||
| HandleTableBucket bucket = Read<HandleTableBucket>(bucketPtr); | ||
| for (uint j = 0; j < tableCount; j++) | ||
| { | ||
| TargetPointer htPtr = ReadPointer(bucket.Table + j * PointerSize); | ||
| if (htPtr == null) continue; | ||
| HandleTable ht = Read<HandleTable>(htPtr); | ||
| if (ht.SegmentList == null) continue; | ||
| TargetPointer seg = ht.SegmentList; | ||
| TargetPointer first = seg; | ||
| do | ||
| { | ||
| TableSegment ts = Read<TableSegment>(seg); | ||
| regions.Add(new GCMemoryRegionData { Start = seg, Size = handleSegmentSize, Heap = (int)j }); | ||
| seg = ts.NextSegment; | ||
| maxRegions--; | ||
| } while (seg != null && seg != first && maxRegions >= 0); | ||
| } | ||
| } | ||
| handleTableMap = map.Next; | ||
| maxRegions--; | ||
| } |
| void AddSegmentList(TargetPointer start, FreeRegionKind kind, List<GCMemoryRegionData> regions, int heap = 0) | ||
| { | ||
| int iterationMax = MaxSegmentListIterations; | ||
| TargetPointer curr = start; | ||
| while (curr != null && iterationMax-- > 0) | ||
| { | ||
| HeapSegment seg = Read<HeapSegment>(curr); | ||
| if (seg.Mem != null) | ||
| { | ||
| ulong size = (seg.Mem < seg.Committed) ? seg.Committed - seg.Mem : 0; | ||
| regions.Add(new GCMemoryRegionData { Start = seg.Mem, Size = size, ExtraData = kind, Heap = heap }); | ||
| } | ||
| curr = seg.Next; | ||
| if (curr == start) break; | ||
| } |
Implements three
ISOSDacInterface13APIs natively in the cDAC by extending theIGCcontract:GetHandleTableMemoryRegions,GetGCBookkeepingMemoryRegions, andGetGCFreeRegions. These APIs previously delegated to legacy DAC or returnedE_NOTIMPL.Description
SOSMemoryEnum
SOSMemoryEnumimplementation following theSOSHandleEnumpattern, with built-in#if DEBUGvalidation against the legacy DAC enum (validatesStart,Size,ExtraData,Heapfields onNext(),Skip(),Reset(), andGetCount())IGC Contract Extensions
IGCcontract with three new memory region enumeration methodsMaxHandleTableRegions(8192),MaxBookkeepingRegions(32),MaxSegmentListIterations(2048)GetGCFreeRegionsusesTryReadGlobalforCountFreeRegionKindsto gracefully handle non-regions GC buildsheap.FreeRegionsvalue in server GC pathnext > cardTableInfoSize) matching native DACdaccess.cpp:8313Data Types & Descriptors
CardTableInfoandRegionFreeListdata classesGCHeaptype descriptors with optional fields (FreeableSohSegment,FreeableUohSegment,FreeRegions) conditional on GC build configuration (BACKGROUND_GC,USE_REGIONS)Documentation & Tests
docs/design/datacontracts/GC.mdwith new API types, data descriptors, globals, constants, and pseudocodeGCMemoryRegionTests.cs) covering single/multiple segments, empty buckets, linked segments, bookkeeping entries, and free regionsServerGCDumpTests.csandWorkstationGCDumpTests.cs💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.