Skip to content

[Crossgen2] Deduplicate identical IL method bodies in R2R composite image #126047

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
kotlarmilos wants to merge 5 commits into from
Closed

[Crossgen2] Deduplicate identical IL method bodies in R2R composite image #126047

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
7f9f268
[Crossgen2] Add --dedup-il-bodies to deduplicate identical IL method …
kotlarmilos Mar 24, 2026
13e5ad0
Make IL body dedup always-on, remove --dedup-il-bodies option
kotlarmilos Mar 25, 2026
27a15ce
Use single byte[] -> Node mapping for IL body dedup
kotlarmilos Mar 26, 2026
c235831
Merge branch 'main' into r2r-dedup-il-bodies-measure
kotlarmilos Apr 8, 2026
28021fa
Dedup IL bodies after stripping
kotlarmilos Apr 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 19 additions & 10 deletions ...ls/aot/ILCompiler.ReadyToRun/Compiler/DependencyAnalysis/ReadyToRun/CopiedMethodILNode.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,21 +55,30 @@ public override ObjectData GetData(NodeFactory factory, bool relocsOnly = false)
definedSymbols: new ISymbolDefinitionNode[] { this });
}

if (factory.OptimizationFlags.StripILBodies
&& factory.OptimizationFlags.CompiledMethodDefs.Contains(_method)
&& !_method.HasInstantiation
&& !_method.OwningType.HasInstantiation)
byte[] bodyBytes = ReadBodyBytes(_method, factory.OptimizationFlags);

return new ObjectData(bodyBytes, Array.Empty<Relocation>(), 4, new ISymbolDefinitionNode[] { this });
}
Comment thread
kotlarmilos marked this conversation as resolved.

internal static byte[] ReadBodyBytes(EcmaMethod method, NodeFactoryOptimizationFlags optimizationFlags)
{
method = (EcmaMethod)method.GetTypicalMethodDefinition();

if (optimizationFlags.StripILBodies
&& optimizationFlags.CompiledMethodDefs?.Contains(method) == true
&& !method.HasInstantiation
&& !method.OwningType.HasInstantiation)
{
return new ObjectData(s_minimalILBody, Array.Empty<Relocation>(), 4, new ISymbolDefinitionNode[] { this });
return s_minimalILBody;
}

var rva = _method.MetadataReader.GetMethodDefinition(_method.Handle).RelativeVirtualAddress;
var peReader = _method.Module.PEReader;
var reader = peReader.GetSectionData(rva).GetReader();
int rva = method.MetadataReader.GetMethodDefinition(method.Handle).RelativeVirtualAddress;
if (rva == 0)
return Array.Empty<byte>();
BlobReader reader = method.Module.PEReader.GetSectionData(rva).GetReader();
int size = MethodBodyBlock.Create(reader).Size;
byte[] bodyBytes = peReader.GetSectionData(rva).GetReader().ReadBytes(size);

return new ObjectData(bodyBytes, Array.Empty<Relocation>(), 4, new ISymbolDefinitionNode[] { this });
return reader.ReadBytes(size);
}

public override int ClassCode => 541651465;
Expand Down
10 changes: 3 additions & 7 deletions ...ols/aot/ILCompiler.ReadyToRun/Compiler/DependencyAnalysis/ReadyToRunCodegenNodeFactory.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -355,11 +355,6 @@ private void CreateNodeCaches()
return new CopiedMetadataBlobNode(module);
});

_copiedMethodIL = new NodeCache<MethodDesc, CopiedMethodILNode>(method =>
{
return new CopiedMethodILNode((EcmaMethod)method);
});

_copiedFieldRvas = new NodeCache<ModuleAndIntValueKey, CopiedFieldRvaNode>(key =>
{
return new CopiedFieldRvaNode(key.Module, key.IntValue);
Expand Down Expand Up @@ -1148,11 +1143,12 @@ public CopiedMetadataBlobNode CopiedMetadataBlob(EcmaModule module)
return _copiedMetadataBlobs.GetOrAdd(module);
}

private NodeCache<MethodDesc, CopiedMethodILNode> _copiedMethodIL;
private readonly ConcurrentDictionary<byte[], CopiedMethodILNode> _copiedMethodIL = new(ByteArrayComparer.Instance);

public CopiedMethodILNode CopiedMethodIL(EcmaMethod method)
{
return _copiedMethodIL.GetOrAdd(method);
byte[] bodyBytes = CopiedMethodILNode.ReadBodyBytes(method, OptimizationFlags);
return _copiedMethodIL.GetOrAdd(bodyBytes, _ => new CopiedMethodILNode(method));
Comment on lines +1150 to +1151
Copy link
Copy Markdown
Member

@MichalStrehovsky MichalStrehovsky Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may not work. If first time someone asks for CopiedMethodIL and optimizationFlags.CompiledMethodDefs is still null, we'll get a node for the real method body bytes. This node will become part of the dependency graph and will get written to the output no matter if there's still a reference to it at the writing phase (if the node is marked, it will get written to the output).

If this is not a concern, we should remove the question mark from optimizationFlags.CompiledMethodDefs?.Contains(method) so that this is a hard crash if anyone wants to use this before we do the analysis.

If it is a concern, there are ways to address this. Probably the easiest would be to delete the CopiedMethodILNode completely and instead emit the bodies as part of CopiedMetadataBlobNode. We don't need a separate symbol for each method body - the builder.EmitReloc(factory.CopiedMethodIL(method), RelocType.IMAGE_REL_BASED_ADDR32NB); can be builder.EmitReloc(this, RelocType.IMAGE_REL_BASED_ADDR32NB, someOffsetFromStart); (where someOffsetFromStart is _sourceModule.PEReader.GetMetadata().Length + cumulativeSizeOfMethodBodiesEmittedSoFar).

}
Comment thread
kotlarmilos marked this conversation as resolved.
Comment thread
kotlarmilos marked this conversation as resolved.
Comment thread
kotlarmilos marked this conversation as resolved.

private NodeCache<ModuleAndIntValueKey, CopiedFieldRvaNode> _copiedFieldRvas;
Expand Down
Loading