200kB size delta for minimal NativeAOT shared lib with reflection disabled mode

[Sergio0694]

Note

Follow up to #82607

Overview

I'm experimenting with NativeAOT to create self-contained native shared libraries using ComputeSharp that can be used by native applications (or other native libraries). I'm currently using sizoscope to go through everything's causing binary size increase and trying to address them. After having found and fixed the two main issues I had there (one Regex use, and Type.GUID pulling in a bunch of garbage), I'm left with what I think is pretty much as minimal as I can reasonably be the library to be, and it all works with reflection disabled as well (as it's not really using reflection anywhere either). But, I'm noticing quite a big size delta when trying with reflection free to get another comparison point.

Reflection disabled	Binary size
No	1030 KB
Yes	830 KB

Delta: +200 KB (~24%)

This seems a bit much, and looking at the delta with sizoscope I'm not sure I understand what's causing this exactly 😅

Data

I can provide exact repro steps (right now stuff is just hacked together on a local branch).
But, here's all the sizoscope files, which I think should help already:

• baseline.zip
• disabled_reflection.zip

Additional context

@MichalStrehovsky mentioned on Discord he thinks (at least part of?) this might be caused by the object.GetType() goo that's pulled in by default because that's how object.ToString() works when no additional implementation is provided. This seems highly unfortunate, as eg. for libraries that's not a method you'd implement for usually most of your types (as you'd just never need to call that), and paying such a high cost in terms of binary size for that doesn't seem great 🥲

Would it make sense to consider adding some new option to explicitly disable that, and just get back the behavior that both .NET Native and NativeAOT in reflection disabled mode have when trying to call ToString(), ie. just printing EETypeRva:0x033EE048? That seems like an option that would certainly be useful in cases like this, especially for native library authors 🤔

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

---

[!NOTE]
Follow up to #82607

Overview

I'm experimenting with NativeAOT to create self-contained native shared libraries using ComputeSharp that can be used by native applications (or other native libraries). I'm currently using sizoscope to go through everything's causing binary size increase and trying to address them. After having found and fixed the two main issues I had there (one Regex use, and Type.GUID pulling in a bunch of garbage), I'm left with what I think is pretty much as minimal as I can reasonably be the library to be, and it all works with reflection disabled as well (as it's not really using reflection anywhere either). But, I'm noticing quite a big size delta when trying with reflection free to get another comparison point.

Reflection disabled	Binary size
No	1030 KB
Yes	830 KB

Delta: +200 KB (~24%)

This seems a bit much, and looking at the delta with sizoscope I'm not sure I understand what's causing this exactly.

Data

I can provide exact repro steps (right now stuff is just hacked together on a local branch).
But, here's all the sizoscope files, which I think should help already:

• baseline.zip
• disabled_reflection.zip

Additional context

@MichalStrehovsky mentioned on Discord he thinks (at least part of?) this might be caused by the object.GetType() goo that's pulled in by default because that's how object.ToString() works when no additional implementation is provided. This seems highly unfortunate, as eg. for libraries that's not a method you'd implement for usually most of your types (as you'd just never need to call that), and paying such a high cost in terms of binary size for that doesn't seem great 🥲

Would it make sense to consider adding some new option to explicitly disable that, and just get back the behavior that both .NET Native and NativeAOT in reflection disabled mode have when trying to call ToString(), ie. just printing EETypeRva:0x033EE048? That seems like an option that would certainly be useful in cases like this, especially for native library authors 🤔

Author:	Sergio0694
Assignees:	-
Labels:	area-NativeAOT-coreclr
Milestone:	-

[Sergio0694]

Remove two typeof(T).FullName calls I had and that shaved off another ~20 KB from both cases 🤔

Reflection disabled	Binary size
No	1006 KB
Yes	811 KB

Delta: +195 KB (~23%)

Updated diffs:

• baseline2.zip
• disabled_reflection2.zip

[Sergio0694]

Including a bunch of random notes here as I investigate this more.

3 P/Invoke methods take up 56 KB??

There's a whole bunch of stuff (like these "dispencers"..?) that seem to be rooted by 3 P/Invoke-s I have. Removing them saves 56 KB of binary size. That seems... A bit much? Is there no way to make P/Invoke-s have less of a binary size impact?

StringBuilder from Exception.StackTrace

Why is StringBuilder rooted from a bunch of Exception.StackTrace even though I'm using StackTraceSupport=false?

AppContext.OnUnhandledException and generic math interfaces

It seems like AppContext.OnUnhandledException is being preserved (though I'm never using it..?) and this shows up in a ton of dependency chains, and it seems to be causing quite a big binary size increase. Eg. this seems to be rooting all the generic math interfaces:

AppContext.OnUnhandledException and more random stuff

The same AppContext.OnUnhandledException also seems to be rooting just... Random stuff? Eg. why are Rune UTF8 encoding APIs being preserved?

[MichalStrehovsky]

Isn't ~200 kB around the size we established to be the expected difference in #82607 (comment)?

3 P/Invoke methods take up 56 KB??

If you dig into FixupModuleCell (the place where p/invoke starts depending on reflection), you'll see it's needed because NativeLibrary APIs could redirect the p/invoke to a different DLL:

runtime/src/coreclr/nativeaot/System.Private.CoreLib/src/Internal/Runtime/CompilerHelpers/InteropHelpers.cs

Lines 302 to 305 in 741ab87

	Assembly callingAssembly = RuntimeAugments.Callbacks.GetAssemblyForHandle(new RuntimeTypeHandle(pCell->CallingAssemblyType));
	// First check if there's a NativeLibrary callback and call it to attempt the resolution
	IntPtr hModule = NativeLibrary.LoadLibraryCallbackStub(moduleName, callingAssembly, hasDllImportSearchPath, dllImportSearchPath);

We could potentially replace this direct call with a function pointer stored in a static field that gets created only if NativeLibrary.SetDllImportResolver is called (we'd just do a nullcheck here and if the delegate is non-null, call it). But it's questionable if the savings will be worth the hassle. An example of what I mean: #83064.

You can also get rid of this by turning the p/invoke into a direct p/invoke: https://learn.microsoft.com/en-us/dotnet/core/deploying/native-aot/interop

StringBuilder from Exception.StackTrace

StackTraceSupport stops generating stack trace data, but we still print list of addresses. This is StringBuilding used to print those.

AppContext.OnUnhandledException and generic math interfaces

That's just a "display artifact". It's the first thing that referenced UInt. I'm pretty sure something else would reference UInt if it wasn't for this codepath. This could also be re-done to use a delegate. Not clear if it's worth it.

AppContext.OnUnhandledException and more random stuff

You can also just search the repo to see why UTF8 is needed. The method name to search for is right there in the log.

runtime/src/coreclr/nativeaot/System.Private.CoreLib/src/System/AppContext.NativeAot.cs

Lines 13 to 26 in 741ab87

	private static unsafe Dictionary<string, object?> InitializeDataStore()
	{
	uint count = RuntimeImports.RhGetKnobValues(out byte** keys, out byte** values);
	var dataStore = new Dictionary<string, object?>((int)count);
	for (int i = 0; i < count; i++)
	{
	dataStore.Add(
	Encoding.UTF8.GetString(keys[i], string.strlen(keys[i])),
	Encoding.UTF8.GetString(values[i], string.strlen(values[i])));
	}
	return dataStore;
	}

Pretty sure this is not the only thing that depends on UTF-8 decoding (or AppContext) in the app. It's just the first thing. MSTAT only captures the first thing. Things get very complicated when the dependency tree becomes a dependency graph in the visualization.

[Sergio0694]

"Isn't ~200 kB around the size we established to be the expected difference in #82607?"

I guess so, but given this library was even smaller and still with the same amount of delta I figured I might as well open an issue just to check whether there could be other improvement opportunities. That's especially because minimizing size as much as possible is even more important when authoring shared libraries like in this case, as opposed to apps you own 😅

"We could potentially replace this direct call with a function pointer stored in a static field that gets created only if NativeLibrary.SetDllImportResolver is called (we'd just do a nullcheck here and if the delegate is non-null, call it)."

That seems like a minimal change to achieve that and I think those size savings when using P/Invoke-s would be very nice to have, again especially when authoring shared libraries. Is that something you'd be willing to take contributions from? Because I'd love to try my first NativeAOT PR for that if so 😄

"That's just a "display artifact". It's the first thing that referenced UInt. I'm pretty sure something else would reference UInt if it wasn't for this codepath. This could also be re-done to use a delegate. Not clear if it's worth it."

I guess given there seem to be several things showing up from there, it might make sense to at least try using that delegate trick for this event to, make a local build and double check the diff and see whether that does in fact save anything or whether it was just the MSTAT tree being misleading? 🤔

[MichalStrehovsky]

That seems like a minimal change to achieve that and I think those size savings when using P/Invoke-s would be very nice to have, again especially when authoring shared libraries. Is that something you'd be willing to take contributions from? Because I'd love to try my first NativeAOT PR for that if so 😄

Would be good to measure what the actual savings would be. I don't think we'll get the full 56 kB. If it's 50, it might be interesting. If it's 10, less so. It's always a compromise between maintainability and savings. It's worth a shot. The PR with the function pointer trick I linked to was a stepping stone to get rid of the type loader (it got rid of one of the code paths what would bring it in) and is essentially a couple hundred kB saving. Converting the p/invoke to a direct p/invoke is the ultimate size saving in this case.

I guess given there seem to be several things showing up from there, it might make sense to at least try using that delegate trick for this event to, make a local build and double check the diff and see whether that does in fact save anything or whether it was just the MSTAT tree being misleading? 🤔

Doesn't hurt to try.

Here's the workflow I typically use:

• Download a recent enough version of the installer ZIP (https://github.com/dotnet/installer#table). The reason you want recent enough is to be as close as possible to the runtime repo versions of things. We very recently shifted to v9.0 so hopefully things are not too broken. I didn't check today. Sometimes you could be unlucky and things are broken, check back in a couple days or use an older daily build. I usually just get the ZIP and extract it somewhere and put it on PATH, no need to install.
• Build the repo. For this experiment running build.cmd clr.aot -c Release will get you all you need.
• Create a new AOT app (make sure the installer ZIP you downloaded is on your path): dotnet new console --aot -o MyTestApp. Cd to MyTestApp and add nuget.config with the contents the installer repo tells you to add: https://github.com/dotnet/installer#installers-and-binaries
• dotnet publish the app just to check it works
• Delete obj and bin
• Open the csproj and add this to a PropertyGroup: <IlcSdkPath>C:\git\runtime\artifacts\bin\coreclr\windows.x64.Release\aotsdk\</IlcSdkPath><IlcToolsPath>C:\git\runtime\artifacts\bin\coreclr\windows.x64.Release\ilc-published\</IlcToolsPath> (replace the paths with where you put your runtime repo.
• dotnet publish again, this time it should use ILC you built and CoreLib you built.

If you change corelib or compiler, delete obj and bin directories on the test project, the build won't be incremental because we don't check there.