Proposal: expose 'IlcDehydrate' as a supported option

[Sergio0694]

NativeAOT currently exposes the undocumented 'IlcDehydrate' option (enabled by default), which reduces binary size by dehydrating pointer values in the Native AOT runtime data structures, and then hydrates them at startup. This trades binary size for perf and memory use. We experimented with disabling this setting in the Microsoft Store, and after profiling we had some positive results that make the tradeoff worth it for us:

• 20-30ms startup perf improvement
• ~50 MB memory use reduction

Would it be possible to expose this as a documented option? Probably with a different name, of course.
Ideally, this should be a standalone option (like it is now), and not just implied by something else (eg. 'OptimizationPreference').

cc. @MichalStrehovsky

[dotnet-policy-service]

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

[MichalStrehovsky]

Ideally, this should be a standalone option (like it is now), and not just implied by something else (eg. 'OptimizationPreference').

I would actually lean towards turning on dehydration on Windows only if OptimizationPreference=Size. Exactly because of the characteristics you saw - this option makes startup and working set meaningfully worse and startup and working set are the perf characteristics we advertise as the biggest selling points of native AOT (size too, but size is third in importance).

We can keep the undocumented switch to turn it on/off individually, but we'd change the default to false on Windows.

IlcDehydrate was mostly built for Linux because the considerations are different there. We turned it on on Windows for parity.

[Sergio0694]

"We can keep the undocumented switch to turn it on/off individually, but we'd change the default to false on Windows."

Changing the default on Windows makes sense to, but it feels like it'd make sense to document the option too? I mean, is the idea is to still keep it available for advanced users to potentially change the default, why keep it effectively as "unsupported"? The reason I'm worried is that the impact on size is not small (can very well be over 10 MBs), so it might make the delta between optimizing for size and speed too big in some scenarios. Being able to have some knobs to have more fine tuned control there would make sense to me, without having to use something that is undocumented? 😅

[MichalStrehovsky]

Here's some numbers collected on the ASP.NET perf infrastructure (don't ask me why there's two reported numbers for Max Working Set):

BasicMinimalApi app, IlcDehydrate on:

	Attempt 1	Attempt 2	Attempt 3	Attempt 4	Attempt 5
Max Working Set (MB)	67	67	66	65	67
Max Private Memory (MB)	75	74	76	74	75
Start Time (ms)	66	65	66	76	65
Max Working Set (MB)	71	74	69	67	68
Load Working Set - P90 (MB)	64	67	66	65	65
First Request (ms)	108	79	80	78	78
Requests/sec	697,342	689,366	686,463	686,094	686,936

BasicMinimalApi app, IlcDehydrate off:

	Attempt 1	Attempt 2	Attempt 3	Attempt 4	Attempt 5
Max Working Set (MB)	63	62	68	60	58
Max Private Memory (MB)	69	70	77	64	63
Start Time (ms)	69	68	68	68	68
Max Working Set (MB)	76	61	70	63	60
Load Working Set - P90 (MB)	63	59	64	59	58
First Request (ms)	79	78	78	80	79
Requests/sec	690,509	664,070	688,901	654,688	681,821

TodosApi, IlcDehydrate on:

	Attempt 1	Attempt 2	Attempt 3	Attempt 4	Attempt 5
Max Working Set (MB)	158	153	146	156	150
Max Private Memory (MB)	172	164	157	167	161
Start Time (ms)	154	158	159	153	177
Max Working Set (MB)	165	160	153	163	157
Load Working Set - P90 (MB)	158	153	146	156	150
First Request (ms)	85	87	87	81	81
Requests/sec	199,312	199,266	199,348	200,852	197,801

TodosApi, IlcDehydrate off:

	Attempt 1	Attempt 2	Attempt 3	Attempt 4	Attempt 5
Max Working Set (MB)	147	140	140	147	146
Max Private Memory (MB)	159	151	151	159	157
Start Time (ms)	180	160	182	168	162
Max Working Set (MB)	154	146	146	153	152
Load Working Set - P90 (MB)	147	140	140	147	146
First Request (ms)	84	88	83	82	89
Requests/sec	198,655	198,684	198,983	198,452	199,499

The working set is definitely lower, but it's in the grey zone between "noise level change" and "meaningful improvement" (it's neither of those). There's an overlap between worst case of Dehydrate=Off and best case of Dehydrate=On.

The size regression is pretty meaningful though:

	Dehydrate=On	Dehydrate=Off
BasicMinimalApi	8,954	10,347
TodosApi	22,701	26,680

I'm starting to be not too sure about tying this into OptimizationPreference=Size. But also don't like the idea of adding weird knobs.