net9.0-ios regression causing `InvalidProgramException` in certain circumstances

[rdavisau]

Description

Regression introduced in .net9 preview 5, and still present in rc1. On iOS, certain code that worked on .net9 preview 4 and all previous versions of .net now crashes with InvalidProgramException in release builds.

Triggering code is a path of DryIoc's interpreter. From what I could isolate, it occurs in these circumstances:

• multiple singleton service types (A, B), depend on another singleton type (Z)
• the Z type is injected lazily (Lazy<Z> constructor parameter)
• DryIoC is configured to use its own interpreter
• DryIoC attempts to resolve one of the service types (e.g. B), after having previously resolved the other service type (e.g. A)

Circumstances that don't cause it:

• Z type is injected directly, not lazily
• Only one of the service types is resolved, including repeated resolutions of the same type.

Tagging @dadhi the DryIoC library author who may be able to provide information on what specific parts of the interpreter are exercised in this "resolution of second service injecting lazy dependency" case that differ from the cases that don't trigger the crash.

Reproduction Steps

• be running .net9 p5 or later, with the ios workload installed
• have physical ios device plugged in
• clone https://github.com/rdavisau/ios-net9-invalidprogram-repro
• run ./net9.sh {Your Physical Device UDID}

Expected behavior

Application launches and remains running with log output:

Resolve a service that depends on Lazy<Thing> once
Ok
Resolve the same service that depends on Lazy<Thing> a second time
Ok
Resolve a different service that depends on Lazy<Thing>
Ok

Actual behavior

Application launches and quickly crashes, with log output:

Resolve a service that depends on Lazy<Thing> once
Ok
Resolve the same service that depends on Lazy<Thing> a second time
Ok
Resolve a different service that depends on Lazy<Thing>

Terminating app due to uncaught exception 'System.InvalidProgramException', reason: ' (System.InvalidProgramException) 
at DryIoc.Interpreter.TryInterpretSingletonAndUnwrapContainerException(IResolverContext r, Expression expr, ImMapEntry`1 itemRef, Object& result) 
at DryIoc.Factory.ApplyReuse(Expression serviceExpr, Request request) at DryIoc.Factory.GetExpressionOrDefault(Request request) 
at DryIoc.Container.ResolveAndCache(Int32 serviceTypeHash, Type serviceType, IfUnresolved ifUnresolved) 
at DryIoc.Container.DryIoc.IResolver.Resolve(Type serviceType, IfUnresolved ifUnresolved) 
at DryIoc.Resolver.Resolve[IServiceB](IResolver resolver, IfUnresolved ifUnresolved) 
at InvalidProgram.AppDelegate.FinishedLaunching(UIApplication application, NSDictionary launchOptions) 
at InvalidProgram.AppDelegate.Registrar_Callbacks.callback_2_InvalidProgram_AppDelegate_FinishedLaunching(IntPtr pobj, IntPtr sel, IntPtr p0, IntPtr p1, IntPtr* exception_gchandle) '

(sometimes, app may crash before exception details are written to console)

Regression?

Yes. This was working in .net9 preview 4 and earlier stables.
Introduced in .net9 preview 5 and still present in .net9 rc1.

Known Workarounds

No response

Configuration

dotnet --info

.NET SDK:
 Version:           9.0.100-rc.1.24452.12
 Commit:            81a714c6d3
 Workload version:  9.0.100-manifests.b772b23d
 MSBuild version:   17.12.0-preview-24422-09+d17ec720d

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  14.6
 OS Platform: Darwin
 RID:         osx-arm64
 Base Path:   /usr/local/share/dotnet/sdk/9.0.100-rc.1.24452.12/

.NET workloads installed:
Configured to use loose manifests when installing new manifests.
 [ios]
   Installation Source: SDK 9.0.100-rc.1
   Manifest Version:    17.5.9270-net9-rc1/9.0.100-rc.1
   Manifest Path:       /usr/local/share/dotnet/sdk-manifests/9.0.100-rc.1/microsoft.net.sdk.ios/17.5.9270-net9-rc1/WorkloadManifest.json
   Install Type:        FileBased

 [maui]
   Installation Source: SDK 9.0.100-rc.1
   Manifest Version:    9.0.0-rc.1.24453.9/9.0.100-rc.1
   Manifest Path:       /usr/local/share/dotnet/sdk-manifests/9.0.100-rc.1/microsoft.net.sdk.maui/9.0.0-rc.1.24453.9/WorkloadManifest.json
   Install Type:        FileBased


Host:
  Version:      9.0.0-rc.1.24431.7
  Architecture: arm64
  Commit:       static

.NET SDKs installed:
  6.0.417 [/usr/local/share/dotnet/sdk]
  7.0.404 [/usr/local/share/dotnet/sdk]
  8.0.100 [/usr/local/share/dotnet/sdk]
  8.0.204 [/usr/local/share/dotnet/sdk]
  9.0.100-preview.3.24204.13 [/usr/local/share/dotnet/sdk]
  9.0.100-preview.4.24267.66 [/usr/local/share/dotnet/sdk]
  9.0.100-preview.5.24307.3 [/usr/local/share/dotnet/sdk]
  9.0.100-preview.6.24328.19 [/usr/local/share/dotnet/sdk]
  9.0.100-preview.7.24407.12 [/usr/local/share/dotnet/sdk]
  9.0.100-rc.1.24452.12 [/usr/local/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.25 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.14 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 8.0.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 8.0.4 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 9.0.0-preview.3.24172.13 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 9.0.0-preview.4.24267.6 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 9.0.0-preview.5.24306.11 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 9.0.0-preview.6.24328.4 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 9.0.0-preview.7.24406.2 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 9.0.0-rc.1.24452.1 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 6.0.25 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.14 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 8.0.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 8.0.4 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 9.0.0-preview.3.24172.9 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 9.0.0-preview.4.24266.19 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 9.0.0-preview.5.24306.7 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 9.0.0-preview.6.24327.7 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 9.0.0-preview.7.24405.7 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 9.0.0-rc.1.24431.7 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

Other architectures found:
  None

Environment variables:
  Not set

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download

Other information

No response

[ivanpovazan]

Hello @rdavisau
I have a question about the repro project attached to the issue.
It seems like you overwriting default configuration for app trimming (like forcing TrimMode=full):
https://github.com/rdavisau/ios-net9-invalidprogram-repro/blob/cca4dae68e8f2abdb2b7ab9bad01db60d6d8d78b/InvalidProgram-net9.csproj#L26

I was wondering what is the reason for overwriting the defaults and if the issue is reproducible if you use the default settings (so we can rule out potential trimming problem)?

[rdavisau]

Thanks @ivanpovazan

There were various extra properties set in the reproduction that mirror the real project I observed the issue in. I have removed as many customisations as possible while being able to continue to reproduce the issue. I found two things:

• different combinations of mtouchlink/trimmode did not appear to affect the crash (full, partial, sdk only etc.)
• from the current state of the repro, which now only includes a <PublishTrimmed>true</PublishTrimmed> element, you can fix the issue by removing the PublishedTrimmed element.

The second point is surprising to me, since we can't specify <PublishTrimmed>false</PublishTrimmed>, so I suppose that the inclusion of PublishTrimmed at all causes some other properties to be set? In this case it seems it could be trimming-related after all.

[vitek-karas]

I tried this without trying to run the app, just to see the difference in trimming.
The main difference in the input to the trimmer are these feature switches (this is without PublishTrimmed -> with PublishTrimmed):

--feature Microsoft.Extensions.DependencyInjection.VerifyOpenGenericServiceTrimmability false->true
--feature System.ComponentModel.DefaultValueAttribute.IsSupported ->false
--feature System.ComponentModel.Design.IDesignerHost.IsSupported ->false
--feature System.Resources.UseSystemResourceKeys true->false
--feature System.Runtime.CompilerServices.RuntimeFeature.IsDynamicCodeSupported false->true
--feature System.Runtime.InteropServices.Marshalling.EnableGeneratedComInterfaceComImportInterop ->false

The rest of the command line is basically identical. Specifically IsDynamicCodeSupported and (for DI especially) VerifyOpenGenericServiceTrimmability might affect the app.
I don't have a device to test the app on, but it would be really interesting to try to set these feature switches one by one to see which one causes the problem. From there it should be much easier to figure out the trimming difference.

[rdavisau]

I can go through the different combinations, but I’m not sure how to override the feature switches 💭

[ivanpovazan]

The problem we are having seems to be a mismatch in the feature switch configuration when PublishTrimmed property is set on a project level (or forced) or resolved by the SDK:

• if it is not set on the project level, we will use feature switch configuration as set in the Xamarin SDK early in Xamarin.Shared.Sdk.targets:
  https://github.com/xamarin/xamarin-macios/blob/ec50934897ef1b98c1cd73ca780cdf5a3750faaa/dotnet/targets/Xamarin.Shared.Sdk.targets#L121-L158
  The SDK later sets: PublishTrimmed to true so that we run the linker in:
  https://github.com/xamarin/xamarin-macios/blob/ec50934897ef1b98c1cd73ca780cdf5a3750faaa/dotnet/targets/Xamarin.Shared.Sdk.targets#L298

• if it is set on project level, Microsoft.NET.ILLink.targets will set its own feature switch configuration in:

  runtime/src/tools/illink/src/ILLink.Tasks/build/Microsoft.NET.ILLink.targets

  Lines 34 to 61 in a16e0d5

  	<PropertyGroup Condition="'$(PublishTrimmed)' == 'true'">
  	<StartupHookSupport Condition="'$(StartupHookSupport)' == ''">false</StartupHookSupport>
  	<CustomResourceTypesSupport Condition="'$(CustomResourceTypesSupport)' == ''">false</CustomResourceTypesSupport>
  	<EnableUnsafeBinaryFormatterInDesigntimeLicenseContextSerialization Condition="'$(EnableUnsafeBinaryFormatterInDesigntimeLicenseContextSerialization)' == ''">false</EnableUnsafeBinaryFormatterInDesigntimeLicenseContextSerialization>
  	<EnableUnsafeBinaryFormatterSerialization Condition="'$(EnableUnsafeBinaryFormatterSerialization)' == ''">false</EnableUnsafeBinaryFormatterSerialization>
  	<EnableUnsafeUTF7Encoding Condition="'$(EnableUnsafeUTF7Encoding)' == ''">false</EnableUnsafeUTF7Encoding>
  	<BuiltInComInteropSupport Condition="'$(BuiltInComInteropSupport)' == ''">false</BuiltInComInteropSupport>
  	<!-- GeneratedComInterface/ComImport interop requires built-in COM -->
  	<EnableGeneratedComInterfaceComImportInterop Condition="'$(BuiltInComInteropSupport)' == 'false'">false</EnableGeneratedComInterfaceComImportInterop>
  	<EnableGeneratedComInterfaceComImportInterop Condition="'$(EnableGeneratedComInterfaceComImportInterop)' == ''">false</EnableGeneratedComInterfaceComImportInterop>
  	<AutoreleasePoolSupport Condition="'$(AutoreleasePoolSupport)' == ''">false</AutoreleasePoolSupport>
  	<EnableCppCLIHostActivation Condition="'$(EnableCppCLIHostActivation)' == ''">false</EnableCppCLIHostActivation>
  	<!-- C++/CLI activation requires native hosting -->
  	<_EnableConsumingManagedCodeFromNativeHosting Condition="'$(EnableCppCLIHostActivation)' == 'true'">true</_EnableConsumingManagedCodeFromNativeHosting>
  	<_EnableConsumingManagedCodeFromNativeHosting Condition="'$(_EnableConsumingManagedCodeFromNativeHosting)' == ''">false</_EnableConsumingManagedCodeFromNativeHosting>
  	<VerifyDependencyInjectionOpenGenericServiceTrimmability Condition="'$(VerifyDependencyInjectionOpenGenericServiceTrimmability)' == ''">true</VerifyDependencyInjectionOpenGenericServiceTrimmability>
  	<JsonSerializerIsReflectionEnabledByDefault Condition="'$(JsonSerializerIsReflectionEnabledByDefault)' == ''">false</JsonSerializerIsReflectionEnabledByDefault>
  	<!-- Linux Bionic doesn't ship GSSAPI, so enable managed implementation -->
  	<_UseManagedNtlm Condition="'$(_UseManagedNtlm)' == '' and $(RuntimeIdentifier.StartsWith('linux-bionic'))">true</_UseManagedNtlm>
  	<!-- Trim managed NTLM on Linux when it's not explicitly requested -->
  	<_UseManagedNtlm Condition="'$(_UseManagedNtlm)' == '' and $(RuntimeIdentifier.StartsWith('linux'))">false</_UseManagedNtlm>
  	<_ComObjectDescriptorSupport Condition="'$(_ComObjectDescriptorSupport)' == ''">false</_ComObjectDescriptorSupport>
  	<_DesignerHostSupport Condition="'$(_DesignerHostSupport)' == ''">false</_DesignerHostSupport>
  	<_DefaultValueAttributeSupport Condition="'$(_DefaultValueAttributeSupport)' == ''">false</_DefaultValueAttributeSupport>
  	<DynamicCodeSupport Condition="'$(DynamicCodeSupport)' == ''">true</DynamicCodeSupport>
  	<UseSystemResourceKeys Condition="'$(UseSystemResourceKeys)' == ''">false</UseSystemResourceKeys>
  	<_DataSetXmlSerializationSupport Condition="'$(_DataSetXmlSerializationSupport)' == ''">false</_DataSetXmlSerializationSupport>
  	</PropertyGroup>

  as it is imported before Xamarin.Shared.Sdk.targets

This is specifically problematic in case of for example: DynamicCodeSupport , which should not be true on iOS platforms if interpreter is not enabled.

So I suggest as a workaround to not set manually PublishTrimmed=true on a project level and we should look into how to properly align the feature switch configuration.

/cc: @rolfbjarne do you have any other suggestion?

[rolfbjarne]

My suggestion would be to not set the PublishTrimmed defaults (or any other trimmer/linker default really) until right before they're needed (i.e. in a target), that way we (xamarin-macios) can provide different defaults earlier in the build.