NrbfReader tests not working in WASM/mobile

[adamsitnik]

In #104104 I've tried to re-enable all BF tests and got following error for WASM/Mobile builds:

[00:49:29] info: [FAIL] System.Formats.Nrbf.Tests.ArraySinglePrimitiveRecordTests.CanReadArrayOfAnySize_Bool(size: 127, canSeek: False)
[00:49:29] info: System.TypeInitializationException : The type initializer for 'System.Runtime.Serialization.Formatters.Binary.Converter' threw an exception.
[00:49:29] info: ---- System.IO.FileNotFoundException :
[00:49:29] info:    at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.GetAssemblyId(WriteObjectInfo )
[00:49:29] info:    at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.Serialize(Object , BinaryFormatterWriter )
[00:49:29] info:    at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Serialize(Stream , Object )
[00:49:29] info:    at System.Formats.Nrbf.Tests.ReadTests.Serialize[Boolean[]](Boolean[] instance)
[00:49:29] info:    at System.Formats.Nrbf.Tests.ArraySinglePrimitiveRecordTests.Test[Boolean](Int32 size, Boolean canSeek)
[00:49:29] info:    at System.Object.InvokeStub_ArraySinglePrimitiveRecordTests.CanReadArrayOfAnySize_Bool(Object , Span`1 )

Most likely it's trying to load "mscorlib" which is not available for these platforms:

runtime/src/libraries/System.Runtime.Serialization.Formatters/src/System/Runtime/Serialization/Formatters/Binary/Converter.cs

Line 38 in d6f4bee

internal static readonly Assembly s_urtAssembly = Assembly.Load("mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089");

Prior to #103255 BF would simply throw PNSE for them:

runtime/src/libraries/System.Runtime.Serialization.Formatters/src/System.Runtime.Serialization.Formatters.csproj

Line 3 in a82225e

<TargetFrameworks>$(NetCoreAppCurrent);$(NetCoreAppCurrent)-browser;$(NetCoreAppCurrent)-ios;$(NetCoreAppCurrent)-tvos;$(NetCoreAppCurrent)-android</TargetFrameworks>

runtime/src/libraries/System.Runtime.Serialization.Formatters/src/System.Runtime.Serialization.Formatters.csproj

Lines 77 to 79 in a82225e

	<ItemGroup Condition="'$(TargetPlatformIdentifier)' != ''">
	<Compile Include="System\Runtime\Serialization\Formatters\Binary\BinaryFormatter.PlatformNotSupported.cs" />
	</ItemGroup>

@bartonjs should we change the OOB package to keep throwing PNSE for WASM/Mobile?

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-runtime
See info in area-owners.md if you want to be subscribed.

[jkotas]

Most likely it's trying to load "mscorlib" which is not available for these platforms

mscorlib is available for these platforms.

I think that the problem that you are seeing is that these tests have trimming enabled when running on mobile/WASM, but binary serialization is incompatible with trimming. The trimming strips mscorlib since it does not see it used anywhere. I expect that you would see similar problems if you enabled these tests e.g. on native AOT on any platforms.

[eiriktsarpalis]

@adamsitnik @bartonjs If I'm understanding the failing stack trace correctly, it is not product code from NrbfReader that is failing in these platforms but BinaryFormatter which is being used to generate test inputs. I see that the test class is already conditioned on BinaryFormatter being supported, so presumably some platforms are being missed out in the relevant platform detection logic?

Assuming NrbfReader is supported in trimmed/AOT applications perhaps a better solution longer term is to precompute the blobs and store them as files in the test project.

[adamsitnik]

If I'm understanding the failing stack trace correctly, it is not product code from NrbfReader that is failing in these platforms but BinaryFormatter which is being used to generate test inputs. I see that the test class is already conditioned on BinaryFormatter being supported, so presumably some platforms are being missed out in the relevant platform detection logic?

That is correct. IMO we should make sure that these tests are disabled on WASM/Mobile but also change the new OOB package to throw PNS rather than FileNotFound.

[eiriktsarpalis]

Unless I'm missing something the detection logic already excludes these platforms:

runtime/src/libraries/Common/tests/TestUtilities/System/PlatformDetection.cs

Lines 734 to 737 in 0f2ce10

	else if (IsNativeAot || IsBrowser || IsMobile)
	{
	return false;
	}

cc @dotnet/dotnet-webassembly-admin

[adamsitnik]

Unless I'm missing something the detection logic already excludes these platforms:

It does. But I am not sure if the line that follows this if block won't try to load the referenced type anyway (and run the static ctor and produce the FileNotFound exception wrapped by TypeInitializationException).

runtime/src/libraries/Common/tests/TestUtilities/System/PlatformDetection.cs

Line 739 in 0f2ce10

Assembly assembly = typeof(System.Runtime.Serialization.Formatters.Binary.BinaryFormatter).Assembly;

@eiriktsarpalis I may be paranoid here and it may be working as expected, just sharing my thoughts here.

[adamsitnik]

System.Formats.Nrbf.Tests is not listed here, so the condition most likely works as expected:

runtime/src/libraries/tests.proj

Lines 437 to 440 in c4624d0

	<ItemGroup Condition="'$(TestNativeAot)' == 'true' or '$(TargetsMobile)' == 'true'">
	<ProjectExclusions Include="$(MSBuildThisFileDirectory)System.Resources.Extensions\tests\BinaryFormatTests\System.Resources.Extensions.BinaryFormat.Tests.csproj" />
	<ProjectExclusions Include="$(MSBuildThisFileDirectory)System.Resources.Extensions\tests\CompatTests\System.Resources.Extensions.Compat.Tests.csproj" />
	</ItemGroup>

[eiriktsarpalis]

I think I misunderstood the issue. It's not that it's failing in CI but rather it was failing when you were working on the tests and you then got disabled in ec45dad. I don't think there is much that we can do for .NET 9. In the future can try to enable the tests in those platforms by precomputing the BF blobs ahead of time.