[wasm] Perf benchmark failures - `System.Text.Json` serializer related failures

[radical]

Failing for AOT, and interpereter.

Benchmark: Json_ToStream<Location>.SystemTextJson_Reflection:

console.info: Initializing dotnet version 8.0.0-ci commit hash 36481bfd079093fe7d29ef024b1fb801e6c787fe
...
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.NotSupportedException: JsonTypeInfo metadata for type 'MicroBenchmarks.Serializers.Location' was not provided by TypeInfoResolver of type '[]'. If using source generation, ensure that all root types passed to the serializer have been annotated with 'JsonSerializableAttribute', along with any types that might be serialized polymorphically.
   at BenchmarkDotNet.Autogenerated.Runnable_7.Run(IHost host, String benchmarkName)
   at System.Reflection.MethodInvoker.Invoke(Object , IntPtr* , BindingFlags )
   --- End of inner exception stack trace ---
   at System.Reflection.MethodInvoker.Invoke(Object , IntPtr* , BindingFlags )
   at BenchmarkDotNet.Autogenerated.UniqueProgramName.AfterAssemblyLoadingAttached(String[] args)
// AfterAll
test-main.js exiting dba61141-3925-4bba-8dea-e01f32679898.dll --benchmarkName with result -1
MONO_WASM: {"name":"ExitStatus","message":"Program terminated with exit(-1)","status":-1}
console.info: WASM EXIT -1

Benchmark ColdStartSerialization<SimpleStructWithProperties>.NewCustomizedOptions:

console.info: Initializing dotnet version 8.0.0-ci commit hash 36481bfd079093fe7d29ef024b1fb801e6c787fe
...
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.InvalidOperationException: JsonSerializerOptions instance must specify a TypeInfoResolver setting before being marked as read-only.
   at System.Text.Json.Serialization.Tests.ColdStartSerialization`1[[MicroBenchmarks.Serializers.SimpleStructWithProperties, MicroBenchmarks, Version=42.42.42.42, Culture=neutral, PublicKeyToken=null]].Setup()
   at BenchmarkDotNet.Autogenerated.Runnable_142.Run(IHost host, String benchmarkName)
   at System.Reflection.MethodInvoker.Invoke(Object , IntPtr* , BindingFlags )
   --- End of inner exception stack trace ---
   at System.Reflection.MethodInvoker.Invoke(Object , IntPtr* , BindingFlags )
   at BenchmarkDotNet.Autogenerated.UniqueProgramName.AfterAssemblyLoadingAttached(String[] args)
// AfterAll
test-main.js exiting dba61141-3925-4bba-8dea-e01f32679898.dll --benchmarkName with result -1
MONO_WASM: {"name":"ExitStatus","message":"Program terminated with exit(-1)","status":-1}
console.info: WASM EXIT -1

Changes 60799cc...53a10ea ([20230707.3](https://dev.azure.com/dnceng/internal/_build/results?buildId=2216705) to https://dev.azure.com/dnceng/internal/_build/results?buildId=2217531&view=results). The pipeline had other failures earlier, so the starting hash could be a little off.

Sdk version changed from 8.0.100-preview.7.23357.5 to 8.0.100-preview.7.23359.2.
.. which corresponds to dotnet/sdk@f8336e3...cfa4244

dotnet/performance commit used for both was dotnet/performance@a719dec

Known Issue Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Report

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	0

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Failing for AOT, and interpereter.

Benchmark: Json_ToStream<Location>.SystemTextJson_Reflection:

console.info: Initializing dotnet version 8.0.0-ci commit hash 36481bfd079093fe7d29ef024b1fb801e6c787fe
...
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.NotSupportedException: JsonTypeInfo metadata for type 'MicroBenchmarks.Serializers.Location' was not provided by TypeInfoResolver of type '[]'. If using source generation, ensure that all root types passed to the serializer have been annotated with 'JsonSerializableAttribute', along with any types that might be serialized polymorphically.
   at BenchmarkDotNet.Autogenerated.Runnable_7.Run(IHost host, String benchmarkName)
   at System.Reflection.MethodInvoker.Invoke(Object , IntPtr* , BindingFlags )
   --- End of inner exception stack trace ---
   at System.Reflection.MethodInvoker.Invoke(Object , IntPtr* , BindingFlags )
   at BenchmarkDotNet.Autogenerated.UniqueProgramName.AfterAssemblyLoadingAttached(String[] args)
// AfterAll
test-main.js exiting dba61141-3925-4bba-8dea-e01f32679898.dll --benchmarkName with result -1
MONO_WASM: {"name":"ExitStatus","message":"Program terminated with exit(-1)","status":-1}
console.info: WASM EXIT -1

Benchmark ColdStartSerialization<SimpleStructWithProperties>.NewCustomizedOptions:

console.info: Initializing dotnet version 8.0.0-ci commit hash 36481bfd079093fe7d29ef024b1fb801e6c787fe
...
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.InvalidOperationException: JsonSerializerOptions instance must specify a TypeInfoResolver setting before being marked as read-only.
   at System.Text.Json.Serialization.Tests.ColdStartSerialization`1[[MicroBenchmarks.Serializers.SimpleStructWithProperties, MicroBenchmarks, Version=42.42.42.42, Culture=neutral, PublicKeyToken=null]].Setup()
   at BenchmarkDotNet.Autogenerated.Runnable_142.Run(IHost host, String benchmarkName)
   at System.Reflection.MethodInvoker.Invoke(Object , IntPtr* , BindingFlags )
   --- End of inner exception stack trace ---
   at System.Reflection.MethodInvoker.Invoke(Object , IntPtr* , BindingFlags )
   at BenchmarkDotNet.Autogenerated.UniqueProgramName.AfterAssemblyLoadingAttached(String[] args)
// AfterAll
test-main.js exiting dba61141-3925-4bba-8dea-e01f32679898.dll --benchmarkName with result -1
MONO_WASM: {"name":"ExitStatus","message":"Program terminated with exit(-1)","status":-1}
console.info: WASM EXIT -1

Changes 60799cc...53a10ea ([20230707.3](https://dev.azure.com/dnceng/internal/_build/results?buildId=2216705) to https://dev.azure.com/dnceng/internal/_build/results?buildId=2217531&view=results). The pipeline had other failures earlier, so the starting hash could be a little off.

Sdk version changed from 8.0.100-preview.7.23357.5 to 8.0.100-preview.7.23359.2.
.. which corresponds to dotnet/sdk@f8336e3...cfa4244

dotnet/performance commit used for both was dotnet/performance@a719dec

Author:	radical
Assignees:	-
Labels:	arch-wasm, blocking-clean-ci, perf-pipeline
Milestone:	-

[radical]

@eiriktsarpalis Any idea what might be wrong here?

[eiriktsarpalis]

Seems related to #88480. These benchmarks target the reflection serializer and thus should never be running in trimmed applications or AOT. Were they even working in AOT before the feature switch got turned on?

[lewing]

Seems related to #88480. These benchmarks target the reflection serializer and thus should never be running in trimmed applications or AOT. Were they even working in AOT before the feature switch got turned on?

why did you merge that pr on red?

[lewing]

opened #88612

[radical]

ColdStartSerialization - this one seems to have been around since April, and I haven't seen it fail before this.

[lewing]

yeah I was reading across too many failures apologies for the noise

[eiriktsarpalis]

@radical I just created this PR that should fix the issue. Can we trigger CI against it to validate?