Using System.Text.Json, IAsyncEnumerable, and source generation together doesn't work

[richlander]

I'm using .NET 6 RC2.

using System.Text.Json;
using System.Text.Json.Serialization;

using Stream stream = Console.OpenStandardOutput();
Measurements data = new(DateTime.Now, GetValues(10));
await JsonSerializer.SerializeAsync<Measurements>(stream, data, JsonContext.Default.Measurements);
Console.WriteLine();

static async IAsyncEnumerable<int> GetValues(int n)
{
    for (int i = 0; i < n; i++) yield return Random.Shared.Next(i);
}
    
internal record Measurements(DateTime time, IAsyncEnumerable<int> Values);

// Source generator definition
[JsonSerializable(typeof(Measurements))]
internal partial class JsonContext : JsonSerializerContext
{
}

This code produces the following result:

{"time":"2021-08-26T15:08:56.93747-07:00","Values":{}}

If I switch out the JsonSerializer call to the following, I get the following desired result.

await JsonSerializer.SerializeAsync<Measurements>(stream, data);

and

{"time":"2021-08-26T15:07:58.698976-07:00","Values":[0,0,0,1,0,3,1,3,0,1]}

Is this scenario unsupported or I'm doing something wrong?

Tagging subscribers to this area: @eiriktsarpalis, @layomia
See info in area-owners.md if you want to be subscribed.

Issue Details

---

I'm using .NET 6 RC2.

using System.Text.Json;
using System.Text.Json.Serialization;

using Stream stream = Console.OpenStandardOutput();
Measurements data = new(DateTime.Now, GetValues(10));
await JsonSerializer.SerializeAsync<Measurements>(stream, data, JsonContext.Default.Measurements);
Console.WriteLine();

static async IAsyncEnumerable<int> GetValues(int n)
{
    for (int i = 0; i < n; i++) yield return Random.Shared.Next(i);
}
    
internal record Measurements(DateTime time, IAsyncEnumerable<int> Values);

// Source generator definition
[JsonSerializable(typeof(Measurements))]
internal partial class JsonContext : JsonSerializerContext
{
}

This code produces the following result:

{"time":"2021-08-26T15:08:56.93747-07:00","Values":{}}

If I switch out the JsonSerializer call to the following, I get the following result.

await JsonSerializer.SerializeAsync<Measurements>(stream, data);

and

{"time":"2021-08-26T15:07:58.698976-07:00","Values":[0,0,0,1,0,3,1,3,0,1]}

Is this scenario unsupported or I'm doing something wrong?

Author:	richlander
Assignees:	-
Labels:	area-System.Text.Json, untriaged
Milestone:	-

[eiriktsarpalis]

Related to #53393. We discussed implementing this but ultimately didn't get around to doing it. @layomia @steveharter would it make sense to try and get this in for RC2? Note that it would require new APIs.

[richlander]

I am not asking for a new .NET 6 feature at this point. This seemed like a natural scenario to try. I suggest we document it as non-functional for .NET 6 and re-consider for 7.

[eiriktsarpalis]

Triage: let's keep in 6.0.0 and investigate potential fallbacks. We probably should not add new JsonMetadataServices APIs in 6.0.0.

[richlander]

As an aside, this pattern wasn't immediately intuitive to me. I started out with this:

internal record Measurements(DateTime time, int[] Values);

Naturally, that didn't work with this:

Measurements data = new(DateTime.Now, GetValues(10));
static async IAsyncEnumerable<int> GetValues(int n)
{
    for (int i = 0; i < n; i++) yield return Random.Shared.Next(i);
}

Normally, we don't model POCOs with either async or IEnumerable. Once the data is realized, there is no need or desire for these mechanisms. I suspect that my problem is that I conflated the types for T and data into one. I suspect that I won't be the first person to run into this modeling problem. It probably deserves some guidance. This challenge would be more obvious if I was using the same POCO for both serialization and deserialization.

[steveharter]

Related to #53393. We discussed implementing this but ultimately didn't get around to doing it. @layomia @steveharter would it make sense to try and get this in for RC2? Note that it would require new APIs.

Supporting IAsyncEnumerable requires additional public API(s) since the implementation of IAsyncEnumerable is done via a custom converter, like other collection types, and helper method(s) for source-gen would need to be added. However these helpers are just used by the generated code and not generally visible\usable by the consumer.

Something like this (following the pattern used to support other collection types):

namespace System.Text.Json.Serialization.Metadata
{
    public static partial class JsonMetadataServices
    {
+                public static JsonTypeInfo<IAsyncEnumerable> CreateIAsyncEnumerable(System.Text.Json.JsonSerializerOptions options, System.Func<IAsyncEnumerable>? createObjectFunc, System.Text.Json.Serialization.Metadata.JsonTypeInfo elementInfo, System.Text.Json.Serialization.JsonNumberHandling numberHandling, System.Action<Utf8JsonWriter, IAsyncEnumerable>? serializeFunc) { throw null; }
    }
}

Note that although the standard SerializeAsync<T>(Stream) methods work with T being IAsyncEnumerable, when deserializing the explicit DeserializeAsyncEnumerable() must be used, and currently there is not an overload for JsonTypeInfo. Thus if we fix the serialize side, there isn't a way to deserialize without adding additional public DeserializeAsyncEnumerable overloads that takeJsonTypeInfo.

Options for V6:

• Add the support for serialize, but not deserialize.
• Add support for both serialize and deserialize
• Throw NotSupportedException when serializing\deserializing IAsyncEnumerable for source-gen cases.
• Do nothing (silent failure not a good thing)
• ...?

[eiriktsarpalis]

Good point about IAE deserialization requiring special treatment. Throwing NotSupportedException (or perhaps emitting a compile-time error) seems like a reasonable measure for 6.0.0.

[richlander]

Throwing seems like the best approach for 6. It is worth taking that fix IMO.