JsonSerializer falls into cycle

[Arthi93]

I try to run my API call in order to verify Google reCaptcha

`

        var content = new FormUrlEncodedContent(new[]
            {
                new KeyValuePair<string, string>("secret", Secret),
                new KeyValuePair<string, string>("response", Token)
            }); 

        using (var client = new HttpClient()) {

            var response = await client.PostAsync("https://www.google.com/recaptcha/api/siteverify", content);
            if (response.IsSuccessStatusCode)
            {

                string JSONres = response.Content.ReadAsStringAsync().Result;
                dynamic result = JObject.Parse(JSONres);

                return result.success;

            }
            else return false;

        }`

But i cant serialize ANY json with .net Core 3.0. I got following Exception returning 500 to my client:

System.Text.Json.JsonException: A possible object cycle was detected which is not supported. This can either be due to a cycle or if the object depth is larger than the maximum allowed depth of 32. at System.Text.Json.ThrowHelper.ThrowInvalidOperationException_SerializerCycleDetected(Int32 maxDepth) at System.Text.Json.JsonSerializer.Write(Utf8JsonWriter writer, Int32 originalWriterDepth, Int32 flushThreshold, JsonSerializerOptions options, WriteStack& state) at System.Text.Json.JsonSerializer.WriteAsyncCore(Stream utf8Json, Object value, Type inputType, JsonSerializerOptions options, CancellationToken cancellationToken) at Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonOutputFormatter.WriteResponseBodyAsync(OutputFormatterWriteContext context, Encoding selectedEncoding) at Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonOutputFormatter.WriteResponseBodyAsync(OutputFormatterWriteContext context, Encoding selectedEncoding) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeResultAsync>g__Logged|21_0(ResourceInvoker invoker, IActionResult result) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResultFilterAsync>g__Awaited|29_0[TFilter,TFilterAsync](ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResultExecutedContextSealed context) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.ResultNext[TFilter,TFilterAsync](State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeResultFilters() --- End of stack trace from previous location where exception was thrown --- at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|19_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker) at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger) at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

[Youssef1313]

I think it might be helpful to put an example of the string JSONres. I'd also suggest to see if this helps.

[Arthi93]

It will throw this exception for literally every string. If I set

Secret = "qwerty"
Token = "asd"

it will also throw this exception.

[Arthi93]

I had to configure Newtonsoft.Json options like this:

services.AddControllers().AddNewtonsoftJson(options =>
            {
                options.SerializerSettings.ContractResolver = new DefaultContractResolver();
                options.SerializerSettings.ReferenceLoopHandling = ReferenceLoopHandling.Ignore;
            });

and than magically it started to work. I had to install Microsoft.AspNetCore.Mvc.NewtonsoftJson in order to make it work.

Still there is an issue with System.Text.Json pointed in this answer:

https://stackoverflow.com/a/58517316/7690242

[ahsonkhan]

@Arthi93 - cycles in the object graph are currently not supported in System.Text.Json (for 3.1).
We are working on adding that feature. See this issue and spec:
https://github.com/dotnet/corefx/issues/41002
#354

One thing I was wondering about your workaround. Why did you choose ReferenceLoopHandling.Ignore;?

Would PreserveReferenceHandling.All work for your scenario? https://www.newtonsoft.com/json/help/html/T_Newtonsoft_Json_PreserveReferencesHandling.htm

I am asking because the semantics/requirements of Ignore are not clear for us (and serialization becomes lossy in that mode), which is why we decided against implementing that and are only going to provide a "preserve" mode.

@jozkee - here's a scenario that might be worth testing.

@Arthi93 - can you please share a small app that repros the error (including more info on the type FormUrlEncodedContent).

[Arthi93]

@ahsonkhan - thanks for your answer. I choose ReferenceLoopHandling.Ignore because it allows me to return my objects intact with only specific references specified by .Include() statement, while using PreserveReferenceHandling.All ended with differently serialized structure of my responses. The good part about PreserveReferenceHandling.All is, that it keeps information about deeply nested references even the cyclic ones, but still - it affects my responses. Here is an example:

Using this code: return _ctx.Events.Include(e => e.Client).Where(e => e.Id == id).Single();

ReferenceLoopHandling.Ignore returns this json: https://codebeautify.org/online-json-editor/cb6604cb

PreserveReferenceHandling.All returns this json https://codebeautify.org/online-json-editor/cbae39c1

As you can see, ignoring references loops ends with broken connection between client and event - only one reference level works (client has many events, event has one client), sol i have an event, with a client, but this client has no events (broken reference).

Preserving reference handing keeps the reference as an ID not to fall into a loop, but als alters the JSON structure

[jozkee]

@Arthi93 But you can still read back the JSON created with PreserveReferenceHandling.All and keep the integrity of the information.
May I ask, what is the problem with changing the JSON structure? Are you expecting that other serializer reads your payload?

Also you can use PreserveReferencesHandling.Objects to avoid the JSON array structure to change.

[ahsonkhan]

As you can see, ignoring references loops ends with broken connection between client and event - only one reference level works (client has many events, event has one client), sol i have an event, with a client, but this client has no events (broken reference).

Exactly, so why use ignore? Is round-tripping with full fidelity not relevant/important in your use case? I am asking not to push-back on the use of ignore, but to more deeply understand why folks end up using Ignore (along with pros/cons of doing so) and why having support for only Preserve may not be enough (which is our current plan for System.Text.Json). Is the concern around readability? Size? Change of schema? Ignore works good enough and lossy transformation isn't a concern (essentially, in your case, is Ignore actually the right/desired thing to do)?

Preserving reference handing keeps the reference as an ID not to fall into a loop, but als alters the JSON structure

As @jozkee asked, how does the JSON structure being altered affect your scenario? Essentially, metadata is being injected into the JSON payload to maintain the reference/loop information so that it can round-trip.

What is your .NET object graph look like that corresponds to the example JSON you are trying to deserialize into? Where in the object do you end up with a cycle?

[jozkee]

The preview version for ReferenceHandling feature is here.

• Download the System.Text.Json preview package version 5.0.0-alpha.1.20071.1 from dotnet5 gallery (through Visual Studio's NuGet client):
  https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5/nuget/v3/index.json

• How to use it:
  Set ReferenceHandling.Preserve to ReferenceHandling property in JsonSerializerOptions.

var options = new JsonSerializerOptions
{
    ReferenceHandling = ReferenceHandling.Preserve
};

string json = JsonSerializer.Serialize(objectWithLoops, options);

Here's the spec doc including samples and notes about many scenarios.
If you want to add feedback, feel free to leave a comment in https://github.com/dotnet/corefx/issues/41002.