Skip to content
This repository was archived by the owner on Jan 23, 2023. It is now read-only.

Add extension data support to Json serializer#37578

Closed
steveharter wants to merge 11 commits intodotnet:masterfrom
steveharter:Overflow
Closed

Add extension data support to Json serializer#37578
steveharter wants to merge 11 commits intodotnet:masterfrom
steveharter:Overflow

Conversation

@steveharter
Copy link
Copy Markdown
Contributor

@steveharter steveharter commented May 10, 2019

Adds the overflow or data extension support via the new JsonExtensionDataAttribute

Other changes:

Open issues:

  • For both here, and for the object->JsonElement support (already in) we will likely need to pass the JsonSerializerOptions to the JsonDocument and have the JsonDocument use the JsonSerializerOptions reader options (pretty printing, comment support, etc), plus two possible serializer-only options:
    • IgnoreNullValues.
    • A property naming policy, such as a new ExtensionDataPropertyNaming property.
  • This does not support Stream scenarios yet where the current buffer may run out of data. This will be addressed along with similar cases where this is encountered including the object->JsonElement converter and the extensibility model where the reader is exposed.

@steveharter steveharter requested a review from ahsonkhan May 10, 2019 00:49
@steveharter steveharter requested a review from layomia May 10, 2019 00:49
@steveharter steveharter changed the title Add data extension support Add extension data support to Json serializer May 10, 2019
@ahsonkhan
Copy link
Copy Markdown

/azp run corefx-ci

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

@steveharter
Copy link
Copy Markdown
Contributor Author

cc @rynowak @JamesNK

@ahsonkhan
Copy link
Copy Markdown

  • For both here, and for the object->JsonElement support (already in) we will likely need to pass the JsonSerializerOptions to the JsonDocument and have the JsonDocument use the JsonSerializerOptions reader options (pretty printing, comment support, etc), plus two possible serializer-only options:
    • IgnoreNullValues.
    • A property naming policy, such as a new ExtensionDataPropertyNaming property.

cc @bartonjs

@ahsonkhan
Copy link
Copy Markdown

Refactors the "ignore" code and some tests added.

cc @MarcoRossignoli since this may affect your PR: #37586

Comment thread src/System.Text.Json/src/System/Text/Json/Serialization/JsonClassInfo.cs Outdated

string keyName = Encoding.UTF8.GetString(unescapedPropertyName);

// Currently we don't apply any naming policy. If we do, we'd have to pass it onto the JsonDocument.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So there are two different things that can have a naming policy applied to it.

  1. The dictionary key
  2. The JSON content

Json.NET has an extension point for changing the dictionary key. On the other hand the Json.NET serializer doesn't modify the extension data content. An approach I took with the serializer is JToken (i.e. JsonElement) is already JSON, and the serializer always writes it unchanged.

Copy link
Copy Markdown

@ahsonkhan ahsonkhan May 11, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the other hand the Json.NET serializer doesn't modify the extension data content. An approach I took with the serializer is JToken (i.e. JsonElement) is already JSON, and the serializer always writes it unchanged.

I am fine with this and it would remove the need to pass in the serializer options to the JsonDocument.

@rynowak, @KrzysztofCwalina - what do you think? Is it necessary for the "overflow" data to honor the serializer options?

@JamesNK, what about the read/deserialize side of things or is that not relevant here?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's the settings not on the reader\writer but could be applied:

  • Naming on serialize (not in reader options, plus would be a new option on JsonSerializerOptions)
  • IgnoreNulls (not in reader options)

Here's the settings on reader\writer options that could applied (they should be applied IMO but currently aren't):

  • AllowTrailingCommas
  • MaxDepth
  • ReadCommentHandling
  • WriteIndented

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now, pending future discussion, I will pass in the JsonReaderOptions\JsonWriterOptions like we do for other calls into the reader\writer. I will not add any support for property naming or IgnoreNull at this time.

Comment thread src/System.Text.Json/src/Resources/Strings.resx Outdated
Comment thread src/System.Text.Json/src/System/Text/Json/Serialization/JsonClassInfo.cs Outdated
Comment thread src/System.Text.Json/src/System/Text/Json/Serialization/JsonClassInfo.cs Outdated
Comment thread src/System.Text.Json/src/System/Text/Json/Serialization/JsonClassInfo.cs Outdated
Comment thread src/System.Text.Json/src/System/Text/Json/Serialization/JsonClassInfo.cs Outdated
Comment thread src/System.Text.Json/src/System/Text/Json/Serialization/ReadStackFrame.cs Outdated
@ahsonkhan
Copy link
Copy Markdown

Other changes

In the future, can we make sure that the PRs are isolated to a single change? Refactoring/bug fixes should be in separate PRs to make it easier to review each independently. Reviewing the ExtensionData in isolation was a bit difficult, especially when some of the changes involved indentation only.

@steveharter
Copy link
Copy Markdown
Contributor Author

In the future, can we make sure that the PRs are isolated to a single change?...especially when some of the changes involved indentation only.

The other changes were related except for some of the "ignore" changes - however even that the new code I added needed to use ignore and refactor a bit to share that ignore logic with the new dictionary and existing object code.

For the indent, I try to do the indent\moving code on the second commit, which I did partially here.

@steveharter steveharter self-assigned this May 14, 2019
@steveharter
Copy link
Copy Markdown
Contributor Author

Closing and replacing this with #37690 due to issues with corrupt branch history.

@steveharter steveharter deleted the Overflow branch May 16, 2019 00:49
@karelz karelz added this to the 3.0 milestone May 22, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Deserializer should support only getter for IDictionary properties

6 participants