stricter behavior for parse_json, add try_parse_json, remove to_json#12920
Merged
gianm merged 1 commit intoapache:masterfrom Aug 23, 2022
Merged
stricter behavior for parse_json, add try_parse_json, remove to_json#12920gianm merged 1 commit intoapache:masterfrom
gianm merged 1 commit intoapache:masterfrom
Conversation
Contributor
|
I have not had the pleasure yet of checking out this branch and playing with it but from the description provided it sounds great. Very excited for |
cryptoe
approved these changes
Aug 22, 2022
| public static class ParseJsonExprMacro implements ExprMacroTable.ExprMacro | ||
| { | ||
| public static final String NAME = "to_json_string"; | ||
| public static final String NAME = "parse_json"; |
Contributor
There was a problem hiding this comment.
Is there a doc update required for this ?
Member
Author
There was a problem hiding this comment.
no, docs in #12922 were written as if this PR were already merged, (to_json_string still exists, but to_json is removed and try_parse_json is added, so this is a result of funny diff stuff i think)
gianm
approved these changes
Aug 23, 2022
Contributor
gianm
left a comment
There was a problem hiding this comment.
LGTM. The new behavior is more internally consistent, and is therefore an improvement.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR adjusts the behavior of
PARSE_JSONto be strict and a bit more intuitive, adds a newTRY_PARSE_JSONwhich will produceNULLvalues instead of errors for bad or non-string inputs, and removesTO_JSONwhich no longer had a use since its behavior has been made intrinsic to all JSON expressions.Previously
PARSE_JSONwas far far too forgiving, allowing stuff likePARSE_JSON('hello world')to pass through into a string root literal even though it isn't valid JSON. Meanwhile, something likePARSE_JSON('{')would be an error. Basically, the behavior wasTO_JSONif the input was not actually JSON, which is far too magical and not particularly useful (apologies this made it in like this, it was a relic of an earlier iteration).Now,
PARSE_JSONis strict and will fail with an error if provided non-string input (with the exception of null, which will output a null) or if the string contains invalid JSON.TRY_PARSE_JSONhas also been added, which will returnNULLfor anything that is not a string containing valid JSON in the cases wherePARSE_JSONwould have resulted in an error.I made some adjustments to
StructuredDataProcessorto look for and 'unwrap' anyStructuredDataobjects it happens to encounter, which would allow for the JSON expressions which produceCOMPLEX<json>outputs to return them asStructuredData, but have not actually made this change to the expressions at this time. This change should be harmless, but isn't currently necessary so I can revert it if any reviewers wish.This PR has: