Use discriminator to parse correct union type#131
Use discriminator to parse correct union type#131joel-bach merged 4 commits intoHaskell-OpenAPI-Code-Generator:masterfrom
discriminator to parse correct union type#131Conversation
The example (I believe) is good. Changes that are made to the generated code will help me check that my changes to the generator are correct. This is documented/defined here: https://swagger.io/docs/specification/v3_0/data-models/inheritance-and-polymorphism/#discriminator
The docs that I'm referencing are here: https://swagger.io/docs/specification/v3_0/data-models/inheritance-and-polymorphism/#discriminator They are a little confusing, underly prescriptive (IMHO), and lack specificity so I may have gotten some things wrong here. I figure though, as a first usable pass, this might be "good enough" and we can iron out the edge cases should they appear. **What's `discriminator`?** It specifies a `propertyName` which is used as a key to discriminate between different types that a type might take (e.g. `oneOf`). I'm fixing this in my Freckle day job because we have a few types which don't have meaningful distinction in structure (e.g. `{ tag: 'comma' } | { tag: 'newline' }`). Without `discriminator`, or something like it, our generated parsers are just picking the first case when they should respect `tag`. `discriminator` is designed to give this hint to the clients. **Key notes** - The docs say this should work in `anyOf` but I can't really comprehend how that _could_ work. I'm just ignoring it for now and putting it in the "later improvement" bucket since Freckle doesn't need that feature. - The docs also say that `mappings` are optional. If they're not present, then the schema's name will be used as the property value that's checked. The `Lizard` test case tries this. - I don't really know how far to take schema validation in this codebase. For example, the docs don't really say what should happen if there's a `ref` in `mappings` that's not in the `oneOf` (or vice versa). I _could_ check this and log a warning but I opted for less until I hear otherwise. I really wish the schema was better structured to just eliminate these cases but here we are. - This is my first template Haskell so please lemme know if I should be doing something different.
|
Should be sufficient to resolve #125 and provide the functionality that Freckle needs. |
|
Ping @joel-bach to help ensure you see this. |
| Lizard: | ||
| type: object | ||
| oneOf: | ||
| - $ref: '#/components/schemas/gecko' | ||
| - $ref: '#/components/schemas/gilaMonster' | ||
| discriminator: | ||
| propertyName: lizardType | ||
| gecko: | ||
| type: object | ||
| properties: | ||
| hasTail: | ||
| type: boolean | ||
| lizardType: | ||
| type: string | ||
| required: | ||
| - lizardType | ||
| gilaMonster: | ||
| type: object | ||
| properties: | ||
| hasTail: | ||
| type: boolean | ||
| lizardType: | ||
| type: string | ||
| required: | ||
| - lizardType |
There was a problem hiding this comment.
Is there a reason only the lizard schema names start with a lower case letter? Might be good to keep them consistent to clarify that that's not something pertinent to what the test is covering, because currently it stands out and makes me wonder.
| Lizard: | |
| type: object | |
| oneOf: | |
| - $ref: '#/components/schemas/gecko' | |
| - $ref: '#/components/schemas/gilaMonster' | |
| discriminator: | |
| propertyName: lizardType | |
| gecko: | |
| type: object | |
| properties: | |
| hasTail: | |
| type: boolean | |
| lizardType: | |
| type: string | |
| required: | |
| - lizardType | |
| gilaMonster: | |
| type: object | |
| properties: | |
| hasTail: | |
| type: boolean | |
| lizardType: | |
| type: string | |
| required: | |
| - lizardType | |
| Lizard: | |
| type: object | |
| oneOf: | |
| - $ref: '#/components/schemas/Gecko' | |
| - $ref: '#/components/schemas/GilaMonster' | |
| discriminator: | |
| propertyName: lizardType | |
| Gecko: | |
| type: object | |
| properties: | |
| hasTail: | |
| type: boolean | |
| lizardType: | |
| type: string | |
| required: | |
| - lizardType | |
| GilaMonster: | |
| type: object | |
| properties: | |
| hasTail: | |
| type: boolean | |
| lizardType: | |
| type: string | |
| required: | |
| - lizardType |
There was a problem hiding this comment.
@chris-martin I made them lower-case to test that we're not accidentally using haskell-ified propertyName discriminator values in the case obj .:? propertyName cases.
There was a problem hiding this comment.
I think it's great to test this behavior but it would make it clearer if you'd use more distinct values in the mapping, e.g. guppieType or guppieDiscriminator (etc.) so it's clear we're not just lowercasing some value. And then you can keep the rest consistent with the casing. What do you think?
| oneOfSchemaRefs = do | ||
| (ref, (_, name')) <- Map.toList schemaLookupFromRef | ||
| pure (name', ref) | ||
| propertyNamesWithReferences = maybe oneOfSchemaRefs Map.toList $ OAS.discriminatorObjectMapping disc |
There was a problem hiding this comment.
Some comments would help me here
| oneOfSchemaRefs = do | |
| (ref, (_, name')) <- Map.toList schemaLookupFromRef | |
| pure (name', ref) | |
| propertyNamesWithReferences = maybe oneOfSchemaRefs Map.toList $ OAS.discriminatorObjectMapping disc | |
| -- Association of discriminator values to schema names | |
| propertyNamesWithReferences = | |
| case OAS.discriminatorObjectMapping disc of | |
| Just objectMapping -> | |
| -- When present, use the associations explicitly given by the `mapping` property in the spec | |
| Map.toList objectMapping | |
| Nothing -> do | |
| -- When the spec does not provide a `mapping` property, default to using the names of the schemas as discriminator values. | |
| (ref, (_, name')) <- Map.toList schemaLookupFromRef | |
| pure (name', ref) | |
There was a problem hiding this comment.
I thought about commenting this stuff well but there were almost no comments around here. That's probably a bad excuse though. I can add some if the maintainer(s) agree(s).
There was a problem hiding this comment.
Please feel free to add comments! I also prefer case over maybe here but both are fine.
I think I can explain. If the components of your subtype have their discriminator field pinned to a particular value using components:
schemas:
Pet:
type: object
oneOf: # ⚠️
- $ref: '#/components/schemas/Cat'
- $ref: '#/components/schemas/Dog'
discriminator:
propertyName: petType
mapping:
cat: '#/components/schemas/Cat'
dog: '#/components/schemas/Dog'
Cat:
type: object
properties:
petType:
type: string
enum: ['cat'] # ⚠️
required:
- petType
Dog:
type: object
properties:
petType:
type: string
enum: ['dog'] # ⚠️
required:
- petTypeIn this situation, I believe it doesn't matter whether you use
(Aside: One of the bugs in this library, iirc, is that an enum with a single value won't be validated at all - e.g. But what if the components:
schemas:
Pet:
type: object
anyOf: # ⚠️
- $ref: '#/components/schemas/Cat'
- $ref: '#/components/schemas/Dog'
discriminator:
propertyName: petType
mapping:
cat: '#/components/schemas/Cat'
dog: '#/components/schemas/Dog'
Cat:
type: object
properties:
petType:
type: string
# ⚠️
required:
- petType
Dog:
type: object
properties:
petType:
type: string
# ⚠️
required:
- petTypeThis is dumb, but technically allowed. In this case, the I think your sense is right that most cases where you'd be using |
| GHC.Maybe.Just propertyName -> case propertyName :: Data.Text.Internal.Text of | ||
| {"guppie" -> FishGuppie Data.Functor.<$> Data.Aeson.Types.FromJSON.parseJSON val; | ||
| "minnow" -> FishMinnow Data.Functor.<$> Data.Aeson.Types.FromJSON.parseJSON val; | ||
| "shark" -> FishShark Data.Functor.<$> Data.Aeson.Types.FromJSON.parseJSON val; | ||
| _unmatched -> Control.Monad.Fail.fail "No match for discriminator property"}}}) val} |
There was a problem hiding this comment.
Just commenting here to highlight for other reviewers: This case expression is the substantive effect of the PR.
chris-martin
left a comment
There was a problem hiding this comment.
I have always really struggled to read Template Haskell code, but I think the results in the golden output mostly speak for themselves.
Golden tests FTW! |
joel-bach
left a comment
There was a problem hiding this comment.
Thank you very much for your work @mjgpy3 ! This looks good to me 👍
Regarding anyOf: I agree with @chris-martin 's sentiment
Really the larger conclusion to me from this thought exercise is that anyOf is crazy and one ought to avoid it altogether. But, that's me as a Haskeller: I like all my unions disjoint.
If someone wants to use this and has a legitimate case, they can let us know and provide a contribution. From my POV, your PR addresses the main use case which makes it worth merging.
I don't really know how far to take schema validation in this codebase.
I am fine with the current approach. This is not a schema validator and there are better tools for that out there. If the generated code does not compile due to a non-sense schema (which I think is currently the case), it's not ideal but at least you'll get a hint something's up. If you want to add a warning that's even better but not a requirement from my side.
This is my first template Haskell so please lemme know if I should be doing
something different.
It is fine from my POV, as the others said, the golden test output is the relevant part here.
The indexed-variant setting was tricky. I think I've gotten it right here but could use some checking (perhaps I should write a test?)
Ideally, we'd have multiple configurations for the golden tests (with minimal specs to demonstrate the specific setting). I do not have the time on my hands to adjust this atm but if you want to give it a crack, feel free. I'd suggest doing that in a separate PR though. But in any case, I am fine with the current implementation of this.
One part which I would very much like to have is a level 3 test (essentially an e2e test). It's not a strict requirement to merge this PR but it would be helpful to not only verify that the code looks correct in the output but also does the deserialization correctly. Let me know if you can do this and if you need pointers.
I've also let the CI run and the only part failing is the formatting. Could you make sure to let the ormolu formatting run in the pre-commit hook?
| Lizard: | ||
| type: object | ||
| oneOf: | ||
| - $ref: '#/components/schemas/gecko' | ||
| - $ref: '#/components/schemas/gilaMonster' | ||
| discriminator: | ||
| propertyName: lizardType | ||
| gecko: | ||
| type: object | ||
| properties: | ||
| hasTail: | ||
| type: boolean | ||
| lizardType: | ||
| type: string | ||
| required: | ||
| - lizardType | ||
| gilaMonster: | ||
| type: object | ||
| properties: | ||
| hasTail: | ||
| type: boolean | ||
| lizardType: | ||
| type: string | ||
| required: | ||
| - lizardType |
There was a problem hiding this comment.
I think it's great to test this behavior but it would make it clearer if you'd use more distinct values in the mapping, e.g. guppieType or guppieDiscriminator (etc.) so it's clear we're not just lowercasing some value. And then you can keep the rest consistent with the casing. What do you think?
| oneOfSchemaRefs = do | ||
| (ref, (_, name')) <- Map.toList schemaLookupFromRef | ||
| pure (name', ref) | ||
| propertyNamesWithReferences = maybe oneOfSchemaRefs Map.toList $ OAS.discriminatorObjectMapping disc |
There was a problem hiding this comment.
Please feel free to add comments! I also prefer case over maybe here but both are fine.
| Nothing -> [] | ||
| Just (n, caseName) -> do | ||
| let | ||
| suffix = if OAO.settingUseNumberedVariantConstructors settings then "Variant" <> T.pack (show n) else "" |
There was a problem hiding this comment.
Ideally, you'd extract the suffix generation above and reuse it here but I will not block the PR on this.
**Why?** To show that we're not just following some incorrect convention.
|
Thanks @joel-bach. I believe that I've applied the more important suggestions. Please let me know your thoughts! |
joel-bach
left a comment
There was a problem hiding this comment.
This PR is ready to be merged. Are you planning to address other parts of my comment here or in a separate PR?
|
@joel-bach I can try to follow up on some of those points in a separate PR if that's okay |
That's fine with me, I'll merge the PR 👍 |
c01eecc
into
Haskell-OpenAPI-Code-Generator:master
The docs that I'm referencing are here: https://swagger.io/docs/specification/v3_0/data-models/inheritance-and-polymorphism/#discriminator
They are a little confusing, underly prescriptive (IMHO), and lack specificity
so I may have gotten some things wrong here. I figure though, as a first usable
pass, this might be "good enough" and we can iron out the edge cases should they
appear.
What's
discriminator?It specifies a
propertyNamewhich is used as a key to discriminate betweendifferent types that a type might take (e.g.
oneOf).I'm fixing this in my Freckle day job because we have a few types which don't
have meaningful distinction in structure (e.g.
{ tag: 'comma' } | { tag: 'newline' }). Withoutdiscriminator, or something like it, our generatedparsers are just picking the first case when they should respect
tag.discriminatoris designed to give this hint to the clients.Key notes
anyOfbut I can't really comprehend howthat could work. I'm just ignoring it for now and putting it in the "later
improvement" bucket since Freckle doesn't need that feature.
mappingsare optional. If they're not present, thenthe schema's name will be used as the property value that's checked. The
Lizardtest case tries this.example, the docs don't really say what should happen if there's a
refinmappingsthat's not in theoneOf(or vice versa). I could check this andlog a warning but I opted for less until I hear otherwise. I really wish the
schema was better structured to just eliminate these cases but here we are.
something different.