Skip to content

Finish ignored validation conformance gaps from spec suite #58

@craigsmitham

Description

@craigsmitham

Summary

Issue #57 adds a new GraphZen.SpecConformance.Tests project for GraphQL draft Section 5 validation traceability. The suite is green locally, but only by explicitly ignoring the cases that currently expose implementation gaps or where the broader graphql-js matrix still needs to be ported.

This follow-up tracks the ignored validation conformance work so the suite can move from pass + ignore to fully enforced conformance.

Current state

  • dotnet test passes on main with the new spec suite in place
  • GraphZen.SpecConformance.Tests currently reports 91 passed, 21 skipped, 0 failed
  • The skipped cases are intentional and correspond to product gaps or incomplete graphql-js parity ports

Ignored conformance areas

  • 5.2.4.1 Single Root Field
    Negative subscription root-field validation cases are ignored
  • 5.3.2 Field Selection Merging
    The broader graphql-js overlap matrix is not yet ported
  • 5.3.3 Leaf Field Selections
    Negative validation cases are ignored
  • 5.4.1 Argument Names
    Negative validation cases are ignored
  • 5.4.2 Argument Uniqueness
    Negative validation cases are ignored
  • 5.4.3 Required Arguments
    Negative validation cases are ignored
  • 5.5.1.1 Fragment Name Uniqueness
    Negative validation cases are ignored
  • 5.5.1.4 Fragments Must Be Used
    Negative validation cases are ignored
  • 5.5.2.2 Fragment Spreads Must Not Form Cycles
    Negative validation cases are ignored
  • 5.5.2.3 Fragment Spread Is Possible
    The broader graphql-js spread matrix is not yet ported
  • 5.6.1, 5.6.2, 5.6.4
    The literal coercion / input field name / required field matrix is not yet ported
  • 5.6.3 Input Object Field Uniqueness
    Negative validation cases are ignored
  • 5.7.3 Directives Are Unique per Location
    Negative validation cases are ignored
  • 5.8.1 Variable Uniqueness
    Negative validation case is ignored
  • 5.8.2 Variables Are Input Types
    Negative validation case is ignored
  • 5.8.3 All Variable Uses Defined
    Negative validation cases are ignored
  • 5.8.4 All Variables Used
    Negative validation cases are ignored
  • 5.8.5 All Variable Usages Are Allowed
    Negative validation cases are ignored, plus the OneOf-specific cases still need a dedicated schema harness

Acceptance criteria

  • Replace ignored tests with enforced assertions where GraphZen already behaves correctly
  • Fix implementation gaps uncovered by the ignored cases
  • Port the remaining graphql-js validation matrices now represented only by placeholders
  • Remove the temporary skips from GraphZen.SpecConformance.Tests
  • Keep dotnet test green without relying on the ignored cases above

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions