Skip to content

schema: refactor validators and add pre-check for version#254

Merged
bcumming merged 9 commits intoeth-cscs:mainfrom
albestro:alby/refactor-schema-validators
Aug 11, 2025
Merged

schema: refactor validators and add pre-check for version#254
bcumming merged 9 commits intoeth-cscs:mainfrom
albestro:alby/refactor-schema-validators

Conversation

@albestro
Copy link
Copy Markdown
Contributor

Close #247

Rationale:
Current version of stackinator does not support older versions of config.yaml (as stated by the error message reported below), so I don't see the point in having a back-compatible schema (like I was trying with #247).

self._logger.error(
"\nThe recipe is an old version 1 recipe for Spack v0.23 and earlier.\n"
"This version of Stackinator supports Spack 1.0, and has deprecated support for Spack v0.23.\n"
"Use version 5 of stackinator, which can be accessed via the releases/v5 branch:\n"
" git switch releases/v5\n\n"
"If this recipe is to be used with Spack 1.0, then please add the field 'version: 2' to\n"
"config.yaml in your recipe.\n\n"
"For more information: https://eth-cscs.github.io/stackinator/recipes/#configuration\n"
)
raise RuntimeError("incompatible uenv recipe version")

Proposal:
the schema should support just the current supported version, same for tests which should just care about currently supported version (at most, check that unsupported version are correctly reported with a user-friendly error message).

Changelog:

  • move version check logic as a pre-check in the validation step
  • adapt tests and test resources to version: 2
  • took the chance to customize the error message (more in following comment)

@albestro albestro requested review from bcumming and msimberg July 30, 2025 15:02
@albestro albestro self-assigned this Jul 30, 2025
@albestro
Copy link
Copy Markdown
Contributor Author

About the error messages:

  • it does not print anymore the schema. not very useful if not used to schemas, and it can easily be a wall of text hiding the actual problem.
  • it reports all error messages and not just the first one. In the use-cases I faced, the first error is not always the most informative one to solve the problem.
  • it prints the json path pointing to where the error is in the instance, opposed to the schema path
  • it prints "ValidationError for {title}", where title is the one read from schema (I should probably change from for to in).

Note1: trivial examples verified against schema fixed as in #252

Example 1

name: prgenv-gnu
spack:
  repo: https://github.com/spack/spack.git
  commit: c702f5b89eea1950c76472fbf8f0919ad12880a6
  packages:
    repo: https://github.com/spack/spack-packages.git
store: /user-environment
description: GNU Compiler toolchain with cray-mpich, Python, CMake and other development tools.
version: 2

BEFORE

'commit' is a required property

Failed validating 'required' in schema['properties']['spack']['properties']['packages']:
    {'type': 'object',
     'additionalProperties': False,
     'required': ['repo', 'commit'],
     'properties': {'repo': {'type': 'string'},
                    'commit': {'oneOf': [{'type': 'string'},
                                         {'type': 'null'}],
                               'default': None}}}

On instance['spack']['packages']:
    {'repo': 'https://github.com/spack/spack-packages.git'}

NOW

ValidationError for 'Schema for Spack Stack config.yaml recipe file'
- Failed validating 'required' in $.spack.packages : 'commit' is a required property

Example 2

name: prgenv-gnu
spack:
  repo: https://github.com/spack/spack.git
  packages:
    repo: https://github.com/spack/spack-packages.git
store: /user-environment
description: GNU Compiler toolchain with cray-mpich, Python, CMake and other development tools.
version: 2

BEFORE

'commit' is a required property

Failed validating 'required' in schema['properties']['spack']:
    {'type': 'object',
     'additionalProperties': False,
     'required': ['repo', 'commit', 'packages'],
     'properties': {'repo': {'type': 'string'},
                    'commit': {'oneOf': [{'type': 'string'},
                                         {'type': 'null'}],
                               'default': None},
                    'packages': {'type': 'object',
                                 'additionalProperties': False,
                                 'required': ['repo', 'commit'],
                                 'properties': {'repo': {'type': 'string'},
                                                'commit': {'oneOf': [{'type': 'string'},
                                                                     {'type': 'null'}],
                                                           'default': None}}}}}

On instance['spack']:
    {'repo': 'https://github.com/spack/spack.git',
     'packages': {'repo': 'https://github.com/spack/spack-packages.git'}}

NOW

ValidationError for 'Schema for Spack Stack config.yaml recipe file'
- Failed validating 'required' in $.spack : 'commit' is a required property
- Failed validating 'required' in $.spack.packages : 'commit' is a required property

Copy link
Copy Markdown
Member

@bcumming bcumming left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @albestro. The new error messages for schema validation parsers are a big improvement to the usability of the tool.

@bcumming bcumming merged commit 4a4be66 into eth-cscs:main Aug 11, 2025
2 checks passed
@bcumming bcumming mentioned this pull request Aug 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants