Skip to content

schema: validate schemas against meta-schemas#253

Merged
bcumming merged 6 commits intoeth-cscs:mainfrom
albestro:alby/test-schemas
Aug 18, 2025
Merged

schema: validate schemas against meta-schemas#253
bcumming merged 6 commits intoeth-cscs:mainfrom
albestro:alby/test-schemas

Conversation

@albestro
Copy link
Copy Markdown
Contributor

@albestro albestro commented Jul 29, 2025

The idea behind this is to add a CI check that validates schemas against meta schemas, aiming at early catching small errors like #252.

Changelog:

  • CI: use pipx to install both uv and check-jsonschema
  • CI: use check-jsonschema to validate all schemas (json files) in stackiantor/schema directory

@albestro
Copy link
Copy Markdown
Contributor Author

albestro commented Jul 29, 2025

Nice to add:

  • do not exit at first fail
  • (optional) more concise error message
  • if nothing custom is needed (e.g. versioning), evaluate switching to check-jsonschema

@albestro albestro self-assigned this Jul 30, 2025
@albestro albestro force-pushed the alby/test-schemas branch from 7e28cc4 to 50fc057 Compare July 31, 2025 06:46
@albestro
Copy link
Copy Markdown
Contributor Author

The note in previous comment has been fully addressed: switched to check-jsonschema that provides a better error message, and put in a script that accumulates errors from all validation and just errors in the end if at least one validation failed.

See https://github.com/eth-cscs/stackinator/actions/runs/16642216221/job/47094558980 for an example of failure.

stackinator/schema/cache.json
ok -- validation done
stackinator/schema/environments.json
ok -- validation done
stackinator/schema/compilers.json
ok -- validation done
stackinator/schema/config.json
Schema validation errors were encountered.
  stackinator/schema/config.json::$.properties.spack.properties.required: ['repo', 'commit', 'packages'] is not of type 'object', 'boolean'

@albestro
Copy link
Copy Markdown
Contributor Author

As soon as #252 gets merged, which fixes the schema reported as error in the previous CI run, this one can be reviewed and merged.

Copy link
Copy Markdown
Member

@bcumming bcumming left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, just a few little fixes to stick with using uv for package management.

Comment thread .github/workflows/main.yaml Outdated
- uses: actions/checkout@v3
- name: Install uv
- name: Install Tools
uses: threeal/pipx-install-action@v1.0.0
Copy link
Copy Markdown
Member

@bcumming bcumming Aug 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using pipx to install uv is a little bit meta - uv is supposed to be a replacement for pipx.
Instead, you can use uvx to run check-jsonschema.

Comment thread .github/workflows/main.yaml Outdated
errors=0
for schema in `find stackinator/schema -type f -name "*.json"`; do
echo $schema
check-jsonschema --check-metaschema $schema
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
check-jsonschema --check-metaschema $schema
uvx check-jsonschema --check-metaschema $schema

@albestro albestro marked this pull request as ready for review August 12, 2025 12:41
@albestro albestro requested a review from bcumming August 18, 2025 09:06
@bcumming bcumming merged commit 2211d65 into eth-cscs:main Aug 18, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants