Skip to content

Conversation

@hudlow
Copy link
Collaborator

@hudlow hudlow commented Sep 23, 2025

  • Removes the old Go-and-Python tooling to ingest/convert CEL conformance tests and replaces it with a new pure-Python script (tools/gherkinize.py)
  • Adds a new tools/wip.txt list to annotate individual conformance test scenarios with @wip
  • Updates to the latest version of CEL conformance tests
  • Seeds tools/wip.txt with currently-failing tests
  • Updates tools/README.rst to describe the new tooling
  • Updates .github/workflows/ci.yaml to use uv, as the rest of the repo does (I needed to do this so I wouldn't have to get the new build dependencies working in two separate systems)
  • Updates Makefile and features/Makefile to remove obsolete targets and add more granularity for testing
  • Updates features/steps/integration_binding.py to remove workarounds no longer needed for the cleaner, more consistent output of the new script.

I recommend starting with the new tools/README.rst.

Apologies for the large change set, but I have tried to make the changes as reviewable as I could:

  • Despite an entirely new code path, generated Gherkin scenarios in features/*.feature are VERY similar to the old output.
  • src/ has only been touched to tweak some linter annotations — this PR does not touch the runtime, and this is very intentional!
  • I carefully translated everything that seemed applicable from the old test_pb2g.py to the new test_gherkinize.py as a way of providing some confidence that the new test generation path considers everything the old one did. Unfortunately GitHub does not show this as a rename, so if you want to see them side-by-side you'll have to pull them down and do a diff (I recommend running an aggressive formatter on both first).

@hudlow hudlow requested a review from slott56 September 23, 2025 23:02
Copy link
Contributor

@stefanvanburen stefanvanburen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I won't pretend to have fully reviewed or understand the existing gherkin process, but what I did see looked good to me. just a couple of passing nits, really

@hudlow hudlow force-pushed the refactor-conformance-ingestion branch from 51e0072 to 3839ec5 Compare September 24, 2025 15:36
Copy link
Collaborator

@slott56 slott56 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very, very nice. Thoughtful, careful, and wonderfully complete. I'm delighted with the resulting simplification to managing conformance testing with the baseline CEL-spec. This is a super-helpful foundation.

Two things:

  1. Can we switch the wip.txt to be a wip.toml so that it can be enriched with other attributes, like plans, status, dates, etc.? (Maybe this is too much work for only a potential value.)

  2. Can we add a module-level docstring at the top of the new gherkinize.py. This would help create a unified Sphinx-friendly source for internal documentation.

Feel free to explain to me why these might be bad ideas.

@slott56 slott56 changed the title Refactor conformance ingestion Refactor conformance test case ingestion Sep 25, 2025
@hudlow hudlow force-pushed the refactor-conformance-ingestion branch 2 times, most recently from 12952df to 1122865 Compare September 26, 2025 22:50
@hudlow hudlow force-pushed the refactor-conformance-ingestion branch from 1122865 to 81f5894 Compare September 26, 2025 23:03
Copy link
Collaborator

@slott56 slott56 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The structure of the TOML for WIP tags looks really, really helpful. It parallels the source and leaves plenty of room to add notes or other fields and attributes and things.

I'm not convinced the unique classes are helpful. I would like some docstring comments to provide a few sentences on why this structure is helpful. Especially because toml already produces a nested dict[str, dict[str, dict[str, tags]]] structure that we're replacing with something almost but not quite the same.

@hudlow hudlow force-pushed the refactor-conformance-ingestion branch from 9829119 to a4d8844 Compare September 30, 2025 21:52
Copy link
Collaborator

@slott56 slott56 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent docstring for the new app. The configuration reflects an inherent complexity in the problem domain and a couple of comments explain this.

(Most important. It eliminates the legacy, clumsy, multi-step mess.)

@slott56 slott56 merged commit 443a79f into cloud-custodian:main Oct 1, 2025
8 checks passed
@slott56 slott56 mentioned this pull request Oct 1, 2025
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants