From 849411a1a542760e91615bfc1cc99e708950ed39 Mon Sep 17 00:00:00 2001 From: Leynos Date: Fri, 15 Aug 2025 01:54:26 +0100 Subject: [PATCH 1/5] Document YAML-first templating pipeline --- docs/netsuke-design.md | 62 ++++++++++++++++++++++++------------------ docs/roadmap.md | 19 +++++++------ 2 files changed, 46 insertions(+), 35 deletions(-) diff --git a/docs/netsuke-design.md b/docs/netsuke-design.md index 381d6bca..ea235def 100644 --- a/docs/netsuke-design.md +++ b/docs/netsuke-design.md @@ -31,11 +31,11 @@ understand and execute. This separation of concerns—Netsuke managing build logic and Ninja managing execution—is the foundational principle of the entire architecture. -### 1.2 The Five Stages of a Netsuke Build +### 1.2 The Six Stages of a Netsuke Build The process of transforming a user's `Netsukefile` manifest into a completed -build artifact follows a distinct, five-stage pipeline. This multi-stage data -flow ensures that dynamic configurations are fully resolved into a static plan +build artefact now follows a six-stage pipeline. This data flow validates the +manifest as YAML first, then resolves all dynamic logic into a static plan before execution, a critical requirement for compatibility with Ninja. 1. Stage 1: Manifest Ingestion @@ -43,23 +43,27 @@ before execution, a critical requirement for compatibility with Ninja. The process begins by locating and reading the user's project manifest file (e.g., Netsukefile) from the filesystem into memory as a raw string. -2. Stage 2: Jinja Evaluation +2. Stage 2: Initial YAML Parsing - The raw manifest string is treated as a Jinja template. Netsuke's templating - engine processes this string, evaluating all expressions, executing control - structures (loops, conditionals), and applying filters. This stage resolves - all dynamic aspects of the build, producing a static, pure YAML string as - its output. + The raw string is parsed into an untyped `serde_yml::Value`. This step + ensures the manifest is valid YAML before any templating takes place. -3. Stage 3: YAML Parsing & Deserialization +3. Stage 3: Template Expansion - The static YAML string generated in the previous stage is passed to a YAML - parser. This parser validates the YAML syntax and deserializes the content - into a set of strongly typed Rust data structures. This collection of - structs, which directly mirrors the YAML schema, can be considered an - "unprocessed" Abstract Syntax Tree (AST) of the build plan. + Netsuke walks the YAML `Value`, evaluating Jinja macros, variables, and the + `foreach` and `when` keys. Each mapping containing these keys is expanded by + the Jinja engine, and the environment from each iteration is preserved. At + this stage Jinja may not modify the YAML structure directly; control + constructs live only within these explicit keys. -4. Stage 4: IR Generation & Validation +4. Stage 4: Deserialisation & Final Rendering + + The expanded `Value` is deserialised into strongly typed Rust structs. Jinja + expressions are then rendered, but only within string fields. Structural + templating using `{% %}` blocks is forbidden; all control flow must appear + in YAML values. + +5. Stage 5: IR Generation & Validation The AST is traversed to construct a canonical, fully resolved Intermediate Representation (IR) of the build. This IR represents the build as a static @@ -70,10 +74,10 @@ before execution, a critical requirement for compatibility with Ninja. exactly one of `rule`, `command`, or `script`. Circular dependencies and missing inputs are also detected at this stage. -5. Stage 5: Ninja Synthesis & Execution +6. Stage 6: Ninja Synthesis & Execution The final, validated IR is traversed by a code generator. This generator - synthesizes the content of a `build.ninja` file, translating the IR's nodes + synthesises the content of a `build.ninja` file, translating the IR's nodes and edges into corresponding Ninja rule and build statements. Once the file is written, Netsuke invokes the `ninja` executable as a subprocess, passing control to it for the final dependency checking and command-execution phase. @@ -120,8 +124,8 @@ Representation (IR) generated in Stage 4. The IR serves as a static snapshot of the build plan after all Jinja logic has been resolved. It is the "object code" that the Netsuke "compiler" produces, which can then be handed off to the Ninja "assembler" for execution. This mandate for a pre-computed static graph -dictates the entire five-stage pipeline and establishes a clean boundary -between the user-facing logic layer and the machine-facing execution layer. +dictates the entire six-stage pipeline and establishes a clean boundary between +the user-facing logic layer and the machine-facing execution layer. ## Section 2: The Netsuke Manifest: A User-Centric YAML Schema @@ -336,19 +340,23 @@ are specified. Large sets of similar outputs can clutter a manifest when written individually. Netsuke supports a `foreach` entry within `targets` to generate multiple -outputs succinctly. The expression assigned to `foreach` is evaluated during -the Jinja render phase, and each value becomes `item` in the target context. +outputs succinctly. The `foreach` and optional `when` keys accept bare Jinja +expressions that are evaluated after the initial YAML pass. Each resulting +value becomes `item` in the target context, and the environment at that +iteration is preserved for later rendering. ```yaml -- foreach: "{{ glob('assets/svg/*.svg') }}" +- foreach: glob('assets/svg/*.svg') + when: item | basename != 'logo.svg' name: "{{ outdir }}/{{ item | basename | replace('.svg', '.png') }}" rule: rasterise sources: "{{ item }}" ``` -Each element in the sequence produces a separate target. The resulting build -graph is still fully static and behaves the same as if every target were -declared explicitly. +Each element in the sequence produces a separate target. Jinja control +structures cannot shape the YAML; all templating must occur within the string +values. The resulting build graph is still fully static and behaves the same as +if every target were declared explicitly. ### 2.6 Table: Netsuke Manifest vs. Makefile @@ -1426,7 +1434,7 @@ treated as the default subcommand if none is provided, allowing for the common* The behaviour of each subcommand is clearly defined: - `Netsuke build [--emit FILE] [targets...]`: This is the primary and default - command. It executes the full five-stage pipeline: ingestion, Jinja +command. It executes the full six-stage pipeline: ingestion, Jinja rendering, YAML parsing, IR generation, and Ninja synthesis. By default the generated Ninja file is written to a securely created temporary location and removed after the build completes. Supplying `--emit FILE` writes the Ninja diff --git a/docs/roadmap.md b/docs/roadmap.md index d1354ad2..c0597a79 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -79,19 +79,21 @@ configurations with variables, control flow, and custom functions. - [x] Integrate the `minijinja` crate into the build pipeline. - - [x] Implement the two-pass parsing mechanism: the first pass renders the - manifest as a Jinja template, and the second pass parses the resulting pure - YAML string with serde_yml. + - [x] Implement data-first parsing: parse the manifest into a + `serde_yml::Value`, expand `foreach` and `when` entries with a Jinja + environment, then deserialize the expanded tree into the typed AST and + render remaining string fields. - [x] Create a minijinja::Environment and populate its initial context with the global vars defined in the manifest. - [ ] **Dynamic Features and Custom Functions:** - - [x] Implement support for basic Jinja control structures (`{% if %}` and - `{% for %}`) + - [x] Evaluate Jinja expressions only within string values, forbidding + structural tags such as `{% if %}` and `{% for %}`. - - [ ] Implement the foreach key for target generation. + - [ ] Implement the `foreach` and `when` keys for target generation, carrying + the iteration context into subsequent rendering phases. - [ ] Implement the essential custom Jinja function env(var_name) to read system environment variables. @@ -105,8 +107,9 @@ configurations with variables, control flow, and custom functions. - **Success Criterion:** - [ ] Netsuke can successfully build a manifest that uses variables, - conditional logic, the foreach loop, custom macros, and the glob() function - to discover and operate on source files. + conditional logic within string values, the `foreach` and `when` keys, + custom macros, and the `glob()` function to discover and operate on source + files. ## Phase 3: The "Friendly" Polish 🛡️ From a34751334c8eb12532be0b21ba7dad93ad017de0 Mon Sep 17 00:00:00 2001 From: Leynos Date: Fri, 15 Aug 2025 02:11:43 +0100 Subject: [PATCH 2/5] Align docs with six-stage pipeline --- ...havioural-testing-in-rust-with-cucumber.md | 2 +- docs/netsuke-design.md | 40 ++++++++----------- docs/roadmap.md | 2 +- scripts/assert-file-absent.sh | 6 +-- scripts/assert-file-exists.sh | 6 +-- 5 files changed, 25 insertions(+), 31 deletions(-) diff --git a/docs/behavioural-testing-in-rust-with-cucumber.md b/docs/behavioural-testing-in-rust-with-cucumber.md index c49f15bb..3c8f9f8b 100644 --- a/docs/behavioural-testing-in-rust-with-cucumber.md +++ b/docs/behavioural-testing-in-rust-with-cucumber.md @@ -18,7 +18,7 @@ stakeholders can use to describe and agree upon software requirements.[^2] This process is centred on conversation; the discussions about how a feature should behave are the most valuable output of BDD.[^3] -The tangible artefact of these conversations is a set of specifications written +The tangible artifact of these conversations is a set of specifications written in a structured, natural language format. These specifications serve a dual purpose: they are human-readable documentation of the system's features, and they are also executable tests that verify the system's behaviour. This diff --git a/docs/netsuke-design.md b/docs/netsuke-design.md index ea235def..fb7d7859 100644 --- a/docs/netsuke-design.md +++ b/docs/netsuke-design.md @@ -34,7 +34,7 @@ architecture. ### 1.2 The Six Stages of a Netsuke Build The process of transforming a user's `Netsukefile` manifest into a completed -build artefact now follows a six-stage pipeline. This data flow validates the +build artifact now follows a six-stage pipeline. This data flow validates the manifest as YAML first, then resolves all dynamic logic into a static plan before execution, a critical requirement for compatibility with Ninja. @@ -77,7 +77,7 @@ before execution, a critical requirement for compatibility with Ninja. 6. Stage 6: Ninja Synthesis & Execution The final, validated IR is traversed by a code generator. This generator - synthesises the content of a `build.ninja` file, translating the IR's nodes + synthesizes the content of a `build.ninja` file, translating the IR's nodes and edges into corresponding Ninja rule and build statements. Once the file is written, Netsuke invokes the `ninja` executable as a subprocess, passing control to it for the final dependency checking and command-execution phase. @@ -88,18 +88,12 @@ before execution, a critical requirement for compatibility with Ninja. output suitable for caching or source control. ```mermaid -sequenceDiagram - participant User - participant Netsuke - participant IR_Generator - participant Ninja - - User->>Netsuke: Provide Netsukefile and environment - Netsuke->>IR_Generator: Parse and validate manifest - IR_Generator->>IR_Generator: Generate deterministic BuildGraph (IR) - IR_Generator->>Netsuke: Return BuildGraph (byte-for-byte deterministic) - Netsuke->>Ninja: Synthesize build.ninja and invoke Ninja - Ninja-->>User: Execute build and report results +flowchart TD + A[Stage 1:\nManifest Ingestion] --> B[Stage 2:\nInitial YAML Parsing] + B --> C[Stage 3:\nTemplate Expansion] + C --> D[Stage 4:\nDeserialisation & Final Rendering] + D --> E[Stage 5:\nIR Generation & Validation] + E --> F[Stage 6:\nNinja Synthesis & Execution] ``` ### 1.3 The Static Graph Mandate @@ -598,7 +592,7 @@ preserved for Jinja control flow. Targets also accept optional `phony` and `always` booleans. They default to `false`, making it explicit when an action should run regardless of file timestamps. Targets listed in the `actions` section are deserialised using a custom helper so they are always treated as -`phony` tasks. This ensures preparation actions never generate build artefacts. +`phony` tasks. This ensures preparation actions never generate build artifacts. Convenience functions in `src/manifest.rs` load a manifest from a string or a file path, returning `anyhow::Result` for straightforward error handling. @@ -1406,7 +1400,7 @@ struct Cli { /// Path to the Netsuke manifest file to use. enum Commands { /// Build specified targets (or default targets if none are given). /// This is the default subcommand. Build(BuildArgs), - /// Remove build artefacts and intermediate files. Clean, + /// Remove build artifacts and intermediate files. Clean, /// Display the build dependency graph in DOT format for visualisation. Graph, @@ -1434,13 +1428,13 @@ treated as the default subcommand if none is provided, allowing for the common* The behaviour of each subcommand is clearly defined: - `Netsuke build [--emit FILE] [targets...]`: This is the primary and default -command. It executes the full six-stage pipeline: ingestion, Jinja - rendering, YAML parsing, IR generation, and Ninja synthesis. By default the - generated Ninja file is written to a securely created temporary location and - removed after the build completes. Supplying `--emit FILE` writes the Ninja - file to `FILE` and retains it. If no targets are provided on the command - line, the targets listed in the `defaults` section of the manifest are - built. +command. It executes the full six-stage pipeline: Manifest Ingestion, Initial +YAML Parsing, Template Expansion, Deserialisation & Final Rendering, IR +Generation & Validation, and Ninja Synthesis & Execution. By default the +generated Ninja file is written to a securely created temporary location and +removed after the build completes. Supplying `--emit FILE` writes the Ninja +file to `FILE` and retains it. If no targets are provided on the command line, +the targets listed in the `defaults` section of the manifest are built. - `Netsuke clean`: This command provides a convenient way to clean the build directory. It will invoke the Ninja backend with the appropriate flags, such diff --git a/docs/roadmap.md b/docs/roadmap.md index c0597a79..3056dcc1 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -68,7 +68,7 @@ compilation pipeline from parsing to execution. - [x] Netsuke can successfully take a Netsukefile without any Jinja syntax, compile it to a `build.ninja` file, and execute it via the ninja subprocess - to produce the correct build artefacts. *(validated via CI workflow)* + to produce the correct build artifacts. *(validated via CI workflow)* ## Phase 2: The Dynamic Engine ✨ diff --git a/scripts/assert-file-absent.sh b/scripts/assert-file-absent.sh index eb3cb6c3..c99bc1ed 100755 --- a/scripts/assert-file-absent.sh +++ b/scripts/assert-file-absent.sh @@ -1,6 +1,6 @@ #!/usr/bin/env bash -# Ensures the Netsuke build did not produce an unexpected artefact. -# If the artefact is present and `NINJA_MANIFEST` is set, the referenced +# Ensures the Netsuke build did not produce an unexpected artifact. +# If the artifact is present and `NINJA_MANIFEST` is set, the referenced # Ninja manifest is dumped to stderr for debugging. set -euo pipefail @@ -12,7 +12,7 @@ fi file="$1" if [[ -f "$file" ]]; then - echo "Unexpected build artefact '$file' present." >&2 + echo "Unexpected build artifact '$file' present." >&2 if [[ -n "${NINJA_MANIFEST:-}" && -f "$NINJA_MANIFEST" ]]; then echo "Ninja manifest '$NINJA_MANIFEST' for debugging:" >&2 echo "-----BEGIN NINJA MANIFEST-----" >&2 diff --git a/scripts/assert-file-exists.sh b/scripts/assert-file-exists.sh index 7c7f0c76..ca16e4e4 100755 --- a/scripts/assert-file-exists.sh +++ b/scripts/assert-file-exists.sh @@ -1,6 +1,6 @@ #!/usr/bin/env bash -# Ensures the Netsuke build produced the expected artefact. -# If the artefact is absent and `NINJA_MANIFEST` is set, the referenced +# Ensures the Netsuke build produced the expected artifact. +# If the artifact is absent and `NINJA_MANIFEST` is set, the referenced # Ninja manifest is dumped to stderr for debugging. set -euo pipefail @@ -12,7 +12,7 @@ fi file="$1" if [[ ! -f "$file" ]]; then - echo "Expected build artefact '$file' to exist." >&2 + echo "Expected build artifact '$file' to exist." >&2 if [[ -n "${NINJA_MANIFEST:-}" && -f "$NINJA_MANIFEST" ]]; then echo "Ninja manifest '$NINJA_MANIFEST' for debugging:" >&2 echo "-----BEGIN NINJA MANIFEST-----" >&2 From f933a281f8ccf47e6a31cda486d0ca000c2b3a70 Mon Sep 17 00:00:00 2001 From: Leynos Date: Fri, 15 Aug 2025 13:41:18 +0100 Subject: [PATCH 3/5] Use EX_USAGE in file absence script --- scripts/assert-file-absent.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/assert-file-absent.sh b/scripts/assert-file-absent.sh index c99bc1ed..ecb84432 100755 --- a/scripts/assert-file-absent.sh +++ b/scripts/assert-file-absent.sh @@ -6,7 +6,7 @@ set -euo pipefail if [[ $# -ne 1 ]]; then echo "Usage: $(basename "$0") " >&2 - exit 2 # usage error + exit 64 # EX_USAGE fi file="$1" From 306cd224387656bc01a70f591820a4f39fca0345 Mon Sep 17 00:00:00 2001 From: Leynos Date: Fri, 15 Aug 2025 13:41:24 +0100 Subject: [PATCH 4/5] Document foreach context and use artefact spelling --- Makefile | 2 +- ...havioural-testing-in-rust-with-cucumber.md | 4 ++-- docs/netsuke-design.md | 23 +++++++++++-------- docs/roadmap.md | 9 ++++---- scripts/assert-file-absent.sh | 6 ++--- scripts/assert-file-exists.sh | 6 ++--- src/cli.rs | 2 +- tests/runner_tests.rs | 6 ++--- 8 files changed, 32 insertions(+), 26 deletions(-) diff --git a/Makefile b/Makefile index bdc7ea1b..704a9daa 100644 --- a/Makefile +++ b/Makefile @@ -12,7 +12,7 @@ release: target/release/$(APP) ## Build release binary all: release ## Default target builds release binary -clean: ## Remove build artifacts +clean: ## Remove build artefacts $(CARGO) clean test: ## Run tests with warnings treated as errors diff --git a/docs/behavioural-testing-in-rust-with-cucumber.md b/docs/behavioural-testing-in-rust-with-cucumber.md index 3c8f9f8b..85965f8b 100644 --- a/docs/behavioural-testing-in-rust-with-cucumber.md +++ b/docs/behavioural-testing-in-rust-with-cucumber.md @@ -18,7 +18,7 @@ stakeholders can use to describe and agree upon software requirements.[^2] This process is centred on conversation; the discussions about how a feature should behave are the most valuable output of BDD.[^3] -The tangible artifact of these conversations is a set of specifications written +The tangible artefact of these conversations is a set of specifications written in a structured, natural language format. These specifications serve a dual purpose: they are human-readable documentation of the system's features, and they are also executable tests that verify the system's behaviour. This @@ -979,7 +979,7 @@ The process involves two main steps: 2. **Publish reports:** Many CI platforms can parse and display test results in a structured format. The `cucumber` crate supports generating JUnit XML reports via the `output-junit` feature flag.[^16] These XML files can then - be published as test artifacts for platforms like GitHub Actions, GitLab + be published as test artefacts for platforms like GitHub Actions, GitLab CI,[^34] or Jenkins to consume.[^33] This CI integration closes the BDD loop. The `.feature` files, once checked diff --git a/docs/netsuke-design.md b/docs/netsuke-design.md index fb7d7859..cc3fd776 100644 --- a/docs/netsuke-design.md +++ b/docs/netsuke-design.md @@ -34,7 +34,7 @@ architecture. ### 1.2 The Six Stages of a Netsuke Build The process of transforming a user's `Netsukefile` manifest into a completed -build artifact now follows a six-stage pipeline. This data flow validates the +build artefact now follows a six-stage pipeline. This data flow validates the manifest as YAML first, then resolves all dynamic logic into a static plan before execution, a critical requirement for compatibility with Ninja. @@ -347,10 +347,15 @@ iteration is preserved for later rendering. sources: "{{ item }}" ``` -Each element in the sequence produces a separate target. Jinja control -structures cannot shape the YAML; all templating must occur within the string -values. The resulting build graph is still fully static and behaves the same as -if every target were declared explicitly. +Each element in the sequence produces a separate target. Per-iteration context: + +- item: current element +- index: 0-based index (optional) +- vars: resolved in order `globals` < `target.vars` < per-iteration locals + +Jinja control structures cannot shape the YAML; all templating must occur +within the string values. The resulting build graph is still fully static and +behaves the same as if every target were declared explicitly. ### 2.6 Table: Netsuke Manifest vs. Makefile @@ -592,7 +597,7 @@ preserved for Jinja control flow. Targets also accept optional `phony` and `always` booleans. They default to `false`, making it explicit when an action should run regardless of file timestamps. Targets listed in the `actions` section are deserialised using a custom helper so they are always treated as -`phony` tasks. This ensures preparation actions never generate build artifacts. +`phony` tasks. This ensures preparation actions never generate build artefacts. Convenience functions in `src/manifest.rs` load a manifest from a string or a file path, returning `anyhow::Result` for straightforward error handling. @@ -1400,7 +1405,7 @@ struct Cli { /// Path to the Netsuke manifest file to use. enum Commands { /// Build specified targets (or default targets if none are given). /// This is the default subcommand. Build(BuildArgs), - /// Remove build artifacts and intermediate files. Clean, + /// Remove build artefacts and intermediate files. Clean, /// Display the build dependency graph in DOT format for visualisation. Graph, @@ -1497,7 +1502,7 @@ goal. - **Success Criterion:** Netsuke can successfully take a `Netsukefile` file *without any Jinja syntax* and compile it to a `build.ninja` file, then - execute it to produce the correct artifacts. This phase validates the + execute it to produce the correct artefacts. This phase validates the entire static compilation pipeline. - **Phase 2: The Dynamic Engine** @@ -1567,7 +1572,7 @@ powerful build tool. The use of a decoupled IR, in particular, opens up many possibilities for future enhancements beyond the initial scope. - **Advanced Caching:** While Ninja provides excellent file-based incremental - build caching, Netsuke could implement a higher-level artifact caching layer. + build caching, Netsuke could implement a higher-level artefact caching layer. This could involve caching build outputs in a shared network location (e.g., S3) or a local content-addressed store, allowing for cache hits across different machines or clean checkouts. diff --git a/docs/roadmap.md b/docs/roadmap.md index 3056dcc1..19eb0bdb 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -68,7 +68,7 @@ compilation pipeline from parsing to execution. - [x] Netsuke can successfully take a Netsukefile without any Jinja syntax, compile it to a `build.ninja` file, and execute it via the ninja subprocess - to produce the correct build artifacts. *(validated via CI workflow)* + to produce the correct build artefacts. *(validated via CI workflow)* ## Phase 2: The Dynamic Engine ✨ @@ -80,9 +80,10 @@ configurations with variables, control flow, and custom functions. - [x] Integrate the `minijinja` crate into the build pipeline. - [x] Implement data-first parsing: parse the manifest into a - `serde_yml::Value`, expand `foreach` and `when` entries with a Jinja - environment, then deserialize the expanded tree into the typed AST and - render remaining string fields. + `serde_yml::Value` (Stage 2: Initial YAML Parsing), expand `foreach` and + `when` entries with a Jinja environment (Stage 3: Template Expansion), then + deserialise the expanded tree into the typed AST and render remaining + string fields (Stage 4: Deserialisation & Final Rendering). - [x] Create a minijinja::Environment and populate its initial context with the global vars defined in the manifest. diff --git a/scripts/assert-file-absent.sh b/scripts/assert-file-absent.sh index ecb84432..8fabbdd3 100755 --- a/scripts/assert-file-absent.sh +++ b/scripts/assert-file-absent.sh @@ -1,6 +1,6 @@ #!/usr/bin/env bash -# Ensures the Netsuke build did not produce an unexpected artifact. -# If the artifact is present and `NINJA_MANIFEST` is set, the referenced +# Ensures the Netsuke build did not produce an unexpected artefact. +# If the artefact is present and `NINJA_MANIFEST` is set, the referenced # Ninja manifest is dumped to stderr for debugging. set -euo pipefail @@ -12,7 +12,7 @@ fi file="$1" if [[ -f "$file" ]]; then - echo "Unexpected build artifact '$file' present." >&2 + echo "Unexpected build artefact '$file' present." >&2 if [[ -n "${NINJA_MANIFEST:-}" && -f "$NINJA_MANIFEST" ]]; then echo "Ninja manifest '$NINJA_MANIFEST' for debugging:" >&2 echo "-----BEGIN NINJA MANIFEST-----" >&2 diff --git a/scripts/assert-file-exists.sh b/scripts/assert-file-exists.sh index ca16e4e4..7c7f0c76 100755 --- a/scripts/assert-file-exists.sh +++ b/scripts/assert-file-exists.sh @@ -1,6 +1,6 @@ #!/usr/bin/env bash -# Ensures the Netsuke build produced the expected artifact. -# If the artifact is absent and `NINJA_MANIFEST` is set, the referenced +# Ensures the Netsuke build produced the expected artefact. +# If the artefact is absent and `NINJA_MANIFEST` is set, the referenced # Ninja manifest is dumped to stderr for debugging. set -euo pipefail @@ -12,7 +12,7 @@ fi file="$1" if [[ ! -f "$file" ]]; then - echo "Expected build artifact '$file' to exist." >&2 + echo "Expected build artefact '$file' to exist." >&2 if [[ -n "${NINJA_MANIFEST:-}" && -f "$NINJA_MANIFEST" ]]; then echo "Ninja manifest '$NINJA_MANIFEST' for debugging:" >&2 echo "-----BEGIN NINJA MANIFEST-----" >&2 diff --git a/src/cli.rs b/src/cli.rs index 664e7c87..2812dd9c 100644 --- a/src/cli.rs +++ b/src/cli.rs @@ -97,7 +97,7 @@ pub enum Commands { /// Build specified targets (or default targets if none are given) `default`. Build(BuildArgs), - /// Remove build artifacts and intermediate files. + /// Remove build artefacts and intermediate files. Clean, /// Display the build dependency graph in DOT format for visualization. diff --git a/tests/runner_tests.rs b/tests/runner_tests.rs index 76c8ddfc..39ff6248 100644 --- a/tests/runner_tests.rs +++ b/tests/runner_tests.rs @@ -119,7 +119,7 @@ fn run_executes_ninja_without_persisting_file() { // Ensure no ninja file remains in project directory assert!(!temp.path().join("build.ninja").exists()); - // Drop the fake ninja artifacts. PATH is restored by guard drop. + // Drop the fake ninja artefacts. PATH is restored by guard drop. drop(ninja_path); } @@ -150,7 +150,7 @@ fn run_build_with_emit_keeps_file() { assert!(emitted.contains("build ")); assert!(!temp.path().join("build.ninja").exists()); - // Drop the fake ninja artifacts. PATH is restored by guard drop. + // Drop the fake ninja artefacts. PATH is restored by guard drop. drop(ninja_path); } @@ -179,7 +179,7 @@ fn run_build_with_emit_creates_parent_dirs() { assert!(emit_path.exists()); assert!(nested_dir.exists()); - // Drop the fake ninja artifacts. PATH is restored by guard drop. + // Drop the fake ninja artefacts. PATH is restored by guard drop. drop(ninja_path); } From 6148a07eadd4ea2608887c8a60e56d3395729d04 Mon Sep 17 00:00:00 2001 From: Leynos Date: Fri, 15 Aug 2025 19:30:24 +0100 Subject: [PATCH 5/5] Clarify foreach context; forbid structural Jinja --- docs/netsuke-design.md | 25 ++++++++++++++----------- docs/roadmap.md | 6 ++++-- 2 files changed, 18 insertions(+), 13 deletions(-) diff --git a/docs/netsuke-design.md b/docs/netsuke-design.md index cc3fd776..7b64eb23 100644 --- a/docs/netsuke-design.md +++ b/docs/netsuke-design.md @@ -51,10 +51,13 @@ before execution, a critical requirement for compatibility with Ninja. 3. Stage 3: Template Expansion Netsuke walks the YAML `Value`, evaluating Jinja macros, variables, and the - `foreach` and `when` keys. Each mapping containing these keys is expanded by - the Jinja engine, and the environment from each iteration is preserved. At - this stage Jinja may not modify the YAML structure directly; control - constructs live only within these explicit keys. + `foreach` and `when` keys. Each mapping containing these keys is expanded + with an iteration context providing `item` and optional `index`. Variable + lookups respect the precedence `globals` < `target.vars` < per-iteration + locals, and this context is preserved for later rendering. At this stage + Jinja must not modify the YAML structure directly; control constructs live + only within these explicit keys. Structural Jinja blocks (`{% ... %}`) are + not permitted to reshape mappings or sequences. 4. Stage 4: Deserialisation & Final Rendering @@ -335,9 +338,9 @@ are specified. Large sets of similar outputs can clutter a manifest when written individually. Netsuke supports a `foreach` entry within `targets` to generate multiple outputs succinctly. The `foreach` and optional `when` keys accept bare Jinja -expressions that are evaluated after the initial YAML pass. Each resulting -value becomes `item` in the target context, and the environment at that -iteration is preserved for later rendering. +expressions evaluated after the initial YAML pass. Each resulting value becomes +`item` in the target context, and the per-iteration environment is carried +forward to later rendering. ```yaml - foreach: glob('assets/svg/*.svg') @@ -347,11 +350,11 @@ iteration is preserved for later rendering. sources: "{{ item }}" ``` -Each element in the sequence produces a separate target. Per-iteration context: +Each element in the sequence produces a separate target. The iteration context: -- item: current element -- index: 0-based index (optional) -- vars: resolved in order `globals` < `target.vars` < per-iteration locals +- `item`: current element +- `index`: 0-based index (optional) +- Variables resolve with precedence `globals` < `target.vars` < iteration locals Jinja control structures cannot shape the YAML; all templating must occur within the string values. The resulting build graph is still fully static and diff --git a/docs/roadmap.md b/docs/roadmap.md index 19eb0bdb..eb6ecf52 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -93,8 +93,10 @@ configurations with variables, control flow, and custom functions. - [x] Evaluate Jinja expressions only within string values, forbidding structural tags such as `{% if %}` and `{% for %}`. - - [ ] Implement the `foreach` and `when` keys for target generation, carrying - the iteration context into subsequent rendering phases. + - [ ] Implement the `foreach` and `when` keys for target generation, + exposing `item` and optional `index` variables and layering + per-iteration locals over `target.vars` and manifest globals for + subsequent rendering phases. - [ ] Implement the essential custom Jinja function env(var_name) to read system environment variables.