From 4f88bb2c5e1b5c7ea637af4dce8ad3fde1928655 Mon Sep 17 00:00:00 2001 From: Leynos Date: Sun, 13 Jul 2025 20:26:55 +0100 Subject: [PATCH 1/7] Clarify expansion of ins/outs --- docs/netsuke-design.md | 133 +++++++++++++++++++++++++---------------- 1 file changed, 82 insertions(+), 51 deletions(-) diff --git a/docs/netsuke-design.md b/docs/netsuke-design.md index 4b4e82e1..3086a32f 100644 --- a/docs/netsuke-design.md +++ b/docs/netsuke-design.md @@ -72,11 +72,16 @@ before execution, a critical requirement for compatibility with Ninja. 5. Stage 5: Ninja Synthesis & Execution The final, validated IR is traversed by a code generator. This generator - synthesizes the content of a build.ninja file, translating the IR's nodes + synthesizes the content of a `build.ninja` file, translating the IR's nodes and edges into corresponding Ninja rule and build statements. Once the file - is written, Netsuke invokes the ninja executable as a subprocess, passing + is written, Netsuke invokes the `ninja` executable as a subprocess, passing control to it for the final dependency checking and command execution phase. + Netsuke's pipeline is **deterministic**. Given the same `Netsukefile` and + environment variables, the generated `build.ninja` will be byte-for-byte + identical. This property is essential for reproducible builds and makes the + output suitable for caching or source control. + ### 1.3 The Static Graph Mandate The architecture's multi-stage pipeline is a direct consequence of a fundamental @@ -168,18 +173,23 @@ Each entry in the `rules` list is a mapping that defines a reusable action. - `name`: A unique string identifier for the rule. -- `command`: A single command string to be executed. This string uses - placeholders like `{{ ins }}` and `{{ outs }}` for input and output files. - Netsuke's IR-to-Ninja generator will translate these into Ninja's native $in - and $out variables. After interpolation, the value must be parsable by the - [`shlex`](https://docs.rs/shlex/latest/shlex/) crate. Any interpolation other - than `ins` or `outs` is automatically shell escaped. +- `command`: A single command string to be executed. It may include the + placeholders `{{ ins }}` and `{{ outs }}` to represent input and output files. + Netsuke expands these placeholders to space-separated, shell-escaped lists + of file paths before hashing the action. When generating the Ninja rule, the + lists are replaced with Ninja's `$in` and `$out` macros. After interpolation + the command must be parsable by [`shlex`]( + shlex/). Any interpolation other than `ins` or `outs` is automatically shell + escaped. - `script`: A multi-line script declared with the YAML `|` block style. The - entire block is passed to an interpreter (currently `/bin/sh`). For `/bin/sh` - scripts, each interpolation is automatically passed through the `shell_escape` - filter unless a `| raw` filter is applied. Future versions will allow - configurable script languages with their own escaping rules. + entire block is passed to an interpreter. If the first line begins with `#! + ` Netsuke executes the script verbatim, respecting the shebang. Otherwise the + block is wrapped in the interpreter specified by the optional `interpreter` + field (defaulting to `/bin/sh -e`). For `/bin/sh` scripts, each interpolation + is automatically passed through the `shell_escape` filter unless a `| raw` + filter is applied. Future versions will allow configurable script languages + with their own escaping rules. Exactly one of `command` or `script` must be provided. The manifest parser enforces this rule to prevent invalid states. @@ -219,8 +229,10 @@ rule: - `script`: A multi-line script passed to the interpreter. When present, it is defined using the YAML `|` block style. - Only one of `rule`, `command`, or `script` may be specified. The parser - validates this exclusivity during deserialisation. +Only one of `rule`, `command`, or `script` may be specified. The parser +validates this exclusivity during deserialisation. If multiple fields are +provided, Netsuke emits a `RecipeConflict` error with the message "rule, command +and script are mutually exclusive". This union deserialises into the same `Recipe` enum used for rules. The parser enforces that only one variant is present, maintaining backward compatibility @@ -336,7 +348,7 @@ use std::collections::HashMap; /// Represents the top-level structure of a Netsukefile file. #[serde(deny_unknown_fields)] pub struct NetsukeManifest { - pub Netsuke_version: String, + pub netsuke_version: String, #[serde(default)] pub vars: HashMap, @@ -561,8 +573,9 @@ providing a secure bridge to the underlying system. - `glob(pattern: &str) -> Result, Error>`: A function that performs file path globbing. This is a critical feature for any modern build tool, allowing users to easily specify sets of source files (e.g., `src/**/*.c`). - This function bridges a key feature gap, as Ninja itself does not support - globbing.[^3] + The results are returned sorted lexicographically and symlinks are followed to + keep builds deterministic. This function bridges a key feature gap, as Ninja + itself does not support globbing.[^3] - `python_version(requirement: &str) -> Result`: An example of a domain-specific helper function that demonstrates the extensibility of this @@ -657,7 +670,10 @@ operations. | `base64` / `hex` | Encode bytes or string | | `slugify` | Make filename-safe slug | | `snake_case` / `camel_case` / `kebab-case` | Rename helpers | -| `shell_escape` | POSIX-quote for safe `$()` usage | + +All built-in filters use `snake_case`. The `camel_case` helper is provided in +place of `camelCase` so naming remains consistent with `snake_case` and `kebab- +case`. #### Generic collection filters @@ -667,7 +683,7 @@ operations. | `flatten` | One-level flatten of nested lists | | `group_by(attr)` | Dict keyed on `attr` of list items | | `zip(other)` | Pairwise tuples of two lists | -| `version_compare(other, op='=>')` | SemVer comparison (`'<'`, `'<=', '==', …`) | +| `version_compare(other, op='>=')` | SemVer comparison (`'<'`, `'<=', '==', …`) | #### Network & command functions / filters @@ -679,6 +695,10 @@ operations. | `shell(cmd)` | **filter** | Pipe value to arbitrary shell command; marks template **impure** | | `grep`, `sed`, `awk`, `cut`, `wc`, `tr` | filters | Canonical wrappers implemented via `shell()` for convenience | +Using `shell()` marks the template as *impure* and disables caching of the +rendered YAML between Stage 2 and Stage 3. This avoids accidental +reuse of results that depend on external commands. + Custom external commands can be registered as additional filters. Those should be marked `pure` if safe for caching or `impure` otherwise. @@ -726,6 +746,12 @@ a distributed build system), only a new generator module (`IR -> NewBackend`) would need to be written, leaving the entire front-end parsing and validation logic untouched. +Importantly, the IR contains **no Ninja-isms**. Placeholders such as `$in` +and `$out` are resolved to plain lists of file paths, and command strings are +expanded before hashing. This deliberate absence of Ninja-specific syntax makes +the IR a stable contract that future back-ends—distributed builders, remote +executors, or otherwise—can consume without modification. + Furthermore, the IR is the ideal stage at which to perform graph-level analysis and optimizations, such as detecting circular dependencies, pruning unused build targets, or identifying duplicate build actions. @@ -803,9 +829,11 @@ consumes a `NetsukeManifest` (the AST) and produces a `BuildGraph` (the IR). This transformation involves several steps: 1. **Action Consolidation:** Iterate through the `manifest.rules` from the AST. - For each rule, create a corresponding `ir::Action` struct. These actions are - stored in the `BuildGraph`'s `actions` map, keyed by a hash of their contents - to automatically deduplicate identical rules. + For each rule, create a corresponding `ir::Action` struct. These actions + are stored in the `BuildGraph`'s `actions` map, keyed by a hash of their + fully resolved command text, interpreter, local variables, and depfile + options. This ensures deduplication only occurs when two actions are truly + interchangeable. 2. **Target Expansion:** Iterate through the `manifest.targets` and the optional `manifest.actions`. Entries in `actions` are treated identically to targets @@ -831,12 +859,10 @@ structures to the Ninja file syntax. 1. **Write Variables:** Any global variables that need to be passed to Ninja can be written at the top of the file (e.g., `msvc_deps_prefix` for Windows - builds.[^8] - 2. **Write Rules:** Iterate through the `graph.actions` map. For each - `ir::Action`, write a corresponding Ninja `rule` statement to the output - file. The command placeholders (`{{ ins }}`, `{{ outs }}`) are replaced with - Ninja's variables (`$in`, `$out`). + `ir::Action`, write a corresponding Ninja `rule` statement. The input and + output lists stored in the action replace the `ins` and `outs` placeholders. + These lists are then rewritten as Ninja's `$in` and `$out` When an action's `recipe` is a script, the generated rule wraps the script in an invocation of `/bin/sh -e -c` so that multi-line scripts execute @@ -911,6 +937,10 @@ The command construction will follow this pattern: streamed to the user's console, potentially with additional formatting or status updates from Netsuke itself. +On Windows, Netsuke defaults to executing scripts via `powershell -Command` +unless an `interpreter` field is set at the manifest root. This early +acknowledgement aims to minimise surprises on CI systems. + ### 6.2 The Criticality of Shell Escaping A primary security responsibility for Netsuke is the prevention of command @@ -941,17 +971,14 @@ for building command strings by pushing quoted components into a buffer: The command generation logic within the `ninja_gen.rs` module must not use simple string formatting (like `format!`) to construct the final command strings -for the `build.ninja` file. Doing so would be inherently insecure. - -Instead, the implementation must parse the Netsuke command template (e.g., -`{cc} -c {ins} -o {outs}`) and build the final command string piece by piece. -The placeholders `{{ ins }}` and `{{ outs }}` remain as Ninja's $in and $out -variables. After substitution, Netsuke verifies that the resulting command -string can be parsed by the `shlex` crate. For each segment of the command, if -it is a variable substitution (like `{ins}`), the value of that variable must be -passed through the `shell-quote` API before being appended to the output string. -This ensures that every dynamic part of the command is correctly and safely -quoted for the target shell. +Instead, parse the Netsuke command template (e.g., `{cc} -c {ins} -o {outs}`) +and build the final command string step by step. The placeholders `{{ ins }} +` and `{{ outs }}` are expanded to space-separated lists of file paths within +Netsuke itself. Each path is shell escaped using the `shell-quote` API. When the +command is written to `build.ninja`, these lists are replaced with Ninja's `$in` +and `$out` macros. After substitution the command is validated with [`shlex`] +() to ensure it parses correctly. This +approach guarantees that every dynamic part of the command is securely quoted. ### 6.4 Automatic Security as a "Friendliness" Feature @@ -1060,6 +1087,10 @@ enrichment: the user with a rich, layered explanation of the failure, from the general to the specific. +For automation use cases Netsuke will support a `--diag-json` flag. When +enabled, the entire error chain is serialised to JSON, allowing editors and CI +tools to annotate failures inline. + ### 7.4 Table: Transforming Errors into User-Friendly Messages This table provides a specification for the desired output of Netsuke's error @@ -1155,11 +1186,11 @@ The behaviour of each subcommand is clearly defined: - `Netsuke graph`: This command is an introspection and debugging tool. It will run the Netsuke pipeline up to Stage 4 (IR Generation) and then invoke Ninja - with the graph tool, `ninja -t graph`.3 This will output the complete build - dependency graph in the DOT language, which can be rendered into an image - using tools like Graphviz. This allows users to visually inspect their build - dependencies, which is invaluable for understanding and debugging complex - projects. + with the graph tool, `ninja -t graph`.3 This outputs the complete build + dependency graph in the DOT language. The result can be piped through `dot + -Tsvg` or displayed via `netsuke graph --html` using an embedded Dagre.js + viewer. Visualising the graph is invaluable for understanding and debugging + complex projects. ## Section 9: Implementation Roadmap and Strategic Recommendations @@ -1248,14 +1279,14 @@ goal. This table serves as a quick-reference guide to the core third-party crates selected for this project and the rationale for their inclusion. -| Component | Recommended Crate | Rationale | -| -------------- | ------------------ | --------------------------------------------------------------------------------------------------------------------- | -| CLI Parsing | clap | The Rust standard for powerful, derive-based CLI development. | -| YAML Parsing | serde_yaml | Mature, stable, and provides seamless integration with the serde framework. | -| Templating | minijinja | High compatibility with Jinja2, minimal dependencies, and supports runtime template loading. | -| Shell Quoting | shell-quote | A critical security component; provides robust, shell-specific escaping for command arguments. | -| Error Handling | anyhow + thiserror | An idiomatic and powerful combination for creating rich, contextual, and user-friendly error reports. | -| Versioning | semver | The standard library for parsing and evaluating Semantic Versioning strings, essential for the Netsuke_version field. | +| Component | Recommended Crate | Rationale | +| -------------- | ------------------ | ----------------------------------------------------------------------------------------------------------------------- | +| CLI Parsing | clap | The Rust standard for powerful, derive-based CLI development. | +| YAML Parsing | serde_yaml | Mature, stable, and provides seamless integration with the serde framework. | +| Templating | minijinja | High compatibility with Jinja2, minimal dependencies, and supports runtime template loading. | +| Shell Quoting | shell-quote | A critical security component; provides robust, shell-specific escaping for command arguments. | +| Error Handling | anyhow + thiserror | An idiomatic and powerful combination for creating rich, contextual, and user-friendly error reports. | +| Versioning | semver | The standard library for parsing and evaluating Semantic Versioning strings, essential for the `netsuke_version` field. | ### 9.3 Future Enhancements From 556c370eaf73ab1e39702a22a74d4818b6781779 Mon Sep 17 00:00:00 2001 From: Leynos Date: Sun, 13 Jul 2025 23:07:44 +0100 Subject: [PATCH 2/7] Fix shlex link formatting --- docs/netsuke-design.md | 27 +++++++++++++-------------- 1 file changed, 13 insertions(+), 14 deletions(-) diff --git a/docs/netsuke-design.md b/docs/netsuke-design.md index 3086a32f..c5b2db6e 100644 --- a/docs/netsuke-design.md +++ b/docs/netsuke-design.md @@ -72,10 +72,10 @@ before execution, a critical requirement for compatibility with Ninja. 5. Stage 5: Ninja Synthesis & Execution The final, validated IR is traversed by a code generator. This generator - synthesizes the content of a `build.ninja` file, translating the IR's nodes + synthesises the content of a `build.ninja` file, translating the IR's nodes and edges into corresponding Ninja rule and build statements. Once the file is written, Netsuke invokes the `ninja` executable as a subprocess, passing - control to it for the final dependency checking and command execution phase. + control to it for the final dependency checking and command-execution phase. Netsuke's pipeline is **deterministic**. Given the same `Netsukefile` and environment variables, the generated `build.ninja` will be byte-for-byte @@ -178,13 +178,12 @@ Each entry in the `rules` list is a mapping that defines a reusable action. Netsuke expands these placeholders to space-separated, shell-escaped lists of file paths before hashing the action. When generating the Ninja rule, the lists are replaced with Ninja's `$in` and `$out` macros. After interpolation - the command must be parsable by [`shlex`]( - shlex/). Any interpolation other than `ins` or `outs` is automatically shell - escaped. + the command must be parsable by [shlex](https://docs.rs/shlex/latest/shlex/). + Any interpolation other than `ins` or `outs` is automatically shell-escaped. - `script`: A multi-line script declared with the YAML `|` block style. The - entire block is passed to an interpreter. If the first line begins with `#! - ` Netsuke executes the script verbatim, respecting the shebang. Otherwise the + entire block is passed to an interpreter. If the first line begins with `#!` + Netsuke executes the script verbatim, respecting the shebang. Otherwise, the block is wrapped in the interpreter specified by the optional `interpreter` field (defaulting to `/bin/sh -e`). For `/bin/sh` scripts, each interpolation is automatically passed through the `shell_escape` filter unless a `| raw` @@ -862,7 +861,7 @@ structures to the Ninja file syntax. 2. **Write Rules:** Iterate through the `graph.actions` map. For each `ir::Action`, write a corresponding Ninja `rule` statement. The input and output lists stored in the action replace the `ins` and `outs` placeholders. - These lists are then rewritten as Ninja's `$in` and `$out` + These lists are then rewritten as Ninja's `$in` and `$out`. When an action's `recipe` is a script, the generated rule wraps the script in an invocation of `/bin/sh -e -c` so that multi-line scripts execute @@ -974,9 +973,9 @@ simple string formatting (like `format!`) to construct the final command strings Instead, parse the Netsuke command template (e.g., `{cc} -c {ins} -o {outs}`) and build the final command string step by step. The placeholders `{{ ins }} ` and `{{ outs }}` are expanded to space-separated lists of file paths within -Netsuke itself. Each path is shell escaped using the `shell-quote` API. When the -command is written to `build.ninja`, these lists are replaced with Ninja's `$in` -and `$out` macros. After substitution the command is validated with [`shlex`] +Netsuke itself, each path being shell-escaped using the `shell-quote` API. +When the command is written to `build.ninja`, these lists replace Ninja's `$in` +and `$out` macros. After substitution, the command is validated with [`shlex`] () to ensure it parses correctly. This approach guarantees that every dynamic part of the command is securely quoted. @@ -1087,7 +1086,7 @@ enrichment: the user with a rich, layered explanation of the failure, from the general to the specific. -For automation use cases Netsuke will support a `--diag-json` flag. When +For automation use cases, Netsuke will support a `--diag-json` flag. When enabled, the entire error chain is serialised to JSON, allowing editors and CI tools to annotate failures inline. @@ -1185,8 +1184,8 @@ The behaviour of each subcommand is clearly defined: as `ninja -t clean`, to remove the outputs of the build rules. - `Netsuke graph`: This command is an introspection and debugging tool. It will - run the Netsuke pipeline up to Stage 4 (IR Generation) and then invoke Ninja - with the graph tool, `ninja -t graph`.3 This outputs the complete build + run the Netsuke pipeline up to Stage 4 (IR Generation) and then invoke + Ninja with the graph tool, `ninja -t graph`. This outputs the complete build dependency graph in the DOT language. The result can be piped through `dot -Tsvg` or displayed via `netsuke graph --html` using an embedded Dagre.js viewer. Visualising the graph is invaluable for understanding and debugging From 64449a34842f55c43e8b32b00a51909062c1062a Mon Sep 17 00:00:00 2001 From: Leynos Date: Sun, 13 Jul 2025 23:22:36 +0100 Subject: [PATCH 3/7] Fix spelling and add architecture diagram --- docs/netsuke-design.md | 25 ++++++++++++++++++++----- 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/docs/netsuke-design.md b/docs/netsuke-design.md index c5b2db6e..09a523e0 100644 --- a/docs/netsuke-design.md +++ b/docs/netsuke-design.md @@ -72,7 +72,7 @@ before execution, a critical requirement for compatibility with Ninja. 5. Stage 5: Ninja Synthesis & Execution The final, validated IR is traversed by a code generator. This generator - synthesises the content of a `build.ninja` file, translating the IR's nodes + synthesizes the content of a `build.ninja` file, translating the IR's nodes and edges into corresponding Ninja rule and build statements. Once the file is written, Netsuke invokes the `ninja` executable as a subprocess, passing control to it for the final dependency checking and command-execution phase. @@ -82,6 +82,21 @@ before execution, a critical requirement for compatibility with Ninja. identical. This property is essential for reproducible builds and makes the output suitable for caching or source control. +```mermaid +sequenceDiagram + participant User + participant Netsuke + participant IR_Generator + participant Ninja + + User->>Netsuke: Provide Netsukefile and environment + Netsuke->>IR_Generator: Parse and validate manifest + IR_Generator->>IR_Generator: Generate deterministic BuildGraph (IR) + IR_Generator->>Netsuke: Return BuildGraph (byte-for-byte deterministic) + Netsuke->>Ninja: Synthesize build.ninja and invoke Ninja + Ninja-->>User: Execute build and report results +``` + ### 1.3 The Static Graph Mandate The architecture's multi-stage pipeline is a direct consequence of a fundamental @@ -193,7 +208,7 @@ Each entry in the `rules` list is a mapping that defines a reusable action. Exactly one of `command` or `script` must be provided. The manifest parser enforces this rule to prevent invalid states. - Internally, these options deserialise into a shared `Recipe` enum tagged with + Internally, these options deserialize into a shared `Recipe` enum tagged with a `kind` field. Serde aliases ensure manifests that omit the tag continue to load correctly. @@ -229,11 +244,11 @@ rule: defined using the YAML `|` block style. Only one of `rule`, `command`, or `script` may be specified. The parser -validates this exclusivity during deserialisation. If multiple fields are +validates this exclusivity during deserialization. If multiple fields are provided, Netsuke emits a `RecipeConflict` error with the message "rule, command and script are mutually exclusive". - This union deserialises into the same `Recipe` enum used for rules. The parser + This union deserializes into the same `Recipe` enum used for rules. The parser enforces that only one variant is present, maintaining backward compatibility through serde aliases when `kind` is omitted. @@ -1087,7 +1102,7 @@ enrichment: the specific. For automation use cases, Netsuke will support a `--diag-json` flag. When -enabled, the entire error chain is serialised to JSON, allowing editors and CI +enabled, the entire error chain is serialized to JSON, allowing editors and CI tools to annotate failures inline. ### 7.4 Table: Transforming Errors into User-Friendly Messages From d50aed6ff3521bade87522a955e2eeb30f0eb5a7 Mon Sep 17 00:00:00 2001 From: Leynos Date: Sun, 13 Jul 2025 23:43:35 +0100 Subject: [PATCH 4/7] Refine design document --- docs/netsuke-design.md | 28 +++++++++++++--------------- 1 file changed, 13 insertions(+), 15 deletions(-) diff --git a/docs/netsuke-design.md b/docs/netsuke-design.md index 09a523e0..2dd54374 100644 --- a/docs/netsuke-design.md +++ b/docs/netsuke-design.md @@ -203,7 +203,8 @@ Each entry in the `rules` list is a mapping that defines a reusable action. field (defaulting to `/bin/sh -e`). For `/bin/sh` scripts, each interpolation is automatically passed through the `shell_escape` filter unless a `| raw` filter is applied. Future versions will allow configurable script languages - with their own escaping rules. + with their own escaping rules. On Windows, scripts default to `powershell + -Command` unless the manifest's `interpreter` field overrides the setting. Exactly one of `command` or `script` must be provided. The manifest parser enforces this rule to prevent invalid states. @@ -244,8 +245,8 @@ rule: defined using the YAML `|` block style. Only one of `rule`, `command`, or `script` may be specified. The parser -validates this exclusivity during deserialization. If multiple fields are -provided, Netsuke emits a `RecipeConflict` error with the message "rule, command +validates this exclusivity during deserialization. When multiple fields are +present, Netsuke emits a `RecipeConflict` error with the message "rule, command and script are mutually exclusive". This union deserializes into the same `Recipe` enum used for rules. The parser @@ -951,10 +952,6 @@ The command construction will follow this pattern: streamed to the user's console, potentially with additional formatting or status updates from Netsuke itself. -On Windows, Netsuke defaults to executing scripts via `powershell -Command` -unless an `interpreter` field is set at the manifest root. This early -acknowledgement aims to minimise surprises on CI systems. - ### 6.2 The Criticality of Shell Escaping A primary security responsibility for Netsuke is the prevention of command @@ -985,14 +982,15 @@ for building command strings by pushing quoted components into a buffer: The command generation logic within the `ninja_gen.rs` module must not use simple string formatting (like `format!`) to construct the final command strings -Instead, parse the Netsuke command template (e.g., `{cc} -c {ins} -o {outs}`) -and build the final command string step by step. The placeholders `{{ ins }} -` and `{{ outs }}` are expanded to space-separated lists of file paths within -Netsuke itself, each path being shell-escaped using the `shell-quote` API. -When the command is written to `build.ninja`, these lists replace Ninja's `$in` -and `$out` macros. After substitution, the command is validated with [`shlex`] -() to ensure it parses correctly. This -approach guarantees that every dynamic part of the command is securely quoted. +Instead, parse the Netsuke command template (e.g., `{{ cc }} -c {{ ins }} -o` +`{{ outs }}`) and build the final command string step by step. The placeholders +`{{ ins }}` and `{{ outs }}` are expanded to space-separated lists of file +paths within Netsuke itself, each path being shell-escaped using the `shell- +quote` API. When the command is written to `build.ninja`, these lists replace +Ninja's `$in` and `$out` macros. After substitution, the command is validated +with [`shlex`] () to ensure it parses +correctly. This approach guarantees that every dynamic part of the command is +securely quoted. ### 6.4 Automatic Security as a "Friendliness" Feature From 1f12f0d4689cfd1d491d823ff8bf7522e1275103 Mon Sep 17 00:00:00 2001 From: Leynos Date: Sun, 13 Jul 2025 23:43:42 +0100 Subject: [PATCH 5/7] Add class diagram and fix placeholder --- docs/netsuke-design.md | 87 +++++++++++++++++++++++++++++++++++------- 1 file changed, 73 insertions(+), 14 deletions(-) diff --git a/docs/netsuke-design.md b/docs/netsuke-design.md index 2dd54374..dfdc2292 100644 --- a/docs/netsuke-design.md +++ b/docs/netsuke-design.md @@ -182,6 +182,65 @@ level keys. invoked without any specific targets on the command line. This maps directly to Ninja's `default` target statement.[^3] +The class diagram below summarises the structure of a `Netsukefile` and the +relationships between its components. + +```mermaid +classDiagram + class NetsukeManifest { + +String netsuke_version + +HashMap vars + +Vec rules + +Vec actions + +Vec targets + +Vec defaults + } + class Target { + +StringOrList name + +Recipe recipe + +StringOrList sources + +StringOrList deps + +StringOrList order_only_deps + +HashMap vars + +bool phony + +bool always + } + class Rule { + +String name + +Recipe recipe + +String description + +String deps + } + class Recipe { + <> + Command + Script + Rule + } + class StringOrList { + <> + Empty + String + List + } + class Value { + <> + Null + Bool + Number + String + Sequence + Mapping + Tagged + } + NetsukeManifest "1" o-- "*" Target + NetsukeManifest "1" o-- "*" Rule + NetsukeManifest "1" o-- "*" Value + Target "1" -- "1" Recipe + Target "1" -- "1" StringOrList + Rule "1" -- "1" Recipe +``` + ### 2.3 Defining `rules` Each entry in the `rules` list is a mapping that defines a reusable action. @@ -309,13 +368,13 @@ table compares a simple C compilation project defined in both a traditional `Makefile` and a `Netsukefile` file. The comparison highlights Netsuke's explicit, structured, and self-documenting nature. -| Feature | Makefile Example | Netsukefile Example | -| --------------- | ---------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | -| Variables | CC=gcc | { vars: { cc: gcc } } | -| Macros | define greet\\t@echo Hello $$1endef | { macros: { signature: "greet(name)", body: "Hello {{ name }}" } } | -| Rule Definition | %.o: %.c\\n\\t$(CC) -c $< -o $@ | { rules: { name: compile, command: "{{ cc }} -c {{ ins }} -o {{ out }}", description: "Compiling {{ outs }}" } } | -| Target Build | my_program: main.o utils.o\\t$(CC) $^ -o $@ | { targets: { name: my_program, rule: link, sources: \[main.o, utils.o\] } | -| Readability | Relies on cryptic automatic variables ($@, $\<, $^) and implicit pattern matching. | Uses explicit, descriptive keys (name, rule, sources) and standard YAML list/map syntax. | +| Feature | Makefile Example | Netsukefile Example | +| --------------- | ---------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- | +| Variables | CC=gcc | { vars: { cc: gcc } } | +| Macros | define greet\\t@echo Hello $$1endef | { macros: { signature: "greet(name)", body: "Hello {{ name }}" } } | +| Rule Definition | %.o: %.c\\n\\t$(CC) -c $< -o $@ | { rules: { name: compile, command: "{{ cc }} -c {{ ins }} -o {{ outs }}", description: "Compiling {{ outs }}" } } | +| Target Build | my_program: main.o utils.o\\t$(CC) $^ -o $@ | { targets: { name: my_program, rule: link, sources: \[main.o, utils.o\] } | +| Readability | Relies on cryptic automatic variables ($@, $\<, $^) and implicit pattern matching. | Uses explicit, descriptive keys (name, rule, sources) and standard YAML list/map syntax. | ## Section 3: Parsing and Deserialization Strategy @@ -984,13 +1043,13 @@ The command generation logic within the `ninja_gen.rs` module must not use simple string formatting (like `format!`) to construct the final command strings Instead, parse the Netsuke command template (e.g., `{{ cc }} -c {{ ins }} -o` `{{ outs }}`) and build the final command string step by step. The placeholders -`{{ ins }}` and `{{ outs }}` are expanded to space-separated lists of file -paths within Netsuke itself, each path being shell-escaped using the `shell- -quote` API. When the command is written to `build.ninja`, these lists replace -Ninja's `$in` and `$out` macros. After substitution, the command is validated -with [`shlex`] () to ensure it parses -correctly. This approach guarantees that every dynamic part of the command is -securely quoted. +`{{ ins }}` and `{{ outs }}` are expanded to space-separated lists of file paths +within Netsuke itself, each path being shell-escaped using the `shell- quote` +API. When the command is written to `build.ninja`, these lists replace Ninja's +`$in` and `$out` macros. After substitution, the command is validated with +[`shlex`] () to ensure it parses correctly. +This approach guarantees that every dynamic part of the command is securely +quoted. ### 6.4 Automatic Security as a "Friendliness" Feature From 2fd30f8651b95b6ebbab03fb94400319316e1c9a Mon Sep 17 00:00:00 2001 From: Leynos Date: Mon, 14 Jul 2025 00:25:10 +0100 Subject: [PATCH 6/7] =?UTF-8?q?Use=20Oxford=20=E2=80=9C-ize=E2=80=9D=20and?= =?UTF-8?q?=20re-flow?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> --- docs/netsuke-design.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/netsuke-design.md b/docs/netsuke-design.md index dfdc2292..046f1797 100644 --- a/docs/netsuke-design.md +++ b/docs/netsuke-design.md @@ -182,7 +182,7 @@ level keys. invoked without any specific targets on the command line. This maps directly to Ninja's `default` target statement.[^3] -The class diagram below summarises the structure of a `Netsukefile` and the +The class diagram below summarizes the structure of a `Netsukefile` and the relationships between its components. ```mermaid From b920a8036837b76e5bb84d8708a46b61aa3430ad Mon Sep 17 00:00:00 2001 From: Leynos Date: Mon, 14 Jul 2025 00:25:45 +0100 Subject: [PATCH 7/7] =?UTF-8?q?Inline=20=E2=80=9Cpowershell=20-Command?= =?UTF-8?q?=E2=80=9D=20and=20remove=20stray=20newline?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> --- docs/netsuke-design.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/netsuke-design.md b/docs/netsuke-design.md index 046f1797..381b0d5b 100644 --- a/docs/netsuke-design.md +++ b/docs/netsuke-design.md @@ -262,9 +262,9 @@ Each entry in the `rules` list is a mapping that defines a reusable action. field (defaulting to `/bin/sh -e`). For `/bin/sh` scripts, each interpolation is automatically passed through the `shell_escape` filter unless a `| raw` filter is applied. Future versions will allow configurable script languages - with their own escaping rules. On Windows, scripts default to `powershell - -Command` unless the manifest's `interpreter` field overrides the setting. - + with their own escaping rules. + On Windows, scripts default to `powershell -Command` unless the manifest's + `interpreter` field overrides the setting. Exactly one of `command` or `script` must be provided. The manifest parser enforces this rule to prevent invalid states.