Drive Stage 6 task progress from Ninja status lines monotonic updates#268
Drive Stage 6 task progress from Ninja status lines monotonic updates#268
Conversation
…progress Add a new parser module for Ninja status lines to extract task-level progress updates and integrate it with subprocess output forwarding. Extend status reporters to reflect task progress during Stage 6 execution. Implement fallback textual updates for non-TTY stdout and accessible mode, controlled by the existing progress configuration with localized CLI help. Add comprehensive unit and behavioural tests to cover happy, unhappy, and edge cases. Update user and design documentation to reflect new progress reporting. This implements roadmap item 3.9.2, enabling Netsuke to show deterministic task progress from Ninja build output. Co-authored-by: devboxerhub[bot] <devboxerhub[bot]@users.noreply.github.com>
Reviewer's GuideImplements a Ninja status-line parsing and reporting pipeline so Stage 6 task progress is driven from Ninja output while preserving streaming behavior, adds deterministic textual fallbacks (non-TTY and accessible), and updates reporters, localization, tests, and documentation accordingly. Sequence diagram for Stage 6 Ninja-driven task progress reportingsequenceDiagram
actor User
participant Cli as Cli
participant Runner as Runner_run
participant Reporter as StatusReporter
participant Process as Process_run_ninja_invocation
participant Streaming as Streaming_forward
participant Ninja as Ninja_process
User->>Cli: netsuke build args
Cli->>Runner: run(cli, prefs)
Runner->>Runner: output_mode::resolve(cli.accessible)
Runner->>Runner: stdout_is_tty = stdout.is_terminal()
Runner->>Runner: reporter = make_reporter(mode, progress_enabled, prefs, stdout_is_tty)
Runner->>Process: run_ninja_invocation(NinjaInvocation::Build, Some(on_task_progress))
Process->>Process: configure_ninja_build_command(cmd, cli, build_file, targets)
Process->>Process: run_command_and_stream(cmd, status_observer)
Process->>Ninja: spawn()
Process->>Streaming: spawn_and_stream_output(child, status_observer)
par stderr_forwarding
Streaming->>Streaming: forward_child_output(stderr_reader, stderr_writer, stderr)
Streaming->>Reporter: no task updates (stderr only)
and stdout_forwarding
Streaming->>Streaming: forward_child_output_with_ninja_status(stdout_reader, stdout_writer, observer, stdout)
loop per_status_line
Ninja-->>Streaming: "[current/total] description\n"
Streaming->>Streaming: parse_ninja_status_line(line)
Streaming->>Streaming: NinjaTaskProgressTracker.accept(progress)
alt valid_and_monotonic
Streaming->>Runner: observer(current, total, description)
Runner->>Reporter: report_task_progress(current, total, description)
alt AccessibleReporter
Reporter->>Reporter: task_progress_update()
Reporter->>stderr: write textual task line
else IndicatifReporter
Reporter->>Reporter: validate_stage6_active_and_monotonic()
alt state_is_hidden_or_force_text_task_updates
Reporter->>stderr: write textual task line
else interactive_progress
Reporter->>Reporter: stage_label()
Reporter->>Reporter: task_progress_update()
Reporter->>MultiProgress: set_message(stage6_bar, summary_with_task)
end
end
else invalid_or_regressive
Streaming->>Streaming: ignore update
end
end
end
Streaming-->>Process: ForwardStats for stdout and stderr
Ninja-->>Process: ExitStatus
Process->>Process: check_exit_status(status)
Process-->>Runner: Ok(())
Runner->>Reporter: report_complete(tool_key)
Runner-->>Cli: Result
Cli-->>User: exit code
Class diagram for StatusReporter and task progress implementationsclassDiagram
class StatusReporter {
<<trait>>
+report_stage(current: StageNumber, total: StageNumber, description: &str) void
+report_task_progress(current: u32, total: u32, description: &str) void
+report_complete(tool_key: LocalizationKey) void
}
class AccessibleReporter {
+new(prefs: OutputPrefs) AccessibleReporter
+report_stage(current: StageNumber, total: StageNumber, description: &str) void
+report_task_progress(current: u32, total: u32, description: &str) void
+report_complete(tool_key: LocalizationKey) void
-write_prefix() String
}
class SilentReporter {
+report_stage(current: StageNumber, total: StageNumber, description: &str) void
+report_task_progress(current: u32, total: u32, description: &str) void
+report_complete(tool_key: LocalizationKey) void
}
class IndicatifReporter {
+new(force_text_task_updates: bool) IndicatifReporter
+default() IndicatifReporter
+report_stage(current: StageNumber, total: StageNumber, description: &str) void
+report_task_progress(current: u32, total: u32, description: &str) void
+report_complete(tool_key: LocalizationKey) void
-state: Arc<Mutex<IndicatifState>>
-is_valid_task_progress(current: u32, total: u32) bool
-is_stage6_active(state: IndicatifState, stage_index: usize) bool
-is_task_progress_monotonic(state: IndicatifState, current: u32, total: u32) bool
}
class IndicatifState {
-progress: MultiProgress
-bars: Vec<ProgressBar>
-descriptions: Vec<String>
-running_index: Option<usize>
-completed: bool
-is_hidden: bool
-force_text_task_updates: bool
-last_task_progress: Option<(u32, u32)>
}
class Helpers {
<<module>>
+stage_summary(current: StageNumber, total: StageNumber, description: &str) String
+task_progress_label(current: u32, total: u32) String
+task_progress_update(current: u32, total: u32, description: &str) String
+stage_label(current: StageNumber, total: StageNumber, description: &str) String
+stage6_index() usize
}
StatusReporter <|.. AccessibleReporter
StatusReporter <|.. SilentReporter
StatusReporter <|.. IndicatifReporter
IndicatifReporter o--> IndicatifState
Helpers ..> AccessibleReporter
Helpers ..> IndicatifReporter
Class diagram for Ninja status parsing and streaming pipelineclassDiagram
class NinjaTaskProgress {
+new(current: u32, total: u32, description: String) NinjaTaskProgress
+current() u32
+total() u32
+description() &str
-current: u32
-total: u32
-description: String
}
class NinjaTaskProgressTracker {
+accept(update: &NinjaTaskProgress) bool
-total: Option<u32>
-last_current: u32
-is_invalid_update(update: &NinjaTaskProgress) bool
}
class ninja_status_module {
<<module>>
+parse_ninja_status_line(line: &str) Option<NinjaTaskProgress>
}
class ForwardStats {
<<struct>>
+bytes_read: usize
+bytes_written: usize
+write_failed: bool
}
class CountingReader~R: Read~ {
-inner: &mut R
-read: u64
+read(buf: &mut [u8]) Result<usize>
}
class CountingWriter~W: Write~ {
-inner: &mut W
-written: u64
+write(buf: &[u8]) Result<usize>
+flush() Result
}
class NinjaStatusParsingReader~R: Read, F: FnMut(u32,u32,&str)~ {
-inner: &mut R
-tracker: NinjaTaskProgressTracker
-pending_line: Vec<u8>
-observer: &mut F
+read(buf: &mut [u8]) Result<usize>
-consume_bytes(bytes: &[u8]) void
-finish_line() void
}
class streaming_module {
<<module>>
+forward_child_output(reader: R, writer: W, stream_name: &'static str) ForwardStats
+forward_child_output_with_ninja_status(reader: R, writer: W, observer: F, stream_name: &'static str) ForwardStats
-copy_with_stats(reader: &mut R, writer: &mut W, stream_name: &'static str) ForwardStats
-clamp_u64_to_usize(value: u64) usize
}
ninja_status_module --> NinjaTaskProgress
ninja_status_module --> NinjaTaskProgressTracker
NinjaTaskProgressTracker --> NinjaTaskProgress
streaming_module --> ForwardStats
streaming_module --> CountingReader
streaming_module --> CountingWriter
streaming_module --> NinjaStatusParsingReader
NinjaStatusParsingReader --> NinjaTaskProgressTracker
NinjaStatusParsingReader --> NinjaTaskProgress
NinjaStatusParsingReader ..> ninja_status_module
class Process_module {
<<module>>
+NinjaInvocation
+run_ninja_invocation(invocation: &NinjaInvocation, status_observer: Option<StatusObserver>) io::Result
+spawn_and_stream_output(child: Child, status_observer: Option<StatusObserver>) io::Result<ExitStatus>
-handle_forwarding_stats(stats: ForwardStats, stream_name: &str) void
}
Process_module --> streaming_module
Process_module --> ForwardStats
Process_module ..> NinjaTaskProgressTracker
Process_module ..> ninja_status_module
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
WalkthroughSummarise parsing Ninja status lines to drive Stage 6 task progress, add a Ninja status parser and streaming plumbing, wire progress-aware reporter selection with stdout TTY detection and OrthoConfig layering, extend localisation and docs, and add unit, BDD and localisation tests. Changes
Sequence Diagram(s)sequenceDiagram
actor User
participant CLI as CLI/Runner
participant Reporter as Status Reporter
participant Ninja as Ninja Process
participant Streaming as Output Streaming
participant Parser as Status Parser
User->>CLI: Run with --progress or config
CLI->>CLI: Detect stdout TTY and compute progress_enabled
CLI->>Reporter: Create reporter (force_text_task_updates?)
CLI->>Ninja: Spawn ninja_synthesis_execute (with status observer when enabled)
Ninja-->>Streaming: Emit stdout/stderr containing status lines
Streaming->>Parser: Deliver complete lines for parsing
Parser->>Parser: Parse and validate [current/total] description
Parser-->>Streaming: Emit NinjaTaskProgress if valid
Streaming-->>Reporter: Invoke on_task_progress observer
Reporter->>Reporter: Render Stage 6 update or produce text update
Reporter-->>User: Emit text/animated update (TTY-aware)
Ninja-->>Streaming: Continue emitting other output
Streaming-->>User: Forward remaining output and stats
Estimated code review effort🎯 4 (Complex) | ⏱️ ~50 minutes Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches
🧪 Generate unit tests (beta)
Comment |
- Added parsing of Ninja status lines in `src/runner/process/ninja_status.rs`. - Enhanced process streaming to handle and observe parsed task progress. - Integrated task progress reporting in `StatusReporter` trait and implementations. - Updated `IndicatifReporter` to display task progress and fallback to textual output when needed. - Enable textual task updates on non-TTY stdout or accessible mode. - Added BDD and unit tests covering happy/unhappy parsing cases and progress emission. - Updated CLI flags documentation and localization keys/messages for task progress. - Added a fake ninja executable test fixture emitting task status lines. - Synchronized docs, roadmap, and user guides with the new progress feature. - Ensured monotonic acceptance of task progress to avoid regressive updates. Co-authored-by: devboxerhub[bot] <devboxerhub[bot]@users.noreply.github.com>
|
@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix: src/runner/process/ninja_status.rs Comment on file impl NinjaTaskProgressTracker {
/// Accept a new update when it is consistent and monotonic.
pub(super) const fn accept(&mut self, update: &NinjaTaskProgress) -> bool {
if update.total() == 0 || update.current() == 0 || update.current() > update.total() {❌ New issue: Complex Conditional |
|
@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix: Comment on file if total == 0 || current == 0 || current > total {
return;
}
let mut state = self
.state
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
let stage_index = stage6_index();
if stage_index >= state.bars.len() || state.running_index != Some(stage_index) {
return;
}
if let Some((last_current, last_total)) = state.last_task_progress
&& (total != last_total || current < last_current)❌ New issue: Complex Conditional |
|
@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix: src/runner/process/ninja_status.rs Comment on file if current_raw.is_empty()
|| total_raw.is_empty()
|| !current_raw.bytes().all(|byte| byte.is_ascii_digit())
|| !total_raw.bytes().all(|byte| byte.is_ascii_digit())❌ New issue: Complex Conditional |
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
… validation of task progress updates - Introduced helper functions for validating numeric strings and task progress consistency. - Encapsulated invalid update checks inside NinjaTaskProgressTracker. - Added monotonicity and active stage checks in IndicatifReporter for safer progress reporting. - Simplified and clarified conditions to reject invalid or inconsistent progress updates. This refactoring improves code clarity, reusability, and correctness by centralizing validation logic and enforcing strict progress monotonocity and state consistency. Co-authored-by: devboxerhub[bot] <devboxerhub[bot]@users.noreply.github.com>
…ctor process calls - Introduce a `progress_enabled` boolean controlling whether task progress is reported during Ninja execution. - Refactor Ninja invocation APIs to accept typed requests and optional task progress callbacks. - Update build, clean, and graph commands to honor `progress_enabled` for conditional progress reporting. - Simplify IndicatifReporter task progress handling by removing state and improving stage 6 task progress display logic. - Update tests and BDD scenarios to exercise progress enabling/disabling. This change enables disabling progress output while maintaining standard invocation flow, improving user control over task progress reporting. Co-authored-by: devboxerhub[bot] <devboxerhub[bot]@users.noreply.github.com>
|
@coderabbitai Have the following now been resolved? |
|
@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix: Comment on file fn handle_clean(cli: &Cli, reporter: &dyn StatusReporter, progress_enabled: bool) -> Result<()> {
handle_ninja_tool(
cli,
NinjaToolSpec {
name: "clean",
key: keys::STATUS_TOOL_CLEAN.into(),
},
reporter,
progress_enabled,
)❌ New issue: Code Duplication |
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
- Implement parsing of Ninja status lines like `[current/total]` to drive task-level progress updates during build execution (Stage 6). - Add option to force textual task progress output fallback when stdout is non-TTY or accessible mode is active. - Update IndicatifReporter with ability to control forced text fallback for task progress. - Enhance tests for task progress reporting with new fixtures and verify behavior in different modes. - Document the parsing and progress reporting approach in execplans and user guide. - Improve concurrency in process streaming to avoid blocking on stderr while reporting task progress. This feature fulfills roadmap item 3.9.2 by enabling deterministic task progress reporting from Ninja output and ensures accessibility and CI-friendly textual fallbacks. Co-authored-by: devboxerhub[bot] <devboxerhub[bot]@users.noreply.github.com>
|
@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix: Code Duplicationsrc/runner/mod.rs: What lead to degradation?The module contains 2 functions with similar structure: handle_clean,handle_graph Why does this problem occur?Duplicated code often leads to code that's harder to change since the same logical change has to be done in multiple functions. More duplication gives lower code health. How to fix it?A certain degree of duplicated code might be acceptable. The problems start when it is the same behavior that is duplicated across the functions in the module, ie. a violation of the Don't Repeat Yourself (DRY) principle. DRY violations lead to code that is changed together in predictable patterns, which is both expensive and risky. DRY violations can be identified using CodeScene's X-Ray analysis to detect clusters of change coupled functions with high code similarity. Read More |
This comment was marked as resolved.
This comment was marked as resolved.
…rocess Replaced manual buffer slicing with split_at to safely obtain a byte slice in `streaming.rs`, reducing bounds checking complexity and potential errors. Also removed an unnecessary example in `runner/mod.rs` and fixed minor doc markdown formatting. Co-authored-by: devboxerhub[bot] <devboxerhub[bot]@users.noreply.github.com>
|
@coderabbitai Have the following now been resolved? Please address the following concerns and ensure all commit gates succeed:
|
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
Introduce a new constructor `with_text_task_updates` on `IndicatifReporter` that forces text-only task updates. This is useful for non-TTY environments or accessibility fallbacks where graphical task updates are not desired. Co-authored-by: devboxerhub[bot] <devboxerhub[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@docs/execplans/3-9-2-parse-ninja-status-lines-to-drive-task-progress.md`:
- Around line 23-24: Update the sentence containing "Standard interactive mode
keeps the existing `indicatif`-based UX, now with" by expanding the acronym on
first use: replace "UX" with "user experience (UX)" so the text reads
"...indicatif-based user experience (UX), now with..."; ensure the line wrapping
adheres to the 80-column guideline when editing this sentence in the document.
- Around line 26-27: Update the help-copy list so it uses the Oxford comma: in
the sentence containing the items `--progress`, `NETSUKE_PROGRESS`, and config
file key (the line reading "Progress control continues through OrthoConfig
layering with localized help copy (`--progress`, `NETSUKE_PROGRESS`, config file
key)."), insert a comma before the final conjunction so the list reads "...
`--progress`, `NETSUKE_PROGRESS`, and config file key."
ℹ️ Review info
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (4)
docs/execplans/3-9-2-parse-ninja-status-lines-to-drive-task-progress.mdsrc/runner/mod.rssrc/runner/process/streaming.rssrc/status.rs
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: Length of output: 666 🏁 Script executed: Length of output: 1196 🏁 Script executed: Length of output: 2247
Concern 1 (src/runner/tests.rs module doc) — ✅ ResolvedThe module doc has been updated to: //! Unit tests for runner path resolution, predicate helpers, and core helpers.
This is actually more specific and descriptive than the suggested generic "internal helpers". Concern 2 (src/status.rs named constructor) — ✅ Resolved (Better Than Suggested!)The implementation went beyond the suggested fix by adding two named constructors:
The actual call sites use the second variant: Box::new(IndicatifReporter::with_force_text_task_updates(force_text_task_updates))This is superior to the suggested fix because:
Both concerns are fully addressed. The code is now more readable and maintainable. 🐇 |
Refined description of the standard interactive mode from "indicatif-based UX" to "indicatif-based user experience (UX)" and clarified the progress control copy listing by adding "and" before the config file key. Co-authored-by: devboxerhub[bot] <devboxerhub[bot]@users.noreply.github.com>
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@docs/execplans/3-9-2-parse-ninja-status-lines-to-drive-task-progress.md`:
- Around line 399-406: The "## Artefacts and notes" section still contains
template placeholders; replace them with concrete evidence: list the exact
parser unit test names and their pass/fail status (e.g.,
"TestParser_ProgressLines — PASSED"), the specific BDD scenario names proving
fallback behaviour (e.g., "Fallback to non-TTY progress updates — PASSED"),
include a short stderr excerpt that shows the Stage 6 textual updates in non-TTY
mode (capture 3–4 lines verbatim), and record the actual exit codes for the
commands `make check-fmt`, `make lint`, and `make test` (e.g., "make check-fmt:
0"); update the bullets under "## Artefacts and notes" accordingly so reviewers
see concrete results.
- Around line 133-135: The term "bidi" is an unexpanded abbreviation in the
sentence "Fluent rendering includes bidi isolation markers..."—expand it on
first use by replacing "bidi" with "bidirectional (bidi)" so the phrase becomes
"Fluent rendering includes bidirectional (bidi) isolation markers..." and keep
the rest of the sentence unchanged.
…ed test artefacts - Expanded artefacts and notes section with explicit unit test and BDD scenario names and statuses. - Included a representative stderr excerpt from non-TTY mode illustrating progress output. - Added command exit code results for formatting, linting, and testing commands. - Minor text correction from 'bidi' to 'bidirectional' for clarity. Co-authored-by: devboxerhub[bot] <devboxerhub[bot]@users.noreply.github.com>
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@docs/execplans/3-9-2-parse-ninja-status-lines-to-drive-task-progress.md`:
- Around line 416-423: The fenced ```text block under the bullet "Non-TTY stderr
excerpt (stdout redirected):" is missing blank lines before and after the fence
which triggers MD031; edit the markdown in
docs/execplans/3-9-2-parse-ninja-status-lines-to-drive-task-progress.md to
insert one blank line immediately after the bullet line and one blank line
immediately after the closing ``` fence, preserving the existing indentation and
the exact fence contents.
- Line 26: Replace American English "localized"/"localization" prose with
British "localised"/"localisation" throughout the document; specifically update
the sentence "Progress control continues through OrthoConfig layering with
localized help" and the other prose occurrences listed (lines referenced in the
review: 47, 133, 147, 150, 191, 263, 350) while leaving code spans and file-path
references (e.g., `src/cli_l10n.rs`, `locales/en-US/messages.ftl`) untouched.
- Around line 56-57: Edit the sentence that begins "Run required quality gates
with logged output:" so the command list reads "make check-fmt, make lint, and
make test" (i.e., insert "and" before the final item `make test`) to match the
Oxford-comma style used elsewhere; update the line containing the three commands
to include the conjunction while preserving the surrounding backticks and
formatting.
…entation Consistently update spelling of 'localized' to 'localised' across the execplans documentation and related localized help text, reflecting preferred UK English spelling. No functional code changes; purely documentation and comment spelling corrections. Co-authored-by: devboxerhub[bot] <devboxerhub[bot]@users.noreply.github.com>
Summary
Changes
Plan of work (high-level)
Stage A: Ninja status parser and process plumbing
Stage B: Reporter integration for Stage 6 task progress
Stage C: Fallback textual updates and OrthoConfig-aligned behaviour
Stage D: Tests with rstest and rstest-bdd v0.5.0
Stage E: Documentation, design records, and gates
Validation and acceptance
Risks and mitigations
Interfaces and dependencies
No new external crates; reuse existing dependencies:
Expected internal interfaces (names may vary):
src/runner/process/ninja_status.rswith a parsed update type andparse_ninja_status_line(&str) -> Option<NinjaTaskProgress>Artifacts and notes
Capture concise evidence for review:
📎 Task: https://www.devboxer.com/task/c6ade565-184d-4bd9-8649-a5f1bcbbc945