diff --git a/docs/adr-008-zero-copy-public-byte-container.md b/docs/adr-008-zero-copy-public-byte-container.md new file mode 100644 index 00000000..65fac5bc --- /dev/null +++ b/docs/adr-008-zero-copy-public-byte-container.md @@ -0,0 +1,205 @@ +# Architectural decision record (ADR) 008: zero-copy public byte container + +## Status + +Proposed + +## Date + +2026-04-12 + +## Context and Problem Statement + +`FrameCodec` already supports `Bytes`-backed frame types, and the default +length-delimited codec uses `Bytes` today. The inventory in +[`frame-vec-u8-inventory.md`](frame-vec-u8-inventory.md), however, shows that +the public packet, middleware, serializer, and client hook surfaces still +assume owned `Vec` payloads. + +Those APIs are not equivalent in how they use bytes: + +- `PacketParts`, `Envelope`, and serializer output are primarily transport and + routing hand-off surfaces. +- `ServiceRequest`, `ServiceResponse`, and `BeforeSendHook` promise editable + bytes and therefore embed a mutation model into the public API. +- The default codec path requires shared, cheap-to-clone byte buffers to + eliminate the final copy identified in epic 284. + +The project needs a single public byte-container strategy that preserves +zero-copy behaviour for read-only paths without forcing every caller to manage +buffer taxonomy manually. + +## Traceability + +This ADR governs the Epic 284 work tracked in: + +- [`frame-vec-u8-inventory.md`](frame-vec-u8-inventory.md), especially the + public payload surfaces and resolved direction in sections "Public + `payload-bound` surfaces" and "Resolved direction for epic 284". +- [`zero-copy-frame-and-payload-migration-roadmap.md`](zero-copy-frame-and-payload-migration-roadmap.md), + which tracks the dedicated zero-copy migration phases that apply this public + byte-container decision across the internal migration, public API flip, and + release workstream. +- [`roadmap.md`](roadmap.md), specifically: + - roadmap item `10.1.1`, which approves the stable public byte-container and + edit-on-demand model; + - roadmap item `11.1.1`, which converts internal packet payload storage and + serializer output to the approved byte representation; + - roadmap items `12.1.1` and `12.1.2`, which migrate packet, envelope, and + middleware surfaces to the approved byte and editing model; + - roadmap items `12.2.1` and `12.2.3`, which migrate client hooks and + serializers and publish downstream migration examples; + - roadmap item `13.1.2`, which writes the migration guide section covering + the zero-copy API flip. + +## Decision Drivers + +- Remove the final owned-byte copy from the default outbound path. +- Keep read-only packet and routing paths zero-copy by default. +- Preserve a clear and ergonomic mutation story for middleware and client + hooks. +- Avoid exposing two equally primary byte-container APIs forever. +- Keep the design compatible with the existing `FrameCodec` and `Bytes` + default. + +## Requirements + +### Functional requirements + +- Public packet and routing surfaces must support cheap cloning and transport + without copying when the underlying bytes are already shared. +- Middleware and client hooks must still support intentional mutation of + serialized bytes. +- The selected API must support migration examples that downstream users can + apply without learning multiple internal buffer types. + +### Technical requirements + +- The default codec path must be able to move from serialization to + `wrap_payload` without materializing a fresh `Vec`. +- The design must not require `F::Frame = Vec` or any equivalent + frame-level coupling. +- Mutation must be explicit, and it must not trigger copies on read-only + paths. + +## Options Considered + +### Option A: switch every public byte surface directly to `Bytes` + +This would maximize zero-copy reuse for read-only paths, but it would also +replace the current `frame_mut()` and `Fn(&mut Vec)` contracts with a more +awkward mutation story. Middleware authors would need to choose when to clone, +freeze, or reallocate, and the API would no longer describe the intended +editing workflow clearly. + +### Option B: use `Bytes` for stable storage and expose explicit edit-on-demand wrappers (preferred) + +Under this option, public packet and routing surfaces store `Bytes` (or a +single project-defined wrapper over `Bytes`) as the stable representation. +Editable surfaces expose mutation through an explicit helper or editor that +performs copy-on-write only when a caller actually mutates the bytes. + +This preserves zero-copy behaviour for pass-through traffic while keeping the +current "inspect, optionally edit, then forward" workflow legible. + +### Option C: keep `Vec` as the stable public representation + +This preserves the existing mutation model, but it also preserves the final +copy that epic 284 is trying to remove. Any internal `Bytes` use would still +collapse back to owned vectors at the public boundary. + +### Option D: make the public API permanently generic or dual-surfaced over `Vec` and `Bytes` + +This avoids choosing one primary abstraction, but it increases API surface +area, complicates documentation, and pushes conversion logic onto downstream +users. The inventory explicitly calls out the risk of forcing consumers to +write repetitive adapters between byte containers. + +| Topic | Option A: `Bytes` only | Option B: `Bytes` + explicit editor | Option C: keep `Vec` | Option D: dual support | +| ------------------------- | ---------------------- | ----------------------------------- | ------------------------ | ---------------------- | +| Zero-copy read path | Excellent | Excellent | Poor | Good | +| Editing ergonomics | Weak | Strong | Strong | Medium | +| Public API complexity | Medium | Medium | Low | High | +| Long-term maintainability | Good | Good | Poor | Poor | +| Migration burden | Medium | Medium | Low short-term | High | + +_Table 1: Trade-offs for the public byte-container choice._ + +## Decision Outcome / Proposed Direction + +Adopt Option B: use `Bytes`-compatible storage as the stable public payload +representation, and expose mutation through an explicit edit-on-demand helper +for middleware and client hooks. + +The proposed direction is: + +- Packet and routing surfaces (`PacketParts`, `Envelope`, serializer output, + and equivalent internal hand-offs) standardize on shared bytes. +- Middleware and hook surfaces expose an explicit editing entry point that + copies only when the caller mutates the payload. +- `Vec` remains available only through explicit compatibility helpers, + adapters, or migration constructors defined by the rollout ADR. + +This keeps the zero-copy path obvious and makes the mutating path explicit. + +## Goals and Non-Goals + +### Goals + +- Remove the final default-path copy without sacrificing middleware + ergonomics. +- Make the read-only versus mutating cost model visible in the API. +- Give downstream users a single primary byte model to target. + +### Non-Goals + +- Guarantee zero allocation for every mutation path. +- Eliminate every internal `Vec` immediately, including bounded one-shot + replay buffers that remain intentionally separate. +- Rework unrelated message-body APIs that are outside the frame and payload + migration scope. + +## Migration Plan + +### Phase 1: establish the stable byte representation + +Convert packet payload storage and serializer output to the shared byte +representation, and add explicit conversions for compatibility callers. + +### Phase 2: introduce edit-on-demand wrappers + +Replace `Vec`-backed middleware and hook edit points with explicit editing +helpers that preserve the current workflow while copying only when modified. + +### Phase 3: update docs and examples + +Update the user guide, middleware examples, and migration guide to teach the +new read-only and mutating workflows distinctly. + +## Known Risks and Limitations + +- Copy-on-write editing helpers still need a concrete public shape, and poor + naming could hide when a clone occurs. +- Some downstream users may rely on direct `Vec` methods such as `push` or + `extend_from_slice`; the migration guide must map those patterns to the new + editor API explicitly. +- If the project exposes both raw `Bytes` and a custom editor type too widely, + the API could still feel like a dual-surface design in practice. + +## Outstanding Decisions + +- Should the public editing surface expose `BytesMut`, a project-defined editor + wrapper, or closure-based mutation helpers? +- Should serializer output move to the new byte representation in the same + release as packet and middleware changes, or in a preparatory release + beforehand? +- Which compatibility constructors or `into_vec` helpers are required for the + migration window? + +## Architectural Rationale + +The wider architecture already treats frame transport and payload ownership as +separate concerns. Standardizing stable payload storage on a `Bytes`-compatible +representation aligns the public API with the default codec path, while +explicit editing helpers preserve the middleware-first ergonomics that made +`Vec` attractive originally. diff --git a/docs/adr-009-vec-u8-migration-rollout.md b/docs/adr-009-vec-u8-migration-rollout.md new file mode 100644 index 00000000..707d5c2c --- /dev/null +++ b/docs/adr-009-vec-u8-migration-rollout.md @@ -0,0 +1,168 @@ +# Architectural decision record (ADR) 009: rollout strategy for the `Vec` to zero-copy migration + +## Status + +Proposed + +## Date + +2026-04-12 + +## Context and Problem Statement + +Epic 284 is a breaking change. The inventory in +[`frame-vec-u8-inventory.md`](frame-vec-u8-inventory.md) shows that downstream +users are exposed to `Vec` not only through one public `Frame` alias, but +through packet construction, middleware wrappers, client hooks, serializer +output, tests, and documentation. A zero-copy internal implementation is not +enough; the project also needs a release and compatibility strategy that keeps +the change reviewable and predictable for downstream users. + +The key rollout question is whether the project should: + +- switch to the new byte model in one hard break, +- support both `Vec` and the new representation indefinitely, or +- ship a staged major release with finite compatibility helpers and explicit + migration guidance. + +## Traceability + +This ADR governs the Epic 284 compatibility and rollout work tracked in: + +- [`frame-vec-u8-inventory.md`](frame-vec-u8-inventory.md), especially the + "Generalization paths and conceptual risks", "Resolved direction for epic + 284", and "Coordination notes" sections. +- [`zero-copy-frame-and-payload-migration-roadmap.md`](zero-copy-frame-and-payload-migration-roadmap.md), + which tracks the dedicated migration publish and breaking-release workstream + that this rollout policy governs, including the Phase 1 decision closure and + the later publication milestones. +- [`roadmap.md`](roadmap.md), specifically: + - roadmap item `10.1.2`, which approves the compatibility and rollout + policy; + - roadmap item `10.2.3`, which publishes the migration-guide outline and + exact public surfaces affected by the change; + - roadmap item `12.1.2`, which preserves explicit edit-on-demand ergonomics + as part of the public migration story; + - roadmap item `12.2.2`, which documents how client preamble leftovers fit + into the compatibility plan; + - roadmap items `13.1.1` and `13.1.2`, which remove obsolete `Vec` + compatibility surfaces and publish the migration guide section; + - roadmap items `14.1.1`, `14.1.2`, `14.1.3`, and `14.2.1`, which prepare + and review the breaking release and any retained helpers. + +## Decision Drivers + +- Minimize downstream boilerplate during the migration. +- Keep the long-term public API coherent rather than permanently dual-shaped. +- Make the breaking change easy to communicate in release notes and examples. +- Avoid indefinite maintenance of compatibility shims that preserve the old + bottlenecks. +- Leave room for downstream users to migrate in bounded, observable steps. + +## Options Considered + +### Option A: one-shot hard break with no compatibility helpers + +Ship the new zero-copy API and require all downstream users to update in one +step. This keeps the final API clean, but it also maximizes upgrade pain and +forces every consumer to solve migration details independently. + +### Option B: permanent dual support for `Vec` and the zero-copy type + +Expose both old and new constructors, accessors, and hook signatures as first- +class supported APIs. This reduces short-term friction, but it risks turning a +breaking release into a permanent maintenance burden and keeps the old +allocation-heavy path alive indefinitely. + +### Option C: staged breaking release with finite compatibility helpers (preferred) + +Adopt the zero-copy API as the long-term default, but ship the breaking release +with bounded helper conversions, migration examples, and clear removal criteria +for any retained `Vec` adapters. + +| Topic | Option A: hard break | Option B: permanent dual support | Option C: staged release | +| ------------------------- | -------------------- | -------------------------------- | ------------------------ | +| Long-term API coherence | Strong | Weak | Strong | +| Short-term upgrade pain | High | Low | Medium | +| Maintenance burden | Low | High | Medium | +| Migration guidance needed | High | High | High | +| Risk of preserving copies | Low | High | Low | + +_Table 1: Trade-offs for the migration rollout policy._ + +## Decision Outcome / Proposed Direction + +Adopt Option C: ship the zero-copy API as the new primary surface in the next +breaking release, and include finite compatibility helpers plus a committed +migration guide. + +The proposed rollout policy is: + +- The new byte model becomes the canonical API in the breaking release. +- Compatibility helpers remain only where they clearly reduce upgrade cost and + do not reintroduce the old bottlenecks as first-class behaviour. +- Release notes, examples, and migration documentation explicitly show how to + move middleware, hooks, serializer code, and custom codecs off `Vec`. +- Any retained compatibility surface has a documented review point for later + removal or retention. + +## Goals and Non-Goals + +### Goals + +- Make the breaking release adoptable without forcing users to reverse-engineer + internal design intent. +- Prevent the project from carrying two equally primary byte APIs + indefinitely. +- Bound the lifetime of compatibility helpers. + +### Non-Goals + +- Preserve full source compatibility. +- Hide the fact that this is a semver-significant release. +- Guarantee that every downstream crate can upgrade without any code changes. + +## Migration Plan + +### Phase 1: announce the target API and migration path + +Publish the roadmap, proposed ADRs, and a migration-guide outline before the +public API flip lands. + +### Phase 2: land the breaking release with bounded helpers + +Ship the new canonical API together with explicitly documented adapters such as +`from_vec`, `into_vec`, or equivalent helper wrappers where they materially +reduce migration churn. + +### Phase 3: evaluate retained helpers after the release + +Review which helpers are still needed after the initial migration cycle, and +either accept them as narrow compatibility tools or schedule their removal in a +follow-up release plan. + +## Known Risks and Limitations + +- Compatibility helpers can easily become permanent in practice if their removal + criteria are not written down. +- A breaking release without concrete before-and-after examples will still feel + abrupt even if helper APIs exist. +- If too many helpers survive unchanged, downstream code may continue to depend + on the `Vec` mental model, weakening the benefits of the migration. + +## Outstanding Decisions + +- Which compatibility helpers are important enough to ship in the breaking + release? +- Should any helpers be feature-gated, or should they ship in the default + build for one release cycle? +- What benchmark or adoption signals justify removing retained helpers later? + +## Architectural Rationale + +The zero-copy migration is not only a transport optimization. It changes how +middleware, hooks, serializers, and packet routing represent bytes. A staged +breaking release acknowledges that architectural reality: the new API becomes +the primary one immediately, but the release still carries enough local +documentation and narrowly scoped adapters to keep downstream adoption +manageable. diff --git a/docs/adr-010-transport-frame-boundary-for-zero-copy.md b/docs/adr-010-transport-frame-boundary-for-zero-copy.md new file mode 100644 index 00000000..83e9caf1 --- /dev/null +++ b/docs/adr-010-transport-frame-boundary-for-zero-copy.md @@ -0,0 +1,180 @@ +# Architectural decision record (ADR) 010: transport-frame boundary for the zero-copy migration + +## Status + +Proposed + +## Date + +2026-04-12 + +## Context and Problem Statement + +The inventory in [`frame-vec-u8-inventory.md`](frame-vec-u8-inventory.md) shows +that the remaining `Vec` coupling is not only about public payload APIs. +The transport pipeline also has an architectural boundary problem: + +- `ConnectionActor` operates on types satisfying + `FrameLike + CorrelatableFrame + Packet`. +- The default codec frame type is `Bytes`, which does not satisfy those actor + traits. +- The current server path therefore routes actor output through `Envelope` and + lets the codec driver perform the final `serialize -> wrap_payload` step. +- Protocol hook alignment across the actor and codec boundary is still a + follow-up concern after roadmap item 9.3.1. + +The zero-copy migration needs an explicit long-term boundary between actor +logic, protocol hooks, serializer output, and transport frame emission so the +project does not keep `Vec` bridges alive only because the architecture is +underspecified. + +## Traceability + +This ADR governs the Epic 284 runtime-boundary work tracked in: + +- [`frame-vec-u8-inventory.md`](frame-vec-u8-inventory.md), especially the + "`internal-only` runtime surfaces", "Adjacent constraints that matter but do + not name `Vec`", and "Resolved direction for epic 284" sections. +- [`zero-copy-frame-and-payload-migration-roadmap.md`](zero-copy-frame-and-payload-migration-roadmap.md), + which tracks the dedicated rollout workstream that applies this boundary + decision and reviews the migration until the codec driver becomes the sole + owner of transport frame emission. During implementation review, monitor all + non-driver `FrameCodec::wrap_payload` call sites against this ADR until they + are removed or relocated into the codec-driver boundary. +- [`roadmap.md`](roadmap.md), specifically: + - roadmap item `10.1.3`, which approves the actor and codec-driver boundary; + - roadmap item `11.1.2`, which removes the final default-path `Vec` copy + between serialization and `FrameCodec::wrap_payload`; + - roadmap items `11.2.1` and `11.2.2`, which implement the boundary and + move `Vec`-specific runtime traits or bridges out of the core runtime; + - roadmap item `11.2.3`, which adds allocation and pointer-reuse regressions + for the internal actor and codec path; + - roadmap item `14.2.1`, which reviews whether any runtime-only compatibility + bridges still need to survive after the breaking release. + +## Decision Drivers + +- Preserve zero-copy transport framing for the default codec path. +- Avoid expanding `ConnectionActor` bounds just to make codec frame types look + like packets. +- Keep fragmentation, correlation handling, and protocol hooks explicit. +- Reuse the existing `FramePipeline` direction from roadmap item 9.3.1 where + it remains sound. +- Minimize public API churn outside the migration's intended scope. + +## Options Considered + +### Option A: make `ConnectionActor` operate on codec frame types directly + +This would require codec frame types such as `Bytes` to satisfy packet- and +correlation-oriented traits, or it would require a new actor-specific wrapper +for every codec frame. That increases coupling between transport framing and +application packet semantics. + +### Option B: keep the actor envelope-oriented and let the codec driver own transport frames (preferred) + +Under this option, the actor continues to work with `Envelope` or another +packet-shaped type, while the codec driver owns serialization, payload +wrapping, and final transport frame emission. Protocol hooks must be defined +explicitly at whichever boundary they truly need. + +### Option C: add a new bridging abstraction that makes codec frames actor-compatible + +This could formalize the boundary, but it also risks creating a second layer of +wrapper types whose only job is to satisfy the actor's existing bounds. That +adds complexity without necessarily removing copies. + +| Topic | Option A: actor on codec frames | Option B: actor on envelopes | Option C: bridge layer | +| --------------------------- | ------------------------------- | ---------------------------- | ---------------------- | +| Zero-copy default path | Good | Good | Medium | +| Actor/transport separation | Weak | Strong | Medium | +| Required trait expansion | High | Low | Medium | +| Implementation complexity | High | Medium | High | +| Fit with roadmap item 9.3.1 | Weak | Strong | Medium | + +_Table 1: Trade-offs for the actor and transport-frame boundary._ + +## Decision Outcome / Proposed Direction + +Adopt Option B: keep the actor packet-oriented, and make the codec driver the +only component that owns transport frame emission. + +The proposed direction is: + +- `ConnectionActor` continues to reason about packet-shaped values such as + `Envelope`. +- The serializer and codec driver own the final + `packet -> bytes -> transport frame` transition. +- Any protocol hook that requires codec-frame visibility is attached at the + codec-driver boundary, not by forcing codec frames through actor traits they + do not naturally satisfy. +- `Vec` frame bridges are removed from the core runtime once the chosen + codec-driver path is proven. + +## Goals and Non-Goals + +### Goals + +- Keep transport framing and packet semantics decoupled. +- Remove accidental `Vec` dependencies at the actor boundary. +- Give the zero-copy migration a stable place to enforce pointer reuse and + allocation tests. + +### Non-Goals + +- Redesign the full protocol hook API in this ADR. +- Force every custom codec frame to implement packet semantics. +- Remove all test-only `Vec` helpers immediately if they still provide + migration value. + +## Migration Plan + +### Phase 1: formalize the packet-to-frame boundary + +Document which component owns serialization, protocol hook invocation, and +`wrap_payload`, and update the runtime so that ownership is reflected in code. +Consolidate any non-driver `FrameCodec::wrap_payload` callers into the +codec-driver boundary, including the current runtime inbound emission path, so +`ConnectionActor` no longer participates in transport frame emission outside +the codec driver. + +### Phase 2: remove obsolete core bridges + +Delete or move `Vec`-specific core frame bridges once the actor no longer +needs them for production behaviour. + +### Phase 3: validate the boundary + +Add regression coverage showing that the default codec path stays zero-copy and +that correlation, fragmentation, and protocol hooks still run at the intended +stage. + +## Known Risks and Limitations + +- The protocol hook story still needs a precise statement of which hooks run on + packets and which run on transport frames. +- If the codec driver becomes the only framing boundary, its tests must carry + more of the performance and correctness burden than they do today. +- Some example or test harness code may continue to use `Vec` as a + convenience type even after the production boundary is cleaned up. +- Until every non-driver `FrameCodec::wrap_payload` caller is relocated, the + project needs an explicit tracked list of remaining call sites, including the + inbound emission path adjacent to `ConnectionActor`, so reviews can verify + that the codec driver is becoming the sole owner of transport frame emission. + +## Outstanding Decisions + +- Which existing protocol hooks should move to the codec-driver boundary, and + which should remain packet-oriented? +- Should any remaining `Vec` bridge live in `test_support`, a feature-gated + compatibility module, or nowhere at all? +- Does the project need a new internal trait to express "serializable packet" + separately from transport frame semantics? + +## Architectural Rationale + +Roadmap item 9.3.1 already established that treating codec frames and actor +packets as the same abstraction leads to awkward constraints, especially when +the default codec frame is `Bytes`. Making the codec driver the explicit +transport-frame boundary preserves that lesson, supports the zero-copy goal, +and avoids rebuilding actor semantics around transport-specific frame types. diff --git a/docs/contents.md b/docs/contents.md index 161ef214..1efafd9d 100644 --- a/docs/contents.md +++ b/docs/contents.md @@ -29,6 +29,9 @@ reference when navigating the project's design and architecture material. - [Wireframe 1.0 roadmap](wireframe-1-0-detailed-development-roadmap.md) Detailed tasks leading to Wireframe 1.0. - [Project roadmap](roadmap.md) High-level development roadmap. +- [Zero-copy frame and payload migration roadmap](zero-copy-frame-and-payload-migration-roadmap.md) + Phased plan for migrating from `Vec`-owned frame APIs to a zero-copy + alternative. - [1.0 philosophy][philosophy] Philosophy and feature set for Wireframe 1.0. diff --git a/docs/developers-guide.md b/docs/developers-guide.md index 837c1ec6..ab243ed7 100644 --- a/docs/developers-guide.md +++ b/docs/developers-guide.md @@ -43,6 +43,23 @@ Use this checklist before merging API naming changes: - Label cross-layer terms (for example, `correlation_id`) explicitly as shared metadata. +## Message sequence validation architecture + +Message continuation ordering lives in `src/message_assembler/series.rs`. The +public-facing control point is `validate_and_advance_sequence()`, which keeps +the series state machine readable by delegating to two private helpers: + +- `handle_untracked_first_sequence()` handles the first numbered continuation + after an untracked start. It switches the series into tracked mode and + derives the next expected sequence. +- `advance_tracked_sequence()` handles all later numbered continuations once + tracking is active. It rejects duplicates, gaps, and sequence overflow before + advancing the expected sequence. + +This split is intentional. The helpers isolate the untracked-to-tracked +transition from steady-state tracked validation so sequence policy stays flat, +testable, and documentation-friendly during future refactors. + ## Vocabulary normalization outcome (2026-02-20) The 2026-02-20 normalization pass aligned docs and rustdoc terminology to this diff --git a/docs/execplans/10-2-1-client-message-api.md b/docs/execplans/16-2-1-client-message-api.md similarity index 97% rename from docs/execplans/10-2-1-client-message-api.md rename to docs/execplans/16-2-1-client-message-api.md index 6f3bc08c..1290fbf5 100644 --- a/docs/execplans/10-2-1-client-message-api.md +++ b/docs/execplans/16-2-1-client-message-api.md @@ -23,11 +23,11 @@ Success is observable when: - Unit tests cover correlation ID generation, stamping, and mismatch detection. - Cucumber behavioural tests validate the messaging flow. - `docs/users-guide.md` documents the new client APIs. -- `docs/roadmap.md` marks 10.2.1 as done. +- `docs/roadmap.md` marks 16.2.1 as done. ## Progress -- [x] (2026-01-09) Draft ExecPlan for 10.2.1. +- [x] (2026-01-09) Draft ExecPlan for 16.2.1. - [x] (2026-01-09) Add `correlation_counter: AtomicU64` field to `WireframeClient`. - [x] (2026-01-09) Add `CorrelationMismatch` error variant to `ClientError`. @@ -39,7 +39,7 @@ Success is observable when: - [x] (2026-01-09) Add Cucumber feature and steps for client messaging. - [x] (2026-01-09) Update `docs/users-guide.md` with client messaging API documentation. -- [x] (2026-01-09) Mark roadmap 10.2.1 as done. +- [x] (2026-01-09) Mark roadmap 16.2.1 as done. - [ ] Run full validation (fmt, lint, test). ## Surprises & Discoveries @@ -176,7 +176,7 @@ Finally, update documentation and mark the roadmap item as done. - Document `send_envelope`, `receive_envelope`, `call_correlated` methods - Provide usage examples -11. Update `docs/roadmap.md` to mark 10.2.1 as done. +11. Update `docs/roadmap.md` to mark 16.2.1 as done. ## Validation and Acceptance @@ -186,7 +186,7 @@ Acceptance requires all of the following: - Unit tests cover correlation ID generation, stamping, and validation. - Behavioural tests for client messaging pass. - Documentation updated in users guide. -- Roadmap item 10.2.1 marked as done. +- Roadmap item 16.2.1 marked as done. Run validation from the repository root (use `tee` to capture full output): diff --git a/docs/execplans/10-2-2-client-decode-and-transport-failures.md b/docs/execplans/16-2-2-client-decode-and-transport-failures.md similarity index 95% rename from docs/execplans/10-2-2-client-decode-and-transport-failures.md rename to docs/execplans/16-2-2-client-decode-and-transport-failures.md index e8bad4cc..9997174f 100644 --- a/docs/execplans/10-2-2-client-decode-and-transport-failures.md +++ b/docs/execplans/16-2-2-client-decode-and-transport-failures.md @@ -10,7 +10,7 @@ No `PLANS.md` file exists in the repository root at the time of writing. ## Purpose / big picture -Phase `10.2.2` closes a contract gap in the client request/response pipeline. +Phase `16.2.2` closes a contract gap in the client request/response pipeline. Today, client decode and transport failures are surfaced as direct `ClientError` variants. This work makes those failures flow through `WireframeError` variants, so client behaviour matches the server-side error @@ -25,7 +25,7 @@ Success is observable when: - Unit tests (`rstest`) and behavioural tests (`rstest-bdd` v0.5.0) pass. - `docs/wireframe-client-design.md` records the error-mapping decision. - `docs/users-guide.md` reflects public client error-handling changes. -- `docs/roadmap.md` marks `10.2.2` as done. +- `docs/roadmap.md` marks `16.2.2` as done. ## Constraints @@ -73,7 +73,7 @@ Success is observable when: ## Progress -- [x] (2026-02-12 08:59Z) Draft ExecPlan for roadmap item `10.2.2`. +- [x] (2026-02-12 08:59Z) Draft ExecPlan for roadmap item `16.2.2`. - [x] (2026-02-12 09:13Z) Confirm final client error surface and mapping strategy. - [x] (2026-02-12 09:15Z) Implement `WireframeError` mapping in client @@ -88,7 +88,7 @@ Success is observable when: decisions. - [x] (2026-02-12 09:25Z) Update `docs/users-guide.md` with public interface guidance. -- [x] (2026-02-12 09:25Z) Mark roadmap entry `10.2.2` done in +- [x] (2026-02-12 09:25Z) Mark roadmap entry `16.2.2` done in `docs/roadmap.md`. - [x] (2026-02-12 09:31Z) Run formatting, lint, and full test gates. @@ -102,7 +102,7 @@ Success is observable when: - Observation: Behavioural tests already use `rstest-bdd`, but repository pins `0.4.0`. Evidence: `Cargo.toml` dev-dependencies list `rstest-bdd = "0.4.0"` - and `rstest-bdd-macros = "0.4.0"`. Impact: `10.2.2` includes a required + and `rstest-bdd-macros = "0.4.0"`. Impact: `16.2.2` includes a required dependency/version alignment task. - Observation: `docs/behavioural-testing-in-rust-with-cucumber.md` is @@ -120,7 +120,7 @@ Success is observable when: - Decision: Model decode and transport failures through `WireframeError` variants and expose that mapping from client runtime errors. Rationale: - aligns client and server failure semantics and matches roadmap item `10.2.2` + aligns client and server failure semantics and matches roadmap item `16.2.2` intent. Date/Author: 2026-02-12 / Codex. - Decision: Keep preamble/serialization/correlation mismatch failures as @@ -136,7 +136,7 @@ Success is observable when: ## Outcomes & retrospective -Implemented `10.2.2` end-to-end. +Implemented `16.2.2` end-to-end. - Request/response decode and transport failures now map through `ClientError::Wireframe(...)` backed by `WireframeError` variants. @@ -146,7 +146,7 @@ Implemented `10.2.2` end-to-end. - Behavioural tests were upgraded to `rstest-bdd` `0.5.0` and extended with a decode-mapping scenario. - Client design and user documentation were updated with migration guidance. -- `docs/roadmap.md` marks `10.2.2` complete. +- `docs/roadmap.md` marks `16.2.2` complete. ## Context and orientation @@ -179,7 +179,7 @@ Documentation touchpoints for this item: - `docs/wireframe-client-design.md` for design rationale. - `docs/users-guide.md` for public API/error-handling guidance. -- `docs/roadmap.md` to mark completion of `10.2.2`. +- `docs/roadmap.md` to mark completion of `16.2.2`. ## Plan of work @@ -213,7 +213,7 @@ Go/no-go check: integration tests and behavioural tests pass locally. ### Stage D: Documentation + roadmap closure Update design and user documentation to reflect the new error surface. Mark -`10.2.2` done in `docs/roadmap.md` only after all tests and quality gates pass. +`16.2.2` done in `docs/roadmap.md` only after all tests and quality gates pass. Go/no-go check: documentation lint/format gates pass; roadmap status changed in same change-set. @@ -250,7 +250,7 @@ same change-set. - migration guidance for callers matching previous `ClientError` variants 8. Mark roadmap item done in `docs/roadmap.md`: - - change `10.2.2` from `[ ]` to `[x]` after successful validation. + - change `16.2.2` from `[ ]` to `[x]` after successful validation. 9. Run quality gates from repository root with logs captured: @@ -285,7 +285,7 @@ Acceptance criteria: client outcomes. - `docs/wireframe-client-design.md` and `docs/users-guide.md` reflect the new behaviour. -- `docs/roadmap.md` marks `10.2.2` as done. +- `docs/roadmap.md` marks `16.2.2` as done. - `make fmt`, `make markdownlint`, `make check-fmt`, `make lint`, and `make test` all succeed. diff --git a/docs/execplans/10-3-1-client-response-stream-and-response-multipacket.md b/docs/execplans/16-3-1-client-response-stream-and-response-multipacket.md similarity index 99% rename from docs/execplans/10-3-1-client-response-stream-and-response-multipacket.md rename to docs/execplans/16-3-1-client-response-stream-and-response-multipacket.md index ab1a5444..53141d22 100644 --- a/docs/execplans/10-3-1-client-response-stream-and-response-multipacket.md +++ b/docs/execplans/16-3-1-client-response-stream-and-response-multipacket.md @@ -403,7 +403,7 @@ Update `tests/fixtures/mod.rs` to add `pub mod client_streaming;`. Update - Example code showing the streaming API end-to-end. 3. Update `docs/roadmap.md`: - mark item 10.3.1 as done. + mark item 16.3.1 as done. ### Stage H: Validation @@ -492,7 +492,7 @@ Quality criteria: Quality method: - Run `make check-fmt && make lint && make test` and verify exit 0. -- Manually verify `docs/roadmap.md` shows 10.3.1 as done. +- Manually verify `docs/roadmap.md` shows 16.3.1 as done. - Verify `docs/users-guide.md` contains streaming client documentation. ## Idempotence and recovery diff --git a/docs/execplans/10-3-2-exercise-interleaved-queues.md b/docs/execplans/16-3-2-exercise-interleaved-queues.md similarity index 95% rename from docs/execplans/10-3-2-exercise-interleaved-queues.md rename to docs/execplans/16-3-2-exercise-interleaved-queues.md index 006d5b7e..f6955b04 100644 --- a/docs/execplans/10-3-2-exercise-interleaved-queues.md +++ b/docs/execplans/16-3-2-exercise-interleaved-queues.md @@ -37,7 +37,7 @@ both queues and proves three properties: Observable outcome: running `make test` passes, including new unit tests in `tests/interleaved_push_queues.rs` and new Behaviour-Driven Development (BDD) scenarios in `tests/features/interleaved_push_queues.feature`. The existing -test suite remains green. The roadmap entry 10.3.2 is marked as done. +test suite remains green. The roadmap entry 16.3.2 is marked as done. ## Constraints @@ -89,7 +89,7 @@ test suite remains green. The roadmap entry 10.3.2 is marked as done. - [x] (2026-02-19) Gathered context from roadmap, client/runtime code, existing fairness/rate tests, and referenced design/testing documents. -- [x] (2026-02-19) Drafted ExecPlan for roadmap item `10.3.2`. +- [x] (2026-02-19) Drafted ExecPlan for roadmap item `16.3.2`. - [x] (2026-02-19) Defined parity assertions and mapped them to unit/behavioural tests. - [x] (2026-02-19) Added `rstest` unit tests for interleaved fairness and @@ -99,7 +99,7 @@ test suite remains green. The roadmap entry 10.3.2 is marked as done. - [x] (2026-02-19) Applied minimal fixture/runtime test-harness updates; no production API changes were required. - [x] (2026-02-19) Updated design and user docs with final parity notes. -- [x] (2026-02-19) Marked roadmap item `10.3.2` done. +- [x] (2026-02-19) Marked roadmap item `16.3.2` done. - [x] (2026-02-19) Ran full quality and documentation gates with captured logs. ## Surprises & discoveries @@ -136,7 +136,7 @@ test suite remains green. The roadmap entry 10.3.2 is marked as done. - Decision: Use `ConnectionActor::run()` as the integration boundary for fairness tests rather than testing `FairnessTracker` in isolation. Rationale: the tracker is already unit-tested in `src/fairness.rs::tests`; the value of - 10.3.2 is proving that the tracker integrates correctly with the actor's + 16.3.2 is proving that the tracker integrates correctly with the actor's `select!` loop and drain logic. Date: 2026-02-21. - Decision: use virtual time in timing-sensitive unit tests and outcome-based @@ -151,7 +151,7 @@ test suite remains green. The roadmap entry 10.3.2 is marked as done. ## Outcomes & retrospective -Implemented roadmap item `10.3.2` end-to-end. +Implemented roadmap item `16.3.2` end-to-end. ### Deliverables @@ -215,7 +215,7 @@ The wireframe crate lives at the repository root. Key paths for this task: wireframe_testing/src/lib.rs — TestResult, push_expect!, recv_expect! - docs/roadmap.md — Roadmap (10.3.2 to mark done) + docs/roadmap.md — Roadmap (16.3.2 to mark done) docs/users-guide.md — User's guide docs/multi-packet-and-streaming-responses-design.md — Design document @@ -346,7 +346,7 @@ decisions. C2. Update `docs/users-guide.md` — add a note to the "Push queues and connection actors" section about interleaved push queue validation. -C3. Update `docs/roadmap.md` — change the 10.3.2 line from `- [ ]` to `- [x]`. +C3. Update `docs/roadmap.md` — change the 16.3.2 line from `- [ ]` to `- [x]`. ### Stage D: Validation @@ -392,25 +392,25 @@ Stage D: ```shell set -o pipefail - make fmt 2>&1 | tee /tmp/wireframe-10-3-2-fmt.log + make fmt 2>&1 | tee /tmp/wireframe-16-3-2-fmt.log set -o pipefail - make check-fmt 2>&1 | tee /tmp/wireframe-10-3-2-check-fmt.log + make check-fmt 2>&1 | tee /tmp/wireframe-16-3-2-check-fmt.log set -o pipefail - make lint 2>&1 | tee /tmp/wireframe-10-3-2-lint.log + make lint 2>&1 | tee /tmp/wireframe-16-3-2-lint.log set -o pipefail - make test-bdd 2>&1 | tee /tmp/wireframe-10-3-2-test-bdd.log + make test-bdd 2>&1 | tee /tmp/wireframe-16-3-2-test-bdd.log set -o pipefail - make test 2>&1 | tee /tmp/wireframe-10-3-2-test.log + make test 2>&1 | tee /tmp/wireframe-16-3-2-test.log set -o pipefail - make markdownlint 2>&1 | tee /tmp/wireframe-10-3-2-markdownlint.log + make markdownlint 2>&1 | tee /tmp/wireframe-16-3-2-markdownlint.log set -o pipefail - make nixie 2>&1 | tee /tmp/wireframe-10-3-2-nixie.log + make nixie 2>&1 | tee /tmp/wireframe-16-3-2-nixie.log ``` @@ -432,7 +432,7 @@ Quality method: - Run `make check-fmt && make lint && make test && make markdownlint` and verify exit 0. -- Manually verify `docs/roadmap.md` shows 10.3.2 as done. +- Manually verify `docs/roadmap.md` shows 16.3.2 as done. - Verify `docs/users-guide.md` mentions interleaved push queue validation. ## Idempotence and recovery diff --git a/docs/execplans/10-4-1-client-documentation-and-examples.md b/docs/execplans/16-4-1-client-documentation-and-examples.md similarity index 88% rename from docs/execplans/10-4-1-client-documentation-and-examples.md rename to docs/execplans/16-4-1-client-documentation-and-examples.md index 225159dd..8c6d16f6 100644 --- a/docs/execplans/10-4-1-client-documentation-and-examples.md +++ b/docs/execplans/16-4-1-client-documentation-and-examples.md @@ -12,7 +12,7 @@ requirements. ## Purpose / big picture -Roadmap item 10.4 requires two concrete outcomes: +Roadmap item 16.4 requires two concrete outcomes: - A runnable client example that connects to the `echo` server, sends a login request, and decodes the echoed acknowledgement. @@ -33,7 +33,7 @@ Observable success is: - `docs/users-guide.md` and `docs/wireframe-client-design.md` contain configuration tables, lifecycle diagrams, troubleshooting guidance, and the design decision record for the example contract. -- `docs/roadmap.md` marks 10.4.1 and 10.4.2 done. +- `docs/roadmap.md` marks 16.4.1 and 16.4.2 done. ## Constraints @@ -52,14 +52,14 @@ Observable success is: the feature. - Record implementation decisions in the relevant design document (`docs/wireframe-client-design.md`). -- On feature completion, mark roadmap entries 10.4.1 and 10.4.2 done in +- On feature completion, mark roadmap entries 16.4.1 and 16.4.2 done in `docs/roadmap.md`. ## Tolerances (exception triggers) - Scope: if implementation requires more than 14 files or 500 net lines, pause and escalate. -- Interface: if satisfying 10.4 requires changing existing public API +- Interface: if satisfying 16.4 requires changing existing public API signatures in `src/client/`, pause and escalate. - Dependencies: if a new crate is required, pause and escalate. - Behavioural ambiguity: if "acknowledgement" cannot be represented clearly @@ -92,7 +92,7 @@ Observable success is: ## Progress -- [x] (2026-02-23 00:00Z) Drafted ExecPlan for roadmap items 10.4.1 and 10.4.2. +- [x] (2026-02-23 00:00Z) Drafted ExecPlan for roadmap items 16.4.1 and 16.4.2. - [x] (2026-02-23 00:18Z) Stage A complete: confirmed the acknowledgement contract as an echoed login payload decode and scoped file targets. - [x] (2026-02-23 00:30Z) Stage B complete: added @@ -104,7 +104,7 @@ Observable success is: - [x] (2026-02-23 01:10Z) Stage E complete: expanded `docs/users-guide.md` and `docs/wireframe-client-design.md` with configuration tables, lifecycle diagrams, troubleshooting, and the - 10.4.1 decision record. + 16.4.1 decision record. - [x] (2026-02-23 01:55Z), Stage F complete: updated roadmap checkboxes and ran all required quality/documentation gates successfully. @@ -112,7 +112,7 @@ Observable success is: - Discovery: strict `-D warnings` on `make test-bdd` surfaced existing `unused_must_use` test helper paths in interleaved push queue fixtures that - were unrelated to 10.4. Fixing them was required for green behavioural + were unrelated to 16.4. Fixing them was required for green behavioural validation. - Discovery: `make test-doc` initially failed due to stale doctest imports in @@ -121,10 +121,10 @@ Observable success is: ## Decision Log -- Decision: implement 10.4 behavioural coverage by extending the existing +- Decision: implement 16.4 behavioural coverage by extending the existing client-runtime BDD suite instead of creating a separate BDD domain. Rationale: this keeps setup reuse high, avoids duplicate worlds, and keeps - roadmap 10.x client validation centralized. Date/Author: 2026-02-23 / Codex. + roadmap 16.x client validation centralized. Date/Author: 2026-02-23 / Codex. - Decision: define login acknowledgement in the runnable example as the echoed login message decoded by the client. Rationale: this honours the explicit @@ -154,7 +154,7 @@ Shipped functionality: - Expanded client docs and design guidance in `docs/users-guide.md` and `docs/wireframe-client-design.md`, including configuration tables, lifecycle diagrams, troubleshooting, runnable commands, and decision rationale. -- Marked roadmap 10.4.1 and 10.4.2 complete in `docs/roadmap.md`. +- Marked roadmap 16.4.1 and 16.4.2 complete in `docs/roadmap.md`. Repository-health fixes needed for validation: @@ -167,28 +167,28 @@ Repository-health fixes needed for validation: Validation evidence (all passing): - `cargo test --test client_runtime -- --nocapture`: - `/tmp/10-4-1-client-runtime-targeted.log` -- `make test-bdd`: `/tmp/10-4-1-bdd.log` -- `make fmt`: `/tmp/10-4-1-make-fmt.log` -- `make check-fmt`: `/tmp/10-4-1-check-fmt.log` -- `make lint`: `/tmp/10-4-1-lint.log` -- `make test`: `/tmp/10-4-1-test.log` -- `make test-doc`: `/tmp/10-4-1-test-doc.log` -- `make doctest-benchmark`: `/tmp/10-4-1-doctest-benchmark.log` -- `make markdownlint`: `/tmp/10-4-1-markdownlint.log` -- `make nixie`: `/tmp/10-4-1-nixie.log` + `/tmp/16-4-1-client-runtime-targeted.log` +- `make test-bdd`: `/tmp/16-4-1-bdd.log` +- `make fmt`: `/tmp/16-4-1-make-fmt.log` +- `make check-fmt`: `/tmp/16-4-1-check-fmt.log` +- `make lint`: `/tmp/16-4-1-lint.log` +- `make test`: `/tmp/16-4-1-test.log` +- `make test-doc`: `/tmp/16-4-1-test-doc.log` +- `make doctest-benchmark`: `/tmp/16-4-1-doctest-benchmark.log` +- `make markdownlint`: `/tmp/16-4-1-markdownlint.log` +- `make nixie`: `/tmp/16-4-1-nixie.log` - Manual runtime proof: - `cargo run --example echo --features examples` (background server): - `/tmp/10-4-1-echo-example.log` + `/tmp/16-4-1-echo-example.log` - `RUST_LOG=info cargo run --example client_echo_login --features examples`: - `/tmp/10-4-1-client-echo-login-example.log` + `/tmp/16-4-1-client-echo-login-example.log` - Observed success lines include: `decoded login acknowledgement username=guest correlation_id=Some(1)` and `client echo-login example completed successfully`. Roadmap status confirmation: -- `docs/roadmap.md` now marks both 10.4.1 and 10.4.2 as done. +- `docs/roadmap.md` now marks both 16.4.1 and 16.4.2 as done. Lessons learned: @@ -215,12 +215,12 @@ The relevant repository areas are: runtime and streaming APIs. - `docs/wireframe-client-design.md`: client design source of truth and the correct location for decision rationale. -- `docs/roadmap.md`: roadmap checklist containing 10.4.1 and 10.4.2. +- `docs/roadmap.md`: roadmap checklist containing 16.4.1 and 16.4.2. Terminology used in this plan: - "Echo server" means the server sends back the same decoded message payload. -- "Acknowledgement" in 10.4.1 means the client receives and decodes the echoed +- "Acknowledgement" in 16.4.1 means the client receives and decodes the echoed login payload as a successful reply. ## Plan of work @@ -302,8 +302,8 @@ validation. Mark both roadmap checkboxes complete in `docs/roadmap.md`: -- 10.4.1 done -- 10.4.2 done +- 16.4.1 done +- 16.4.2 done Run all required gates and capture output logs. Do not finish until every gate passes. @@ -317,56 +317,56 @@ All commands run from repository root (`/home/user/project`). Use ```sh set -o pipefail - cargo test --test client_runtime -- --nocapture | tee /tmp/10-4-1-client-runtime-targeted.log + cargo test --test client_runtime -- --nocapture | tee /tmp/16-4-1-client-runtime-targeted.log ``` ```sh set -o pipefail - make test-bdd | tee /tmp/10-4-1-bdd.log + make test-bdd | tee /tmp/16-4-1-bdd.log ``` 2. Full Rust quality gates before completion: ```sh set -o pipefail - make check-fmt | tee /tmp/10-4-1-check-fmt.log + make check-fmt | tee /tmp/16-4-1-check-fmt.log ``` ```sh set -o pipefail - make lint | tee /tmp/10-4-1-lint.log + make lint | tee /tmp/16-4-1-lint.log ``` ```sh set -o pipefail - make test | tee /tmp/10-4-1-test.log + make test | tee /tmp/16-4-1-test.log ``` ```sh set -o pipefail - make test-doc | tee /tmp/10-4-1-test-doc.log + make test-doc | tee /tmp/16-4-1-test-doc.log ``` ```sh set -o pipefail - make doctest-benchmark | tee /tmp/10-4-1-doctest-benchmark.log + make doctest-benchmark | tee /tmp/16-4-1-doctest-benchmark.log ``` 3. Documentation quality gates: ```sh set -o pipefail - make fmt | tee /tmp/10-4-1-make-fmt.log + make fmt | tee /tmp/16-4-1-make-fmt.log ``` ```sh set -o pipefail - make markdownlint | tee /tmp/10-4-1-markdownlint.log + make markdownlint | tee /tmp/16-4-1-markdownlint.log ``` ```sh set -o pipefail - make nixie | tee /tmp/10-4-1-nixie.log + make nixie | tee /tmp/16-4-1-nixie.log ``` 4. Manual runtime proof for the example: @@ -399,7 +399,7 @@ Acceptance criteria: - `docs/users-guide.md` and `docs/wireframe-client-design.md` include configuration tables, lifecycle diagrams, and troubleshooting guidance. - Design decision rationale is recorded in `docs/wireframe-client-design.md`. -- `docs/roadmap.md` marks 10.4.1 and 10.4.2 as done. +- `docs/roadmap.md` marks 16.4.1 and 16.4.2 as done. - All gates in "Concrete steps" pass. Quality method: @@ -422,9 +422,9 @@ Quality method: Capture concise evidence in commit/PR notes: -- command log file paths under `/tmp/10-4-1-*.log`, +- command log file paths under `/tmp/16-4-1-*.log`, - terminal output line showing decoded acknowledgement, -- and final roadmap diff lines marking 10.4.1/10.4.2 complete. +- and final roadmap diff lines marking 16.4.1/16.4.2 complete. ## Interfaces and dependencies @@ -442,8 +442,8 @@ Dependency policy: - Reuse existing crates (`tokio`, `wireframe`, `rstest`, `rstest-bdd`). - Do not add external dependencies. -Revision note (2026-02-23): Initial draft created for roadmap items 10.4.1 and -10.4.2. This revision defines acknowledgement semantics for the echo contract, +Revision note (2026-02-23): Initial draft created for roadmap items 16.4.1 and +16.4.2. This revision defines acknowledgement semantics for the echo contract, scopes file targets, and sets mandatory validation gates for implementation. Revision note (2026-02-23): Implementation completed. This revision records diff --git a/docs/execplans/11-1-1-outgoing-request-middleware-hooks.md b/docs/execplans/17-1-1-outgoing-request-middleware-hooks.md similarity index 98% rename from docs/execplans/11-1-1-outgoing-request-middleware-hooks.md rename to docs/execplans/17-1-1-outgoing-request-middleware-hooks.md index 07266d91..77693df3 100644 --- a/docs/execplans/11-1-1-outgoing-request-middleware-hooks.md +++ b/docs/execplans/17-1-1-outgoing-request-middleware-hooks.md @@ -1,4 +1,4 @@ -# Add middleware hooks for outgoing requests and incoming frames (11.1.1) +# Add middleware hooks for outgoing requests and incoming frames (17.1.1) This ExecPlan (execution plan) is a living document. The sections `Constraints`, `Tolerances`, `Risks`, `Progress`, `Surprises & Discoveries`, @@ -9,7 +9,7 @@ Status: COMPLETE ## Purpose / big picture -Roadmap item 11.1.1 requires middleware hooks on the wireframe client so that +Roadmap item 17.1.1 requires middleware hooks on the wireframe client so that metrics, retries, and authentication tokens can be injected symmetrically with server middleware. Today the server has a rich hook/middleware stack (`WireframeProtocol::before_send` in `src/hooks.rs`, `Service`/`Transform` @@ -50,7 +50,7 @@ Hard invariants. Violation requires escalation, not workarounds. `BytesMut` — are already available). - Design decisions must be recorded in `docs/wireframe-client-design.md`. - `docs/users-guide.md` must be updated with the new public API surface. -- `docs/roadmap.md` item 11.1.1 must be marked done only after all gates pass. +- `docs/roadmap.md` item 17.1.1 must be marked done only after all gates pass. ## Tolerances (exception triggers) @@ -468,7 +468,7 @@ lifecycle hooks" section (line ~1252). Include: **F3. Mark roadmap item as done:** -In `docs/roadmap.md`, change `- [ ] 11.1.1.` to `- [x] 11.1.1.`. +In `docs/roadmap.md`, change `- [ ] 17.1.1.` to `- [x] 17.1.1.`. ## Concrete steps @@ -499,7 +499,7 @@ Quality criteria (what "done" means): verified by a dedicated test. - `docs/users-guide.md` documents the new builder methods. - `docs/wireframe-client-design.md` records the design decisions. -- `docs/roadmap.md` marks 11.1.1 as done. +- `docs/roadmap.md` marks 17.1.1 as done. ## Idempotence and recovery @@ -539,7 +539,7 @@ beginning of the failed stage. No destructive operations are involved. | `tests/scenarios/mod.rs` | Add `mod client_request_hooks_scenarios` | 33 → ~34 | | `docs/wireframe-client-design.md` | New "Request hooks" section | +~40 | | `docs/users-guide.md` | New subsection + table row | +~50 | -| `docs/roadmap.md` | Mark 11.1.1 done | 1 char change | +| `docs/roadmap.md` | Mark 17.1.1 done | 1 char change | Total: 6 new files + 16 modified files = 22 files. All under 400 lines. diff --git a/docs/execplans/11-1-2-structured-logging-and-tracing-spans.md b/docs/execplans/17-1-2-structured-logging-and-tracing-spans.md similarity index 98% rename from docs/execplans/11-1-2-structured-logging-and-tracing-spans.md rename to docs/execplans/17-1-2-structured-logging-and-tracing-spans.md index 5c5d5bbb..a565c031 100644 --- a/docs/execplans/11-1-2-structured-logging-and-tracing-spans.md +++ b/docs/execplans/17-1-2-structured-logging-and-tracing-spans.md @@ -1,4 +1,4 @@ -# 11.1.2 Structured logging and tracing spans +# 17.1.2 Structured logging and tracing spans This ExecPlan (execution plan) is a living document. The sections `Constraints`, `Tolerances`, `Risks`, `Progress`, `Surprises & Discoveries`, @@ -9,7 +9,7 @@ Status: COMPLETE ## Purpose / big picture -Roadmap item 11.1.2 requires structured logging and tracing spans around client +Roadmap item 17.1.2 requires structured logging and tracing spans around client connect, call, send, receive, close, and stream lifecycle events, plus configuration for per-command timing. After this change, every client operation emits a `tracing` span with structured fields (frame size, correlation ID, @@ -45,7 +45,7 @@ Hard invariants. Violation requires escalation, not workarounds. `tracing-test` are already in `Cargo.toml`). - Design decisions must be recorded in `docs/wireframe-client-design.md`. - `docs/users-guide.md` must be updated with the new public API surface. -- `docs/roadmap.md` item 11.1.2 must be marked done only after all gates pass. +- `docs/roadmap.md` item 17.1.2 must be marked done only after all gates pass. ## Tolerances (exception triggers) @@ -563,7 +563,7 @@ usage, span output examples, and per-command timing example. **E3. Mark roadmap item done.** -In `docs/roadmap.md`, change `- [ ] 11.1.2.` to `- [x] 11.1.2.`. +In `docs/roadmap.md`, change `- [ ] 17.1.2.` to `- [x] 17.1.2.`. **E4. Final validation.** @@ -605,7 +605,7 @@ Quality criteria (what "done" means): - The `frame_count()` method on `ResponseStream` is public. - `docs/users-guide.md` documents the new builder methods and `TracingConfig`. - `docs/wireframe-client-design.md` records the design decisions. -- `docs/roadmap.md` marks 11.1.2 as done. +- `docs/roadmap.md` marks 17.1.2 as done. ## Idempotence and recovery @@ -671,7 +671,7 @@ client.call_streaming{correlation_id=1, frame.bytes=42} [DEBUG] | `tests/scenarios/mod.rs` | ~36 | ~37 | `mod client_tracing_scenarios;` | | `docs/wireframe-client-design.md` | 369 | ~400 | "Tracing instrumentation" section | | `docs/users-guide.md` | 1926 | ~1975 | "Client tracing" subsection | -| `docs/roadmap.md` | ~500 | ~500 | Mark 11.1.2 done | +| `docs/roadmap.md` | ~500 | ~500 | Mark 17.1.2 done | Total: 8 new + 15 modified = 23 files. All source files under 400 lines. diff --git a/docs/execplans/11-2-1-configurable-connection-pool-preserving-preamble.md b/docs/execplans/17-2-1-configurable-connection-pool-preserving-preamble.md similarity index 93% rename from docs/execplans/11-2-1-configurable-connection-pool-preserving-preamble.md rename to docs/execplans/17-2-1-configurable-connection-pool-preserving-preamble.md index 12736f3e..b9371cfc 100644 --- a/docs/execplans/11-2-1-configurable-connection-pool-preserving-preamble.md +++ b/docs/execplans/17-2-1-configurable-connection-pool-preserving-preamble.md @@ -1,4 +1,4 @@ -# Implement hybrid client connection pooling with `bb8` and custom per-socket admission/fairness controls (11.2.1) +# Implement hybrid client connection pooling with `bb8` and custom per-socket admission/fairness controls (17.2.1) This ExecPlan (execution plan) is a living document. The sections `Constraints`, `Tolerances`, `Risks`, `Progress`, `Surprises & Discoveries`, @@ -9,7 +9,7 @@ Status: COMPLETE (all validation commands currently pass) ## Purpose / big picture -Roadmap item `11.2.1` requires a configurable client connection pool that keeps +Roadmap item `17.2.1` requires a configurable client connection pool that keeps already-negotiated preamble state on reused sockets, enforces a bounded number of admitted operations per socket, and recycles idle sockets to avoid stale connections. @@ -25,7 +25,7 @@ and acquire pooled client leases for request work. Reused leases stay on warm connections (no repeated preamble handshake), while idle connections are re-established automatically when they exceed the configured idle threshold. -Trade-off accepted for `11.2.1`: more functionality now with battle-tested pool +Trade-off accepted for `17.2.1`: more functionality now with battle-tested pool operations. If deeper per-socket multiplex control is needed later, the pool internals will be reviewed and potentially revised in a follow-up phase. @@ -37,11 +37,11 @@ Observable success: through user-facing scenarios; - `docs/users-guide.md` documents the public pool API and migration guidance; - design decisions are recorded in `docs/wireframe-client-design.md`; and -- `docs/roadmap.md` marks `11.2.1` as done only after all quality gates pass. +- `docs/roadmap.md` marks `17.2.1` as done only after all quality gates pass. ## Constraints -- Scope is limited to roadmap item `11.2.1`. +- Scope is limited to roadmap item `17.2.1`. - Existing `WireframeClient` single-connection APIs must remain backward compatible. - No single source file may exceed 400 lines; extract submodules before @@ -60,7 +60,7 @@ Observable success: names exactly. - Design decisions must be added to `docs/wireframe-client-design.md`. - Public API changes must be documented in `docs/users-guide.md`. -- `docs/roadmap.md` `11.2.1` is updated to done only after full validation. +- `docs/roadmap.md` `17.2.1` is updated to done only after full validation. - Follow guidance from: - `docs/generic-message-fragmentation-and-re-assembly-design.md` - `docs/multi-packet-and-streaming-responses-design.md` @@ -80,7 +80,7 @@ Observable success: - Dependencies: if a new crate other than `bb8` is required, stop and escalate. - Ambiguity: if in-flight semantics cannot be made coherent without also - implementing `11.2.2` fairness/pool-handle work, stop and escalate with + implementing `17.2.2` fairness/pool-handle work, stop and escalate with alternatives. - Iterations: if the same gate fails 3 times after focused fixes, stop and escalate. @@ -116,7 +116,7 @@ Observable success: ## Progress -- [x] (2026-03-05 17:47Z) Drafted ExecPlan for `11.2.1`. +- [x] (2026-03-05 17:47Z) Drafted ExecPlan for `17.2.1`. - [x] (2026-03-06 09:42Z) Revised strategy to the hybrid `bb8` + custom admission/fairness approach. - [x] Stage A: finalize pool interface and internal module boundaries. @@ -133,7 +133,7 @@ Observable success: `docs/wireframe-client-design.md`. Evidence: repository search for `PoolHandle`, `connection pool`, and client-side pool code only matches roadmap/design placeholders. Impact: this plan defines initial pool semantics - explicitly to avoid implicit assumptions leaking into `11.2.2`. + explicitly to avoid implicit assumptions leaking into `17.2.2`. - Observation: `src/client/runtime.rs` is at 399 lines. Evidence: `wc -l src/client/runtime.rs`. Impact: all pool implementation must @@ -166,11 +166,11 @@ Observable success: fairness controls. Rationale: this delivers battle-tested pooling behaviour quickly while preserving Wireframe-specific control points. If deeper multiplex control is needed later, that can be reviewed without blocking - `11.2.1`. Date/Author: 2026-03-06 / plan revision. + `17.2.1`. Date/Author: 2026-03-06 / plan revision. - Decision: define in-flight limiting as a per-socket admission bound enforced by permits (not transport-level multiplexing fairness). Rationale: satisfies - `11.2.1` while keeping `11.2.2` (`PoolHandle` fairness) as a follow-on + `17.2.1` while keeping `17.2.2` (`PoolHandle` fairness) as a follow-on feature. Date/Author: 2026-03-05 / plan phase. - Decision: perform idle recycling on acquisition path (lazy recycle) instead @@ -340,13 +340,13 @@ alternatives before implementation continues. Update docs after implementation passes tests: - `docs/wireframe-client-design.md`: - add a new section/decision record for `11.2.1` covering pool semantics, + add a new section/decision record for `17.2.1` covering pool semantics, in-flight definition, preamble lifecycle, and idle recycle strategy. - `docs/users-guide.md`: add public API guidance, config reference row(s), and usage example for pooled connections. - `docs/roadmap.md`: - mark `11.2.1` as done. + mark `17.2.1` as done. Also add/adjust rustdoc examples for new public pool API and run doctest gates. @@ -356,22 +356,22 @@ Run from repository root (`/home/user/project`). Use logs for every gate. ```plaintext set -o pipefail -cargo test --all-features src::client::tests::pool 2>&1 | tee /tmp/11-2-1-unit-red.log -cargo test --test bdd --all-features -- client_pool 2>&1 | tee /tmp/11-2-1-bdd-red.log +cargo test --all-features src::client::tests::pool 2>&1 | tee /tmp/17-2-1-unit-red.log +cargo test --test bdd --all-features -- client_pool 2>&1 | tee /tmp/17-2-1-bdd-red.log ``` Expected during Stage B: at least one new test fails before implementation. ```plaintext set -o pipefail -make fmt 2>&1 | tee /tmp/11-2-1-fmt.log -make check-fmt 2>&1 | tee /tmp/11-2-1-check-fmt.log -make lint 2>&1 | tee /tmp/11-2-1-lint.log -make test 2>&1 | tee /tmp/11-2-1-test.log -make test-doc 2>&1 | tee /tmp/11-2-1-test-doc.log -make doctest-benchmark 2>&1 | tee /tmp/11-2-1-doctest-benchmark.log -make markdownlint MDLINT=/root/.bun/bin/markdownlint-cli2 2>&1 | tee /tmp/11-2-1-markdownlint.log -make nixie 2>&1 | tee /tmp/11-2-1-nixie.log +make fmt 2>&1 | tee /tmp/17-2-1-fmt.log +make check-fmt 2>&1 | tee /tmp/17-2-1-check-fmt.log +make lint 2>&1 | tee /tmp/17-2-1-lint.log +make test 2>&1 | tee /tmp/17-2-1-test.log +make test-doc 2>&1 | tee /tmp/17-2-1-test-doc.log +make doctest-benchmark 2>&1 | tee /tmp/17-2-1-doctest-benchmark.log +make markdownlint MDLINT=/root/.bun/bin/markdownlint-cli2 2>&1 | tee /tmp/17-2-1-markdownlint.log +make nixie 2>&1 | tee /tmp/17-2-1-nixie.log ``` Expected final result: all commands exit `0` and logs contain no new warnings @@ -396,7 +396,7 @@ Acceptance is behavioural and must be demonstrated by tests and docs. - Documentation: - `docs/users-guide.md` includes new consumer-facing pool interface details; - `docs/wireframe-client-design.md` records design decisions; - - `docs/roadmap.md` marks `11.2.1` done. + - `docs/roadmap.md` marks `17.2.1` done. ## Idempotence and recovery @@ -410,7 +410,7 @@ Acceptance is behavioural and must be demonstrated by tests and docs. ## Artifacts and notes -Capture implementation evidence in temporary logs under `/tmp/11-2-1-*.log`. At +Capture implementation evidence in temporary logs under `/tmp/17-2-1-*.log`. At minimum retain: - first red test run proving tests fail before implementation; @@ -488,7 +488,7 @@ Dependencies for this milestone: ## Revision note -- 2026-03-05: Initial draft created for roadmap item `11.2.1`, with staged +- 2026-03-05: Initial draft created for roadmap item `17.2.1`, with staged implementation, test strategy (`rstest` + `rstest-bdd`), explicit tolerance gates, and required documentation/roadmap updates. - 2026-03-06: Revised to a hybrid pool strategy: `bb8` for lifecycle/reaping/ diff --git a/docs/execplans/11-2-2-expose-pool-handle-api.md b/docs/execplans/17-2-2-expose-pool-handle-api.md similarity index 92% rename from docs/execplans/11-2-2-expose-pool-handle-api.md rename to docs/execplans/17-2-2-expose-pool-handle-api.md index 83e51772..a978d85c 100644 --- a/docs/execplans/11-2-2-expose-pool-handle-api.md +++ b/docs/execplans/17-2-2-expose-pool-handle-api.md @@ -1,4 +1,4 @@ -# Expose a `PoolHandle` API with fair pooled acquisition for logical sessions (11.2.2) +# Expose a `PoolHandle` API with fair pooled acquisition for logical sessions (17.2.2) This ExecPlan (execution plan) is a living document. The sections `Constraints`, `Tolerances`, `Risks`, `Progress`, `Surprises & Discoveries`, @@ -9,7 +9,7 @@ Status: COMPLETE ## Purpose / big picture -Roadmap item `11.2.2` exists because the current pooled-client surface is still +Roadmap item `17.2.2` exists because the current pooled-client surface is still too low-level for larger deployments. Today, all contention flows through `WireframeClientPool::acquire()`. That API is correct for manual control, but it gives the pool no durable notion of "logical session A" versus "logical @@ -27,10 +27,10 @@ access. Observable success is: preserves back-pressure, and does not regress warm reuse or idle recycle; - behavioural tests written with `rstest-bdd` v0.5.0 prove the same behaviour through user-visible scenarios; -- `docs/wireframe-client-design.md` records the `11.2.2` design decisions; +- `docs/wireframe-client-design.md` records the `17.2.2` design decisions; - `docs/users-guide.md` explains when to use `PoolHandle` instead of direct `pool.acquire()`; and -- `docs/roadmap.md` marks `11.2.2` as done only after all quality gates pass. +- `docs/roadmap.md` marks `17.2.2` as done only after all quality gates pass. This plan deliberately keeps `PooledClientLease` as the low-level escape hatch for split-phase workflows (`send` followed later by `receive`). `PoolHandle` @@ -39,13 +39,13 @@ only where they remain safe on a shared pool. ## Constraints -- Scope is limited to roadmap item `11.2.2`. +- Scope is limited to roadmap item `17.2.2`. - Existing `WireframeClientPool`, `ClientPoolConfig`, and `PooledClientLease` APIs must remain backward compatible. - The `pool` Cargo feature remains the opt-in gate for all pooled client code. - Fairness must be additive above the existing slot-permit back-pressure. The new API must not bypass `max_in_flight_per_socket`, socket serialization, or - idle recycle rules introduced in `11.2.1`. + idle recycle rules introduced in `17.2.1`. - `PoolHandle` must represent a logical-session fairness identity, not a promise of transport affinity to one physical socket. - Split-phase receive semantics must remain explicit. Do not imply that a @@ -63,7 +63,7 @@ only where they remain safe on a shared pool. names. - Design decisions must be recorded in `docs/wireframe-client-design.md`. - Public interface changes must be recorded in `docs/users-guide.md`. -- `docs/roadmap.md` `11.2.2` is updated to done only after full validation. +- `docs/roadmap.md` `17.2.2` is updated to done only after full validation. - Follow guidance from: - `docs/generic-message-fragmentation-and-re-assembly-design.md` - `docs/multi-packet-and-streaming-responses-design.md` @@ -78,7 +78,7 @@ only where they remain safe on a shared pool. - Scope: if implementation requires a full response router for uncorrelated `receive()` traffic across shared handles, stop and escalate. That is a - materially larger feature than `11.2.2`. + materially larger feature than `17.2.2`. - Interface: if delivering `PoolHandle` requires removing or breaking `WireframeClientPool::acquire()` or `PooledClientLease`, stop and escalate. - Surface area: if more than 20 files or roughly 1600 net lines change before @@ -128,8 +128,8 @@ only where they remain safe on a shared pool. ## Progress -- [x] (2026-03-10 00:00Z) Reviewed roadmap item `11.2.2`, the completed - `11.2.1` pool implementation, relevant design/testing docs, and project notes. +- [x] (2026-03-10 00:00Z) Reviewed roadmap item `17.2.2`, the completed + `17.2.1` pool implementation, relevant design/testing docs, and project notes. - [x] (2026-03-10 00:00Z) Drafted this ExecPlan. - [x] (2026-03-12 00:00Z) Stage A: confirmed the public API boundary and internal scheduler shape around `ClientPoolInner`, `PoolHandle`, and @@ -148,10 +148,10 @@ only where they remain safe on a shared pool. Evidence: `src/client/pool/client_pool.rs`. Impact: current contention is "works eventually" rather than a documented fairness contract. -- Observation: the existing `11.2.1` design already treats the pool as a +- Observation: the existing `17.2.1` design already treats the pool as a hybrid of `bb8` lifecycle management plus Wireframe-owned admission control. - Evidence: `docs/wireframe-client-design.md` decision record for `11.2.1`. - Impact: `11.2.2` should extend the Wireframe-owned layer rather than fight or + Evidence: `docs/wireframe-client-design.md` decision record for `17.2.1`. + Impact: `17.2.2` should extend the Wireframe-owned layer rather than fight or replace `bb8`. - Observation: pooled BDD coverage already lives in a separate @@ -408,7 +408,7 @@ intended scope. ### Stage D: documentation, doctests, and roadmap updates -Update `docs/wireframe-client-design.md` with a new `11.2.2` decision record +Update `docs/wireframe-client-design.md` with a new `17.2.2` decision record covering: - why `PoolHandle` represents logical-session fairness instead of socket @@ -430,7 +430,7 @@ Add or update Rustdoc examples on every new public type and method. Keep the examples realistic and runnable so `make test-doc` and `make doctest-benchmark` stay green. -Finally, mark roadmap item `11.2.2` done in `docs/roadmap.md` only after all +Finally, mark roadmap item `17.2.2` done in `docs/roadmap.md` only after all quality gates pass. ## Validation and evidence @@ -439,13 +439,13 @@ Run every gate through `tee` with `set -o pipefail` so failures are visible and logs remain inspectable: ```bash -set -o pipefail && make check-fmt | tee /tmp/wireframe-11-2-2-check-fmt.log -set -o pipefail && make lint | tee /tmp/wireframe-11-2-2-lint.log -set -o pipefail && make test | tee /tmp/wireframe-11-2-2-test.log -set -o pipefail && make test-doc | tee /tmp/wireframe-11-2-2-test-doc.log -set -o pipefail && make doctest-benchmark | tee /tmp/wireframe-11-2-2-doctest-benchmark.log -set -o pipefail && make markdownlint MDLINT=/root/.bun/bin/markdownlint-cli2 | tee /tmp/wireframe-11-2-2-markdownlint.log -set -o pipefail && make nixie | tee /tmp/wireframe-11-2-2-nixie.log +set -o pipefail && make check-fmt | tee /tmp/wireframe-17-2-2-check-fmt.log +set -o pipefail && make lint | tee /tmp/wireframe-17-2-2-lint.log +set -o pipefail && make test | tee /tmp/wireframe-17-2-2-test.log +set -o pipefail && make test-doc | tee /tmp/wireframe-17-2-2-test-doc.log +set -o pipefail && make doctest-benchmark | tee /tmp/wireframe-17-2-2-doctest-benchmark.log +set -o pipefail && make markdownlint MDLINT=/root/.bun/bin/markdownlint-cli2 | tee /tmp/wireframe-17-2-2-markdownlint.log +set -o pipefail && make nixie | tee /tmp/wireframe-17-2-2-nixie.log ``` Observable completion criteria: @@ -454,7 +454,7 @@ Observable completion criteria: - the new BDD scenarios fail before the scheduler work and pass afterward; - no existing pooled-client tests regress; - doctests for the new public API compile and run; and -- `docs/roadmap.md` shows `11.2.2` as done. +- `docs/roadmap.md` shows `17.2.2` as done. ## Outcomes & Retrospective @@ -474,16 +474,16 @@ Observable completion criteria: - `cargo test --lib client::tests::pool_handle --features pool -- --nocapture` - `cargo test --test bdd_pool --features "advanced-tests pool" -- --nocapture` - Final quality gates passed: - - `set -o pipefail && make fmt | tee /tmp/wireframe-11-2-2-fmt.log` - - `set -o pipefail && make check-fmt | tee /tmp/wireframe-11-2-2-check-fmt.log` - - `set -o pipefail && make lint | tee /tmp/wireframe-11-2-2-lint.log` - - `set -o pipefail && make test | tee /tmp/wireframe-11-2-2-test.log` - - `set -o pipefail && make test-doc | tee /tmp/wireframe-11-2-2-test-doc.log` - - `set -o pipefail && make doctest-benchmark | tee /tmp/wireframe-11-2-2-doctest-benchmark.log` + - `set -o pipefail && make fmt | tee /tmp/wireframe-17-2-2-fmt.log` + - `set -o pipefail && make check-fmt | tee /tmp/wireframe-17-2-2-check-fmt.log` + - `set -o pipefail && make lint | tee /tmp/wireframe-17-2-2-lint.log` + - `set -o pipefail && make test | tee /tmp/wireframe-17-2-2-test.log` + - `set -o pipefail && make test-doc | tee /tmp/wireframe-17-2-2-test-doc.log` + - `set -o pipefail && make doctest-benchmark | tee /tmp/wireframe-17-2-2-doctest-benchmark.log` - `set -o pipefail && make markdownlint MDLINT=/root/.bun/bin/markdownlint-cli2 | tee - /tmp/wireframe-11-2-2-markdownlint.log` - - `set -o pipefail && make nixie | tee /tmp/wireframe-11-2-2-nixie.log` + /tmp/wireframe-17-2-2-markdownlint.log` + - `set -o pipefail && make nixie | tee /tmp/wireframe-17-2-2-nixie.log` - Lessons learned: - fairness is easiest to explain when the scheduler operates on logical session handles rather than trying to retrofit identity into repeated diff --git a/docs/execplans/11-3-1-helpers-for-consuming-streaming-responses.md b/docs/execplans/17-3-1-helpers-for-consuming-streaming-responses.md similarity index 94% rename from docs/execplans/11-3-1-helpers-for-consuming-streaming-responses.md rename to docs/execplans/17-3-1-helpers-for-consuming-streaming-responses.md index d74810c9..49cb260e 100644 --- a/docs/execplans/11-3-1-helpers-for-consuming-streaming-responses.md +++ b/docs/execplans/17-3-1-helpers-for-consuming-streaming-responses.md @@ -1,4 +1,4 @@ -# Add ergonomic helpers for consuming streaming responses (11.3.1) +# Add ergonomic helpers for consuming streaming responses (17.3.1) This ExecPlan (execution plan) is a living document. The sections `Constraints`, `Tolerances`, `Risks`, `Progress`, `Surprises & Discoveries`, @@ -9,7 +9,7 @@ Status: DONE ## Purpose / big picture -Roadmap item `11.3.1` exists because the client can already *receive* streaming +Roadmap item `17.3.1` exists because the client can already *receive* streaming responses, but larger protocol implementations still need to hand-roll the same consumption loop over and over: @@ -45,12 +45,12 @@ Observable success: - `docs/users-guide.md` documents the new public interface and when to prefer it over manual `StreamExt::next` loops; - the relevant streaming/client design documents record the decision; and -- `docs/roadmap.md` marks `11.3.1` done only after the full validation suite +- `docs/roadmap.md` marks `17.3.1` done only after the full validation suite passes. ## Constraints -- Scope is limited to roadmap item `11.3.1`. +- Scope is limited to roadmap item `17.3.1`. - Existing public streaming APIs must remain backward compatible: `WireframeClient::call_streaming`, `WireframeClient::receive_streaming`, `client::ResponseStream`, `response::Response`, and `response::FrameStream` @@ -82,7 +82,7 @@ Observable success: `docs/wireframe-client-design.md` for the client-facing API surface and `docs/multi-packet-and-streaming-responses-design.md` for the streaming model. - Public interface changes must be added to `docs/users-guide.md`. -- `docs/roadmap.md` item `11.3.1` is updated to done only after full +- `docs/roadmap.md` item `17.3.1` is updated to done only after full validation. - Follow guidance from: - `docs/generic-message-fragmentation-and-re-assembly-design.md` @@ -98,7 +98,7 @@ Observable success: - Scope: if implementing the helper requires a response-router or a true multi-request demultiplexer on one client connection, stop and escalate. That - is a materially larger feature than `11.3.1`. + is a materially larger feature than `17.3.1`. - Surface area: if more than 18 files or roughly 1200 net lines must change before documentation additions, stop and re-evaluate the design. - Dependencies: do not add a new crate. Reuse `futures`, Tokio, and existing @@ -147,10 +147,10 @@ Observable success: ## Progress -- [x] (2026-03-21 00:00Z) Reviewed roadmap item `11.3.1`, the execplan +- [x] (2026-03-21 00:00Z) Reviewed roadmap item `17.3.1`, the execplan authoring rules, project instructions, and stored project notes. - [x] (2026-03-21 00:00Z) Read the referenced design, testing, and user-guide - documents plus the existing 11.x execplans and current client streaming + documents plus the existing 17.x execplans and current client streaming implementation. - [x] (2026-03-21 00:00Z) Drafted this ExecPlan. - [x] Stage A: finalized the public helper shape and module boundaries with @@ -166,7 +166,7 @@ Observable success: - Observation: `client::ResponseStream` already handles transport reads, deserialization, terminator detection, correlation validation, and request - hook integration. Evidence: `src/client/response_stream.rs`. Impact: `11.3.1` + hook integration. Evidence: `src/client/response_stream.rs`. Impact: `17.3.1` should not add a second transport abstraction; it should layer *above* the existing typed frame stream. @@ -195,7 +195,7 @@ Observable success: the helper requires `Unpin` at the call site. Evidence: `src/client/streaming_helpers.rs`. Impact: `typed_with` remains broadly usable with `ResponseStream` and simple test streams without introducing a - new pin-projection dependency for `11.3.1`. + new pin-projection dependency for `17.3.1`. ## Decision Log @@ -389,7 +389,7 @@ public surface only so it participates cleanly in `make test-doc` and If the generic helper naturally works with `response::FrameStream` too, add a small integration test under `tests/` proving that path. If not, do not force -it in `11.3.1`; note the narrower scope in the design docs and leave the +it in `17.3.1`; note the narrower scope in the design docs and leave the generic `Response::Stream` support as a follow-up. ### Stage D: update documentation and roadmap @@ -414,7 +414,7 @@ client" section. Add: fidelity. Only after implementation, docs, and validation are complete, mark roadmap item -`11.3.1` as done in `docs/roadmap.md`. +`17.3.1` as done in `docs/roadmap.md`. ## Validation and evidence @@ -422,14 +422,14 @@ Run the full validation flow with `set -o pipefail` and `tee`, keeping the logs for review: ```sh -set -o pipefail && make fmt | tee /tmp/11-3-1-fmt.log -set -o pipefail && make markdownlint MDLINT=/root/.bun/bin/markdownlint-cli2 | tee /tmp/11-3-1-markdownlint.log -set -o pipefail && make check-fmt | tee /tmp/11-3-1-check-fmt.log -set -o pipefail && make lint | tee /tmp/11-3-1-lint.log -set -o pipefail && make test | tee /tmp/11-3-1-test.log -set -o pipefail && make test-doc | tee /tmp/11-3-1-test-doc.log -set -o pipefail && make doctest-benchmark | tee /tmp/11-3-1-doctest-benchmark.log -set -o pipefail && make nixie | tee /tmp/11-3-1-nixie.log +set -o pipefail && make fmt | tee /tmp/17-3-1-fmt.log +set -o pipefail && make markdownlint MDLINT=/root/.bun/bin/markdownlint-cli2 | tee /tmp/17-3-1-markdownlint.log +set -o pipefail && make check-fmt | tee /tmp/17-3-1-check-fmt.log +set -o pipefail && make lint | tee /tmp/17-3-1-lint.log +set -o pipefail && make test | tee /tmp/17-3-1-test.log +set -o pipefail && make test-doc | tee /tmp/17-3-1-test-doc.log +set -o pipefail && make doctest-benchmark | tee /tmp/17-3-1-doctest-benchmark.log +set -o pipefail && make nixie | tee /tmp/17-3-1-nixie.log ``` Before declaring success, confirm all of the following: @@ -448,12 +448,12 @@ To be completed during implementation. At minimum, record: generalized to `response::FrameStream`; - whether a trait-only design was sufficient or a macro was also added; - the validation commands that passed; and -- any follow-up work intentionally deferred to `11.3.2` or later. +- any follow-up work intentionally deferred to `17.3.2` or later. - Final public API: `client::StreamingResponseExt::typed_with` returning `client::TypedResponseStream`. The shipped helper remains client-side for `Stream>`; it was not generalized to - `response::FrameStream` in `11.3.1`. + `response::FrameStream` in `17.3.1`. - Trait-only outcome: sufficient. No macro was added. - Validation status so far: targeted `cargo test typed_response_stream --all-features` and `cargo test client_streaming diff --git a/docs/execplans/12-3-2-in-process-server-and-client-pair-test-harness.md b/docs/execplans/17-3-2-in-process-server-and-client-pair-test-harness.md similarity index 96% rename from docs/execplans/12-3-2-in-process-server-and-client-pair-test-harness.md rename to docs/execplans/17-3-2-in-process-server-and-client-pair-test-harness.md index e45aba2a..b55982ae 100644 --- a/docs/execplans/12-3-2-in-process-server-and-client-pair-test-harness.md +++ b/docs/execplans/17-3-2-in-process-server-and-client-pair-test-harness.md @@ -1,4 +1,4 @@ -# Publish an in-process server and client pair test harness (12.3.2) +# Publish an in-process server and client pair test harness (17.3.2) This ExecPlan (execution plan) is a living document. The sections `Constraints`, `Tolerances`, `Risks`, `Progress`, `Surprises & Discoveries`, @@ -9,7 +9,7 @@ Status: COMPLETE ## Purpose / big picture -Roadmap item `12.3.2` exists because client-facing tests currently repeat the +Roadmap item `17.3.2` exists because client-facing tests currently repeat the same setup in many places: bind a loopback listener, spawn a server task, record its address, connect a `WireframeClient`, hold onto one or more join handles, and remember to abort or shut everything down in `Drop`. @@ -42,12 +42,12 @@ Observable success: - relevant design documentation records the chosen API boundary and rationale; - `docs/users-guide.md` explains when to use the harness and what public surface it adds for library consumers; and -- `docs/roadmap.md` marks `12.3.2` done only after the full validation suite +- `docs/roadmap.md` marks `17.3.2` done only after the full validation suite passes. ## Constraints -- Scope is limited to roadmap item `12.3.2`. +- Scope is limited to roadmap item `17.3.2`. - Existing public client APIs must remain backward compatible: `WireframeClient`, `WireframeClientBuilder`, `WireframeServer`, `wireframe::testkit`, and the existing `wireframe_testing` helpers must keep @@ -86,7 +86,7 @@ Observable success: - `docs/wireframe-client-design.md` - `docs/wireframe-testing-crate.md` - `docs/users-guide.md` must describe the public harness surface for consumers. -- `docs/roadmap.md` item `12.3.2` is updated to done only after validation. +- `docs/roadmap.md` item `17.3.2` is updated to done only after validation. - Follow guidance from: - `docs/generic-message-fragmentation-and-re-assembly-design.md` - `docs/multi-packet-and-streaming-responses-design.md` @@ -102,7 +102,7 @@ Observable success: - Scope: if the public API cannot be implemented without also introducing response demultiplexing, transport abstraction, or a general-purpose async process supervisor, stop and escalate. Those are larger features than - `12.3.2`. + `17.3.2`. - Surface area: if implementation requires changes to more than 16 files or roughly 1200 net lines before documentation updates, stop and re-evaluate. - Dependencies: do not add a new crate. Reuse Tokio, existing Wireframe types, @@ -200,7 +200,7 @@ Observable success: currently share a comparable public loopback TCP harness. Evidence: `src/testkit/support.rs`, `wireframe_testing/src/helpers/runtime.rs`, and the many ad hoc listener-based client fixtures under - `tests/fixtures/client_*.rs`. Impact: `12.3.2` should focus on loopback + `tests/fixtures/client_*.rs`. Impact: `17.3.2` should focus on loopback server/client orchestration rather than inventing another in-memory driver. - Observation: `wireframe_testing` already exposes exactly the supporting @@ -321,7 +321,7 @@ The implementation will touch four main areas. - `docs/wireframe-client-design.md` for the client/testing design record; - `docs/wireframe-testing-crate.md` for the companion crate's public design; - `docs/users-guide.md` for consumer-facing usage; - - `docs/roadmap.md` to mark `12.3.2` done after validation. + - `docs/roadmap.md` to mark `17.3.2` done after validation. 4. Existing client fixture examples @@ -516,7 +516,7 @@ Update `docs/users-guide.md` with a consumer-facing section that explains: - the crate or feature dependency shape required to use it. Only after all code and tests are complete, mark `docs/roadmap.md` item -`12.3.2` as done. +`17.3.2` as done. ## Stage F: Validation and evidence capture @@ -527,45 +527,45 @@ preserved even when the terminal truncates. Suggested targeted commands: ```sh -set -o pipefail && cargo test --test client_pair_harness 2>&1 | tee /tmp/12-3-2-client-pair-unit.log +set -o pipefail && cargo test --test client_pair_harness 2>&1 | tee /tmp/17-3-2-client-pair-unit.log ``` ```sh -set -o pipefail && cargo test --test bdd --all-features -- client_pair_harness 2>&1 | tee /tmp/12-3-2-client-pair-bdd.log +set -o pipefail && cargo test --test bdd --all-features -- client_pair_harness 2>&1 | tee /tmp/17-3-2-client-pair-bdd.log ``` Required repo gates after the targeted tests pass: ```sh -set -o pipefail && make fmt 2>&1 | tee /tmp/12-3-2-fmt.log +set -o pipefail && make fmt 2>&1 | tee /tmp/17-3-2-fmt.log ``` ```sh -set -o pipefail && make markdownlint MDLINT=/root/.bun/bin/markdownlint-cli2 2>&1 | tee /tmp/12-3-2-markdownlint.log +set -o pipefail && make markdownlint MDLINT=/root/.bun/bin/markdownlint-cli2 2>&1 | tee /tmp/17-3-2-markdownlint.log ``` ```sh -set -o pipefail && make check-fmt 2>&1 | tee /tmp/12-3-2-check-fmt.log +set -o pipefail && make check-fmt 2>&1 | tee /tmp/17-3-2-check-fmt.log ``` ```sh -set -o pipefail && make lint 2>&1 | tee /tmp/12-3-2-lint.log +set -o pipefail && make lint 2>&1 | tee /tmp/17-3-2-lint.log ``` ```sh -set -o pipefail && make test 2>&1 | tee /tmp/12-3-2-test.log +set -o pipefail && make test 2>&1 | tee /tmp/17-3-2-test.log ``` ```sh -set -o pipefail && make test-doc 2>&1 | tee /tmp/12-3-2-test-doc.log +set -o pipefail && make test-doc 2>&1 | tee /tmp/17-3-2-test-doc.log ``` ```sh -set -o pipefail && make doctest-benchmark 2>&1 | tee /tmp/12-3-2-doctest-benchmark.log +set -o pipefail && make doctest-benchmark 2>&1 | tee /tmp/17-3-2-doctest-benchmark.log ``` ```sh -set -o pipefail && make nixie 2>&1 | tee /tmp/12-3-2-nixie.log +set -o pipefail && make nixie 2>&1 | tee /tmp/17-3-2-nixie.log ``` If repo-wide markdown linting still fails on unrelated historical files, @@ -596,7 +596,7 @@ Files changed (existing): - `docs/wireframe-client-design.md` — added harness design section. - `docs/wireframe-testing-crate.md` — added crate layout entry and API section. - `docs/users-guide.md` — added consumer-facing harness section. -- `docs/roadmap.md` — marked `12.3.2` done. +- `docs/roadmap.md` — marked `17.3.2` done. Observations: diff --git a/docs/execplans/11-4-2-troubleshooting-section.md b/docs/execplans/17-4-2-troubleshooting-section.md similarity index 90% rename from docs/execplans/11-4-2-troubleshooting-section.md rename to docs/execplans/17-4-2-troubleshooting-section.md index a1658bb2..292ab72c 100644 --- a/docs/execplans/11-4-2-troubleshooting-section.md +++ b/docs/execplans/17-4-2-troubleshooting-section.md @@ -1,4 +1,4 @@ -# Expand the client troubleshooting guide and validate misconfiguration diagnostics (11.4.2) +# Expand the client troubleshooting guide and validate misconfiguration diagnostics (17.4.2) This ExecPlan (execution plan) is a living document. The sections `Constraints`, `Tolerances`, `Risks`, `Progress`, `Surprises & Discoveries`, @@ -9,7 +9,7 @@ Status: COMPLETED ## Purpose / big picture -Roadmap item `11.4.2` is not greenfield documentation work. The user guide +Roadmap item `17.4.2` is not greenfield documentation work. The user guide already contains a short `Client troubleshooting` section, but it is only a bullet list and does not yet give operators enough signal to distinguish the three misconfiguration classes named in the roadmap: codec length mismatch, @@ -26,7 +26,7 @@ be backed by executable evidence: user-facing scenarios. 3. Design-document updates in [`docs/wireframe-client-design.md`] recording any behavioural or wording decisions taken during implementation. -4. A roadmap update in [`docs/roadmap.md`] marking `11.4.2` done only after all +4. A roadmap update in [`docs/roadmap.md`] marking `17.4.2` done only after all quality gates pass. Observable success means: @@ -43,15 +43,15 @@ Observable success means: ## Constraints -- Scope is limited to roadmap item `11.4.2`. +- Scope is limited to roadmap item `17.4.2`. - Existing public client APIs must remain source-compatible. This milestone is about diagnostics and documentation, not a broader client redesign. - The final documentation must update [`docs/users-guide.md`] even if no public - API signatures change, because the observable client failure surface is part + API signatures change because the observable client failure surface is part of the public contract. - Record all design decisions taken during implementation in [`docs/wireframe-client-design.md`]. -- Mark [`docs/roadmap.md`] entry `11.4.2` done only after all tests, lint, and +- Mark [`docs/roadmap.md`] entry `17.4.2` done only after all tests, lint, and documentation quality gates pass. - Unit coverage must use `rstest`. - Behavioural coverage must use `rstest-bdd` v0.5.0 and follow the repository's @@ -65,7 +65,7 @@ Observable success means: - Do not document `ClientError::PreambleWrite` as a user-observable signal unless implementation first makes that variant reachable from the actual preamble path. -- TLS remains future work in roadmap item `12.3.1`, so the troubleshooting +- TLS remains future work in roadmap item `18.3.1`, so the troubleshooting section may describe TLS-related deployment mistakes and their symptoms, but must not imply first-party TLS transport support exists today. - No new external dependencies should be introduced for this milestone. @@ -124,7 +124,7 @@ Observable success means: ## Progress - [x] (2026-03-21 00:00Z) Drafted the initial ExecPlan for roadmap item - `11.4.2`. + `17.4.2`. - [x] (2026-03-21 00:45Z) Stage A: confirmed that [`src/client/preamble_exchange.rs`] still reports write-side preamble failures through `ClientError::PreambleEncode`, not `PreambleWrite`, and @@ -139,7 +139,7 @@ Observable success means: scenario wiring, to cover the same diagnostics behaviourally. - [x] (2026-03-21 01:15Z) Stage D: rewrote the troubleshooting guidance in [`docs/users-guide.md`], aligned [`docs/wireframe-client-design.md`], and - added an explicit `11.4.2` decision record. + added an explicit `17.4.2` decision record. - [x] (2026-03-21 01:35Z) Stage E: ran the Rust and doc quality gates, updated [`docs/roadmap.md`], and recorded the final validation outcome below. Full `make markdownlint` still reports unrelated legacy MD029 failures in older @@ -170,8 +170,8 @@ Observable success means: before the troubleshooting docs are expanded. - TLS support is not yet a built-in client feature. [`docs/roadmap.md`] still - lists `12.3.1` ("Provide built-in middleware or guides for implementing TLS") - as future work, so `11.4.2` must stay within documentation and diagnostics + lists `18.3.1` ("Provide built-in middleware or guides for implementing TLS") + as future work, so `17.4.2` must stay within documentation and diagnostics for deployment mismatches. - The initial full-suite `make test` run failed once in the existing @@ -219,8 +219,8 @@ Shipped outcomes: contention, and transport disconnects. - [`docs/wireframe-client-design.md`] now mirrors that wording, records that `PreambleWrite` is not currently user-observable, and adds a dedicated - `11.4.2` decision record. -- [`docs/roadmap.md`] marks `11.4.2` complete. + `17.4.2` decision record. +- [`docs/roadmap.md`] marks `17.4.2` complete. - New executable evidence now covers the roadmap's missing cases: - [`tests/client_runtime.rs`] adds `client_surfaces_tls_protocol_mismatch_as_wireframe_io`; @@ -243,7 +243,7 @@ Validation summary: [`docs/users-guide.md`], [`docs/wireframe-client-design.md`], [`docs/roadmap.md`], and this ExecPlan - Known unrelated baseline: full `make markdownlint` still fails on legacy - MD029 ordered-list numbering in older execplans outside the scope of `11.4.2` + MD029 ordered-list numbering in older execplans outside the scope of `17.4.2` ## Context and orientation @@ -254,7 +254,7 @@ The relevant repository areas are already in place: - [`docs/wireframe-client-design.md`] contains the client design record, including an existing troubleshooting bullet list that must stay aligned with the user guide. -- [`docs/roadmap.md`] tracks roadmap item `11.4.2`, which should remain +- [`docs/roadmap.md`] tracks roadmap item `17.4.2`, which should remain unchecked until the end of implementation. - [`src/client/error.rs`] defines the public client error variants that the docs must describe accurately. @@ -338,12 +338,12 @@ Targeted commands during this stage: ```sh set -o pipefail -cargo test --test client_runtime 2>&1 | tee /tmp/11-4-2-client-runtime.log +cargo test --test client_runtime 2>&1 | tee /tmp/17-4-2-client-runtime.log ``` ```sh set -o pipefail -cargo test --test client_preamble 2>&1 | tee /tmp/11-4-2-client-preamble.log +cargo test --test client_preamble 2>&1 | tee /tmp/17-4-2-client-preamble.log ``` ## Stage C: add behavioural coverage with `rstest-bdd` v0.5.0 @@ -368,12 +368,12 @@ Targeted commands during this stage: ```sh set -o pipefail -cargo test --test bdd --all-features -- client_runtime 2>&1 | tee /tmp/11-4-2-bdd-runtime.log +cargo test --test bdd --all-features -- client_runtime 2>&1 | tee /tmp/17-4-2-bdd-runtime.log ``` ```sh set -o pipefail -cargo test --test bdd --all-features -- client_preamble 2>&1 | tee /tmp/11-4-2-bdd-preamble.log +cargo test --test bdd --all-features -- client_preamble 2>&1 | tee /tmp/17-4-2-bdd-preamble.log ``` ## Stage D: rewrite the documentation and record the decisions @@ -403,54 +403,63 @@ The documentation must be explicit that TLS guidance is about diagnosing a misconfigured transport boundary today, not about enabling built-in TLS on the client. -## Stage E: run the full quality gates and close the roadmap item +## Stage E: run the agreed quality gates and close the roadmap item -After code and docs are complete, run the full validation suite from the +After code and docs are complete, run the agreed validation suite from the repository root. Use `tee` and `set -o pipefail` so failures are not hidden. +For Markdown linting, the gate for `17.4.2` is targeted validation of the +touched documentation files because full `make markdownlint` still reports +unrelated legacy MD029 failures in older execplans outside this roadmap item's +scope. ```sh set -o pipefail -make fmt 2>&1 | tee /tmp/11-4-2-fmt.log +make fmt 2>&1 | tee /tmp/17-4-2-fmt.log ``` ```sh set -o pipefail -make markdownlint MDLINT=/root/.bun/bin/markdownlint-cli2 2>&1 | tee /tmp/11-4-2-markdownlint.log +/root/.bun/bin/markdownlint-cli2 \ + docs/users-guide.md \ + docs/wireframe-client-design.md \ + docs/roadmap.md \ + docs/execplans/17-4-2-troubleshooting-section.md \ + 2>&1 | tee /tmp/17-4-2-markdownlint.log ``` ```sh set -o pipefail -make check-fmt 2>&1 | tee /tmp/11-4-2-check-fmt.log +make check-fmt 2>&1 | tee /tmp/17-4-2-check-fmt.log ``` ```sh set -o pipefail -make lint 2>&1 | tee /tmp/11-4-2-lint.log +make lint 2>&1 | tee /tmp/17-4-2-lint.log ``` ```sh set -o pipefail -make test 2>&1 | tee /tmp/11-4-2-test.log +make test 2>&1 | tee /tmp/17-4-2-test.log ``` ```sh set -o pipefail -make test-doc 2>&1 | tee /tmp/11-4-2-test-doc.log +make test-doc 2>&1 | tee /tmp/17-4-2-test-doc.log ``` ```sh set -o pipefail -make doctest-benchmark 2>&1 | tee /tmp/11-4-2-doctest-benchmark.log +make doctest-benchmark 2>&1 | tee /tmp/17-4-2-doctest-benchmark.log ``` ```sh set -o pipefail -make nixie 2>&1 | tee /tmp/11-4-2-nixie.log +make nixie 2>&1 | tee /tmp/17-4-2-nixie.log ``` -Only after these pass should implementation: +Only after these agreed gates pass should the implementation: -1. mark `11.4.2` as done in [`docs/roadmap.md`]; +1. mark `17.4.2` as done in [`docs/roadmap.md`]; 2. update the `Progress` and `Outcomes & Retrospective` sections in this ExecPlan; and 3. summarize the final evidence in the commit message and handoff note. @@ -468,5 +477,6 @@ Implementation is complete when all of the following are true: decisions taken for the troubleshooting guidance. 4. `rstest` coverage exists for the documented failure classes. 5. `rstest-bdd` coverage exists for the same failure classes. -6. [`docs/roadmap.md`] marks `11.4.2` done. -7. All validation commands in Stage E pass. +6. [`docs/roadmap.md`] marks `17.4.2` done. +7. All agreed validation commands in Stage E pass, including targeted + Markdown lint for the `17.4.2` documentation files. diff --git a/docs/frame-vec-u8-inventory.md b/docs/frame-vec-u8-inventory.md index 55ca79a5..90e1692c 100644 --- a/docs/frame-vec-u8-inventory.md +++ b/docs/frame-vec-u8-inventory.md @@ -356,22 +356,33 @@ The current direction is now explicit: ## Coordination notes -No matching roadmap item for this inventory work was found in the local docs -set on 2026-04-11, so nothing was marked done. If epic 284 is updated outside -the repository, the prepared link text and next-step summary are: +Epic 284 is now tracked locally in the dedicated +[`zero-copy-frame-and-payload-migration-roadmap.md`](zero-copy-frame-and-payload-migration-roadmap.md). + +The combined [`roadmap.md`](roadmap.md) carries the corresponding zero-copy +migration phases. That workstream turns the inventory into phased +implementation work, benchmark planning, and release preparation. + +The remaining unresolved design choices now have draft Architecture Decision +Records (ADRs): + +- [ADR 008](adr-008-zero-copy-public-byte-container.md) covers the public byte + container and edit-on-demand model. +- [ADR 009](adr-009-vec-u8-migration-rollout.md) covers the compatibility and + release rollout strategy. +- [ADR 010](adr-010-transport-frame-boundary-for-zero-copy.md) covers the + actor, codec-driver, and transport-frame boundary for the zero-copy path. + +If epic 284 is updated outside the repository, the local source-of-truth links +to add are: ```plaintext -Add inventory reference: docs/frame-vec-u8-inventory.md -Link label: Frame = Vec inventory - -Epic 284 workstreams: -1. Transport-frame substitution and actor/codec boundary work. -2. Public payload-owned API work: PacketParts, Envelope, middleware, and - client byte hooks, with minimal consumer boilerplate. -3. Compatibility cleanup: move Vec-only bridges such as - CorrelatableFrame for Vec out of the core surface once no longer needed. -4. Deferred: keep client preamble leftovers on Vec until broader public - byte APIs justify standardization. +Inventory: docs/frame-vec-u8-inventory.md +Roadmap: docs/zero-copy-frame-and-payload-migration-roadmap.md +ADRs: +- docs/adr-008-zero-copy-public-byte-container.md +- docs/adr-009-vec-u8-migration-rollout.md +- docs/adr-010-transport-frame-boundary-for-zero-copy.md ``` [^adr-004]: See diff --git a/docs/multi-packet-and-streaming-responses-design.md b/docs/multi-packet-and-streaming-responses-design.md index 7271e285..bb467e02 100644 --- a/docs/multi-packet-and-streaming-responses-design.md +++ b/docs/multi-packet-and-streaming-responses-design.md @@ -819,7 +819,7 @@ consistent with the server-side back-pressure architecture described in Section ### 12.6 Interleaved queue parity validation -Roadmap item `10.3.2` is validated by exercising interleaved high- and +Roadmap item `16.3.2` is validated by exercising interleaved high- and low-priority push queues through the client streaming surface. The parity coverage deliberately uses the same `ConnectionActor`, `FairnessConfig`, and `PushQueues` machinery as production paths, then feeds the resulting frame diff --git a/docs/roadmap.md b/docs/roadmap.md index bb6a9eb4..d42e6922 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -441,265 +441,396 @@ integration boundaries. `CodecError` taxonomy and recovery policy behaviours defined in 9.1.2. Requires 9.1.2. -## 10. Formal verification +## 10. Decision closure and baseline + +This phase turns the `Frame = Vec` inventory into approved design choices, +benchmark thresholds, and a concrete migration baseline before public API work +starts. See `docs/frame-vec-u8-inventory.md` and ADRs 008 through 010. + +### 10.1. Design decisions + +- [ ] 10.1.1. Approve the stable public byte-container and edit-on-demand model + for `PacketParts`, `Envelope`, middleware, client hooks, and serializer + output. See `docs/adr-008-zero-copy-public-byte-container.md`. +- [ ] 10.1.2. Approve the compatibility and rollout policy for downstream + users, including which `Vec` helpers survive the breaking release. See + `docs/adr-009-vec-u8-migration-rollout.md`. +- [ ] 10.1.3. Approve the actor and codec-driver boundary so `Vec` bridges + leave the core runtime deliberately rather than incidentally. See + `docs/adr-010-transport-frame-boundary-for-zero-copy.md`. + +### 10.2. Baselines and migration planning + +- [ ] 10.2.1. Capture allocation, copied-byte, throughput, and latency + baselines for inbound decode, middleware pass-through, request hooks, and + outbound encode on the default codec path. Requires 10.1.1. +- [ ] 10.2.2. Record benchmark acceptance thresholds that require removal of + the final default-path `Vec` copy between serialization and + `FrameCodec::wrap_payload`. Requires 10.2.1. +- [ ] 10.2.3. Publish a migration-guide outline listing the exact public + signatures and workflows that will change for packets, middleware, client + hooks, and serializers. Requires 10.1.2. + +## 11. Internal zero-copy foundations + +This phase removes the remaining internal `Vec` bottlenecks without +changing the public API shape prematurely. + +### 11.1. Internal byte surfaces + +- [ ] 11.1.1. Convert internal packet payload storage and serializer output to + the approved zero-copy-capable byte representation. Requires 10.1.1. +- [ ] 11.1.2. Remove the final default-path `Vec` copy between + serialization and `FrameCodec::wrap_payload`. Requires 11.1.1. +- [ ] 11.1.3. Update zero-copy-capable internal channels and replay buffers + identified in the inventory so they no longer force `Vec` hand-offs. + Requires 11.1.1. + +### 11.2. Actor and codec-driver boundary + +- [ ] 11.2.1. Implement the approved actor and codec-driver boundary in the + runtime so zero-copy transport framing does not depend on `Vec` bridges. + Requires 10.1.3. +- [ ] 11.2.2. Move `CorrelatableFrame for Vec` and similar runtime-only + bridges out of production paths once the replacement boundary is covered. + Requires 11.2.1. +- [ ] 11.2.3. Add allocation and pointer-reuse regressions for the internal + byte hand-off path on the default codec and at least one protocol-native + codec. Requires 11.1.2 and 11.2.1. + +## 12. Public API migration + +This phase flips the public packet, middleware, and client surfaces to the new +byte model while preserving explicit editing ergonomics. + +### 12.1. Packet and middleware surfaces + +- [ ] 12.1.1. Update `PacketParts`, `Envelope`, `ServiceRequest`, and + `ServiceResponse` to the approved byte model. Requires 10.1.1 and 11.1.1. +- [ ] 12.1.2. Preserve explicit edit-on-demand ergonomics for middleware, with + compatibility helpers only where the rollout policy requires them. Requires + 10.1.2 and 12.1.1. +- [ ] 12.1.3. Update server-side examples and behavioural coverage so + `Vec` is no longer taught as the default middleware frame shape. Requires + 12.1.2. + +### 12.2. Client-facing byte APIs + +- [ ] 12.2.1. Update `BeforeSendHook` and `Serializer::serialize` to the + approved byte surface. Requires 10.1.1 and 11.1.1. +- [ ] 12.2.2. Re-evaluate client preamble leftovers and document whether they + remain `Vec` for the first breaking release or migrate with the rest of + the client byte surface. Requires 10.1.2 and 12.2.1. +- [ ] 12.2.3. Add migration examples showing how existing `Vec` hook and + serializer call sites move to the new API. Requires 10.2.3 and 12.2.1. + +## 13. Validation, ecosystem updates, and documentation + +This phase proves the new byte model operationally and updates the companion +docs and test utilities to match. + +### 13.1. Compatibility cleanup + +- [ ] 13.1.1. Extend `wireframe_testing`, behavioural coverage, and examples so + `Vec` is no longer the default taught frame shape for codec and client + workflows. Requires 12.1.3 and 12.2.3. +- [ ] 13.1.2. Publish the migration guide and breaking-change summary for the + zero-copy API flip. Requires 10.2.3 and 12.2.3. +- [ ] 13.1.3. Close or supersede ADRs 008 through 010 once the implementation + and published documentation match the approved design. Requires 13.1.2. + +### 13.2. Performance and downstream validation + +- [ ] 13.2.1. Run the benchmark suite and compare results against the approved + thresholds from 10.2.2. Requires 11.1.2 and 12.2.1. +- [ ] 13.2.2. Add regression coverage proving read-only paths stay zero-copy + while mutation paths copy only on demand. Requires 12.1.2 and 12.2.1. +- [ ] 13.2.3. Run downstream canaries against representative middleware, hook, + and custom codec examples before release. Requires 13.1.2 and 13.2.1. + +## 14. Release rollout + +This phase packages the breaking change, publishes the upgrade story, and +captures any follow-up compatibility cleanup. + +### 14.1. Breaking release preparation + +- [ ] 14.1.1. Finalize the versioning plan for the public API break and record + whether it ships as the next major or pre-1.0 point release. Requires 10.1.2 + and 13.2.3. +- [ ] 14.1.2. Publish release notes, changelog entries, and upgrade guidance + covering removed `Vec` contracts and retained compatibility helpers. + Requires 13.1.2 and 14.1.1. +- [ ] 14.1.3. Move `CorrelatableFrame for Vec` and any similar temporary + bridges fully out of production paths once release validation is complete. + Requires 11.2.2 and 14.1.2. + +### 14.2. Post-release cleanup + +- [ ] 14.2.1. Review retained compatibility helpers and either accept them as + narrow shims or schedule their removal in follow-up work. Requires 14.1.2. +- [ ] 14.2.2. Capture any intentionally deferred `Vec` surfaces, together + with explicit rationale and the next review point. Requires 14.1.2. + +## 15. Formal verification This phase establishes the formal verification infrastructure and applies bounded model checking, state-space exploration, and deductive proofs to Wireframe's protocol, framing, and message assembly layers. -### 10.1. Verification workspace and tooling +### 15.1. Verification workspace and tooling -- [ ] 10.1.1. Convert the root manifest into a hybrid workspace while keeping +- [ ] 15.1.1. Convert the root manifest into a hybrid workspace while keeping the root package as the default member. See [formal-verification-methods-in-wireframe.md §Root `Cargo.toml` changes](formal-verification-methods-in-wireframe.md#root-cargotoml-changes). Success criteria: `cargo build` and `cargo test --workspace` pass with the new layout. -- [ ] 10.1.2. Add `crates/wireframe-verification` as an internal crate for - Stateright models and shared verification harnesses. Requires 10.1.1. See +- [ ] 15.1.2. Add `crates/wireframe-verification` as an internal crate for + Stateright models and shared verification harnesses. Requires 15.1.1. See [formal-verification-methods-in-wireframe.md §Why Stateright belongs in a separate verification crate](formal-verification-methods-in-wireframe.md#why-stateright-belongs-in-a-separate-verification-crate) and [§Suggested Stateright file layout](formal-verification-methods-in-wireframe.md#suggested-stateright-file-layout). Success criteria: the crate compiles, is included as a workspace member, and contains a placeholder Stateright model that passes `cargo test`. -- [ ] 10.1.3. Add pinned Kani and Verus tool metadata plus repo-local install +- [ ] 15.1.3. Add pinned Kani and Verus tool metadata plus repo-local install and run scripts. See [formal-verification-methods-in-wireframe.md §Recommended repository layout](formal-verification-methods-in-wireframe.md#recommended-repository-layout) and [§Verus should *not* live inside the main build](formal-verification-methods-in-wireframe.md#why-verus-should-not-live-inside-the-main-build). Success criteria: a contributor can run `./scripts/install-kani.sh` and `./scripts/install-verus.sh` to obtain pinned versions. -- [ ] 10.1.4. Add `make test-verification`, `make kani`, `make kani-full`, +- [ ] 15.1.4. Add `make test-verification`, `make kani`, `make kani-full`, `make verus`, `make formal-pr`, and `make formal-nightly` Makefile targets. - Requires 10.1.2 and 10.1.3. See + Requires 15.1.2 and 15.1.3. See [formal-verification-methods-in-wireframe.md §Recommended Makefile changes](formal-verification-methods-in-wireframe.md#recommended-makefile-changes). Success criteria: each target is accepted by `mbake validate Makefile` and returns exit 0 on a clean tree. -- [ ] 10.1.5. Add separate CI jobs for Stateright, Kani smoke, and Verus +- [ ] 15.1.5. Add separate CI jobs for Stateright, Kani smoke, and Verus proofs without changing the existing `build-test` coverage flow. See [formal-verification-methods-in-wireframe.md §Recommended CI changes](formal-verification-methods-in-wireframe.md#recommended-ci-changes). Success criteria: CI pipelines pass on the default branch with the new jobs visible and green. -### 10.2. Protocol contract decisions +### 15.2. Protocol contract decisions -- [ ] 10.2.1. Support a determined set of length-prefix widths (either `1`, +- [ ] 15.2.1. Support a determined set of length-prefix widths (either `1`, `2`, `4`, and `8`, or the full `1..=8` range) and enforce them in constructors, conversions, and tests; record the decision in an ADR. Requires - 10.1.1. See the formal verification guide[^fv-guide] §"What widths does + 15.1.1. See the formal verification guide[^fv-guide] §"What widths does Wireframe actually support for length prefixes?". Success criteria: an ADR records the decision, constructors enforce the chosen set, and existing tests cover rejected widths. -- [ ] 10.2.2. Treat `total_body_len` as either authoritative or advisory and +- [ ] 15.2.2. Treat `total_body_len` as either authoritative or advisory and enforce or rename it consistently across the message assembly path; record the decision in an ADR and add tests for both conforming and violating - inputs. Requires 10.1.1. See the formal verification guide[^fv-guide] §"Is + inputs. Requires 15.1.1. See the formal verification guide[^fv-guide] §"Is `total_body_len` authoritative or advisory?". Success criteria: an ADR records the decision, runtime code enforces the chosen semantics, and tests verify both conforming and violating inputs. -- [ ] 10.2.3. Publish named fairness and priority guarantees for +- [ ] 15.2.3. Publish named fairness and priority guarantees for `ConnectionActor` and encode them as model properties for Stateright checks. - Requires 10.1.1. See the formal verification guide[^fv-guide] §"What fairness + Requires 15.1.1. See the formal verification guide[^fv-guide] §"What fairness guarantee does `ConnectionActor` actually make?". Success criteria: the design document enumerates each guarantee as a named property that can be referenced by Stateright model checks. -### 10.3. Kani bounded model checks +### 15.3. Kani bounded model checks -- [ ] 10.3.1. Add smoke harnesses for supported length-prefix round-trips and +- [ ] 15.3.1. Add smoke harnesses for supported length-prefix round-trips and unsupported-width rejection in `src/frame/*`. See [formal-verification-methods-in-wireframe.md §Phase 1 smoke harnesses](formal-verification-methods-in-wireframe.md#phase-1-smoke-harnesses). - Requires 10.1.3 and 10.2.1. Success criteria: `make kani` completes with all + Requires 15.1.3 and 15.2.1. Success criteria: `make kani` completes with all harnesses verified. -- [ ] 10.3.2. Add harnesses for `FragmentSeries`, `Reassembler`, and +- [ ] 15.3.2. Add harnesses for `FragmentSeries`, `Reassembler`, and `MessageSeries` covering duplicates, gaps, completion, and oversize cleanup. See [formal-verification-methods-in-wireframe.md §Phase 1 smoke harnesses](formal-verification-methods-in-wireframe.md#phase-1-smoke-harnesses) and [§Phase 2 full harnesses](formal-verification-methods-in-wireframe.md#phase-2-full-harnesses). - Requires 10.1.3 and 10.2.2. Success criteria: `make kani-full` completes + Requires 15.1.3 and 15.2.2. Success criteria: `make kani-full` completes with all fragment and assembly harnesses verified. -- [ ] 10.3.3. Extend existing Proptest coverage for fragment round-trips and +- [ ] 15.3.3. Extend existing Proptest coverage for fragment round-trips and mixed actor action traces where Kani bounds would be too small. See [formal-verification-methods-in-wireframe.md §Second priority: `src/fragment/*`](formal-verification-methods-in-wireframe.md#second-priority-srcfragment) and [§How Proptest and Loom fit after these changes](formal-verification-methods-in-wireframe.md#how-proptest-and-loom-fit-after-these-changes). - Requires 10.3.1. Success criteria: `make test` includes the new Proptest + Requires 15.3.1. Success criteria: `make test` includes the new Proptest property tests and they pass. -### 10.4. Stateright model checks +### 15.4. Stateright model checks -- [ ] 10.4.1. Model queue arrivals, active response and multi-packet outputs, +- [ ] 15.4.1. Model queue arrivals, active response and multi-packet outputs, shutdown races, fairness state, and terminal markers in `crates/wireframe-verification`. See [formal-verification-methods-in-wireframe.md §Model scope](formal-verification-methods-in-wireframe.md#model-scope). - Requires 10.1.2 and 10.2.3. Success criteria: the model compiles and a + Requires 15.1.2 and 15.2.3. Success criteria: the model compiles and a bounded BFS run completes without panics or assertion failures. -- [ ] 10.4.2. Add a shared checker harness that separates safety properties +- [ ] 15.4.2. Add a shared checker harness that separates safety properties from reachability properties and reports both deterministically. See [formal-verification-methods-in-wireframe.md §Properties to encode](formal-verification-methods-in-wireframe.md#properties-to-encode) and [§Shared checker harness](formal-verification-methods-in-wireframe.md#shared-checker-harness). - Requires 10.4.1. Success criteria: `make test-verification` exercises the + Requires 15.4.1. Success criteria: `make test-verification` exercises the checker and reports property results. -- [ ] 10.4.3. Gate a bounded breadth-first search (BFS) model run in pull +- [ ] 15.4.3. Gate a bounded breadth-first search (BFS) model run in pull request CI and a deeper run in scheduled or manual workflows. See [formal-verification-methods-in-wireframe.md §Shared checker harness](formal-verification-methods-in-wireframe.md#shared-checker-harness) and [§Recommended CI changes](formal-verification-methods-in-wireframe.md#recommended-ci-changes). - Requires 10.1.5 and 10.4.2. Success criteria: the PR CI job completes within + Requires 15.1.5 and 15.4.2. Success criteria: the PR CI job completes within 5 minutes; the nightly job explores a deeper state space. -### 10.5. Verus proofs for message assembly +### 15.5. Verus proofs for message assembly -- [ ] 10.5.1. Enforce the chosen `total_body_len` contract in runtime code +- [ ] 15.5.1. Enforce the chosen `total_body_len` contract in runtime code before relying on proofs. See [formal-verification-methods-in-wireframe.md §"Is `total_body_len` authoritative or advisory?"](formal-verification-methods-in-wireframe.md#2-is-total_body_len-authoritative-or-advisory) and [§"What Verus should prove in Wireframe"](formal-verification-methods-in-wireframe.md#what-verus-should-prove-in-wireframe). - Requires 10.2.2. Success criteria: runtime assertions or checks enforce the + Requires 15.2.2. Success criteria: runtime assertions or checks enforce the contract, and existing tests confirm the enforcement. -- [ ] 10.5.2. Add proof-only modules under `verus/` for declared-total and +- [ ] 15.5.2. Add proof-only modules under `verus/` for declared-total and buffered-byte accounting invariants. See [formal-verification-methods-in-wireframe.md §Proof style recommendation](formal-verification-methods-in-wireframe.md#proof-style-recommendation) and [§Representative proof tree](formal-verification-methods-in-wireframe.md#representative-proof-tree). - Requires 10.1.3 and 10.5.1. Success criteria: `make verus` verifies all + Requires 15.1.3 and 15.5.1. Success criteria: `make verus` verifies all proof modules without errors. -- [ ] 10.5.3. Document proof trigger discipline and contributor expectations +- [ ] 15.5.3. Document proof trigger discipline and contributor expectations for running `make verus`. See [formal-verification-methods-in-wireframe.md §Trigger discipline](formal-verification-methods-in-wireframe.md#trigger-discipline) and [§Recommended Makefile changes](formal-verification-methods-in-wireframe.md#recommended-makefile-changes). - Requires 10.5.2. Success criteria: a contributor guide section explains + Requires 15.5.2. Success criteria: a contributor guide section explains trigger patterns, and `CONTRIBUTING.md` or the user guide references it. -## 11. Wireframe client library foundation +## 16. Wireframe client library foundation This phase delivers a first-class client runtime that mirrors the server's framing, serialization, and lifecycle layers, so both sides share the same behavioural guarantees. -### 11.1. Connection runtime +### 16.1. Connection runtime -- [x] 11.1.1. Implement `WireframeClient` and its builder so callers can +- [x] 16.1.1. Implement `WireframeClient` and its builder so callers can configure serializers, codec settings (including `max_frame_length` parity), and socket options before connecting. -- [x] 11.1.2. Integrate the existing preamble helpers so clients can emit and +- [x] 16.1.2. Integrate the existing preamble helpers so clients can emit and verify preambles before exchanging frames, with integration tests covering success and failure callbacks. -- [x] 11.1.3. Expose connection lifecycle hooks (setup, teardown, and error) +- [x] 16.1.3. Expose connection lifecycle hooks (setup, teardown, and error) that mirror the server hooks so middleware and instrumentation receive matching events. -### 11.2. Request and response pipeline +### 16.2. Request and response pipeline -- [x] 11.2.1. Provide async `send`, `receive`, and `call` APIs that encode +- [x] 16.2.1. Provide async `send`, `receive`, and `call` APIs that encode `Message` implementers, forward correlation identifiers, and deserialize typed responses using the configured serializer. -- [x] 11.2.2. Map decode and transport failures into `WireframeError` variants +- [x] 16.2.2. Map decode and transport failures into `WireframeError` variants and add integration tests that round-trip multiple message types through a sample server. -### 11.3. Streaming and multi-packet parity +### 16.3. Streaming and multi-packet parity -- [x] 11.3.1. Support `Response::Stream` and `Response::MultiPacket` on the +- [x] 16.3.1. Support `Response::Stream` and `Response::MultiPacket` on the client by propagating back-pressure, validating terminator frames, and draining push traffic without starving request-driven responses. -- [x] 11.3.2. Exercise interleaved high- and low-priority push queues to prove +- [x] 16.3.2. Exercise interleaved high- and low-priority push queues to prove fairness and rate limits remain symmetrical. -### 11.4. Documentation and examples +### 16.4. Documentation and examples -- [x] 11.4.1. Publish a runnable example where a client connects to the `echo` +- [x] 16.4.1. Publish a runnable example where a client connects to the `echo` server, issues a login request, and decodes the acknowledgement. -- [x] 11.4.2. Extend `docs/users-guide.md` and `docs/wireframe-client-design.md` +- [x] 16.4.2. Extend `docs/users-guide.md` and `docs/wireframe-client-design.md` with configuration tables, lifecycle diagrams, and troubleshooting guidance for the new APIs. -## 12. Client ergonomics and extensions +## 17. Client ergonomics and extensions This phase layers on the ergonomic features outlined in the client design document so larger deployments can adopt the library confidently. -### 12.1. Middleware and observability +### 17.1. Middleware and observability -- [x] 12.1.1. Add middleware hooks for outgoing requests and incoming frames so +- [x] 17.1.1. Add middleware hooks for outgoing requests and incoming frames so metrics, retries, and authentication tokens can be injected symmetrically with server middleware. -- [x] 12.1.2. Provide structured logging and tracing spans around connect, +- [x] 17.1.2. Provide structured logging and tracing spans around connect, send, receive, call, stream, and close lifecycle events, plus configuration for per-command timing. -### 12.2. Connection pooling and concurrency +### 17.2. Connection pooling and concurrency -- [x] 12.2.1. Implement a configurable connection pool that preserves preamble +- [x] 17.2.1. Implement a configurable connection pool that preserves preamble state, enforces in-flight request limits per socket, and recycles idle connections. -- [x] 12.2.2. Expose a `PoolHandle` API with fairness policies, so callers can +- [x] 17.2.2. Expose a `PoolHandle` API with fairness policies, so callers can multiplex many logical sessions without violating back-pressure. -### 12.3. Streaming helpers and test utilities +### 17.3. Streaming helpers and test utilities -- [x] 12.3.1. Ship helper traits or macros for consuming streaming responses +- [x] 17.3.1. Ship helper traits or macros for consuming streaming responses (for example typed iterators over `Response::Stream`) so multiplexed protocols remain ergonomic. -- [x] 12.3.2. Publish reusable test harnesses that spin up an in-process server +- [x] 17.3.2. Publish reusable test harnesses that spin up an in-process server and client pair, allowing downstream crates to verify compatibility. -### 12.4. Docs and adoption +### 17.4. Docs and adoption -- [x] 12.4.1. Update the user guide with migration advice for the pooled client +- [x] 17.4.1. Update the user guide with migration advice for the pooled client and document known limitations or out-of-scope behaviours. -- [x] 12.4.2. Add a troubleshooting section that enumerates the most common +- [x] 17.4.2. Add a troubleshooting section that enumerates the most common client misconfigurations (codec length mismatch, preamble errors, TLS issues) and how to detect them. -## 13. Advanced features and ecosystem (future) +## 18. Advanced features and ecosystem (future) This phase includes features that will broaden the library's applicability and ecosystem. -### 13.1. Alternative transports +### 18.1. Alternative transports -- [ ] 13.1.1. Abstract the transport layer to support protocols other than raw +- [ ] 18.1.1. Abstract the transport layer to support protocols other than raw TCP (e.g., WebSockets, QUIC). -### 13.2. Message versioning +### 18.2. Message versioning -- [ ] 13.2.1. Implement a formal message versioning system to allow for +- [ ] 18.2.1. Implement a formal message versioning system to allow for protocol evolution. -- [ ] 13.2.2. Ensure version negotiation can consume codec metadata without +- [ ] 18.2.2. Ensure version negotiation can consume codec metadata without leaking framing details into handlers.[^message-versioning] -### 13.3. Security +### 18.3. Security -- [ ] 13.3.1. Provide built-in middleware or guides for implementing TLS. +- [ ] 18.3.1. Provide built-in middleware or guides for implementing TLS. -## 14. Documentation and community (ongoing) +## 19. Documentation and community (ongoing) Continuous improvement of documentation and examples is essential for adoption and usability. -### 14.1. Initial documentation +### 19.1. Initial documentation -- [x] 14.1.1. Write comprehensive doc comments for all public APIs. -- [x] 14.1.2. Create a high-level `README.md` and a `docs/contents.md`. +- [x] 19.1.1. Write comprehensive doc comments for all public APIs. +- [x] 19.1.2. Create a high-level `README.md` and a `docs/contents.md`. -### 14.2. Examples +### 19.2. Examples -- [x] 14.2.1. Create a variety of examples demonstrating core features +- [x] 19.2.1. Create a variety of examples demonstrating core features (`ping_pong`, `echo`, `metadata_routing`, and `async_stream`). -### 14.3. Website and user guide +### 19.3. Website and user guide -- [ ] 14.3.1. Develop a dedicated website with a detailed user guide. -- [ ] 14.3.2. Write tutorials for common use cases. +- [ ] 19.3.1. Develop a dedicated website with a detailed user guide. +- [ ] 19.3.2. Write tutorials for common use cases. -### 14.4. API documentation +### 19.4. API documentation -- [ ] 14.4.1. Ensure all public items have clear, useful documentation +- [ ] 19.4.1. Ensure all public items have clear, useful documentation examples. -- [ ] 14.4.2. Publish documentation to `docs.rs`. +- [ ] 19.4.2. Publish documentation to `docs.rs`. [^adr-0001]: Refer to [ADR 0001](adr-001-multi-packet-streaming-response-api.md). diff --git a/docs/users-guide.md b/docs/users-guide.md index 6840e927..8691c630 100644 --- a/docs/users-guide.md +++ b/docs/users-guide.md @@ -2376,7 +2376,7 @@ channel until the client drains data. No explicit flow-control messages are required.[^50] Interleaved high- and low-priority push behaviour is validated against this -streaming path as part of roadmap item `11.3.2`. The parity suite confirms +streaming path as part of roadmap item `16.3.2`. The parity suite confirms fairness-driven low-priority progress and shared cross-priority rate limiting without changing the public `WireframeClient` interface. diff --git a/docs/wireframe-client-design.md b/docs/wireframe-client-design.md index 9c6099a7..f4615fcc 100644 --- a/docs/wireframe-client-design.md +++ b/docs/wireframe-client-design.md @@ -112,12 +112,12 @@ The parity checks cover two guarantees: - Rate limiting remains cross-priority and symmetric, so one priority cannot bypass limits imposed on another. -This validation was added for roadmap item `11.3.2` and does not introduce new +This validation was added for roadmap item `16.3.2` and does not introduce new public client API methods. ### In-process server and client pair harness -Roadmap item `12.3.2` adds a reusable harness in `wireframe_testing` that +Roadmap item `17.3.2` adds a reusable harness in `wireframe_testing` that starts a `WireframeServer` and a connected `WireframeClient` inside one test process. "In-process" means both sides run in the same process and communicate over a real loopback TCP socket, keeping compatibility checks honest while @@ -443,16 +443,16 @@ cargo run --example client_echo_login --features examples usually means peer closure or transport interruption; retry policies should treat these as network faults. -## Decision record for 11.4.1 +## Decision record for 16.4.1 - Decision: treat login acknowledgement in the echo example as the echoed login payload decoded as `LoginAck`. -- Rationale: roadmap item 11.4.1 explicitly targets the existing `echo` server. +- Rationale: roadmap item 16.4.1 explicitly targets the existing `echo` server. The echo server does not synthesize new response payloads, so decoding the echoed payload as the acknowledgement provides a runnable, typed, end-to-end demonstration without introducing server-only behaviour. -## Decision record for 11.2.1 +## Decision record for 17.2.1 - Decision: ship client pooling as a hybrid of `bb8` and Wireframe-owned slot admission. @@ -516,7 +516,7 @@ flowchart TD or true per-socket concurrent transport use, revisit this boundary and decide whether to extend or replace the slot wrapper. -## Decision record for 11.3.1 +## Decision record for 17.3.1 - Decision: ship streaming-response consumption as a trait plus adaptor stream instead of a macro-first API. @@ -529,7 +529,7 @@ flowchart TD back-pressure, and the exclusive `&mut WireframeClient` borrow all continue to behave exactly as the base streaming API documents. -## Decision record for 11.2.2 +## Decision record for 17.2.2 - Decision: `PoolHandle` represents the fairness identity of one logical session, not affinity to one physical socket. @@ -561,9 +561,9 @@ flowchart TD - Back-pressure posture: - handle fairness is additive above the existing slot-permit budget; - the scheduler never bypasses `max_in_flight_per_socket`, serialized socket - access, or idle recycle behaviour inherited from `11.2.1`. + access, or idle recycle behaviour inherited from `17.2.1`. -## Decision record for 11.4.2 +## Decision record for 17.4.2 - Decision: document only the currently reachable troubleshooting surface for client preamble failures. @@ -575,8 +575,8 @@ flowchart TD write variant. - Decision: frame TLS guidance as wrong-protocol troubleshooting, not as a built-in client capability. -- Rationale: roadmap item `12.3.1` still tracks first-party TLS guidance or - middleware, so roadmap item `11.4.2` should help users diagnose a TLS port +- Rationale: roadmap item `18.3.1` still tracks first-party TLS guidance or + middleware, so roadmap item `17.4.2` should help users diagnose a TLS port mismatch without implying native TLS transport support already exists. ## Future work diff --git a/docs/wireframe-testing-crate.md b/docs/wireframe-testing-crate.md index c7ea3cce..f777ce41 100644 --- a/docs/wireframe-testing-crate.md +++ b/docs/wireframe-testing-crate.md @@ -38,7 +38,7 @@ deterministically without external exporters. for the default codec and example codecs. - `src/multi_packet.rs` keeps the `collect_multi_packet` helper. - `src/client_pair.rs` provides the in-process server/client pair harness for - loopback integration tests (roadmap `12.3.2`). + loopback integration tests (roadmap `17.3.2`). ## Dependencies @@ -499,7 +499,7 @@ async fn example() -> TestResult<()> { ### Rationale The harness uses real loopback TCP rather than `tokio::io::duplex` because the -purpose of `12.3.2` is client/server compatibility, not in-memory app driving. +purpose of `17.3.2` is client/server compatibility, not in-memory app driving. It lives in `wireframe_testing` rather than `wireframe::testkit` to avoid widening the optional production feature surface for a purely test-facing capability. diff --git a/docs/zero-copy-frame-and-payload-migration-roadmap.md b/docs/zero-copy-frame-and-payload-migration-roadmap.md new file mode 100644 index 00000000..beacfd6f --- /dev/null +++ b/docs/zero-copy-frame-and-payload-migration-roadmap.md @@ -0,0 +1,211 @@ +# Zero-copy frame and payload migration roadmap + +This roadmap turns [`frame-vec-u8-inventory.md`](frame-vec-u8-inventory.md) +into an implementation plan for epic 284. It keeps transport-frame +substitution, payload-facing API migration, performance validation, and release +rollout as separate workstreams under one breaking-change programme so the work +can be reviewed and delivered in observable increments. + +The roadmap assumes the current direction captured by the inventory: + +- The default codec path should become `Bytes`-compatible end to end. +- Transport-frame migration and public payload API migration should not be + treated as one undifferentiated refactor. +- Downstream middleware and client hooks need an editing model that does not + force avoidable `Vec` conversions. + +## Scope and success measures + +In scope: + +- Replacing the remaining `Vec`-centric frame and payload surfaces that + block zero-copy decode or force the final outbound copy. +- Defining the public API shape for packet, middleware, serializer, and client + byte-editing surfaces. +- Benchmarking the default codec path and documenting rollout for a breaking + release. + +Out of scope: + +- Reworking unrelated body-buffering paths where `Vec` is not part of the + frame or payload hand-off contract. +- Runtime protocol negotiation. +- A permanent dual-surface API that keeps `Vec` and the zero-copy + alternative equally primary forever. + +Success is measured by the following outcomes: + +- The default length-delimited path no longer materializes a final + `Vec` copy between serialization and `FrameCodec::wrap_payload`. +- Public packet, middleware, serializer, and client byte APIs use the + selected zero-copy representation or an explicit edit-on-demand wrapper. +- Benchmark results are recorded with acceptance thresholds before the public + API flip lands. +- Release documentation includes a migration guide, a breaking-change summary, + and compatibility notes for downstream middleware, hooks, and codecs. + +## 1. Decision closure and baseline + +### 1.1. Resolve the remaining design choices + +- [ ] 1.1.1. Approve the public byte-container and editing model. See + [ADR 008](adr-008-zero-copy-public-byte-container.md). + - [ ] Confirm the stable read-only representation for packet, middleware, + serializer, and hook surfaces. + - [ ] Confirm how mutation becomes explicit without forcing every caller to + clone into `Vec`. + - [ ] Success criteria: a single approved API direction exists for + `PacketParts`, `Envelope`, `ServiceRequest`, `ServiceResponse`, + `BeforeSendHook`, and `Serializer::serialize`. +- [ ] 1.1.2. Approve the compatibility and rollout policy. See + [ADR 009](adr-009-vec-u8-migration-rollout.md). + - [ ] Decide whether the major release carries finite compatibility shims, + and document their lifetime. + - [ ] Decide what compile-time adapters or helper constructors remain for + `Vec` callers during the migration window. + - [ ] Success criteria: the release branch has an agreed downstream migration + story before code changes start landing. +- [ ] 1.1.3. Approve the transport-frame boundary for the zero-copy path. See + [ADR 010](adr-010-transport-frame-boundary-for-zero-copy.md). + - [ ] Confirm whether `ConnectionActor` remains envelope-oriented or takes on + codec-frame responsibilities. + - [ ] Confirm where protocol hooks execute when `F::Frame` differs from the + actor payload type. + - [ ] Success criteria: the actor, codec driver, and protocol hook boundary + are documented and no longer depend on ad hoc `Vec` bridges. + +### 1.2. Establish performance and migration baselines + +- [ ] 1.2.1. Define the benchmark matrix for the migration. Requires 1.1.1. + - [ ] Cover inbound decode, middleware pass-through, request-hook mutation, + and outbound encode for the default codec path. + - [ ] Include at least one protocol-native example codec to confirm the plan + is not length-delimited specific. +- [ ] 1.2.2. Record baseline measurements and acceptance thresholds. Requires + 1.2.1. + - [ ] Capture allocations, copied bytes, throughput, and latency for the + current `Vec`-bound surfaces. + - [ ] Set acceptance thresholds that require removal of the final outbound + copy on the default path and forbid more than a small agreed regression in + throughput or latency. + - [ ] Success criteria: the repository contains a benchmark note or report + that later phases can compare against directly. +- [ ] 1.2.3. Inventory downstream breakpoints and draft the migration guide + outline. Requires 1.1.2. + - [ ] List the exact public signatures that will change. + - [ ] Capture before-and-after examples for middleware, hooks, serializers, + and custom codecs. + +## 2. Internal zero-copy foundations + +### 2.1. Remove the remaining internal `Vec` bottlenecks + +- [ ] 2.1.1. Introduce the chosen internal byte abstraction for packet payloads + and outbound serialization. Requires 1.1.1. + - [ ] Update the serializer boundary so the default path can hand encoded + payload bytes to the codec without an intermediate `Vec` round-trip. + - [ ] Success criteria: the default outbound path is capable of reusing + shared byte storage end to end. +- [ ] 2.1.2. Convert `PacketParts` and `Envelope` to the new payload + representation. Requires 2.1.1. + - [ ] Preserve explicit adapter constructors or conversions defined by the + rollout policy from 1.1.2. + - [ ] Update packet reconstruction points in app routing and response + handling. +- [ ] 2.1.3. Update internal channels and helper structs that still pin + `Vec` without needing to. Requires 2.1.2. + - [ ] Cover dead-letter queue hand-offs, response forwarding, and any + zero-copy-capable internal replay buffers. + +### 2.2. Stabilize the actor and codec-driver boundary + +- [ ] 2.2.1. Implement the boundary chosen in ADR 010. Requires 1.1.3. + - [ ] Ensure the default codec path does not regain a hidden copy while + crossing between actor output and transport frame emission. + - [ ] Keep correlation, fragmentation, and protocol hook behaviour explicit. +- [ ] 2.2.2. Remove core `Vec` frame bridges that exist only because the + actor and codec layers were previously misaligned. Requires 2.2.1. + - [ ] Move `CorrelatableFrame for Vec` out of the core runtime once the + replacement path is proven, keeping it only where the rollout policy says + it still belongs. +- [ ] 2.2.3. Add allocation- and pointer-reuse regressions for the internal + boundary. Requires 2.2.1. + - [ ] Cover the default codec path. + - [ ] Cover at least one codec whose frame stores payload bytes directly. + +## 3. Public API migration + +### 3.1. Move packet and middleware APIs off owned `Vec` + +- [ ] 3.1.1. Update `PacketParts`, `Envelope`, and `Packet` consumers to the + selected byte surface. Requires 2.1.2. + - [ ] Preserve developer-facing ergonomics for payload inspection and + reconstruction. +- [ ] 3.1.2. Replace the middleware request and response wrappers with the new + editing model. Requires 1.1.1 and 2.1.2. + - [ ] Port the current `frame()`, `frame_mut()`, and `into_inner()` workflows + to their zero-copy equivalents. + - [ ] Update middleware tests to prove read-only paths stay zero-copy and + mutation paths copy only on demand. +- [ ] 3.1.3. Update server-side examples and behavioural tests that currently + teach `Vec` mutation. Requires 3.1.2. + - [ ] Cover packet rewriting, response rewriting, and correlation-preserving + middleware flows. + +### 3.2. Move client-facing byte APIs off owned `Vec` + +- [ ] 3.2.1. Replace the `before_send` hook contract with the approved + edit-on-demand surface. Requires 1.1.1. + - [ ] Keep hook behaviour observable in tracing and tests. +- [ ] 3.2.2. Update `Serializer::serialize` and its callers to the selected + byte representation. Requires 2.1.1. + - [ ] Confirm the new contract across runtime, examples, and docs. +- [ ] 3.2.3. Re-evaluate client preamble leftovers against the chosen public + byte model. Requires 1.1.1. + - [ ] Keep the existing `Vec` replay path only if the approved API + direction still makes that the least surprising compatibility boundary. + - [ ] Success criteria: the preamble policy is either explicitly retained and + documented, or migrated consistently with the rest of the client surface. + +## 4. Validation, ecosystem updates, and documentation + +### 4.1. Remove stale teaching and compatibility drift + +- [ ] 4.1.1. Update `wireframe_testing`, examples, and design documents to stop + treating `Vec` as the default frame shape. Requires 3.1.3 and 3.2.2. + - [ ] Leave only the compatibility examples that the rollout policy still + promises. +- [ ] 4.1.2. Publish the migration guide and breaking-change summary. Requires + 1.2.3 and 3.2.3. + - [ ] Include before-and-after code for middleware, hooks, serializers, and + custom codecs. + - [ ] Include a short decision summary for why the public surface changed. +- [ ] 4.1.3. Close or supersede ADRs 008 through 010 once implementation and + documentation match the approved design. Requires 4.1.2. + +### 4.2. Validate the final performance and correctness story + +- [ ] 4.2.1. Run the benchmark suite and compare against the thresholds from + 1.2.2. Requires 3.2.2. + - [ ] Record the results in a committed benchmark note or release document. +- [ ] 4.2.2. Run targeted downstream canaries. Requires 4.1.2. + - [ ] Compile and test representative middleware, hook, and custom codec + examples against the release candidate. + - [ ] Success criteria: downstream breakage is either fixed or called out in + the migration guide with a documented workaround. + +## 5. Release rollout + +### 5.1. Prepare the breaking release + +- [ ] 5.1.1. Finalize the versioning plan for the public API break. Requires + 1.1.2 and 4.2.2. + - [ ] Decide whether the change ships as the next major release or a pre-1.0 + point release under the project's semver policy. +- [ ] 5.1.2. Publish release notes, changelog entries, and upgrade guidance. + Requires 4.1.2. + - [ ] Call out the removed `Vec` contracts, the new zero-copy defaults, + and any finite compatibility helpers. +- [ ] 5.1.3. Capture post-release follow-up work. Requires 5.1.2. + - [ ] List any compatibility helpers scheduled for later removal. + - [ ] List any deferred areas that intentionally remained on `Vec`. diff --git a/src/frame/conversion.rs b/src/frame/conversion.rs index a0c96467..85ef32dd 100644 --- a/src/frame/conversion.rs +++ b/src/frame/conversion.rs @@ -29,6 +29,33 @@ fn checked_prefix_cast>(len: usize) -> io::Result { T::try_from(len).map_err(|_| io::Error::new(io::ErrorKind::InvalidInput, ERR_FRAME_TOO_LARGE)) } +/// Copies `prefix` into an 8-byte buffer, right-aligned for big-endian or +/// left-aligned for little-endian. +/// +/// The caller guarantees that `prefix` is `1`, `2`, `4`, or `8` bytes, so the +/// slice operations are infallible. +fn fill_prefix_buffer(prefix: &[u8], endianness: Endianness) -> [u8; 8] { + let mut buf = [0u8; 8]; + let size = prefix.len(); + debug_assert!( + matches!(size, 1 | 2 | 4 | 8), + "prefix must be 1, 2, 4, or 8 bytes" + ); + let offset = match endianness { + Endianness::Big => 8 - size, + Endianness::Little => 0, + }; + let dst = buf.get_mut(offset..offset + size); + debug_assert!( + dst.is_some(), + "validated prefix length always fits within the buffer" + ); + if let Some(dst) = dst { + dst.copy_from_slice(prefix); + } + buf +} + /// Converts a byte slice into a `u64` according to `size` and `endianness`. /// /// Only prefix sizes of `1`, `2`, `4`, or `8` bytes are supported. `bytes` must @@ -51,35 +78,10 @@ pub fn bytes_to_u64(bytes: &[u8], size: usize, endianness: Endianness) -> io::Re )); } - let mut buf = [0u8; 8]; - // NOTE: size is validated above; this is a defensive fallback. let prefix = bytes .get(..size) .ok_or_else(|| io::Error::new(io::ErrorKind::UnexpectedEof, ERR_INCOMPLETE_PREFIX))?; - match endianness { - Endianness::Big => { - if let Some(dst) = buf.get_mut(8 - size..) { - dst.copy_from_slice(prefix); - } else { - debug_assert!(false, "validated size should fit into prefix buffer"); - return Err(io::Error::new( - io::ErrorKind::InvalidInput, - ERR_UNSUPPORTED_PREFIX, - )); - } - } - Endianness::Little => { - if let Some(dst) = buf.get_mut(..size) { - dst.copy_from_slice(prefix); - } else { - debug_assert!(false, "validated size should fit into prefix buffer"); - return Err(io::Error::new( - io::ErrorKind::InvalidInput, - ERR_UNSUPPORTED_PREFIX, - )); - } - } - } + let buf = fill_prefix_buffer(prefix, endianness); // Wire prefix declares its endianness; normalising into an 8-byte buffer and // using explicit conversion helpers keeps decoding deterministic on any host diff --git a/src/message_assembler/series.rs b/src/message_assembler/series.rs index 3f2c55c7..527b5006 100644 --- a/src/message_assembler/series.rs +++ b/src/message_assembler/series.rs @@ -197,51 +197,74 @@ impl MessageSeries { Ok(MessageSeriesStatus::Incomplete) } - fn validate_and_advance_sequence( + /// Starts explicit sequence tracking when the first numbered continuation + /// arrives after an untracked series. + /// + /// The helper records the next expected sequence number and reports + /// overflow only when additional frames are still required. + fn handle_untracked_first_sequence( &mut self, incoming: FrameSequence, is_last: bool, ) -> Result<(), MessageSeriesError> { - match self.sequence_tracking { - SequenceTracking::Untracked => { - // First continuation with a sequence number; start tracking - self.sequence_tracking = SequenceTracking::Tracked; - self.next_sequence = incoming.checked_increment(); - // Overflow is only an error if more frames are expected - if self.next_sequence.is_none() && !is_last { - return Err(MessageSeriesError::SequenceOverflow { last: incoming }); - } - Ok(()) - } - SequenceTracking::Tracked => { - let expected = self - .next_sequence - .ok_or(MessageSeriesError::SequenceOverflow { last: incoming })?; + // First continuation with a sequence number; start tracking. + self.sequence_tracking = SequenceTracking::Tracked; + self.next_sequence = incoming.checked_increment(); + // Overflow is only an error if more frames are expected. + if self.next_sequence.is_none() && !is_last { + return Err(MessageSeriesError::SequenceOverflow { last: incoming }); + } + Ok(()) + } + + /// Validates a numbered continuation against the tracked expectation and + /// advances the series state when the sequence is correct. + /// + /// The helper rejects duplicates, gaps, and unexpected overflow so + /// `validate_and_advance_sequence` can delegate the tracked path without + /// re-embedding the decision tree. + fn advance_tracked_sequence( + &mut self, + incoming: FrameSequence, + is_last: bool, + ) -> Result<(), MessageSeriesError> { + let expected = self + .next_sequence + .ok_or(MessageSeriesError::SequenceOverflow { last: incoming })?; + + if incoming.0 < expected.0 { + // Duplicate: sequence already seen. + return Err(MessageSeriesError::DuplicateFrame { + key: self.message_key, + sequence: incoming, + }); + } - if incoming.0 < expected.0 { - // Duplicate: sequence already seen - return Err(MessageSeriesError::DuplicateFrame { - key: self.message_key, - sequence: incoming, - }); - } + if incoming != expected { + // Out of order or gap. + return Err(MessageSeriesError::SequenceMismatch { + expected, + found: incoming, + }); + } - if incoming != expected { - // Out of order or gap - return Err(MessageSeriesError::SequenceMismatch { - expected, - found: incoming, - }); - } + // Advance expected sequence. + self.next_sequence = incoming.checked_increment(); + // Overflow is only an error if more frames are expected. + if self.next_sequence.is_none() && !is_last { + return Err(MessageSeriesError::SequenceOverflow { last: incoming }); + } + Ok(()) + } - // Advance expected sequence - self.next_sequence = incoming.checked_increment(); - // Overflow is only an error if more frames are expected - if self.next_sequence.is_none() && !is_last { - return Err(MessageSeriesError::SequenceOverflow { last: incoming }); - } - Ok(()) - } + fn validate_and_advance_sequence( + &mut self, + incoming: FrameSequence, + is_last: bool, + ) -> Result<(), MessageSeriesError> { + match self.sequence_tracking { + SequenceTracking::Untracked => self.handle_untracked_first_sequence(incoming, is_last), + SequenceTracking::Tracked => self.advance_tracked_sequence(incoming, is_last), } } } diff --git a/tests/fixtures/memory_budget_hard_cap.rs b/tests/fixtures/memory_budget_hard_cap.rs index b85deea4..7237400e 100644 --- a/tests/fixtures/memory_budget_hard_cap.rs +++ b/tests/fixtures/memory_budget_hard_cap.rs @@ -221,33 +221,38 @@ impl MemoryBudgetHardCapWorld { self.send_payload(payload) } + /// Validates that a server-side `io::Error` represents a budget-enforcement + /// abort: confirms `InvalidData` kind, records the error string, drains any + /// late payloads, and rejects non-empty payload observations. + fn verify_abort_outcome(&mut self, error: &std::io::Error) -> TestResult { + if error.kind() != std::io::ErrorKind::InvalidData { + return Err(format!( + "expected budget-related InvalidData error, got {:?}: {error}", + error.kind(), + ) + .into()); + } + self.connection_error = Some(error.to_string()); + // Drain after the server task has finished so payloads emitted + // during teardown are caught. + self.drain_ready_payloads()?; + if !self.observed_payloads.is_empty() { + return Err(format!( + "expected no payloads before abort, but received: {:?}", + self.observed_payloads + ) + .into()); + } + Ok(()) + } + /// Assert that the connection terminated with an `InvalidData` error /// (the kind produced by budget enforcement) and no payloads were delivered. pub fn assert_connection_aborted(&mut self) -> TestResult { self.spin_runtime()?; match self.join_server()? { Ok(()) => Err("expected connection to abort, but it completed successfully".into()), - Err(error) => { - if error.kind() != std::io::ErrorKind::InvalidData { - return Err(format!( - "expected budget-related InvalidData error, got {:?}: {error}", - error.kind(), - ) - .into()); - } - self.connection_error = Some(error.to_string()); - // Drain after the server task has finished so payloads emitted - // during teardown are caught. - self.drain_ready_payloads()?; - if !self.observed_payloads.is_empty() { - return Err(format!( - "expected no payloads before abort, but received: {:?}", - self.observed_payloads - ) - .into()); - } - Ok(()) - } + Err(error) => self.verify_abort_outcome(&error), } }