From c7711f8d78df4fd09613dc88140e15886a04e72d Mon Sep 17 00:00:00 2001 From: Leynos Date: Wed, 11 Jun 2025 13:13:01 +0100 Subject: [PATCH 1/2] Restore markdown rules --- AGENTS.md | 157 ++- docs/roadmap.md | 41 + docs/rust-binary-router-library-design.md | 1260 ++++++++++++++++----- 3 files changed, 1098 insertions(+), 360 deletions(-) create mode 100644 docs/roadmap.md diff --git a/AGENTS.md b/AGENTS.md index ee74597c..e14cc30c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -2,72 +2,123 @@ ## Code Style and Structure -* **Code is for humans.** Write your code with clarity and empathy—assume a tired teammate will need to debug it at 3 a.m. -* **Comment *why*, not *what*.** Explain assumptions, edge cases, trade-offs, or complexity. Don't echo the obvious. -* **Clarity over cleverness.** Be concise, but favour explicit over terse or obscure idioms. Prefer code that's easy to follow. -* **Use functions and composition.** Avoid repetition by extracting reusable logic. Prefer generators or comprehensions, and declarative code to imperative repetition when readable. -* **Small, meaningful functions.** Functions must be small, clear in purpose, single responsibility, and obey command/query segregation. -* **Clear commit messages.** Commit messages should be descriptive, explaining what was changed and why. -* **Name things precisely.** Use clear, descriptive variable and function names. For booleans, prefer names with `is`, `has`, or `should`. -* **Structure logically.** Each file should encapsulate a coherent module. Group related code (e.g., models + utilities + fixtures) close together. -* **Group by feature, not layer.** Colocate views, logic, fixtures, and helpers related to a domain concept rather than splitting by type. +- **Code is for humans.** Write your code with clarity and empathy—assume a + tired teammate will need to debug it at 3 a.m. +- **Comment *why*, not *what*.** Explain assumptions, edge cases, trade-offs, or + complexity. Don't echo the obvious. +- **Clarity over cleverness.** Be concise, but favour explicit over terse or + obscure idioms. Prefer code that's easy to follow. +- **Use functions and composition.** Avoid repetition by extracting reusable + logic. Prefer generators or comprehensions, and declarative code to imperative + repetition when readable. +- **Small, meaningful functions.** Functions must be small, clear in purpose, + single responsibility, and obey command/query segregation. +- **Clear commit messages.** Commit messages should be descriptive, explaining + what was changed and why. +- **Name things precisely.** Use clear, descriptive variable and function names. + For booleans, prefer names with `is`, `has`, or `should`. +- **Structure logically.** Each file should encapsulate a coherent module. Group + related code (e.g., models + utilities + fixtures) close together. +- **Group by feature, not layer.** Colocate views, logic, fixtures, and helpers + related to a domain concept rather than splitting by type. ## Documentation Maintenance -* **Reference:** Use the markdown files within the `docs/` directory as a knowledge base and source of truth for project requirements, dependency choices, and architectural decisions. -* **Update:** When new decisions are made, requirements change, libraries are added/removed, or architectural patterns evolve, **proactively update** the relevant file(s) in the `docs/` directory to reflect the latest state. Ensure the documentation remains accurate and current. +- **Reference:** Use the markdown files within the `docs/` directory as a + knowledge base and source of truth for project requirements, dependency + choices, and architectural decisions. +- **Update:** When new decisions are made, requirements change, libraries are + added/removed, or architectural patterns evolve, **proactively update** the + relevant file(s) in the `docs/` directory to reflect the latest state. Ensure + the documentation remains accurate and current. ## Change Quality & Committing -* **Atomicity:** Aim for small, focused, atomic changes. Each change (and subsequent commit) should represent a single logical unit of work. -* **Quality Gates:** Before considering a change complete or proposing a commit, ensure it meets the following criteria: - * Passes all relevant unit and behavioral tests according to the guidelines above. - * Passes lint checks - * Adheres to formatting standards tested using a formatting validator. -* **Committing:** - * Only changes that meet all the quality gates above should be committed. - * Write clear, descriptive commit messages summarizing the change, following these formatting guidelines: - * **Imperative Mood:** Use the imperative mood in the subject line (e.g., "Fix bug", "Add feature" instead of "Fixed bug", "Added feature"). - * **Subject Line:** The first line should be a concise summary of the change (ideally 50 characters or less). - * **Body:** Separate the subject from the body with a blank line. Subsequent lines should explain the *what* and *why* of the change in more detail, including rationale, goals, and scope. Wrap the body at 72 characters. - * **Formatting:** Use Markdown for any formatted text (like bullet points or code snippets) within the commit message body. - * Do not commit changes that fail any of the quality gates. +- **Atomicity:** Aim for small, focused, atomic changes. Each change (and + subsequent commit) should represent a single logical unit of work. +- **Quality Gates:** Before considering a change complete or proposing a commit, + ensure it meets the following criteria: + - Passes all relevant unit and behavioral tests according to the guidelines + above. + - Passes lint checks + - Adheres to formatting standards tested using a formatting validator. +- **Committing:** + - Only changes that meet all the quality gates above should be committed. + - Write clear, descriptive commit messages summarizing the change, following + these formatting guidelines: + - **Imperative Mood:** Use the imperative mood in the subject line (e.g., + "Fix bug", "Add feature" instead of "Fixed bug", "Added feature"). + - **Subject Line:** The first line should be a concise summary of the change + (ideally 50 characters or less). + - **Body:** Separate the subject from the body with a blank line. Subsequent + lines should explain the *what* and *why* of the change in more detail, + including rationale, goals, and scope. Wrap the body at 72 characters. + - **Formatting:** Use Markdown for any formatted text (like bullet points or + code snippets) within the commit message body. + - Do not commit changes that fail any of the quality gates. ## Refactoring Heuristics & Workflow -* **Recognizing Refactoring Needs:** Regularly assess the codebase for potential refactoring opportunities. Consider refactoring when you observe: - * **Long Methods/Functions:** Functions or methods that are excessively long or try to do too many things. - * **Duplicated Code:** Identical or very similar code blocks appearing in multiple places. - * **Complex Conditionals:** Deeply nested or overly complex `if`/`else` or `switch` statements (high cyclomatic complexity). - * **Large Code Blocks for Single Values:** Significant chunks of logic dedicated solely to calculating or deriving a single value. - * **Primitive Obsession / Data Clumps:** Groups of simple variables (strings, numbers, booleans) that are frequently passed around together, often indicating a missing class or object structure. - * **Excessive Parameters:** Functions or methods requiring a very long list of parameters. - * **Feature Envy:** Methods that seem more interested in the data of another class/object than their own. - * **Shotgun Surgery:** A single change requiring small modifications in many different classes or functions. -* **Post-Commit Review:** After committing a functional change or bug fix (that meets all quality gates), review the changed code and surrounding areas using the heuristics above. -* **Separate Atomic Refactors:** If refactoring is deemed necessary: - * Perform the refactoring as a **separate, atomic commit** *after* the functional change commit. - * Ensure the refactoring adheres to the testing guidelines (behavioral tests pass before and after, unit tests added for new units). - * Ensure the refactoring commit itself passes all quality gates. +- **Recognizing Refactoring Needs:** Regularly assess the codebase for potential + refactoring opportunities. Consider refactoring when you observe: + - **Long Methods/Functions:** Functions or methods that are excessively long + or try to do too many things. + - **Duplicated Code:** Identical or very similar code blocks appearing in + multiple places. + - **Complex Conditionals:** Deeply nested or overly complex `if`/`else` or + `switch` statements (high cyclomatic complexity). + - **Large Code Blocks for Single Values:** Significant chunks of logic + dedicated solely to calculating or deriving a single value. + - **Primitive Obsession / Data Clumps:** Groups of simple variables (strings, + numbers, booleans) that are frequently passed around together, often + indicating a missing class or object structure. + - **Excessive Parameters:** Functions or methods requiring a very long list of + parameters. + - **Feature Envy:** Methods that seem more interested in the data of another + class/object than their own. + - **Shotgun Surgery:** A single change requiring small modifications in many + different classes or functions. +- **Post-Commit Review:** After committing a functional change or bug fix (that + meets all quality gates), review the changed code and surrounding areas using + the heuristics above. +- **Separate Atomic Refactors:** If refactoring is deemed necessary: + - Perform the refactoring as a **separate, atomic commit** *after* the + functional change commit. + - Ensure the refactoring adheres to the testing guidelines (behavioral tests + pass before and after, unit tests added for new units). + - Ensure the refactoring commit itself passes all quality gates. ## Rust Specific Guidance -This repository is written in Rust and uses Cargo for building and dependency management. Contributors should follow these best practices when working on the project: -* Run `cargo fmt`, `cargo clippy -- -D warnings`, and `cargo test` before committing. -* Clippy warnings MUST be disallowed. -* Where a function is too long, extract meaningfully named helper functions adhering to separation of concerns and CQRS. -* Where a function has too many parameters, group related parameters in meaningfully named structs. -* Where a function is returning a large error consider using `Arc` to reduce the amount of data returned. -* Write unit and behavioural tests for new functionality. Run both before and after making any change. -* Document public APIs using Rustdoc comments (`///`) so documentation can be generated with cargo doc. -* Prefer immutable data and avoid unnecessary `mut` bindings. -* Handle errors with the `Result` type instead of panicking where feasible. -* Use explicit version ranges in `Cargo.toml` and keep dependencies up-to-date. -* Avoid `unsafe` code unless absolutely necessary and document any usage clearly. +This repository is written in Rust and uses Cargo for building and dependency +management. Contributors should follow these best practices when working on the +project: + +- Run `cargo fmt`, `cargo clippy -- -D warnings`, and `cargo test` before + committing. +- Clippy warnings MUST be disallowed. +- Where a function is too long, extract meaningfully named helper functions + adhering to separation of concerns and CQRS. +- Where a function has too many parameters, group related parameters in + meaningfully named structs. +- Where a function is returning a large error consider using `Arc` to reduce the + amount of data returned. +- Write unit and behavioural tests for new functionality. Run both before and + after making any change. +- Document public APIs using Rustdoc comments (`///`) so documentation can be + generated with cargo doc. +- Prefer immutable data and avoid unnecessary `mut` bindings. +- Handle errors with the `Result` type instead of panicking where feasible. +- Use explicit version ranges in `Cargo.toml` and keep dependencies up-to-date. +- Avoid `unsafe` code unless absolutely necessary and document any usage + clearly. ## Markdown Guidance -* Validate Markdown files using `markdownlint`. -* Validate Markdown Mermaid diagrams using `nixie`. +- Validate Markdown files using `markdownlint`. +- Validate Markdown Mermaid diagrams using `nixie`. + +### Key Takeaway -**These practices will help maintain a high-quality codebase and make collaboration easier** +These practices help maintain a high-quality codebase and facilitate +collaboration. diff --git a/docs/roadmap.md b/docs/roadmap.md new file mode 100644 index 00000000..bc0d5454 --- /dev/null +++ b/docs/roadmap.md @@ -0,0 +1,41 @@ +# Roadmap Summary + +This document distills the key development goals from +[rust-binary-router-library-design.md](rust-binary-router-library-design.md) +after formatting. Line numbers below refer to that file. + +## 1. Core Library Foundations + +- Lines 316-345 outline the layered architecture comprising the Transport Layer + Adapter, Framing Layer, Serialization engine, Routing engine, Handler + invocation, and Middleware chain. +- Implement derive macros or wrappers for message serialization (lines 329-333). +- Build the Actix-inspired API around `WireframeApp` and `WireframeServer` as + described in lines 586-676. + +## 2. Middleware and Extractors + +- Develop a minimal middleware system and extractor traits for payloads, + connection metadata, and shared state. + +## 3. Initial Examples and Documentation + +- Provide examples demonstrating routing, serialization, and middleware. + Document configuration and usage reflecting the API design section. + +## 4. Extended Features + +- Add UDP and other transport implementations (lines 1366-1379). +- Develop built-in `FrameProcessor` variants (lines 1381-1389). +- Address schema evolution and versioning strategies (lines 1394-1409). +- Investigate multiplexing and flow control mechanisms (lines 1411-1422). + +## 5. Developer Tooling + +- Create a CLI for protocol scaffolding and testing (lines 1424-1429). +- Improve debugging support and expand documentation (lines 1430-1435). + +## 6. Community Engagement and Integration + +- Collaborate with `wire-rs` for trait derivation and future enhancements (lines + 1437-1442). diff --git a/docs/rust-binary-router-library-design.md b/docs/rust-binary-router-library-design.md index 77edb70d..40085bbb 100644 --- a/docs/rust-binary-router-library-design.md +++ b/docs/rust-binary-router-library-design.md @@ -2,103 +2,300 @@ ## 1. Introduction -The development of applications requiring communication over custom binary wire protocols often involves significant complexities in source code. These complexities arise from manual data serialization and deserialization, intricate framing logic, stateful connection management, and the imperative dispatch of messages to appropriate handlers. Such low-level concerns can obscure the core application logic, increase development time, and introduce a higher propensity for errors. The Rust programming language, with its emphasis on safety, performance, and powerful compile-time abstractions, offers a promising foundation for mitigating these challenges.1 - -This report outlines the design of "wireframe," a novel Rust library aimed at substantially reducing source code complexity when building applications that handle arbitrary frame-based binary wire protocols. The design draws inspiration from the ergonomic API of Actix Web 4, a popular Rust web framework known for its intuitive routing, data extraction, and middleware systems.4 "wireframe" intends to adapt these successful patterns to the domain of binary protocols. - -A key aspect of the proposed design is the utilization of `wire-rs` 6 for message serialization and deserialization, contingent upon its ability to support or be augmented with derivable `Encode` and `Decode` traits. This, combined with a layered architecture and high-level abstractions, seeks to provide developers with a more declarative and less error-prone environment for building robust network services. +The development of applications requiring communication over custom binary wire +protocols often involves significant complexities in source code. These +complexities arise from manual data serialization and deserialization, intricate +framing logic, stateful connection management, and the imperative dispatch of +messages to appropriate handlers. Such low-level concerns can obscure the core +application logic, increase development time, and introduce a higher propensity +for errors. The Rust programming language, with its emphasis on safety, +performance, and powerful compile-time abstractions, offers a promising +foundation for mitigating these challenges.1 + +This report outlines the design of "wireframe," a novel Rust library aimed at +substantially reducing source code complexity when building applications that +handle arbitrary frame-based binary wire protocols. The design draws inspiration +from the ergonomic API of Actix Web 4, a popular Rust web framework known for +its intuitive routing, data extraction, and middleware systems.4 "wireframe" +intends to adapt these successful patterns to the domain of binary protocols. + +A key aspect of the proposed design is the utilization of `wire-rs` 6 for +message serialization and deserialization, contingent upon its ability to +support or be augmented with derivable `Encode` and `Decode` traits. This, +combined with a layered architecture and high-level abstractions, seeks to +provide developers with a more declarative and less error-prone environment for +building robust network services. ## 2. Methodology The design of the "wireframe" library is predicated on a multi-faceted approach: -1. **Literature Survey**: A comprehensive review of existing Rust libraries and patterns for binary serialization, protocol implementation, and network programming was conducted. This included an examination of serialization libraries such as `wire-rs`, `bincode`, `postcard`, and `bin-proto`, as well as protocol frameworks like `protocol` and `tarpc`. The Tokio ecosystem, a foundational element for asynchronous networking in Rust, was also considered. -2. **API Analysis**: The Application Programming Interface (API) of Actix Web was studied in detail, focusing on its mechanisms for routing, request data extraction, middleware, and application state management. The objective was to identify design principles that contribute to its developer-friendliness and could be effectively translated to the context of binary protocols. -3. **Comparative Synthesis**: Learnings from the literature survey and API analysis were synthesized to inform the core design decisions for "wireframe". This involved identifying common strategies for complexity reduction in Rust, such as the use of derive macros and trait-based abstractions, and evaluating how Actix Web's patterns could be adapted. -4. **Iterative Design**: Based on these findings, a conceptual design for "wireframe" was developed, outlining its architecture, core components, and API. This design prioritizes the user query's central goal: reducing source code complexity for arbitrary frame-based binary protocols. - -The inaccessibility of the `leynos/mxd` repository 7, mentioned in the user query as an example of a system whose complexity "wireframe" aims to reduce, means that a direct comparative analysis is not possible. Therefore, the focus of this report is on demonstrating complexity reduction through the inherent advantages of the proposed abstractions and API design, contrasting them with common manual implementation pitfalls. +1. **Literature Survey**: A comprehensive review of existing Rust libraries and + patterns for binary serialization, protocol implementation, and network + programming was conducted. This included an examination of serialization + libraries such as `wire-rs`, `bincode`, `postcard`, and `bin-proto`, as well + as protocol frameworks like `protocol` and `tarpc`. The Tokio ecosystem, a + foundational element for asynchronous networking in Rust, was also + considered. +1. **API Analysis**: The Application Programming Interface (API) of Actix Web + was studied in detail, focusing on its mechanisms for routing, request data + extraction, middleware, and application state management. The objective was + to identify design principles that contribute to its developer-friendliness + and could be effectively translated to the context of binary protocols. +1. **Comparative Synthesis**: Learnings from the literature survey and API + analysis were synthesized to inform the core design decisions for + "wireframe". This involved identifying common strategies for complexity + reduction in Rust, such as the use of derive macros and trait-based + abstractions, and evaluating how Actix Web's patterns could be adapted. +1. **Iterative Design**: Based on these findings, a conceptual design for + "wireframe" was developed, outlining its architecture, core components, and + API. This design prioritizes the user query's central goal: reducing source + code complexity for arbitrary frame-based binary protocols. + +The inaccessibility of the `leynos/mxd` repository 7, mentioned in the user +query as an example of a system whose complexity "wireframe" aims to reduce, +means that a direct comparative analysis is not possible. Therefore, the focus +of this report is on demonstrating complexity reduction through the inherent +advantages of the proposed abstractions and API design, contrasting them with +common manual implementation pitfalls. ## 3. Literature Survey and Precedent Analysis -A survey of the existing Rust ecosystem provides valuable context for the design of "wireframe," highlighting established patterns for serialization, protocol implementation, and API ergonomics. +A survey of the existing Rust ecosystem provides valuable context for the design +of "wireframe," highlighting established patterns for serialization, protocol +implementation, and API ergonomics. ### 3.1. Binary Serialization Libraries in Rust -Effective and efficient binary serialization is fundamental to any library dealing with wire protocols. Several Rust crates offer solutions in this space, each with distinct characteristics. - -- `wire-rs`: The user query specifically suggests considering `wire-rs`.6 This library provides an extensible interface for converting data to and from wire protocols, supporting non-contiguous buffers and `no_std` environments. It features `WireReader` and `WireWriter` for manual data reading and writing, with explicit control over endianness.6 However, the available information does not clearly indicate the presence or nature of derivable `Encode` and `Decode` traits for automatic (de)serialization of complex types.6 The ability to automatically generate (de)serialization logic via derive macros is crucial for achieving "wireframe's" goal of reducing source code complexity. If such derive macros are not a core feature of `wire-rs`, "wireframe" would need to either contribute them, provide its own wrapper traits that enable derivation while using `wire-rs` internally, or consider alternative serialization libraries. - -- `bincode`: `bincode` is a widely used binary serialization library that integrates well with Serde.8 It offers high performance and configurable options for endianness and integer encoding.10 `bincode` 2.0 makes Serde an optional dependency and provides its own `Encode`/`Decode` traits that can be derived.11 Its flexibility and performance make it a strong candidate if `wire-rs` proves unsuitable for derivable (de)serialization. The choice between fixed-width integers and Varint encoding offers trade-offs in terms of size and speed.10 - -- `postcard`: `postcard` is another Serde-compatible library, specifically designed for `no_std` and embedded environments, prioritizing resource efficiency (memory, code size).9 It uses LEB128 for variable-length integer encoding and has a stable, documented wire format.10 `postcard` is noted for being less configurable but potentially easier to use than `bincode` due to fewer options.10 Benchmarks suggest `postcard` can produce smaller outputs than `bincode` in many cases, sometimes at a slight performance cost.9 Its focus on minimalism and a simple specification is appealing. - -- `bin-proto`: `bin-proto` offers simple and fast structured bit-level binary encoding and decoding.14 It provides `BitEncode` and `BitDecode` traits, along with custom derive macros (e.g., `#`) for ease of use.14 It allows fine-grained control over bit-level layout, such as specifying the number of bits for fields and enum discriminants.14 `bin-proto` also supports context-aware parsing, where deserialization logic can depend on external context, a feature potentially valuable for complex protocols.14 - -The common thread among the more ergonomic libraries (`bincode`, `postcard`, `bin-proto`) is the provision of derive macros. This significantly simplifies the process of making custom data structures serializable, aligning directly with "wireframe's" complexity reduction objective. The choice of serialization library for "wireframe" will heavily depend on the feasibility of integrating `wire-rs` with such derivable traits or the need to adopt an alternative like `bincode` or `postcard` that already offers this crucial feature. +Effective and efficient binary serialization is fundamental to any library +dealing with wire protocols. Several Rust crates offer solutions in this space, +each with distinct characteristics. + +- `wire-rs`: The user query specifically suggests considering `wire-rs`.6 This + library provides an extensible interface for converting data to and from wire + protocols, supporting non-contiguous buffers and `no_std` environments. It + features `WireReader` and `WireWriter` for manual data reading and writing, + with explicit control over endianness.6 However, the available information + does not clearly indicate the presence or nature of derivable `Encode` and + `Decode` traits for automatic (de)serialization of complex types.6 The ability + to automatically generate (de)serialization logic via derive macros is crucial + for achieving "wireframe's" goal of reducing source code complexity. If such + derive macros are not a core feature of `wire-rs`, "wireframe" would need to + either contribute them, provide its own wrapper traits that enable derivation + while using `wire-rs` internally, or consider alternative serialization + libraries. + +- `bincode`: `bincode` is a widely used binary serialization library that + integrates well with Serde.8 It offers high performance and configurable + options for endianness and integer encoding.10 `bincode` 2.0 makes Serde an + optional dependency and provides its own `Encode`/`Decode` traits that can be + derived.11 Its flexibility and performance make it a strong candidate if + `wire-rs` proves unsuitable for derivable (de)serialization. The choice + between fixed-width integers and Varint encoding offers trade-offs in terms of + size and speed.10 + +- `postcard`: `postcard` is another Serde-compatible library, specifically + designed for `no_std` and embedded environments, prioritizing resource + efficiency (memory, code size).9 It uses LEB128 for variable-length integer + encoding and has a stable, documented wire format.10 `postcard` is noted for + being less configurable but potentially easier to use than `bincode` due to + fewer options.10 Benchmarks suggest `postcard` can produce smaller outputs + than `bincode` in many cases, sometimes at a slight performance cost.9 Its + focus on minimalism and a simple specification is appealing. + +- `bin-proto`: `bin-proto` offers simple and fast structured bit-level binary + encoding and decoding.14 It provides `BitEncode` and `BitDecode` traits, along + with custom derive macros (e.g., `#`) for ease of use.14 It allows + fine-grained control over bit-level layout, such as specifying the number of + bits for fields and enum discriminants.14 `bin-proto` also supports + context-aware parsing, where deserialization logic can depend on external + context, a feature potentially valuable for complex protocols.14 + +The common thread among the more ergonomic libraries (`bincode`, `postcard`, +`bin-proto`) is the provision of derive macros. This significantly simplifies +the process of making custom data structures serializable, aligning directly +with "wireframe's" complexity reduction objective. The choice of serialization +library for "wireframe" will heavily depend on the feasibility of integrating +`wire-rs` with such derivable traits or the need to adopt an alternative like +`bincode` or `postcard` that already offers this crucial feature. ### 3.2. Existing Rust Libraries for Custom Network Protocols -Several Rust libraries provide frameworks or utilities for implementing custom network protocols, offering insights into effective abstractions. - -- `protocol` **(dylanmckay/protocol)**: This crate aims to simplify protocol definitions in Rust by using a custom derive `#[derive(protocol::Protocol)]`.16 It allows structured data to be sent and received from any I/O stream, with built-in support for TCP and UDP. Any type implementing its `Parcel` trait can be serialized. The derive macro handles the implementation for custom types, though it requires types to also implement `Clone`, `Debug`, and `PartialEq`, which might be overly restrictive for some use cases.16 The library also supports middleware for transforming sent/received data. This library exemplifies the use of derive macros to reduce boilerplate in protocol definitions, a core strategy for "wireframe". - -- `message-io`: This library provides abstractions for message-based network communication over various transports like TCP, UDP, and WebSockets.17 Notably, it offers `FramedTcp`, which prefixes messages with their size, managing data as packets rather than a raw stream.17 This distinction between connection-oriented and packet-based transports, and the provision of framing solutions, is relevant to "wireframe's" design for handling frame-based protocols. - -- `tokio-util::codec`: Part of the extensive Tokio ecosystem 18, `tokio-util::codec` provides `Encoder` and `Decoder` traits. These traits are fundamental for implementing framing logic, allowing the conversion of a raw byte stream into a sequence of frames, and vice-versa. While not a full routing solution, these traits are essential low-level building blocks upon which "wireframe" would likely build its framing layer. - -- `tarpc`: `tarpc` is an RPC framework for Rust focusing on ease of use.20 Although designed for RPC, its approach of defining service schemas directly in Rust code (using the `#[tarpc::service]` attribute to generate service traits and client/server boilerplate) is an interesting parallel to "wireframe's" goal of reducing boilerplate for message handlers.20 Features like pluggable transports and serde serialization further highlight its modern design. - -A clear pattern emerges from these libraries: the use of derive macros and trait-based designs is a prevalent and effective strategy in Rust for simplifying protocol handling and reducing boilerplate code. Both `bin-proto` 14 and `protocol` 16 leverage custom derives to generate (de)serialization logic directly from struct and enum definitions. This is a proven pattern for enhancing developer ergonomics and reducing the likelihood of manual implementation errors. "wireframe" should strongly consider adopting a similar macro-driven approach for defining message structures and potentially for handler registration. +Several Rust libraries provide frameworks or utilities for implementing custom +network protocols, offering insights into effective abstractions. + +- `protocol` **(dylanmckay/protocol)**: This crate aims to simplify protocol + definitions in Rust by using a custom derive + `#[derive(protocol::Protocol)]`.16 It allows structured data to be sent and + received from any I/O stream, with built-in support for TCP and UDP. Any type + implementing its `Parcel` trait can be serialized. The derive macro handles + the implementation for custom types, though it requires types to also + implement `Clone`, `Debug`, and `PartialEq`, which might be overly restrictive + for some use cases.16 The library also supports middleware for transforming + sent/received data. This library exemplifies the use of derive macros to + reduce boilerplate in protocol definitions, a core strategy for "wireframe". + +- `message-io`: This library provides abstractions for message-based network + communication over various transports like TCP, UDP, and WebSockets.17 + Notably, it offers `FramedTcp`, which prefixes messages with their size, + managing data as packets rather than a raw stream.17 This distinction between + connection-oriented and packet-based transports, and the provision of framing + solutions, is relevant to "wireframe's" design for handling frame-based + protocols. + +- `tokio-util::codec`: Part of the extensive Tokio ecosystem 18, + `tokio-util::codec` provides `Encoder` and `Decoder` traits. These traits are + fundamental for implementing framing logic, allowing the conversion of a raw + byte stream into a sequence of frames, and vice-versa. While not a full + routing solution, these traits are essential low-level building blocks upon + which "wireframe" would likely build its framing layer. + +- `tarpc`: `tarpc` is an RPC framework for Rust focusing on ease of use.20 + Although designed for RPC, its approach of defining service schemas directly + in Rust code (using the `#[tarpc::service]` attribute to generate service + traits and client/server boilerplate) is an interesting parallel to + "wireframe's" goal of reducing boilerplate for message handlers.20 Features + like pluggable transports and serde serialization further highlight its modern + design. + +A clear pattern emerges from these libraries: the use of derive macros and +trait-based designs is a prevalent and effective strategy in Rust for +simplifying protocol handling and reducing boilerplate code. Both `bin-proto` 14 +and `protocol` 16 leverage custom derives to generate (de)serialization logic +directly from struct and enum definitions. This is a proven pattern for +enhancing developer ergonomics and reducing the likelihood of manual +implementation errors. "wireframe" should strongly consider adopting a similar +macro-driven approach for defining message structures and potentially for +handler registration. ### 3.3. Lessons from Actix Web's API Design -Actix Web, a high-performance web framework for Rust 5, is cited in the user query as a model for API aesthetics. Its design offers valuable lessons for "wireframe". - -- **Routing**: Actix Web provides flexible and declarative routing. Attribute macros like `#[get("/")]` and `#[post("/echo")]` allow handlers to be associated with specific HTTP methods and paths directly in their definition.4 Alternatively, routes can be registered programmatically using `App::service()` for macro-routed handlers or `App::route()` for manual configuration.4 This declarative style simplifies route management. - -- **Data Extraction**: A key feature of Actix Web is its powerful extractor system.22 Request handlers are `async` functions that accept parameters which are automatically extracted from the incoming request if they implement the `FromRequest` trait.4 Built-in extractors include `web::Path` for path parameters, `web::Json` for JSON payloads, `web::Query` for query parameters, and `web::Data` for accessing shared application state.21 Users can also define custom extractors by implementing `FromRequest`.24 This system promotes type safety and decouples data parsing and validation from the core handler logic. - -- **Middleware**: Actix Web has a robust middleware system allowing developers to insert custom processing logic into the request/response lifecycle.5 Middleware is registered using `App::wrap()` and typically implements `Transform` and `Service` traits.26 Common use cases include logging 27, authentication, and request/response modification. Middleware functions can be simple `async fn`s when using `middleware::from_fn()`.26 - -- **Application Structure and State**: Applications are built around an `App` instance, which is then used to configure an `HttpServer`.4 Shared application state can be managed using `web::Data`, making state accessible to handlers and middleware.21 For globally shared mutable state, careful use of `Arc` and `Mutex` (or atomics) is required, initialized outside the `HttpServer::new` closure.21 - -The ergonomic success of Actix Web for web application development can be significantly attributed to these powerful and intuitive abstractions. Extractors, for example, cleanly separate the concern of deriving data from a request from the business logic within a handler.22 Middleware allows for modular implementation of cross-cutting concerns like logging or authentication, preventing handler functions from becoming cluttered.26 If "wireframe" can adapt these patterns—such as creating extractors for deserialized message payloads or connection-specific metadata, and a middleware system for binary message streams—it can achieve comparable benefits in reducing complexity and improving code organization for its users. +Actix Web, a high-performance web framework for Rust 5, is cited in the user +query as a model for API aesthetics. Its design offers valuable lessons for +"wireframe". + +- **Routing**: Actix Web provides flexible and declarative routing. Attribute + macros like `#[get("/")]` and `#[post("/echo")]` allow handlers to be + associated with specific HTTP methods and paths directly in their definition.4 + Alternatively, routes can be registered programmatically using + `App::service()` for macro-routed handlers or `App::route()` for manual + configuration.4 This declarative style simplifies route management. + +- **Data Extraction**: A key feature of Actix Web is its powerful extractor + system.22 Request handlers are `async` functions that accept parameters which + are automatically extracted from the incoming request if they implement the + `FromRequest` trait.4 Built-in extractors include `web::Path` for path + parameters, `web::Json` for JSON payloads, `web::Query` for query parameters, + and `web::Data` for accessing shared application state.21 Users can also + define custom extractors by implementing `FromRequest`.24 This system promotes + type safety and decouples data parsing and validation from the core handler + logic. + +- **Middleware**: Actix Web has a robust middleware system allowing developers + to insert custom processing logic into the request/response lifecycle.5 + Middleware is registered using `App::wrap()` and typically implements + `Transform` and `Service` traits.26 Common use cases include logging 27, + authentication, and request/response modification. Middleware functions can be + simple `async fn`s when using `middleware::from_fn()`.26 + +- **Application Structure and State**: Applications are built around an `App` + instance, which is then used to configure an `HttpServer`.4 Shared application + state can be managed using `web::Data`, making state accessible to handlers + and middleware.21 For globally shared mutable state, careful use of `Arc` and + `Mutex` (or atomics) is required, initialized outside the `HttpServer::new` + closure.21 + +The ergonomic success of Actix Web for web application development can be +significantly attributed to these powerful and intuitive abstractions. +Extractors, for example, cleanly separate the concern of deriving data from a +request from the business logic within a handler.22 Middleware allows for +modular implementation of cross-cutting concerns like logging or authentication, +preventing handler functions from becoming cluttered.26 If "wireframe" can adapt +these patterns—such as creating extractors for deserialized message payloads or +connection-specific metadata, and a middleware system for binary message +streams—it can achieve comparable benefits in reducing complexity and improving +code organization for its users. ### 3.4. Key Learnings and Implications for "wireframe" Design -The preceding survey underscores several critical considerations for the design of "wireframe": - -1. **Derive Macros are Essential**: For message definition and (de)serialization, derive macros are paramount for reducing boilerplate and improving developer experience, as seen in `bin-proto`, `protocol`, `bincode`, and `postcard`. -2. **Robust (De)serialization Strategy**: The choice of `wire-rs` is contingent on its ability to support derivable `Encode`/`Decode` traits. If this is not feasible, well-established alternatives like `bincode` or `postcard` offer proven Serde integration and derive capabilities.10 -3. **Actix-Inspired API Patterns**: Leveraging Actix Web's patterns for routing, data extraction (extractors), and middleware is key to achieving the desired developer-friendliness and reducing source code complexity. -4. **Asynchronous Foundation**: Integration with an asynchronous runtime like Tokio is non-negotiable for a modern, performant networking library in Rust.18 - -Given the inaccessibility of `leynos/mxd` 7, a direct benchmark of complexity reduction is not possible. Therefore, "wireframe" must demonstrate its benefits through the qualitative improvements offered by its abstractions. The design must clearly articulate how its features—such as declarative APIs, automatic code generation via macros, and separation of concerns inspired by Actix Web—lead to simpler, more maintainable code compared to hypothetical manual implementations of binary protocol handlers. This involves illustrating "before" (manual, complex, error-prone) versus "after" (wireframe, simplified, robust) scenarios through its API design and examples. +The preceding survey underscores several critical considerations for the design +of "wireframe": + +1. **Derive Macros are Essential**: For message definition and + (de)serialization, derive macros are paramount for reducing boilerplate and + improving developer experience, as seen in `bin-proto`, `protocol`, + `bincode`, and `postcard`. +1. **Robust (De)serialization Strategy**: The choice of `wire-rs` is contingent + on its ability to support derivable `Encode`/`Decode` traits. If this is not + feasible, well-established alternatives like `bincode` or `postcard` offer + proven Serde integration and derive capabilities.10 +1. **Actix-Inspired API Patterns**: Leveraging Actix Web's patterns for routing, + data extraction (extractors), and middleware is key to achieving the desired + developer-friendliness and reducing source code complexity. +1. **Asynchronous Foundation**: Integration with an asynchronous runtime like + Tokio is non-negotiable for a modern, performant networking library in + Rust.18 + +Given the inaccessibility of `leynos/mxd` 7, a direct benchmark of complexity +reduction is not possible. Therefore, "wireframe" must demonstrate its benefits +through the qualitative improvements offered by its abstractions. The design +must clearly articulate how its features—such as declarative APIs, automatic +code generation via macros, and separation of concerns inspired by Actix +Web—lead to simpler, more maintainable code compared to hypothetical manual +implementations of binary protocol handlers. This involves illustrating "before" +(manual, complex, error-prone) versus "after" (wireframe, simplified, robust) +scenarios through its API design and examples. ## 4. "wireframe" Library Design -The design of "wireframe" is guided by the primary objective of reducing source code complexity for developers working with arbitrary frame-based binary protocols in Rust. This section details the core philosophy, architecture, and key mechanisms of the proposed library. +The design of "wireframe" is guided by the primary objective of reducing source +code complexity for developers working with arbitrary frame-based binary +protocols in Rust. This section details the core philosophy, architecture, and +key mechanisms of the proposed library. ### 4.1. Core Philosophy and Design Goals The development of "wireframe" adheres to the following principles: -- **Primary Goal: Reduce Source Code Complexity**: This is the central tenet. Every design decision is evaluated against its potential to simplify development. This is achieved through high-level abstractions, declarative APIs, and minimizing boilerplate code. -- **Generality for Arbitrary Frame-Based Protocols**: The library must not be opinionated about the specific content or structure of the binary protocols it handles beyond the assumption of a frame-based structure. Users should be able to define their own framing logic and message types. -- **Performance**: Leveraging Rust's inherent performance characteristics is crucial.2 While developer ergonomics is a primary focus, the design must avoid introducing unnecessary overhead. Asynchronous operations, powered by a runtime like Tokio, are essential for efficient I/O and concurrency.18 -- **Safety**: The library will harness Rust's strong type system and ownership model to prevent common networking bugs, such as data races and use-after-free errors, contributing to more reliable software.1 -- **Developer Ergonomics**: The API should be intuitive, well-documented, and easy to learn, particularly for developers familiar with patterns from Actix Web. - -A potential tension exists between the goal of supporting "arbitrary" protocols and maintaining simplicity. An overly generic library might force users to implement many low-level details, negating the complexity reduction benefits. Conversely, an excessively opinionated library might not be flexible enough for diverse protocol needs. "wireframe" aims to strike a balance by providing robust conventions and helper traits for common protocol patterns (e.g., message ID-based routing, standard framing methods) while still allowing "escape hatches" for highly customized requirements. This involves identifying commonalities in frame-based binary protocols and offering streamlined abstractions for them, without precluding more bespoke implementations. +- **Primary Goal: Reduce Source Code Complexity**: This is the central tenet. + Every design decision is evaluated against its potential to simplify + development. This is achieved through high-level abstractions, declarative + APIs, and minimizing boilerplate code. +- **Generality for Arbitrary Frame-Based Protocols**: The library must not be + opinionated about the specific content or structure of the binary protocols it + handles beyond the assumption of a frame-based structure. Users should be able + to define their own framing logic and message types. +- **Performance**: Leveraging Rust's inherent performance characteristics is + crucial.2 While developer ergonomics is a primary focus, the design must avoid + introducing unnecessary overhead. Asynchronous operations, powered by a + runtime like Tokio, are essential for efficient I/O and concurrency.18 +- **Safety**: The library will harness Rust's strong type system and ownership + model to prevent common networking bugs, such as data races and use-after-free + errors, contributing to more reliable software.1 +- **Developer Ergonomics**: The API should be intuitive, well-documented, and + easy to learn, particularly for developers familiar with patterns from Actix + Web. + +A potential tension exists between the goal of supporting "arbitrary" protocols +and maintaining simplicity. An overly generic library might force users to +implement many low-level details, negating the complexity reduction benefits. +Conversely, an excessively opinionated library might not be flexible enough for +diverse protocol needs. "wireframe" aims to strike a balance by providing robust +conventions and helper traits for common protocol patterns (e.g., message +ID-based routing, standard framing methods) while still allowing "escape +hatches" for highly customized requirements. This involves identifying +commonalities in frame-based binary protocols and offering streamlined +abstractions for them, without precluding more bespoke implementations. ### 4.2. High-Level Architecture -"wireframe" will employ a layered architecture, promoting modularity and separation of concerns. This structure allows different aspects of protocol handling to be managed and customized independently. +"wireframe" will employ a layered architecture, promoting modularity and +separation of concerns. This structure allows different aspects of protocol +handling to be managed and customized independently. -**(Conceptual Block Diagram)** +#### Conceptual Block Diagram -``` +````text +-------------------------+ +---------------------+ +--------------------------+ | Network Connection |<---->| Transport Layer |<---->| Framing Layer | | (e.g., TCP/UDP Socket) | | Adapter (Tokio I/O) | | (User-defined/Built-in) | @@ -114,78 +311,183 @@ A potential tension exists between the goal of supporting "arbitrary" protocols | Serialization Engine |<---->| Routing Engine |<---->| Deserialization Engine | | (e.g., wire-rs) | | (Message ID based) | | (e.g., wire-rs) | +--------------------------+ +---------------------+ +--------------------------+ -``` +```rust **Core Components**: -- **Transport Layer Adapter**: This component interfaces with the underlying network I/O, typically provided by Tokio's `TcpStream`, `UdpSocket`, or similar asynchronous I/O primitives.18 It is responsible for the raw reading of bytes from the network and writing bytes to it. -- **Framing Layer**: Responsible for processing the incoming byte stream from the transport layer into discrete logical units called "frames." Conversely, it serializes outgoing messages from the application into byte sequences suitable for transmission, adding any necessary framing information (e.g., length prefixes, delimiters). This layer will utilize user-defined or built-in framing logic, potentially by implementing traits like Tokio's `Decoder` and `Encoder`. -- **Deserialization/Serialization Engine**: This engine converts the byte payload of incoming frames into strongly-typed Rust data structures (messages) and serializes outgoing Rust messages into byte payloads for outgoing frames. This is the primary role intended for `wire-rs` 6 or an alternative like `bincode` 11 or `postcard`.12 -- **Routing Engine**: After a message is deserialized (or at least a header containing an identifier is processed), the routing engine inspects it to determine which user-defined handler function is responsible for processing this type of message. -- **Handler Invocation Logic**: This component is responsible for calling the appropriate user-defined handler function. It will manage the extraction of necessary data (e.g., the message payload, connection information) and provide it to the handler in a type-safe manner, inspired by Actix Web's extractors. -- **Middleware/Interceptor Chain**: This allows for pre-processing of incoming frames/messages before they reach the handler and post-processing of responses (or frames generated by the handler) before they are serialized and sent. This enables cross-cutting concerns like logging, authentication, or metrics. +- **Transport Layer Adapter**: This component interfaces with the underlying + network I/O, typically provided by Tokio's `TcpStream`, `UdpSocket`, or + similar asynchronous I/O primitives.18 It is responsible for the raw reading + of bytes from the network and writing bytes to it. +- **Framing Layer**: Responsible for processing the incoming byte stream from + the transport layer into discrete logical units called "frames." Conversely, + it serializes outgoing messages from the application into byte sequences + suitable for transmission, adding any necessary framing information (e.g., + length prefixes, delimiters). This layer will utilize user-defined or built-in + framing logic, potentially by implementing traits like Tokio's `Decoder` and + `Encoder`. +- **Deserialization/Serialization Engine**: This engine converts the byte + payload of incoming frames into strongly-typed Rust data structures (messages) + and serializes outgoing Rust messages into byte payloads for outgoing frames. + This is the primary role intended for `wire-rs` 6 or an alternative like + `bincode` 11 or `postcard`.12 +- **Routing Engine**: After a message is deserialized (or at least a header + containing an identifier is processed), the routing engine inspects it to + determine which user-defined handler function is responsible for processing + this type of message. +- **Handler Invocation Logic**: This component is responsible for calling the + appropriate user-defined handler function. It will manage the extraction of + necessary data (e.g., the message payload, connection information) and provide + it to the handler in a type-safe manner, inspired by Actix Web's extractors. +- **Middleware/Interceptor Chain**: This allows for pre-processing of incoming + frames/messages before they reach the handler and post-processing of responses + (or frames generated by the handler) before they are serialized and sent. This + enables cross-cutting concerns like logging, authentication, or metrics. **Data Flow**: 1. **Incoming**: Raw bytes arrive at the **Transport Layer Adapter**. -2. The bytes are passed to the **Framing Layer**, which identifies and extracts a complete frame. -3. The frame's payload is passed to the **Deserialization Engine**, which converts it into a Rust message type. -4. The deserialized message is passed to the **Routing Engine**, which identifies the target handler based on message type or ID. -5. The message (and associated request context) passes through the **Middleware Chain (request path)**. -6. The **Handler Invocation Logic** calls the designated **User Handler Function**. -7. **Outgoing**: If the handler produces a response message: a. The response passes through the **Middleware Chain (response path)**. b. The response message is passed to the **Serialization Engine** to be converted into a byte payload. c. This payload is given to the **Framing Layer** to be encapsulated in a frame. d. The framed bytes are sent via the **Transport Layer Adapter**. - -This layered architecture mirrors the conceptual separation found in network protocol stacks, such as the OSI or TCP/IP models.28 Each component addresses a distinct set of problems. This modularity is fundamental to managing the overall complexity of the library and the applications built with it. It allows individual components, such as the framing mechanism or the serialization engine, to be potentially customized or even replaced by users with specific needs, without requiring modifications to other parts of the system. +2. The bytes are passed to the **Framing Layer**, which identifies and extracts + a complete frame. +3. The frame's payload is passed to the **Deserialization Engine**, which + converts it into a Rust message type. +4. The deserialized message is passed to the **Routing Engine**, which + identifies the target handler based on message type or ID. +5. The message (and associated request context) passes through the **Middleware + Chain (request path)**. +6. The **Handler Invocation Logic** calls the designated **User Handler + Function**. +7. **Outgoing**: If the handler produces a response message: a. The response + passes through the **Middleware Chain (response path)**. b. The response + message is passed to the **Serialization Engine** to be converted into a byte + payload. c. This payload is given to the **Framing Layer** to be encapsulated + in a frame. d. The framed bytes are sent via the **Transport Layer Adapter**. + +This layered architecture mirrors the conceptual separation found in network +protocol stacks, such as the OSI or TCP/IP models.28 Each component addresses a +distinct set of problems. This modularity is fundamental to managing the overall +complexity of the library and the applications built with it. It allows +individual components, such as the framing mechanism or the serialization +engine, to be potentially customized or even replaced by users with specific +needs, without requiring modifications to other parts of the system. ### 4.3. Frame Definition and Processing -To handle "arbitrary frame-based protocols," "wireframe" must provide a flexible way to define and process frames. - -- `FrameProcessor` **(or Tokio** `Decoder`**/**`Encoder` **integration)**: The core of frame handling will revolve around a user-implementable trait, tentatively named `FrameProcessor`. Alternatively, and perhaps preferably for ecosystem compatibility, "wireframe" could directly leverage Tokio's `tokio_util::codec::{Decoder, Encoder}` traits. Users would implement these traits to define: - - - **Decoding**: How a raw byte stream from the network is parsed into discrete frames. This logic would handle issues like partial reads and buffering, accumulating bytes until one or more complete frames can be extracted. Examples include length-prefixed framing (where a header indicates the payload size, similar to `message-io`'s `FramedTcp` 17), delimiter-based framing (where frames are separated by a specific byte sequence), or fixed-size framing. - - **Encoding**: How an outgoing message (or its serialized byte payload) is encapsulated into a frame for transmission, including adding any necessary headers, length prefixes, or delimiters. - - "wireframe" could provide common `FrameProcessor` implementations (e.g., for length-prefixed frames) as part of its standard library, simplifying setup for common protocol types. - -- **Optional** `FrameMetadata` **Trait**: For protocols where routing decisions or pre-handler middleware logic might depend on information in a frame header *before* the entire payload is deserialized (e.g., for performance reasons or if the payload type depends on a header field), an optional trait like `FrameMetadata` could be introduced. Types implementing this trait would represent the quickly parsable header part of a frame, providing access to identifiers or flags. This allows for a two-stage deserialization: first the metadata, then, based on that, the full payload. - -This separation of framing logic via traits is crucial. Different binary protocols employ vastly different methods for delimiting messages on the wire. Some use fixed-size headers with length fields, others rely on special start/end byte sequences, and some might have no explicit framing beyond what the transport layer provides (e.g., UDP datagrams). A trait-based approach for frame processing, akin to how `tokio-util::codec` operates, endows "wireframe" with the necessary flexibility to adapt to this diversity without embedding assumptions about any single framing strategy into its core. +To handle "arbitrary frame-based protocols," "wireframe" must provide a flexible +way to define and process frames. + +- `FrameProcessor` **(or Tokio** `Decoder`**/**`Encoder` **integration)**: The + core of frame handling will revolve around a user-implementable trait, + tentatively named `FrameProcessor`. Alternatively, and perhaps preferably for + ecosystem compatibility, "wireframe" could directly leverage Tokio's + `tokio_util::codec::{Decoder, Encoder}` traits. Users would implement these + traits to define: + + - **Decoding**: How a raw byte stream from the network is parsed into discrete + frames. This logic would handle issues like partial reads and buffering, + accumulating bytes until one or more complete frames can be extracted. + Examples include length-prefixed framing (where a header indicates the + payload size, similar to `message-io`'s `FramedTcp` 17), delimiter-based + framing (where frames are separated by a specific byte sequence), or + fixed-size framing. + - **Encoding**: How an outgoing message (or its serialized byte payload) is + encapsulated into a frame for transmission, including adding any necessary + headers, length prefixes, or delimiters. + + "wireframe" could provide common `FrameProcessor` implementations (e.g., for + length-prefixed frames) as part of its standard library, simplifying setup for + common protocol types. + +- **Optional** `FrameMetadata` **Trait**: For protocols where routing decisions + or pre-handler middleware logic might depend on information in a frame header + *before* the entire payload is deserialized (e.g., for performance reasons or + if the payload type depends on a header field), an optional trait like + `FrameMetadata` could be introduced. Types implementing this trait would + represent the quickly parsable header part of a frame, providing access to + identifiers or flags. This allows for a two-stage deserialization: first the + metadata, then, based on that, the full payload. + +This separation of framing logic via traits is crucial. Different binary +protocols employ vastly different methods for delimiting messages on the wire. +Some use fixed-size headers with length fields, others rely on special start/end +byte sequences, and some might have no explicit framing beyond what the +transport layer provides (e.g., UDP datagrams). A trait-based approach for frame +processing, akin to how `tokio-util::codec` operates, endows "wireframe" with +the necessary flexibility to adapt to this diversity without embedding +assumptions about any single framing strategy into its core. ### 4.4. Message Serialization and Deserialization -The conversion of frame payloads to and from Rust types is a critical source of complexity that "wireframe" aims to simplify. +The conversion of frame payloads to and from Rust types is a critical source of +complexity that "wireframe" aims to simplify. - Primary Strategy: wire-rs with Derivable Traits: - The preferred approach is to utilize wire-rs 6 as the underlying serialization and deserialization engine. However, this is critically dependent on wire-rs supporting, or being extended to support, derivable Encode and Decode traits (e.g., through #). The ability to automatically generate this logic from struct/enum definitions is paramount. Manual serialization/deserialization using WireReader::read_u32(), WireWriter::write_string(), etc., for every field would not meet the complexity reduction goals. + The preferred approach is to utilize wire-rs 6 as the underlying serialization + and deserialization engine. However, this is critically dependent on wire-rs + supporting, or being extended to support, derivable Encode and Decode traits + (e.g., through #). The ability to automatically generate this logic from + struct/enum definitions is paramount. Manual serialization/deserialization + using WireReader::read_u32(), WireWriter::write_string(), etc., for every + field would not meet the complexity reduction goals. - If wire-rs itself does not offer derive macros, "wireframe" might need to provide its own wrapper traits and derive macros that internally use wire-rs's WireReader and WireWriter primitives but expose a user-friendly, derivable interface. + If wire-rs itself does not offer derive macros, "wireframe" might need to + provide its own wrapper traits and derive macros that internally use wire-rs's + WireReader and WireWriter primitives but expose a user-friendly, derivable + interface. - Alternative/Fallback Strategies: - If wire-rs proves unsuitable (e.g., due to the difficulty of implementing derivable traits, performance characteristics not matching the needs, or fundamental API incompatibilities), well-supported alternatives with excellent Serde integration will be considered: + If wire-rs proves unsuitable (e.g., due to the difficulty of implementing + derivable traits, performance characteristics not matching the needs, or + fundamental API incompatibilities), well-supported alternatives with excellent + Serde integration will be considered: - - `bincode`: Offers high performance, configurability, and its own derivable `Encode`/`Decode` traits in version 2.x.10 It is a strong contender for general-purpose binary serialization. - - `postcard`: Ideal for scenarios where serialized size is a primary concern, such as embedded systems, and also provides Serde-based derive macros.9 Its simpler configuration might be an advantage.10 The choice would be guided by the specific requirements of typical "wireframe" use cases. + - `bincode`: Offers high performance, configurability, and its own derivable + `Encode`/`Decode` traits in version 2.x.10 It is a strong contender for + general-purpose binary serialization. + - `postcard`: Ideal for scenarios where serialized size is a primary concern, + such as embedded systems, and also provides Serde-based derive macros.9 Its + simpler configuration might be an advantage.10 The choice would be guided by + the specific requirements of typical "wireframe" use cases. - Context-Aware Deserialization: - Some binary protocols require deserialization logic to vary based on preceding data or external state. For instance, the interpretation of a field might depend on the value of an earlier field in the same message or on the current state of the connection. bin-proto hints at such capabilities with its context parameter in BitDecode::decode.14 "wireframe" should aim to support this, either through features in the chosen serialization library (e.g., wire-rs supporting a context argument) or by layering this capability on top. This could involve a multi-stage deserialization process: + Some binary protocols require deserialization logic to vary based on preceding + data or external state. For instance, the interpretation of a field might + depend on the value of an earlier field in the same message or on the current + state of the connection. bin-proto hints at such capabilities with its context + parameter in BitDecode::decode.14 "wireframe" should aim to support this, + either through features in the chosen serialization library (e.g., wire-rs + supporting a context argument) or by layering this capability on top. This + could involve a multi-stage deserialization process: 1. Deserialize a generic frame header or a preliminary part of the message. 2. Use information from this initial part to establish a context. 3. Deserialize the remainder of the payload using this context. -The cornerstone of "reducing source code complexity" in this domain is the automation of serialization and deserialization. Manual implementation of this logic is not only tedious but also a frequent source of subtle bugs related to endianness, offset calculations, and data type mismatches. Libraries like Serde, and by extension `bincode` and `postcard`, have demonstrated the immense value of derive macros in Rust for automating these tasks.8 If "wireframe" were to force users into manual field-by-field reading and writing, it would fail to deliver on its primary promise. Therefore, ensuring a smooth, derivable (de)serialization experience is a non-negotiable aspect of its design. +The cornerstone of "reducing source code complexity" in this domain is the +automation of serialization and deserialization. Manual implementation of this +logic is not only tedious but also a frequent source of subtle bugs related to +endianness, offset calculations, and data type mismatches. Libraries like Serde, +and by extension `bincode` and `postcard`, have demonstrated the immense value +of derive macros in Rust for automating these tasks.8 If "wireframe" were to +force users into manual field-by-field reading and writing, it would fail to +deliver on its primary promise. Therefore, ensuring a smooth, derivable +(de)serialization experience is a non-negotiable aspect of its design. ### 4.5. Routing and Dispatch Mechanism -Once a frame is received and its payload (or at least a routing-relevant header) is deserialized into a Rust message, "wireframe" needs an efficient and clear mechanism to dispatch this message to the appropriate user-defined handler. +Once a frame is received and its payload (or at least a routing-relevant header) +is deserialized into a Rust message, "wireframe" needs an efficient and clear +mechanism to dispatch this message to the appropriate user-defined handler. - Route Definition: - Inspired by Actix Web's declarative routing 4, but adapted for binary protocols, routes in "wireframe" will be defined based on message identifiers rather than URL paths. This identifier could be an integer, a string, or an enum variant extracted from the message itself or its frame header. + Inspired by Actix Web's declarative routing 4, but adapted for binary + protocols, routes in "wireframe" will be defined based on message identifiers + rather than URL paths. This identifier could be an integer, a string, or an + enum variant extracted from the message itself or its frame header. Two primary approaches for defining routes are envisaged: @@ -193,49 +495,61 @@ Once a frame is received and its payload (or at least a routing-relevant header) Rust - ``` + ```rust // Assuming MessageType is an enum identifying different messages Router::new() .message(MessageType::LoginRequest, handle_login) .message(MessageType::ChatMessage, handle_chat) //... other routes - + ``` + 2. **Attribute Macro on Handler Functions**: Rust - ``` + ```rust # async fn handle_login(msg: Message, conn_info: ConnectionInfo) -> Result { //... handler logic } - + # async fn handle_chat(msg: Message, state: SharedState) { //... handler logic, no direct response needed } - + ``` - The attribute macro approach is generally preferred for its conciseness and co-location of routing information with handler logic, similar to Actix Web's `#[get(...)]` macros. + The attribute macro approach is generally preferred for its conciseness and + co-location of routing information with handler logic, similar to Actix Web's + `#[get(...)]` macros. - Dispatch Logic: - Internally, the router will maintain a mapping (e.g., using a HashMap or a specialized dispatch table if the identifiers are dense integers) from message identifiers to handler functions. Upon receiving a deserialized message, the router extracts its identifier, looks up the corresponding handler, and invokes it. The efficiency of this lookup is critical for high-throughput systems that process a large number of messages per second. + Internally, the router will maintain a mapping (e.g., using a HashMap or a + specialized dispatch table if the identifiers are dense integers) from message + identifiers to handler functions. Upon receiving a deserialized message, the + router extracts its identifier, looks up the corresponding handler, and + invokes it. The efficiency of this lookup is critical for high-throughput + systems that process a large number of messages per second. - Dynamic Routing and Guards: - For more complex routing scenarios where the choice of handler might depend on multiple fields within a message or the state of the connection, "wireframe" could incorporate a "guard" system, analogous to Actix Web's route guards.21 Guards would be functions that evaluate conditions on the incoming message or connection context before a handler is chosen. + For more complex routing scenarios where the choice of handler might depend on + multiple fields within a message or the state of the connection, "wireframe" + could incorporate a "guard" system, analogous to Actix Web's route guards.21 + Guards would be functions that evaluate conditions on the incoming message or + connection context before a handler is chosen. Rust - ``` + ```rust Router::new() .message_guarded( MessageType::GenericCommand, - - ``` + +```` |msg_header: &CommandHeader| msg_header.sub_type == CommandSubType::Special, @@ -247,44 +561,60 @@ handle_special_command \`\`\` -The routing mechanism essentially implements a form of pattern matching or a state machine that operates on message identifiers. A clear, declarative API for this routing, as opposed to manual `match` statements over message types within a single monolithic receive loop, significantly simplifies the top-level structure of a protocol server. This declarative approach makes it easier for developers to understand the mapping between incoming messages and their respective processing logic, thereby improving the maintainability and comprehensibility of the application, especially as the number of message types grows. +The routing mechanism essentially implements a form of pattern matching or a +state machine that operates on message identifiers. A clear, declarative API for +this routing, as opposed to manual `match` statements over message types within +a single monolithic receive loop, significantly simplifies the top-level +structure of a protocol server. This declarative approach makes it easier for +developers to understand the mapping between incoming messages and their +respective processing logic, thereby improving the maintainability and +comprehensibility of the application, especially as the number of message types +grows. ## 5. "wireframe" API Design (Inspired by Actix Web) -The API of "wireframe" will be heavily influenced by Actix Web, aiming for a similar level of developer ergonomics and clarity. This section details the proposed API components. +The API of "wireframe" will be heavily influenced by Actix Web, aiming for a +similar level of developer ergonomics and clarity. This section details the +proposed API components. ### 5.1. Router Configuration and Service Definition -Similar to Actix Web's `App` and `HttpServer` structure 4, "wireframe" will provide a builder pattern for configuring the application and a server component to run it. +Similar to Actix Web's `App` and `HttpServer` structure 4, "wireframe" will +provide a builder pattern for configuring the application and a server component +to run it. -- `WireframeApp` **or** `Router` **Builder**: A central builder struct, let's call it `WireframeApp`, will serve as the primary point for configuring the protocol handling logic. +- `WireframeApp` **or** `Router` **Builder**: A central builder struct, let's + call it `WireframeApp`, will serve as the primary point for configuring the + protocol handling logic. Rust - ``` + ````rustrust use wireframe::{WireframeApp, WireframeServer, Message, error::Result}; use my_protocol::{LoginRequest, LoginResponse, ChatMessage, AppState, MyFrameProcessor, MessageType}; use std::sync::Arc; use tokio::sync::Mutex; - + // Example handler functions (defined elsewhere) async fn handle_login(req: Message) -> Result { /*... */ Ok(LoginResponse::default()) } async fn handle_chat_message(msg: Message, state: SharedState) { /*... */ } - + // --- Assuming message_handler attribute macro --- # async fn login_handler(req: Message) -> Result { /*... */ Ok(LoginResponse::default()) } - + # async fn chat_handler(msg: Message, state: SharedState) { /*... */ } // --- - + async fn main_server_setup() -> std::io::Result<()> { let app_state = Arc::new(Mutex::new(AppState::new())); - + WireframeServer::new(move | - - ``` + + ```rust + + ```` | { // Closure provides App per worker thread @@ -322,27 +652,42 @@ The WireframeApp builder would offer methods like: \* .frame_processor(impl FrameProcessor): Sets the framing logic. -\* .service(handler_function): Registers a handler function, potentially inferring the message type it handles if attribute macros are used. +\* .service(handler_function): Registers a handler function, potentially +inferring the message type it handles if attribute macros are used. -\* .route(message_id, handler_function): Explicitly maps a message identifier to a handler. +\* .route(message_id, handler_function): Explicitly maps a message identifier to +a handler. -\* .app_data(SharedState<T>) or .data(T): Provides shared application state, similar to Actix Web's web::Data.21 +\* .app_data(SharedState\) or .data(T): Provides shared application state, +similar to Actix Web's web::Data.21 \* .wrap(middleware_factory): Adds middleware to the processing pipeline.26 -- **Server Initialization**: A `WireframeServer` component (analogous to `HttpServer`) would take the configured `WireframeApp` factory (a closure that creates an `App` instance per worker thread), bind to a network address, and manage incoming connections, task spawning for each connection, and the overall server lifecycle. This would likely be built on Tokio's networking and runtime primitives.18 +- **Server Initialization**: A `WireframeServer` component (analogous to + `HttpServer`) would take the configured `WireframeApp` factory (a closure that + creates an `App` instance per worker thread), bind to a network address, and + manage incoming connections, task spawning for each connection, and the + overall server lifecycle. This would likely be built on Tokio's networking and + runtime primitives.18 -This structural similarity to Actix Web is intentional. Developers familiar with Actix Web's application setup will find "wireframe's" approach intuitive, reducing the learning curve. Actix Web is a widely adopted framework 5, and reusing its successful patterns makes "wireframe" feel like a natural extension for a different networking domain, rather than an entirely new and unfamiliar library. +This structural similarity to Actix Web is intentional. Developers familiar with +Actix Web's application setup will find "wireframe's" approach intuitive, +reducing the learning curve. Actix Web is a widely adopted framework 5, and +reusing its successful patterns makes "wireframe" feel like a natural extension +for a different networking domain, rather than an entirely new and unfamiliar +library. ### 5.2. Handler Functions -Handler functions are the core of the application logic, processing incoming messages and optionally producing responses. +Handler functions are the core of the application logic, processing incoming +messages and optionally producing responses. -- **Signature**: Handler functions will be `async` and their parameters will be resolved using "wireframe" extractors. +- **Signature**: Handler functions will be `async` and their parameters will be + resolved using "wireframe" extractors. Rust - ``` + ````rustrust async fn handler_name( param1: ExtractedType1, param2: ExtractedType2, @@ -350,103 +695,188 @@ Handler functions are the core of the application logic, processing incoming mes ) -> Result { // Handler logic } - - ``` -- **Asynchronous Nature**: Handlers *must* be `async` functions to integrate seamlessly with the Tokio runtime and enable non-blocking operations. This is standard practice in modern Rust network programming.4 -- **Parameters as Extractors**: Arguments to handler functions are not passed directly by the router. Instead, their types must implement an "extractor" trait (see Section 5.3). "wireframe" will attempt to resolve each argument by calling its extraction logic. -- **Return Types**: Handlers can return various types, which "wireframe" will process accordingly: - - A specific message type that implements a `wireframe::Responder` trait (analogous to Actix Web's `Responder` trait 4). This trait defines how the returned value is serialized and sent back to the client. - - `Result`: For explicit error handling. If `Ok(response_message)`, the message is sent. If `Err(error_value)`, the error is processed by "wireframe's" error handling mechanism (see Section 5.5). `ErrorType` should be convertible into a general `WireframeError`. - - `()`: If the handler does not need to send a direct response to the specific incoming message (e.g., broadcasting a message to other clients, or if the protocol is one-way for this message). - -The following table outlines core "wireframe" API components and their Actix Web analogies, illustrating how the "aesthetic sense" of Actix Web is translated: -**Table 1: Core "wireframe" API Components and Actix Web Analogies** - - - -

"wireframe" Component

Actix Web Analogy

Purpose in "wireframe"

WireframeApp / Router

actix_web::App

Overall application/service configuration, route registration, middleware, state.

WireframeServer

actix_web::HttpServer

Binds to network, manages connections, runs the application.

#[message_handler(MsgId)]

#[get("/path")] / #[post("/path")]

Declarative routing for handlers based on message identifiers.

Message<T> (Extractor)

web::Json<T> / web::Payload

Extracts and deserializes the main message payload of type T.

ConnectionInfo (Extractor)

HttpRequest

Provides access to connection-specific data (e.g., peer address, connection ID).

SharedState<T> (Extractor)

web::Data<T>

Provides access to shared application state.

impl WireframeResponder

impl Responder

Defines how handler return values are serialized and sent back to the client.

WireframeMiddleware (Transform)

impl Transform

Factory for middleware services that process messages/frames.

- -This mapping is valuable because it leverages existing mental models for developers familiar with Actix Web, thereby lowering the barrier to adoption for "wireframe". The use of `async fn` and extractor patterns directly contributes to cleaner, more readable, and more testable handler logic. By delegating tasks like data deserialization and access to connection metadata to extractors (similar to how Actix Web extractors handle HTTP-specific parsing 22), "wireframe" handler functions can remain focused on the core business logic associated with each message type. + ```rust + + ```` + +- **Asynchronous Nature**: Handlers *must* be `async` functions to integrate + seamlessly with the Tokio runtime and enable non-blocking operations. This is + standard practice in modern Rust network programming.4 + +- **Parameters as Extractors**: Arguments to handler functions are not passed + directly by the router. Instead, their types must implement an "extractor" + trait (see Section 5.3). "wireframe" will attempt to resolve each argument by + calling its extraction logic. + +- **Return Types**: Handlers can return various types, which "wireframe" will + process accordingly: + + - A specific message type that implements a `wireframe::Responder` trait + (analogous to Actix Web's `Responder` trait 4). This trait defines how the + returned value is serialized and sent back to the client. + - `Result`: For explicit error handling. If + `Ok(response_message)`, the message is sent. If `Err(error_value)`, the + error is processed by "wireframe's" error handling mechanism (see Section + 5.5). `ErrorType` should be convertible into a general `WireframeError`. + - `()`: If the handler does not need to send a direct response to the specific + incoming message (e.g., broadcasting a message to other clients, or if the + protocol is one-way for this message). + +The following table outlines core "wireframe" API components and their Actix Web +analogies, illustrating how the "aesthetic sense" of Actix Web is translated: + +#### Table 1: Core `wireframe` API Components and Actix Web Analogies + +| `wireframe` Component | Actix Web Analogy | Purpose in `wireframe` | | --- | +--- | --- | | `WireframeApp` / `Router` | `actix_web::App` | Overall +application/service configuration, route registration, middleware, state. | | +`WireframeServer` | `actix_web::HttpServer` | Binds to network, manages +connections, runs the application. | | `#[message_handler(MsgId)]` | +`#[get("/path")]` / `#[post("/path")]` | Declarative routing for handlers based +on message identifiers. | | `Message` (Extractor) | `web::Json` / +`web::Payload` | Extracts and deserializes the main message payload of type `T`. +| | `ConnectionInfo` (Extractor) | `HttpRequest` | Provides access to +connection-specific data (e.g., peer address, connection ID). | | +`SharedState` (Extractor) | `web::Data` | Provides access to shared +application state. | | `impl WireframeResponder` | `impl Responder` | Defines +how handler return values are serialized and sent back to the client. | | +`WireframeMiddleware` (Transform) | `impl Transform` | Factory for middleware +services that process messages/frames. | + +This mapping is valuable because it leverages existing mental models for +developers familiar with Actix Web, thereby lowering the barrier to adoption for +"wireframe". The use of `async fn` and extractor patterns directly contributes +to cleaner, more readable, and more testable handler logic. By delegating tasks +like data deserialization and access to connection metadata to extractors +(similar to how Actix Web extractors handle HTTP-specific parsing 22), +"wireframe" handler functions can remain focused on the core business logic +associated with each message type. ### 5.3. Data Extraction and Type Safety -Inspired by Actix Web's extractors 22, "wireframe" will provide a type-safe mechanism for accessing data from incoming messages and connection context within handlers. +Inspired by Actix Web's extractors 22, "wireframe" will provide a type-safe +mechanism for accessing data from incoming messages and connection context +within handlers. -- `FromMessageRequest` **Trait**: A central trait, analogous to Actix Web's `FromRequest` 24, will be defined. Types implementing `FromMessageRequest` can be used as handler arguments. +- `FromMessageRequest` **Trait**: A central trait, analogous to Actix Web's + `FromRequest` 24, will be defined. Types implementing `FromMessageRequest` can + be used as handler arguments. Rust - ``` + ````rustrust use wireframe::dev::{MessageRequest, Payload}; // Hypothetical types use std::future::Future; - + pub trait FromMessageRequest: Sized { type Error: Into; // Error type if extraction fails type Future: Future>; - + fn from_message_request(req: &MessageRequest, payload: &mut Payload) -> Self::Future; } - - ``` - The `MessageRequest` would encapsulate information about the current incoming message context (like connection details, already parsed headers if any), and `Payload` would represent the raw or partially processed frame data. + ```rust + + The `MessageRequest` would encapsulate information about the current incoming + message context (like connection details, already parsed headers if any), and + `Payload` would represent the raw or partially processed frame data. + + ```` - **Built-in Extractors**: "wireframe" will provide several common extractors: - - `Message`: This would be the most common extractor. It attempts to deserialize the incoming frame's payload into the specified type `T`. `T` must implement the relevant deserialization trait (e.g., `Decode` from `wire-rs` or `serde::Deserialize` if using `bincode`/`postcard`). + - `Message`: This would be the most common extractor. It attempts to + deserialize the incoming frame's payload into the specified type `T`. `T` + must implement the relevant deserialization trait (e.g., `Decode` from + `wire-rs` or `serde::Deserialize` if using `bincode`/`postcard`). Rust - ``` + ````rustrust async fn handle_user_update(update: Message) -> Result<()> { // update.into_inner() gives UserUpdateData //... } - - ``` - - `ConnectionInfo`: Provides access to metadata about the current connection, such as the peer's network address, a unique connection identifier assigned by "wireframe", or transport-specific details. + + ```rust + + ```` + + - `ConnectionInfo`: Provides access to metadata about the current connection, + such as the peer's network address, a unique connection identifier assigned + by "wireframe", or transport-specific details. Rust - ``` + ````rustrust async fn handle_connect_event(conn_info: ConnectionInfo) { println!("New connection from: {}", conn_info.peer_addr()); } - - ``` - - `SharedState`: Allows handlers to access shared application state that was registered with `WireframeApp::app_data()`, similar to `actix_web::web::Data`.21 + + ```rust + + ```` + + - `SharedState`: Allows handlers to access shared application state that + was registered with `WireframeApp::app_data()`, similar to + `actix_web::web::Data`.21 Rust - ``` + ````rustrust async fn get_user_count(state: SharedState>>) -> Result { let count = state.lock().await.get_user_count(); //... } - - ``` -- **Custom Extractors**: Developers can implement `FromMessageRequest` for their own types. This is a powerful extensibility point, allowing encapsulation of custom logic for deriving specific pieces of data from an incoming message or its context. For example, a custom extractor could parse a session token from a specific field in all messages, validate it, and provide a `UserSession` object to the handler. + ```rust + + ```` + +- **Custom Extractors**: Developers can implement `FromMessageRequest` for their + own types. This is a powerful extensibility point, allowing encapsulation of + custom logic for deriving specific pieces of data from an incoming message or + its context. For example, a custom extractor could parse a session token from + a specific field in all messages, validate it, and provide a `UserSession` + object to the handler. -This extractor system, backed by Rust's strong type system, ensures that handlers receive correctly typed and validated data, significantly reducing the likelihood of runtime errors and boilerplate parsing code within the handler logic itself. Custom extractors are particularly valuable as they allow common, protocol-specific data extraction and validation logic (e.g., extracting and verifying a session token from a custom frame header) to be encapsulated into reusable components. This further reduces code duplication across multiple handlers and keeps the handler functions lean and focused on their specific business tasks, mirroring the benefits seen with Actix Web's `FromRequest` trait.24 +This extractor system, backed by Rust's strong type system, ensures that +handlers receive correctly typed and validated data, significantly reducing the +likelihood of runtime errors and boilerplate parsing code within the handler +logic itself. Custom extractors are particularly valuable as they allow common, +protocol-specific data extraction and validation logic (e.g., extracting and +verifying a session token from a custom frame header) to be encapsulated into +reusable components. This further reduces code duplication across multiple +handlers and keeps the handler functions lean and focused on their specific +business tasks, mirroring the benefits seen with Actix Web's `FromRequest` +trait.24 ### 5.4. Middleware and Extensibility -"wireframe" will incorporate a middleware system conceptually similar to Actix Web's 5, allowing developers to inject custom logic into the message processing pipeline. +"wireframe" will incorporate a middleware system conceptually similar to Actix +Web's 5, allowing developers to inject custom logic into the message processing +pipeline. -- `WireframeMiddleware` **Concept**: Middleware in "wireframe" will be defined by implementing a pair of traits, analogous to Actix Web's `Transform` and `Service` traits.25 +- `WireframeMiddleware` **Concept**: Middleware in "wireframe" will be defined + by implementing a pair of traits, analogous to Actix Web's `Transform` and + `Service` traits.25 - The `Transform` trait would act as a factory for the middleware service. - - The `Service` trait would define the actual request/response processing logic. Middleware would operate on "wireframe's" internal request and response types, which could be raw frames at one level or deserialized messages at another, depending on the middleware's purpose. + - The `Service` trait would define the actual request/response processing + logic. Middleware would operate on "wireframe's" internal request and + response types, which could be raw frames at one level or deserialized + messages at another, depending on the middleware's purpose. - A simplified functional middleware approach, similar to `actix_web::middleware::from_fn` 26, could also be provided for simpler use cases: + A simplified functional middleware approach, similar to + `actix_web::middleware::from_fn` 26, could also be provided for simpler use + cases: Rust - ``` + ````rustrust use wireframe::middleware::{Next, ServiceRequest, ServiceResponse}; // Hypothetical types - + async fn logging_mw_fn( req: ServiceRequest, // Represents an incoming message/context next: Next // Call to proceed to the next middleware or handler @@ -458,54 +888,91 @@ This extractor system, backed by Rust's strong type system, ensures that handler } Ok(res) } - - ``` -- **Registration**: Middleware would be registered with the `WireframeApp` builder: + ```rust + + ```` + +- **Registration**: Middleware would be registered with the `WireframeApp` + builder: Rust - ``` + ````rustrust WireframeApp::new() .wrap(LoggingMiddleware::new()) .wrap(AuthMiddleware::new(/* config */)) // For functional middleware: //.wrap(wireframe::middleware::from_fn(logging_mw_fn)) - - ``` - Middleware is typically executed in the reverse order of registration for incoming messages and in the registration order for outgoing responses.26 + ```rust -- **Use Cases**: - - - **Logging**: Recording details of incoming and outgoing messages, connection events (e.g., Actix Web's `Logger` middleware 27). - - **Authentication/Authorization**: Validating credentials present in messages or authorizing operations based on connection state or message content. - - **Metrics**: Collecting data such as message counts per type, processing times for handlers, or active connection counts. - - **Frame/Message Transformation**: On-the-fly modification of frames or messages, such as encryption/decryption or compression/decompression. (Though complex transformations might be better suited to the `FrameProcessor` layer). - - **Request/Response Manipulation**: Modifying message content before it reaches a handler or before a response is sent. - - **Connection Lifecycle Hooks**: Performing actions when connections are established or terminated. + Middleware is typically executed in the reverse order of registration for + incoming messages and in the registration order for outgoing responses.26 -The middleware system promotes a clean separation of concerns. Cross-cutting functionalities like logging, authentication, or metrics collection can be implemented as modular, reusable middleware components. This prevents the core handler logic from being cluttered with these auxiliary tasks, leading to more maintainable and focused handlers. This modularity is a key factor in reducing overall application complexity, particularly in larger systems, mirroring the advantages provided by Actix Web's middleware architecture.26 For instance, a middleware could transparently handle session management for stateful binary protocols, abstracting this complexity away from individual message handlers. + ```` -### 5.5. Error Handling Strategy +- **Use Cases**: -Robust and clear error handling is crucial for any network service. "wireframe" will provide a comprehensive error handling strategy. + - **Logging**: Recording details of incoming and outgoing messages, connection + events (e.g., Actix Web's `Logger` middleware 27). + - **Authentication/Authorization**: Validating credentials present in messages + or authorizing operations based on connection state or message content. + - **Metrics**: Collecting data such as message counts per type, processing + times for handlers, or active connection counts. + - **Frame/Message Transformation**: On-the-fly modification of frames or + messages, such as encryption/decryption or compression/decompression. + (Though complex transformations might be better suited to the + `FrameProcessor` layer). + - **Request/Response Manipulation**: Modifying message content before it + reaches a handler or before a response is sent. + - **Connection Lifecycle Hooks**: Performing actions when connections are + established or terminated. + +The middleware system promotes a clean separation of concerns. Cross-cutting +functionalities like logging, authentication, or metrics collection can be +implemented as modular, reusable middleware components. This prevents the core +handler logic from being cluttered with these auxiliary tasks, leading to more +maintainable and focused handlers. This modularity is a key factor in reducing +overall application complexity, particularly in larger systems, mirroring the +advantages provided by Actix Web's middleware architecture.26 For instance, a +middleware could transparently handle session management for stateful binary +protocols, abstracting this complexity away from individual message handlers. -- **Comprehensive Error Enum(s)**: Specific error types will be defined for different stages of processing to provide clear diagnostics: +### 5.5. Error Handling Strategy - - `FramingError`: Errors occurring during the parsing of byte streams into frames or the construction of frames from outgoing messages (e.g., incomplete frames, invalid length prefixes, delimiter mismatches). - - `SerializationError` / `DeserializationError`: Errors originating from the underlying (de)serialization library (e.g., `wire-rs`, `bincode`). This would include issues like malformed payloads, type mismatches, or data integrity failures (e.g., failed CRC checks if integrated at this level). - - `RoutingError`: Errors related to message dispatch, such as a message identifier not corresponding to any registered handler, or guards preventing access. - - `HandlerError`: Errors explicitly returned by user-defined handler functions, indicating a problem in the application's business logic. - - `IoError`: Errors from the underlying network I/O operations (e.g., connection reset, broken pipe). - - `ExtractorError`: Errors occurring during the data extraction process for handler parameters. +Robust and clear error handling is crucial for any network service. "wireframe" +will provide a comprehensive error handling strategy. + +- **Comprehensive Error Enum(s)**: Specific error types will be defined for + different stages of processing to provide clear diagnostics: + + - `FramingError`: Errors occurring during the parsing of byte streams into + frames or the construction of frames from outgoing messages (e.g., + incomplete frames, invalid length prefixes, delimiter mismatches). + - `SerializationError` / `DeserializationError`: Errors originating from the + underlying (de)serialization library (e.g., `wire-rs`, `bincode`). This + would include issues like malformed payloads, type mismatches, or data + integrity failures (e.g., failed CRC checks if integrated at this level). + - `RoutingError`: Errors related to message dispatch, such as a message + identifier not corresponding to any registered handler, or guards preventing + access. + - `HandlerError`: Errors explicitly returned by user-defined handler + functions, indicating a problem in the application's business logic. + - `IoError`: Errors from the underlying network I/O operations (e.g., + connection reset, broken pipe). + - `ExtractorError`: Errors occurring during the data extraction process for + handler parameters. - `MiddlewareError`: Errors originating from middleware components. -- `WireframeError`: A top-level public error enum will be defined to encompass all possible errors that can occur within the "wireframe" system. This provides a single error type that users can match on for top-level error management. +- `WireframeError`: A top-level public error enum will be defined to encompass + all possible errors that can occur within the "wireframe" system. This + provides a single error type that users can match on for top-level error + management. Rust - ``` + ````rustrust pub enum WireframeError { Io(std::io::Error), Framing(FramingError), @@ -517,87 +984,114 @@ Robust and clear error handling is crucial for any network service. "wireframe" Middleware(Box), //... other variants } - - ``` -- **Error Conversion**: Standard `From` traits will be implemented to allow easy conversion from specific error types (and common standard library errors like `std::io::Error`) into `WireframeError`. This simplifies error propagation within the library and in user code. + ```rust + + ```` + +- **Error Conversion**: Standard `From` traits will be implemented to allow easy + conversion from specific error types (and common standard library errors like + `std::io::Error`) into `WireframeError`. This simplifies error propagation + within the library and in user code. - **Error Propagation**: - - Errors from extractors will typically prevent the handler from being called, and "wireframe" will handle the error (e.g., log it and potentially close the connection). + - Errors from extractors will typically prevent the handler from being called, + and "wireframe" will handle the error (e.g., log it and potentially close + the connection). - Errors returned by handlers (as `Result::Err`) will be caught and processed. - Middleware can intercept errors or generate its own. -- **Custom Error Responses**: If the specific binary protocol supports sending error messages back to the client, "wireframe" should provide a mechanism for handlers or middleware to return such protocol-specific error responses. This might involve a special `Responder`-like trait for error types, similar to how Actix Web's `ResponseError` trait allows custom types to be converted into HTTP error responses.4 By default, unhandled errors or errors that cannot be translated into a protocol-specific response might result in logging the error and closing the connection. - -A well-defined error strategy is paramount for building resilient network applications. It simplifies debugging by providing clear information about the source and nature of failures. Actix Web's `ResponseError` trait 4 serves as a good model for how errors can be consistently processed and potentially translated into meaningful responses for the client. "wireframe" will adopt similar principles to ensure that errors are handled gracefully and informatively. +- **Custom Error Responses**: If the specific binary protocol supports sending + error messages back to the client, "wireframe" should provide a mechanism for + handlers or middleware to return such protocol-specific error responses. This + might involve a special `Responder`-like trait for error types, similar to how + Actix Web's `ResponseError` trait allows custom types to be converted into + HTTP error responses.4 By default, unhandled errors or errors that cannot be + translated into a protocol-specific response might result in logging the error + and closing the connection. + +A well-defined error strategy is paramount for building resilient network +applications. It simplifies debugging by providing clear information about the +source and nature of failures. Actix Web's `ResponseError` trait 4 serves as a +good model for how errors can be consistently processed and potentially +translated into meaningful responses for the client. "wireframe" will adopt +similar principles to ensure that errors are handled gracefully and +informatively. ### 5.6. Illustrative API Usage Examples -To demonstrate the intended simplicity and the Actix-Web-inspired API, concrete examples are invaluable. They make the abstract design tangible and showcase how "wireframe" aims to reduce source code complexity. +To demonstrate the intended simplicity and the Actix-Web-inspired API, concrete +examples are invaluable. They make the abstract design tangible and showcase how +"wireframe" aims to reduce source code complexity. - **Example 1: Simple Echo Protocol** - This example illustrates a basic server that echoes back any string it receives. + This example illustrates a basic server that echoes back any string it + receives. - 1. **Message Definitions** (assuming `wire-rs` with derive macros, or `bincode`/`postcard` with Serde): + 1. **Message Definitions** (assuming `wire-rs` with derive macros, or + `bincode`/`postcard` with Serde): Rust - ``` + ````rustrust // Crate: my_protocol_messages.rs // Using bincode and serde for this example use serde::{Serialize, Deserialize}; - + # pub struct EchoRequest { pub payload: String, } - + # pub struct EchoResponse { pub original_payload: String, pub echoed_at: u64, // Example: timestamp } - - ``` - 2. **Frame Processor Implementation** (Simple Length-Prefixed Framing using `tokio-util`): + ```rust + + ```` + + 1. **Frame Processor Implementation** (Simple Length-Prefixed Framing using + `tokio-util`): Rust - ``` + ````rustrust // Crate: my_frame_processor.rs use bytes::{BytesMut, Buf, BufMut}; use tokio_util::codec::{Encoder, Decoder}; use std::io; - + pub struct LengthPrefixedCodec; - + impl Decoder for LengthPrefixedCodec { type Item = BytesMut; // Raw frame payload type Error = io::Error; - + fn decode(&mut self, src: &mut BytesMut) -> Result, Self::Error> { if src.len() < 4 { return Ok(None); } // Not enough data for length prefix - + let mut length_bytes = [0u8; 4]; length_bytes.copy_from_slice(&src[..4]); let length = u32::from_be_bytes(length_bytes) as usize; - + if src.len() < 4 + length { src.reserve(4 + length - src.len()); return Ok(None); // Not enough data for full frame } - + src.advance(4); // Consume length prefix Ok(Some(src.split_to(length))) } } - + impl> Encoder for LengthPrefixedCodec { type Error = io::Error; - + fn encode(&mut self, item: T, dst: &mut BytesMut) -> Result<(), Self::Error> { let data = item.as_ref(); dst.reserve(4 + data.len()); @@ -606,26 +1100,29 @@ To demonstrate the intended simplicity and the Actix-Web-inspired API, concrete Ok(()) } } - - ``` - (Note: "wireframe" would abstract the direct use of `Encoder`/`Decoder` behind its own `FrameProcessor` trait or provide helpers.) + ```rust + + (Note: "wireframe" would abstract the direct use of `Encoder`/`Decoder` + behind its own `FrameProcessor` trait or provide helpers.) + + ```` - 3. **Server Setup and Handler**: + 1. **Server Setup and Handler**: Rust - ``` + ````rustrust // Crate: main.rs use wireframe::{WireframeApp, WireframeServer, Message, error::Result as WireframeResult, config::SerializationFormat}; use my_protocol_messages::{EchoRequest, EchoResponse}; use my_frame_processor::LengthPrefixedCodec; // Or wireframe's abstraction use std::time::{SystemTime, UNIX_EPOCH}; - + // Define a message ID enum if not using type-based routing directly # enum MyMessageType { Echo = 1 } - + // Handler function async fn handle_echo(req: Message) -> WireframeResult { println!("Received echo request with payload: {}", req.payload); @@ -634,11 +1131,11 @@ To demonstrate the intended simplicity and the Actix-Web-inspired API, concrete echoed_at: SystemTime::now().duration_since(UNIX_EPOCH).unwrap().as_secs(), }) } - + #[tokio::main] async fn main() -> std::io::Result<()> { println!("Starting echo server on 127.0.0.1:8000"); - + WireframeServer::new(|| { WireframeApp::new() //.frame_processor(LengthPrefixedCodec) // Simplified @@ -651,10 +1148,15 @@ To demonstrate the intended simplicity and the Actix-Web-inspired API, concrete .run() .await } - - ``` - This example, even in outline, demonstrates how derive macros for messages, a separable framing component, and a clear handler signature with extractors (`Message`) and a return type (`WireframeResult`) simplify server implementation. + ```rust + + This example, even in outline, demonstrates how derive macros for messages, + a separable framing component, and a clear handler signature with + extractors (`Message`) and a return type + (`WireframeResult`) simplify server implementation. + + ```` - **Example 2: Basic Chat Message Protocol** @@ -662,16 +1164,16 @@ To demonstrate the intended simplicity and the Actix-Web-inspired API, concrete Rust - ``` + ````rustrust // Crate: my_chat_messages.rs use serde::{Serialize, Deserialize}; - + # pub enum ClientMessage { Join { user_name: String }, Post { content: String }, } - + # pub enum ServerMessage { UserJoined { user_name: String }, @@ -679,42 +1181,46 @@ To demonstrate the intended simplicity and the Actix-Web-inspired API, concrete JoinError { reason: String }, } // Assume ClientMessage and ServerMessage have associated IDs for routing/serialization - - ``` - 2. **Application State**: + ```rust + + ```` + + 1. **Application State**: Rust - ``` + ````rustrust // Crate: main.rs (or app_state.rs) use std::collections::HashMap; use tokio::sync::Mutex; use wireframe::net::ConnectionId; // Hypothetical type for connection tracking - + pub struct ChatRoomState { users: HashMap, // Map connection ID to username } impl ChatRoomState { /*... methods to add/remove users, broadcast messages... */ } pub type SharedChatRoomState = wireframe::SharedState>>; - - ``` - 3. **Server Setup and Handlers**: + ```rust + + ```` + + 1. **Server Setup and Handlers**: Rust - ``` + ````rustrust // Crate: main.rs use wireframe::{WireframeApp, WireframeServer, Message, ConnectionInfo, error::Result as WireframeResult, config::SerializationFormat}; use my_chat_messages::{ClientMessage, ServerMessage}; //... use ChatRoomState, SharedChatRoomState... use std::sync::Arc; - + # enum ChatMessageType { ClientJoin = 10, ClientPost = 11 } - - + + async fn handle_join( msg: Message, // Assume it's ClientMessage::Join conn_info: ConnectionInfo, @@ -730,7 +1236,7 @@ To demonstrate the intended simplicity and the Actix-Web-inspired API, concrete } Ok(Some(ServerMessage::JoinError { reason: "Invalid Join message".to_string() })) } - + async fn handle_post( msg: Message, // Assume it's ClientMessage::Post conn_info: ConnectionInfo, @@ -743,13 +1249,15 @@ To demonstrate the intended simplicity and the Actix-Web-inspired API, concrete // println!("User '{}' posted: {}", user_name, content); } } - + #[tokio::main] async fn main() -> std::io::Result<()> { let chat_state = Arc::new(Mutex::new(ChatRoomState { users: HashMap::new() })); WireframeServer::new(move | - - ``` + + ```rust + + ```` | { @@ -777,111 +1285,249 @@ WireframeApp::new() \`\`\` -This chat example hints at how shared state (SharedChatRoomState) and connection information (ConnectionInfo) would be used, and how handlers might not always send a direct response but could trigger other actions (like broadcasting). +This chat example hints at how shared state (SharedChatRoomState) and connection +information (ConnectionInfo) would be used, and how handlers might not always +send a direct response but could trigger other actions (like broadcasting). -These examples, even if simplified, begin to illustrate how "wireframe" aims to abstract away the low-level details of network programming, allowing developers to focus on defining messages and implementing the logic for handling them. The clarity achieved by these abstractions is central to the goal of reducing source code complexity. Actix Web's "Hello, world!" 4 effectively showcases its simplicity; "wireframe" aims for similar illustrative power with its examples. +These examples, even if simplified, begin to illustrate how "wireframe" aims to +abstract away the low-level details of network programming, allowing developers +to focus on defining messages and implementing the logic for handling them. The +clarity achieved by these abstractions is central to the goal of reducing source +code complexity. Actix Web's "Hello, world!" 4 effectively showcases its +simplicity; "wireframe" aims for similar illustrative power with its examples. ## 6. Addressing Source Code Complexity -A primary motivation for "wireframe" is to reduce the inherent source code complexity often encountered when developing systems that communicate over custom binary protocols. The inaccessibility of the `leynos/mxd` repository 7 prevents a direct before-and-after comparison, but we can identify common sources of complexity in such projects and articulate how "wireframe's" design choices aim to mitigate them. +A primary motivation for "wireframe" is to reduce the inherent source code +complexity often encountered when developing systems that communicate over +custom binary protocols. The inaccessibility of the `leynos/mxd` repository 7 +prevents a direct before-and-after comparison, but we can identify common +sources of complexity in such projects and articulate how "wireframe's" design +choices aim to mitigate them. **Common Sources of Complexity in Binary Protocol Implementations**: -- **Manual Byte Manipulation**: Directly reading and writing bytes, handling endianness conversions, bit shifting, managing offsets within buffers, and ensuring correct data alignment are tedious and highly error-prone. -- **Stateful Frame Assembly**: When using stream-based transports like TCP, developers must manually handle partial reads, buffer incoming data, and implement logic to detect and extract complete frames from the byte stream. This state management can become complex. -- **Monolithic Dispatch Logic**: Routing incoming messages to the correct handler often results in large `match` statements or deeply nested if-else chains based on message type codes, which can be difficult to maintain and extend. -- **Boilerplate for Network Operations**: Setting up network listeners, accepting new connections, managing connection lifecycles (including graceful shutdown and error handling), and spawning tasks or threads for concurrent connection processing involves substantial boilerplate code. -- **Intertwined Concerns**: Business logic often becomes tightly coupled with low-level networking details, (de)serialization code, and framing logic, making the codebase harder to understand, test, and modify. -- **Error Handling**: Managing and propagating errors from various sources (I/O, deserialization, framing, application logic) in a consistent and robust manner can be challenging. +- **Manual Byte Manipulation**: Directly reading and writing bytes, handling + endianness conversions, bit shifting, managing offsets within buffers, and + ensuring correct data alignment are tedious and highly error-prone. +- **Stateful Frame Assembly**: When using stream-based transports like TCP, + developers must manually handle partial reads, buffer incoming data, and + implement logic to detect and extract complete frames from the byte stream. + This state management can become complex. +- **Monolithic Dispatch Logic**: Routing incoming messages to the correct + handler often results in large `match` statements or deeply nested if-else + chains based on message type codes, which can be difficult to maintain and + extend. +- **Boilerplate for Network Operations**: Setting up network listeners, + accepting new connections, managing connection lifecycles (including graceful + shutdown and error handling), and spawning tasks or threads for concurrent + connection processing involves substantial boilerplate code. +- **Intertwined Concerns**: Business logic often becomes tightly coupled with + low-level networking details, (de)serialization code, and framing logic, + making the codebase harder to understand, test, and modify. +- **Error Handling**: Managing and propagating errors from various sources (I/O, + deserialization, framing, application logic) in a consistent and robust manner + can be challenging. **How "wireframe" Abstractions Simplify These Areas**: - Automated (De)serialization via Derive Macros: - By relying on wire-rs (ideally with derivable Encode/Decode traits) or alternatives like bincode/postcard that offer #, "wireframe" automates the conversion between Rust structs/enums and byte sequences. This eliminates almost all manual byte manipulation, a major source of complexity and bugs. Developers define their message structures, and the library handles the low-level details. + By relying on wire-rs (ideally with derivable Encode/Decode traits) or + alternatives like bincode/postcard that offer #, "wireframe" automates the + conversion between Rust structs/enums and byte sequences. This eliminates + almost all manual byte manipulation, a major source of complexity and bugs. + Developers define their message structures, and the library handles the + low-level details. - Declarative Routing: - The proposed #\[message_handler(...)\] attribute macro or WireframeApp::route(...) method provides a declarative way to map message types or identifiers to handler functions. This replaces verbose manual dispatch logic (e.g., large match statements) with clear, high-level definitions, making the overall message flow easier to understand and manage. + The proposed #[message_handler(...)] attribute macro or + WireframeApp::route(...) method provides a declarative way to map message + types or identifiers to handler functions. This replaces verbose manual + dispatch logic (e.g., large match statements) with clear, high-level + definitions, making the overall message flow easier to understand and manage. - Extractors for Decoupled Data Access: - Inspired by Actix Web, "wireframe" extractors (Message<T>, ConnectionInfo, SharedState<T>, and custom extractors) decouple the process of obtaining data for a handler from the handler's core business logic. Handlers simply declare what data they need as function parameters, and "wireframe" takes care of providing it. This makes handlers cleaner, more focused, and easier to test in isolation. + Inspired by Actix Web, "wireframe" extractors (Message\, ConnectionInfo, + SharedState\, and custom extractors) decouple the process of obtaining data + for a handler from the handler's core business logic. Handlers simply declare + what data they need as function parameters, and "wireframe" takes care of + providing it. This makes handlers cleaner, more focused, and easier to test in + isolation. - Middleware for Modular Cross-Cutting Concerns: - The middleware system allows common functionalities like logging, authentication, metrics, or even message transformations to be implemented as separate, reusable components. This keeps such concerns out of the primary handler logic, adhering to the principle of separation of concerns and reducing clutter in business-critical code. + The middleware system allows common functionalities like logging, + authentication, metrics, or even message transformations to be implemented as + separate, reusable components. This keeps such concerns out of the primary + handler logic, adhering to the principle of separation of concerns and + reducing clutter in business-critical code. - Managed Connection Lifecycle and Task Spawning: - The WireframeServer component will abstract away the complexities of setting up network listeners (e.g., TCP listeners via Tokio), accepting incoming connections, and managing the lifecycle of each connection (including reading, writing, and handling disconnections). For each connection, it will typically spawn an asynchronous task to handle message processing, isolating connection-specific logic. This frees the developer from writing significant networking boilerplate. + The WireframeServer component will abstract away the complexities of setting + up network listeners (e.g., TCP listeners via Tokio), accepting incoming + connections, and managing the lifecycle of each connection (including reading, + writing, and handling disconnections). For each connection, it will typically + spawn an asynchronous task to handle message processing, isolating + connection-specific logic. This frees the developer from writing significant + networking boilerplate. - Trait-Based Framing (FrameProcessor): - By allowing users to define or select a FrameProcessor implementation, "wireframe" separates the logic of how bytes are grouped into frames from the rest of the application. This means users can plug in different framing strategies (length-prefixed, delimiter-based, etc.) without altering their message definitions or handler logic. The library itself can provide common framing implementations. - -These abstractions collectively contribute to code that is not only less verbose but also more readable, maintainable, and testable. By reducing complexity in these common areas, "wireframe" allows developers to concentrate more on the unique aspects of their application's protocol and business logic. This focus, in turn, can lead to faster development cycles and a lower incidence of bugs. While Rust's inherent safety features prevent many classes of memory-related errors 1, logical errors in protocol implementation remain a significant challenge. By providing well-tested, high-level abstractions for common but error-prone tasks like framing and (de)serialization, "wireframe" aims to help developers avoid entire categories of these logical bugs, leading to more robust systems. Ultimately, a simpler, more intuitive library enhances developer productivity and allows teams to build and iterate on binary protocol-based applications more effectively. + By allowing users to define or select a FrameProcessor implementation, + "wireframe" separates the logic of how bytes are grouped into frames from the + rest of the application. This means users can plug in different framing + strategies (length-prefixed, delimiter-based, etc.) without altering their + message definitions or handler logic. The library itself can provide common + framing implementations. + +These abstractions collectively contribute to code that is not only less verbose +but also more readable, maintainable, and testable. By reducing complexity in +these common areas, "wireframe" allows developers to concentrate more on the +unique aspects of their application's protocol and business logic. This focus, +in turn, can lead to faster development cycles and a lower incidence of bugs. +While Rust's inherent safety features prevent many classes of memory-related +errors 1, logical errors in protocol implementation remain a significant +challenge. By providing well-tested, high-level abstractions for common but +error-prone tasks like framing and (de)serialization, "wireframe" aims to help +developers avoid entire categories of these logical bugs, leading to more robust +systems. Ultimately, a simpler, more intuitive library enhances developer +productivity and allows teams to build and iterate on binary protocol-based +applications more effectively. ## 7. Future Development and Roadmap -While the initial design focuses on core functionality for reducing complexity in frame-based binary protocol development, several avenues for future development could enhance "wireframe's" capabilities and broaden its applicability. +While the initial design focuses on core functionality for reducing complexity +in frame-based binary protocol development, several avenues for future +development could enhance "wireframe's" capabilities and broaden its +applicability. - **Transport Agnosticism and Broader Protocol Support**: - - **UDP Support**: Explicit, first-class support for UDP-based protocols, adapting the routing and handler model to connectionless message passing. This would involve a different server setup (e.g., `UdpWireframeServer`) and potentially different extractor types relevant to datagrams (e.g., source address). - - **Other Transports**: Exploration of support for other transport layers where frame-based binary messaging is relevant. While WebSockets are often text-based (JSON), they can carry binary messages; `message-io` lists Ws separately from `FramedTcp` 17, suggesting distinct handling. - - **In-Process Communication**: Adapting "wireframe" concepts for efficient, type-safe in-process message passing, perhaps using Tokio's MPSC channels as a "transport." + - **UDP Support**: Explicit, first-class support for UDP-based protocols, + adapting the routing and handler model to connectionless message passing. + This would involve a different server setup (e.g., `UdpWireframeServer`) and + potentially different extractor types relevant to datagrams (e.g., source + address). + - **Other Transports**: Exploration of support for other transport layers + where frame-based binary messaging is relevant. While WebSockets are often + text-based (JSON), they can carry binary messages; `message-io` lists Ws + separately from `FramedTcp` 17, suggesting distinct handling. + - **In-Process Communication**: Adapting "wireframe" concepts for efficient, + type-safe in-process message passing, perhaps using Tokio's MPSC channels as + a "transport." - **Advanced Framing Options**: - - **Built-in Codecs**: Providing a richer set of built-in `FrameProcessor` implementations for common framing techniques beyond simple length-prefixing, such as: - - COBS (Consistent Overhead Byte Stuffing), which `postcard` already supports for its serialization output.12 + - **Built-in Codecs**: Providing a richer set of built-in `FrameProcessor` + implementations for common framing techniques beyond simple + length-prefixing, such as: + - COBS (Consistent Overhead Byte Stuffing), which `postcard` already + supports for its serialization output.12 - SLIP (Serial Line Internet Protocol) framing. - - Protocols using fixed-size frames or more complex header/delimiter patterns. - - **Composable Framing**: Allowing framing strategies to be composed or layered. + - Protocols using fixed-size frames or more complex header/delimiter + patterns. + - **Composable Framing**: Allowing framing strategies to be composed or + layered. - Schema Evolution and Message Versioning: - Handling changes to message structures over time (schema evolution) is a common challenge in long-lived systems. While bincode and postcard generally require data structures to be known and fixed for a given version 10, future work could involve: - - - Guidance or helper utilities for implementing version negotiation within protocols built on "wireframe". - - If using a (de)serialization format that inherently supports schema evolution (e.g., Protocol Buffers, Avro – though these are outside the current scope of `wire-rs`/`bincode`/`postcard`), providing integration points or wrappers. - - Exploring how `wire-rs` itself might be extended to facilitate optional fields or forward/backward compatibility, or how "wireframe" could layer such capabilities. + Handling changes to message structures over time (schema evolution) is a + common challenge in long-lived systems. While bincode and postcard generally + require data structures to be known and fixed for a given version 10, future + work could involve: + + - Guidance or helper utilities for implementing version negotiation within + protocols built on "wireframe". + - If using a (de)serialization format that inherently supports schema + evolution (e.g., Protocol Buffers, Avro – though these are outside the + current scope of `wire-rs`/`bincode`/`postcard`), providing integration + points or wrappers. + - Exploring how `wire-rs` itself might be extended to facilitate optional + fields or forward/backward compatibility, or how "wireframe" could layer + such capabilities. - Multiplexing and Logical Streams: - Support for protocols that multiplex multiple logical streams over a single underlying network connection. This might involve extensions to the routing mechanism and connection management to handle stream identifiers. + Support for protocols that multiplex multiple logical streams over a single + underlying network connection. This might involve extensions to the routing + mechanism and connection management to handle stream identifiers. - Flow Control and Backpressure: - Exposing more granular hooks or mechanisms for application-level flow control and backpressure management, especially for high-throughput scenarios or when dealing with producers and consumers of messages that operate at different speeds. + Exposing more granular hooks or mechanisms for application-level flow control + and backpressure management, especially for high-throughput scenarios or when + dealing with producers and consumers of messages that operate at different + speeds. - **Tooling and Developer Experience**: - **CLI Tool**: A potential command-line interface tool for tasks like: - - Generating boilerplate message definitions or handler stubs from a simple protocol description language. + - Generating boilerplate message definitions or handler stubs from a simple + protocol description language. - Basic protocol testing or debugging. - - **Enhanced Debugging Support**: Features or integrations that make it easier to inspect message flows, connection states, and errors within a "wireframe" application. - - **More Examples and Documentation**: Continuously expanding the set of examples and detailed documentation for various use cases and advanced features. + - **Enhanced Debugging Support**: Features or integrations that make it easier + to inspect message flows, connection states, and errors within a "wireframe" + application. + - **More Examples and Documentation**: Continuously expanding the set of + examples and detailed documentation for various use cases and advanced + features. - Deeper wire-rs Integration (if chosen): - If wire-rs is adopted as the primary (de)serialization library, ongoing collaboration or contributions to wire-rs could be beneficial to ensure its API and features (especially around derivable traits and context-aware deserialization) align optimally with "wireframe's" requirements. + If wire-rs is adopted as the primary (de)serialization library, ongoing + collaboration or contributions to wire-rs could be beneficial to ensure its + API and features (especially around derivable traits and context-aware + deserialization) align optimally with "wireframe's" requirements. -These potential future directions aim to evolve "wireframe" from a core routing library into a more comprehensive ecosystem for binary protocol development in Rust. Anticipating such enhancements demonstrates a commitment to the library's long-term viability and its potential to serve a growing range of user needs. Addressing common advanced requirements like schema evolution 10 or diverse transport needs early in the roadmap can guide architectural decisions to ensure future extensibility. +These potential future directions aim to evolve "wireframe" from a core routing +library into a more comprehensive ecosystem for binary protocol development in +Rust. Anticipating such enhancements demonstrates a commitment to the library's +long-term viability and its potential to serve a growing range of user needs. +Addressing common advanced requirements like schema evolution 10 or diverse +transport needs early in the roadmap can guide architectural decisions to ensure +future extensibility. ## 8. Conclusion -The "wireframe" library, as proposed in this report, aims to significantly reduce the source code complexity associated with developing applications that use arbitrary frame-based binary wire protocols in Rust. Its core value proposition lies in providing high-level abstractions, a declarative API inspired by the ergonomics of Actix Web, and leveraging efficient (de)serialization mechanisms, ideally through a `wire-rs` implementation that supports derivable traits, or proven alternatives like `bincode` or `postcard`. +The "wireframe" library, as proposed in this report, aims to significantly +reduce the source code complexity associated with developing applications that +use arbitrary frame-based binary wire protocols in Rust. Its core value +proposition lies in providing high-level abstractions, a declarative API +inspired by the ergonomics of Actix Web, and leveraging efficient +(de)serialization mechanisms, ideally through a `wire-rs` implementation that +supports derivable traits, or proven alternatives like `bincode` or `postcard`. The key design elements contributing to this complexity reduction include: -- **Automated (De)serialization**: Derive macros for message structs and enums eliminate manual, error-prone byte manipulation. -- **Declarative Routing**: Attribute macros or builder methods provide a clear and concise way to map messages to handlers. -- **Type-Safe Data Extraction**: An extractor system simplifies access to message payloads and connection context within handlers, promoting cleaner code. -- **Modular Middleware**: Cross-cutting concerns like logging and authentication can be implemented independently of core business logic. -- **Abstracted Framing and Connection Management**: The library handles low-level framing details and network connection lifecycles, allowing developers to focus on application-specific logic. - -By adopting these strategies, "wireframe" seeks to improve developer productivity, enhance code maintainability, and allow applications to fully benefit from Rust's performance and safety guarantees.1 The design prioritizes creating an intuitive and familiar experience for developers, especially those with a background in Actix Web, while remaining flexible enough to accommodate a wide variety of binary protocols. - -The ultimate success of "wireframe" will depend on the effective execution of its primary goal—complexity reduction—while maintaining the robustness and performance expected of network infrastructure software. The design presented here provides a strong foundation. Prototyping these concepts, particularly the integration with a (de)serialization library offering derivable traits and the Actix-like API components, along with gathering community feedback, will be crucial next steps to validate this approach and refine the library's features into a valuable tool for the Rust ecosystem. \ No newline at end of file +- **Automated (De)serialization**: Derive macros for message structs and enums + eliminate manual, error-prone byte manipulation. +- **Declarative Routing**: Attribute macros or builder methods provide a clear + and concise way to map messages to handlers. +- **Type-Safe Data Extraction**: An extractor system simplifies access to + message payloads and connection context within handlers, promoting cleaner + code. +- **Modular Middleware**: Cross-cutting concerns like logging and authentication + can be implemented independently of core business logic. +- **Abstracted Framing and Connection Management**: The library handles + low-level framing details and network connection lifecycles, allowing + developers to focus on application-specific logic. + +By adopting these strategies, "wireframe" seeks to improve developer +productivity, enhance code maintainability, and allow applications to fully +benefit from Rust's performance and safety guarantees.1 The design prioritizes +creating an intuitive and familiar experience for developers, especially those +with a background in Actix Web, while remaining flexible enough to accommodate a +wide variety of binary protocols. + +The ultimate success of "wireframe" will depend on the effective execution of +its primary goal—complexity reduction—while maintaining the robustness and +performance expected of network infrastructure software. The design presented +here provides a strong foundation. Prototyping these concepts, particularly the +integration with a (de)serialization library offering derivable traits and the +Actix-like API components, along with gathering community feedback, will be +crucial next steps to validate this approach and refine the library's features +into a valuable tool for the Rust ecosystem. From 9555f65a69c5333e3af22e1be25d5a85fd5b25d1 Mon Sep 17 00:00:00 2001 From: Leynos Date: Wed, 11 Jun 2025 13:27:58 +0100 Subject: [PATCH 2/2] Add task checkboxes to roadmap and clean doc --- docs/roadmap.md | 31 ++++++++++++----------- docs/rust-binary-router-library-design.md | 22 ++++++++-------- 2 files changed, 27 insertions(+), 26 deletions(-) diff --git a/docs/roadmap.md b/docs/roadmap.md index bc0d5454..268457b1 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -6,36 +6,37 @@ after formatting. Line numbers below refer to that file. ## 1. Core Library Foundations -- Lines 316-345 outline the layered architecture comprising the Transport Layer - Adapter, Framing Layer, Serialization engine, Routing engine, Handler +- [ ] Lines 316-345 outline the layered architecture comprising the Transport + Layer Adapter, Framing Layer, Serialization engine, Routing engine, Handler invocation, and Middleware chain. -- Implement derive macros or wrappers for message serialization (lines 329-333). -- Build the Actix-inspired API around `WireframeApp` and `WireframeServer` as - described in lines 586-676. +- [ ] Implement derive macros or wrappers for message serialization (lines + 329-333). +- [ ] Build the Actix-inspired API around `WireframeApp` and `WireframeServer` + as described in lines 586-676. ## 2. Middleware and Extractors -- Develop a minimal middleware system and extractor traits for payloads, +- [ ] Develop a minimal middleware system and extractor traits for payloads, connection metadata, and shared state. ## 3. Initial Examples and Documentation -- Provide examples demonstrating routing, serialization, and middleware. +- [ ] Provide examples demonstrating routing, serialization, and middleware. Document configuration and usage reflecting the API design section. ## 4. Extended Features -- Add UDP and other transport implementations (lines 1366-1379). -- Develop built-in `FrameProcessor` variants (lines 1381-1389). -- Address schema evolution and versioning strategies (lines 1394-1409). -- Investigate multiplexing and flow control mechanisms (lines 1411-1422). +- [ ] Add UDP and other transport implementations (lines 1366-1379). +- [ ] Develop built-in `FrameProcessor` variants (lines 1381-1389). +- [ ] Address schema evolution and versioning strategies (lines 1394-1409). +- [ ] Investigate multiplexing and flow control mechanisms (lines 1411-1422). ## 5. Developer Tooling -- Create a CLI for protocol scaffolding and testing (lines 1424-1429). -- Improve debugging support and expand documentation (lines 1430-1435). +- [ ] Create a CLI for protocol scaffolding and testing (lines 1424-1429). +- [ ] Improve debugging support and expand documentation (lines 1430-1435). ## 6. Community Engagement and Integration -- Collaborate with `wire-rs` for trait derivation and future enhancements (lines - 1437-1442). +- [ ] Collaborate with `wire-rs` for trait derivation and future enhancements + (lines 1437-1442). diff --git a/docs/rust-binary-router-library-design.md b/docs/rust-binary-router-library-design.md index 40085bbb..a9beb915 100644 --- a/docs/rust-binary-router-library-design.md +++ b/docs/rust-binary-router-library-design.md @@ -37,17 +37,17 @@ The design of the "wireframe" library is predicated on a multi-faceted approach: as protocol frameworks like `protocol` and `tarpc`. The Tokio ecosystem, a foundational element for asynchronous networking in Rust, was also considered. -1. **API Analysis**: The Application Programming Interface (API) of Actix Web +2. **API Analysis**: The Application Programming Interface (API) of Actix Web was studied in detail, focusing on its mechanisms for routing, request data extraction, middleware, and application state management. The objective was to identify design principles that contribute to its developer-friendliness and could be effectively translated to the context of binary protocols. -1. **Comparative Synthesis**: Learnings from the literature survey and API +3. **Comparative Synthesis**: Learnings from the literature survey and API analysis were synthesized to inform the core design decisions for "wireframe". This involved identifying common strategies for complexity reduction in Rust, such as the use of derive macros and trait-based abstractions, and evaluating how Actix Web's patterns could be adapted. -1. **Iterative Design**: Based on these findings, a conceptual design for +4. **Iterative Design**: Based on these findings, a conceptual design for "wireframe" was developed, outlining its architecture, core components, and API. This design prioritizes the user query's central goal: reducing source code complexity for arbitrary frame-based binary protocols. @@ -225,14 +225,14 @@ of "wireframe": (de)serialization, derive macros are paramount for reducing boilerplate and improving developer experience, as seen in `bin-proto`, `protocol`, `bincode`, and `postcard`. -1. **Robust (De)serialization Strategy**: The choice of `wire-rs` is contingent +2. **Robust (De)serialization Strategy**: The choice of `wire-rs` is contingent on its ability to support derivable `Encode`/`Decode` traits. If this is not feasible, well-established alternatives like `bincode` or `postcard` offer proven Serde integration and derive capabilities.10 -1. **Actix-Inspired API Patterns**: Leveraging Actix Web's patterns for routing, +3. **Actix-Inspired API Patterns**: Leveraging Actix Web's patterns for routing, data extraction (extractors), and middleware is key to achieving the desired developer-friendliness and reducing source code complexity. -1. **Asynchronous Foundation**: Integration with an asynchronous runtime like +4. **Asynchronous Foundation**: Integration with an asynchronous runtime like Tokio is non-negotiable for a modern, performant networking library in Rust.18 @@ -1055,7 +1055,7 @@ examples are invaluable. They make the abstract design tangible and showcase how ```` - 1. **Frame Processor Implementation** (Simple Length-Prefixed Framing using + 2. **Frame Processor Implementation** (Simple Length-Prefixed Framing using `tokio-util`): Rust @@ -1108,7 +1108,7 @@ examples are invaluable. They make the abstract design tangible and showcase how ```` - 1. **Server Setup and Handler**: + 3. **Server Setup and Handler**: Rust @@ -1186,7 +1186,7 @@ examples are invaluable. They make the abstract design tangible and showcase how ```` - 1. **Application State**: + 2. **Application State**: Rust @@ -1206,7 +1206,7 @@ examples are invaluable. They make the abstract design tangible and showcase how ```` - 1. **Server Setup and Handlers**: + 3. **Server Setup and Handlers**: Rust @@ -1518,7 +1518,7 @@ The key design elements contributing to this complexity reduction include: By adopting these strategies, "wireframe" seeks to improve developer productivity, enhance code maintainability, and allow applications to fully -benefit from Rust's performance and safety guarantees.1 The design prioritizes +benefit from Rust's performance and safety guarantees. The design prioritizes creating an intuitive and familiar experience for developers, especially those with a background in Actix Web, while remaining flexible enough to accommodate a wide variety of binary protocols.