From 8fb2fc2436420e60f4ac2042b0c23549b431cef3 Mon Sep 17 00:00:00 2001 From: Leynos Date: Tue, 24 Jun 2025 20:36:59 +0100 Subject: [PATCH 1/3] Clarify connection actor diagrams --- .../asynchronous-outbound-messaging-design.md | 40 +++++++++++-------- 1 file changed, 24 insertions(+), 16 deletions(-) diff --git a/docs/asynchronous-outbound-messaging-design.md b/docs/asynchronous-outbound-messaging-design.md index 24ea48ce..e0d4bb17 100644 --- a/docs/asynchronous-outbound-messaging-design.md +++ b/docs/asynchronous-outbound-messaging-design.md @@ -32,25 +32,26 @@ protocols. Only the queue management utilities in `src/push.rs` exist at present. The connection actor and its write loop are still to be implemented. The remaining sections describe how to build that actor from first principles using the biased -`select!` loop presented in [Section 3](#3-core-architecture-the-connection-actor). +`select!` loop presented in +[Section 3](#3-core-architecture-the-connection-actor). ## 2. Design Goals & Requirements The implementation must satisfy the following core requirements: -| ID | Requirement | +| ID | Requirement | | --- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | -| G1 | Any async task must be able to push frames to a live connection. | -| G2 | Ordering-safety: Pushed frames must interleave correctly with normal request/response traffic and respect any per-message sequencing rules. | -| G3 | Back-pressure: Writers must block (or fail fast) when the peer cannot drain the socket, preventing unbounded memory consumption. | -| G4 | Generic—independent of any particular protocol; usable by both servers and clients built on wireframe. | -| G5 | Preserve the simple “return a reply” path for code that does not need pushes, ensuring backward compatibility and low friction for existing users. | +| G1 | Any async task must be able to push frames to a live connection. | +| G2 | Ordering-safety: Pushed frames must interleave correctly with normal request/response traffic and respect any per-message sequencing rules. | +| G3 | Back-pressure: Writers must block (or fail fast) when the peer cannot drain the socket, preventing unbounded memory consumption. | +| G4 | Generic—independent of any particular protocol; usable by both servers and clients built on wireframe. | +| G5 | Preserve the simple “return a reply” path for code that does not need pushes, ensuring backward compatibility and low friction for existing users. | ## 3. Core Architecture: The Connection Actor The foundation of this design is the **actor-per-connection** model, where each network connection is managed by a dedicated, isolated asynchronous task. This -approach serialises all I/O for a given connection, eliminating the need for +approach serializes all I/O for a given connection, eliminating the need for complex locking and simplifying reasoning about concurrency. In previous iterations, a connection's logic lived in short-lived worker tasks @@ -69,7 +70,7 @@ manage two distinct, bounded `tokio::mpsc` channels for pushed frames: messages like heartbeats, session control notifications, or protocol-level pings. -2. `low_priority_push_rx: mpsc::Receiver`: For standard, non-urgent +1. `low_priority_push_rx: mpsc::Receiver`: For standard, non-urgent background messages like log forwarding or secondary status updates. The bounded nature of these channels provides an inherent and robust @@ -89,13 +90,13 @@ The polling order will be: 1. **Graceful Shutdown Signal:** The `CancellationToken` will be checked first to ensure immediate reaction to a server-wide shutdown request. -2. **High-Priority Push Channel:** Messages from `high_priority_push_rx` will be +1. **High-Priority Push Channel:** Messages from `high_priority_push_rx` will be drained next. -3. **Low-Priority Push Channel:** Messages from `low_priority_push_rx` will be +1. **Low-Priority Push Channel:** Messages from `low_priority_push_rx` will be processed after all high-priority messages. -4. **Handler Response Stream:** Frames from the active request's +1. **Handler Response Stream:** Frames from the active request's `Response::Stream` will be processed last. ```rust @@ -164,13 +165,16 @@ classDiagram sequenceDiagram participant Client participant ConnectionActor + participant Outbox participant Socket Client->>ConnectionActor: Initiate connection/request Note over ConnectionActor: Manages high/low priority queues - ConnectionActor->>ConnectionActor: enqueue outbound frame + ConnectionActor->>Outbox: enqueue outbound frame + Outbox->>ConnectionActor: dequeue frame ConnectionActor->>Socket: Write outbound frame Socket-->>Client: Delivers outbound message + Note over Outbox: Holds frames while socket busy ``` ## 4. Public API Surface @@ -420,10 +424,12 @@ sequenceDiagram participant AppTask as Application Task participant SessionRegistry participant ConnectionActor + participant Outbox participant Socket AppTask->>SessionRegistry: get PushHandle for session AppTask->>ConnectionActor: push(OK packet or LOCAL INFILE) - ConnectionActor->>ConnectionActor: queue frame in outbox + ConnectionActor->>Outbox: enqueue frame + Outbox->>ConnectionActor: dequeue frame ConnectionActor->>Socket: write frame (when idle or after command completes) ``` @@ -437,9 +443,11 @@ even while a large response stream is in progress. sequenceDiagram participant Timer as Heart-beat Timer participant ConnectionActor + participant Outbox participant Socket Timer->>ConnectionActor: push_high_priority(Ping frame) - ConnectionActor->>ConnectionActor: enqueue Ping in high-priority queue + ConnectionActor->>Outbox: enqueue Ping in high-priority queue + Outbox->>ConnectionActor: dequeue Ping ConnectionActor->>Socket: write Ping frame (even during response stream) ``` @@ -460,7 +468,7 @@ sequenceDiagram alt Queue not full ConnectionActor->>Socket: write PUBLISH frame else Queue full - ConnectionActor->>ConnectionActor: drop frame + Note over ConnectionActor: Drop frame due to full queue end ``` From db3941eaafe2b8259c969d5d12a0ae738df8b7f2 Mon Sep 17 00:00:00 2001 From: Leynos Date: Tue, 24 Jun 2025 22:42:38 +0100 Subject: [PATCH 2/3] Address review feedback --- .../asynchronous-outbound-messaging-design.md | 32 +++++++++---------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/docs/asynchronous-outbound-messaging-design.md b/docs/asynchronous-outbound-messaging-design.md index e0d4bb17..754c9ef4 100644 --- a/docs/asynchronous-outbound-messaging-design.md +++ b/docs/asynchronous-outbound-messaging-design.md @@ -39,13 +39,13 @@ sections describe how to build that actor from first principles using the biased The implementation must satisfy the following core requirements: -| ID | Requirement | +| ID | Requirement | | --- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | -| G1 | Any async task must be able to push frames to a live connection. | -| G2 | Ordering-safety: Pushed frames must interleave correctly with normal request/response traffic and respect any per-message sequencing rules. | -| G3 | Back-pressure: Writers must block (or fail fast) when the peer cannot drain the socket, preventing unbounded memory consumption. | -| G4 | Generic—independent of any particular protocol; usable by both servers and clients built on wireframe. | -| G5 | Preserve the simple “return a reply” path for code that does not need pushes, ensuring backward compatibility and low friction for existing users. | +| G1 | Any async task must be able to push frames to a live connection. | +| G2 | Ordering-safety: Pushed frames must interleave correctly with normal request/response traffic and respect any per-message sequencing rules. | +| G3 | Back-pressure: Writers must block (or fail fast) when the peer cannot drain the socket, preventing unbounded memory consumption. | +| G4 | Generic—independent of any particular protocol; usable by both servers and clients built on wireframe. | +| G5 | Preserve the simple “return a reply” path for code that does not need pushes, ensuring backward compatibility and low friction for existing users. | ## 3. Core Architecture: The Connection Actor @@ -59,7 +59,7 @@ spawned per request. Converting those workers into long-running actors allows `wireframe` to maintain per-connection state—such as sequence counters, command metadata, and pending pushes—without cross-task sharing. Handlers now send commands back to the actor instead of writing directly to the socket, -centralising all output in one place. +centralizing all output in one place. ### 3.1 Prioritised Message Queues @@ -70,7 +70,7 @@ manage two distinct, bounded `tokio::mpsc` channels for pushed frames: messages like heartbeats, session control notifications, or protocol-level pings. -1. `low_priority_push_rx: mpsc::Receiver`: For standard, non-urgent +2. `low_priority_push_rx: mpsc::Receiver`: For standard, non-urgent background messages like log forwarding or secondary status updates. The bounded nature of these channels provides an inherent and robust @@ -90,13 +90,13 @@ The polling order will be: 1. **Graceful Shutdown Signal:** The `CancellationToken` will be checked first to ensure immediate reaction to a server-wide shutdown request. -1. **High-Priority Push Channel:** Messages from `high_priority_push_rx` will be +2. **High-Priority Push Channel:** Messages from `high_priority_push_rx` will be drained next. -1. **Low-Priority Push Channel:** Messages from `low_priority_push_rx` will be +3. **Low-Priority Push Channel:** Messages from `low_priority_push_rx` will be processed after all high-priority messages. -1. **Handler Response Stream:** Frames from the active request's +4. **Handler Response Stream:** Frames from the active request's `Response::Stream` will be processed last. ```rust @@ -171,10 +171,10 @@ sequenceDiagram Client->>ConnectionActor: Initiate connection/request Note over ConnectionActor: Manages high/low priority queues ConnectionActor->>Outbox: enqueue outbound frame - Outbox->>ConnectionActor: dequeue frame + ConnectionActor->>Outbox: dequeue frame ConnectionActor->>Socket: Write outbound frame Socket-->>Client: Delivers outbound message - Note over Outbox: Holds frames while socket busy + Note over Outbox: Holds frames while the socket is busy ``` ## 4. Public API Surface @@ -429,7 +429,7 @@ sequenceDiagram AppTask->>SessionRegistry: get PushHandle for session AppTask->>ConnectionActor: push(OK packet or LOCAL INFILE) ConnectionActor->>Outbox: enqueue frame - Outbox->>ConnectionActor: dequeue frame + ConnectionActor->>Outbox: dequeue frame ConnectionActor->>Socket: write frame (when idle or after command completes) ``` @@ -447,7 +447,7 @@ sequenceDiagram participant Socket Timer->>ConnectionActor: push_high_priority(Ping frame) ConnectionActor->>Outbox: enqueue Ping in high-priority queue - Outbox->>ConnectionActor: dequeue Ping + ConnectionActor->>Outbox: dequeue Ping ConnectionActor->>Socket: write Ping frame (even during response stream) ``` @@ -468,7 +468,7 @@ sequenceDiagram alt Queue not full ConnectionActor->>Socket: write PUBLISH frame else Queue full - Note over ConnectionActor: Drop frame due to full queue + Note over ConnectionActor: Drop frame due to full queue. end ``` From 189db9b1a3e3362783a30e9227ebb7a56c04bb67 Mon Sep 17 00:00:00 2001 From: Leynos Date: Tue, 24 Jun 2025 23:10:36 +0100 Subject: [PATCH 3/3] Clarify dequeue interaction --- docs/asynchronous-outbound-messaging-design.md | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/docs/asynchronous-outbound-messaging-design.md b/docs/asynchronous-outbound-messaging-design.md index 754c9ef4..2e558797 100644 --- a/docs/asynchronous-outbound-messaging-design.md +++ b/docs/asynchronous-outbound-messaging-design.md @@ -171,10 +171,11 @@ sequenceDiagram Client->>ConnectionActor: Initiate connection/request Note over ConnectionActor: Manages high/low priority queues ConnectionActor->>Outbox: enqueue outbound frame - ConnectionActor->>Outbox: dequeue frame + ConnectionActor->>Outbox: dequeue request + Outbox-->>ConnectionActor: frame ConnectionActor->>Socket: Write outbound frame Socket-->>Client: Delivers outbound message - Note over Outbox: Holds frames while the socket is busy + Note over Outbox: Holds frames while the socket is busy. ``` ## 4. Public API Surface @@ -429,7 +430,8 @@ sequenceDiagram AppTask->>SessionRegistry: get PushHandle for session AppTask->>ConnectionActor: push(OK packet or LOCAL INFILE) ConnectionActor->>Outbox: enqueue frame - ConnectionActor->>Outbox: dequeue frame + ConnectionActor->>Outbox: dequeue request + Outbox-->>ConnectionActor: frame ConnectionActor->>Socket: write frame (when idle or after command completes) ``` @@ -447,7 +449,8 @@ sequenceDiagram participant Socket Timer->>ConnectionActor: push_high_priority(Ping frame) ConnectionActor->>Outbox: enqueue Ping in high-priority queue - ConnectionActor->>Outbox: dequeue Ping + ConnectionActor->>Outbox: dequeue request + Outbox-->>ConnectionActor: Ping ConnectionActor->>Socket: write Ping frame (even during response stream) ```