Skip to content

wolfMQTT broker: ordering, persistence, offline queue, AES-GCM at rest#538

Open
dgarske wants to merge 8 commits into
wolfSSL:masterfrom
dgarske:broker_features
Open

wolfMQTT broker: ordering, persistence, offline queue, AES-GCM at rest#538
dgarske wants to merge 8 commits into
wolfSSL:masterfrom
dgarske:broker_features

Conversation

@dgarske
Copy link
Copy Markdown
Member

@dgarske dgarske commented May 13, 2026

  • Per-subscriber outbound queue with MQTT v5 Receive Maximum and inflight cap
  • Persistence hooks API + POSIX backend for sessions, subscriptions, retained messages (schema-wipe-on-mismatch, static-memory restore)
  • Offline message queue with cross-restart replay (orphan sessions, NS_OUTQ persist, v5 Session Expiry Interval)
  • Optional AES-GCM encryption at rest for persisted records
  • Fixes: CI flake, retransmit DUP=1, key cache, schema header, mqtt-sub non-blocking MQTT_CODE_CONTINUE handling

@dgarske dgarske self-assigned this May 13, 2026
Copilot AI review requested due to automatic review settings May 13, 2026 21:54
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds opt-in features to the wolfMQTT broker: per-subscriber outbound queueing with v5 Receive Maximum honoring, a hook-based persistence layer with a default POSIX backend, an offline message queue surviving reconnects/restarts, and optional AES-GCM encryption at rest. Also fixes a v5 CONNACK protocol error (Maximum QoS=2 was illegal).

Changes:

  • New per-subscriber outbound queue / ordering / inflight cap in mqtt_broker.c.
  • New persistence layer (mqtt_broker_persist.c, mqtt_broker_persist_posix.c) with schema-wipe-on-mismatch and optional AES-GCM.
  • New configure flags, broker.test cases, and CI matrix entries.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 17 comments.

Show a summary per file
File Description
wolfmqtt/mqtt_broker.h Public types/macros for outbound queue, orphan sessions, persistence hooks.
src/mqtt_broker.c Outbound queue, orphan session pool, persist shadow-write call sites, CONNACK fix.
src/mqtt_broker_persist.c Record encoders/decoders, restore, schema wipe, AES-GCM wrap/unwrap.
src/mqtt_broker_persist_posix.c Default POSIX file-based backend (kv_put/get/del/iter/sync).
src/include.am Build the new persistence sources into the broker binary.
scripts/broker.test New tests: ordering burst, persist round-trip, schema wipe, offline queue, AES-GCM.
configure.ac --enable-broker-persist and --enable-broker-persist-encrypt flags.
.github/workflows/broker-check.yml CI matrix entries for the new build configurations.
Comments suppressed due to low confidence (1)

src/mqtt_broker.c:1

  • The iterator advance sub = sub->next is now inside #ifndef WOLFMQTT_STATIC_MEMORY. This is presumably correct because the static-memory loop uses indexed iteration, but the asymmetry (and the fact that the dynamic-mode body lives just above the #ifndef and is already inside the linked-list loop) makes this very fragile to future edits. Consider keeping the sub = sub->next advance outside the orphan-handling guard, or refactoring so the loop structure is single-version per memory mode.
/* mqtt_broker.c

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/mqtt_broker_persist.c Outdated
Comment thread src/mqtt_broker_persist.c Outdated
Comment thread src/mqtt_broker_persist.c
Comment thread src/mqtt_broker_persist.c Outdated
Comment thread src/mqtt_broker.c
Comment thread src/mqtt_broker_persist.c
Comment thread src/mqtt_broker_persist_posix.c Outdated
Comment thread src/mqtt_broker.c
Comment thread configure.ac
Comment thread src/mqtt_broker_persist.c
@dgarske dgarske requested a review from Copilot May 14, 2026 21:15
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 28 comments.

Comment thread src/mqtt_broker.c
Comment thread src/mqtt_broker.c
Comment thread src/mqtt_broker.c
Comment thread src/mqtt_broker.c
Comment thread src/mqtt_broker_persist.c
Comment thread src/mqtt_broker.c
Comment thread src/mqtt_broker.c
Comment thread src/mqtt_broker_persist.c
Comment thread .github/workflows/broker-check.yml
Comment thread src/mqtt_broker.c
@dgarske dgarske force-pushed the broker_features branch from 2a1d383 to d05b204 Compare May 14, 2026 22:47
When broker.test fails, the autotools test-suite.log only shows the
test-runner's last few lines, not the per-test broker/sub/pub logs
created under $TMPDIR/tmp.XXXXXX. Expand the "Show logs on failure"
step to walk both $TMPDIR/tmp.* and /tmp/tmp.* and cat every .log,
and upload the same set as a build artifact so the failure on the
macos-latest ARM runner can be diagnosed off-line.
@dgarske dgarske force-pushed the broker_features branch from 807fd2e to 7361662 Compare May 14, 2026 23:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants