feat(500-application): add 514-wasm-msg-to-dss WASM map operator with DSS enrichment pattern#356
Merged
feat(500-application): add 514-wasm-msg-to-dss WASM map operator with DSS enrichment pattern#356
Conversation
Add msg-to-dss-key WASM operator that writes JSON messages to the AIO Distributed State Store under a configurable key extracted via JSON Pointer (RFC 6901) with configurable TTL and passthrough behavior. - Implement operator with OnceLock config, init validation, state store write, and 22 unit tests (all passing) - Create graph definition, build/push scripts, and blueprint tfvars example - Document enrichment stitching pattern with syntax - Update src/500-application/README.md with new component entry
🔗 - Generated by Copilot
📚 Documentation Health ReportGenerated on: 2026-04-07 13:45:05 UTC 📈 Documentation Statistics
🏗️ Three-Tree Architecture Status
🔍 Quality Metrics
This report is automatically generated by the Documentation Automation workflow. |
📚 Documentation Health ReportGenerated on: 2026-04-07 14:37:10 UTC 📈 Documentation Statistics
🏗️ Three-Tree Architecture Status
🔍 Quality Metrics
This report is automatically generated by the Documentation Automation workflow. |
📚 Documentation Health ReportGenerated on: 2026-04-07 20:00:18 UTC 📈 Documentation Statistics
🏗️ Three-Tree Architecture Status
🔍 Quality Metrics
This report is automatically generated by the Documentation Automation workflow. |
WilliamBerryiii
approved these changes
Apr 7, 2026
…e_config in msg-to-dss-key - Add food manufacturing lot traceability use case with ASCII diagram to README - Extract parse_config() from init for validation logic reuse and testability - Add comprehensive parse_config unit tests covering valid and invalid inputs 🚀 - Generated by Copilot
📚 Documentation Health ReportGenerated on: 2026-04-08 06:40:39 UTC 📈 Documentation Statistics
🏗️ Three-Tree Architecture Status
🔍 Quality Metrics
This report is automatically generated by the Documentation Automation workflow. |
…uild pipeline 🔒 - Generated by Copilot
📚 Documentation Health ReportGenerated on: 2026-04-09 08:20:44 UTC 📈 Documentation Statistics
🏗️ Three-Tree Architecture Status
🔍 Quality Metrics
This report is automatically generated by the Documentation Automation workflow. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Added
src/500-application/514-wasm-msg-to-dss— a reusable WASM map operator that writes any incoming JSON message to the AIO Distributed State Store (DSS) under a configurable key, then passes the message through unchanged to downstream nodes. This operator enables no-code DSS enrichment in any dataflow graph using AIO's built-in transform capabilities.The operator extracts keys using RFC 6901 JSON Pointer syntax, supports optional TTL and namespace prefix configuration, and implements a passthrough-first architecture where the original message is always returned regardless of DSS write outcome.
Operator Implementation
The core operator in lib.rs (423 lines) accepts four configuration parameters — keyPath and ttlSeconds (required), keyPrefix and onMissing (optional) — validated exhaustively at init time. The operator extracts scalar values at the specified JSON Pointer path, constructs a namespaced key, writes the full payload to DSS, and returns the message unchanged. Graceful degradation handles non-UTF8 payloads, invalid JSON, missing keys, and DSS write failures with warning logs and passthrough behavior. 22 unit tests cover value conversion, JSON Pointer extraction, and key construction.
Build and Deployment
build-wasm.sh compiles to the
wasm32-wasip2target with an optimized release profile (LTO,panic=abort, size-optimized, stripped binary). push-to-acr.sh publishes the WASM module and graph definition as separate ORAS artifacts to Azure Container Registry, injecting the version from Cargo.toml into the graph YAML at push time.Blueprint Integration
A dataflow-graphs-msg-to-dss.tfvars.example file was added to
blueprints/full-single-node-cluster/terraform/demonstrating a complete three-node dataflow graph: MQTT source, msg-to-dss-key operator, and MQTT destination. The example indexes device state by/deviceIdwith adevice:prefix and 24-hour TTL, and includes commented alternatives for nested keys, array-indexed keys, and strict error handling.Related Issue
Closes #354
Type of Change
Implementation Details
The operator uses
OnceLock<OperatorConfig>for thread-safe singleton configuration andserde_json::Value::pointer()for RFC 6901-compliant JSON Pointer resolution. Thewasm_graph_sdk(v1.1.3) from the Azure IoT SDKs registry provides the map operator macro, state store API, and logging infrastructure. The Cargo configuration targetswasm32-wasip2with--export-tablerustflag for Component Model compliance.The graph definition (graph-msg-to-dss-key.yaml) declares a source → map → sink topology with all four configuration parameters documented. The
${VERSION}placeholder is resolved byenvsubstduring the ACR push step.The component directory structure mirrors the existing
512-avro-to-jsonpattern with.cargo/config.toml,operators/,resources/graphs/, andscripts/directories.Testing Performed
22 unit tests validate value conversion (string, integer, float, boolean, null, object, array), JSON Pointer extraction (top-level, nested, array-indexed, deep nesting, missing paths), and key construction (with/without prefix, numeric/boolean values). WASM binary built and pushed to ACR. Dataflow graph deployed and validated with
terraform applyon the full-single-node-cluster blueprint.Validation Steps
cargo testinsrc/500-application/514-wasm-msg-to-dss/operators/msg-to-dss-key/— 22 tests should pass./scripts/build-wasm.shfromsrc/500-application/514-wasm-msg-to-dss/— WASM binary produced atoperators/msg-to-dss-key/target/wasm32-wasip2/release/msg_to_dss_key.wasmdataflow-graphs-msg-to-dss.tfvars.examplefor correct HCL syntax and graph topologyresources/graphs/graph-msg-to-dss-key.yamlChecklist
terraform fmton all Terraform codeterraform validateon all Terraform codeaz bicep formaton all Bicep codeaz bicep buildto validate all Bicep codeSecurity Review
Additional Notes
.examplefile and does not affect existing blueprint deployments.onMissing=errormode drops the message when the key path is not found, which is a deliberate design choice for strict validation scenarios documented in the issue.