From 5bc3d74d2937a1e221feb6cbc5c55f79a6ff2768 Mon Sep 17 00:00:00 2001 From: Wil Wade Date: Wed, 22 May 2024 15:22:21 -0400 Subject: [PATCH 1/5] Messages Pallet Readme docs --- pallets/messages/README.md | 65 +++++++++++++++++++++++++++++++++++++ pallets/messages/src/lib.rs | 29 ++--------------- 2 files changed, 67 insertions(+), 27 deletions(-) create mode 100644 pallets/messages/README.md diff --git a/pallets/messages/README.md b/pallets/messages/README.md new file mode 100644 index 0000000000..72c3beface --- /dev/null +++ b/pallets/messages/README.md @@ -0,0 +1,65 @@ +# Messages Pallet + +Stores block-indexed data for a Schema using `OnChain` or `IPFS` payload location. + +## Summary + +Messages stores metadata and message payload data for a set Schema. +Use of the Message pallet payloads is designed for time-centric data. +Discovery is via the `MessagesInBlock` on new blocks or requesting messages from a block. +Retrieval is via RPC. + +### Metadata vs Payload + +Messages have both metadata and payloads. +The payload should always match the data structure or the messages is considered invalid. +The metadata is the Block Number, Schema Id, and other data to help discover and organize the payload information. + +### Payload Options + +- `IPFS`: Storage of the CID and length of the file on IFPS +- `OnChain`: Storage of the entire payload data. Usually for tiny payloads only. + +### Message Ordered + +Messages are ordered by block number, but the have a set order based on transaction order in the block. +This order is immutable. + +### Actions + +The Messages pallet provides for: + +- Adding messages for a given schema +- Enabling the retrieval of messages for a given schema + +## Interactions + +### Extrinsics + +| Name/Description | Caller | Payment | Key Events | Runtime Added | +| ---------------------------------------------------------------------------------------- | -------- | ------------------ | --------------------------------------------------------------------------------------------------------------------------------- | ------------- | +| `add_ipfs_message`
Add a message to a Schema with an `IPFS` payload location | Provider | Capacity or Tokens | [`MessagesInBlock`](https://frequency-chain.github.io/frequency/pallet_messages/pallet/enum.Event.html#variant.MessagesInBlock)\* | 1 | +| `add_onchain_message`
Add a message to a Schema with an `ON_CHAIN` payload location | Provider | Capacity or Tokens | [`MessagesInBlock`](https://frequency-chain.github.io/frequency/pallet_messages/pallet/enum.Event.html#variant.MessagesInBlock)\* | 1 | + +\* The `MessagesInBlock` may not occur more than once per block and does _not_ indicate which schema received messages. + +See [Rust Docs](https://frequency-chain.github.io/frequency/pallet_messages/pallet/struct.Pallet.html) for more details. + +### State Queries + +| Name | Description | Query | Runtime Added | +| --------------- | ----------------------------------------------------------------------------------------------------------------------------- | ------------ | ------------- | +| Get Messages v2 | _Suggested_: Use RPC instead of this storage directly. Storage for the messages by Block number, Schema Id, and Message Index | `messagesV2` | 61 | +| Get Messages v1 | Removed in Runtime 60 | `messages` | 1-60 | + +See the [Rust Docs](https://frequency-chain.github.io/frequency/pallet_messages/pallet/storage_types/index.html) for additional state queries and details. + +### RPCs + +Note: May be restricted based on node settings and configuration. + +| Name | Description | Call | Node Version | +| ------------------------- | ------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------- | ------------ | +| Get Messages by Schema Id | Fetch paginated messages for a specific Schema Id in the given block range for a given Schema Id | [`getBySchemaId`](https://frequency-chain.github.io/frequency/pallet_messages_rpc/trait.MessagesApiServer.html#tymethod.get_messages_by_schema_id) | v1.0.0+ | + +See [Rust Docs](https://frequency-chain.github.io/frequency/pallet_messages_rpc/trait.MessagesApiServer.html) for more details. diff --git a/pallets/messages/src/lib.rs b/pallets/messages/src/lib.rs index 8a620a1952..121ba16260 100644 --- a/pallets/messages/src/lib.rs +++ b/pallets/messages/src/lib.rs @@ -1,5 +1,4 @@ -//! # Messages pallet -//! A pallet for storing messages. +//! Stores messages `IPFS` and `OnChain` Schema payload locations //! //! - [Configuration: `Config`](Config) //! - [Extrinsics: `Call`](Call) @@ -7,31 +6,7 @@ //! - [Custom RPC API: `MessagesApiServer`](../pallet_messages_rpc/trait.MessagesApiServer.html) //! - [Event Enum: `Event`](Event) //! - [Error Enum: `Error`](Error) -//! -//! ## Overview -//! -//! Messages allow for discovery of new content matching a given Schema. -//! Each message MUST have either a provider or a validated source and provider. -//! Message is the metadata of the source, order, schema for a payload or referenced payload. -//! -//! The Messages Pallet provides functions for: -//! -//! - Adding messages for a given schema. -//! - Enabling retrieving messages for a given schema. -//! - (FUTURE) Removing messages after expiration -//! -//! ## Terminology -//! -//! - **Message:** A message that matches a registered Schema (on-chain or off-chain). -//! - **Payload:** The data in a `Message` that matches a `Schema`. -//! - **MSA Id:** The 64 bit unsigned integer associated with an `Message Source Account`. -//! - **MSA:** Message Source Account. A registered identifier with the [MSA pallet](../pallet_msa/index.html). -//! - **Schema:** A registered data structure and the settings around it. (See [Schemas pallet](../pallet_schemas/index.html)) -//! - **Schema Id:** A U16 bit identifier for a schema stored on-chain. -//! -//! ## Implementations -//! - None -//! +#![doc = include_str!("../README.md")] // Substrate macros are tripping the clippy::expect_used lint. #![allow(clippy::expect_used)] // Ensure we're `no_std` when compiling for Wasm. From b934bc0c44e543e8963313b3ae80b81482e83911 Mon Sep 17 00:00:00 2001 From: Wil Wade Date: Thu, 23 May 2024 15:05:25 -0400 Subject: [PATCH 2/5] Apply suggestions from code review --- pallets/messages/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pallets/messages/README.md b/pallets/messages/README.md index 72c3beface..ebc94b7249 100644 --- a/pallets/messages/README.md +++ b/pallets/messages/README.md @@ -12,8 +12,8 @@ Retrieval is via RPC. ### Metadata vs Payload Messages have both metadata and payloads. -The payload should always match the data structure or the messages is considered invalid. -The metadata is the Block Number, Schema Id, and other data to help discover and organize the payload information. +The payload should always match the data structure or the message is considered invalid. +The metadata is the Block Number, Schema Id, and other data useful for discovering and organizing the payload information. ### Payload Options From dcfb9616cb7a399a09ce5312859dce799db16167 Mon Sep 17 00:00:00 2001 From: Wil Wade Date: Fri, 24 May 2024 08:16:04 -0400 Subject: [PATCH 3/5] Apply suggestions from code review Co-authored-by: Joe Caputo --- pallets/messages/README.md | 6 +++--- pallets/messages/src/lib.rs | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/pallets/messages/README.md b/pallets/messages/README.md index ebc94b7249..c6d7d1e7f6 100644 --- a/pallets/messages/README.md +++ b/pallets/messages/README.md @@ -17,12 +17,12 @@ The metadata is the Block Number, Schema Id, and other data useful for discoveri ### Payload Options -- `IPFS`: Storage of the CID and length of the file on IFPS +- `IPFS`: Storage of the CID and length of the file on IPFS - `OnChain`: Storage of the entire payload data. Usually for tiny payloads only. -### Message Ordered +### Message Ordering -Messages are ordered by block number, but the have a set order based on transaction order in the block. +Messages are ordered by block number, and within each block have a set order based on transaction order in the block. This order is immutable. ### Actions diff --git a/pallets/messages/src/lib.rs b/pallets/messages/src/lib.rs index 121ba16260..c93e051107 100644 --- a/pallets/messages/src/lib.rs +++ b/pallets/messages/src/lib.rs @@ -1,4 +1,4 @@ -//! Stores messages `IPFS` and `OnChain` Schema payload locations +//! Stores messages for `IPFS` and `OnChain` Schema payload locations //! //! - [Configuration: `Config`](Config) //! - [Extrinsics: `Call`](Call) From 5e57ffcc3d80bee24449a1ce6b6edf43e212f60e Mon Sep 17 00:00:00 2001 From: Wil Wade Date: Fri, 24 May 2024 08:45:57 -0400 Subject: [PATCH 4/5] Update pallets/messages/src/lib.rs --- pallets/messages/src/lib.rs | 1 + 1 file changed, 1 insertion(+) diff --git a/pallets/messages/src/lib.rs b/pallets/messages/src/lib.rs index c93e051107..b8ee6b68c0 100644 --- a/pallets/messages/src/lib.rs +++ b/pallets/messages/src/lib.rs @@ -1,5 +1,6 @@ //! Stores messages for `IPFS` and `OnChain` Schema payload locations //! +//! ## Quick Links //! - [Configuration: `Config`](Config) //! - [Extrinsics: `Call`](Call) //! - [Runtime API: `MessagesRuntimeApi`](../pallet_messages_runtime_api/trait.MessagesRuntimeApi.html) From 28fe6f96447fbf049b1c22f0a0256a0ba625455f Mon Sep 17 00:00:00 2001 From: Wil Wade Date: Wed, 29 May 2024 13:31:47 +0000 Subject: [PATCH 5/5] Review feedback --- pallets/messages/README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/pallets/messages/README.md b/pallets/messages/README.md index c6d7d1e7f6..46c490b694 100644 --- a/pallets/messages/README.md +++ b/pallets/messages/README.md @@ -5,7 +5,7 @@ Stores block-indexed data for a Schema using `OnChain` or `IPFS` payload locatio ## Summary Messages stores metadata and message payload data for a set Schema. -Use of the Message pallet payloads is designed for time-centric data. +Message payloads are meant for streaming data, where when the message was sent is the focus. Discovery is via the `MessagesInBlock` on new blocks or requesting messages from a block. Retrieval is via RPC. @@ -18,11 +18,11 @@ The metadata is the Block Number, Schema Id, and other data useful for discoveri ### Payload Options - `IPFS`: Storage of the CID and length of the file on IPFS -- `OnChain`: Storage of the entire payload data. Usually for tiny payloads only. +- `OnChain`: Storage of the entire payload data, usually for sub-256 byte payloads ### Message Ordering -Messages are ordered by block number, and within each block have a set order based on transaction order in the block. +Messages are ordered by block number, and within each block, they follow a specific order based on their transaction sequence within that block. This order is immutable. ### Actions