Skip to content

feat: core domain types and rocksdb storage layer#3

Merged
vieiralucas merged 4 commits intomainfrom
feat/1-3-core-domain-types-storage-layer
Feb 11, 2026
Merged

feat: core domain types and rocksdb storage layer#3
vieiralucas merged 4 commits intomainfrom
feat/1-3-core-domain-types-storage-layer

Conversation

@vieiralucas
Copy link
Copy Markdown
Member

@vieiralucas vieiralucas commented Feb 11, 2026

Summary

  • Add core domain types: FilaError (with thiserror), Message, and QueueConfig with serde serialization
  • Implement binary key encoding (storage::keys) with big-endian numerics and length-prefixed strings for correct lexicographic ordering in RocksDB
  • Add Storage trait abstraction with WriteBatchOp for atomic multi-CF operations
  • Implement RocksDbStorage with 5 column families (messages, leases, lease_expiry, queues, state)

Test plan

  • 6 key encoding unit tests (lexicographic ordering, prefix matching, collision avoidance)
  • 10 RocksDB integration tests (CRUD for all CFs, prefix scan, expired lease scan, atomic write batch, data persistence across reopen)
  • cargo clippy -- -D warnings passes
  • cargo fmt --check passes
  • All 16 tests pass via cargo nextest run

Summary by cubic

Adds core domain types and a RocksDB storage layer with atomic batches and lexicographic key encoding for durable, ordered message persistence. Implements Story 1.3 requirements for queues, messages, leases, and expiry scans.

  • New Features

    • StorageError with StorageResult; removed FilaError from core.
    • Message and QueueConfig with serde/serde_json; UUIDv7 message IDs.
    • Storage trait + WriteBatchOp; RocksDbStorage with CFs: messages, leases, lease_expiry, queues, state; atomic batches and durable reopen.
    • Binary key encoding (big-endian u64, length‑prefixed strings) with prefixes for queue and fairness scans; lease expiry keys enable efficient timeout scanning.
  • Bug Fixes

    • Checked u16 cast for string length prefixes to avoid truncation in key encoding.

Written for commit 899b27b. Summary will update on new commits.

Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 13 files

Prompt for AI agents (all issues)

Check if these issues are valid — if so, understand the root cause of each and fix them.


<file name="crates/fila-core/src/storage/keys.rs">

<violation number="1" location="crates/fila-core/src/storage/keys.rs:16">
P1: Unsafe truncating cast `s.len() as u16` — strings longer than 65535 bytes silently produce a corrupted length prefix, leading to key collisions and data corruption. Use `u16::try_from(s.len()).expect("queue/fairness key exceeds 64 KiB")` (or return a `Result`) to fail loudly instead of silently corrupting keys.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

Comment thread crates/fila-core/src/storage/keys.rs Outdated
Comment thread crates/fila-core/src/error.rs Outdated
@vieiralucas vieiralucas force-pushed the feat/1-2-ci-cd-pipeline branch from d9f7556 to f4d3033 Compare February 11, 2026 12:03
@vieiralucas vieiralucas force-pushed the feat/1-3-core-domain-types-storage-layer branch from 5c62cd7 to b468d09 Compare February 11, 2026 12:06
@vieiralucas vieiralucas force-pushed the feat/1-2-ci-cd-pipeline branch from f4d3033 to 490ef1c Compare February 11, 2026 15:52
Base automatically changed from feat/1-2-ci-cd-pipeline to main February 11, 2026 15:54
add error types (FilaError with thiserror), message and queue domain
types with serde support, binary key encoding for lexicographic
ordering, storage trait abstraction, and rocksdb implementation with
5 column families and atomic write batch support.
replace `s.len() as u16` with `u16::try_from().expect()` to panic
loudly instead of silently truncating strings longer than 64 KiB.
storage operations now return StorageResult<T> with only infrastructure
errors (rocksdb, serialization). domain errors (queue not found, etc.)
are scoped to FilaError at the broker/application layer.
per-command error types will be introduced as each command is added
@vieiralucas vieiralucas force-pushed the feat/1-3-core-domain-types-storage-layer branch from a6cdccc to 899b27b Compare February 11, 2026 15:54
@vieiralucas vieiralucas merged commit e09271a into main Feb 11, 2026
4 checks passed
@vieiralucas vieiralucas deleted the feat/1-3-core-domain-types-storage-layer branch February 11, 2026 15:56
vieiralucas added a commit that referenced this pull request Mar 18, 2026
- apply_to_broker_storage now returns Result and propagates StorageError
  instead of silently swallowing storage failures (cubic #1)
- add DeleteLeaseExpiry mutation in ack/nack replication paths to clean up
  orphaned lease expiry entries (cubic #3)
- fix no-op leased_msg_keys.retain in recovery — now properly clears
  entries for the recovering queue before rebuild (cubic #4)
- warn when create_group is called without broker_storage set (cubic #5)
- check send_command result in watch_leader_changes — only update leading
  state on success so next poll retries on failure (cubic #6, #7)
- trigger RecoverQueue on first-sight leader state to catch messages
  replicated between startup and first poll (cubic #8)
- replace catch-all _ => {} with explicit variant listing in
  apply_to_broker_storage for compiler-enforced exhaustiveness
vieiralucas added a commit that referenced this pull request Mar 18, 2026
- apply_to_broker_storage now returns Result and propagates StorageError
  instead of silently swallowing storage failures (cubic #1)
- add DeleteLeaseExpiry mutation in ack/nack replication paths to clean up
  orphaned lease expiry entries (cubic #3)
- fix no-op leased_msg_keys.retain in recovery — now properly clears
  entries for the recovering queue before rebuild (cubic #4)
- warn when create_group is called without broker_storage set (cubic #5)
- check send_command result in watch_leader_changes — only update leading
  state on success so next poll retries on failure (cubic #6, #7)
- trigger RecoverQueue on first-sight leader state to catch messages
  replicated between startup and first poll (cubic #8)
- replace catch-all _ => {} with explicit variant listing in
  apply_to_broker_storage for compiler-enforced exhaustiveness
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant