feat: crash recovery and graceful shutdown (story 1.9)#11
Merged
vieiralucas merged 3 commits intomainfrom Feb 12, 2026
Merged
Conversation
69e45f6 to
c00fee8
Compare
03b03ce to
e7f66bf
Compare
c00fee8 to
e7a8f09
Compare
581bc40 to
8d2c8f9
Compare
add recovery logic that scans for expired leases on startup and reclaims them so messages re-enter the ready pool. flush the rocksdb wal on graceful shutdown to ensure all writes are durable. includes integration tests for message persistence across restarts, expired lease reclamation, queue definition survival, and wal flush verification.
8d2c8f9 to
30cea52
Compare
- parse_lease_expiry_key roundtrip and corrupt input rejection - recovery skips corrupt lease_expiry keys without panicking - recovery preserves non-expired leases (active messages not reclaimed)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
parse_lease_expiry_key()for extracting queue_id and msg_id from expiry keysflush()method to Storage trait, implemented viaflush_wal(true)in RocksDBTest plan
recovery_preserves_messages_after_restart— enqueue 5 messages, restart scheduler, verify all deliveredrecovery_reclaims_expired_leases— create expired lease, restart, verify reclaimed and message delivered to consumerrecovery_preserves_queue_definitions— create 3 queues, restart, verify all presentshutdown_flushes_wal— enqueue, shutdown, reopen storage from disk, verify data survivedSummary by cubic
Adds crash recovery and graceful shutdown so the broker survives restarts without message loss. On startup we reclaim expired leases and return messages to the ready pool; on shutdown we flush the RocksDB WAL for durability. (Linear story 1.9)
Written for commit 7cd3970. Summary will update on new commits.