Skip to content

[storage] Enable WAL, synchronous=FULL, transactions, and schema versioning #116

@intendednull

Description

@intendednull

Parent: #108

Problem

crates/storage/src/store.rs:15-38 opens the SQLite connection with no durability PRAGMAs:

  • No PRAGMA journal_mode = WAL — default rollback journal mode is slower and prevents concurrent reads during writes.
  • No PRAGMA synchronous = FULL — default NORMAL can lose recent writes on power failure, which is unacceptable for an archival node.
  • No explicit transactions — every store_event() is its own implicit single-row transaction. Batched ingestion does N fsyncs instead of 1.
  • No schema versioning — the schema is hardcoded with CREATE TABLE IF NOT EXISTS. Any future schema change silently re-uses the old schema on existing deployments, leading to silent corruption.

Fix

  1. In StorageEventStore::open, immediately after Connection::open:

    conn.pragma_update(None, "journal_mode", "WAL")?;
    conn.pragma_update(None, "synchronous", "FULL")?;
    conn.pragma_update(None, "foreign_keys", "ON")?;
  2. Add a schema_version table and a linear migrations: Vec<&str> list:

    conn.execute_batch("CREATE TABLE IF NOT EXISTS schema_version (version INTEGER PRIMARY KEY);")?;
    let current: i64 = conn.query_row(
        "SELECT COALESCE(MAX(version), 0) FROM schema_version", [], |r| r.get(0)
    )?;
    for (idx, migration) in MIGRATIONS.iter().enumerate() {
        let version = idx as i64 + 1;
        if version > current {
            let tx = conn.transaction()?;
            tx.execute_batch(migration)?;
            tx.execute("INSERT INTO schema_version (version) VALUES (?)", [version])?;
            tx.commit()?;
        }
    }
  3. Add a batched store_events(events: &[(ServerId, Event)]) that wraps the inserts in a single Transaction.

Test

#[test]
fn wal_mode_is_enabled() {
    let store = StorageEventStore::open(":memory:").unwrap();
    let mode: String = store.conn.query_row(
        "PRAGMA journal_mode", [], |r| r.get(0)
    ).unwrap();
    assert_eq!(mode.to_lowercase(), "wal");
}

#[test]
fn schema_version_table_exists_after_open() {
    // ...
}

#[test]
fn batched_store_is_atomic() {
    // Insert a batch containing one duplicate, assert all-or-nothing.
}

Out of scope

  • Connection pooling for concurrent access (separate issue if the worker needs it).
  • Dead-letter queue for corrupt event rows (separate issue).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions