Skip to content

Storage: ORDER BY timestamp violates DAG ordering; no disk limit; corrupt events silently skipped #83

@intendednull

Description

@intendednull

Problem

Three issues in crates/storage/src/store.rs:

1. ORDER BY violates DAG ordering (CRITICAL)

history() uses ORDER BY timestamp_hint_ms DESC, seq DESC (line 120). sync_since() uses ORDER BY seq ASC (line 164). Neither respects the DAG's topological order. Peers syncing from storage may receive events in wrong order, causing InsertError::SeqGap or state corruption.

The spec requires events to be returned in topological order (causal parents before dependents).

2. No disk size limit (CRITICAL)

Events are inserted indefinitely with no eviction or size limit. A storage node can fill the entire disk and crash with no graceful degradation.

3. Corrupt events silently skipped (CRITICAL)

All deserialization uses .ok() filter:

let events: Vec<Event> = rows.iter()
    .filter_map(|data| bincode::deserialize(data).ok())
    .collect();

Corrupted events are silently dropped — no logging, no alerting. Query returns fewer events than expected with no indication of data loss.

Impact

  1. Peers receive events out of topological order → state corruption
  2. Disk exhaustion → crash with no recovery
  3. Silent data loss from corruption → incomplete state for syncing peers

Suggested fix

  1. Return events via EventDag::topological_sort() order, or store a topo-sort index
  2. Add --max-events or --max-size-gb config with oldest-event eviction
  3. Log warnings on deserialization failures; add corruption counter metric

Location

  • crates/storage/src/store.rs:120,164-165,206-207 (ordering)
  • crates/storage/src/store.rs entire (disk limit)
  • crates/storage/src/store.rs:143,175,224 (deserialization)

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions