Skip to content

[tech-debt] Hardcoded sync batch limit of 500 events lacks configuration and blocks protocol migration #207

@intendednull

Description

@intendednull

Problem

The sync response in crates/client/src/listeners.rs hard-codes a batch size of 500 events with an explicit TODO noting this is a temporary workaround:

// crates/client/src/listeners.rs:293-304
// TODO: migrate to heads-based sync — can't filter by state_hash in DAG model
events.into_iter().take(500).collect()

There are three inconsistent hardcoded limits:

  • 500 events in the sync response batch (listeners.rs:296)
  • 10_000 as a max batch in listeners.rs:256
  • 10_000 in crates/storage/src/store.rs:287

None of these are exposed through ClientConfig or a named constant.

Impact

  • Cannot tune for different network conditions without changing code
  • Inconsistent limits across the sync path
  • The TODO signals this is blocking full DAG-based sync protocol migration

Fix

  1. Extract to named constants:
pub const SYNC_BATCH_SIZE: usize = 500;
pub const SYNC_MAX_BATCH: usize = 10_000;
  1. Expose through ClientConfig for runtime tuning
  2. Track the heads-based sync migration as a follow-up (related to issue Worker sync protocol uses bulk SyncBatch instead of per-author Advertise/Request/Response #65 on worker sync protocol)

Location

  • crates/client/src/listeners.rs:256, 296-304
  • crates/storage/src/store.rs:287

Found during

Codebase audit at commit 18119e343ec53a9393312006d9d3b205129f696d

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions