feat: non-blocking concurrent checkpoint (WAL rotation + MVCC snapshot)

[adsharma]

Summary

This PR ports Vela-Engineering/kuzu's concurrent multi-writer checkpoint implementation
into ladybug. The implementation was developed by Aikins Laryea (Vela Partners) across
4 phases plus a read-only lock fix; the full diff is 2,360 lines across 45 files.

Before: checkpoint drains all transactions (reads + writes) and blocks new writes
until the entire checkpoint completes.

After: checkpoint only drains write transactions, releases the write gate as soon
as the WAL is rotated, and reads continue concurrently via MVCC snapshot isolation.
Shadow pages are applied with per-page CAS locking so optimistic readers detect updates.

What changed

Core mechanism

• WAL rotation: at checkpoint start, the active .wal is atomically renamed to
  .wal.checkpoint (frozen WAL). New write transactions immediately create a fresh
  active WAL, isolated from the checkpoint.
• Write gate release: once WAL is rotated the gate is released; storage
  materialization runs concurrently with new writers via per-node-group locking.
• MVCC catalog snapshots: Catalog::serializeSnapshot(snapshotTS) and
  CatalogSet::serializeSnapshot(snapshotTxn) use a pinned snapshot timestamp for
  consistent serialization after the gate is released.
• Epoch-based change tracking: Table::hasChanges (bool) replaced by
  changeEpoch (atomic uint64). captureChangeEpochs() snapshots per-table
  watermarks under the write gate; Table::checkpoint() uses watermarks to skip
  tables that haven't changed since the last gate.

Thread safety

• StorageManager::mtx upgraded from exclusive std::mutex to
  mutable std::shared_mutex (reads take shared lock, DDL takes unique lock).
• NodeTable::schemaMtx protects the columns vector during concurrent
  checkpoint vacuum.
• PageManager::version and Catalog::version use std::atomic<uint64_t>
  with a stable lastCheckpointVersion baseline.
• ShadowFile::applyShadowPages now calls BufferManager::updateFrameIfPageIsInFrame
  (CAS-loop locked variant) instead of the lock-free variant.

WAL recovery

• WALReplayer::replay now handles both frozen (.wal.checkpoint) and active
  (.wal) WAL files independently via replayFrozenWAL / replayActiveWAL.

Read-only databases

• File lock is skipped when readOnly=true, enabling concurrent read-only opens.

Apple clang compatibility

• Explicit constructors added to SegmentCheckpointState and LazySegmentData.

Testing

• Updated test/transaction/checkpoint_test.cpp to use the new
  logCheckpointAndApplyShadowPages(bool walRotated) signature, WAL rotation-aware
  flaky checkpointer classes, and ShadowFileDatabaseIDMismatchExistingDB now
  validates the frozen WAL (.wal.checkpoint) rather than the active WAL.
• CI workflow at embed-graph-db/.github/workflows/ladybug-concurrent-writes-ci.yml
  covers build (ubuntu + macos), full ctest suite, checkpoint/concurrent-write test
  filter, and benchmark comparison (blocks on >5% single-writer regression or
  zero read-during-checkpoint throughput).

Checkout Latest Benchmark Run

Attribution

Concurrent checkpoint implementation: Aikins Laryea, Vela Partners
(Vela-Engineering/kuzu, commits 3ad9071..d298e9c)

Ported and adapted for ladybug (lbug namespace, mainStorageManager field,
2-arg applyShadowPages API, std::format, graph catalog support) by Logan Powell.