feat: Optimize network traffic with patch-based sync for Node.js server

[kangjoseph90]

PR Checklist

• Have you checked if it works normally in all models? Ignore this if it doesn't use models.
• Have you checked if it works normally in all web, local, and node hosted versions? If it doesn't, have you blocked it in those versions?
• Have you added type definitions?

Description

This pull request introduces a highly efficient patch-based database synchronization system for the Node.js server environment. This fundamentally changes how the client and server communicate for the two most frequent and data-intensive operations: saving the database and managing backups.

Instead of transferring the entire multi-megabyte database on every change, the client now sends only a small patch (a few kilobytes) describing the modifications. This dramatically reduces network traffic, improves application performance, and enhances data integrity.

This feature is off by default and must be explicitly enabled via the --patch-sync flag when running the Node.js server. When disabled, the application functions exactly as it did before, ensuring full backward compatibility.

Background & Motivation

The application's primary performance bottleneck was network traffic. The two most frequent operations were:

• Save Logic (Client → Server): Sending the entire database (tens of MBs) every few seconds.
• Backup Management (Server → Client): Sending the complete list of all files stored on the server (potentially several MBs) for the client to filter for backups.

These two operations occurred repeatedly with every database modification, resulting in constant, heavy data transfer. This constant, heavy data transfer resulted in significant server load and bandwidth costs. This PR optimizes both of these hotspots, reducing network usage to near-negligible levels for most operations.

Implementation Details

• Dependency Added: fast-json-patch is used to efficiently compute data differences on the client.

Client Logic (globalApi.svelte.ts & autoStorage.ts)

• Intelligent Saving: On save, the client compares the current database state with the last synced state to generate a patch. If the server supports it, only this patch is sent.
• Robust Fallback: If a patch fails (e.g., due to a version conflict), the client automatically falls back to the traditional full-save method, ensuring stability.
• Efficient Backup Listing: When fetching backups, the client now sends a key-prefix to the server, allowing the server to pre-filter the list and send back only the relevant entries.

Server Logic (server.cjs)

• New /api/patch Endpoint: A new endpoint handles incoming patches.
• Optimistic Locking & Versioning: The server tracks a version number for the database. A patch is only applied if the client's version matches the server's, preventing data corruption from race conditions. If versions mismatch, the server returns a 409 Conflict, triggering the client's fallback mechanism.
• In-Memory Caching: The database is cached in memory. Patches are applied directly to the cache for near-instantaneous operations, minimizing slow disk I/O.
• Debounced Disk Writes: To further reduce disk load, writes from the cache to the file system are debounced, consolidating multiple rapid saves into a single, efficient write operation after 5 seconds of inactivity.
• Optimized List Endpoint: The /api/list endpoint now uses the key-prefix header to filter file lists before sending the response.

Activation

This feature is opt-in. It can be enabled by running pnpm run runserver:patch or by setting the RISU_PATCH_SYNC environment variable to 1.

🚨 Important: Node.js Version Requirement 🚨

This feature relies on the msgpackr library for efficient data encoding. Due to a bug in the Node.js core, specific versions can cause server-side encoding errors and crashes.

• ✅ Recommended Version: Please use Node.js v22.8.0 or higher (LTS).
• ❌ Incompatible Versions: v22.7.0, v23, and v24 are known to have this issue. Versions below v22.7.0 are not affected.

The server code includes a safeguard that will automatically disable patch-sync if it detects a problematic Node.js version, ensuring stability.

For more details, see the official Node.js issue and pull request:

• Node.js Issue #54518
• Node.js PR #54524

Expected Benefits

• Drastic Traffic Reduction: Server inbound/outbound traffic is reduced from potentially hundreds of gigabytes to mere megabytes over the same period.
• Faster Sync & Responsiveness: The in-memory cache provides near-instant sync confirmations to the client.
• Reduced Server Load: Disk I/O frequency is significantly decreased, lowering overall server strain.
• Enhanced Data Integrity: Versioning and atomic patch operations prevent data corruption.

Trade-offs

• Increased Server Memory Usage: The in-memory cache will increase the server's RAM footprint.
• Slightly Increased CPU Usage: A minor increase in CPU load is expected on both the client (for diffing) and the server (for patching), which is a worthwhile trade-off for the massive network performance gains.