diff --git a/.cursor/rules/DB.mdc b/.cursor/rules/DB.mdc new file mode 100644 index 0000000..d7cfc6c --- /dev/null +++ b/.cursor/rules/DB.mdc @@ -0,0 +1,49 @@ +# DB Module (`src/DB`) + +This module provides the core database interaction layer, abstracting the underlying SQLite database. + +## Core Components + +### `DB.js` + +- **Purpose:** Provides a high-level abstraction for database connections and operations. It likely manages connection pooling and simplifies common database tasks. +- **Key Features:** (To be determined by analyzing the file) +- **Dependencies:** `SQLite.js`, `Statement.js` + +### `SQLite.js` + +- **Purpose:** Wraps the `sqlite3` library to provide a more promise-based and potentially enhanced interface for interacting directly with SQLite. +- **Key Features:** (To be determined by analyzing the file) +- **Dependencies:** `sqlite3`, `Statement.js` + +### `Statement.js` + +- **Purpose:** Represents a prepared SQL statement, likely offering methods for binding parameters and executing the statement efficiently. +- **Key Features:** (To be determined by analyzing the file) +- **Dependencies:** `sqlite3` + +## Usage + +(To be determined by analyzing the code and tests) + +## Dependencies + +- `sqlite3`: The underlying Node.js driver for SQLite. +- `async-sema`: Likely used for managing concurrency or locking around database operations. +- `dataloader`: Potentially used for batching and caching database reads. +- `debug`: Used for logging debug information. +- `lodash`: Utility library, potentially used for various data manipulations. + +## Structure + +``` +src/ +└── DB/ + ├── DB.js # High-level DB abstraction + ├── DB.test.js + ├── SQLite.js # sqlite3 wrapper + ├── SQLite.test.ts + ├── Statement.js # Prepared statement representation + ├── Statement.test.js + └── index.js # Module entry point (re-exports) +``` diff --git a/.cursor/rules/EventQueue.mdc b/.cursor/rules/EventQueue.mdc new file mode 100644 index 0000000..36185f7 --- /dev/null +++ b/.cursor/rules/EventQueue.mdc @@ -0,0 +1,49 @@ +# EventQueue Module (`src/EventQueue.js`) + +This module provides an event queue implementation, likely used for the event sourcing mechanism. + +## Core Components + +### `EventQueue.js` + +- **Purpose:** Implements a persistent event queue backed by a SQLite table (defaulting to `history`). It extends `JsonModel`, suggesting it leverages the JSON storage capabilities. +- **Key Features:** + - Stores events with `v` (auto-incrementing version/ID), `type`, `ts` (timestamp), `data` (JSON), `result` (JSON), and `size` (calculated size of data/result). + - Provides methods for adding events (`add`), retrieving the latest version (`getMaxV`), and fetching the next event after a specific version (`getNext`), including waiting for new events. + - Includes database migrations for adding indices and helper views (`_recentHistory`, `_historyTypes`). + - Manages internal state (`currentV`, `knownV`) for optimization. + - Uses `debug` for logging. + - Handles concurrency within the same process using Promises (`_addP`). + - Optionally keeps the Node.js process alive while waiting (`forever: true`). + - Allows customization of table name and columns. +- **Dependencies:** `JsonModel`, `debug` + +## Usage + +- Instantiate `EventQueue` (likely via `new EventQueueImpl(...)` although the export might simplify this). +- Use `add(type, data, [ts])` to append new events. +- Use `getMaxV()` to get the highest event version number. +- Use `getNext(v, [noWait])` to retrieve events sequentially, potentially waiting for new ones. + +## Schema (Default `history` table) + +- `v`: INTEGER PRIMARY KEY AUTOINCREMENT (Event version/ID) +- `type`: TEXT (Event type identifier) +- `ts`: INTEGER (Timestamp in milliseconds since epoch, indexed) +- `data`: JSON (Payload of the event) +- `result`: JSON (Result associated with processing the event, optional) +- `size`: INTEGER (Calculated size of `data` + `result` JSON strings) + +## Dependencies + +- `debug`: Used for logging. +- `JsonModel`: Base class providing JSON handling and DB interaction. + +## Structure + +``` +src/ +├── EventQueue.js # Main EventQueue implementation +├── EventQueue.test.js +└── ... (other modules) +``` diff --git a/.cursor/rules/EventSourcingDB.mdc b/.cursor/rules/EventSourcingDB.mdc new file mode 100644 index 0000000..7ab6c37 --- /dev/null +++ b/.cursor/rules/EventSourcingDB.mdc @@ -0,0 +1,62 @@ +# EventSourcingDB & ESModel Modules (`src/EventSourcingDB`) + +These modules implement the core event sourcing pattern for the database. + +## Core Components + +### `EventSourcingDB.js` (`EventSourcingDB` class) + +- **Purpose:** Orchestrates the event sourcing process. It manages the flow of events from the `EventQueue` to the `ESModel` instances, ensuring atomic and ordered processing. +- **Key Features:** + - **Event Loop:** Polls or waits for new events from the `EventQueue`. + - **Event Processing Pipeline:** For each event: + 1. **Preprocessing:** Calls `preprocessor` static methods on all registered `ESModel`s. `ESModel.preprocessor` assigns IDs to new objects. + 2. **Reduction:** Calls `reducer` static methods on all `ESModel`s. `ESModel.reducer` calculates the necessary change (`ins`, `upd`, `rm`, `esFail`) based on the event type and current state. + 3. **Application:** Within a database transaction: + - Calls `applyResult` (via `model.applyResult(result)` which calls the imported `applyResult`) on each model with its calculated result, making the actual database changes using the underlying `JsonModel` methods (in writable mode). + - Calls `deriver` methods on models for post-processing (also in writable mode). + 4. **Commit:** Commits the transaction and updates the database `user_version`. + - **Sub-events:** Handles events dispatched *during* the processing of another event, processing them recursively within the same transaction. + - **Error Handling:** Logs errors and halts processing if a reducer or applicator fails. Stores error information in the event's `result` field in the `EventQueue`. + - **State Management:** Manages database connections (potentially separate R/W and RO), tracks the current processed version, and uses `AsyncLocalStorage` for context. + - **Lifecycle Events:** Emits events ('begin', 'result', 'error', 'processed') for monitoring. +- **Dependencies:** `DB`, `EventQueue`, `ESModel`, `debug`, `lodash`, `AsyncLocalStorage`. + +### `ESModel.js` (`ESModel` class) + +- **Purpose:** A wrapper around `JsonModel` that integrates a model into the event sourcing system. +- **Key Features:** + - **Extends `JsonModel`:** Inherits ORM capabilities. + - **Event Generation:** Overrides `set`, `update`, `remove` to dispatch standardized events (`{type: 'es/ModelName', data: [...]}`) instead of directly modifying the DB. + - **Standard Event Format:** Uses `[ACTION_ENUM, id, payload, meta]` for event `data`. + - **Writable Mode (`setWritable`)**: Allows bypassing event generation for direct DB modification (used by `applyResult`, migrations, derivers). + - **Event Helpers (`event.*`)**: Provides methods to easily create standard event objects. + - **Default Handlers:** Implements static `preprocessor` (assigns IDs) and `reducer` (calculates `ins`/`upd`/`rm`/`esFail`) for the standard `es/ModelName` events. + - **ID Prediction (`getNextId`)**: Predicts the next integer ID within a processing cycle. +- **Dependencies:** `JsonModel`, `applyResult.js`, `lodash`, `debug`. + +### `applyResult.js` + +- **Purpose:** A utility function used by `ESModel.applyResult`. +- **Key Features:** Takes a model instance (in writable mode) and a result object (e.g., `{ins: [...], upd: [...], rm: [...]}`) generated by a reducer, and calls the appropriate underlying `JsonModel` methods (`_set`, `remove`) to apply the changes to the database. +- **Dependencies:** `JsonModel` (implicitly via the model instance). + +## Usage + +1. Define models by extending `ESModel` or using it directly, passing a `dispatch` function from `EventSourcingDB`. +2. Instantiate `EventSourcingDB`, providing it with an `EventQueue` instance (or configuration) and a map of `models`. +3. Call `esdb.open()` and `esdb.startPolling()` (or `waitForQueue`). +4. Interact with the database *only* through `ESModel` methods (`set`, `update`, `remove`) or by directly dispatching events via `esdb.dispatch()`. The system handles processing these events asynchronously. +5. Use standard `JsonModel` methods for reads (`get`, `search`, etc.). + +## Structure + +``` +src/ +└── EventSourcingDB/ + ├── EventSourcingDB.js # Main orchestrator class + ├── ESModel.js # JsonModel wrapper for event sourcing + ├── applyResult.js # Utility to apply reducer results + ├── ES*.test.js # Unit tests + └── index.js # Module entry point +``` diff --git a/.cursor/rules/JsonModel.mdc b/.cursor/rules/JsonModel.mdc new file mode 100644 index 0000000..47e9d7d --- /dev/null +++ b/.cursor/rules/JsonModel.mdc @@ -0,0 +1,69 @@ +# JsonModel Module (`src/JsonModel`) + +This module provides the `JsonModel` class, a powerful Object-Relational Mapper (ORM)-like interface for storing and querying JSON-style documents within SQLite tables. + +## Core Components + +### `JsonModel.js` (`JsonModelImpl` class) + +- **Purpose:** Manages a specific SQLite table, mapping rows to JavaScript objects. It handles schema definition, data serialization/deserialization, CRUD operations, querying, indexing, migrations, and caching. +- **Key Features:** + - **Schema Definition:** Defines table structure through a `columns` object, supporting various data types (TEXT, INTEGER, JSON, etc.), primary keys (`idCol`), automatic value generation (`value`), indexing (`index`), and aliasing. + - **JSON Handling:** Stores primary data in a `json` column but can extract specific JSON paths (`path`) into virtual columns for efficient querying and indexing. + - **CRUD:** Provides comprehensive methods: + - `set()`: Inserts or updates (upserts) an object. + - `get()`: Retrieves an object by its ID. + - `getAll()`: Retrieves multiple objects by IDs. + - `update()`: Updates an existing object. + - `remove()` / `delete()`: Deletes an object by ID or the object itself. + - `changeId()`: Updates the ID of an object. + - **Querying:** Offers flexible search capabilities: + - `search()`, `searchOne()`, `searchAll()`: Finds objects based on attribute matching (`where`), sorting (`sort`), limiting results (`limit`), and cursor-based pagination. + - `exists()`: Checks if an object matching criteria exists. + - `count()`: Counts matching objects. + - Aggregate functions: `max()`, `min()`, `sum()`, `avg()`. + - **Caching:** Integrates `dataloader` for efficient batching and caching of `get` operations via `getCached()`. + - **Migrations:** Manages database schema evolution through registered migration functions. + - **Customization:** Allows specifying a custom class (`ItemClass`) for instantiated objects. + - **Utilities:** Includes internal helpers for SQL generation, object cloning, cursor encoding/decoding (`jsurl2`). +- **Dependencies:** `DB` (likely via passed `db` instance), `dataloader`, `lodash`, `debug`, `jsurl2`, internal helpers. + +### Helper Modules + +- `normalizeColumn.js`: Standardizes column definitions. +- `prepareSqlCol.js`: Generates SQL snippets for columns, handles JSON path extraction. +- `verifyOptions.js`: Validates constructor options and column definitions. +- `makeMigrations.js`: Constructs migration steps based on schema definition. +- `makeDefaultIdValue.js`: Creates default value generation logic for ID columns. +- `assignJsonParents.js`: Links JSON path columns to their parent JSON column. + +## Usage + +1. Instantiate `JsonModel` with database connection (`db`), table `name`, and `columns` definition. +2. Use methods like `set()`, `get()`, `search()`, `update()`, `remove()` to interact with the data. +3. Leverage caching (`getCached()`) for performance-sensitive reads. +4. Define migrations to handle schema changes over time. + +## Dependencies + +- `dataloader`: For batching and caching reads. +- `lodash`: General utility functions. +- `debug`: For logging. +- `jsurl2`: For compact URL-safe JSON serialization (used for cursors). +- `DB` module components (implicitly via the `db` instance). + +## Structure + +``` +src/ +└── JsonModel/ + ├── JsonModel.js # Main class implementation + ├── JM-*.test.js # Unit tests + ├── normalizeColumn.js # Helper + ├── prepareSqlCol.js # Helper + ├── verifyOptions.js # Helper + ├── makeMigrations.js # Helper + ├── makeDefaultIdValue.js # Helper + ├── assignJsonParents.js # Helper + └── index.js # Module entry point +``` diff --git a/.cursor/rules/Lib.mdc b/.cursor/rules/Lib.mdc new file mode 100644 index 0000000..a1e8077 --- /dev/null +++ b/.cursor/rules/Lib.mdc @@ -0,0 +1,39 @@ +# Lib Module (`src/lib`) + +This directory contains miscellaneous utility functions used across the codebase. + +## Core Components + +### `settleAll.js` + +- **Purpose:** Provides a function likely analogous to `Promise.allSettled`, allowing multiple promises to be awaited, capturing both fulfilled values and rejection reasons without short-circuiting on the first rejection. +- **Usage:** Useful when you need to process a batch of independent asynchronous operations and handle their individual outcomes. + +### `slugify.js` + +- **Purpose:** Converts a string into a URL-friendly "slug" format. +- **Key Features:** Likely removes special characters, converts to lowercase, and replaces spaces with hyphens. +- **Usage:** Generating clean identifiers for URLs or filenames. + +### `warning.js` + +- **Purpose:** Handles the display of warnings or deprecation notices, potentially only in development environments. +- **Key Features:** Exports `DEV` (boolean indicating development mode) and `deprecated` (function to log deprecation warnings). +- **Usage:** Providing feedback to developers about outdated APIs or potential issues without cluttering production logs. + +### `_test-helpers.js` + +- **Purpose:** Contains helper functions specifically designed for use within the project's test suite (`*.test.js` files). +- **Usage:** Reducing boilerplate and simplifying common testing patterns. + +## Structure + +``` +src/ +└── lib/ + ├── settleAll.js + ├── slugify.js + ├── slugify.test.js + ├── warning.js + └── _test-helpers.js +``` diff --git a/.cursor/rules/Overview.mdc b/.cursor/rules/Overview.mdc new file mode 100644 index 0000000..34d31c7 --- /dev/null +++ b/.cursor/rules/Overview.mdc @@ -0,0 +1,72 @@ +# StratoDB Project Overview + +**Version:** 3.11.0 (from package.json) +**Description:** NoSQL-hybrid with Event Sourcing based on sqlite. + +This document provides a high-level overview of the `strato-db` codebase, intended for AI agent understanding. + +## Core Concepts + +StratoDB is a database system built on SQLite that combines features of NoSQL document stores with the principles of Event Sourcing. + +- **Document Storage (`JsonModel`):** Objects are stored in SQLite tables, often with a primary JSON blob column and potentially indexed virtual columns extracted from the JSON structure. +- **Event Sourcing (`EventSourcingDB`, `ESModel`, `EventQueue`):** Changes to the database state are driven by a sequence of events recorded in a persistent queue (`history` table by default). Instead of direct mutation, operations generate events, which are then processed sequentially and atomically to update the database state. +- **Layered Architecture:** + - **`DB` Module:** Low-level SQLite interaction wrapper (Promises, prepared statements). + - **`JsonModel` Module:** ORM-like layer for interacting with tables as document stores. + - **`EventQueue` Module:** Persistent queue for storing and retrieving events. + - **`EventSourcingDB` Module:** Orchestrates the event processing pipeline, applying events to `ESModel` instances. + - **`ESModel` Module:** Connects `JsonModel` to the event sourcing system by translating operations into events and providing default event handlers. + +## Project Structure + +``` +. +├── .cursor/rules/ # AI-readable documentation (GENERATED) +│ ├── DB.mdc +│ ├── EventQueue.mdc +│ ├── EventSourcingDB.mdc +│ ├── JsonModel.mdc +│ └── Lib.mdc +├── src/ +│ ├── DB/ # Core SQLite wrapper and abstractions +│ ├── EventQueue.js # Event queue implementation +│ ├── EventSourcingDB/ # Event Sourcing orchestration and ESModel +│ ├── JsonModel/ # Document store ORM layer +│ ├── lib/ # Utility functions +│ └── index.js # Main library export +├── dist/ # Compiled output (JS) +├── types.d.ts # Main TypeScript definition file +├── package.json # Project metadata, dependencies, scripts +├── tsconfig.json # TypeScript configuration +├── vite.config.ts # Vite build/test configuration +├── Readme.md # User-facing README +└── ... (config files, tests, etc.) +``` + +## Key Modules (Detailed Documentation) + +- **[DB Module](./DB.mdc):** Covers `DB.js`, `SQLite.js`, `Statement.js`. +- **[EventQueue Module](./EventQueue.mdc):** Covers `EventQueue.js`. +- **[JsonModel Module](./JsonModel.mdc):** Covers `JsonModel.js` and its helpers. +- **[EventSourcingDB & ESModel Modules](./EventSourcingDB.mdc):** Covers `EventSourcingDB.js`, `ESModel.js`, `applyResult.js`. +- **[Lib Module](./Lib.mdc):** Covers utilities in `src/lib`. + +## Technology Stack + +- **Language:** JavaScript (with JSDoc types), some TypeScript for tests and config. +- **Database:** SQLite (via `sqlite3` Node.js package). +- **Build/Test:** Vite, Vitest. +- **Linting/Formatting:** ESLint, Prettier. +- **Package Manager:** pnpm. + +## Entry Point (`src/index.js`) + +Exports the main classes/functions: + +- `DB` and `SQLite` from `./DB` +- `EventQueue` from `./EventQueue` +- `EventSourcingDB`, `applyResult`, `ESModel` from `./EventSourcingDB` +- `JsonModel` from `./JsonModel` + + diff --git a/agent-todo.md b/agent-todo.md new file mode 100644 index 0000000..a18ee58 --- /dev/null +++ b/agent-todo.md @@ -0,0 +1,165 @@ +# StratoDB Refactoring TODO List + +This document outlines the necessary steps to refactor and modernize the `strato-db` codebase. The tasks are ordered to minimize disruption and build upon previous steps. + +**Overall Goals:** + +- Improve maintainability, readability, and type safety. +- Enhance performance by using a synchronous SQLite driver off the main thread. +- Provide a more user-friendly API. +- Ensure proper type definitions are generated for package consumers. + +--- + +## Phase 1: Database Backend and Concurrency Model + +**Goal:** Replace `sqlite3` with `better-sqlite3` and move all database operations to worker threads. + +**Tasks:** + +1. **Install Dependencies:** + + - Remove `sqlite3`. + - Add `better-sqlite3`. + - Add a worker pool library (e.g., `piscina`) or implement a custom pool. + +2. **Design Worker Communication:** + + - Define the message format for commands (e.g., `run`, `get`, `all`, `exec`, `prepare`, transaction operations) and results/errors between the main thread and workers. + - Decide on data transfer strategy (structured cloning vs. `SharedArrayBuffer` where applicable/beneficial, especially for reads returning large datasets). Consider the overhead of serialization. + +3. **Implement Worker Logic:** + + - Create worker script(s) that initialize `better-sqlite3` database connections. + - Implement message handlers within the worker to execute corresponding `better-sqlite3` methods synchronously. + - Handle database connection management within workers (opening, closing). + - Ensure proper error handling and propagation back to the main thread. + +4. **Implement Worker Pool:** + + - Set up the worker pool (e.g., using `piscina`). + - Configure pool size (consider read vs. write workers if necessary - writes _must_ be serialized per DB file, reads can often be parallelized). A single writer thread per database instance and multiple reader threads is a common pattern. + +5. **Refactor `DB`, `SQLite`, `Statement` Modules:** + + - Remove the existing `sqlite3`-based implementation (`src/DB/SQLite.js`, `src/DB/Statement.js`). + - Modify `src/DB/DB.js` (or create a new abstraction) to interact with the worker pool instead of directly with the database. + - All methods in this layer (`get`, `all`, `run`, `exec`, `prepare`, `transaction`, etc.) must become asynchronous, returning Promises that resolve/reject based on worker responses. + - Adapt the `Statement` concept if needed, potentially managing prepared statements within specific workers or re-preparing as necessary. `better-sqlite3`'s statement caching might simplify this. + +6. **Update Dependent Modules:** + - Modify `JsonModel`, `EventQueue`, and `EventSourcingDB` to use the new asynchronous DB layer methods (replace synchronous assumptions or promise-based `sqlite3` calls with `await` on the new worker-based calls). + +**Considerations:** + +- `better-sqlite3` is synchronous. All calls _must_ be offloaded to workers to avoid blocking the main thread. +- Transaction management across worker messages needs careful design. `better-sqlite3` transactions are synchronous within the worker. The main thread might send a "beginTransaction" command, followed by multiple operations, then "commit" or "rollback". +- Error handling across thread boundaries. +- Performance impact of data serialization/deserialization between threads. + +--- + +## Phase 2: TypeScript Conversion + +**Goal:** Convert the entire `src` directory to TypeScript and configure the build process for type generation. + +**Tasks:** + +1. **Setup TypeScript Build:** + + - Ensure `typescript` and necessary `@types/*` packages are dev dependencies. + - Configure `tsconfig.json` for compilation: + - Set `outDir` (e.g., `./dist`). + - Set `declaration: true` and `declarationDir` (e.g., `./dist/types` or `./dist`). + - Set `module` to `NodeNext` (or similar modern target) to support both CJS and ESM output if needed, or configure separate builds. + - Ensure `allowJs: false` (or remove it) once conversion is complete. + - Set `rootDir: ./src`. + - Configure `package.json`: + - Define `types` field pointing to the main declaration file (e.g., `./dist/index.d.ts`). + - Define `exports` field for proper CJS/ESM resolution. + - Update build scripts (`"scripts": { "build": "tsc" }`). Ensure the build _only_ compiles (no bundling). + +2. **Convert Files Incrementally:** + - Rename `.js` files in `src` to `.ts`. + - Start adding types, beginning with core classes and functions. Leverage JSDoc annotations where present. + - Address TypeScript compiler errors (`tsc --noEmit`). + - Pay close attention to types related to the worker communication layer developed in Phase 1. Define clear interfaces for messages and results. + - Convert test files (`.test.js` to `.test.ts`) and update test setup/configuration (e.g., `vite.config.ts` or `vitest.config.ts`) if necessary. + +**Considerations:** + +- Address `any` types progressively. Aim for strong typing. +- Use TypeScript features like interfaces, generics, enums effectively. +- Ensure generated `.d.ts` files accurately represent the public API. + +--- + +## Phase 3: Code Splitting and Refactoring + +**Goal:** Break down large implementation files into smaller, more focused modules. + +**Tasks:** + +1. **Identify Large Files:** + + - Target files like `JsonModel.ts`, `EventSourcingDB.ts`, `DB.ts` (the refactored version), `EventQueue.ts`. + +2. **Refactor `JsonModel.ts`:** + + - Move helper functions (like `parseRow`, `_makeSetFn`, query building logic, cursor logic) into separate files within `src/JsonModel/`. + - Potentially group methods by functionality (e.g., CRUD operations, search operations, caching logic) into separate files/modules. + +3. **Refactor `EventSourcingDB.ts`:** + + - Extract event processing steps (preprocessing, reduction, application, derivation logic) into separate functions or modules within `src/EventSourcingDB/`. + - Move polling/waiting logic (`startPolling`, `waitForQueue`, etc.) into its own module. + - Separate sub-event handling logic. + +4. **Refactor `DB.ts` / Worker Interaction Layer:** + + - Ensure the logic for interacting with the worker pool is well-organized. If `DB.ts` becomes complex, split it (e.g., transaction handling, statement preparation proxy). + +5. **Refactor `EventQueue.ts`:** + - If complex, separate logic for adding events vs. retrieving/waiting for events. + +**Considerations:** + +- Maintain clear module boundaries and responsibilities. +- Use barrel files (`index.ts`) within directories to re-export public APIs of the split modules if needed, but avoid overly complex export chains. +- Ensure internal imports are updated correctly after moving files. + +--- + +## Phase 4: Factory Functions API + +**Goal:** Provide a simplified API for creating instances of core StratoDB components. + +**Tasks:** + +1. **Design Factory Functions:** + + - Define functions like `createJsonModel(...)`, `createEventQueue(...)`, `createEventSourcingDB(...)`, `createDb(...)`. + - Determine the necessary parameters for each factory (e.g., database file path, model definitions, queue options). + - Decide how dependencies are managed (e.g., does `createEventSourcingDB` internally create the DB and Queue instances, or does it accept them as parameters?). Accepting instances often provides more flexibility. + +2. **Implement Factory Functions:** + + - Create a new API entry point file (e.g., `src/factories.ts` or directly in `src/index.ts`). + - Implement the functions to encapsulate the `new Class(...)` calls. + - Hide complex configuration options behind sensible defaults if possible. + +3. **Update Exports:** + + - Export the factory functions from the main entry point (`src/index.ts`). + - Decide whether to continue exporting the classes directly. It might be useful for advanced users or testing, but the factories should be the recommended approach. + +4. **Update Documentation/Examples:** + - Modify README and any usage examples to use the new factory functions. + +**Considerations:** + +- API design should be intuitive and cover common use cases easily. +- Balance simplicity with flexibility. +- Ensure type safety carries through the factory functions. + +--- diff --git a/src/EventQueue.js b/src/EventQueue.js index f9ff514..25b3ad8 100644 --- a/src/EventQueue.js +++ b/src/EventQueue.js @@ -124,6 +124,9 @@ class EventQueueImpl extends JsonModel { return this.getMaxV() } + // We check every minute because we have a race condition where currentV is stale sometimes + _lastRealCheck = 0 + /** * Get the highest version stored in the queue. * @@ -132,12 +135,15 @@ class EventQueueImpl extends JsonModel { async getMaxV() { if (this._addP) await this._addP - const dataV = await this.db.dataVersion() - if (this.currentV >= 0 && this._dataV === dataV) { - // If there was no change on other connections, currentV is correct - return this.currentV + if (this._lastRealCheck + 60_000 > Date.now()) { + const dataV = await this.db.dataVersion() + if (this.currentV >= 0 && this._dataV === dataV) { + // If there was no change on other connections, currentV is correct + return this.currentV + } + this._dataV = dataV } - this._dataV = dataV + this._lastRealCheck = Date.now() if (this._maxSql?.db !== this.db) this._maxSql = this.db.prepare( `SELECT MAX(v) AS v from ${this.quoted}`,