7.4.0: add store.use() middleware

[bitmage]

Summary

This PR adds a store.use(tableId, handler) middleware API that enables business rule enforcement on both local mutations and CRDT sync. This addresses a fundamental limitation identified in #280: listeners cannot prevent invalid data from entering the store—they only observe changes after they're applied.

The .use() format is inspired by Express: hopefully that provides some familiar ergonomics.

There's basically two scenarios I could see people using Tinybase in: client/server and p2p nodes. In both cases, middleware provides a necessary function. In the case of a server it allows you to validate your incoming data. And in the case of p2p nodes: validations are equally important, if not more.

Some really nice patterns result from this, for instance:

store.use('jobs', (rowId, cells) => {
  if (!cells.name || typeof cells.name !== 'string') {
    store.setRow('errors', `err-${rowId}`, {
      table: 'jobs',
      rowId,
      message: 'Job name is required',
      timestamp: Date.now(),
    });
    return null; // reject the row
  }
  return cells; // accept
});

This allows us to create error handling which runs on both client and server (or all nodes in the p2p context). The client blocks bad data and allows the UI to render the error. And in the case where the validation needs to run on the server (it needs access to priveledged data such as cryptographic keys for instance) then the server has the opportunity to bubble errors up through the same store (perhaps tagged with {source: 'server'}).

Middleware gives us a place to create these patterns while also being flexible for an inumerable amount of future use cases. The key innovation here is that this prevents bad data from getting into the model, and prevents peers from accepting bad data over the wire.

How did you test this change?

I added a new test file specifically for middleware.ts. Many additional tests are added.

Risks

It's definitely possible to write infinite cycles using this. That could be a footgun. That's also true of mutating listeners though. I think the only way to protect the user would be through some kind of cycle detection. I'm not sure how you would do that other than code analysis.

Implementation

I tried in every way to take a light touch. I thought at first that we could do this by a small change to MergeableStore. Alas, I later realized:

1. I needed to improve the ergonomics by iterating through the change set and presenting a row-based interception of values
2. modifying MergeableStore wasn't sufficient, I also needed to modify Store in order to prevent local additions

I did my best to stick to the code idioms of the project. The bulk of the code changes are in two files: middleware.ts and middleware.test.ts.

I did run over 6000 tests and I don't think we have regressions, but it's hard for me to say because running the full test suite on my laptop consumes all available memory and hard crashes my laptop. This was true as well in the unaltered codebase before any of my changes. I presume you have CI, or a beefier machine at your disposal.

Peace. I know this is a big change, so let me know if there's anything I can do to make it more digestible.

[jamesgpearce]

Wowzers! Let me get to this amazing contribution...

[bitmage]

@jamesgpearce are you a Clojure guy? Your coding style reminds me of Clojure or some dialect of Lisp. We may share some inspiration: Clojure and FP were what originally got me turned onto Javascript, around 15 years ago.

[jamesgpearce]

Haha not particularly! This started from an obsession with reducing the file size, which means that I start to (pathologically!) functionalize more and more verbose boilerplate idioms... BTW don't worry about the house style in PRs... I can always wrap things up later.

[jamesgpearce]

This is most excellent work! It has taken me a little while to digest it all. A few things, off the top of my mind:

• Fix sync propagation for mutating listener deletions #285 (I think) addresses your Mutating listener deletions during sync don't propagate back to source #280 issue. (Please check v7.3.4!) BUT I still like the middleware concept so we should move ahead. It's an alternative to mutator listeners, but the DX can feel more like you are gating writes before they are even written in instead of patching them up afterwards.

• Returning null - I will probably change to undefined since null is now a valid call value and I'm trying not to add additional semantic overloads from here on. Or maybe there's a 'canSet*' idiom that is literally just a gate on the writes, and a 'transformSet*' that can manipulate the values. Or maybe that's too much.

• I think we should collapse the two types together. Even if you've registered it on a single table, it's ok that you get sent its name. This is analogous to listeners that register themselves to all tables with the 'null' (!) wildcard.

• We might create a more meaningful API than 'use'. Idiomatically, Middleware (like Indexes, Relationships, Persisters etc etc etc) could be a thing that is created with a store as an argument, and (with a few surreptitious private APIs) might be seen to work as a wrapper around a store, rather than something a store seems to need to know about. Also means we can have this in a separate submodule.

• I am nervous about having the middleware having to know too much about the inner workings of sync, but I guess that is unavoidable. Unless we conceptualize middleware as a thing that guards the public API and mutator listeners being the thing that guard the inner transactions. Or maybe not hm. Not sure about a clean way forward here.

• Your test suite is amazing and insane. Thank you!

Let me merge this and riff on a few of these ideas. Before we push to 7.4. I'll also check all the other tests haven't regressed.

[jamesgpearce]

Merging into the beta branch for what hopefully becomes v8!

[jamesgpearce]

Eh, that didn't work as I expected. Apologies if that disrupted your own main branch.

[jamesgpearce]

Cherry picked this as 3271e4c into the beta branch!

[jamesgpearce]

I do see some test regressions, but I'm on it!

[bitmage]

Very cool. Thanks James, its exciting to see it being incorporated. Appreciate your time in reviewing it. Question though: Is a mutating listener semantically equivalent? I think a listener would only receive changes after they have been applied to internal state, vs. gating/protecting that state from changes? I presume them to be different, but maybe I'm missing something about the usage of a mutating listener. You're right about DX - feels good to have something that feels like a guarantee that the rule will be enforced. And ensures that listeners in turn (and other peers) only receive valid state. Thanks again!

…

-------- Original Message --------

On Sunday, 02/15/26 at 14:27 James Pearce ***@***.***> wrote: jamesgpearce left a comment [(tinyplex/tinybase#283)](#283 (comment)) I do see some test regressions, but I'm on it! — Reply to this email directly, [view it on GitHub](#283 (comment)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AACMM37PFSNCTG4HHQUACRT4MDQDVAVCNFSM6AAAAACQZHAV6KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTSMBVGIYTKMJWGY). You are receiving this because you authored the thread.Message ID: ***@***.***>

[jamesgpearce]

Yep I think people have options. But the middleware DX is clearer and understandable to be a 'pre-write' gate or transform. (I am making a small change to your paradigm to have middleware hooks run after schema checks). Mutating a change after it's been set will become more niche with this, I think.

[bitmage]

Ok, thanks for explaining.

…

-------- Original Message --------

On Monday, 02/16/26 at 01:36 James Pearce ***@***.***> wrote: jamesgpearce left a comment [(tinyplex/tinybase#283)](#283 (comment)) Yep I think people have options. But the middleware DX is clearer and understandable to be a 'pre-write' gate or transform. (I am making a small change to your paradigm to have middleware hooks run after schema checks). Mutating a change after it's been set will become more niche with this, I think. — Reply to this email directly, [view it on GitHub](#283 (comment)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AACMM35CZ5LKDTXNFT3Z3H34MF6RVAVCNFSM6AAAAACQZHAV6KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTSMBXGEZDKNJSGA). You are receiving this because you authored the thread.Message ID: ***@***.***>