Broad ideas for an event store

[windley]

One enhancement we'll need at some point is an event source within a pico. One that can be read for logging purposes and possibly used for playback, reliability etc.

This article has information on building an event source in MondgoDB.

Mark @solargroovy had these thoughts:

I'm not sure if our Perl driver supports tailable cursors.

We all but store events now with scheduled events

One issue is that a capped collection has fixed total space. If you are passing env info someone could DOS you by sending to much data.

You could use a ttl index to get similar results.

You could guarantee delivery and provide a replay log of events

A node front end/event collector could allow you to distribute event processing or have certain channels with a higher priority than others

We'd want:

• work on a pico-by-pico basis.
• filterable by ECI

[windley]

The pico event store can snapshot current state from time to time and then just store events from the snapshot. This provides a nice backup for the pico.

This also provides an audit log from the time of the last snapshot.