Conversation
|
@Jegp This is a fantastic addition! It might be nice to add the drivers dependency as a pip extra (so that users can use Format support for non-monotonic timestamps varies:
Besides file formats, most algorithms implicitly assume that timestamps are monotonic. Not doing so makes quite a few operations much more complicated (time deltas used in time surfaces can be negative and break look-up tables, tracking algorithms need to sort the events since they rely on forward Bayesian models, regularization needs to buffer more data and sort non-monotonic events to ensure consistency, and so on). Overall, I am tempted to treat non-monotonic timestamps as bug that should be fixed upstream in the Inivation drivers (they most likely indicate some sort of buffer corruption). As a workaround, we could either sort the events by timestamp (potentially bad for latency since one needs to store a copy of the current buffer before dispatching it, in case events in a future buffer have smaller timestamps) or "freeze time" until it increases again (the timestamp sequence One exception to monotonicity are multiplexed streams (for instance events + frames or events + triggers). Whilst guaranteed monotonicity would be nice to have for such streams (to implement hybrid frame/events algorithms, for instance), it can be hard to implement without sorting data. It is also somewhat ill-defined since frames have multiple timestamps (frame trigger, integration start, integration end, and readout end). This is even worse in the case of a rolling shutter (where technically one needs one timestamp per row to properly describe the frame). At any rate, this is beyond the current scope of Faery (which focuses on events) so I think that the aedat decoder and inivation driver should dump the frame data as it arrives from the source and only check the monotonicity of events. |
|
Thank you for your detailed comments @aMarcireau! I see that it's a good idea to ensure the ordering. If you agree to retain the flag, I definitely think it should be I like the idea of "freeze time", but isn't that costly? Particularly because it can't really be paralleled. Alternatively, wouldn't it be faster to do something like this? events = ... # Assume an array of timestamps
min_timestamp = events[0]
events = min_timestamp + np.clip(events - min_timestamp, 0)I btw agree that this should happen in the event drivers and definitely not be a part of Faery. For now. It may be nice to have a filter in the future. |
This PR fixes #1 by adding support for Inivation camera streaming.
NOTE: this also adds a flag
enforce_monotonic_timestampsto the file encoders to allow timestamps arriving out-of-order. That is, unfortunately, common for camera packets (at least from Inivation).This PR sets the flag to false by default. Partly because it'll be hard to configure this differently for different kinds of input, partly because I'm not sure I understand the need for the check. Are there specific readers/parsers that don't accept timestamps out of order? Care to comment @aMarcireau?