Enhance vector clock performance #1158
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Change SelectVectorClock to return gateway_time with latest originator_sequence_id per originator_node_id and initialize originator stream cursors to improve vector clock performance
This updates the vector clock query and stream initialization to use gateway timestamps instead of raw envelopes. - Replace
SelectVectorClockSQL with a lateral join that returnsoriginator_node_id, latestoriginator_sequence_id, andgateway_time, and regenerate query code to scan intoGatewayTime time.Timein envelopes.sql.go and update the SQL in envelopes.sql. - Create indexesidx_gateway_envelopes_vector_clockandidx_gateway_envelopes_originator_node_idin 00016_vector_clock.up.sql and drop them in 00016_vector_clock.down.sql. - Introduce a lightweightcursor(sequenceID, timestampNS) and refactororiginatorStreamto track validation state using the cursor instead of the last envelope in originator_stream.go with matching test adjustments in originator_stream_test.go. - Initialize stream position fromSelectVectorClockby constructing acursorusingOriginatorSequenceIDandGatewayTime.UnixNano()insyncWorker.setupStreamin sync_worker.go.📍Where to Start
Start with
syncWorker.setupStreamin sync_worker.go, then review theSelectVectorClockchanges in envelopes.sql and the regenerated code in envelopes.sql.go.Macroscope summarized 8c414bd.