Auto conflict resolution for `compact` then `update/delete`

[wjones127]

Scope

• compact then update/delete

compact-compact conflicts are out of scope, as they are pretty rare.

Design

When compaction happens, the row addresses change. To be able to handle row-level conflicts, we will need compaction to emit a change_of_address mapping.

Update and delete transactions generate an affected_rows bitmap that tell them which row addresses need to be masked from the table. When they rebase on top of a compaction, they need run their affected_rows bitmap through the change_of_address mapping.

When compaction rebases on top of update/delete transactions, it needs to run the new deletion files through it's change_of_address mapping to generate new deletion files for the compacted data. Rows that were originally deleted and compacted out of the data will not appear in the mapping and will be dropped. But rows that have been deleted after will be mapped to an actual location.

TODO: compact then update/delete

• Serialize the change of address somewhere, likely the transaction file
  • Investigate whether we should instead use FragmentReuseIndex
• Modify TransactionRebase to read the change of address when other committers create that
• Make sure the change of address is added to metadata cache when possible
• Add a test validating if we commit compaction and then update, we the update can successfully rebase the compaction:
  • Start an upsert with MergeInsertJob::execute_uncommitted()
  • Run compaction, (parameterize test on doing this without and without same Session)
  • Commit Compaction using CommitBuilder, passing along affected_rows to make sure conflict resolution can happen.
  • Validate expected updates happen, and no extra rows were added.

[wjones127]

The change of address mapping is basically the fragment reuse index, which we already write.