Commitment evolving #13884
Description
Commitment evolving
- Research on prefetching branches using RO trie while Write trie computes update and do not use IO Research on prefetching branches using RO trie while Write trie computes update and do not use IO #14273
- make
unfoldmore readable commitment: replaceunfoldheuristics by named funcs/methods #13234 - fold: simplify branch collection by the end of folding (collect update in one place, not in 4)
- fold: do not use closure of
cellGetterto get cell from the row, use internal hph method instead - fold: avoid merge with previous
branchDatavalue of current branch data: encode whole row at once instead - memo: need to research if we can use same field
cell.hashto keep memoised state hashes (which essentially state roots of those cells). During my initial work, i noticed that cell.hash and cell.stateHash never matches but we keep 2 fields for each cell having same meaning but memo hash is safe to use while cell hash may be even some intermediate value. - Binary Patricia Trie revive: long forgotten bph must be ported once again and we must be able to easily run this commitment scheme as well, just do not do state root comparisons.
- Review Commitment interface to ease support of other commitment scheme
- Add support of
ERIGON_COMMITMENT_TRACEvariable to allow collection of statistics per each commitment run (levels heatmap, counters, skip rates, depths, fat accounts, whatever) into csv file so we could visualize stats using python and get better insights collect commitment processing data #14490 - Commitment: rehash code only with asserts #14536
- Commitment: remove unused parameter tempdir #14481
- Commitment: no embedded keccak #14482
- Use MaxUint64 as return value to
HistoryStartFrom()if no files present #17308 - Trie: reduce cell size - by using int16 for prefix lengths #17134
- integration print_commitment (already implemented)
- add and reserve 2 bytes for flags (trie variant, branchSqueezed flag
- Produce comprehensive documentation on HPH trie internals and tricks (memoisation, referenced key in branches)
Experiments
- Concurrent warmup for Commitment #15658
- Parallel subtrie computation for trie Commitment: parallel subtrie computation #13696
- Split read and write trie Hex patricia reader #13784
- Collect Witnesses and Proofs from ReadOnly trie without converting it into intermediate type
trie.Trie