feat: add live state LtHash library#2647

Merged

blindchaser merged 7 commits intomainfrom

yiren/lthash-p0

Jan 2, 2026

Contributor

blindchaser commented Dec 30, 2025

Describe your changes and provide context

Add standalone LtHash implementation (vector, checksum, serialization)
Add parallel delta computation helpers + timings

Testing performed to validate your change

blindchaser added 2 commits

December 29, 2025 21:56


          lthash

290b1c4


          make lthash lib only

61eefbb

github-actions bot commented Dec 30, 2025 •

edited

Loading

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

Build	Format	Lint	Breaking	Updated (UTC)
`✅ passed`	`✅ passed`	`✅ passed`	`✅ passed`	Jan 2, 2026, 6:19 PM


          add length-prefixed fields in serializeKV

726848b

codecov bot commented Dec 30, 2025 •

edited

Loading

Codecov Report

❌ Patch coverage is 81.81818% with 46 lines in your changes missing coverage. Please review.
✅ Project coverage is 43.78%. Comparing base (e6cb591) to head (fc22c7a).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
sei-db/sc/live_state/lthash/lthash.go	78.35%	23 Missing and 6 partials ⚠️
sei-db/sc/live_state/lthash/api.go	85.71%	9 Missing and 8 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2647      +/-   ##
==========================================
+ Coverage   43.71%   43.78%   +0.06%     
==========================================
  Files        1902     1904       +2     
  Lines      158679   158932     +253     
==========================================
+ Hits        69372    69587     +215     
- Misses      82918    82941      +23     
- Partials     6389     6404      +15

Flag	Coverage Δ
sei-chain	`45.82% <81.81%> (+0.13%)`	⬆️
sei-cosmos	`38.20% <ø> (+<0.01%)`	⬆️
sei-db	`69.06% <ø> (ø)`
sei-tendermint	`47.29% <ø> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
sei-db/sc/live_state/lthash/api.go	`85.71% <85.71%> (ø)`
sei-db/sc/live_state/lthash/lthash.go	`78.35% <78.35%> (ø)`

... and 20 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

github-advanced-security bot found potential problems

View reviewed changes

sei-db/sc/live_state/lthash/api.go Fixed Show fixed Hide fixed

sei-db/sc/live_state/lthash/api.go Fixed Show fixed Hide fixed

sei-db/sc/live_state/lthash/api.go Fixed Show fixed Hide fixed

sei-db/sc/live_state/lthash/api.go Fixed Show fixed Hide fixed

sei-db/sc/live_state/lthash/api.go Fixed Show fixed Hide fixed

sei-db/sc/live_state/lthash/api.go Fixed Show fixed Hide fixed

sei-db/sc/live_state/lthash/api.go Fixed Show fixed Hide fixed

sei-db/sc/live_state/lthash/api.go Fixed Show fixed Hide fixed

sei-db/sc/live_state/lthash/api.go Fixed Show fixed Hide fixed

sei-db/sc/live_state/lthash/api.go Fixed Show fixed Hide fixed


          fix lint

88b64cb

blindchaser added the non-app-hash-breaking label

yzang2019 reviewed

View reviewed changes

sei-db/sc/live_state/lthash/parallel.go Outdated Show resolved Hide resolved

yzang2019 reviewed

View reviewed changes

sei-db/sc/live_state/lthash/lthash.go Outdated Show resolved Hide resolved


          refactor(lthash): rename parallel.go to api.go and cleanup types

399f082

github-advanced-security bot found potential problems

View reviewed changes

sei-db/sc/live_state/lthash/api.go

+              // computeDelta computes the LtHash delta for a changeset.
+              func computeDelta(kvPairs []KVPairWithLastValue, numWorkers int) (*LtHash, *LtHashTimings) {
+              	totalStart := time.Now()

Check warning

Code scanning / CodeQL

Calling the system time Warning

Calling the system time may be a possible source of non-determinism

sei-db/sc/live_state/lthash/api.go

+              	}
+              	// Phase 1: Serialize
+              	serializeStart := time.Now()

Check warning

Code scanning / CodeQL

Calling the system time Warning

Calling the system time may be a possible source of non-determinism

sei-db/sc/live_state/lthash/api.go

+              	serializeNs := time.Since(serializeStart).Nanoseconds()
+              	// Phase 2: Hash (parallel)
+              	blake3Start := time.Now()

Check warning

Code scanning / CodeQL

Calling the system time Warning

Calling the system time may be a possible source of non-determinism

sei-db/sc/live_state/lthash/api.go

+              	blake3Ns := time.Since(blake3Start).Nanoseconds()
+              	// Phase 3: MixIn/MixOut (parallel)
+              	mixStart := time.Now()

Check warning

Code scanning / CodeQL

Calling the system time Warning

Calling the system time may be a possible source of non-determinism

sei-db/sc/live_state/lthash/api.go

+              	mixNs := time.Since(mixStart).Nanoseconds()
+              	// Phase 4: Merge
+              	mergeStart := time.Now()

Check warning

Code scanning / CodeQL

Calling the system time Warning

Calling the system time may be a possible source of non-determinism

sei-db/sc/live_state/lthash/api.go

+              	serializeNs := time.Since(serializeStart).Nanoseconds()
+              	// Phase 2: Hash
+              	blake3Start := time.Now()

Check warning

Code scanning / CodeQL

Calling the system time Warning

Calling the system time may be a possible source of non-determinism

sei-db/sc/live_state/lthash/api.go

+              	blake3Ns := time.Since(blake3Start).Nanoseconds()
+              	// Phase 3: MixIn/MixOut
+              	mixStart := time.Now()

Check warning

Code scanning / CodeQL

Calling the system time Warning

Calling the system time may be a possible source of non-determinism

sei-db/sc/live_state/lthash/api.go

+              package lthash
+              import (
+              	"runtime"

Check notice

Code scanning / CodeQL

Sensitive package import Note

Certain system packages contain functions which may be a possible source of non-determinism

sei-db/sc/live_state/lthash/api.go

Comment on lines +110 to +121

+              		go func(startIdx, endIdx int) {
+              			defer wg.Done()
+              			for i := startIdx; i < endIdx; i++ {
+              				skv := serializedPairs[i]
+              				if skv.lastSerialized != nil {
+              					lthashPairs[i].lastLth = hash(skv.lastSerialized)
+              				}
+              				if skv.newSerialized != nil {
+              					lthashPairs[i].newLth = hash(skv.newSerialized)
+              				}
+              			}
+              		}(start, end)

Check notice

Code scanning / CodeQL

Spawning a Go routine Note

Spawning a Go routine may be a possible source of non-determinism

sei-db/sc/live_state/lthash/api.go

Comment on lines +141 to +156

+              		go func(workerID int, startIdx, endIdx int) {
+              			defer wg.Done()
+              			workerLth := getLtHashFromPool()
+              			for i := startIdx; i < endIdx; i++ {
+              				lp := lthashPairs[i]
+              				if lp.lastLth != nil {
+              					workerLth.MixOut(lp.lastLth)
+              					putLtHashToPool(lp.lastLth)
+              				}
+              				if lp.newLth != nil {
+              					workerLth.MixIn(lp.newLth)
+              					putLtHashToPool(lp.newLth)
+              				}
+              			}
+              			results[workerID] = workerLth
+              		}(w, start, end)

Check notice

Code scanning / CodeQL

Spawning a Go routine Note

Spawning a Go routine may be a possible source of non-determinism

yzang2019 approved these changes

View reviewed changes


          lint

5c5d323

jewei1997 approved these changes

View reviewed changes


          Merge branch 'main' into yiren/lthash-p0

fc22c7a

blindchaser enabled auto-merge (squash)

January 2, 2026 18:40

blindchaser merged commit 25bf825 into main

50 of 52 checks passed

blindchaser deleted the yiren/lthash-p0 branch

January 2, 2026 18:48

yzang2019 added a commit that referenced this pull request


          Merge branch 'main' into yzang/seidb-restructure

b7268ff

* main:
  feat: add live state LtHash library (#2647)
  fix: correct TestAsyncComputeMissingRanges (#2655)

yzang2019 pushed a commit that referenced this pull request


          feat: add live state LtHash library (#2647)

d439412

- Add standalone LtHash implementation (vector, checksum, serialization)
- Add parallel delta computation helpers + timings

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

non-app-hash-breaking