Seidb restructure by yzang2019 · Pull Request #2653 · sei-protocol/sei-chain

yzang2019 · 2026-01-02T11:20:27Z

Describe your changes and provide context

Refactor and restructure seidb to prepare for v3. The new structure will look like this:

This PR made changes of:

Move files around to different folder names
Update all dependency references based on the new structure
Add new placeholders for future preparation
Add a new basic pebbledb implementation

Testing performed to validate your change

github-actions · 2026-01-02T11:21:09Z

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

Build	Format	Lint	Breaking	Updated (UTC)
`✅ passed`	`✅ passed`	`✅ passed`	`✅ passed`	Jan 5, 2026, 4:25 PM

codecov · 2026-01-02T11:26:19Z

Codecov Report

❌ Patch coverage is 5.55556% with 102 lines in your changes missing coverage. Please review.
✅ Project coverage is 43.75%. Comparing base (9fda687) to head (bbb7a84).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
sei-db/db_engine/pebbledb/kv/db.go	0.00%	53 Missing ⚠️
sei-db/db_engine/pebbledb/kv/batch.go	0.00%	45 Missing ⚠️
sei-db/state_db/sc/memiavl/snapshot.go	0.00%	2 Missing ⚠️
sei-cosmos/storev2/rootmulti/store.go	66.66%	1 Missing ⚠️
sei-db/state_db/sc/store.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2653      +/-   ##
==========================================
- Coverage   43.78%   43.75%   -0.03%     
==========================================
  Files        1904     1906       +2     
  Lines      158932   159030      +98     
==========================================
+ Hits        69585    69589       +4     
- Misses      82944    83041      +97     
+ Partials     6403     6400       -3

Flag	Coverage Δ
sei-chain	`45.74% <1.94%> (-0.08%)`	⬇️
sei-cosmos	`38.20% <75.00%> (-0.01%)`	⬇️
sei-db	`69.06% <100.00%> (ø)`
sei-tendermint	`47.33% <ø> (+0.05%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
app/app.go	`70.81% <ø> (ø)`
app/seidb.go	`82.69% <ø> (ø)`
app/test_helpers.go	`32.82% <ø> (ø)`
evmrpc/server.go	`91.54% <ø> (ø)`
evmrpc/watermark_manager.go	`74.46% <ø> (ø)`
sei-cosmos/storev2/commitment/store.go	`9.37% <100.00%> (ø)`
sei-cosmos/storev2/state/store.go	`0.00% <ø> (ø)`
sei-db/changelog/changelog/changelog.go	`53.73% <ø> (ø)`
sei-db/changelog/changelog/subscriber.go	`0.00% <ø> (ø)`
sei-db/changelog/changelog/utils.go	`58.20% <ø> (ø)`
... and 58 more

... and 16 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

blindchaser · 2026-01-02T19:54:59Z

sei-db/db_engine/pebbledb/kv/db.go

+}
+
+// Get returns value by key.
+func (db *Database) Get(key []byte) ([]byte, error) {


from ai: Pebble Get() returns a value that is only valid until the returned closer is closed.
value is returned while closer.Close() is also called, which means the caller may observe invalid / corrupted data? We should copy value before closing the closer. we can do:

func (db *Database) Get(key []byte) ([]byte, error) { value, closer, err := db.storage.Get(key) if err != nil { if errors.Is(err, pebble.ErrNotFound) { return nil, nil } return nil, errors.WithStack(err) } defer closer.Close() out := append([]byte(nil), value...) // copy return out, nil }

That's interesting, I think this is copied from v3 and we didn't really see this issue at all in v3. Will still fix it though

blindchaser · 2026-01-02T20:03:01Z

sei-db/db_engine/pebbledb/kv/db.go

+// OpenDB opens an existing or create a new database.
+func OpenDB(dbPath string) *Database {
+	cache := pebble.NewCache(1024 * 1024 * 512)
+	defer cache.Unref()


defer cache.Unref() will run when OpenDB() returns, potentially releasing the cache while the DB is still using it?

We have to keep the defer cache.Unref() - it's the correct pattern recommended by PebbleDB. The idea is to keep reference counting down to 0 when closing the DB.

i double checked this issue. the defer executes immediately when OpenDB returns, not when the DB is closed though

blindchaser · 2026-01-02T20:14:30Z

sei-db/db_engine/pebbledb/kv/batch.go

+}
+
+func getStorePrefix(storeKey string) []byte {
+	return []byte(fmt.Sprintf("s/k:%s/", storeKey))


nitpick perf improve:

func getStorePrefix(storeKey string) []byte { // "s/k:" + storeKey + "/" b := make([]byte, 0, len("s/k:/")+len(storeKey)) b = append(b, "s/k:"...) b = append(b, storeKey...) b = append(b, '/' ) return b }

blindchaser · 2026-01-02T20:30:41Z

sei-db/state_db/sc/types/store.go

+
+	GetProof(key []byte) *ics23.CommitmentProof
+
+	io.Closer


It seems ModuleStore inherits io.Closer, but commitment.NewStore wraps it and never calls Close. Is ModuleStore.Close a no-op? or we should also have it in commitment.Store and call close() on rootmulti

It's called by CMS I think? But this piece of code need more refactory in the future, I would not really touch it in this PR yet

* main: feat: add live state LtHash library (#2647) fix: correct TestAsyncComputeMissingRanges (#2655)

jewei1997 · 2026-01-05T20:29:23Z

sei-db/db_engine/parquet/placeholder.go

@@ -0,0 +1 @@
+package parquet


jewei1997 · 2026-01-05T20:34:04Z

sei-db/ledger_db/event/placeholder.go

@@ -0,0 +1 @@
+package event


is this for evm logs?

## Describe your changes and provide context Refactor and restructure seidb to prepare for v3. The new structure will look like this: <img width="736" height="349" alt="image" src="https://github.com/user-attachments/assets/de643253-8ffc-4631-9152-ec2a6b5f2676" /> This PR made changes of: 1. Move files around to different folder names 2. Update all dependency references based on the new structure 3. Add new placeholders for future preparation 4. Add a new basic pebbledb implementation ## Testing performed to validate your change

Refactor and restructure seidb to prepare for v3. The new structure will look like this: <img width="736" height="349" alt="image" src="https://github.com/user-attachments/assets/de643253-8ffc-4631-9152-ec2a6b5f2676" /> This PR made changes of: 1. Move files around to different folder names 2. Update all dependency references based on the new structure 3. Add new placeholders for future preparation 4. Add a new basic pebbledb implementation

yzang2019 added 2 commits January 2, 2026 03:15

Restructure seidb for v3

915a3dc

Fix go mod

206931d

yzang2019 added the non-app-hash-breaking label Jan 2, 2026

yzang2019 requested review from Kbhat1, blindchaser and jewei1997 January 2, 2026 14:56

yzang2019 added 3 commits January 2, 2026 07:04

Fix lint

a876284

Fix lint

ad238ab

Fix unit test path for rocksdb

98c663c

blindchaser reviewed Jan 2, 2026

View reviewed changes

yzang2019 added 2 commits January 2, 2026 13:37

Address comments

c8f6aa2

Fix unit test

9069021

blindchaser approved these changes Jan 2, 2026

View reviewed changes

yzang2019 and others added 4 commits January 5, 2026 02:35

Merge branch 'main' into yzang/seidb-restructure

b7268ff

* main: feat: add live state LtHash library (#2647) fix: correct TestAsyncComputeMissingRanges (#2655)

Fix golint and merge latest

0a4174c

fix go lint

1cabf1f

Merge branch 'main' into yzang/seidb-restructure

bbb7a84

jewei1997 reviewed Jan 5, 2026

View reviewed changes

sei-db/db_engine/parquet/placeholder.go

@@ -0,0 +1 @@

package parquet

Copy link
Copy Markdown

Contributor

jewei1997 Jan 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

jewei1997 reviewed Jan 5, 2026

View reviewed changes

sei-db/ledger_db/event/placeholder.go

@@ -0,0 +1 @@

package event

Copy link
Copy Markdown

Contributor

jewei1997 Jan 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this for evm logs?

jewei1997 approved these changes Jan 5, 2026

View reviewed changes

yzang2019 merged commit d7d72d9 into main Jan 5, 2026
49 of 52 checks passed

yzang2019 deleted the yzang/seidb-restructure branch January 5, 2026 22:28

Conversation

yzang2019 commented Jan 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Describe your changes and provide context

Testing performed to validate your change

Uh oh!

github-actions bot commented Jan 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Jan 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

blindchaser Jan 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yzang2019 Jan 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

yzang2019 commented Jan 2, 2026 •

edited

Loading

github-actions bot commented Jan 2, 2026 •

edited

Loading

codecov bot commented Jan 2, 2026 •

edited

Loading

blindchaser Jan 2, 2026 •

edited

Loading

yzang2019 Jan 2, 2026 •

edited

Loading