Skip to content

Use clean context snapshot for ProcessProposal gas validation (CON-173)#3197

Merged
wen-coding merged 13 commits intomainfrom
wen/fix_processProposalState
Apr 9, 2026
Merged

Use clean context snapshot for ProcessProposal gas validation (CON-173)#3197
wen-coding merged 13 commits intomainfrom
wen/fix_processProposalState

Conversation

@wen-coding
Copy link
Copy Markdown
Contributor

@wen-coding wen-coding commented Apr 4, 2026

Summary

checkTotalBlockGas in ProcessProposalHandler reads from processProposalState's context, which can contain stale writes from a previous round's optimistic processing. This PR snapshots a clean context at the start of each ProcessProposal call and uses it for all downstream reads (gas validation, upgrade plan checks, optimistic processing).

How it works

At the start of ProcessProposal, after prepareProcessProposalState sets up the header and consensus params, we branch a new CacheMultiStore from the clean source store (deliverState for block 1, committed root store for blocks 2+). This is a lightweight read-through cache — no data is copied, reads fall through to the parent.

The handler reassigns ctx to the clean snapshot so all downstream reads use it. A defensive error is returned if the clean context is not set up (i.e. if the handler is called directly instead of through ProcessProposal).

Changes

  • sei-cosmos/baseapp/abci.go: Snapshot clean context after prepareProcessProposalState, before calling the handler.
  • sei-cosmos/baseapp/baseapp.go: Add processProposalCleanCtx field and GetProcessProposalCleanContext() accessor.
  • sei-cosmos/baseapp/test_helpers.go: Add SetProcessProposalCleanCtx test helper.
  • app/app.go: Reassign ctx from clean snapshot; add comment that ProcessProposalHandler must not be called directly.
  • app/app_test.go, app/upgrade_test.go: Convert tests to go through ProcessProposal instead of calling handler directly.
  • app/optimistic_processing_test.go: Set up clean context in test setup for internal tests.

Test plan

  • TestProcessProposalCleanContextNotAffectedByHandler — verifies writes to processProposalState are NOT visible through clean context
  • TestInvalidProposalWithExcessiveGasWanted, TestInvalidProposalWithExcessiveGasEstimates, TestOverflowGas — gas validation tests
  • TestSkipOptimisticProcessingOnUpgrade — upgrade detection through clean context
  • TestProcessProposalHandlerPanicRecovery — panic recovery via ProcessProposal
  • All optimistic processing suite tests pass
  • All baseapp tests pass

🤖 Generated with Claude Code

processProposalState was only initialized once and reused across
rounds at the same height. If a round timed out, speculative state
from optimistic processing carried over into the next round's
ProcessProposal. This is unnecessary since the old speculative
state is stale and will never be committed.

Now we create a fresh CacheMultiStore on each ProcessProposal call,
except for the very first block where InitChain writes genesis state
into the cache without committing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 4, 2026

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedApr 9, 2026, 4:06 AM

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 4, 2026

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedApr 4, 2026, 9:16 PM

@codecov
Copy link
Copy Markdown

codecov bot commented Apr 4, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 58.66%. Comparing base (9bea80a) to head (eef4bb7).
⚠️ Report is 19 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3197      +/-   ##
==========================================
+ Coverage   58.64%   58.66%   +0.02%     
==========================================
  Files        2055     2055              
  Lines      168494   168601     +107     
==========================================
+ Hits        98810    98916     +106     
+ Misses      60900    60896       -4     
- Partials     8784     8789       +5     
Flag Coverage Δ
sei-chain-pr 56.09% <100.00%> (?)
sei-db 70.41% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
app/app.go 68.56% <100.00%> (+0.02%) ⬆️
sei-cosmos/baseapp/abci.go 61.92% <100.00%> (+0.39%) ⬆️
sei-cosmos/baseapp/baseapp.go 78.89% <100.00%> (+3.02%) ⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

wen-coding and others added 2 commits April 4, 2026 14:27
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Only create a fresh CacheMultiStore when the proposal hash differs
from the previous round. When the same hash is re-proposed after a
timeout, reuse the existing cache so optimistic processing results
remain valid.

On hash mismatch, the app layer now clears optimistic processing
state and starts a new goroutine on the fresh CacheMultiStore.
The old goroutine detects the replacement via hash check and
silently discards its results.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@wen-coding wen-coding changed the title Reset processProposalState between consensus rounds Reset processProposalState on proposal hash change Apr 5, 2026
Unconditionally create a fresh CacheMultiStore on every ProcessProposal
call (except block 1 where InitChain genesis state must be preserved).
Even for the same proposal hash, optimistic ProcessBlock writes (e.g.
address associations) change the results of state-dependent checks
like gasless eligibility, so the store must always be clean.

On the app side, always clear optimistic processing info and start a
new goroutine. The old goroutine detects replacement via hash check
and silently discards its results.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@wen-coding wen-coding changed the title Reset processProposalState on proposal hash change Always reset processProposalState between rounds Apr 5, 2026
wen-coding and others added 2 commits April 5, 2026 16:04
Keep baseapp processProposalState reset, but revert app.go
ProcessProposalHandler to main's original code to isolate which
layer causes the AppHash mismatch.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove processProposalState setup from InitChain — no longer needed
since setProcessProposalState now branches from deliverState for
block 1 (before genesis is committed to the root store).

ProcessProposal unconditionally creates a fresh CacheMultiStore on
every call, preventing speculative state from a previous round's
optimistic processing from leaking into state-dependent checks.

App layer (ProcessProposalHandler) is unchanged — shouldStartOptimistic
Processing prevents concurrent ProcessBlock goroutines.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@wen-coding wen-coding changed the title Always reset processProposalState between rounds Always reset processProposalState with fresh CacheMultiStore Apr 6, 2026
wen-coding and others added 3 commits April 5, 2026 17:05
SetProcessProposalStateToCommit now falls back to deliverState when
processProposalState is nil. This handles test helpers that call
Commit without going through ProcessProposal (e.g. sei-wasmd and
sei-ibc-go snapshotter tests).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Branching processProposalState from deliverState for block 1 does
not work because CacheMultiStore branching from a CacheMultiStore
does not reliably surface uncommitted writes (e.g. distribution
module's previous proposer).

Restore InitChain's processProposalState setup and preserve it for
block 1. Reset unconditionally for all subsequent blocks. The block 1
round timeout case (dirty state from round 0 leaking) is a
pre-existing issue tracked separately.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
checkTotalBlockGas now reads from deliverState context instead of
processProposalState context. deliverState always reflects the last
committed state (or genesis for block 1) and is never written to
between ProcessProposal calls, making gas validation immune to
speculative writes from optimistic processing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@wen-coding wen-coding changed the title Always reset processProposalState with fresh CacheMultiStore Use deliverState for gas validation in ProcessProposal Apr 6, 2026
wen-coding and others added 2 commits April 5, 2026 22:44
GetDeliverStateContext falls back to committed root store when
deliverState is nil (after Commit, before FinalizeBlock recreates it).
Copy consensus params from processProposalState context onto the
check context since they are not stored in the root store.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Snapshot a clean context at the start of ProcessProposal by branching
from the source store (deliverState for block 1, committed root store
for blocks 2+) rather than from processProposalState. This ensures
checkTotalBlockGas is immune to speculative writes from optimistic
processing, while preserving correct consensus params and header from
processProposalState's context.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@wen-coding wen-coding changed the title Use deliverState for gas validation in ProcessProposal Use clean context snapshot for gas validation in ProcessProposal Apr 6, 2026
@wen-coding wen-coding changed the title Use clean context snapshot for gas validation in ProcessProposal Use clean context for gas validation in ProcessProposal Apr 6, 2026
Branch clean context from source store (deliverState or cms) rather
than from processProposalState, since CacheMultiStore branching reads
through to the parent's cached writes.

Update gas validation tests to go through BaseApp.ProcessProposal
instead of calling ProcessProposalHandler directly, and to store
consensus params on deliverState before committing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@wen-coding wen-coding changed the title Use clean context for gas validation in ProcessProposal Use clean context for gas validation in ProcessProposal (CON-173) Apr 8, 2026
if !app.checkTotalBlockGas(ctx, req.Txs) {
// Use the clean context snapshotted at the start of ProcessProposal,
// before optimistic processing can dirty the store.
checkCtx := app.GetProcessProposalCleanContext()
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think we can just assign to ctx here. If ctx is dirty then it probably shouldn't be referenced anywhere below

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that makes sense, had to fix more tests which call ProcessProposalHandler directly (while I think in production we should always go through ProcessProposal).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, tried that and actually it didn't work:
The flow within a single ProcessProposalHandler call:

  1. ctx comes in as processProposalState.ctx (the writable store)
  2. Line 1222: ctx = cleanCtx — now ctx points to the read-only snapshot
  3. Line 1243: app.UpgradeKeeper.GetUpgradePlan(ctx) — reads from clean snapshot, fine
  4. Line 1252: goroutine launches, captures ctx
  5. Line 1261: app.ProcessBlock(ctx, ...) — writes to the clean snapshot's CacheMultiStore
    Those writes are orphaned. Nobody commits the clean snapshot branch back to processProposalState. When FinalizeBlock later commits processProposalState, the state changes from optimistic processing aren't there.
    So all integration tests now fail with "insufficient fund".

@wen-coding wen-coding changed the title Use clean context for gas validation in ProcessProposal (CON-173) Use clean context snapshot for ProcessProposal gas validation Apr 9, 2026
@wen-coding wen-coding changed the title Use clean context snapshot for ProcessProposal gas validation Use clean context snapshot for ProcessProposal gas validation (CON-173) Apr 9, 2026
ProcessBlock writes to ctx's store for optimistic processing, so ctx
must remain on processProposalState. The clean context is only used
for the read-only gas validation check.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@wen-coding wen-coding force-pushed the wen/fix_processProposalState branch from f95cb2e to eef4bb7 Compare April 9, 2026 04:05
Copy link
Copy Markdown
Collaborator

@masih masih left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @wen-coding 🙌

@wen-coding
Copy link
Copy Markdown
Contributor Author

Ran local docker cluster, stopped and restarted one node, looks happy, submitting

@wen-coding wen-coding added this pull request to the merge queue Apr 9, 2026
Merged via the queue into main with commit 1f4724d Apr 9, 2026
39 checks passed
@wen-coding wen-coding deleted the wen/fix_processProposalState branch April 9, 2026 18:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants