Skip to content

Add block header warp sync patch.#1792

Merged
sam0x17 merged 4 commits intodevnet-readyfrom
warp-sync-test6
Jul 1, 2025
Merged

Add block header warp sync patch.#1792
sam0x17 merged 4 commits intodevnet-readyfrom
warp-sync-test6

Conversation

@shamil-gadelshin
Copy link
Collaborator

@shamil-gadelshin shamil-gadelshin commented Jun 27, 2025

Description

This PR introduces a temporary patch for warp sync that was broken after the recent bad blocks issue. The patch switches the warp sync target block header provider from the grandpa to the exact block header that we get from our own API (wss://archive.chain.opentensor.ai). The second part of the patch adds a reference to the patched grandpa crate from Polkadot-sdk (version polkadot-stable2409-7).

Additional comments:

  • The warp sync provider was set behind the feature warp-sync-patch.
  • Grandpa crate patch removes the justification check from two blocks that correspond to our manual grandpa authority changes
  • The next steps are: remove API invocation (switch back to grandpa provider), remove grandpa crate patch
  • Polkadot updates will require updating granpda crate as well.
  • grandpa changes: https://github.com/opentensor/grandpa/tree/block-patch
  • node Cargo.toml contains a new dependency (rustls-webpki) that is required for Linux compilation

Build command (updated)

 cargo build -p node-subtensor --profile=production --features=metadata-hash

Run command

node-subtensor --chain ./subtensor/chainspecs/raw_spec_finney.json --base-path <BASE_PATH> --sync=warp --port 30333 --no-grandpa --no-mdns --database paritydb --db-cache 4096 --trie-cache-size 2048

Tested configuration

m7i.xlarge
or 4 high performance x86 cores, 16gb memory and 300gb storage

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Other (please describe):

Checklist

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have run cargo fmt and cargo clippy to ensure my code is formatted and linted correctly
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published in downstream modules

@shamil-gadelshin shamil-gadelshin added the skip-cargo-audit This PR fails cargo audit but needs to be merged anyway label Jun 27, 2025
@shamil-gadelshin shamil-gadelshin marked this pull request as draft June 27, 2025 15:00
@shamil-gadelshin
Copy link
Collaborator Author

Should I just remove the feature (warp-sync-patch)? We have docker files without the new flag. @sam0x17

@shamil-gadelshin shamil-gadelshin marked this pull request as ready for review June 30, 2025 13:05
@shamil-gadelshin
Copy link
Collaborator Author

I removed the new feature because of the existing docker workflows. Here is the new build command:

cargo build -p node-subtensor --profile=production --features=metadata-hash

@l0r1s l0r1s self-requested a review July 1, 2025 15:53
Copy link
Collaborator

@l0r1s l0r1s left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested on AMD Ryzen 9 7950X3D, works like a charm:

2025-07-01 17:53:33 💤 Idle (40 peers), best: #5902580 (0x3451…ee4b), finalized #5902232 (0x5743…57c5), ⬇ 13.4kiB/s ⬆ 12.8kiB/s
2025-07-01 17:53:36 🏆 Imported #5902581 (0x3451…ee4b → 0x2178…13ba)
2025-07-01 17:53:38 💤 Idle (40 peers), best: #5902581 (0x2178…13ba), finalized #5902232 (0x5743…57c5), ⬇ 45.4kiB/s ⬆ 28.8kiB/s
2025-07-01 17:53:43 💤 Idle (40 peers), best: #5902581 (0x2178…13ba), finalized #5902232 (0x5743…57c5), ⬇ 15.8kiB/s ⬆ 15.4kiB/s
2025-07-01 17:53:48 🏆 Imported #5902582 (0x2178…13ba → 0xd86a…0027)
2025-07-01 17:53:48 💤 Idle (40 peers), best: #5902582 (0xd86a…0027), finalized #5902232 (0x5743…57c5), ⬇ 143.9kiB/s ⬆ 142.0kiB/s
2025-07-01 17:53:53 💤 Idle (40 peers), best: #5902582 (0xd86a…0027), finalized #5902232 (0x5743…57c5), ⬇ 36.2kiB/s ⬆ 27.5kiB/s
2025-07-01 17:53:58 💤 Idle (40 peers), best: #5902582 (0xd86a…0027), finalized #5902232 (0x5743…57c5), ⬇ 23.8kiB/s ⬆ 22.9kiB/s
2025-07-01 17:54:00 🏆 Imported #5902583 (0xd86a…0027 → 0x0527…004b)
2025-07-01 17:54:03 💤 Idle (40 peers), best: #5902583 (0x0527…004b), finalized #5902232 (0x5743…57c5), ⬇ 43.3kiB/s ⬆ 32.9kiB/s
2025-07-01 17:54:08 💤 Idle (40 peers), best: #5902583 (0x0527…004b), finalized #5902232 (0x5743…57c5), ⬇ 39.9kiB/s ⬆ 38.4kiB/s
2025-07-01 17:54:12 🏆 Imported #5902584 (0x0527…004b → 0x17c2…5ea7)
2025-07-01 17:54:13 💤 Idle (40 peers), best: #5902584 (0x17c2…5ea7), finalized #5902232 (0x5743…57c5), ⬇ 180.1kiB/s ⬆ 164.0kiB/s
2025-07-01 17:54:18 💤 Idle (40 peers), best: #5902584 (0x17c2…5ea7), finalized #5902232 (0x5743…57c5), ⬇ 50.2kiB/s ⬆ 48.7kiB/s
2025-07-01 17:54:23 💤 Idle (40 peers), best: #5902584 (0x17c2…5ea7), finalized #5902232 (0x5743…57c5), ⬇ 13.0kiB/s ⬆ 12.0kiB/s
2025-07-01 17:54:25 🏆 Imported #5902585 (0x17c2…5ea7 → 0x081c…ba64)

@sam0x17 sam0x17 merged commit 506cc94 into devnet-ready Jul 1, 2025
52 of 54 checks passed
@sam0x17 sam0x17 mentioned this pull request Jul 1, 2025
@l0r1s
Copy link
Collaborator

l0r1s commented Jul 1, 2025

Tested also with the docker image:

docker build --target subtensor -f Dockerfile -t debug-prod1 .

then run:

docker run -d --name prod1 debug-prod1 --chain ./chainspecs/raw_spec_finney.json --sync=warp --port 30333 --no-grandpa --no-mdns --database paritydb --db-cache 4096 --trie-cache-size 2048

Everything works correctly ✅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-cargo-audit This PR fails cargo audit but needs to be merged anyway

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants