From cbb87c490b789d16b1989137c9bac9bb6d473c6c Mon Sep 17 00:00:00 2001 From: Sam Johnson Date: Wed, 19 Jun 2024 11:49:25 -0400 Subject: [PATCH 1/6] add requirement for `devnet-companion` label for `devnet-ready` PRs --- .github/workflows/devnet-ready-labels.yml | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) create mode 100644 .github/workflows/devnet-ready-labels.yml diff --git a/.github/workflows/devnet-ready-labels.yml b/.github/workflows/devnet-ready-labels.yml new file mode 100644 index 0000000000..25d9bc4791 --- /dev/null +++ b/.github/workflows/devnet-ready-labels.yml @@ -0,0 +1,17 @@ +name: Companion Labels (devnet) +on: + pull_request: + types: [opened, labeled, unlabeled, synchronize] + branches: [devnet-ready] +jobs: + check-labels: + runs-on: ubuntu-latest + permissions: + issues: write + pull-requests: write + steps: + - uses: mheap/github-action-required-labels@v5 + with: + mode: minimum + count: 1 + labels: devnet-companion From 25ceb3b497d53e63010add603ea4e7e02b612fed Mon Sep 17 00:00:00 2001 From: Sam Johnson Date: Wed, 19 Jun 2024 11:50:58 -0400 Subject: [PATCH 2/6] add testnet-ready labels check --- .github/workflows/devnet-ready-labels.yml | 2 +- .github/workflows/testnet-ready-labels.yml | 17 +++++++++++++++++ 2 files changed, 18 insertions(+), 1 deletion(-) create mode 100644 .github/workflows/testnet-ready-labels.yml diff --git a/.github/workflows/devnet-ready-labels.yml b/.github/workflows/devnet-ready-labels.yml index 25d9bc4791..ab53327e71 100644 --- a/.github/workflows/devnet-ready-labels.yml +++ b/.github/workflows/devnet-ready-labels.yml @@ -1,4 +1,4 @@ -name: Companion Labels (devnet) +name: devnet-companion Label Check on: pull_request: types: [opened, labeled, unlabeled, synchronize] diff --git a/.github/workflows/testnet-ready-labels.yml b/.github/workflows/testnet-ready-labels.yml new file mode 100644 index 0000000000..8570d20112 --- /dev/null +++ b/.github/workflows/testnet-ready-labels.yml @@ -0,0 +1,17 @@ +name: testnet-companion Label Check +on: + pull_request: + types: [opened, labeled, unlabeled, synchronize] + branches: [testnet-ready] +jobs: + check-labels: + runs-on: ubuntu-latest + permissions: + issues: write + pull-requests: write + steps: + - uses: mheap/github-action-required-labels@v5 + with: + mode: minimum + count: 1 + labels: testnet-companion From 82cad04be874797143081bb232e25eec823563c2 Mon Sep 17 00:00:00 2001 From: Sam Johnson Date: Wed, 19 Jun 2024 11:52:15 -0400 Subject: [PATCH 3/6] update CONTRIBUTING.md with additional info --- CONTRIBUTING.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index d3616041b2..084f3e9608 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -15,9 +15,9 @@ add appropriate labels to your PR as shown below. Three positive reviews are required. 4. Once the required passing reviews have been obtained, you are ready to request that your PR be included in the next `devnet` deploy. To do this, you should open a companion PR merging - your branch into the `devnet-ready` branch. You must include a link to the parent PR in the - description and preface your PR title with "(Devnet Ready)" or the PR will be - closed/ignored. + a copy of your branch into the `devnet-ready` branch. You must include a link to the parent + PR in the description and preface your PR title with "(Devnet Ready)" or the PR will be + closed/ignored. Your companion PR should have the `devnet-companion` label. 5. A core team administrator will review your "(Devnet Ready)" PR, verifying that it logically matches the changes introduced in the parent PR (there will sometimes be minor differences due to merge conflicts) and will either request changes or approve the PR and merge it. Once From 509af18380478aacb8bf3be5ed9fcde54ca1757a Mon Sep 17 00:00:00 2001 From: Sam Johnson Date: Wed, 19 Jun 2024 11:55:14 -0400 Subject: [PATCH 4/6] update docs --- CONTRIBUTING.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 084f3e9608..132d360b86 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -86,11 +86,13 @@ | `runtime` | PR contains substantive changes to runtime / pallet code | none | | `breaking-change` | PR requires synchronized changes with bittensor | Triggers an automatic bot message so the relevant teams are made aware of the change well in advance | | `migration` | PR contains one or more migrations | none | +| `devnet-companion` | Designates a devnet companion PR | Presence of `devnet-companion` label is checked | | `devnet-ready` | PR's branch has been merged into the `devnet-ready` branch and will be included in the next `devnet` deploy | none | | `on-devnet` | PR has been deployed to `devnet` | Removes `devnet-ready` | | `devnet-pass` | PR has passed manual testing on `devnet` | `devnet-pass` or `devnet-skip` required | | `devnet-skip` | Allows a critical hotfix PR to skip required testing on `devnet` | `devnet-pass` or `devnet-skip` required | | `devnet-fail` | PR has failed manual testing on `devnet` and requires modification | none | +| `testnet-companion` | Designates a testnet companion PR | Presence of `testnet-companion` label is checked | | `on-testnet` | PR has been deployed to `testnet` | none | | `testnet-pass` | PR has passed manual testing on `testnet` | `testnet-pass` or `testnet-skip` required | | `testnet-skip` | Allows a critical hotfix PR to skip required manual testing and SOP on `testnet` | `testnet-pass` or `testnet-skip` required | From 9b221057d92d24fd4b7aafadbb88b7048d450753 Mon Sep 17 00:00:00 2001 From: Liam Date: Sun, 23 Jun 2024 14:37:20 +0300 Subject: [PATCH 5/6] override default pages to 60k --- node/src/command.rs | 50 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 49 insertions(+), 1 deletion(-) diff --git a/node/src/command.rs b/node/src/command.rs index 23674ad176..2423d1456d 100644 --- a/node/src/command.rs +++ b/node/src/command.rs @@ -17,7 +17,7 @@ use sp_runtime::traits::HashingFor; use node_subtensor_runtime::Block; use sc_cli::SubstrateCli; -use sc_service::PartialComponents; +use sc_service::{Configuration, PartialComponents}; impl SubstrateCli for Cli { fn impl_name() -> String { @@ -209,8 +209,56 @@ pub fn run() -> sc_cli::Result<()> { None => { let runner = cli.create_runner(&cli.run)?; runner.run_node_until_exit(|config| async move { + let config = override_default_heap_pages(config, 60_000); service::new_full(config).map_err(sc_cli::Error::Service) }) } } } + +/// Override default heap pages +fn override_default_heap_pages(config: Configuration, pages: u64) -> Configuration { + Configuration { + default_heap_pages: Some(pages), + impl_name: config.impl_name, + impl_version: config.impl_version, + role: config.role, + tokio_handle: config.tokio_handle, + transaction_pool: config.transaction_pool, + network: config.network, + keystore: config.keystore, + database: config.database, + trie_cache_maximum_size: config.trie_cache_maximum_size, + state_pruning: config.state_pruning, + blocks_pruning: config.blocks_pruning, + chain_spec: config.chain_spec, + wasm_method: config.wasm_method, + wasm_runtime_overrides: config.wasm_runtime_overrides, + rpc_addr: config.rpc_addr, + rpc_max_connections: config.rpc_max_connections, + rpc_cors: config.rpc_cors, + rpc_methods: config.rpc_methods, + rpc_max_request_size: config.rpc_max_request_size, + rpc_max_response_size: config.rpc_max_response_size, + rpc_id_provider: config.rpc_id_provider, + rpc_max_subs_per_conn: config.rpc_max_subs_per_conn, + rpc_port: config.rpc_port, + rpc_message_buffer_capacity: config.rpc_message_buffer_capacity, + rpc_batch_config: config.rpc_batch_config, + rpc_rate_limit: config.rpc_rate_limit, + prometheus_config: config.prometheus_config, + telemetry_endpoints: config.telemetry_endpoints, + offchain_worker: config.offchain_worker, + force_authoring: config.force_authoring, + disable_grandpa: config.disable_grandpa, + dev_key_seed: config.dev_key_seed, + tracing_targets: config.tracing_targets, + tracing_receiver: config.tracing_receiver, + max_runtime_instances: config.max_runtime_instances, + announce_block: config.announce_block, + data_path: config.data_path, + base_path: config.base_path, + informant_output_format: config.informant_output_format, + runtime_cache_size: config.runtime_cache_size, + } +} From 43a23f46a9a7d480836398af9a6fdcff628424cd Mon Sep 17 00:00:00 2001 From: Liam Date: Mon, 24 Jun 2024 13:26:54 +0300 Subject: [PATCH 6/6] update doc --- docs/running-subtensor-locally.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/docs/running-subtensor-locally.md b/docs/running-subtensor-locally.md index 4d827d8580..089f4d30a8 100644 --- a/docs/running-subtensor-locally.md +++ b/docs/running-subtensor-locally.md @@ -174,7 +174,7 @@ You can now run the public subtensor node either as a lite node or as an archive To run a lite node connected to the mainchain, execute the below command (note the `--sync=warp` flag which runs the subtensor node in lite mode): ```bash title="With --sync=warp setting, for lite node" -./target/release/node-subtensor --chain raw_spec.json --base-path /tmp/blockchain --sync=warp --execution wasm --wasm-execution compiled --port 30333 --max-runtime-instances 32 --rpc-max-response-size 2048 --rpc-cors all --rpc-port 9944 --bootnodes /ip4/13.58.175.193/tcp/30333/p2p/12D3KooWDe7g2JbNETiKypcKT1KsCEZJbTzEHCn8hpd4PHZ6pdz5 --no-mdns --in-peers 8000 --out-peers 8000 --prometheus-external --rpc-external +./target/release/node-subtensor --chain raw_spec.json --base-path /tmp/blockchain --sync=warp --port 30333 --max-runtime-instances 32 --rpc-max-response-size 2048 --rpc-cors all --rpc-port 9944 --bootnodes /ip4/13.58.175.193/tcp/30333/p2p/12D3KooWDe7g2JbNETiKypcKT1KsCEZJbTzEHCn8hpd4PHZ6pdz5 --no-mdns --in-peers 8000 --out-peers 8000 --prometheus-external --rpc-external ``` ### Archive node on mainchain @@ -182,7 +182,7 @@ To run a lite node connected to the mainchain, execute the below command (note t To run an archive node connected to the mainchain, execute the below command (note the `--sync=full` which syncs the node to the full chain and `--pruning archive` flags, which disables the node's automatic pruning of older historical data): ```bash title="With --sync=full and --pruning archive setting, for archive node" -./target/release/node-subtensor --chain raw_spec.json --base-path /tmp/blockchain --sync=full --pruning archive --execution wasm --wasm-execution compiled --port 30333 --max-runtime-instances 32 --rpc-max-response-size 2048 --rpc-cors all --rpc-port 9944 --bootnodes /ip4/13.58.175.193/tcp/30333/p2p/12D3KooWDe7g2JbNETiKypcKT1KsCEZJbTzEHCn8hpd4PHZ6pdz5 --no-mdns --in-peers 8000 --out-peers 8000 --prometheus-external --rpc-external +./target/release/node-subtensor --chain raw_spec.json --base-path /tmp/blockchain --sync=full --pruning archive --port 30333 --max-runtime-instances 32 --rpc-max-response-size 2048 --rpc-cors all --rpc-port 9944 --bootnodes /ip4/13.58.175.193/tcp/30333/p2p/12D3KooWDe7g2JbNETiKypcKT1KsCEZJbTzEHCn8hpd4PHZ6pdz5 --no-mdns --in-peers 8000 --out-peers 8000 --prometheus-external --rpc-external ``` ### Lite node on testchain @@ -190,7 +190,7 @@ To run an archive node connected to the mainchain, execute the below command (no To run a lite node connected to the testchain, execute the below command: ```bash title="With bootnodes set to testnet and --sync=warp setting, for lite node." -./target/release/node-subtensor --chain raw_testspec.json --base-path /tmp/blockchain --sync=warp --execution wasm --wasm-execution compiled --port 30333 --max-runtime-instances 32 --rpc-max-response-size 2048 --rpc-cors all --rpc-port 9944 --bootnodes /dns/bootnode.test.finney.opentensor.ai/tcp/30333/p2p/12D3KooWPM4mLcKJGtyVtkggqdG84zWrd7Rij6PGQDoijh1X86Vr --no-mdns --in-peers 8000 --out-peers 8000 --prometheus-external --rpc-external +./target/release/node-subtensor --chain raw_testspec.json --base-path /tmp/blockchain --sync=warp --port 30333 --max-runtime-instances 32 --rpc-max-response-size 2048 --rpc-cors all --rpc-port 9944 --bootnodes /dns/bootnode.test.finney.opentensor.ai/tcp/30333/p2p/12D3KooWPM4mLcKJGtyVtkggqdG84zWrd7Rij6PGQDoijh1X86Vr --no-mdns --in-peers 8000 --out-peers 8000 --prometheus-external --rpc-external ``` ### Archive node on testchain @@ -198,8 +198,9 @@ To run a lite node connected to the testchain, execute the below command: To run an archive node connected to the testchain, execute the below command: ```bash title="With bootnodes set to testnet and --sync=full and --pruning archive setting, for archive node" -./target/release/node-subtensor --chain raw_testspec.json --base-path /tmp/blockchain --sync=full --pruning archive --execution wasm --wasm-execution compiled --port 30333 --max-runtime-instances 32 --rpc-max-response-size 2048 --rpc-cors all --rpc-port 9944 --bootnodes /dns/bootnode.test.finney.opentensor.ai/tcp/30333/p2p/12D3KooWPM4mLcKJGtyVtkggqdG84zWrd7Rij6PGQDoijh1X86Vr --no-mdns --in-peers 8000 --out-peers 8000 --prometheus-external --rpc-external +./target/release/node-subtensor --chain raw_testspec.json --base-path /tmp/blockchain --sync=full --pruning archive --port 30333 --max-runtime-instances 32 --rpc-max-response-size 2048 --rpc-cors all --rpc-port 9944 --bootnodes /dns/bootnode.test.finney.opentensor.ai/tcp/30333/p2p/12D3KooWPM4mLcKJGtyVtkggqdG84zWrd7Rij6PGQDoijh1X86Vr --no-mdns --in-peers 8000 --out-peers 8000 --prometheus-external --rpc-external ``` ## Running on cloud + We have not tested these installation scripts on any cloud service. In addition, if you are using Runpod cloud service, then note that this service is already [containerized](https://docs.runpod.io/pods/overview). Hence, the only option available to you is to compile from the source, as described in the above [Method 2: Using Source Code](#method-2-using-source-code) section. Note that these scripts have not been tested on Runpod.