Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 78 additions & 0 deletions docs/node/run-your-node/advanced/pruning.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
# Pruning State

To reduce hardware requirements and improve the overall latency of the node,
operators are encouraged to prune their state (unless also serving historical
state).

As will be explained in the following sections, pruning should be configured from
the start, i.e. late changes to the pruning configuration may not be optimal for
the node's health.

## Consensus Pruning

To configure pruning of the consensus data amend your node's configuration with:

```yaml
# ... sections not relevant are omitted ...
consensus:
# ... sections not relevant are omitted ...
prune:
strategy: "keep_n"
num_kept: <n>
```

where `<n>` is the number of blocks and state versions that will be kept.

Comment thread
martintomazic marked this conversation as resolved.
### Suggested Pruning Configuration

For normal node operation, the minimum requirements is `n=250_000`. However,
it is recommended to set it to `n=400_000`. Assuming blocks are produced every 6
seconds, the latter corresponds to preserving 1 month of the consensus data.

Operators that are not resource-restrained and want to contribute to the overall
network health are encouraged to set `n=5_000_000` (approximately 1 year of data).

### Late Pruning

Due to the [LSM Tree][] design of the underlying databases, enabling pruning after
your node has been running for a while, or changing configuration to retain less data,
may not work as expected.

To change pruning configuration operators must follow this steps:

1. Gracefuly shutdown the node (validators should take [special care][graceful shutdown]).
Comment thread
martintomazic marked this conversation as resolved.
2. Configure pruning as described in the [Consensus Pruning](#consensus-pruning) section above.
3. Run offline [pruning command].
4. Run the [compaction command].
5. Start the node again.

[LSM tree]: https://en.wikipedia.org/wiki/Log-structured_merge-tree
[graceful shutdown]: ../maintenance/shutting-down-a-node.md
[pruning command]: https://github.com/oasisprotocol/oasis-core/blob/master/docs/oasis-node/cli.md#prune-experimental
[compaction command]: https://github.com/oasisprotocol/oasis-core/blob/master/docs/oasis-node/cli.md#compact-experimental

## ParaTime Pruning

Comment thread
martintomazic marked this conversation as resolved.
To configure pruning of the ParaTime data amend your node's configuration with:

```yaml
# ... sections not relevant are omitted ...
runtime:
# ... sections not relevant are omitted ...
prune:
strategy: "keep_last"
num_kept: <n>
```

where `<n>` is the number of ParaTime's state versions that will be kept.

### Suggested Pruning Configuration

ParaTime state is much larger than the consensus state. As a consequence, as the
number of versions in the state database grows, pruning becomes slower and slower.
For this reason **ParaTime state pruning must be configured
from the start or not configured at all**.

To configure ParaTime pruning it is recommended to set `n=250_000`. The maximum value is
`n=400_000`. If you need to preserve more data (e.g. nodes serving historical
state) you will have to keep the entire ParaTime's state from the genesis.
18 changes: 7 additions & 11 deletions docs/node/run-your-node/prerequisites/hardware-recommendations.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,18 +152,12 @@ running a separate archive node.

:::info

You can configure your node _not to_ keep a complete state from the genesis
onward. This will reduce the amount of storage required for the consensus and
ParaTime state.
Unless running a node to preserve historical state you are encouraged _not to_
keep a complete state from the genesis onward. This will significantly reduce
the amount of storage required for the consensus and ParaTime state and improve
the overall latency of the storage related operations.

To enable pruning of the consensus state set the
`consensus.prune.strategy` and
`consensus.prune.num_kept` parameters appropriately in your
[node's configuration].

To enable pruning of the ParaTime state set the
`runtime.prune.strategy` and `runtime.prune.num_kept`
parameters appropriately in your [node's configuration].
To enable it, see relevant [consensus pruning] and [ParaTime pruning] sections.

:::

Expand All @@ -174,6 +168,8 @@ parameters appropriately in your [node's configuration].
[Emerald Mainnet launch]:
https://medium.com/oasis-protocol-project/oasis-emerald-evm-paratime-is-live-on-mainnet-13afe953a4c9
[node's configuration]: ../validator-node.mdx#configuration
[consensus pruning]: ../advanced/pruning.md#consensus-pruning
[ParaTime pruning]: ../advanced/pruning.md#paratime-pruning


### Network
Expand Down
3 changes: 2 additions & 1 deletion sidebarNode.ts
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,7 @@ export const sidebarNode: SidebarsConfig = {
},
items: [
'node/run-your-node/advanced/sync-node-using-state-sync',
'node/run-your-node/advanced/pruning',
Comment thread
martintomazic marked this conversation as resolved.
'node/run-your-node/advanced/copy-state-from-one-node-to-the-other',
'node/run-your-node/advanced/remote-signer',
],
Expand Down Expand Up @@ -165,7 +166,7 @@ export const sidebarNode: SidebarsConfig = {
label: 'Mainnet',
},
{
type: 'doc',
type: 'doc',
id: 'node/reference/upgrade-logs/testnet',
label: 'Testnet',
},
Expand Down