This repository was archived by the owner on Nov 15, 2023. It is now read-only.
state-db: Print warning when using large pruning window on RocksDb#13414
Merged
paritytech-processbot[bot] merged 2 commits intomasterfrom Feb 18, 2023
Merged
Conversation
This pr changes state-db to print a warning when using a large pruning window and running with a database that isn't supporting ref-counting like RocksDb. This makes the user aware of potential out of memory errors because this option together with RocksDb etc puts the entire pruning window into memory. Besides that the pr introduces `LOG_TARGET` for having the target declared central!
bkchr
added
A0-please_review
koute
approved these changes
Feb 18, 2023
client/state-db/src/pruning.rs
Outdated
Comment on lines
+325
to
+332
| log::warn!( | ||
| target: LOG_TARGET, | ||
| "Detected large pruning window of {window_size} blocks while running with a database \n | ||
| that isn't supporting ref counting, e.g. RocksDb. \n\ | ||
| This means that all the blocks in the pruning window are stored in memory which \n\ | ||
| may leads to a high memory usage and to possible out of memory crashes. \n\ | ||
| Use for example ParityDb to not store the entire pruning window in memory.\ | ||
| "); |
Contributor
There was a problem hiding this comment.
Hmm.... as a user visible comment this is perhaps a little to detailed? (: I don't know. Maybe making this a code comment and using something terser and more shouty for the message would be better? Something like:
Large pruning window of {window_size} detected! THIS CAN LEAD TO HIGH MEMORY USAGE AND CRASHES. Reduce the pruning window or switch your database to paritydb.
But I don't really care either way, so it's up to you.
Member
Author
There was a problem hiding this comment.
Yours sound better :P Mine was just me trying to bring over the same point, but way more complicated :P
arkpar
approved these changes
Feb 18, 2023
Member
Author
|
bot merge |
|
Waiting for commit status. |
paritytech-processbot
bot
deleted the
bkchr-state-db-show-warning-on-huge-pruning-window
branch
rossbulat
pushed a commit
that referenced
this pull request
Feb 19, 2023
…13414) * state-db: Print warning when using large pruning window on RocksDb This pr changes state-db to print a warning when using a large pruning window and running with a database that isn't supporting ref-counting like RocksDb. This makes the user aware of potential out of memory errors because this option together with RocksDb etc puts the entire pruning window into memory. Besides that the pr introduces `LOG_TARGET` for having the target declared central! * Review comments
ltfschoen
pushed a commit
to ltfschoen/substrate
that referenced
this pull request
Feb 22, 2023
…aritytech#13414) * state-db: Print warning when using large pruning window on RocksDb This pr changes state-db to print a warning when using a large pruning window and running with a database that isn't supporting ref-counting like RocksDb. This makes the user aware of potential out of memory errors because this option together with RocksDb etc puts the entire pruning window into memory. Besides that the pr introduces `LOG_TARGET` for having the target declared central! * Review comments
ark0f
pushed a commit
to gear-tech/substrate
that referenced
this pull request
Feb 27, 2023
…aritytech#13414) * state-db: Print warning when using large pruning window on RocksDb This pr changes state-db to print a warning when using a large pruning window and running with a database that isn't supporting ref-counting like RocksDb. This makes the user aware of potential out of memory errors because this option together with RocksDb etc puts the entire pruning window into memory. Besides that the pr introduces `LOG_TARGET` for having the target declared central! * Review comments
Ank4n
pushed a commit
that referenced
this pull request
Feb 28, 2023
…13414) * state-db: Print warning when using large pruning window on RocksDb This pr changes state-db to print a warning when using a large pruning window and running with a database that isn't supporting ref-counting like RocksDb. This makes the user aware of potential out of memory errors because this option together with RocksDb etc puts the entire pruning window into memory. Besides that the pr introduces `LOG_TARGET` for having the target declared central! * Review comments
ukint-vs
pushed a commit
to gear-tech/substrate
that referenced
this pull request
Apr 10, 2023
…aritytech#13414) * state-db: Print warning when using large pruning window on RocksDb This pr changes state-db to print a warning when using a large pruning window and running with a database that isn't supporting ref-counting like RocksDb. This makes the user aware of potential out of memory errors because this option together with RocksDb etc puts the entire pruning window into memory. Besides that the pr introduces `LOG_TARGET` for having the target declared central! * Review comments
15 tasks
nathanwhit
pushed a commit
to nathanwhit/substrate
that referenced
this pull request
Jul 19, 2023
…aritytech#13414) * state-db: Print warning when using large pruning window on RocksDb This pr changes state-db to print a warning when using a large pruning window and running with a database that isn't supporting ref-counting like RocksDb. This makes the user aware of potential out of memory errors because this option together with RocksDb etc puts the entire pruning window into memory. Besides that the pr introduces `LOG_TARGET` for having the target declared central! * Review comments
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pr changes state-db to print a warning when using a large pruning window and running with a database that isn't supporting ref-counting like RocksDb. This makes the user aware of potential out of memory errors because this option together with RocksDb etc puts the entire pruning window into memory. Besides that the pr introduces
LOG_TARGETfor having the target declared central!