Skip to content

init: Remove auto-import of bootstrap.dat and associated code #17044

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
laanwj merged 1 commit into from
Nov 5, 2019
Merged

init: Remove auto-import of bootstrap.dat and associated code #17044

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion doc/developer-notes.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -385,7 +385,7 @@ Threads

- ThreadScriptCheck : Verifies block scripts.

- ThreadImport : Loads blocks from blk*.dat files or bootstrap.dat.
- ThreadImport : Loads blocks from `blk*.dat` files or `-loadblock=<file>`.

- StartNode : Starts other threads.

Expand Down
4 changes: 4 additions & 0 deletions doc/release-notes-15954.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Configuration option changes
-----------------------------

Importing blocks upon startup via the `bootstrap.dat` file no longer occurs by default. The file must now be specified with `-loadblock=<file>`.
16 changes: 1 addition & 15 deletions src/init.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -374,7 +374,7 @@ void SetupServerArgs()
gArgs.AddArg("-debuglogfile=<file>", strprintf("Specify location of debug log file. Relative paths will be prefixed by a net-specific datadir location. (-nodebuglogfile to disable; default: %s)", DEFAULT_DEBUGLOGFILE), ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
gArgs.AddArg("-feefilter", strprintf("Tell other nodes to filter invs to us by our mempool min fee (default: %u)", DEFAULT_FEEFILTER), ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::OPTIONS);
gArgs.AddArg("-includeconf=<file>", "Specify additional configuration file, relative to the -datadir path (only useable from configuration file, not command line)", ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
gArgs.AddArg("-loadblock=<file>", "Imports blocks from external blk000??.dat file on startup", ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
gArgs.AddArg("-loadblock=<file>", "Imports blocks from external file on startup", ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
Copy link
Copy Markdown
Member

@maflcko maflcko Oct 23, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this removed? I'd rather add a test for importing blk000.dat files.

Copy link
Copy Markdown
Contributor

@mzumsande mzumsande Oct 31, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MarcoFalke: I suggested removing that above, because it seemed to imply a non-existing naming convention for the file (might as well load from bootstrap.dat or whatever your file is named).

Copy link
Copy Markdown
Member

@maflcko maflcko Nov 1, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See for example this test:

diff --git a/test/functional/feature_loadblock.py b/test/functional/feature_loadblock.py
index bf2a4ff61f..a532a83946 100755
--- a/test/functional/feature_loadblock.py
+++ b/test/functional/feature_loadblock.py
@@ -25,7 +25,7 @@ from test_framework.util import assert_equal, wait_until
 class LoadblockTest(BitcoinTestFramework):
     def set_test_params(self):
         self.setup_clean_chain = True
-        self.num_nodes = 2
+        self.num_nodes = 3
 
     def run_test(self):
         self.nodes[1].setnetworkactive(state=False)
@@ -71,13 +71,16 @@ class LoadblockTest(BitcoinTestFramework):
         subprocess.run([sys.executable, linearize_data_file, cfg_file],
                        check=True)
 
-        self.log.info("Restart second, unsynced node with bootstrap file")
+        self.log.info("Restart unsynced nodes with bootstrap file or raw block file")
         self.stop_node(1)
         self.start_node(1, ["-loadblock=" + bootstrap_file])
-        wait_until(lambda: self.nodes[1].getblockcount() == 100)
+        self.stop_node(2)
+        self.start_node(2, ["-loadblock=" + os.path.join(blocks_dir, 'blk00000.dat')])
+        for i in [1, 2]:
+            wait_until(lambda: self.nodes[i].getblockcount() == 100)
 
-        assert_equal(self.nodes[1].getblockchaininfo()['blocks'], 100)
-        assert_equal(self.nodes[0].getbestblockhash(), self.nodes[1].getbestblockhash())
+            assert_equal(self.nodes[i].getblockchaininfo()['blocks'], 100)
+            assert_equal(self.nodes[0].getbestblockhash(), self.nodes[i].getbestblockhash())
 
 
 if __name__ == '__main__':

Copy link
Copy Markdown
Member

@laanwj laanwj Nov 4, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with removing this. blkXXXXX.dat are the block files and the way to (re)index them is -reindex, it has nothing to do with -loadblock.

Copy link
Copy Markdown
Member

@maflcko maflcko Nov 4, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't the point that you can import the block.dat files from another datadir with -loadblock=blk001.dat -loadblock=blk002.dat -loadblock=...

Copy link
Copy Markdown
Member

@laanwj laanwj Nov 4, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that's the point. In that case it's less hassle to copy over all the blk files and do a reindex.

I think the idea behind -loadblock is to import the output from linearize; huge consecutive 'block archives' (there used to be torrents for this). But I might be wrong. I'm not sure what it's used for nowadays.

Copy link
Copy Markdown
Member

@maflcko maflcko Nov 4, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, so it is multi-arg to support a split-up bootstrap.dat?

Copy link
Copy Markdown
Contributor Author

@fjahr fjahr Nov 5, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that was my understanding as well. But I did see it spelled out anywhere explicitly.

Copy link
Copy Markdown
Member

@laanwj laanwj Nov 5, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, so it is multi-arg to support a split-up bootstrap.dat?

I think so; linearize splits it into 1GB files by default.

gArgs.AddArg("-maxmempool=<n>", strprintf("Keep the transaction memory pool below <n> megabytes (default: %u)", DEFAULT_MAX_MEMPOOL_SIZE), ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
gArgs.AddArg("-maxorphantx=<n>", strprintf("Keep at most <n> unconnectable transactions in memory (default: %u)", DEFAULT_MAX_ORPHAN_TRANSACTIONS), ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
gArgs.AddArg("-mempoolexpiry=<n>", strprintf("Do not keep transactions in the mempool longer than <n> hours (default: %u)", DEFAULT_MEMPOOL_EXPIRY), ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
Expand Down Expand Up @@ -687,20 +687,6 @@ static void ThreadImport(std::vector<fs::path> vImportFiles)
LoadGenesisBlock(chainparams);
}

// hardcoded $DATADIR/bootstrap.dat
fs::path pathBootstrap = GetDataDir() / "bootstrap.dat";
if (fs::exists(pathBootstrap)) {
FILE *file = fsbridge::fopen(pathBootstrap, "rb");
if (file) {
fs::path pathBootstrapOld = GetDataDir() / "bootstrap.dat.old";
LogPrintf("Importing bootstrap.dat...\n");
LoadExternalBlockFile(chainparams, file);
RenameOver(pathBootstrap, pathBootstrapOld);
} else {
LogPrintf("Warning: Could not open bootstrap file %s\n", pathBootstrap.string());
}
}

// -loadblock=
for (const fs::path& path : vImportFiles) {
FILE *file = fsbridge::fopen(path, "rb");
Expand Down