Faster sync by collecting esplora ureq thread handles#579
Faster sync by collecting esplora ureq thread handles#579notmandatory merged 1 commit intobitcoindevkit:masterfrom
Conversation
|
Now that you're fixing the parallel execution should we worry about running out of some resource if a wallet has lots of scripts and transactions to sync? seems like this could swamp a low power esplora server or at least exceed some point of diminishing returns for spawning new threads. You don't need to address it in this PR but but do you have any thoughts for how to address this issue? maybe some sort of thread pool? |
Note that the number of requests is limited. It does them in batches according to |
Signed-off-by: nickfarrow <nick@nickfarrow.com>
Description
Speeds up esplora ureq syncing. Taken from #560
The current sync just creates a map of scripts to joinhandles which doesn't yet spawn the sync threads due to lazy evaluation. In the following
handles.map(), the thread handles are sequentially evaluated and joined. With the fix, the handles are collected so that the threads spawn in parallel, and then joinedNotes to the reviewers
I had to add a
#[allow(clippy::needless_collect)]so that it wouldn't complain about collecting and then iterating. (Perhaps clippy is partially responsible for this issue!)Tested sync performance by doing a fresh sync on an existing gun wallet.
Checklists
All Submissions:
cargo fmtandcargo clippybefore committingBugfixes: