memory issue when syncing

[81jpayne]

While syncing HSD, it would get killed by my OS (Ubuntu 20.04.1) about every 5000 blocks for some memory reason. I had to restart HSD (with --plugins=hnscan as instructed) about ten times to complete the sync. It's been stable since fully syncing.

The above didn't occur when I was running HSD without the plugin.

[chrxn]

same issue, though it is killed for exceeding js-heap stack memory limit

<--- Last few GCs --->

[34566:0x110000000]   173649 ms: Mark-sweep 2025.8 (2062.5) -> 2022.7 (2046.7) MB, 17.4 / 0.0 ms  (+ 739.6 ms in 784 steps since start of marking, biggest step 5.5 ms, walltime since start of marking 784 ms) (average mu = 0.372, current mu = 0.463) finali[34566:0x110000000]   173696 ms: Scavenge 2028.9 (2046.7) -> 2017.8 (2066.5) MB, 12.2 / 0.0 ms  (average mu = 0.372, current mu = 0.463) allocation failure


<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x100093bd5 node::Abort() [/Users/chrxn/.nvm/versions/node/v13.9.0/bin/node]
 2: 0x100093d9b void node::FPrintF<char const*&, char const*&>(__sFILE*, char const*, char const*&, char const*&) [/Users/chrxn/.nvm/versions/node/v13.9.0/bin/node]
 3: 0x1001d40f7 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/Users/chrxn/.nvm/versions/node/v13.9.0/bin/node]
 4: 0x1001d4093 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/chrxn/.nvm/versions/node/v13.9.0/bin/node]
 5: 0x100356b65 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/Users/chrxn/.nvm/versions/node/v13.9.0/bin/node]
 6: 0x100358244 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [/Users/chrxn/.nvm/versions/node/v13.9.0/bin/node]
 7: 0x100354fde v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/Users/chrxn/.nvm/versions/node/v13.9.0/bin/node]
 8: 0x100352fc6 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/chrxn/.nvm/versions/node/v13.9.0/bin/node]
 9: 0x10035b2d9 v8::internal::Heap::FinalizeIncrementalMarkingIfComplete(v8::internal::GarbageCollectionReason) [/Users/chrxn/.nvm/versions/node/v13.9.0/bin/node]
10: 0x10036972f v8::internal::IncrementalMarkingJob::Task::RunInternal() [/Users/chrxn/.nvm/versions/node/v13.9.0/bin/node]
11: 0x1000fa769 node::PerIsolatePlatformData::RunForegroundTask(std::__1::unique_ptr<v8::Task, std::__1::default_delete<v8::Task> >) [/Users/chrxn/.nvm/versions/node/v13.9.0/bin/node]
12: 0x1000f9692 node::PerIsolatePlatformData::FlushForegroundTasksInternal() [/Users/chrxn/.nvm/versions/node/v13.9.0/bin/node]
13: 0x1009161c4 uv__async_io [/Users/chrxn/.nvm/versions/node/v13.9.0/bin/node]
14: 0x100929bbd uv__io_poll [/Users/chrxn/.nvm/versions/node/v13.9.0/bin/node]
15: 0x100916758 uv_run [/Users/chrxn/.nvm/versions/node/v13.9.0/bin/node]
16: 0x1000d5f99 node::NodeMainInstance::Run() [/Users/chrxn/.nvm/versions/node/v13.9.0/bin/node]
17: 0x10006bb25 node::Start(int, char**) [/Users/chrxn/.nvm/versions/node/v13.9.0/bin/node]
18: 0x7fff657fa7fd start [/usr/lib/system/libdyld.dylib]
fish: 'hsd --plugins=hnscan' terminated by signal SIGABRT (Abort)

I haven't confirmed stability yet though as I am still going through the stop-and-start, though it restarts the process at the same block height it left off on, so I assume that all will be fine eventually.

[81jpayne]

@chrxn, are you running hsd in a tmux session by any chance? I was (still am) and suspect this might have been the cause.

[chrxn]

@81jpayne I was not, but my fish is my shell.