This repository was archived by the owner on Feb 12, 2025. It is now read-only.
This repository was archived by the owner on Feb 12, 2025. It is now read-only.
Memory usage is at its highest during cleanup #157
Description
The sqlite approach should be probably adapted to it too - keep track of the files that are in use there, instead of keeping (millions of) Python strings in memory. FYI @joelynch
#161 fixes this somewhat (keep only manifests that are actually active, not historic ones, in memory) but we should really split the manifest to per-node sub-manifests with file references.