Skip to content

use CidHashMap in the plain CAR-file blockstore #3286

@lemmih

Description

@lemmih

Issue summary

The blockstore backed by an uncompressed CAR file uses two HashMaps with CIDs as keys: One map for the write cache and one map for the index. These maps should be replaced with CidHashMap, which uses less memory. Memory usage when validating a snapshot should be measured before and after the change.

Acceptance criteria:

  • Add missing methods to CidHashMap
  • Replace HashMap<Cid, V> with CidHashMap<V> in car/plain.rs
  • Measure change in peak memory use when validating an uncompressed calibnet and mainnet snapshot. See /usr/bin/time -v forest-cli snapshot validate {snapshot}.car

Other information and links

forest/src/db/car/plain.rs

Lines 152 to 153 in 6dbd776

write_cache: ahash::HashMap<Cid, Vec<u8>>,
index: ahash::HashMap<Cid, UncompressedBlockDataLocation>,

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions