Skip to content

Commit 3364f08

Browse files
feat(merk): node_hash_with_sum + proof Node variants for ProvableSumTree
Phase 2 of the ProvableSumTree feature — bakes the per-node sum into the node hash so it becomes cryptographically committed via the parent's hash chain, parallel to how `node_hash_with_count` commits the count for `ProvableCountTree`. After this commit a `ProvableSumTree` with the same {key/value/sum} contents as a plain `SumTree` produces a different root hash, which is the whole point of the Phase 2 divergence. Phase 1 (commit c95cf74) was types-only; aggregation, storage, and hashing all used the SumTree code paths. Phase 2 introduces the new hash function, the new proof-node variants needed to transport sums through proofs, and the dispatch wiring on both prover and verifier sides. Phases 3 (insert/read), 4 (verify_grovedb walk), and 5 (AggregateSumOnRange) remain. HASH DISPATCH - `merk::tree::hash::node_hash_with_sum(kv, l, r, i64)` mirrors `node_hash_with_count` byte-for-byte except the appended 8-byte field is `i64::to_be_bytes()`. Negative sums hash via their two's-complement BE form, which is platform-independent. - New `AggregateData::ProvableSum(i64)` variant. The `From<TreeFeatureType>` conversion now maps `ProvableSummedMerkNode(v) -> ProvableSum(v)` (was `Sum(v)` in Phase 1) so `Tree::hash_for_link` and the commit path can dispatch through the new arm. - `Tree::hash_for_link(TreeType::ProvableSumTree)` and both commit paths (left/right Link::Modified arms) now call `node_hash_with_sum` when the aggregate is `ProvableSum`. `Tree::aggregate_data` for `ProvableSummedMerkNode` yields `ProvableSum` instead of `Sum`. - Helper updates: `child_aggregate_sum_data_as_i64` / `child_aggregate_sum_data_as_i128` treat `ProvableSum` identically to `Sum`; `child_aggregate_count_data_as_u64` returns 0. `child_ref_and_sum_size` covers the new variant. - `Link::encode_into` / `decode_into` learn tag byte 7 for `AggregateData::ProvableSum` (parallel to the existing `ProvableSummedMerkNode` tag byte 7 in `TreeFeatureType`). - `grovedb::batch` `InsertTreeWithRootHash` now reconstructs an `Element::ProvableSumTree` when seeing `AggregateData::ProvableSum`. PROOF NODE VARIANTS Five new `Node` enum variants in `grovedb-query/src/proofs/mod.rs`, mirroring the Count family member-for-member but with `i64` sums: - `KVSum(key, value, sum)` — sum analogue of `KVCount` - `KVHashSum(kv_hash, sum)` — analogue of `KVHashCount` - `KVRefValueHashSum(key, ref_value, ref_elem_hash, sum)` - `KVDigestSum(key, value_hash, sum)` — analogue of `KVDigestCount` - `HashWithSum(kv_hash, l, r, sum)` — analogue of `HashWithCount` `merk::proofs::tree::Tree::hash()` now dispatches each new variant through `node_hash_with_sum`. `KVValueHashFeatureType` / `...WithChildHash` handling gains a `ProvableSummedMerkNode` arm so proof-tree hashes recomputed from a Sum-bearing feature_type match the Merk-tree side. `aggregate_data()` returns `ProvableSum(sum)` for `KVSum` and `HashWithSum`; `key()` lists the three key-bearing new variants alongside their Count counterparts. `grovedb-element::ProofNodeType` gains `KvSum` and `KvRefValueHashSum`; `ElementType::proof_node_type` now picks them when the parent is `ProvableSumTree` (Phase 1 routed Sum-tree children through the Count dispatch). Subtrees inside ProvableSum still use `KvValueHashFeatureType` since the feature_type carries the sum. Proof generation in `merk/src/proofs/query/mod.rs` adds `to_kv_sum_node`, `to_kvhash_sum_node`, `to_kvdigest_sum_node` (parallel to the Count helpers) and an `is_provable_sum_tree` branch that emits Sum-bearing variants. `chunks.rs`'s `create_proof_node_for_chunk` dispatches the new ProofNodeType arms. GroveDB-side reference post-processing in `grovedb/src/operations/proof/generate.rs` rewrites the merk-level `KVValueHashFeatureType(_, _, _, ProvableSummedMerkNode(sum))` to `KVRefValueHashSum`, mirroring the existing `KVValueHashFeatureType -> KVRefValueHashCount` path. Both ref-rewriting loops in that file are updated. The regular query verifier in `merk/src/proofs/query/verify.rs` rejects `HashWithSum` at non-aggregate positions (fail-fast, matching the existing `HashWithCount` guard). `KVSum`, `KVDigestSum`, and `KVRefValueHashSum` are dispatched via `execute_node`. `KVHashSum` joins `KVHash` / `KVHashCount` in the "non-data-bearing on path" branch and in the absence-proof boundary set. WIRE FORMAT Tag bytes 0x30..=0x3D in the previously-unused 0x30..0x3F range: Push variants (V0 short + V1 wrapper for KV-style large values): 0x30 = KVSum (small), 0x31 = KVSum (large) 0x32 = KVHashSum 0x33 = KVRefValueHashSum (small), 0x34 = KVRefValueHashSum (large) 0x35 = KVDigestSum 0x36 = HashWithSum PushInverted parallel: 0x37 = KVSum (small), 0x38 = KVSum (large) 0x39 = KVHashSum 0x3a = KVRefValueHashSum (small), 0x3b = KVRefValueHashSum (large) 0x3c = KVDigestSum 0x3d = HashWithSum 0x3e and 0x3f are intentionally reserved. The on-wire i64 sum uses varint (via `ed::Encode for i64`) for compactness, matching the Count family. The hash recomputation in `node_hash_with_sum` uses the fixed 8-byte big-endian form independently — wire encoding and hash input are deliberately decoupled. `encoding_length()` and `Decode` arms parallel the Count family verbatim. V0 wire format is unchanged. All new tags are V1-only. TESTS - `merk::tree::hash` (4): `node_hash_with_sum` differs from `node_hash` even at sum=0; different sums give different hashes; `i64::MIN` / `i64::MAX` are distinct; determinism. - `merk::tree` (2): a `ProvableSummedMerkNode` tree aggregates to `ProvableSum`, `hash_for_link(ProvableSumTree)` matches `node_hash_with_sum(...)` and diverges from plain `Tree::hash()`; mutating a node sum changes the root hash. - `merk::proofs::tree` (4): forged sums on `HashWithSum`, `KVSum`, `KVHashSum` change the recomputed node hash; Phase 1 -> Phase 2 cornerstone — same {key/value/sum} contents give a different ProvableSumTree hash than a plain SumTree. - `grovedb-query::proofs::encoding` (4): round-trip every new variant through `Op::Push` and `Op::PushInverted` at sum values {`i64::MIN`, -42, -1, 0, 1, 42, `i64::MAX`}; tag-byte sanity check for all 10 new tags. - `merk::tree::tree_feature_type`: extended every existing `AggregateData` test to cover the new `ProvableSum` variant. Workspace `cargo test --all-features` green: 2881 tests passing, zero failures. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent c95cf74 commit 3364f08

19 files changed

Lines changed: 1415 additions & 72 deletions

File tree

grovedb-element/src/element_type.rs

Lines changed: 40 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -122,6 +122,21 @@ pub enum ProofNodeType {
122122
///
123123
/// Used for: Reference (inside ProvableCountTree)
124124
KvRefValueHashCount,
125+
126+
/// Use `Node::KVSum` - sum analogue of `KvCount`. The verifier
127+
/// recomputes `value_hash = H(value)` and includes the i64 sum in the
128+
/// node hash via `node_hash_with_sum`. Phase 2.
129+
///
130+
/// Used for: Item, SumItem, ItemWithSumItem (inside ProvableSumTree)
131+
KvSum,
132+
133+
/// Use `Node::KVRefValueHashSum` - sum analogue of `KvRefValueHashCount`.
134+
/// At the merk layer, this generates `KVValueHashFeatureType` (since
135+
/// merk doesn't know about references). GroveDB post-processes these
136+
/// nodes to `Node::KVRefValueHashSum` with the dereferenced value.
137+
///
138+
/// Used for: Reference (inside ProvableSumTree)
139+
KvRefValueHashSum,
125140
}
126141

127142
/// Element type discriminants.
@@ -405,42 +420,53 @@ impl ElementType {
405420
pub fn proof_node_type(&self, parent_tree_type: Option<ElementType>) -> ProofNodeType {
406421
let parent_base = parent_tree_type.map(|t| t.base());
407422
// "Provable aggregate parents" are those that bake the per-node
408-
// aggregate (count or sum) into the node hash. Items inside them
409-
// must carry the feature in the proof, and subtrees inside them
410-
// must use the feature-aware proof-node variant.
423+
// aggregate into the node hash. The count family
424+
// (`ProvableCountTree`, `ProvableCountSumTree`) hashes the count;
425+
// the sum family (`ProvableSumTree`, Phase 2) hashes the sum.
411426
//
412-
// Phase 1: `ProvableSumTree` joins this family. It mirrors
413-
// `ProvableCountTree`'s proof shape for now; Phase 2 will diverge
414-
// the actual hash computation but keep the proof-node-type
415-
// selection identical.
427+
// Phase 2: the dispatch now distinguishes the two families. Item /
428+
// Reference proof variants diverge (KvSum / KvRefValueHashSum vs
429+
// KvCount / KvRefValueHashCount). Subtrees inside either family
430+
// still use `KvValueHashFeatureType` — the feature_type field on
431+
// that variant carries both the count and sum in their respective
432+
// tagged TreeFeatureType variants, so a single proof-node variant
433+
// suffices for the subtree case.
416434
let is_provable_count_tree = matches!(
417435
parent_base,
418-
Some(ElementType::ProvableCountTree)
419-
| Some(ElementType::ProvableCountSumTree)
420-
| Some(ElementType::ProvableSumTree)
436+
Some(ElementType::ProvableCountTree) | Some(ElementType::ProvableCountSumTree)
421437
);
438+
let is_provable_sum_tree = matches!(parent_base, Some(ElementType::ProvableSumTree));
439+
let is_provable_aggregate_tree = is_provable_count_tree || is_provable_sum_tree;
422440

423441
let base = self.base();
424442
if base.has_simple_value_hash() {
425443
// Items (Item, SumItem, ItemWithSumItem)
426444
if is_provable_count_tree {
427445
ProofNodeType::KvCount
446+
} else if is_provable_sum_tree {
447+
ProofNodeType::KvSum
428448
} else {
429449
ProofNodeType::Kv
430450
}
431451
} else if base.is_reference() {
432452
// References need combined hash (for reference resolution).
433-
// In ProvableCountTree, they also need the count in node_hash.
434-
// GroveDB post-processes these to KVRefValueHash/KVRefValueHashCount.
453+
// In ProvableCountTree they additionally need the count in
454+
// node_hash; in ProvableSumTree they need the sum.
455+
// GroveDB post-processes these to KVRefValueHash /
456+
// KVRefValueHashCount / KVRefValueHashSum.
435457
if is_provable_count_tree {
436458
ProofNodeType::KvRefValueHashCount
459+
} else if is_provable_sum_tree {
460+
ProofNodeType::KvRefValueHashSum
437461
} else {
438462
ProofNodeType::KvRefValueHash
439463
}
440464
} else {
441465
// Subtrees (Tree, SumTree, BigSumTree, CountTree, CountSumTree,
442-
// ProvableCountTree, ProvableSumTree)
443-
if is_provable_count_tree {
466+
// ProvableCountTree, ProvableSumTree). KvValueHashFeatureType
467+
// works for both Count and Sum families because the embedded
468+
// `TreeFeatureType` carries the aggregate.
469+
if is_provable_aggregate_tree {
444470
ProofNodeType::KvValueHashFeatureType
445471
} else {
446472
ProofNodeType::KvValueHash

0 commit comments

Comments
 (0)