feat(grovedb,query): allow AggregateCountOnRange as carrier subquery

[QuantumExplorer]

Summary

• Allow AggregateCountOnRange (ACOR) as a subquery item under an outer Keys/Range* carrier query — the natural multi-outer-key extension of the existing single-leg ACOR. Returns one (outer_key, u64) pair per matched outer key while leaving the leaf wire format byte-identical.
• Split Query::validate_aggregate_count_on_range into a leaf validator (existing semantics, renamed) and a new carrier validator, with the top-level dispatcher routing by query shape and returning the leaf inner range either way.
• Add GroveDb::verify_aggregate_count_query_per_key(...) -> Result<(RootHash, Vec<(Vec<u8>, u64)>), Error> that accepts both leaf and carrier shapes (leaf collapses to a single-entry Vec with an empty key). The legacy verify_aggregate_count_query now strictly validates the leaf shape.

Why

Dash Platform's count-query module needs to answer

SELECT brand, COUNT(*) FROM widgets
WHERE brand IN ('brand_000', 'brand_001') AND color > 'color_00000500'
GROUP BY brand

against a byBrand / <brand> / color / <ProvableCountTree> layout. Today grovedb rejects the nested shape at Query::validate_aggregate_count_on_range, forcing callers to fan out one proof per outer key (k× larger proofs) or fall back to the unproven path. With this change the same GroveDB proof contains all k counts and is verified in one pass.

Asymptotic complexity for |outer matches| = k, outer tree B, leaf tree C: O(k · (log B + log C)).

Wire-format compat

• Existing leaf-only ACOR proofs are byte-identical: drive's existing aggregate_count_path_query + verify_aggregate_count_query round-trip works without changes.
• The new carrier shape is structurally different in the GroveDBProof envelope (multi-key outer merk proof + per-outer-key lower_layers) — it cannot be confused for a leaf proof.

What's NOT in this PR

• Drive-side consumer changes (separate PR in dashpay/platform).
• Conditional-subquery carriers (out of scope per spec).
• Multi-In on prefix (IN × IN) — the carrier accepts ONE outer fan-out dimension.

Test plan

• 10 new validation tests in grovedb-query exercising carrier rules: happy paths, RangeFull/ACOR outer rejection, missing/non-ACOR subquery rejection, conditional-branch rejection, empty-items rejection, Range-outer acceptance, dispatch behavior of the top-level validator.
• 7 new integration tests in grovedb:
  • acor_subquery_two_outer_keys_succeeds — 3-deep tree, two outer brands, 1 000 colors each, RangeAfter on color leaf; expect 2 entries in lex-asc order with the correct per-brand count.
  • acor_subquery_with_unknown_outer_key_returns_present_keys_only — absent outer key contributes no entry (prover doesn't emit a lower layer for it).
  • acor_subquery_rejects_acor_at_both_levels — validation and prove-entry gate both reject.
  • acor_subquery_carrier_with_range_outer_succeeds — carrier with a RangeAfter outer item.
  • acor_leaf_unchanged_under_per_key_verifier — leaf ACOR returns a one-entry Vec with an empty key and the same count verify_aggregate_count_query returns.
  • acor_per_key_rejects_non_acor_path_query — non-ACOR path query rejected up front.
  • acor_subquery_count_forgery_is_caught — bumping a HashWithCount in the leaf bytes through the carrier envelope is rejected by the chain check.
• cargo test --workspace --features 'minimal verify' — all 1516 lib tests pass.
• cargo clippy -p grovedb -p grovedb-query --features minimal -- -D warnings — clean.
• Drive-side integration (separate PR in dashpay/platform).

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features

  • Per-key aggregate-count verification for carrier-shaped queries and clearer leaf vs carrier query validation
  • Better handling of empty leaf subtrees so carrier proofs can report zero counts per outer key
  • query API now enforces leaf-only shape when returning a single aggregate count

• Bug Fixes

  • Require V1 proof envelopes for aggregate-count verification; V0 envelopes rejected with clear errors

• Tests

  • Large expansion of unit and end-to-end tests covering carrier scenarios, validation, and forgery detection

[coderabbitai]

Warning

Rate limit exceeded

@QuantumExplorer has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 43 minutes and 34 seconds before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: c6419b0e-1a71-4b13-9dd0-d50b485c4401

📥 Commits

Reviewing files that changed from the base of the PR and between 2bbf7d1 and d1cd151.

📒 Files selected for processing (3)

• grovedb/src/operations/get/query.rs
• grovedb/src/operations/proof/aggregate_count/per_key.rs
• grovedb/src/tests/aggregate_count_query_tests.rs

📝 Walkthrough

Walkthrough

This PR splits AggregateCountOnRange handling into explicit leaf vs carrier validations, adds query-level ACOR constructors/detectors, gates proof generation to V1 for ACOR, and implements classification-driven V1 verification (leaf-chain and per-key carrier walkers) with extensive carrier-focused tests.

Changes

ACOR Leaf and Carrier Shape Separation

Layer / File(s)	Summary
Query Construction, Detection, and Validation grovedb-query/src/aggregate_count.rs, grovedb-query/src/lib.rs	New module provides Query::new_aggregate_count_on_range constructor, detection helpers (aggregate_count_on_range, has_aggregate_count_on_range_anywhere), and independent validators for leaf and carrier ACOR shapes with comprehensive unit tests.
Query Type Wrapper Validators & API gating grovedb/src/query/mod.rs, grovedb/src/operations/get/query.rs	Adds SizedQuery::validate_leaf_aggregate_count_on_range and PathQuery::validate_leaf_aggregate_count_on_range, updates query_aggregate_count docs and runtime validation to enforce leaf-only shape at the single-u64 API.
Proof Generation Entry Point Version Gating grovedb/src/operations/proof/generate.rs	Detects ACOR requests early in prove_query_non_serialized, validates shape immediately, rejects V0 for ACOR proofs, and ensures V1 generation includes empty-count lower layers for carrier queries.
Proof helpers and lower-chain enforcement grovedb/src/operations/proof/aggregate_count/helpers.rs	Adds canonical GroveDBProof decoding with trailing-bytes check, merk-bytes guard, single-key and carrier-layer verification helpers, OuterMatch, and enforce_lower_chain.
Leaf-chain V1 verifier grovedb/src/operations/proof/aggregate_count/leaf_chain.rs	Implements recursive V1 leaf-chain verification that validates successive single-key proofs and delegates final leaf count verification.
AggregateCount classification grovedb/src/operations/proof/aggregate_count/classification.rs	Adds AggregateCountClassification and classify_aggregate_count_path_query to normalize leaf vs carrier shapes and extract inner-range/subquery_path.
Proof envelope glue / GroveDb entry points grovedb/src/operations/proof/aggregate_count/mod.rs	Adds GroveDb::verify_aggregate_count_query (leaf) and verify_aggregate_count_query_per_key (leaf-or-carrier) with V1-envelope enforcement and dispatch to the new walkers.
Per-key carrier verifier and subquery descent grovedb/src/operations/proof/aggregate_count/per_key.rs	Implements per-key V1 walker that executes carrier outer proofs, recurses subquery_path per matched outer key, and returns per-key counts in query order.
Carrier ACOR End-to-End Tests grovedb/src/tests/aggregate_count_query_tests.rs	Adds extensive carrier-focused integration tests covering multi-key results, absent-key handling, nested ACOR rejection, Range outer-item support, ordering, proof tampering detection, V0-envelope rejection, zero-count behavior, corruption/trailing-bytes checks, and entry-point validations.

🎯 4 (Complex) | ⏱️ ~45 minutes

🐰 A tale of shapes now split so clear,
Leaf and carrier, both so near,
With V1 proof and query-path so deep,
Our counts and keys safe we shall keep!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately captures the main change: enabling AggregateCountOnRange to function as a carrier subquery, which is the core feature addition enabling multi-outer-key aggregate count verification.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch claude/stoic-ardinghelli-508a69

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[codecov]

Codecov Report

❌ Patch coverage is 91.21996% with 95 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.76%. Comparing base (a917d92) to head (d1cd151).

Files with missing lines	Patch %	Lines
...db/src/operations/proof/aggregate_count/helpers.rs	81.76%	33 Missing ⚠️
...rovedb/src/operations/proof/aggregate_count/mod.rs	70.49%	18 Missing ⚠️
grovedb-query/src/aggregate_count.rs	96.71%	16 Missing ⚠️
...db/src/operations/proof/aggregate_count/per_key.rs	90.18%	16 Missing ⚠️
grovedb/src/operations/get/query.rs	83.58%	11 Missing ⚠️
grovedb/src/operations/proof/generate.rs	97.22%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop     #663      +/-   ##
===========================================
- Coverage    90.76%   90.76%   -0.01%     
===========================================
  Files          184      189       +5     
  Lines        55791    56407     +616     
===========================================
+ Hits         50639    51196     +557     
- Misses        5152     5211      +59     

Components	Coverage Δ
grovedb-core	88.49% <86.72%> (-0.03%)	⬇️
merk	92.32% <ø> (ø)
storage	86.36% <ø> (ø)
commitment-tree	96.43% <ø> (ø)
mmr	96.76% <ø> (ø)
bulk-append-tree	89.14% <ø> (ø)
element	95.75% <ø> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[coderabbitai]

Actionable comments posted: 2

🧹 Nitpick comments (2)

grovedb/src/tests/aggregate_count_query_tests.rs (2)

1656-1672: ⚡ Quick win

Make the ordering contract observable in the explicit-key test.

The test currently supplies brand_000 then brand_001 and asserts the same order, so it also passes if results preserve request order rather than canonical lexicographic order. Reversing the input keys would make the ordering guarantee real.

Suggested tweak

         let path_query = carrier_acor_path_query(
-            &[b"brand_000", b"brand_001"],
+            &[b"brand_001", b"brand_000"],
             QueryItem::RangeAfter(b"color_00499".to_vec()..),
         );

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@grovedb/src/tests/aggregate_count_query_tests.rs` around lines 1656 - 1672,
The test's ordering assertion isn't robust because it passes the keys in lex
order; change the carrier_acor_path_query call to provide the input keys in
reverse (e.g., pass b"brand_001" then b"brand_000") and update the subsequent
expectations for results[0].0 and results[1].0 accordingly so the test verifies
that GroveDb::verify_aggregate_count_query_per_key (used after prove_query)
returns canonical lexicographic ordering rather than preserving request order.

---

791-796: ⚡ Quick win

Assert the rejection reason, not just InvalidQuery.

This passes on any entry-point InvalidQuery, so a future regression could still look green even if the carrier/leaf-inner validation path changes. Pin the malformed-inner Key reason here as well.

Suggested tightening

-        assert!(
-            matches!(prove_result, Err(crate::Error::InvalidQuery(_))),
-            "carrier ACOR with malformed leaf-inner Key must be rejected at entry, got {:?}",
-            prove_result.map(|b| b.len())
-        );
+        match prove_result {
+            Err(crate::Error::InvalidQuery(msg)) => assert!(
+                msg.contains("AggregateCountOnRange may not wrap Key"),
+                "expected malformed inner ACOR rejection, got: {msg}"
+            ),
+            other => panic!(
+                "carrier ACOR with malformed leaf-inner Key must be rejected at entry, got {:?}",
+                other.map(|b| b.len())
+            ),
+        }

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@grovedb/src/tests/aggregate_count_query_tests.rs` around lines 791 - 796, The
test currently only checks for Err(crate::Error::InvalidQuery(_)) when calling
prove_query; tighten it to assert the specific invalid-query reason for a
malformed leaf-inner Key (the carrier ACOR path) by matching the concrete
InvalidQuery variant produced (e.g.,
Err(crate::Error::InvalidQuery(crate::InvalidQueryReason::MalformedLeafInnerKey))
or the exact enum/variant used in your codebase), or by asserting the error's
description/message equals the expected malformed-inner-Key reason; update the
assert to match that specific variant/message instead of the wildcard
underscore.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@grovedb/src/operations/proof/aggregate_count.rs`:
- Around line 95-100: The code currently runs ACOR validation on the inner Query
by calling path_query.query.query.validate_leaf_aggregate_count_on_range(),
which skips SizedQuery pagination checks (limit/offset); change the validation
call to run on the SizedQuery/PathQuery level (e.g., call
validate_leaf_aggregate_count_on_range on path_query.query or otherwise ensure
the SizedQuery wrapper is validated) and keep mapping errors via
crate::query_validation_error_to_invalid_query; apply the same change to the
other aggregate-count verification sites in this file that mirror this pattern
(the later blocks referencing inner_range/path_query and
validate_leaf_aggregate_count_on_range).
- Around line 140-141: The doc comments in aggregate_count.rs claim the returned
vector is "ascending lexicographic order" but the implementation respects the
query's left_to_right direction, so update the docs (in the comments around the
vector/return description in aggregate_count.rs and the similar blocks at the
other noted locations) to say "in query-direction (left_to_right) order" instead
of "ascending lexicographic order", or alternatively change the implementation
to sort the per-key result vector into ascending lexicographic order before
returning; mention or use the existing left_to_right flag in your change so the
behavior is explicit.

---

Nitpick comments:
In `@grovedb/src/tests/aggregate_count_query_tests.rs`:
- Around line 1656-1672: The test's ordering assertion isn't robust because it
passes the keys in lex order; change the carrier_acor_path_query call to provide
the input keys in reverse (e.g., pass b"brand_001" then b"brand_000") and update
the subsequent expectations for results[0].0 and results[1].0 accordingly so the
test verifies that GroveDb::verify_aggregate_count_query_per_key (used after
prove_query) returns canonical lexicographic ordering rather than preserving
request order.
- Around line 791-796: The test currently only checks for
Err(crate::Error::InvalidQuery(_)) when calling prove_query; tighten it to
assert the specific invalid-query reason for a malformed leaf-inner Key (the
carrier ACOR path) by matching the concrete InvalidQuery variant produced (e.g.,
Err(crate::Error::InvalidQuery(crate::InvalidQueryReason::MalformedLeafInnerKey))
or the exact enum/variant used in your codebase), or by asserting the error's
description/message equals the expected malformed-inner-Key reason; update the
assert to match that specific variant/message instead of the wildcard
underscore.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: f96586f0-6339-4473-8c8c-6234f3da9593

📥 Commits

Reviewing files that changed from the base of the PR and between a917d92 and f5e39e3.

📒 Files selected for processing (5)

• grovedb-query/src/query.rs
• grovedb/src/lib.rs
• grovedb/src/operations/proof/aggregate_count.rs
• grovedb/src/query/mod.rs
• grovedb/src/tests/aggregate_count_query_tests.rs

[coderabbitai]

🧹 Nitpick comments (1)

grovedb/src/operations/proof/aggregate_count.rs (1)

261-270: 💤 Low value

Trailing-byte tolerance in proof decoding.

bincode::decode_from_slice returns the number of bytes consumed but the value is discarded, so any trailing bytes appended to a valid proof are silently ignored. Since the resulting (RootHash, count) is still bound by the cryptographic chain check, this is not a correctness/security issue, but it does make the encoding non-canonical (multiple byte-strings can verify to the same result). If strict canonical-encoding is desired here, consider rejecting proofs whose consumed != proof.len().

♻️ Optional canonical-encoding tweak

-    let (proof, _) = bincode::decode_from_slice(proof, config)
-        .map_err(|e| Error::CorruptedData(format!("unable to decode proof: {}", e)))?;
-    Ok(proof)
+    let (decoded, consumed) = bincode::decode_from_slice(proof, config)
+        .map_err(|e| Error::CorruptedData(format!("unable to decode proof: {}", e)))?;
+    if consumed != proof.len() {
+        return Err(Error::CorruptedData(format!(
+            "proof has {} trailing bytes after decoding",
+            proof.len() - consumed
+        )));
+    }
+    Ok(decoded)

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@grovedb/src/operations/proof/aggregate_count.rs` around lines 261 - 270,
decode_grovedb_proof currently discards the byte-count returned by
bincode::decode_from_slice which allows trailing bytes to be ignored and makes
the encoding non-canonical; change decode_grovedb_proof to capture the returned
consumed length from bincode::decode_from_slice(proof, config) and if consumed
!= proof.len() return an Error::CorruptedData (or a new descriptive error)
indicating trailing bytes, so only proofs that consume the entire input are
accepted.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@grovedb/src/operations/proof/aggregate_count.rs`:
- Around line 261-270: decode_grovedb_proof currently discards the byte-count
returned by bincode::decode_from_slice which allows trailing bytes to be ignored
and makes the encoding non-canonical; change decode_grovedb_proof to capture the
returned consumed length from bincode::decode_from_slice(proof, config) and if
consumed != proof.len() return an Error::CorruptedData (or a new descriptive
error) indicating trailing bytes, so only proofs that consume the entire input
are accepted.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: c8ccea73-86cd-40f1-bdfc-f9828934a20a

📥 Commits

Reviewing files that changed from the base of the PR and between f5e39e3 and d8ddda5.

📒 Files selected for processing (7)

• grovedb-query/src/aggregate_count.rs
• grovedb-query/src/lib.rs
• grovedb-query/src/query.rs
• grovedb/src/operations/proof/aggregate_count.rs
• grovedb/src/operations/proof/generate.rs
• grovedb/src/query/mod.rs
• grovedb/src/tests/aggregate_count_query_tests.rs

💤 Files with no reviewable changes (1)

• grovedb-query/src/query.rs

✅ Files skipped from review due to trivial changes (1)

• grovedb-query/src/lib.rs

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@grovedb/src/operations/proof/aggregate_count/per_key.rs`:
- Around line 132-136: Replace the panic caused by
classification.carrier_subquery_path.as_ref().expect(...) with error
propagation: change to an ok_or_else that returns Error::InvalidProof with a
clear contextual message (e.g. "missing carrier_subquery_path in
classification") so the verifier treats this as a verification failure instead
of aborting; where subsequent fallible calls are made on that path, wrap/map
errors with context using .map_err(|e| Error::CorruptedData(format!("context:
{}", e))) as per the coding guideline so all failures are returned with useful
context.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 4a72e90e-06f9-4f51-ad86-08bd46a9d9e6

📥 Commits

Reviewing files that changed from the base of the PR and between d8ddda5 and 2bbf7d1.

📒 Files selected for processing (10)

• grovedb-query/src/aggregate_count.rs
• grovedb/src/operations/get/query.rs
• grovedb/src/operations/proof/aggregate_count.rs
• grovedb/src/operations/proof/aggregate_count/classification.rs
• grovedb/src/operations/proof/aggregate_count/helpers.rs
• grovedb/src/operations/proof/aggregate_count/leaf_chain.rs
• grovedb/src/operations/proof/aggregate_count/mod.rs
• grovedb/src/operations/proof/aggregate_count/per_key.rs
• grovedb/src/operations/proof/generate.rs
• grovedb/src/tests/aggregate_count_query_tests.rs

💤 Files with no reviewable changes (1)

• grovedb/src/operations/proof/aggregate_count.rs

🚧 Files skipped from review as they are similar to previous changes (2)

• grovedb-query/src/aggregate_count.rs
• grovedb/src/tests/aggregate_count_query_tests.rs

[QuantumExplorer]

Self Reviewed.