Skip to content

feat: get_queue_elements for input+output queues#2041

Merged
ananas-block merged 8 commits intomainfrom
sergey/get_queue_elements_for_2_queues
Nov 10, 2025
Merged

feat: get_queue_elements for input+output queues#2041
ananas-block merged 8 commits intomainfrom
sergey/get_queue_elements_for_2_queues

Conversation

@sergeytimoshin
Copy link
Contributor

@sergeytimoshin sergeytimoshin commented Nov 7, 2025

Summary by CodeRabbit

  • New Features

    • Indexer APIs and client models now return separate input/output queue results with per-queue pagination.
    • Test environment setup accepts an explicit gRPC port option.
    • Forester config adds sleep timing controls for post-processing and idle waits.
  • Improvements

    • Spawned test processes preserve environment and default to debug logging when unset.
    • Internal queue handling updated to the new input/output response shape.
  • Breaking Changes

    • Single-queue parameters and unified queue responses replaced with per-queue fields and limits.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Nov 7, 2025

Walkthrough

Refactors queue APIs to explicit optional output/input boundaries and separate queue result fields, threads grpcPort through CLI test env/startup, ensures spawned binaries inherit process.env with RUST_LOG defaulted to "debug", and adds configurable sleep timings to Forester config and usage.

Changes

Cohort / File(s) Summary
CLI Setup & Processes
cli/src/commands/test-validator/index.ts, cli/src/utils/initTestEnv.ts, cli/src/utils/processPhotonIndexer.ts, cli/src/utils/process.ts
Reorder static flag entry for prover-port; add optional grpcPort to initTestEnv signature and pass it to startIndexer; move grpcPort parameter earlier in startIndexer; spawnBinary now forwards process.env and ensures RUST_LOG="debug" if unset.
Indexer Trait & RPC
sdk-libs/client/src/indexer/indexer_trait.rs, sdk-libs/client/src/rpc/indexer.rs, sdk-libs/program-test/src/program_test/indexer.rs
Replace QueueType + (limit, start) API with explicit optional output_queue_start_index, output_queue_limit, input_queue_start_index, input_queue_limit; remove QueueType imports and update forwarding.
Photon Indexer Implementation
sdk-libs/client/src/indexer/photon_indexer.rs
Update get_queue_elements signature to accept per-queue boundaries; construct request with output/input start/limit; parse and assemble output_queue_elements/input_queue_elements and corresponding indices.
Queue Result Types & API Models
sdk-libs/client/src/indexer/types.rs, sdk-libs/photon-api/src/models/_get_queue_elements_post_200_response_result.rs, sdk-libs/photon-api/src/models/_get_queue_elements_post_request_params.rs
Replace unified elements/first_value_queue_index with optional output_queue_elements/output_queue_index and input_queue_elements/input_queue_index; request params drop limit/queue_type and add per-queue fields; constructors and serde attrs updated.
Test & Program Indexer Implementations
sdk-libs/program-test/src/indexer/test_indexer.rs, sdk-libs/program-test/src/program_test/indexer.rs
Adapt implementations to new boundary-driven parameters; compute end indices from limits; return separate output/input elements and indices; remove QueueType-based logic.
Call Sites & Forester Integration
forester-utils/src/instructions/state_batch_append.rs, forester-utils/src/instructions/state_batch_nullify.rs, program-tests/utils/src/e2e_test_env.rs, program-tests/utils/src/test_batch_forester.rs
Update get_queue_elements calls to use None/Some(...) boundary params; replace accesses of elements/first_value_queue_index with output_queue_elements.unwrap_or_default()/output_queue_index (or input equivalents); remove QueueType usage.
Forester Config & Timing
forester/src/config.rs, forester/src/epoch_manager.rs, forester/tests/*, forester/tests/legacy/*
Add sleep_after_processing_ms and sleep_when_idle_ms to GeneralConfig (defaults and test overrides); replace hard-coded sleeps with config-driven values; remove an info log in slot processing; update tests/config generators.
Misc & Versioning
prover/server/main.go, scripts/devenv/versions.sh
Bump prover server Version const from "2.0.0" to "2.0.6"; update PHOTON_COMMIT hash in devenv versions script.

Sequence Diagram(s)

sequenceDiagram
    participant CLI as CLI (Test Validator)
    participant Init as initTestEnv
    participant Proc as processPhotonIndexer
    participant IndexerBin as Indexer Binary

    CLI->>Init: run SetupCommand (flags incl. --grpc-port?)
    Init->>Init: read grpcPort param
    Init->>Proc: startIndexer(rpcUrl, indexerPort, grpcPort)
    Proc->>Proc: build args (--rpc-url, --port, --grpc-port)
    Proc->>IndexerBin: spawnBinary(args, env=process.env + ensure RUST_LOG)
    IndexerBin->>IndexerBin: start services, listen on grpcPort
Loading
sequenceDiagram
    participant Caller as Caller
    participant Trait as Indexer Trait
    participant Photon as PhotonIndexer
    participant API as Photon API

    rect rgba(220,220,255,0.12)
      Note over Caller,Trait: Old (QueueType-based)
      Caller->>Trait: get_queue_elements(pubkey, QueueType, limit, start)
      Trait->>Photon: forward queue_type, limit, start
    end

    rect rgba(220,255,220,0.12)
      Note over Caller,Trait: New (Boundary-based)
      Caller->>Trait: get_queue_elements(pubkey, output_start?, output_limit?, input_start?, input_limit?)
      Trait->>Photon: forward per-queue boundaries
      Photon->>API: POST {output_queue_start_index, output_queue_limit, input_queue_start_index, input_queue_limit}
      API-->>Photon: {output_queue_elements?, output_queue_index?, input_queue_elements?, input_queue_index?}
      Photon->>Trait: Response(QueueElementsResult { output_queue_elements?, input_queue_elements?, ... })
    end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

  • Heterogeneous changes across traits, implementations, API models, CLI startup, and tests.
  • Pay extra attention to:
    • sdk-libs/client/src/indexer/photon_indexer.rs — dual-queue parsing and mapping.
    • sdk-libs/program-test/src/indexer/test_indexer.rs — index/limit math and proof assembly.
    • sdk-libs/photon-api/src/models/* — serde renames, constructors, option defaults.
    • cli/src/utils/process.ts and cli/src/utils/initTestEnv.ts — env inheritance and grpcPort wiring.

Possibly related PRs

Suggested reviewers

  • SwenSchaeferjohann
  • ananas-block

Poem

🐇 I hopped through queues both in and out,

boundaries set, no more doubt.
Grpc port carried down the line,
debug logs shining, configs fine.
A tiny hop — the change aligns. 🥕

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main change: refactoring get_queue_elements to support both input and output queues atomically, replacing the previous queue_type-based approach.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch sergey/get_queue_elements_for_2_queues

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e78a060 and b292967.

📒 Files selected for processing (2)
  • prover/server/main.go (1 hunks)
  • scripts/devenv/versions.sh (1 hunks)
✅ Files skipped from review due to trivial changes (1)
  • prover/server/main.go
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (26)
  • GitHub Check: integration-test-lightweight-lazy
  • GitHub Check: lean-verification
  • GitHub Check: test-with-redis (Queue cleanup tests, go test -v -run TestCleanup -timeout 5m)
  • GitHub Check: test-with-redis (Failed job status tests, go test -v -run TestFailedJobStatus -timeout 5m)
  • GitHub Check: integration-test-lightweight
  • GitHub Check: test-no-redis (Unit tests, go test ./prover/... -timeout 60m)
  • GitHub Check: test-with-redis (Redis Queue tests, go test -v -run TestRedis -timeout 10m)
  • GitHub Check: test-with-redis (Queue processing flow tests, go test -v -run TestJobProcessingFlow -timeout 5m)
  • GitHub Check: system-programs (token test, ["cargo-test-sbf -p sdk-token-test"])
  • GitHub Check: system-programs (anchor & pinocchio, ["cargo-test-sbf -p sdk-anchor-test", "cargo-test-sbf -p sdk...
  • GitHub Check: system-programs (sdk-libs, light-sdk-macros light-sdk light-program-test light-client light-compr...
  • GitHub Check: system-programs (native, ["cargo-test-sbf -p sdk-native-test", "cargo-test-sbf -p sdk-v1-native-t...
  • GitHub Check: programs (system-cpi-test-v2-functional-read-only, ["cargo-test-sbf -p system-cpi-v2-test -- func...
  • GitHub Check: programs (system-cpi-test, ["cargo-test-sbf -p system-cpi-test", "cargo test -p light-system-prog...
  • GitHub Check: programs (system-cpi-test-v2-functional-account-infos, ["cargo-test-sbf -p system-cpi-v2-test -- ...
  • GitHub Check: programs (compressed-token-and-e2e, ["cargo-test-sbf -p compressed-token-test --test v1", "cargo-...
  • GitHub Check: programs (compressed-token-batched-tree, ["cargo-test-sbf -p compressed-token-test -- test_transf...
  • GitHub Check: cli-v1
  • GitHub Check: programs (light-system-program-compression, ["cargo-test-sbf -p system-test -- test_with_compress...
  • GitHub Check: stateless-js-v1
  • GitHub Check: cli-v2
  • GitHub Check: Forester e2e test
  • GitHub Check: programs (light-system-program-address, ["cargo-test-sbf -p system-test -- test_with_address", "c...
  • GitHub Check: programs (account-compression-and-registry, ["cargo-test-sbf -p account-compression-test", "cargo...
  • GitHub Check: lint
  • GitHub Check: stateless-js-v2
🔇 Additional comments (1)
scripts/devenv/versions.sh (1)

16-16: Verify the Photon commit update is intentional and compatible.

This line updates the PHOTON_COMMIT hash, but the relationship to the PR objectives (queue API refactors, grpcPort threading, environment configuration) is unclear from this change alone.

Please confirm:

  • The new commit 94b3688b08477668bb946a689b0267319f5c1ae1 exists in the Photon repository.
  • Any breaking changes or compatibility considerations between the old and new commits.
  • Whether this update is necessary for the queue API refactoring changes described in the PR objectives.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Nitpick comments (2)
sdk-libs/client/src/local_test_validator.rs (1)

12-12: LGTM! Optional: Consider u16 for semantic precision.

The addition of the grpc_port field is correctly implemented and follows the existing pattern for optional configuration fields. The default value of None is appropriate.

Optionally, since port numbers range from 0 to 65535, using Option<u16> instead of Option<u64> would be more semantically precise, though the current implementation is functionally correct.

Also applies to: 23-23

sdk-libs/client/src/indexer/photon_indexer.rs (1)

1661-1694: Extract duplicated parsing logic into a helper function.

The input queue parsing logic (lines 1661-1694) is nearly identical to the output queue parsing logic (lines 1626-1659). Consider extracting a helper function to reduce duplication.

Consider this refactor:

fn parse_queue_elements(
    elements: Vec<models::GetQueueElementsResponseValue>,
) -> Result<Vec<MerkleProofWithContext>, IndexerError> {
    elements
        .iter()
        .map(|x| {
            let proof: Result<Vec<_>, _> = x
                .proof
                .iter()
                .map(|x| Hash::from_base58(x))
                .collect();
            let proof = proof?;
            let root = Hash::from_base58(&x.root)?;
            let leaf = Hash::from_base58(&x.leaf)?;
            let merkle_tree = Hash::from_base58(&x.tree)?;
            let tx_hash = x
                .tx_hash
                .as_ref()
                .map(|x| Hash::from_base58(x))
                .transpose()?;
            let account_hash = Hash::from_base58(&x.account_hash)?;

            Ok(MerkleProofWithContext {
                proof,
                root,
                leaf_index: x.leaf_index,
                leaf,
                merkle_tree,
                root_seq: x.root_seq,
                tx_hash,
                account_hash,
            })
        })
        .collect()
}

Then use it for both queues:

let output_queue_elements = api_result
    .output_queue_elements
    .map(|elements| parse_queue_elements(elements))
    .transpose()?;

let input_queue_elements = api_result
    .input_queue_elements
    .map(|elements| parse_queue_elements(elements))
    .transpose()?;
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d83518e and 5e85e0d.

📒 Files selected for processing (20)
  • .github/actions/setup-and-build/action.yml (1 hunks)
  • cli/src/commands/test-validator/index.ts (2 hunks)
  • cli/src/utils/initTestEnv.ts (3 hunks)
  • cli/src/utils/process.ts (1 hunks)
  • cli/src/utils/processPhotonIndexer.ts (2 hunks)
  • forester-utils/src/instructions/state_batch_append.rs (1 hunks)
  • forester-utils/src/instructions/state_batch_nullify.rs (1 hunks)
  • program-tests/utils/src/e2e_test_env.rs (1 hunks)
  • program-tests/utils/src/test_batch_forester.rs (1 hunks)
  • scripts/devenv/versions.sh (1 hunks)
  • sdk-libs/client/src/indexer/indexer_trait.rs (1 hunks)
  • sdk-libs/client/src/indexer/photon_indexer.rs (3 hunks)
  • sdk-libs/client/src/indexer/types.rs (1 hunks)
  • sdk-libs/client/src/local_test_validator.rs (3 hunks)
  • sdk-libs/client/src/rpc/indexer.rs (2 hunks)
  • sdk-libs/photon-api/src/models/_get_queue_elements_post_200_response_result.rs (1 hunks)
  • sdk-libs/photon-api/src/models/_get_queue_elements_post_request_params.rs (1 hunks)
  • sdk-libs/photon-api/src/models/account_context.rs (1 hunks)
  • sdk-libs/program-test/src/indexer/test_indexer.rs (9 hunks)
  • sdk-libs/program-test/src/program_test/indexer.rs (2 hunks)
🧰 Additional context used
🧠 Learnings (15)
📚 Learning: 2025-10-16T06:33:19.426Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: program-libs/compressible/CLAUDE.md:0-0
Timestamp: 2025-10-16T06:33:19.426Z
Learning: Applies to program-libs/compressible/src/config.rs : Ensure serialization compatibility across Anchor, Pinocchio, and Borsh for core account types used by dependent programs

Applied to files:

  • sdk-libs/photon-api/src/models/account_context.rs
  • program-tests/utils/src/test_batch_forester.rs
  • program-tests/utils/src/e2e_test_env.rs
  • sdk-libs/program-test/src/program_test/indexer.rs
  • sdk-libs/client/src/indexer/photon_indexer.rs
📚 Learning: 2025-10-11T21:59:25.222Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: program-libs/account-checks/CLAUDE.md:0-0
Timestamp: 2025-10-11T21:59:25.222Z
Learning: Applies to program-libs/account-checks/src/**/*.rs : Validate account type with 8-byte discriminators using check_discriminator before deserialization

Applied to files:

  • sdk-libs/photon-api/src/models/account_context.rs
📚 Learning: 2025-10-11T21:59:25.222Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: program-libs/account-checks/CLAUDE.md:0-0
Timestamp: 2025-10-11T21:59:25.222Z
Learning: Applies to program-libs/account-checks/src/account_info/account_info_trait.rs : Ensure the crate compiles with no features enabled by keeping trait definitions free of SDK-specific dependencies

Applied to files:

  • sdk-libs/photon-api/src/models/account_context.rs
  • sdk-libs/program-test/src/program_test/indexer.rs
  • sdk-libs/client/src/rpc/indexer.rs
📚 Learning: 2025-10-11T21:59:25.222Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: program-libs/account-checks/CLAUDE.md:0-0
Timestamp: 2025-10-11T21:59:25.222Z
Learning: Applies to program-libs/account-checks/src/account_info/{solana.rs,pinocchio.rs,test_account_info.rs} : Provide SDK-specific AccountInfoTrait implementations in account_info/{solana.rs,pinocchio.rs,test_account_info.rs}

Applied to files:

  • sdk-libs/photon-api/src/models/account_context.rs
  • sdk-libs/program-test/src/program_test/indexer.rs
  • sdk-libs/client/src/rpc/indexer.rs
📚 Learning: 2025-10-15T03:46:26.767Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: programs/registry/CLAUDE.md:0-0
Timestamp: 2025-10-15T03:46:26.767Z
Learning: Applies to programs/registry/src/account_compression_cpi/*.rs : Context structs for wrapper instructions must include standard accounts: optional registered_forester_pda (mut), authority Signer, cpi_authority with seeds/bump for CPI_AUTHORITY_PDA_SEED, registered_program_pda, target program handle, log_wrapper, and mutable target_account.

Applied to files:

  • sdk-libs/photon-api/src/models/account_context.rs
📚 Learning: 2025-10-11T21:59:25.222Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: program-libs/account-checks/CLAUDE.md:0-0
Timestamp: 2025-10-11T21:59:25.222Z
Learning: Applies to program-libs/account-checks/src/**/*.rs : On account initialization, call account_info_init to set the 8-byte discriminator

Applied to files:

  • sdk-libs/photon-api/src/models/account_context.rs
📚 Learning: 2025-10-11T21:59:25.222Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: program-libs/account-checks/CLAUDE.md:0-0
Timestamp: 2025-10-11T21:59:25.222Z
Learning: Applies to program-libs/account-checks/src/!(account_info)/**/*.rs : Use AccountInfoTrait for runtime-agnostic account handling; avoid direct solana-program or pinocchio AccountInfo in general logic

Applied to files:

  • sdk-libs/photon-api/src/models/account_context.rs
  • sdk-libs/program-test/src/program_test/indexer.rs
  • sdk-libs/client/src/rpc/indexer.rs
📚 Learning: 2025-10-11T21:59:25.222Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: program-libs/account-checks/CLAUDE.md:0-0
Timestamp: 2025-10-11T21:59:25.222Z
Learning: Applies to program-libs/account-checks/src/account_info/test_account_info.rs : Use the mock AccountInfo implementation under the test-only feature for unit tests

Applied to files:

  • sdk-libs/photon-api/src/models/account_context.rs
📚 Learning: 2025-10-11T21:59:25.222Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: program-libs/account-checks/CLAUDE.md:0-0
Timestamp: 2025-10-11T21:59:25.222Z
Learning: Applies to program-libs/account-checks/src/account_info/{solana.rs,pinocchio.rs,test_account_info.rs} : Gate SDK-specific implementations with #[cfg(feature = "solana"|"pinocchio"|"test-only")]

Applied to files:

  • sdk-libs/photon-api/src/models/account_context.rs
  • sdk-libs/program-test/src/program_test/indexer.rs
  • sdk-libs/client/src/rpc/indexer.rs
📚 Learning: 2025-10-16T06:33:19.426Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: program-libs/compressible/CLAUDE.md:0-0
Timestamp: 2025-10-16T06:33:19.426Z
Learning: Applies to program-libs/compressible/src/config.rs : Provide default initialization for the CToken V1 config in CompressibleConfig

Applied to files:

  • sdk-libs/client/src/local_test_validator.rs
📚 Learning: 2025-10-15T03:46:26.767Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: programs/registry/CLAUDE.md:0-0
Timestamp: 2025-10-15T03:46:26.767Z
Learning: Applies to programs/registry/src/lib.rs : Load accounts according to type before check_forester: batched via BatchedMerkleTreeAccount::type_from_account_info(); regular via ctx.accounts.account.load()?.metadata; use custom deserialization when required.

Applied to files:

  • program-tests/utils/src/test_batch_forester.rs
  • forester-utils/src/instructions/state_batch_append.rs
  • forester-utils/src/instructions/state_batch_nullify.rs
  • program-tests/utils/src/e2e_test_env.rs
  • sdk-libs/program-test/src/program_test/indexer.rs
  • sdk-libs/client/src/indexer/photon_indexer.rs
📚 Learning: 2025-10-15T03:46:26.767Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: programs/registry/CLAUDE.md:0-0
Timestamp: 2025-10-15T03:46:26.767Z
Learning: Applies to programs/registry/src/lib.rs : Instruction handlers must compute work_units by operation type: batch operations use account.queue_batches.batch_size; single operations use DEFAULT_WORK_V1; custom operations compute based on complexity.

Applied to files:

  • program-tests/utils/src/test_batch_forester.rs
  • forester-utils/src/instructions/state_batch_append.rs
📚 Learning: 2025-10-11T21:59:25.222Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: program-libs/account-checks/CLAUDE.md:0-0
Timestamp: 2025-10-11T21:59:25.222Z
Learning: Applies to program-libs/account-checks/src/packed_accounts.rs : For dynamic account sets, use PackedAccounts for index-based access with bounds checks instead of manual indexing

Applied to files:

  • program-tests/utils/src/e2e_test_env.rs
📚 Learning: 2025-10-11T21:59:25.222Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: program-libs/account-checks/CLAUDE.md:0-0
Timestamp: 2025-10-11T21:59:25.222Z
Learning: Applies to program-libs/account-checks/src/checks.rs : Expose and maintain account validation helpers (check_owner, check_program, check_mut/non_mut, check_signer, check_discriminator, set_discriminator, check_pda_seeds, check_account_balance_is_rent_exempt, account_info_init) in checks.rs

Applied to files:

  • sdk-libs/program-test/src/program_test/indexer.rs
📚 Learning: 2025-10-11T21:59:25.222Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: program-libs/account-checks/CLAUDE.md:0-0
Timestamp: 2025-10-11T21:59:25.222Z
Learning: Applies to program-libs/account-checks/src/**/*.rs : Return AccountError variants (codes 12006–12021) and rely on automatic ProgramError conversions; avoid returning raw ProgramError directly

Applied to files:

  • sdk-libs/program-test/src/program_test/indexer.rs
🧬 Code graph analysis (7)
program-tests/utils/src/test_batch_forester.rs (1)
program-tests/utils/src/e2e_test_env.rs (1)
  • addresses (757-757)
sdk-libs/program-test/src/indexer/test_indexer.rs (4)
program-tests/utils/src/e2e_test_env.rs (2)
  • new (343-417)
  • addresses (757-757)
sdk-libs/photon-api/src/models/_get_queue_elements_post_200_response_result.rs (1)
  • new (48-56)
sdk-libs/photon-api/src/models/_get_queue_elements_post_request_params.rs (1)
  • new (49-57)
program-tests/utils/src/test_batch_forester.rs (1)
  • addresses (664-670)
program-tests/utils/src/e2e_test_env.rs (1)
program-tests/utils/src/test_batch_forester.rs (1)
  • addresses (664-670)
sdk-libs/photon-api/src/models/_get_queue_elements_post_request_params.rs (3)
sdk-libs/photon-api/src/models/_get_queue_elements_post_200_response_result.rs (1)
  • new (48-56)
sdk-libs/photon-api/src/models/_get_batch_address_update_info_post_request_params.rs (1)
  • new (29-35)
sdk-libs/photon-api/src/models/get_queue_elements_response_value.rs (1)
  • new (39-58)
sdk-libs/photon-api/src/models/_get_queue_elements_post_200_response_result.rs (2)
sdk-libs/photon-api/src/models/_get_queue_elements_post_request_params.rs (1)
  • new (49-57)
sdk-libs/program-test/src/indexer/test_indexer.rs (1)
  • new (1357-1418)
sdk-libs/client/src/indexer/photon_indexer.rs (1)
sdk-libs/client/src/indexer/base58.rs (2)
  • from_base58 (8-10)
  • from_base58 (22-24)
sdk-libs/client/src/indexer/indexer_trait.rs (4)
sdk-libs/client/src/indexer/photon_indexer.rs (1)
  • get_queue_elements (1580-1735)
sdk-libs/client/src/rpc/indexer.rs (1)
  • get_queue_elements (204-226)
sdk-libs/program-test/src/indexer/test_indexer.rs (1)
  • get_queue_elements (618-875)
sdk-libs/program-test/src/program_test/indexer.rs (1)
  • get_queue_elements (200-222)
🔇 Additional comments (18)
.github/actions/setup-and-build/action.yml (1)

26-30: Correct addition of protobuf-compiler dependency.

Adding protobuf-compiler to the system dependencies is the right call for supporting the protobuf-based code generation referenced in the PR objectives. The placement and syntax are correct.

sdk-libs/client/src/local_test_validator.rs (1)

51-53: LGTM! Correct argument handling.

The conditional logic correctly appends the --grpc-port argument when configured, following the same pattern as other optional arguments in this function.

scripts/devenv/versions.sh (1)

16-16: PHOTON_COMMIT hash verified and confirmed.

The commit 94b3688b08477668bb946a689b0267319f5c1ae1 exists in the Lightprotocol/photon repository and aligns with this PR's objectives. The commit message "feat: distinct output and input queues in the get_queue_elements" directly corresponds to the queue API refactoring scope of this PR. No issues identified.

sdk-libs/photon-api/src/models/account_context.rs (1)

27-27: Type widening aligns with queue refactoring.

The change from Option<u16> to Option<u64> for nullifier_queue_index significantly increases the maximum index capacity (65K → 18 quintillion). This aligns with the broader PR refactoring that separates input/output queue handling and is appropriate for long-running queues.

cli/src/utils/process.ts (1)

215-218: LGTM!

The environment configuration correctly preserves existing variables while ensuring RUST_LOG defaults to "debug" for spawned binaries. The fallback pattern (process.env.RUST_LOG || "debug") is appropriate for development/test environments.

cli/src/utils/initTestEnv.ts (1)

89-89: LGTM!

The grpcPort parameter is properly wired with a sensible default (50051, standard gRPC port) and correctly propagated to startIndexer. The optional parameter preserves backward compatibility.

Also applies to: 104-104, 133-133

forester-utils/src/instructions/state_batch_nullify.rs (2)

126-127: LGTM!

The extraction of queue elements correctly uses input_queue_elements.unwrap_or_default() and input_queue_index, aligning with the new response structure that separates input and output queue data.


113-122: Parameter mapping verified as correct.

The get_queue_elements call uses the updated API signature correctly:

  • None, None for output queue parameters (appropriate since nullify doesn't fetch output elements)
  • next_queue_index (Option) for input queue start
  • Some(zkp_batch_size) where zkp_batch_size: u16 for input queue limit
  • Parameter order and types match the trait definition in sdk-libs/client/src/indexer/indexer_trait.rs
cli/src/utils/processPhotonIndexer.ts (1)

42-42: LGTM!

The grpcPort parameter (default 50051) is correctly added to the function signature and properly passed to the Photon indexer as --grpc-port. The change maintains backward compatibility through the default value.

Also applies to: 59-60

cli/src/commands/test-validator/index.ts (2)

72-77: LGTM!

The grpc-port flag is well-defined with:

  • Appropriate type (integer)
  • Sensible default (50051, standard gRPC port)
  • Correct exclusivity constraint (cannot be used with skip-indexer)
  • Clear description

203-203: LGTM!

The grpcPort value is correctly propagated from the flag to initTestEnv.

program-tests/utils/src/e2e_test_env.rs (1)

746-757: LGTM!

The update to get_queue_elements correctly fetches output queue elements for the address append operation by passing None, Some(batch.batch_size as u16) for output queue parameters. The extraction using output_queue_elements.unwrap_or_default() aligns with the new API structure.

program-tests/utils/src/test_batch_forester.rs (1)

654-670: LGTM!

The queue element retrieval is correctly updated to use the new API signature with None, Some(batch.zkp_batch_size as u16) for output queue parameters, and the extraction properly accesses output_queue_elements.unwrap_or_default() for address batch updates.

sdk-libs/photon-api/src/models/_get_queue_elements_post_request_params.rs (2)

14-46: LGTM! Well-structured dual-queue parameter design.

The refactoring to separate output and input queue parameters is clear and well-implemented. All optional fields have proper serde configuration.


49-57: LGTM! Constructor design is flexible and appropriate.

Simplifying the constructor to only require the tree parameter while initializing all queue parameters to None provides good API flexibility.

sdk-libs/client/src/rpc/indexer.rs (1)

204-226: LGTM! Clean parameter forwarding.

The method correctly forwards the new queue parameters to the inner indexer implementation.

sdk-libs/photon-api/src/models/_get_queue_elements_post_200_response_result.rs (2)

14-45: LGTM! Response structure aligns well with the new dual-queue API.

The separation of output and input queue elements and indices is clear and consistent with the request parameters.


48-56: LGTM! Constructor design is consistent with the request params.

Initializing all optional queue fields to None provides appropriate flexibility.

- add `--grpc-port` flag for test-validator CLI command.
- Enabled configurable logging via `RUST_LOG` for spawned processes.
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
sdk-libs/program-test/src/indexer/test_indexer.rs (1)

714-730: Return the recovered Merkle proof instead of an empty vector

In both recovery branches (Line 719 and Line 797) you rebuild the tree until get_proof_of_leaf succeeds, but then drop the resulting Vec and return Vec::new(). Downstream consumers receive an empty proof, so verification fails even though the proof exists. Return the Vec produced by get_proof_of_leaf (and apply the same change in the output-queue block).

-                                Err(_) => {
+                                Err(_) => {
                                     let mut next_index =
                                         state_tree_bundle.merkle_tree.get_next_index() as u64;
                                     while next_index < leaf_info.leaf_index as u64 {
                                         state_tree_bundle.merkle_tree.append(&[0u8; 32]).unwrap();
                                         next_index =
                                             state_tree_bundle.merkle_tree.get_next_index() as u64;
                                     }
-                                    state_tree_bundle
-                                        .merkle_tree
-                                        .get_proof_of_leaf(leaf_info.leaf_index as usize, true)
-                                        .unwrap()
-                                        .to_vec();
-                                    Vec::new()
+                                    state_tree_bundle
+                                        .merkle_tree
+                                        .get_proof_of_leaf(leaf_info.leaf_index as usize, true)
+                                        .unwrap()
+                                        .to_vec()
                                 }
...
-                                Err(_) => {
+                                Err(_) => {
                                     let mut next_index =
                                         state_tree_bundle.merkle_tree.get_next_index() as u64;
                                     while next_index < **index {
                                         state_tree_bundle.merkle_tree.append(&[0u8; 32]).unwrap();
                                         next_index =
                                             state_tree_bundle.merkle_tree.get_next_index() as u64;
                                     }
-                                    state_tree_bundle
-                                        .merkle_tree
-                                        .get_proof_of_leaf(**index as usize, true)
-                                        .unwrap()
-                                        .to_vec();
-                                    Vec::new()
+                                    state_tree_bundle
+                                        .merkle_tree
+                                        .get_proof_of_leaf(**index as usize, true)
+                                        .unwrap()
+                                        .to_vec()
                                 }

Also applies to: 790-803

🧹 Nitpick comments (4)
program-tests/compressed-token-test/tests/v1.rs (1)

5441-5448: Explicit grpc_port: None is fine; consider Default struct update to future‑proof.

No behavior change. To reduce churn when new fields are added, rely on Default for untouched fields.

Apply:

 spawn_validator(LightValidatorConfig {
   enable_indexer: false,
   enable_prover: true,
   wait_time: 15,
   sbf_programs: vec![],
-  limit_ledger_size: None,
-  grpc_port: None,
+  ..Default::default()
 })
forester/tests/test_batch_append_spent.rs (1)

47-55: LGTM; optional cleanup using Default for untouched fields.

Keeps behavior unchanged. Prefer struct update with Default for stability across future field additions.

 init(Some(LightValidatorConfig {
   enable_indexer: false,
   enable_prover: true,
   wait_time: 10,
   sbf_programs: vec![],
-  limit_ledger_size: None,
-  grpc_port: None,
+  ..Default::default()
 }))
program-tests/system-cpi-v2-test/tests/event.rs (1)

532-543: Good addition; minor refactor to reduce maintenance.

Use Default struct update to avoid touching tests when new config fields appear.

 spawn_validator(LightValidatorConfig {
   enable_indexer: false,
   enable_prover: true,
   wait_time: 10,
   sbf_programs: vec![(
     create_address_test_program::ID.to_string(),
     "../../target/deploy/create_address_test_program.so".to_string(),
   )],
-  limit_ledger_size: None,
-  grpc_port: None,
+  ..Default::default()
 })
forester/tests/e2e_test.rs (1)

244-255: Looks good; consider Default update (and optional env override hook).

Keep None for current behavior; use Default to future‑proof. If later testing gRPC, you can read an env var to pass Some(port).

 init(Some(LightValidatorConfig {
   enable_indexer: true,
   enable_prover: false,
   wait_time: 60,
   sbf_programs: vec![(
     "FNt7byTHev1k5x2cXZLBr8TdWiC3zoP5vcnZR4P682Uy".to_string(),
     "../target/deploy/create_address_test_program.so".to_string(),
   )],
-  limit_ledger_size: None,
-  grpc_port: None,
+  ..Default::default()
 }))
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 5e85e0d and 331aad5.

📒 Files selected for processing (7)
  • forester/tests/e2e_test.rs (1 hunks)
  • forester/tests/test_batch_append_spent.rs (1 hunks)
  • program-tests/compressed-token-test/tests/v1.rs (1 hunks)
  • program-tests/system-cpi-v2-test/tests/event.rs (1 hunks)
  • sdk-libs/client/src/indexer/photon_indexer.rs (3 hunks)
  • sdk-libs/program-test/src/indexer/test_indexer.rs (9 hunks)
  • sdk-tests/client-test/tests/light_client.rs (1 hunks)
🧰 Additional context used
🧠 Learnings (7)
📚 Learning: 2025-10-16T06:33:19.426Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: program-libs/compressible/CLAUDE.md:0-0
Timestamp: 2025-10-16T06:33:19.426Z
Learning: Applies to program-libs/compressible/src/error.rs : Define error types with numeric codes in the 19xxx range and propagate hasher errors in the 7xxx range; include ProgramError conversions (Anchor, Pinocchio, Solana) in src/error.rs

Applied to files:

  • sdk-libs/client/src/indexer/photon_indexer.rs
📚 Learning: 2025-10-11T21:59:25.222Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: program-libs/account-checks/CLAUDE.md:0-0
Timestamp: 2025-10-11T21:59:25.222Z
Learning: Applies to program-libs/account-checks/src/error.rs : Maintain stable mapping of AccountError to ProgramError, including Pinocchio code mapping (1–11), in error.rs

Applied to files:

  • sdk-libs/client/src/indexer/photon_indexer.rs
📚 Learning: 2025-10-11T21:59:25.222Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: program-libs/account-checks/CLAUDE.md:0-0
Timestamp: 2025-10-11T21:59:25.222Z
Learning: Applies to program-libs/account-checks/src/account_iterator.rs : Use AccountIterator for sequential account retrieval to get precise file:line:column error locations; avoid manual index handling

Applied to files:

  • sdk-libs/client/src/indexer/photon_indexer.rs
  • sdk-libs/program-test/src/indexer/test_indexer.rs
📚 Learning: 2025-10-11T21:59:25.222Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: program-libs/account-checks/CLAUDE.md:0-0
Timestamp: 2025-10-11T21:59:25.222Z
Learning: Applies to program-libs/account-checks/src/**/*.rs : Return AccountError variants (codes 12006–12021) and rely on automatic ProgramError conversions; avoid returning raw ProgramError directly

Applied to files:

  • sdk-libs/client/src/indexer/photon_indexer.rs
📚 Learning: 2025-10-11T21:59:52.712Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: program-libs/account-checks/docs/CLAUDE.md:0-0
Timestamp: 2025-10-11T21:59:52.712Z
Learning: Applies to program-libs/account-checks/docs/**/ERRORS.md : ERRORS.md must document error types with numeric codes in the 12006–12021 range, common causes, resolution strategies, and conversion mappings for Solana and Pinocchio

Applied to files:

  • sdk-libs/client/src/indexer/photon_indexer.rs
📚 Learning: 2025-10-16T06:33:19.426Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: program-libs/compressible/CLAUDE.md:0-0
Timestamp: 2025-10-16T06:33:19.426Z
Learning: Applies to program-libs/compressible/src/config.rs : Ensure serialization compatibility across Anchor, Pinocchio, and Borsh for core account types used by dependent programs

Applied to files:

  • sdk-libs/client/src/indexer/photon_indexer.rs
📚 Learning: 2025-10-15T03:46:26.767Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: programs/registry/CLAUDE.md:0-0
Timestamp: 2025-10-15T03:46:26.767Z
Learning: Applies to programs/registry/src/lib.rs : Load accounts according to type before check_forester: batched via BatchedMerkleTreeAccount::type_from_account_info(); regular via ctx.accounts.account.load()?.metadata; use custom deserialization when required.

Applied to files:

  • sdk-libs/client/src/indexer/photon_indexer.rs
🧬 Code graph analysis (2)
sdk-libs/client/src/indexer/photon_indexer.rs (2)
sdk-libs/client/src/indexer/error.rs (2)
  • from (92-103)
  • from (107-109)
sdk-libs/client/src/indexer/base58.rs (2)
  • from_base58 (8-10)
  • from_base58 (22-24)
sdk-libs/program-test/src/indexer/test_indexer.rs (5)
program-libs/compressed-account/src/pubkey.rs (1)
  • new_from_array (79-81)
sdk-libs/photon-api/src/models/_get_queue_elements_post_200_response_result.rs (1)
  • new (48-56)
sdk-libs/photon-api/src/models/_get_queue_elements_post_request_params.rs (1)
  • new (49-57)
program-tests/utils/src/e2e_test_env.rs (2)
  • new (343-417)
  • addresses (757-757)
program-tests/utils/src/test_batch_forester.rs (1)
  • addresses (664-670)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (18)
  • GitHub Check: system-programs (native, ["cargo-test-sbf -p sdk-native-test", "cargo-test-sbf -p sdk-v1-native-t...
  • GitHub Check: system-programs (token test, ["cargo-test-sbf -p sdk-token-test"])
  • GitHub Check: system-programs (sdk-libs, light-sdk-macros light-sdk light-program-test light-client light-compr...
  • GitHub Check: system-programs (anchor & pinocchio, ["cargo-test-sbf -p sdk-anchor-test", "cargo-test-sbf -p sdk...
  • GitHub Check: cli-v2
  • GitHub Check: lint
  • GitHub Check: cli-v1
  • GitHub Check: programs (system-cpi-test, ["cargo-test-sbf -p system-cpi-test", "cargo test -p light-system-prog...
  • GitHub Check: programs (account-compression-and-registry, ["cargo-test-sbf -p account-compression-test", "cargo...
  • GitHub Check: programs (light-system-program-compression, ["cargo-test-sbf -p system-test -- test_with_compress...
  • GitHub Check: programs (system-cpi-test-v2-functional-account-infos, ["cargo-test-sbf -p system-cpi-v2-test -- ...
  • GitHub Check: programs (compressed-token-batched-tree, ["cargo-test-sbf -p compressed-token-test -- test_transf...
  • GitHub Check: programs (compressed-token-and-e2e, ["cargo-test-sbf -p compressed-token-test --test v1", "cargo-...
  • GitHub Check: programs (system-cpi-test-v2-functional-read-only, ["cargo-test-sbf -p system-cpi-v2-test -- func...
  • GitHub Check: programs (light-system-program-address, ["cargo-test-sbf -p system-test -- test_with_address", "c...
  • GitHub Check: stateless-js-v2
  • GitHub Check: stateless-js-v1
  • GitHub Check: Forester e2e test
🔇 Additional comments (1)
sdk-tests/client-test/tests/light_client.rs (1)

59-59: LGTM! Default gRPC port behavior maintained.

The addition of grpc_port: None correctly initializes the new optional field while preserving the existing default behavior for this test.

@sergeytimoshin sergeytimoshin force-pushed the sergey/get_queue_elements_for_2_queues branch from 331aad5 to d877458 Compare November 8, 2025 13:13
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
cli/src/utils/initTestEnv.ts (1)

130-137: Remove duplicate grpcPort argument at line 136.

The startIndexer function accepts 5 parameters: rpcUrl, indexerPort, grpcPort, checkPhotonVersion, and photonDatabaseUrl. The call at lines 130–137 passes grpcPort twice (lines 133 and 136), which is a copy-paste error. Remove the second occurrence at line 136.

♻️ Duplicate comments (1)
sdk-libs/program-test/src/indexer/test_indexer.rs (1)

645-652: Guard against out-of-bounds queue slicing

When a caller requests a page past the available elements (e.g., start = len + 1 after the queue drains), the current queue[start..end] slices panic because start > end. That makes the test indexer crash instead of returning an empty page, so pagination past the tail is impossible. Please gate all three branches (address output queue, state input queue, state output queue) with a length check that yields an empty Vec when start >= len before slicing.

-                    let end = std::cmp::min(
-                        start + limit as usize,
-                        address_tree_bundle.queue_elements.len(),
-                    );
-                    let queue_elements = address_tree_bundle.queue_elements[start..end].to_vec();
+                    let queue_len = address_tree_bundle.queue_elements.len();
+                    let queue_elements = if start >= queue_len {
+                        Vec::new()
+                    } else {
+                        let end = std::cmp::min(start + limit as usize, queue_len);
+                        address_tree_bundle.queue_elements[start..end].to_vec()
+                    };
...
-                    let end = std::cmp::min(
-                        start + limit as usize,
-                        state_tree_bundle.input_leaf_indices.len(),
-                    );
-                    let queue_elements = state_tree_bundle.input_leaf_indices[start..end].to_vec();
+                    let queue_len = state_tree_bundle.input_leaf_indices.len();
+                    let queue_elements = if start >= queue_len {
+                        Vec::new()
+                    } else {
+                        let end = std::cmp::min(start + limit as usize, queue_len);
+                        state_tree_bundle.input_leaf_indices[start..end].to_vec()
+                    };
...
-                    let end = std::cmp::min(
-                        start + limit as usize,
-                        state_tree_bundle.output_queue_elements.len(),
-                    );
-                    let queue_elements =
-                        state_tree_bundle.output_queue_elements[start..end].to_vec();
+                    let queue_len = state_tree_bundle.output_queue_elements.len();
+                    let queue_elements = if start >= queue_len {
+                        Vec::new()
+                    } else {
+                        let end = std::cmp::min(start + limit as usize, queue_len);
+                        state_tree_bundle.output_queue_elements[start..end].to_vec()
+                    };

Also applies to: 701-707, 768-775

🧹 Nitpick comments (1)
program-tests/utils/src/test_batch_forester.rs (1)

664-673: Response handling correctly updated, but consider cleaning up commented code.

The change from value.elements to value.output_queue_elements.unwrap_or_default() correctly handles the new response structure with separate queue fields.

However, lines 672-673 contain commented-out hash chain assertion code. Consider either:

  • Removing it if the assertion is no longer needed
  • Re-enabling it if it provides valuable test coverage

Apply this diff if you want to remove the commented code:

     let addresses = addresses
         .value
         .output_queue_elements
         .unwrap_or_default()
         .iter()
         .map(|x| x.account_hash)
         .collect::<Vec<[u8; 32]>>();
-    // // local_leaves_hash_chain is only used for a test assertion.
-    // let local_nullifier_hash_chain = create_hash_chain_from_slice(addresses.as_slice()).unwrap();
-    // assert_eq!(leaves_hash_chain, local_nullifier_hash_chain);
     let start_index = merkle_tree.next_index as usize;
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 331aad5 and d877458.

📒 Files selected for processing (16)
  • cli/src/commands/test-validator/index.ts (1 hunks)
  • cli/src/utils/initTestEnv.ts (1 hunks)
  • cli/src/utils/process.ts (1 hunks)
  • cli/src/utils/processPhotonIndexer.ts (2 hunks)
  • forester-utils/src/instructions/state_batch_append.rs (1 hunks)
  • forester-utils/src/instructions/state_batch_nullify.rs (1 hunks)
  • program-tests/utils/src/e2e_test_env.rs (1 hunks)
  • program-tests/utils/src/test_batch_forester.rs (1 hunks)
  • sdk-libs/client/src/indexer/indexer_trait.rs (1 hunks)
  • sdk-libs/client/src/indexer/photon_indexer.rs (3 hunks)
  • sdk-libs/client/src/indexer/types.rs (1 hunks)
  • sdk-libs/client/src/rpc/indexer.rs (2 hunks)
  • sdk-libs/photon-api/src/models/_get_queue_elements_post_200_response_result.rs (1 hunks)
  • sdk-libs/photon-api/src/models/_get_queue_elements_post_request_params.rs (1 hunks)
  • sdk-libs/program-test/src/indexer/test_indexer.rs (9 hunks)
  • sdk-libs/program-test/src/program_test/indexer.rs (2 hunks)
🚧 Files skipped from review as they are similar to previous changes (8)
  • forester-utils/src/instructions/state_batch_nullify.rs
  • cli/src/utils/processPhotonIndexer.ts
  • program-tests/utils/src/e2e_test_env.rs
  • cli/src/utils/process.ts
  • sdk-libs/client/src/rpc/indexer.rs
  • forester-utils/src/instructions/state_batch_append.rs
  • sdk-libs/client/src/indexer/types.rs
  • cli/src/commands/test-validator/index.ts
🧰 Additional context used
🧠 Learnings (13)
📚 Learning: 2025-10-15T03:46:26.767Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: programs/registry/CLAUDE.md:0-0
Timestamp: 2025-10-15T03:46:26.767Z
Learning: Applies to programs/registry/src/lib.rs : Load accounts according to type before check_forester: batched via BatchedMerkleTreeAccount::type_from_account_info(); regular via ctx.accounts.account.load()?.metadata; use custom deserialization when required.

Applied to files:

  • program-tests/utils/src/test_batch_forester.rs
  • sdk-libs/program-test/src/program_test/indexer.rs
  • sdk-libs/client/src/indexer/photon_indexer.rs
📚 Learning: 2025-10-15T03:46:26.767Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: programs/registry/CLAUDE.md:0-0
Timestamp: 2025-10-15T03:46:26.767Z
Learning: Applies to programs/registry/src/lib.rs : Instruction handlers must compute work_units by operation type: batch operations use account.queue_batches.batch_size; single operations use DEFAULT_WORK_V1; custom operations compute based on complexity.

Applied to files:

  • program-tests/utils/src/test_batch_forester.rs
📚 Learning: 2025-10-16T06:33:19.426Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: program-libs/compressible/CLAUDE.md:0-0
Timestamp: 2025-10-16T06:33:19.426Z
Learning: Applies to program-libs/compressible/src/config.rs : Ensure serialization compatibility across Anchor, Pinocchio, and Borsh for core account types used by dependent programs

Applied to files:

  • program-tests/utils/src/test_batch_forester.rs
  • sdk-libs/program-test/src/program_test/indexer.rs
  • sdk-libs/client/src/indexer/photon_indexer.rs
📚 Learning: 2025-10-11T21:59:25.222Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: program-libs/account-checks/CLAUDE.md:0-0
Timestamp: 2025-10-11T21:59:25.222Z
Learning: Applies to program-libs/account-checks/src/account_info/{solana.rs,pinocchio.rs,test_account_info.rs} : Gate SDK-specific implementations with #[cfg(feature = "solana"|"pinocchio"|"test-only")]

Applied to files:

  • sdk-libs/program-test/src/program_test/indexer.rs
📚 Learning: 2025-10-11T21:59:25.222Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: program-libs/account-checks/CLAUDE.md:0-0
Timestamp: 2025-10-11T21:59:25.222Z
Learning: Applies to program-libs/account-checks/src/account_info/{solana.rs,pinocchio.rs,test_account_info.rs} : Provide SDK-specific AccountInfoTrait implementations in account_info/{solana.rs,pinocchio.rs,test_account_info.rs}

Applied to files:

  • sdk-libs/program-test/src/program_test/indexer.rs
📚 Learning: 2025-10-11T21:59:25.222Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: program-libs/account-checks/CLAUDE.md:0-0
Timestamp: 2025-10-11T21:59:25.222Z
Learning: Applies to program-libs/account-checks/src/account_info/account_info_trait.rs : Ensure the crate compiles with no features enabled by keeping trait definitions free of SDK-specific dependencies

Applied to files:

  • sdk-libs/program-test/src/program_test/indexer.rs
📚 Learning: 2025-10-11T21:59:25.222Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: program-libs/account-checks/CLAUDE.md:0-0
Timestamp: 2025-10-11T21:59:25.222Z
Learning: Applies to program-libs/account-checks/src/checks.rs : Expose and maintain account validation helpers (check_owner, check_program, check_mut/non_mut, check_signer, check_discriminator, set_discriminator, check_pda_seeds, check_account_balance_is_rent_exempt, account_info_init) in checks.rs

Applied to files:

  • sdk-libs/program-test/src/program_test/indexer.rs
📚 Learning: 2025-10-11T21:59:25.222Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: program-libs/account-checks/CLAUDE.md:0-0
Timestamp: 2025-10-11T21:59:25.222Z
Learning: Applies to program-libs/account-checks/src/**/*.rs : Return AccountError variants (codes 12006–12021) and rely on automatic ProgramError conversions; avoid returning raw ProgramError directly

Applied to files:

  • sdk-libs/program-test/src/program_test/indexer.rs
  • sdk-libs/client/src/indexer/photon_indexer.rs
📚 Learning: 2025-10-11T21:59:25.222Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: program-libs/account-checks/CLAUDE.md:0-0
Timestamp: 2025-10-11T21:59:25.222Z
Learning: Applies to program-libs/account-checks/src/!(account_info)/**/*.rs : Use AccountInfoTrait for runtime-agnostic account handling; avoid direct solana-program or pinocchio AccountInfo in general logic

Applied to files:

  • sdk-libs/program-test/src/program_test/indexer.rs
📚 Learning: 2025-10-11T21:59:25.222Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: program-libs/account-checks/CLAUDE.md:0-0
Timestamp: 2025-10-11T21:59:25.222Z
Learning: Applies to program-libs/account-checks/src/account_iterator.rs : Use AccountIterator for sequential account retrieval to get precise file:line:column error locations; avoid manual index handling

Applied to files:

  • sdk-libs/program-test/src/indexer/test_indexer.rs
  • sdk-libs/client/src/indexer/photon_indexer.rs
📚 Learning: 2025-10-16T06:33:19.426Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: program-libs/compressible/CLAUDE.md:0-0
Timestamp: 2025-10-16T06:33:19.426Z
Learning: Applies to program-libs/compressible/src/error.rs : Define error types with numeric codes in the 19xxx range and propagate hasher errors in the 7xxx range; include ProgramError conversions (Anchor, Pinocchio, Solana) in src/error.rs

Applied to files:

  • sdk-libs/client/src/indexer/photon_indexer.rs
📚 Learning: 2025-10-11T21:59:25.222Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: program-libs/account-checks/CLAUDE.md:0-0
Timestamp: 2025-10-11T21:59:25.222Z
Learning: Applies to program-libs/account-checks/src/error.rs : Maintain stable mapping of AccountError to ProgramError, including Pinocchio code mapping (1–11), in error.rs

Applied to files:

  • sdk-libs/client/src/indexer/photon_indexer.rs
📚 Learning: 2025-10-11T21:59:52.712Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: program-libs/account-checks/docs/CLAUDE.md:0-0
Timestamp: 2025-10-11T21:59:52.712Z
Learning: Applies to program-libs/account-checks/docs/**/ERRORS.md : ERRORS.md must document error types with numeric codes in the 12006–12021 range, common causes, resolution strategies, and conversion mappings for Solana and Pinocchio

Applied to files:

  • sdk-libs/client/src/indexer/photon_indexer.rs
🧬 Code graph analysis (5)
program-tests/utils/src/test_batch_forester.rs (4)
program-tests/utils/src/e2e_test_env.rs (1)
  • addresses (757-757)
forester/tests/e2e_test.rs (1)
  • addresses (1326-1332)
program-tests/system-cpi-test/tests/test.rs (1)
  • addresses (1768-1775)
program-tests/system-cpi-v2-test/tests/invoke_cpi_with_read_only.rs (2)
  • addresses (269-276)
  • addresses (571-578)
sdk-libs/program-test/src/indexer/test_indexer.rs (4)
program-tests/utils/src/e2e_test_env.rs (2)
  • new (343-417)
  • addresses (757-757)
sdk-libs/photon-api/src/models/_get_queue_elements_post_200_response_result.rs (1)
  • new (48-56)
sdk-libs/photon-api/src/models/_get_queue_elements_post_request_params.rs (1)
  • new (49-57)
program-tests/utils/src/test_batch_forester.rs (1)
  • addresses (664-670)
sdk-libs/client/src/indexer/indexer_trait.rs (4)
sdk-libs/client/src/indexer/photon_indexer.rs (1)
  • get_queue_elements (1580-1738)
sdk-libs/program-test/src/indexer/test_indexer.rs (1)
  • get_queue_elements (618-868)
sdk-libs/client/src/rpc/indexer.rs (1)
  • get_queue_elements (204-226)
sdk-libs/program-test/src/program_test/indexer.rs (1)
  • get_queue_elements (200-222)
sdk-libs/photon-api/src/models/_get_queue_elements_post_200_response_result.rs (1)
sdk-libs/photon-api/src/models/_get_queue_elements_post_request_params.rs (1)
  • new (49-57)
sdk-libs/client/src/indexer/photon_indexer.rs (1)
sdk-libs/client/src/indexer/base58.rs (2)
  • from_base58 (8-10)
  • from_base58 (22-24)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (18)
  • GitHub Check: Forester e2e test
  • GitHub Check: stateless-js-v2
  • GitHub Check: cli-v2
  • GitHub Check: cli-v1
  • GitHub Check: stateless-js-v1
  • GitHub Check: programs (compressed-token-batched-tree, ["cargo-test-sbf -p compressed-token-test -- test_transf...
  • GitHub Check: programs (system-cpi-test, ["cargo-test-sbf -p system-cpi-test", "cargo test -p light-system-prog...
  • GitHub Check: programs (system-cpi-test-v2-functional-account-infos, ["cargo-test-sbf -p system-cpi-v2-test -- ...
  • GitHub Check: programs (system-cpi-test-v2-functional-read-only, ["cargo-test-sbf -p system-cpi-v2-test -- func...
  • GitHub Check: programs (light-system-program-address, ["cargo-test-sbf -p system-test -- test_with_address", "c...
  • GitHub Check: programs (compressed-token-and-e2e, ["cargo-test-sbf -p compressed-token-test --test v1", "cargo-...
  • GitHub Check: programs (light-system-program-compression, ["cargo-test-sbf -p system-test -- test_with_compress...
  • GitHub Check: programs (account-compression-and-registry, ["cargo-test-sbf -p account-compression-test", "cargo...
  • GitHub Check: system-programs (anchor & pinocchio, ["cargo-test-sbf -p sdk-anchor-test", "cargo-test-sbf -p sdk...
  • GitHub Check: system-programs (native, ["cargo-test-sbf -p sdk-native-test", "cargo-test-sbf -p sdk-v1-native-t...
  • GitHub Check: system-programs (token test, ["cargo-test-sbf -p sdk-token-test"])
  • GitHub Check: system-programs (sdk-libs, light-sdk-macros light-sdk light-program-test light-client light-compr...
  • GitHub Check: Analyze (go)
🔇 Additional comments (3)
program-tests/utils/src/test_batch_forester.rs (1)

653-663: LGTM! API call correctly updated for new queue parameter structure.

The updated get_queue_elements call appropriately requests elements from the output queue by passing None for the output queue start boundary and Some(batch.zkp_batch_size as u16) as the limit, while passing None for input queue parameters. This aligns with the address tree batch update operation.

cli/src/utils/initTestEnv.ts (1)

82-112: LGTM: Parameter addition looks correct.

The grpcPort parameter is properly added with an appropriate default value and type annotation.

sdk-libs/program-test/src/program_test/indexer.rs (1)

200-222: Verification complete: all call sites have been successfully updated to the new signature.

All 7 call sites found (2 trait implementations + 5 actual usage sites) are using the new 6-parameter signature correctly:

  • test_indexer.rs trait impl and calls ✓
  • test_batch_forester.rs ✓
  • e2e_test_env.rs ✓
  • state_batch_nullify.rs ✓
  • state_batch_append.rs ✓
  • client indexer.rs trait impl and delegation ✓

The refactoring from enum-based queue selection to explicit optional parameters for output and input queues is complete and consistent across the codebase.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (2)
forester/src/config.rs (2)

86-87: Consider adding documentation for the new timing fields.

The new sleep_after_processing_ms and sleep_when_idle_ms fields would benefit from doc comments explaining their purpose and units (milliseconds). For example:

/// Duration in milliseconds to sleep after successfully processing items
pub sleep_after_processing_ms: u64,
/// Duration in milliseconds to sleep when no work is available
pub sleep_when_idle_ms: u64,

287-288: Sleep timings initialized consistently across configuration paths.

Both new_for_start() and new_for_status() correctly initialize the sleep timing fields with production defaults (10_000ms/45_000ms).

Note: These values are currently hardcoded and not exposed as CLI arguments. If users need to tune these timings in production, consider adding optional CLI arguments in StartArgs with these values as defaults.

Also applies to: 345-346

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 023568c and e78a060.

📒 Files selected for processing (5)
  • forester/src/config.rs (6 hunks)
  • forester/src/epoch_manager.rs (2 hunks)
  • forester/tests/e2e_test.rs (1 hunks)
  • forester/tests/legacy/test_utils.rs (1 hunks)
  • forester/tests/test_utils.rs (1 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-10-16T06:33:19.426Z
Learnt from: CR
Repo: Lightprotocol/light-protocol PR: 0
File: program-libs/compressible/CLAUDE.md:0-0
Timestamp: 2025-10-16T06:33:19.426Z
Learning: Applies to program-libs/compressible/src/config.rs : Provide default initialization for the CToken V1 config in CompressibleConfig

Applied to files:

  • forester/src/config.rs
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (18)
  • GitHub Check: programs (system-cpi-test-v2-functional-account-infos, ["cargo-test-sbf -p system-cpi-v2-test -- ...
  • GitHub Check: programs (system-cpi-test-v2-functional-read-only, ["cargo-test-sbf -p system-cpi-v2-test -- func...
  • GitHub Check: programs (system-cpi-test, ["cargo-test-sbf -p system-cpi-test", "cargo test -p light-system-prog...
  • GitHub Check: programs (compressed-token-and-e2e, ["cargo-test-sbf -p compressed-token-test --test v1", "cargo-...
  • GitHub Check: programs (compressed-token-batched-tree, ["cargo-test-sbf -p compressed-token-test -- test_transf...
  • GitHub Check: programs (light-system-program-compression, ["cargo-test-sbf -p system-test -- test_with_compress...
  • GitHub Check: programs (account-compression-and-registry, ["cargo-test-sbf -p account-compression-test", "cargo...
  • GitHub Check: programs (light-system-program-address, ["cargo-test-sbf -p system-test -- test_with_address", "c...
  • GitHub Check: stateless-js-v2
  • GitHub Check: cli-v1
  • GitHub Check: system-programs (sdk-libs, light-sdk-macros light-sdk light-program-test light-client light-compr...
  • GitHub Check: system-programs (native, ["cargo-test-sbf -p sdk-native-test", "cargo-test-sbf -p sdk-v1-native-t...
  • GitHub Check: system-programs (token test, ["cargo-test-sbf -p sdk-token-test"])
  • GitHub Check: system-programs (anchor & pinocchio, ["cargo-test-sbf -p sdk-anchor-test", "cargo-test-sbf -p sdk...
  • GitHub Check: cli-v2
  • GitHub Check: stateless-js-v1
  • GitHub Check: Forester e2e test
  • GitHub Check: lint
🔇 Additional comments (6)
forester/tests/test_utils.rs (1)

112-113: LGTM! Test-appropriate sleep timings configured.

The test configuration correctly initializes the new sleep timing fields with reduced values (50ms after processing, 100ms when idle) compared to production defaults (10s/45s), which will make tests run faster while still validating the sleep behavior.

forester/tests/e2e_test.rs (1)

234-235: LGTM! E2E test timing configured appropriately.

The configuration correctly sets reduced sleep durations for the end-to-end test, matching the values used in other test utilities (50ms/100ms). This ensures consistent test behavior across the test suite.

forester/src/config.rs (1)

101-102: Well-structured timing defaults for production and tests.

The implementation correctly provides:

  • Production-appropriate defaults (10s after processing, 45s when idle) for conservative resource usage
  • Test-appropriate values (50ms/100ms) in test variants for faster test execution

This separation ensures tests run quickly while production maintains reasonable polling intervals.

Also applies to: 118-119, 133-134

forester/tests/legacy/test_utils.rs (1)

98-99: LGTM! Legacy test configuration updated correctly.

The legacy test utilities correctly initialize the new sleep timing fields with test-appropriate values (50ms/100ms), consistent with the other test configurations in the codebase.

forester/src/epoch_manager.rs (2)

1331-1335: Excellent refactor: Hardcoded sleep durations now configurable.

This change successfully replaces hardcoded sleep values with config-driven durations, allowing:

  • Different sleep times for production (10s/45s) vs tests (50ms/100ms)
  • Potential future tuning without code changes
  • Clear distinction between post-processing sleep and idle sleep

The implementation correctly uses self.config.general_config.sleep_after_processing_ms when items were processed and self.config.general_config.sleep_when_idle_ms when idle, maintaining the original intent while improving configurability.


2182-2183: Test configuration properly includes new timing fields.

The test helper function correctly initializes the sleep timing fields with test-appropriate values, ensuring tests that use this configuration will have the expected fast sleep behavior.

@ananas-block ananas-block merged commit 1add664 into main Nov 10, 2025
41 checks passed
@ananas-block ananas-block deleted the sergey/get_queue_elements_for_2_queues branch November 10, 2025 16:27
@coderabbitai coderabbitai bot mentioned this pull request Jan 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants