Open
Conversation
Phase 2 of StoragePrograms feature implementation: - Create GCR_StorageProgram TypeORM entity with full field support - Add indexes for owner, programName, encoding, storageLocation - Support JSON and Binary encoding via 'encoding' field - Include robust ACL (mode, allowed, blacklisted, groups) as jsonb - Add IPFS stubs (storageLocation, ipfsCid) for future hybrid storage - Track fees paid (totalFeesPaid) and soft delete (isDeleted) - Update datasource.ts to register the new entity - Update to demosdk 2.8.11 with unified StorageProgram types Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Phase 3 of StoragePrograms feature implementation: - Add GCRStorageProgramRoutines with validation and fee calculation - validateStorageProgramPayload() for confirm flow validation - GCRStorageProgramRoutines.apply() for broadcast flow GCREdit handling - Support CREATE, WRITE, UPDATE_ACL, DELETE operations - Integrate with handleGCR.ts routing and repository management - Follow SDK GCREditStorageProgram structure (target, context.operation, context.data) Fee calculation: 1 DEM per 10KB chunk (minimum 1 DEM) ACL validation: mode, allowed, blacklisted, groups with permissions Soft delete pattern with isDeleted flag preservation Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Phase 5 of StoragePrograms implementation: - Add GET /storage-program/:address endpoint for reading by address - Add GET /storage-program/owner/:owner endpoint for listing by owner - Implement checkReadPermission in GCRStorageProgramRoutines - ACL check enforces: public (allow all), owner (only owner), restricted (allowed/groups) - Proper error handling: NOT_FOUND, PERMISSION_DENIED, INTERNAL_ERROR - Update SDK to 2.8.13 for HandleStorageProgramOperations support Related: DEM-548 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add full transaction flow diagram (confirm/broadcast) - Include examples for all CRUD operations - Document ACL helper methods and validation utilities - Add complete working example with error handling Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add missing owner field to CREATE_STORAGE_PROGRAM spec docs - Add INVALID_REQUEST error code for malformed requests (routes.ts) - Remove READ_STORAGE from valid transaction operations (reads are RPC-only) - Add write permission check in handleWriteStorage (was missing ACL enforcement) - Fix ACL priority: owner check now before blacklist (owner cannot be blacklisted) - Add checkWritePermission helper for ACL group/public mode validation Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix logging inconsistency: rename chunks variable to rawChunks for raw value, use chunks for effective value (min 1) ensuring log output matches actual fee calculation - Add null/object type guard in ACL group validation to prevent runtime errors from malformed ACL input Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…ctions
StorageProgram transactions use stor-{40 hex chars} format for the 'to'
field instead of Ed25519 public keys. Added validateStorageAddress()
method and modified structured() to route validation based on tx type.
This fixes the TypeError when confirming storage transactions where
validateToField() expected 32-byte Ed25519 keys but received stor- addresses.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add searchStorageProgramsByName to GCRStorageProgramRoutines - Support exact match and ILIKE partial matching - Add pagination support (limit, offset) - Add /storage-program/search?q=name endpoint - ACL filtering applied to search results Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Modified HandleGCR.apply() to update sender's assignedTxs on success - Added addAssignedTx() helper with duplicate prevention - Non-blocking: assignedTxs update failure doesn't fail the operation Relates to: DEM-549 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add field-level read/write operations for storage programs instead of full blob operations. This enables efficient granular access to storage program data. Read methods (manageNodeCall.ts): - getStorageProgramFields - list top-level field names - getStorageProgramValue - get specific field value - getStorageProgramItem - get array element by index - hasStorageProgramField - check field existence - getStorageProgramFieldType - get field type info - getStorageProgramAll - full data (retrocompat) Write routines (GCRStorageProgramRoutines.ts): - SET_FIELD - set/create field value - SET_ITEM - set array element at index - APPEND_ITEM - push to array - DELETE_FIELD - remove field - DELETE_ITEM - remove array element Features: - ACL enforcement on all operations - Fee calculation based on size delta - Binary encoding detection (error for granular) - Bounds checking for array operations Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add detailed specifications for Storage Program granular operations: - 03-operations.mdx: Add GRANULAR_WRITE operation with 5 operation types (SET_FIELD, SET_ITEM, APPEND_ITEM, DELETE_FIELD, DELETE_ITEM), GranularWriteOperation interface, and comparison table vs WRITE_STORAGE - 05-rpc-endpoints.mdx: Add all granular read endpoints (/fields, /field/:field, /field/:field/item/:index, /has/:field, /type/:field, /all) with request/ response examples and error codes Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add Serena session memories for storage program API work - Add key derivation specification document
…iles and the `run` script, modify `.beads` files and `.gitignore`, and create `AGENTS.md`.
- Implement stress tests for concurrent operations and payload boundaries in `stress.test.ts`. - Add comprehensive validation tests for storage program payloads in `validation.test.ts`, covering operation validation, data size, JSON nesting depth, and ACL structure. - Mock necessary dependencies to isolate tests and ensure reliability.
Contributor
|
Important Review skippedToo many files! This PR contains 280 files, which is 130 over the limit of 150. ⛔ Files ignored due to path filters (6)
📒 Files selected for processing (280)
You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
ⓘ You are approaching your monthly quota for Qodo. Upgrade your plan Review Summary by QodoStorage v2 - StorageProgram GCR routines with comprehensive testing and code formatting improvements
WalkthroughsDescription• Implemented comprehensive GCRStorageProgramRoutines module with 1685 lines supporting CREATE, WRITE, UPDATE_ACCESS_CONTROL, DELETE operations and granular field operations (SET_FIELD, SET_ITEM, APPEND_ITEM, DELETE_FIELD, DELETE_ITEM) • Added fee calculation system based on data size with 10KB chunk pricing (1 DEM per chunk) • Implemented ACL (Access Control List) validation and permission checking with support for owner, public, and restricted modes • Added comprehensive test suite with 1000+ lines covering all storage program operations, permission validation, data size limits, and edge cases • Added stress test suite for concurrent operations and boundary testing (payload size limits, nesting depth limits, transaction capping) • Fixed bugs including data validation in simulate mode (Bug #2) and interactionTxs capping (Bug #3) • Applied consistent code formatting across multiple files (4-space indentation, improved line wrapping, consistent import/method signature formatting) • Updated version from 0.47.0 to 0.49.6 Diagramflowchart LR
A["StorageProgram<br/>Operations"] -->|CREATE, WRITE,<br/>UPDATE_ACCESS_CONTROL,<br/>DELETE| B["GCRStorageProgramRoutines"]
B -->|Field Operations| C["SET_FIELD, SET_ITEM,<br/>APPEND_ITEM,<br/>DELETE_FIELD, DELETE_ITEM"]
B -->|Fee Calculation| D["Data Size Based<br/>Pricing<br/>1 DEM per 10KB"]
B -->|ACL Validation| E["Permission Checking<br/>Owner, Public,<br/>Restricted Modes"]
B -->|Tested By| F["Comprehensive<br/>Test Suite<br/>1000+ lines"]
B -->|Stress Tested| G["Concurrent Operations<br/>Boundary Testing<br/>Payload & Nesting Limits"]
File Changes |
Contributor
Code Review by Qodo
1. Storage routes return error.message
|
![qodo-code-review[bot]](https://avatars.githubusercontent.com/in/484649?s=60&v=4){kind=small}
Comment on lines
+252
to
+255
| error: | ||
| error instanceof Error | ||
| ? error.message | ||
| : "Internal server error", |
Contributor
There was a problem hiding this comment.
1. Storage routes return error.message 📘 Rule violation ⛨ Security
StorageProgram route handlers return error.message directly in HTTP JSON responses, potentially exposing internal implementation details to callers. This violates secure error handling expectations for user-facing APIs.
Agent Prompt
## Issue description
Several StorageProgram endpoints return `error.message` in API responses, which can leak internal exception details.
## Issue Context
User-facing errors should be generic (e.g., "Internal server error") while detailed error data is written to internal logs only.
## Fix Focus Areas
- src/features/storageprogram/routes.ts[248-258]
- src/features/storageprogram/routes.ts[291-297]
- src/features/storageprogram/routes.ts[359-367]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Comment on lines
+221
to
+232
| process.stdout?.on("data", (data: Buffer) => { | ||
| proxyInfo.lastActivity = Date.now() | ||
| log.debug( | ||
| `[TLSNotary] Proxy ${proxyInfo.domain} stdout: ${data.toString().trim()}`, | ||
| ) | ||
| }) | ||
|
|
||
| process.stderr?.on("data", (data: Buffer) => { | ||
| proxyInfo.lastActivity = Date.now() | ||
| log.debug( | ||
| `[TLSNotary] Proxy ${proxyInfo.domain} stderr: ${data.toString().trim()}`, | ||
| ) |
Contributor
There was a problem hiding this comment.
2. wstcp output logged verbatim 📘 Rule violation ⛨ Security
TLSNotary proxy monitoring logs raw wstcp stdout/stderr contents, which may contain sensitive connection/request data. This creates a high risk of sensitive data leakage into logs.
Agent Prompt
## Issue description
The TLSNotary proxy manager logs raw `wstcp` stdout/stderr, which can leak sensitive information into application logs.
## Issue Context
Secure logging requires preventing any PII/PHI/secrets from appearing in logs. Subprocess output is untrusted and may contain sensitive content.
## Fix Focus Areas
- src/features/tlsnotary/proxyManager.ts[221-232]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Comment on lines
+305
to
+350
| const storageAddress = pathParts[2] | ||
| const field = decodeURIComponent(pathParts[4] || "") | ||
| const requesterAddress = getRequesterAddress(req) | ||
|
|
||
| if (!field) { | ||
| return jsonResponse( | ||
| { | ||
| success: false, | ||
| error: "Field name is required", | ||
| errorCode: "INVALID_REQUEST", | ||
| } satisfies StorageProgramGranularResponse, | ||
| 400, | ||
| ) | ||
| } | ||
|
|
||
| const result = await getAccessibleProgram(storageAddress, requesterAddress) | ||
| if (result.response) { | ||
| return result.response | ||
| } | ||
|
|
||
| const program = result.program as GCRStorageProgram | ||
| if (!program.data || typeof program.data !== "object" || Array.isArray(program.data)) { | ||
| return jsonResponse( | ||
| { | ||
| success: false, | ||
| error: "Field operations are only available for JSON object data", | ||
| errorCode: "INVALID_FIELD_TYPE", | ||
| } satisfies StorageProgramGranularResponse, | ||
| 400, | ||
| ) | ||
| } | ||
|
|
||
| const data = program.data as Record<string, unknown> | ||
| if (!(field in data)) { | ||
| return jsonResponse( | ||
| { | ||
| success: false, | ||
| error: `Field not found: ${field}`, | ||
| errorCode: "FIELD_NOT_FOUND", | ||
| } satisfies StorageProgramGranularResponse, | ||
| 404, | ||
| ) | ||
| } | ||
|
|
||
| const value = data[field] | ||
| const response: StorageProgramGranularResponse = { |
Contributor
There was a problem hiding this comment.
3. Untrusted field indexes object 📘 Rule violation ⛨ Security
StorageProgram granular read endpoints use a URL-derived field to check field in data and then access data[field], which can hit prototype keys like __proto__/constructor. This is unsafe handling of external input and can cause unexpected behavior or security issues.
Agent Prompt
## Issue description
The `field` URL parameter is used directly for `in` checks and object indexing, which can access prototype keys and behaves unsafely for untrusted input.
## Issue Context
External inputs (URL path parts) should be validated/sanitized. For object key access, prefer own-property checks and disallow dangerous keys like `__proto__`, `prototype`, and `constructor`.
## Fix Focus Areas
- src/features/storageprogram/routes.ts[305-350]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Comment on lines
+803
to
+814
| export function registerStorageProgramRoutes(server: BunServer): void { | ||
| // Register specific routes first (more specific paths before wildcards) | ||
| server.get("/storage-program/search/*", searchByNameHandler) | ||
| server.get("/storage-program/search", searchByNameHandler) | ||
| server.get("/storage-program/*/field/*/item/*", getStorageProgramArrayItemHandler) | ||
| server.get("/storage-program/*/field/*", getStorageProgramFieldValueHandler) | ||
| server.get("/storage-program/*/fields", getStorageProgramFieldsHandler) | ||
| server.get("/storage-program/*/has/*", hasStorageProgramFieldHandler) | ||
| server.get("/storage-program/*/type/*", getStorageProgramFieldTypeHandler) | ||
| server.get("/storage-program/*/all", getStorageProgramAllDataHandler) | ||
| server.get("/storage-program/owner/*", listByOwnerHandler) | ||
| server.get("/storage-program/*", getStorageProgramHandler) |
Contributor
There was a problem hiding this comment.
4. Wildcard routes don't match 🐞 Bug ✓ Correctness
StorageProgram routes are registered with wildcard patterns (e.g. "/storage-program/*"), but BunServer resolves handlers via exact Map.get(path) lookup, so dynamic StorageProgram endpoints will 404.
Agent Prompt
## Issue description
StorageProgram routes are registered using wildcard patterns (e.g. `/storage-program/*`), but `BunServer` only performs an exact pathname lookup, so dynamic routes will always return 404.
## Issue Context
The new feature assumes a router with wildcard/param support, but the in-house `BunServer` currently stores routes in a `Map` keyed by exact `url.pathname`.
## Fix Focus Areas
- src/libs/network/bunServer.ts[39-61]
- src/features/storageprogram/routes.ts[803-814]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
|
This PR modifies Files that will be reverted:
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.