Claude/ipfs yellowpaper docs zw ekp

[tcsenpai]

PR Type

Enhancement, Documentation, Bug fix

---

Description

• Complete IPFS integration with core IPFSManager class providing content operations (add, get, pin, unpin), streaming support for large files, swarm management, and public bridge functionality

• IPFS tokenomics system with tier-based pricing (regular: 1GB/1000 pins, genesis: 10GB/10000 pins), pin expiration with duration-based multipliers, and fair pricing via custom_charges

• Account IPFS state management in GCR with concurrency control, quota enforcement, pin expiration cleanup, and comprehensive statistics tracking

• Comprehensive RPC endpoints for IPFS operations: status, add/get/pin/unpin, streaming uploads/downloads, swarm management, public bridge access, cost estimation, and quota checking

• Pin expiration system with background worker service for monitoring and cleanup, grace period enforcement, and batch processing

• Private network support via swarm key utilities and peer discovery through hello_peer protocol with capability exchange

• Public bridge implementation with rate limiting, gateway availability checks, and content publishing to public IPFS network

• TUI enhancements for IPFS status display, peer capabilities visualization, and debounced log updates

• IPFS yellowpaper specification documenting complete architecture, tokenomics, quotas, expiration, security, and all RPC endpoints

• Docker orchestration for IPFS container lifecycle management with port configuration and swarm key setup

• Bug fix: TTY safety checks in index.ts to prevent crashes in Docker/CI environments

---

Diagram Walkthrough

flowchart LR
  A["User Transaction"] -->|"ipfs_add/pin/unpin"| B["IPFSOperations Handler"]
  B -->|"Cost Calculation"| C["IPFSTokenomics"]
  C -->|"Pricing & Validation"| D["Custom Charges"]
  B -->|"State Updates"| E["GCRIPFSRoutines"]
  E -->|"Quota & Expiration"| F["Account IPFS State"]
  G["IPFSManager"] -->|"Content Operations"| H["Kubo HTTP API"]
  H -->|"Streaming"| I["Large Files"]
  G -->|"Swarm Management"| J["Private Network"]
  K["RPC Endpoints"] -->|"NodeCall Handlers"| L["IPFS Operations"]
  L -->|"Manager Access"| G
  M["ExpirationWorker"] -->|"Periodic Cleanup"| F
  N["PeerManager"] -->|"Capability Exchange"| O["hello_peer Protocol"]
  O -->|"Peer Discovery"| J

Loading

File Walkthrough

	Relevant files
Enhancement	52 files IPFSManager.ts Core IPFS Manager with streaming and public bridge support src/features/ipfs/IPFSManager.ts Implements core IPFSManager class wrapping Kubo HTTP API for IPFS operations with health checking and lifecycle management Provides content operations (add, get, pin, unpin, listPins, isPinned, getSize) with comprehensive error handling Adds streaming support for large files via addStream and getStream methods with progress callbacks Implements swarm management (peer connections, bootstrap nodes) and cluster pinning for distributed content replication Includes public bridge operations for fetching/publishing content to public IPFS network with rate limiting Features retry logic with exponential backoff, shared state updates, and verbose logging for debugging +1655/-0 ipfsOperations.ts IPFS transaction handlers with tokenomics integration        src/libs/blockchain/routines/ipfsOperations.ts Implements transaction handlers for IPFS operations (ipfs_add, ipfs_pin, ipfs_unpin, ipfs_extend_pin) Integrates IPFS tokenomics with pricing calculations based on file size and genesis account detection Validates quotas, processes fee payments, and updates account IPFS state with cost tracking Supports pin expiration system with duration-based pricing multipliers and extension functionality Implements fair pricing using custom_charges for cost control and balance validation +857/-0  GCRIPFSRoutines.ts Account IPFS state management with concurrency control      src/libs/blockchain/gcr/gcr_routines/GCRIPFSRoutines.ts Provides static methods for managing IPFS state in user accounts (addPin, removePin, updatePin) Implements per-pubkey locking mechanism to prevent race conditions in concurrent operations Tracks rewards and costs with bigint serialization for account IPFS state Includes pin expiration cleanup and comprehensive statistics retrieval Manages free tier allocation tracking and quota enforcement +526/-0  IPFSTypes.ts IPFS types with quotas and expiration system                          src/model/entities/types/IPFSTypes.ts Defines PinnedContent and AccountIPFSState interfaces for IPFS data in account records Implements consensus-critical quota system with tier-based limits (regular: 1GB/1000 pins, genesis: 10GB/10000 pins) Adds pin expiration system with preset durations and custom duration support with pricing multipliers Provides validation functions for quotas, durations, and expiration checking Includes helper functions for quota checks, duration validation, and pin time remaining calculations +518/-0  ipfsPublicFetch.ts Public IPFS gateway fetch RPC endpoint                                      src/libs/network/routines/nodecalls/ipfs/ipfsPublicFetch.ts Implements RPC endpoint for fetching content from public IPFS gateways Validates public bridge enablement and returns appropriate error codes (403 for disabled, 429 for rate limit) Returns content as base64 or text with source gateway and response time metadata +109/-0  PointSystem.ts Add Nomis reputation scoring integration                                  src/features/incentive/PointSystem.ts Adds linkedNomisIdentities field to account points structure for Nomis reputation scoring integration +2/-0      PeerManager.ts Phase 9 - IPFS peer discovery via hello_peer protocol        src/libs/peer/PeerManager.ts Enhanced peer list format to support capabilities (IPFS info) alongside URLs Added savePeerList() method to persist peer data with capabilities to disk Implemented getOurCapabilities() to advertise node's IPFS capabilities to peers Added isLocalNetwork() helper to determine if peer is on local network for address filtering Updated sayHelloToPeer() to include capabilities in hello_peer handshake Made helloPeerCallback() async and added capability extraction from peer responses +244/-11 types.ts IPFS integration type definitions and configuration            src/features/ipfs/types.ts Comprehensive IPFS type definitions for configuration, node info, and health status Streaming types for large file uploads/downloads with progress callbacks Swarm and cluster types for private IPFS network management Public bridge types for optional public IPFS gateway access Helper functions to load configuration from environment variables Default configuration constants for API, timeouts, and rate limiting +416/-0  ipfsTokenomics.ts IPFS tokenomics and pricing calculations module                    src/libs/blockchain/routines/ipfsTokenomics.ts Pricing configuration for regular and genesis accounts with different rates Genesis account detection with caching to avoid repeated block queries Cost calculation functions for pin operations with free tier support Fee distribution logic (MVP: 100% to host RPC) Validation helpers for balance and transaction amount checks +358/-0  ExpirationWorker.ts IPFS pin expiration background worker service                        src/features/ipfs/ExpirationWorker.ts Background worker service for monitoring and managing expired IPFS pins Periodic scanning of account IPFS states with configurable check intervals Grace period enforcement before actual cleanup of expired pins Batch processing to prevent overwhelming the system Statistics tracking and error handling for expiration operations +363/-0  TUIManager.ts TUI enhancements for IPFS status and peer capabilities display src/utilities/tui/TUIManager.ts Added IPFS status display in right column showing node status, peer ID, and swarm peers Reorganized keyboard shortcuts: IPFS tab on -, DAHR on =, TLSN on backtick Enhanced peers command to display peer capabilities including IPFS multiaddresses Implemented debounced log updates to prevent event loop saturation from rapid logs Added support for connecting/retrying/error states in IPFS status display +145/-19 ipfsAddStream.ts IPFS streaming add endpoint for large file uploads              src/libs/network/routines/nodecalls/ipfs/ipfsAddStream.ts Streaming endpoint for chunked IPFS file uploads with session management Base64-encoded chunk handling with sequential chunk index validation Session ID generation and tracking for multi-request uploads Automatic cleanup of stale upload sessions after 30 minutes Support for single-chunk finalization and multi-chunk streaming +282/-0  swarmKey.ts IPFS swarm key utilities for private network setup              src/features/ipfs/swarmKey.ts Swarm key generation and management for private IPFS network isolation Official Demos network swarm key constant with security documentation Swarm key file format parsing and validation (go-ipfs/kubo format) Environment variable integration for custom swarm keys Private network mode detection and status logging +251/-0  ipfsGetStream.ts IPFS streaming get endpoint for large file downloads          src/libs/network/routines/nodecalls/ipfs/ipfsGetStream.ts Streaming endpoint for chunked IPFS file downloads with session caching Metadata requests returning file size and total chunk count Chunk-by-chunk retrieval with base64 encoding Race condition prevention for concurrent downloads of same CID Sliding window timeout for download sessions (10 minutes since last access) +257/-0  manageHelloPeer.ts Phase 9 - IPFS capabilities in hello_peer handshake            src/libs/network/manageHelloPeer.ts Added IpfsCapabilities and PeerCapabilities interfaces for capability exchange Updated HelloPeerRequest to include optional capabilities field Implemented handleIpfsCapabilities() to dynamically connect to peer IPFS nodes Added self-peer detection bypass via DEBUG_SKIP_SELF_CHECK environment variable Enhanced response to include our capabilities and save peer list after updates +102/-2  ipfsQuote.ts IPFS cost quote endpoint for operation estimation                src/libs/network/routines/nodecalls/ipfs/ipfsQuote.ts Cost estimation endpoint for IPFS operations without creating transactions Supports IPFS_ADD, IPFS_PIN, IPFS_UNPIN operations with file size input Genesis account detection for preferential pricing Detailed cost breakdown including free tier and chargeable bytes Input validation with NaN check for file size +188/-0  manageNodeCall.ts IPFS NodeCall handlers integration and routing                      src/libs/network/manageNodeCall.ts Added comprehensive IPFS NodeCall handler imports for all operations Integrated handlers for status, add, get, pin, unpin, list operations Added streaming endpoints (ipfsAddStream, ipfsGetStream) Integrated swarm management endpoints (peers, connect, disconnect, bootstrap) Added public bridge endpoints and cost estimation (ipfsQuote, ipfsQuota) Updated manageNodeCall() signature to accept optional sender parameter +107/-1  ipfsPins.ts IPFS account pins query endpoint with expiration info        src/libs/network/routines/nodecalls/ipfs/ipfsPins.ts Account-based pins query endpoint returning pins from account IPFS state Expiration summary calculation showing permanent, expiring, and expired pins Breakdown of pins expiring within 7 and 30 days Returns total pinned bytes, earned rewards, and paid costs Null safety checks for pins array and IPFS state +142/-0  ipfsQuota.ts IPFS quota check endpoint for storage limits                          src/libs/network/routines/nodecalls/ipfs/ipfsQuota.ts Storage quota status endpoint for account IPFS usage Tier-based quota limits (regular vs genesis accounts) Usage percentage calculation and remaining capacity reporting Free tier allocation tracking for genesis accounts Pin count limits and remaining pins available +139/-0  endpointHandlers.ts IPFS custom_charges validation in transaction handling      src/libs/network/endpointHandlers.ts Added IPFS custom_charges validation in transaction confirmation Validates actual IPFS cost against signed max_cost_dem Genesis account detection for preferential pricing validation Attaches actual cost breakdown to validity data for client review Prevents transaction if actual cost exceeds signed maximum +77/-0    ipfsManager.ts IPFS manager singleton initialization and access                  src/libs/network/routines/nodecalls/ipfs/ipfsManager.ts Singleton access pattern for IPFS manager with lazy initialization Thread-safe initialization handling concurrent calls Automatic retry capability on initialization failure Shutdown method for graceful cleanup during node shutdown Debug logging for initialization status +80/-0    index.ts IPFS module main exports and factory functions                      src/features/ipfs/index.ts Main export file for IPFS integration module with comprehensive documentation Exports IPFSManager class and factory function createIpfsManager Exports types for IPFS configuration, health status, pins, and streaming Exports error classes and swarm key utilities for private network support Exports ExpirationWorker for managing pin expiration lifecycle +130/-0  ipfsAdd.ts IPFS add content NodeCall handler with DoS protection        src/libs/network/routines/nodecalls/ipfs/ipfsAdd.ts NodeCall handler for adding content to IPFS with automatic pinning Implements DoS prevention with 16MB content size limit and dual validation Supports base64 encoding/decoding for binary-safe content transfer Returns CID, size, and success status in RPC response format +112/-0  errors.ts IPFS error classes and error code definitions                        src/features/ipfs/errors.ts Defines custom error hierarchy for IPFS operations with base IPFSError class Implements specialized error types: IPFSConnectionError, IPFSTimeoutError, IPFSNotFoundError, IPFSInvalidCIDError, IPFSAPIError Each error includes code, message, and optional cause for debugging Exports IPFS_ERROR_CODES constant for error code reference +108/-0  ipfsPublicPublish.ts IPFS public network publish endpoint handler                          src/libs/network/routines/nodecalls/ipfs/ipfsPublicPublish.ts NodeCall handler for publishing content to public IPFS network via DHT Checks if public bridge is enabled before allowing publish operations Handles rate limiting and permission errors with appropriate HTTP status codes Part of Phase 5 - Public Bridge implementation +106/-0  ipfsGet.ts IPFS content retrieval NodeCall handler                                    src/libs/network/routines/nodecalls/ipfs/ipfsGet.ts NodeCall handler for retrieving content from IPFS by CID Defaults to base64 encoding for binary safety in RPC responses Handles IPFSNotFoundError and IPFSInvalidCIDError with appropriate HTTP status codes Returns content, size, and encoding information +100/-0  ipfsClusterPin.ts IPFS cluster-wide pinning endpoint handler                              src/libs/network/routines/nodecalls/ipfs/ipfsClusterPin.ts NodeCall handler for pinning content across multiple Demos network nodes Supports replication factor, expiration, and metadata parameters Returns cluster pin result with replication status and per-node errors Part of Phase 4 - Private Network Implementation +100/-0  CategorizedLogger.ts Logger categories and getAllEntries caching optimization  src/utilities/tui/CategorizedLogger.ts Adds IPFS and DEBUG log categories for better log organization Implements caching mechanism for getAllEntries() to prevent event loop saturation Cache is invalidated when new entries are added or logs are cleared Improves TUI responsiveness by avoiding repeated sorting of all log entries +29/-1    ipfsPublicCheck.ts IPFS public gateway availability check endpoint                    src/libs/network/routines/nodecalls/ipfs/ipfsPublicCheck.ts NodeCall handler for checking content availability on public IPFS gateways Performs lightweight HEAD requests without downloading full content Returns availability status, gateway info, and response time metrics Part of Phase 5 - Public Bridge implementation +91/-0    ipfsSwarmConnect.ts IPFS swarm peer connection endpoint handler                            src/libs/network/routines/nodecalls/ipfs/ipfsSwarmConnect.ts NodeCall handler for connecting to IPFS swarm peers by multiaddress Supports marking peers as Demos network nodes for tracking Returns connection result with peer ID and error information Part of Phase 4 - Private Network Implementation +88/-0    ipfsPin.ts IPFS content pinning NodeCall handler                                        src/libs/network/routines/nodecalls/ipfs/ipfsPin.ts NodeCall handler for pinning existing IPFS content to local node Validates CID format and handles not-found errors appropriately Prevents garbage collection of pinned content Returns success status and pinned CID +93/-0    executeOperations.ts IPFS transaction operation handlers integration                    src/libs/blockchain/routines/executeOperations.ts Adds IPFS operation handlers: ipfs_add, ipfs_pin, ipfs_unpin, ipfs_extend_pin Imports IPFSOperations module for transaction processing Improves operation result handling to preserve custom results from operations Properly propagates operation failures and updates results accordingly +42/-6    ipfsSwarmDisconnect.ts IPFS swarm peer disconnection endpoint handler                      src/libs/network/routines/nodecalls/ipfs/ipfsSwarmDisconnect.ts NodeCall handler for disconnecting from IPFS swarm peers by peer ID Unregisters peer from Demos peer tracking upon successful disconnection Returns disconnection result with peer ID Part of Phase 4 - Private Network Implementation +85/-0    ipfsUnpin.ts IPFS content unpinning NodeCall handler                                    src/libs/network/routines/nodecalls/ipfs/ipfsUnpin.ts NodeCall handler for unpinning content from local IPFS node Allows garbage collection of unpinned content Validates CID format and handles invalid CID errors Returns success status and unpinned CID +81/-0    index.ts IPFS NodeCall handlers index and exports                                  src/libs/network/routines/nodecalls/ipfs/index.ts Central export file for all IPFS NodeCall handlers Exports content operations: ipfsAdd, ipfsGet, ipfsPin, ipfsUnpin, ipfsListPins Exports swarm management: ipfsSwarmPeers, ipfsSwarmConnect, ipfsSwarmDisconnect, ipfsBootstrapList Exports public bridge and utility endpoints with phase-based organization +41/-0    ipfsRateLimitStatus.ts IPFS public bridge rate limit status endpoint                        src/libs/network/routines/nodecalls/ipfs/ipfsRateLimitStatus.ts NodeCall handler for retrieving public bridge rate limit status Returns current usage against configured limits for requests and bytes per minute Includes reset time and limit configuration information Part of Phase 5 - Public Bridge implementation +68/-0    ipfsDemosPeers.ts IPFS Demos network peers discovery endpoint                            src/libs/network/routines/nodecalls/ipfs/ipfsDemosPeers.ts NodeCall handler for listing known Demos network peers in IPFS swarm Returns peer IDs and multiaddresses for Demos network nodes Indicates if private network mode is enabled Part of Phase 4 - Private Network Implementation +67/-0    ipfsBootstrapList.ts IPFS bootstrap nodes list endpoint handler                              src/libs/network/routines/nodecalls/ipfs/ipfsBootstrapList.ts NodeCall handler for retrieving configured bootstrap nodes Returns bootstrap node multiaddresses and count Indicates private network status and configured nodes Part of Phase 4 - Private Network Implementation +64/-0    ipfsSwarmPeers.ts IPFS connected swarm peers list endpoint                                  src/libs/network/routines/nodecalls/ipfs/ipfsSwarmPeers.ts NodeCall handler for listing currently connected IPFS swarm peers Returns peer information including ID, address, direction, and latency Indicates if operating in private network mode Part of Phase 4 - Private Network Implementation +62/-0    ipfsStatus.ts IPFS node health status check endpoint                                      src/libs/network/routines/nodecalls/ipfs/ipfsStatus.ts NodeCall handler for checking IPFS node health and status Returns peer ID, peer count, and healthy flag Provides timestamp and error information if unhealthy Used for monitoring IPFS daemon availability +62/-0    ipfsListPins.ts IPFS list pinned content endpoint handler                                src/libs/network/routines/nodecalls/ipfs/ipfsListPins.ts NodeCall handler for listing all pinned content CIDs on local IPFS node Returns array of pinned CIDs and total count Used for inventory and quota management +50/-0    control.ts Debug logging for hello_peer protocol handler                        src/libs/omniprotocol/protocol/handlers/control.ts Adds debug logging for hello_peer NodeCall with peer identity truncation Logs manageHelloPeer response extra field for debugging Writes detailed response to custom log file for full inspection +5/-0      tagCategories.ts Log tag to category mappings for IPFS and DEBUG                    src/utilities/tui/tagCategories.ts Adds IPFS tag mappings: IPFS, IPFS MANAGER, IPFS UPLOAD, IPFS DOWNLOAD, IPFS PIN, IPFS UNPIN, IPFS ERROR Adds DEBUG tag mappings: DEBUG, DEBUG HELLO PEER, DEBUG CAPABILITIES Enables proper categorization of IPFS and debug logs in TUI +14/-0    Peer.ts Peer capabilities field for Phase 9 implementation              src/libs/peer/Peer.ts Adds capabilities field to Peer class for Phase 9 peer capabilities tracking Initializes capabilities as undefined in constructor Allows storing peer capabilities like IPFS support information +5/-0      peerBootstrap.ts Debug logging for peer bootstrap routine                                  src/libs/peer/routines/peerBootstrap.ts Adds debug logging for bootstrap entry with peer count Logs peer identity before calling sayHelloToPeer for debugging Improves visibility into peer discovery process +3/-0      sharedState.ts IPFS status tracking in shared application state                  src/utilities/sharedState.ts Adds ipfsStatus field to track IPFS node status with states: active, connecting, retrying, error, disabled Includes peer ID, peer count, retry attempt tracking, and error messages Enables TUI display of IPFS status and connection state +14/-0    GCR_Main.ts IPFS state field in GCR account entity                                      src/model/entities/GCRv2/GCR_Main.ts Adds ipfs field to GCRMain entity for storing account IPFS state Uses JSONB column type with empty object default Enables consensus-level tracking of pins, quotas, and rewards per account +4/-0      deriveMempoolOperation.ts Custom charges inclusion in operation parameters                  src/libs/utils/demostdlib/deriveMempoolOperation.ts Includes custom_charges in operation params for Phase 9 IPFS cost control Spreads transaction custom_charges into operation params Enables per-operation cost limits and user-defined pricing +5/-1      index.ts IPFS routines integration in GCR operations                            src/libs/blockchain/gcr/gcr_routines/index.ts Imports GCRIPFSRoutines module for IPFS-related GCR operations Adds ipfs property to gcrRoutines object Enables Phase 3 IPFS integration with consensus registry +3/-0      server_rpc.ts Sender identity parameter for NodeCall handlers                    src/libs/network/server_rpc.ts Passes sender parameter to manageNodeCall function Enables NodeCall handlers to access sender identity for authorization +1/-1      init-ipfs.sh IPFS daemon initialization script with swarm key                  ipfs_53550/init-ipfs.sh Initialization script for IPFS with fixed swarm port 4001 configuration Writes Demos Network swarm key for private network isolation Configures API, gateway, and swarm addresses with environment variables Enables LAN address announcement and NAT port mapping +68/-0    init-ipfs.sh IPFS daemon initialization script with swarm key                  ipfs/init-ipfs.sh Initialization script for IPFS with fixed swarm port 4001 configuration Writes Demos Network swarm key for private network isolation Configures API, gateway, and swarm addresses with environment variables Enables LAN address announcement and NAT port mapping +68/-0
Bug fix	1 files index.ts Phase 9 - TTY safety and TUI stdin handler fixes                  src/index.ts Added TTY check before calling setRawMode() to prevent crashes in Docker/CI Conditional stdin handler setup only when TUI is disabled TUI mode now waits for hello_peer without stdin handlers (TUI manages input) Improved error handling with proper cleanup in finally block +55/-36
Documentation	1 files IPFS_YELLOWPAPER.md Complete IPFS integration yellowpaper specification            IPFS_YELLOWPAPER.md Comprehensive 744-line specification document for IPFS integration Covers architecture, tokenomics, quotas, pin expiration, private network, and public bridge Documents all RPC endpoints, error handling, security considerations, and implementation phases Includes configuration reference, constants, and glossary for complete protocol specification +744/-0
	1 files run IPFS container orchestration and lifecycle management        run Adds IPFS container startup with port management (API port = NODE_PORT + 1000) Implements orphaned container cleanup for PostgreSQL and IPFS Adds IPFS port availability checks and automatic cleanup of leftover containers Implements graceful shutdown for IPFS daemon with proper signal handling Adds IPFS swarm key verification for private network configuration +266/-4
Additional files	45 files _continue_here.md +27/-20  devnet_docker_setup.md +0/-53    feature_devnet_docker.md +29/-0    feature_ipfs_custom_charges.md +109/-0  feature_ipfs_overview.md +44/-0    feature_ipfs_phase9_config.md +25/-0    feature_ipfs_technical.md +50/-0    feature_ipfs_tokenomics.md +40/-0    feature_omniprotocol.md +37/-0    ipfs_architecture_overview.md +71/-0    ipfs_expiration_system.md +123/-0  ipfs_gcr_integration.md +139/-0  ipfs_nodecall_api.md +141/-0  ipfs_tokenomics.md +95/-0    ipfs_transaction_types.md +79/-0    omniprotocol_complete_2025_11_11.md +0/-407  omniprotocol_session_2025-12-01.md +0/-48    omniprotocol_wave8.1_complete.md +0/-345  omniprotocol_wave8_tcp_physical_layer.md +0/-485  session_dem481_completion_2026_01_10.md +57/-0    session_ipfs_ux_fixes_2026_01_09.md +54/-0    session_pr_review_fixes_2026_01_08.md +86/-0    session_ud_ownership_verification_2025_10_21.md +0/-138  session_ud_points_implementation_2025_01_31.md +0/-103  tlsnotary_integration_context.md +0/-79    typescript_audit_complete_2025_12_17.md +0/-70    ud_architecture_patterns.md +0/-146  ud_integration_complete.md +0/-143  ud_phase5_complete.md +0/-260  ud_phases_tracking.md +0/-466  ud_security_patterns.md +0/-157  ud_technical_reference.md +0/-65    README.md +32/-0    README.md +49/-19  docker-compose.yml +140/-0  README.md +255/-0  docker-compose.yml +50/-0    start_ipfs +46/-0    stop_ipfs +27/-0    README.md +177/-0  docker-compose.yml +50/-0    start_ipfs +46/-0    stop_ipfs +27/-0    package.json +1/-1      PROXY_MANAGER_PLAN.md +0/-301

---

Summary by CodeRabbit

• New Features

  • Added IPFS decentralized content storage with pinning, expiration management, and automatic cleanup.
  • Introduced streaming upload/download for large files.
  • Added storage quotas and cost quoting for IPFS operations.
  • Implemented private IPFS network mode for security.

• Documentation

  • Added comprehensive IPFS architecture and operation guides.
  • Documented network port requirements and firewall setup.

• Infrastructure

  • Updated DevNet Docker Compose to include IPFS nodes.
  • Enhanced TUI with IPFS status monitoring and peer information display.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Caution

Review failed

The pull request is closed.

Walkthrough

This PR implements comprehensive IPFS (InterPlanetary File System) integration for the Demos Network, including a decentralized storage system with cost accounting, pin expiration mechanisms, multi-phase infrastructure, Docker containerization, and extensive RPC endpoint support for storage operations.

Changes

Cohort / File(s)	Summary
Documentation & Setup .gitignore, README.md, IPFS_YELLOWPAPER.md, devnet/README.md, ipfs/README.md, ipfs_53550/README.md	Updated ignore patterns; added network requirements and firewall setup guides; comprehensive IPFS integration blueprint covering architecture, tokenomics, quotas, and operations.
Session/Memory Files .serena/memories/*	Deleted legacy session logs and OmniProtocol documentation; added new IPFS feature docs (overview, technical, tokenomics, custom charges, architecture, expiration, GCR integration, NodeCall API, transaction types); documented recent session completions and UX fixes.
Core IPFS Manager & Infrastructure src/features/ipfs/IPFSManager.ts, src/features/ipfs/types.ts, src/features/ipfs/errors.ts, src/features/ipfs/swarmKey.ts, src/features/ipfs/index.ts	Introduced IPFSManager class with Kubo HTTP API integration, retry logic, health checks, content operations, streaming, swarm management, and bootstrap configuration; added comprehensive type definitions, error classes, swarm key utilities, and public API exports.
IPFS Expiration & GCR Integration src/features/ipfs/ExpirationWorker.ts, src/libs/blockchain/gcr/gcr_routines/GCRIPFSRoutines.ts, src/libs/blockchain/gcr/gcr_routines/index.ts	Implemented background expiration worker with configurable intervals and grace periods; added GCRIPFSRoutines for per-pubkey synchronized pin/reward/cost management; integrated IPFS routines into GCR export.
IPFS Operations & Tokenomics src/libs/blockchain/routines/ipfsOperations.ts, src/libs/blockchain/routines/ipfsTokenomics.ts, src/libs/blockchain/routines/executeOperations.ts, src/libs/network/endpointHandlers.ts	Implemented IPFS transaction handlers (add, pin, unpin, extend_pin) with cost validation and state updates; added genesis account detection, pricing tiers, fee distribution; integrated custom_charges validation into transaction confirmation; wired operation execution.
IPFS NodeCall Handlers (Bulk) src/libs/network/routines/nodecalls/ipfs/{ipfsStatus,ipfsAdd,ipfsGet,ipfsPin,ipfsUnpin,ipfsListPins,ipfsPins,ipfsAddStream,ipfsGetStream,ipfsSwarmPeers,ipfsSwarmConnect,ipfsSwarmDisconnect,ipfsBootstrapList,ipfsClusterPin,ipfsDemosPeers,ipfsPublicFetch,ipfsPublicPublish,ipfsPublicCheck,ipfsRateLimitStatus,ipfsQuote,ipfsQuota}.ts, src/libs/network/routines/nodecalls/ipfs/index.ts, src/libs/network/routines/nodecalls/ipfs/ipfsManager.ts	Added 21 RPC endpoint handlers for content operations, streaming, swarm management, bootstrap, clustering, public bridge, rate limiting, quoting, and quota queries; centralized manager initialization with lazy loading and lifecycle control.
Network Integration & Peer Management src/libs/network/manageNodeCall.ts, src/libs/network/server_rpc.ts, src/libs/network/manageHelloPeer.ts, src/libs/peer/Peer.ts, src/libs/peer/PeerManager.ts, src/libs/peer/routines/peerBootstrap.ts	Extended NodeCall routing to support 21+ IPFS operations with sender context; added IPFS capabilities exchange in hello_peer flow; enhanced peer discovery with IPFS peer info storage and registration; added local network detection and capability discovery utilities.
Data Model & Types src/model/entities/types/IPFSTypes.ts, src/model/entities/GCRv2/GCR_Main.ts	Defined comprehensive IPFS types (PinnedContent, AccountIPFSState, quotas, durations, pricing); implemented quota system with tiered limits and expiration logic; added IPFS state jsonb column to GCRMain.
Docker & Infrastructure devnet/docker-compose.yml, ipfs/docker-compose.yml, ipfs/init-ipfs.sh, ipfs/start_ipfs, ipfs/stop_ipfs, ipfs_53550/{docker-compose.yml,init-ipfs.sh,start_ipfs,stop_ipfs}	Added IPFS node services to devnet with 4 containerized instances; created standalone IPFS docker-compose and initialization scripts with private swarm key support, port isolation, health checks, and resource limits.
Runtime & Startup run	Enhanced startup script with IPFS container management, per-port folder isolation, IPFS readiness checks, swarm key verification, external database validation, improved signal handling, and expanded system requirement checks.
Shared State & UI src/utilities/sharedState.ts, src/utilities/tui/TUIManager.ts, src/utilities/tui/CategorizedLogger.ts, src/utilities/tui/tagCategories.ts	Added IPFS status tracking (active/connecting/retrying/error/disabled) to shared state; integrated IPFS status column in TUI header; added IPFS and DEBUG log categories with caching optimization; added IPFS-specific tag mappings for logging.
Other Updates src/index.ts, src/features/incentive/PointSystem.ts, src/libs/blockchain/routines/deriveMempoolOperation.ts, src/libs/omniprotocol/protocol/handlers/control.ts, package.json	Enhanced stdin handling with TUI awareness and safer raw mode; added linkedNomisIdentities field to UserPoints; propagated custom_charges into operation params; added hello_peer debug logging; bumped SDK to v2.8.6 for custom_charges support.

Sequence Diagram(s)

sequenceDiagram
    participant Client as Client/SDK
    participant Node as Demos Node
    participant IPFS as IPFS Manager
    participant GCR as GCR State
    participant IPFS_Net as IPFS Network

    Note over Client,IPFS_Net: IPFS Add with Custom Charges (Two-Step)
    
    Client->>Node: ipfsQuote(file_size, operation)
    activate Node
    Node->>Node: Determine genesis status
    Node->>Node: Calculate cost via tokenomics
    Node-->>Client: cost_dem, breakdown
    deactivate Node
    
    Client->>Client: Construct ipfs_add transaction<br/>with max_cost_dem in custom_charges
    Client->>Client: Sign transaction
    
    Client->>Node: submitTransaction(ipfs_add_tx)
    activate Node
    Node->>Node: Validate custom_charges.ipfs
    Node->>Node: Calculate actual_cost_dem
    Node->>Node: Compare actual vs max_cost_dem
    alt actual_cost > max_cost_dem
        Node-->>Client: Invalid (exceeded budget)
    else Within budget
        Node->>GCR: Deduct balance (processFeePayment)
        Node->>IPFS: Add content
        activate IPFS
        IPFS->>IPFS_Net: Store & pin content
        IPFS-->>Node: Return CID
        deactivate IPFS
        Node->>GCR: Update pins, costs, rewards
        Node-->>Client: Success with CID
    end
    deactivate Node

Loading

sequenceDiagram
    participant Host as Host A
    participant Node as Demos Node
    participant IPFS_Node as IPFS Node
    participant Peer as Peer B (Host)
    participant Peer_IPFS as Peer B IPFS

    Note over Host,Peer_IPFS: IPFS Peer Discovery & Connection (Hello Peer Flow)
    
    Node->>Peer: hello_peer request
    activate Peer
    Peer->>Peer_IPFS: Get node info (peerId, addresses)
    Peer_IPFS-->>Peer: Return capabilities
    Peer-->>Node: hello_peer response<br/>(with capabilities)
    deactivate Peer
    
    activate Node
    Node->>Node: Parse IpfsCapabilities<br/>(peerId, addresses)
    Node->>IPFS_Node: handleIpfsCapabilities
    activate IPFS_Node
    IPFS_Node->>IPFS_Node: Filter local vs remote addresses
    IPFS_Node->>Peer_IPFS: connectPeer(multiaddr)
    Peer_IPFS-->>IPFS_Node: Connected
    IPFS_Node->>Node: Register Demos peer
    deactivate IPFS_Node
    Node-->>Node: Store peer capabilities
    deactivate Node

Loading

sequenceDiagram
    participant Worker as ExpirationWorker
    participant DB as GCR Database
    participant IPFS as IPFS Manager
    participant GCRRoutines as GCRIPFSRoutines

    Note over Worker,GCRRoutines: IPFS Pin Expiration Cycle
    
    Worker->>Worker: runExpirationCheck (periodic)
    Worker->>DB: Get all accounts with pins
    activate DB
    DB-->>Worker: Account list
    deactivate DB
    
    loop For each account
        Worker->>GCRRoutines: getIPFSState(account)
        activate GCRRoutines
        GCRRoutines->>DB: Query account IPFS state
        DB-->>GCRRoutines: pins array
        GCRRoutines-->>Worker: AccountIPFSState
        deactivate GCRRoutines
        
        Worker->>Worker: Identify expired pins<br/>(current_time > expiresAt)
        
        alt Pin within grace period
            Worker->>Worker: Wait & log
        else Pin past grace period
            loop For each expired pin (batch)
                Worker->>IPFS: unpin(cid)
                IPFS-->>Worker: Unpinned
                Worker->>GCRRoutines: removePin(account, cid)
                GCRRoutines->>DB: Update IPFS state
                DB-->>GCRRoutines: Updated
            end
        end
    end
    
    Worker->>Worker: Update stats & log

Loading

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~120 minutes

Reasoning: This PR introduces a massive IPFS subsystem with 1500+ new lines across 50+ files, spanning infrastructure (Docker, startup scripts), core manager/types, tokenomics/costing, 21 NodeCall handlers, transaction handlers, GCR integration, peer discovery enhancements, and UI updates. The changes exhibit high logic density (custom_charges validation, cost calculations, expiration logic, quota enforcement), multi-pattern implementation (handlers, state routines, operations), deep integration across consensus/network/storage layers, and multiple edge cases (grace periods, genesis accounts, streaming uploads, rate limiting). While many handlers follow similar patterns (reducing per-file effort), the cohesive complexity of the entire feature—including cross-module dependencies, race condition prevention (per-pubkey locking), and tokenomics correctness—demands comprehensive scrutiny of core logic, test coverage, and integration flow.

Possibly related issues

• IPFS Cost Estimation in Confirm/Execute Flow #586: Directly implements the IPFS custom_charges two-step confirm/execute flow with ipfsQuote, validateTransaction custom_charges validation, and SDK type updates.
• [SDK] Add ipfsQuote nodeCall helper #603: Adds ipfsQuote nodeCall handler and related server-side quote/types with SDK version update aligning with retrieved SDK integration request.
• [Node] Validate custom_charges in confirmTransaction #584: Implements exact confirmTransaction validation for custom_charges.ipfs in endpointHandlers.ts via calculatePinCost and breakdown attachment.
• [Node] Add ipfsQuote nodeCall endpoint #585: Implements ipfsQuote nodeCall handler registration and IPFS nodecall routing as requested in the issue.
• Phase 4: Transaction Types - IPFS Writes (SDK + Node) #595: Implements complete Phase 4 IPFS transaction types (add, pin, unpin, extend_pin) with validation, tokenomics, and state updates.
• [SDK] Update IPFSOperations to include custom_charges in payloads #602: Adds end-to-end IPFS support (ipfsQuote, operations, custom_charges validation) enabling SDK custom_charges field flow through node validation and execution.

Possibly related PRs

• Tg identities v2 implementation #468: Modifies PointSystem.ts public UserPoints shape, overlapping with main PR's addition of linkedNomisIdentities.
• better run options #469: Modifies run/startup scripts with per-port isolation and enhanced checks, sharing infrastructure changes with main PR.
• Integrated Nomis Identity #552: Modifies PointSystem.ts to add Nomis-related fields/behavior, directly overlapping with main PR's linkedNomisIdentities addition.

Suggested labels

Possible security concern, Review effort 5/5, Feature: IPFS, Phase 3, High complexity, Multiple integrations

Suggested reviewers

• cwilvx

---

🐰 Through the IPFS looking glass, content flows free,
From quotes to pins to expiration's decree,
With genesis tiers and peer connections bright,
The Demos network stores with economic might! ✨

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch claude/ipfs-yellowpaper-docs-ZwEkp

---

📜 Recent review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Disabled knowledge base sources:

• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between bf4f233 and e6cda63.

📒 Files selected for processing (101)

• .gitignore
• .serena/memories/_continue_here.md
• .serena/memories/devnet_docker_setup.md
• .serena/memories/feature_devnet_docker.md
• .serena/memories/feature_ipfs_custom_charges.md
• .serena/memories/feature_ipfs_overview.md
• .serena/memories/feature_ipfs_phase9_config.md
• .serena/memories/feature_ipfs_technical.md
• .serena/memories/feature_ipfs_tokenomics.md
• .serena/memories/feature_omniprotocol.md
• .serena/memories/ipfs_architecture_overview.md
• .serena/memories/ipfs_expiration_system.md
• .serena/memories/ipfs_gcr_integration.md
• .serena/memories/ipfs_nodecall_api.md
• .serena/memories/ipfs_tokenomics.md
• .serena/memories/ipfs_transaction_types.md
• .serena/memories/omniprotocol_complete_2025_11_11.md
• .serena/memories/omniprotocol_session_2025-12-01.md
• .serena/memories/omniprotocol_wave8.1_complete.md
• .serena/memories/omniprotocol_wave8_tcp_physical_layer.md
• .serena/memories/session_dem481_completion_2026_01_10.md
• .serena/memories/session_ipfs_ux_fixes_2026_01_09.md
• .serena/memories/session_pr_review_fixes_2026_01_08.md
• .serena/memories/session_ud_ownership_verification_2025_10_21.md
• .serena/memories/session_ud_points_implementation_2025_01_31.md
• .serena/memories/tlsnotary_integration_context.md
• .serena/memories/typescript_audit_complete_2025_12_17.md
• .serena/memories/ud_architecture_patterns.md
• .serena/memories/ud_integration_complete.md
• .serena/memories/ud_phase5_complete.md
• .serena/memories/ud_phases_tracking.md
• .serena/memories/ud_security_patterns.md
• .serena/memories/ud_technical_reference.md
• IPFS_YELLOWPAPER.md
• README.md
• devnet/README.md
• devnet/docker-compose.yml
• ipfs/README.md
• ipfs/docker-compose.yml
• ipfs/init-ipfs.sh
• ipfs/start_ipfs
• ipfs/stop_ipfs
• ipfs_53550/README.md
• ipfs_53550/docker-compose.yml
• ipfs_53550/init-ipfs.sh
• ipfs_53550/start_ipfs
• ipfs_53550/stop_ipfs
• package.json
• run
• src/features/incentive/PointSystem.ts
• src/features/ipfs/ExpirationWorker.ts
• src/features/ipfs/IPFSManager.ts
• src/features/ipfs/errors.ts
• src/features/ipfs/index.ts
• src/features/ipfs/swarmKey.ts
• src/features/ipfs/types.ts
• src/features/tlsnotary/PROXY_MANAGER_PLAN.md
• src/index.ts
• src/libs/blockchain/gcr/gcr_routines/GCRIPFSRoutines.ts
• src/libs/blockchain/gcr/gcr_routines/index.ts
• src/libs/blockchain/routines/executeOperations.ts
• src/libs/blockchain/routines/ipfsOperations.ts
• src/libs/blockchain/routines/ipfsTokenomics.ts
• src/libs/network/endpointHandlers.ts
• src/libs/network/manageHelloPeer.ts
• src/libs/network/manageNodeCall.ts
• src/libs/network/routines/nodecalls/ipfs/index.ts
• src/libs/network/routines/nodecalls/ipfs/ipfsAdd.ts
• src/libs/network/routines/nodecalls/ipfs/ipfsAddStream.ts
• src/libs/network/routines/nodecalls/ipfs/ipfsBootstrapList.ts
• src/libs/network/routines/nodecalls/ipfs/ipfsClusterPin.ts
• src/libs/network/routines/nodecalls/ipfs/ipfsDemosPeers.ts
• src/libs/network/routines/nodecalls/ipfs/ipfsGet.ts
• src/libs/network/routines/nodecalls/ipfs/ipfsGetStream.ts
• src/libs/network/routines/nodecalls/ipfs/ipfsListPins.ts
• src/libs/network/routines/nodecalls/ipfs/ipfsManager.ts
• src/libs/network/routines/nodecalls/ipfs/ipfsPin.ts
• src/libs/network/routines/nodecalls/ipfs/ipfsPins.ts
• src/libs/network/routines/nodecalls/ipfs/ipfsPublicCheck.ts
• src/libs/network/routines/nodecalls/ipfs/ipfsPublicFetch.ts
• src/libs/network/routines/nodecalls/ipfs/ipfsPublicPublish.ts
• src/libs/network/routines/nodecalls/ipfs/ipfsQuota.ts
• src/libs/network/routines/nodecalls/ipfs/ipfsQuote.ts
• src/libs/network/routines/nodecalls/ipfs/ipfsRateLimitStatus.ts
• src/libs/network/routines/nodecalls/ipfs/ipfsStatus.ts
• src/libs/network/routines/nodecalls/ipfs/ipfsSwarmConnect.ts
• src/libs/network/routines/nodecalls/ipfs/ipfsSwarmDisconnect.ts
• src/libs/network/routines/nodecalls/ipfs/ipfsSwarmPeers.ts
• src/libs/network/routines/nodecalls/ipfs/ipfsUnpin.ts
• src/libs/network/server_rpc.ts
• src/libs/omniprotocol/protocol/handlers/control.ts
• src/libs/peer/Peer.ts
• src/libs/peer/PeerManager.ts
• src/libs/peer/routines/peerBootstrap.ts
• src/libs/utils/demostdlib/deriveMempoolOperation.ts
• src/model/entities/GCRv2/GCR_Main.ts
• src/model/entities/types/IPFSTypes.ts
• src/utilities/sharedState.ts
• src/utilities/tui/CategorizedLogger.ts
• src/utilities/tui/TUIManager.ts
• src/utilities/tui/tagCategories.ts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[sonarqubecloud]

Quality Gate failed

Failed conditions
1 Security Hotspot
3.3% Duplication on New Code (required ≤ 3%)
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

[qodo-code-review]

PR Compliance Guide 🔍

Below is a summary of compliance checks for this PR:

Security Compliance
⚪	BigInt precision loss Description: processFeePayment converts amount/feeDistribution.hostShare from bigint to number via Number(...) (e.g., lines 793-803), which can lose precision above 2^53 and can lead to incorrect balance debits/credits (potentially enabling underpayment or balance corruption/minting when large fees are processed). ipfsOperations.ts [780-804] Referred Code private static async processFeePayment( from: string, amount: bigint, txHash: string, ): Promise<OperationResult> { try { // Get current balances const senderBalance = await GCR.getGCRNativeBalance(from) // Calculate fee distribution (MVP: 100% to host) const feeDistribution = calculateFeeDistribution(amount) // Deduct from sender const newSenderBalance = senderBalance - Number(amount) await GCR.setGCRNativeBalance(from, newSenderBalance, txHash) // Credit hosting RPC (this node) // REVIEW: In MVP, 100% goes to the hosting RPC if (feeDistribution.hostShare > 0n) { const hostAddress = getSharedState.publicKeyHex if (hostAddress) { ... (clipped 4 lines)
	Multipart header injection Description: addStream interpolates the user-controlled filename directly into the multipart Content-Disposition header (lines 609-616) without sanitizing CR/LF or quotes, allowing multipart header injection that could alter the request body sent to the Kubo API (e.g., injecting additional headers/parts) if an attacker supplies a crafted filename. IPFSManager.ts [608-616] Referred Code const boundary = `----DemosIPFSBoundary${Date.now()}` const contentDisposition = filename ? `form-data; name="file"; filename="${filename}"` : "form-data; name=\"file\"; filename=\"file\"" // Build the multipart header and footer const headerStr = `--${boundary}\r\nContent-Disposition: ${contentDisposition}\r\nContent-Type: application/octet-stream\r\n\r\n` const footerStr = `\r\n--${boundary}--\r\n`
Ticket Compliance
⚪	🎫 No ticket provided Create ticket/issue
Codebase Duplication Compliance
⚪	Codebase context is not defined Follow the guide to enable codebase context checks.
Custom Compliance
🟢	Generic: Meaningful Naming and Self-Documenting Code Objective: Ensure all identifiers clearly express their purpose and intent, making code self-documenting Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
🔴	Generic: Comprehensive Audit Trails Objective: To create a detailed and reliable record of critical system actions for security analysis and compliance. Status: Missing audit context: Critical actions (e.g., add/pin/unpin) are only logged behind debug mode and do not consistently capture actor/user identity, timestamp, action, and outcome needed to reconstruct events. Referred Code async add(content: Buffer | Uint8Array | string, filename?: string): Promise<string> { this.logDebug(`Adding content to IPFS (size: ${content.length} bytes)`) // Convert content to Blob for multipart form data // Use Uint8Array to ensure compatibility with BlobPart type const contentBuffer = typeof content === "string" ? Buffer.from(content) : content const blob = new Blob([new Uint8Array(contentBuffer)]) // Create multipart form data const formData = new FormData() formData.append("file", blob, filename || "file") const response = await this.apiRequest("/api/v0/add", "POST", formData) const data = await response.json() const cid = data.Hash if (!cid) { throw new IPFSAPIError("IPFS add response missing Hash field", undefined, JSON.stringify(data)) } this.logDebug(`Content added successfully. CID: ${cid}`) ... (clipped 82 lines) Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Robust Error Handling and Edge Case Management Objective: Ensure comprehensive error handling that provides meaningful context and graceful degradation Status: Unvalidated numeric parsing: External input custom_charges.max_cost_dem is parsed via parseFloat and converted to BigInt without validation, which can throw on NaN/non-numeric values and cause unexpected failures. Referred Code if (customCharges?.max_cost_dem !== undefined) { // Parse max_cost_dem as bigint for comparison const maxCostDem = BigInt( Math.floor(parseFloat(String(customCharges.max_cost_dem)) * 1e8), ) if (costResult.totalCost > maxCostDem) { return { success: false, message: `Actual cost ${costResult.totalCost} exceeds signed maximum ${maxCostDem} DEM`, } } Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Secure Error Handling Objective: To prevent the leakage of sensitive system information through error messages while providing sufficient detail for internal debugging. Status: Leaky user errors: User-facing message values return detailed internal error strings and sensitive financial context (exact balances/costs), rather than generic client-safe errors with details restricted to internal logs. Referred Code // REVIEW: Check sender has sufficient balance const senderBalance = await GCR.getGCRNativeBalance(from) if (hasInsufficientBalance(BigInt(senderBalance), costResult.totalCost)) { return { success: false, message: `Insufficient balance: required ${costResult.totalCost} DEM, have ${senderBalance} DEM`, } } Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Secure Logging Practices Objective: To ensure logs are useful for debugging and auditing without exposing sensitive information like PII, PHI, or cardholder data. Status: Sensitive debug logging: The code logs and persists full hello_peer responses (including peer URLs, peer lists, and capability/IPFS address data) via log.custom/JSON.stringify, risking exposure of sensitive network topology information. Referred Code // REVIEW: Phase 9 - Extract capabilities from response log.info("[DEBUG HELLO PEER] Raw response.extra: " + JSON.stringify(response.extra), false) // DEBUG: Write to file for full inspection await log.custom("hello_peer_debug", `CLIENT RECEIVED response.extra: ${JSON.stringify(response.extra, null, 2)}`, false) if (response.extra.capabilities) { peer.capabilities = response.extra.capabilities log.info( "[DEBUG CAPABILITIES] Received capabilities from peer: " + JSON.stringify(response.extra.capabilities), false, ) } else { log.info("[DEBUG CAPABILITIES] No capabilities in response from peer", false) } log.debug( "[DEBUG HELLO PEER] Final Peer sync data: " + JSON.stringify(peer.sync), ) log.debug( "[DEBUG CAPABILITIES] Final Peer capabilities: " + JSON.stringify(peer.capabilities), ... (clipped 8 lines) Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Security-First Input Validation and Data Handling Objective: Ensure all data inputs are validated, sanitized, and handled securely to prevent vulnerabilities Status: Unsafe filename handling: Streaming multipart upload builds Content-Disposition using the unescaped filename, allowing header/body injection (e.g., quotes/CRLF) and malformed multipart requests. Referred Code const boundary = `----DemosIPFSBoundary${Date.now()}` const contentDisposition = filename ? `form-data; name="file"; filename="${filename}"` : "form-data; name=\"file\"; filename=\"file\"" // Build the multipart header and footer const headerStr = `--${boundary}\r\nContent-Disposition: ${contentDisposition}\r\nContent-Type: application/octet-stream\r\n\r\n` const footerStr = `\r\n--${boundary}--\r\n` Learn more about managing compliance generic rules or creating your own custom rules

Compliance status legend

🟢 - Fully Compliant
🟡 - Partial Compliant
🔴 - Not Compliant
⚪ - Requires Further Human Verification
🏷️ - Compliance label

[qodo-code-review]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
High-level	Streaming implementation defeats its purpose The ipfsGetStream handler for large file downloads incorrectly loads the entire file into memory instead of streaming it. This should be changed to stream data directly from the IPFS node to avoid high memory usage. Examples: src/libs/network/routines/nodecalls/ipfs/ipfsGetStream.ts [133-166] Solution Walkthrough: Before: // file: ipfsGetStream.ts // A map to store entire file contents in memory const downloadSessions = new Map<string, { content: Buffer }>() async function ipfsGetStream(data) { // ... let session = downloadSessions.get(data.cid) if (!session) { // This call loads the ENTIRE file into memory const content = await ipfs.get(data.cid) // The entire file buffer is stored in the session map session = { content: content, ... } downloadSessions.set(data.cid, session) } // A chunk is sliced from the full in-memory buffer const chunkData = session.content.subarray(start, end) return { chunk: chunkData.toString("base64") } } After: // file: ipfsGetStream.ts // A map to store stream readers, not full content const downloadSessions = new Map<string, { streamReader: ReadableStreamDefaultReader }>() async function ipfsGetStream(data) { let session = downloadSessions.get(data.cid) if (!session) { // This call gets a true stream, not the full content const stream = await ipfs.getStream(data.cid) session = { streamReader: stream.getReader(), ... } downloadSessions.set(data.cid, session) } // Read the next chunk directly from the stream const { done, value } = await session.streamReader.read() if (done) { downloadSessions.delete(data.cid) // Clean up } return { chunk: value?.toString("base64") } } Suggestion importance[1-10]: 9 __ Why: The suggestion correctly identifies a critical design flaw where the ipfsGetStream handler buffers the entire file in memory, which negates the benefits of streaming and introduces a memory vulnerability for large files.	High
Possible issue	Use a random boundary for uploads Replace Date.now() with crypto.randomUUID() when generating multipart boundaries to prevent race conditions in concurrent addStream calls. src/features/ipfs/IPFSManager.ts [608-610] // Note: We need to use a custom approach since FormData doesn't support streaming -const boundary = `----DemosIPFSBoundary${Date.now()}` +const boundary = `----DemosIPFSBoundary${crypto.randomUUID()}` const contentDisposition = filename ? `form-data; name="file"; filename="${filename}"` [To ensure code accuracy, apply this suggestion manually] Suggestion importance[1-10]: 9 __ Why: The suggestion correctly identifies a critical race condition in multipart boundary generation that could cause data corruption during concurrent uploads and proposes a robust, standard fix.	High
	Fix race condition in locking utility Fix a race condition in the withPubkeyLock utility by chaining promises to ensure sequential execution of operations for the same pubkey. src/libs/blockchain/gcr/gcr_routines/GCRIPFSRoutines.ts [27-49] async function withPubkeyLock<T>( pubkey: string, operation: () => Promise<T>, ): Promise<T> { - // Wait for any pending operation on this pubkey - const pending = pendingOperations.get(pubkey) - if (pending) { - await pending.catch(() => {}) // Ignore errors from previous operation - } + // Get the promise for the currently pending operation for this pubkey. + const pending = pendingOperations.get(pubkey) ?? Promise.resolve() - // Create and track this operation - const currentOp = operation() + // Chain the new operation to run after the pending one completes. + const currentOp = pending.then(operation, operation) + + // Update the map with the promise for the new operation. pendingOperations.set(pubkey, currentOp) - try { - return await currentOp - } finally { - // Clean up if this is still the tracked operation - if (pendingOperations.get(pubkey) === currentOp) { - pendingOperations.delete(pubkey) - } - } + // Return the promise for the new operation. + return currentOp } [To ensure code accuracy, apply this suggestion manually] Suggestion importance[1-10]: 9 __ Why: The suggestion correctly identifies a critical race condition in the locking mechanism that would allow concurrent operations and potential data corruption, and it provides a correct and robust fix using promise chaining.	High
	Avoid floating-point math for costs Avoid using parseFloat for financial calculations by parsing the cost string into integer and fractional parts to construct a BigInt, ensuring precision. src/libs/blockchain/routines/ipfsOperations.ts [155-170] if (customCharges?.max_cost_dem !== undefined) { - // Parse max_cost_dem as bigint for comparison - const maxCostDem = BigInt( - Math.floor(parseFloat(String(customCharges.max_cost_dem)) * 1e8), - ) + // Parse max_cost_dem as bigint for comparison without using floating point math + const maxCostStr = String(customCharges.max_cost_dem); + const [integerPart, fractionalPart = ""] = maxCostStr.split("."); + const fractionalPadded = fractionalPart.padEnd(8, "0").slice(0, 8); + const maxCostDem = BigInt(integerPart + fractionalPadded); + if (costResult.totalCost > maxCostDem) { return { success: false, message: `Actual cost ${costResult.totalCost} exceeds signed maximum ${maxCostDem} DEM`, } } // Log the fair pricing in action log.debug( `[IPFSOperations] IPFS_ADD: Using custom_charges - max=${maxCostDem}, actual=${costResult.totalCost}`, ) } else { [To ensure code accuracy, apply this suggestion manually] Suggestion importance[1-10]: 8 __ Why: The suggestion correctly points out the risk of using floating-point math for financial calculations and provides a safer, more precise implementation using string manipulation to avoid potential precision loss.	Medium
	Use a regex for robust LAN address detection Replace the string-based LAN address detection with a more robust regular expression to prevent false positives from parts of the multiaddress string like port numbers. src/libs/peer/PeerManager.ts [649-666] // Check if this is a private/LAN address -const isLanAddr = addr.includes("/192.168.") || - addr.includes("/10.") || - addr.includes("/172.16.") || - addr.includes("/172.17.") || - addr.includes("/172.18.") || - addr.includes("/172.19.") || - addr.includes("/172.20.") || - addr.includes("/172.21.") || - addr.includes("/172.22.") || - addr.includes("/172.23.") || - addr.includes("/172.24.") || - addr.includes("/172.25.") || - addr.includes("/172.26.") || - addr.includes("/172.27.") || - addr.includes("/172.28.") || - addr.includes("/172.29.") || - addr.includes("/172.30.") || - addr.includes("/172.31.") +const lanRegex = /^\/ip4\/(192\.168\.|10\.|172\.(1[6-9]|2[0-9]|3[0-1]))/ +const isLanAddr = lanRegex.test(addr) [To ensure code accuracy, apply this suggestion manually] Suggestion importance[1-10]: 8 __ Why: The suggestion correctly identifies a fragile implementation for LAN address detection and provides a robust regex-based solution, preventing potential false positives and improving the reliability of peer discovery logic.	Medium
	Validate duration presets Add validation to ensure the provided string duration is a known preset before accessing PIN_DURATION_SECONDS and PIN_DURATION_PRICING to prevent potential errors. src/model/entities/types/IPFSTypes.ts [380-390] if (typeof duration === "string") { + if (!(duration in PIN_DURATION_SECONDS)) { + return { + valid: false, + durationSeconds: 0, + pricingMultiplier: 0, + errorMessage: `IPFS_INVALID_DURATION: preset "${duration}" is unknown`, + } + } const durationSeconds = PIN_DURATION_SECONDS[duration] const pricingMultiplier = PIN_DURATION_PRICING[duration] return { valid: true, durationSeconds, expiresAt: durationSeconds > 0 ? currentTimestamp + durationSeconds : undefined, pricingMultiplier, } } [To ensure code accuracy, apply this suggestion manually] Suggestion importance[1-10]: 8 __ Why: The suggestion correctly identifies a missing validation check that could lead to runtime errors if an unknown duration string is passed, and the proposed fix makes the function more robust.	Medium
	Halt execution on first failure Modify the operation execution loop to break immediately upon the first failed operation, ensuring transactional atomicity and preventing subsequent operations from running after a failure. src/libs/blockchain/routines/executeOperations.ts [161-182] // REVIEW: Phase 9 - Properly propagate operation results // Check if the operation returned a failure and propagate it if (!result.success) { valid = false error = result.message || "Operation failed" + // Set the failed operation's result + results.operations.set(operations[i], { + success: false, + message: error, + }) + // Halt execution on first failure + break } -operations[i].status = valid -// Only set generic result for operations that didn't set their own result -// (genesis, transfer_native, and default case) + +operations[i].status = true +// If an operation-specific result was already set (like for IPFS), +// it will be preserved. Otherwise, we might set a generic one if needed, +// but the current IPFS operations all set their own results. if (!results.operations.has(operations[i])) { results.operations.set(operations[i], { - success: valid, - message: valid - ? "Transaction executed" - : "Transaction failed due to: " + error, - }) -} else if (!valid) { - // Update existing result if operation failed - results.operations.set(operations[i], { - success: false, - message: result.message || "Transaction failed due to: " + error, + success: true, + message: "Transaction executed", }) } [To ensure code accuracy, apply this suggestion manually] Suggestion importance[1-10]: 8 __ Why: This suggestion correctly identifies a critical flaw in transaction processing logic. Continuing to execute operations after a failure is a bug, and the proposed change to break the loop ensures atomic execution, which is crucial for correctness.	Medium
	Roll back pin on state update failure Roll back the IPFS pin by calling ipfs.unpin() if the subsequent database state update fails, preventing orphaned data and maintaining consistency. src/libs/blockchain/routines/ipfsOperations.ts [246-253] // Update account IPFS state const stateResult = await GCRIPFSRoutines.addPin(from, pin) if (!stateResult.success) { - // REVIEW: Should we unpin from IPFS if state update fails? + // Roll back the IPFS pin when state update fails + await ipfs.unpin(cid).catch(() => {}) log.warning( `[IPFSOperations] IPFS_ADD: Content added but state update failed: ${stateResult.message}`, ) } [To ensure code accuracy, apply this suggestion manually] Suggestion importance[1-10]: 7 __ Why: The suggestion correctly proposes adding a rollback mechanism to maintain state consistency by unpinning from IPFS if the corresponding database update fails, which is a good practice for error handling.	Medium
	Prevent crashes with null-safe access Use optional chaining (?.) and nullish coalescing (??) to safely access ipfsState properties, providing default zero values to prevent crashes for accounts without existing IPFS state. src/libs/network/routines/nodecalls/ipfs/ipfsQuota.ts [79-115] // Get account IPFS state const ipfsState = await GCRIPFSRoutines.getIPFSState(data.address) const isGenesis = await isGenesisAccount(data.address) // Determine tier and get quota limits const tier: QuotaTier = isGenesis ? "genesis" : "regular" const quota = getQuotaForTier(tier) // Calculate quota status -const usedBytes = ipfsState.totalPinnedBytes -const pinCount = ipfsState.pins.length +const usedBytes = ipfsState?.totalPinnedBytes ?? 0 +const pinCount = ipfsState?.pins?.length ?? 0 const remainingBytes = Math.max(0, quota.maxPinnedBytes - usedBytes) const remainingPins = Math.max(0, quota.maxPinCount - pinCount) const usagePercent = quota.maxPinnedBytes > 0 ? Math.round((usedBytes / quota.maxPinnedBytes) * 10000) / 100 : 0 ... // Add free tier info for genesis accounts -if (isGenesis && ipfsState.freeAllocationBytes > 0) { +if (isGenesis && (ipfsState?.freeAllocationBytes ?? 0) > 0) { + const freeAllocation = ipfsState?.freeAllocationBytes ?? 0 + const usedFree = ipfsState?.usedFreeBytes ?? 0 response.freeTier = { - allocation: ipfsState.freeAllocationBytes, - used: ipfsState.usedFreeBytes, - remaining: Math.max(0, ipfsState.freeAllocationBytes - ipfsState.usedFreeBytes), + allocation: freeAllocation, + used: usedFree, + remaining: Math.max(0, freeAllocation - usedFree), } } [To ensure code accuracy, apply this suggestion manually] Suggestion importance[1-10]: 7 __ Why: The suggestion correctly identifies a potential null pointer exception if ipfsState is null for a new account and provides a robust fix using optional chaining, which prevents a crash and correctly reflects a zero-usage state.	Medium
	Validate numeric cost values Validate that max_cost_dem parses to a valid, non-negative number before comparing it to the actual cost, and return an error if the value is NaN or negative to prevent invalid inputs. src/libs/network/endpointHandlers.ts [130-137] const actualCostDemNum = Number(actualCostResult.totalCost) const maxCostNum = parseFloat(maxCostDem) +if (isNaN(maxCostNum) || maxCostNum < 0) { + log.warning( + `[handleValidateTransaction] Invalid signed max_cost_dem: ${maxCostDem}`, + ) + validationData.data.valid = false + validationData.data.message = + `[IPFS Validation] Invalid max_cost_dem value: ${maxCostDem}` + // re-sign & return… +} // Validate: actual cost must not exceed signed max_cost_dem if (actualCostDemNum > maxCostNum) { - log.warning( - `[handleValidateTransaction] IPFS cost exceeds max: actual=${actualCostDemNum} > max=${maxCostNum}`, - ) … } [To ensure code accuracy, apply this suggestion manually] Suggestion importance[1-10]: 6 __ Why: The suggestion correctly points out that parseFloat can result in NaN, which would lead to incorrect cost validation. Adding a check for NaN and negative values makes the cost validation logic more robust against invalid transaction data.	Low
General	Use BigInt for balance updates Use BigInt for all balance calculations to prevent precision loss, especially when the amount could be large. src/libs/blockchain/routines/ipfsOperations.ts [793-794] -// Deduct from sender -const newSenderBalance = senderBalance - Number(amount) -await GCR.setGCRNativeBalance(from, newSenderBalance, txHash) +// Deduct from sender using BigInt +const currentBal = BigInt(senderBalance) +const newSenderBal = currentBal - amount +await GCR.setGCRNativeBalance(from, newSenderBal.toString(), txHash) [To ensure code accuracy, apply this suggestion manually] Suggestion importance[1-10]: 8 __ Why: The suggestion correctly identifies that using Number for balance calculations is unsafe and can lead to precision loss, proposing a necessary switch to BigInt for all financial arithmetic.	Medium
	Optimize DB query to fetch only relevant accounts Optimize the database query in getAllAccountsWithIPFS to fetch only accounts with expirable pins, reducing unnecessary processing and improving performance. src/features/ipfs/ExpirationWorker.ts [186-204] private async getAllAccountsWithIPFS(): Promise<string[]> { try { const db = await Datasource.getInstance() const repo = db.getDataSource().getRepository(GCRMain) - // Find all accounts with IPFS pins - // Using QueryBuilder to search for accounts where ipfs.pins is not empty + // Find all accounts with at least one expirable IPFS pin + // This is more efficient than fetching all accounts with pins const results = await repo .createQueryBuilder("gcr") .select("gcr.pubkey") - .where("jsonb_array_length(gcr.ipfs -> 'pins') > 0") + .where(`gcr.ipfs @> '{"pins": [{"expiresAt": 0}]}'`) .getRawMany() return results.map((r) => r.gcr_pubkey as string) } catch (error) { log.error(`[ExpirationWorker] Failed to get accounts with IPFS: ${error}`) return [] } } [To ensure code accuracy, apply this suggestion manually] Suggestion importance[1-10]: 7 __ Why: The suggestion provides a valid optimization to the database query, which improves performance by reducing the workload of the expiration worker, although the original query is not incorrect.	Medium
	Use accurate byte length Replace data.content.length with Buffer.byteLength(data.content, "utf8") to accurately measure the content's byte size for the DoS prevention check, correctly handling multi-byte characters. src/libs/network/routines/nodecalls/ipfs/ipfsAdd.ts [44-48] // DoS prevention - limit content size before buffer allocation const MAX_CONTENT_SIZE = 16 * 1024 * 1024 -const contentLength = data.content.length +const contentLength = Buffer.byteLength(data.content, "utf8") // Base64 encoded content is ~33% larger than raw, so decoded size is ~75% of encoded -const estimatedDecodedSize = data.base64 ? Math.ceil(contentLength * 0.75) : contentLength +const estimatedDecodedSize = data.base64 + ? Math.ceil(contentLength * 0.75) + : contentLength [To ensure code accuracy, apply this suggestion manually] Suggestion importance[1-10]: 5 __ Why: The suggestion improves the accuracy of the DoS protection check by using Buffer.byteLength instead of string.length, which is more correct for multi-byte characters, though the impact is moderate given the existing multi-layered size checks.	Low
More