Add WASM-based process isolation + DHT-based message routing#24
Merged
Add WASM-based process isolation + DHT-based message routing#24
Conversation
- Add Read() method that returns EOF when ReadWriteCloser is nil - Add Write() method that discards output when ReadWriteCloser is nil - This prevents nil pointer dereference when WASM modules access stdin/stdout - Enables clean initialization without requiring a dummy ReadWriteCloser
- Add Async bool field to ProcConfig to gate sync vs async behavior - Update module instantiation to conditionally use WithStartFunctions() - Add ProcessMessage method that handles both sync and async modes - In sync mode: _start runs automatically, processes one message, module closes - In async mode: _start prevented from running, poll() called for each stream - Add Poll method as alias for backward compatibility - Implement proper error handling for sys.ExitError with exit code 0
- Add main() function for sync mode (processes one message and exits) - Add poll() export function for async mode (processes one message per call) - Both functions implement the same echo logic: copy stdin to stdout - Support both synchronous and asynchronous execution patterns - One stream (start to EOF) equals one message protocol
- Add Async field to all ProcConfig instances in tests - Update TestEcho_Synchronous to use sync mode with proper stdin/stdout setup - Update TestEcho_Asynchronous to use async mode with poll() function - Update TestEcho_RepeatedAsync to test multiple async messages - Fix TestProc_Integration_WithRealWasm to use async mode - Remove await function checks (not used in current design) - All tests now pass with both sync and async modes
- Add examples/echo/README.md with dual-mode pattern explanation - Add system/SPEC-PROC.md with detailed process specification - Document WASM lifecycle constraint and dual-mode solution - Explain message delivery protocol: one stream (start to EOF) = one message - Provide configuration examples and use cases for both modes - Document benefits: explicit mode selection, WASM compliance, unified API
- Add Read/Write methods to Endpoint for better nil handling - Return io.EOF when ReadWriteCloser is nil (allows graceful completion) - Discard output when ReadWriteCloser is nil (prevents errors) - Support both sync and async modes with proper I/O abstraction - Maintain backward compatibility with existing code
- Update run.go to support both sync and async process execution - Modify env.go to handle dual-mode configuration - Update main.go to integrate with new ProcConfig.Async field - Support both client (sync) and server (async) modes from CLI - Maintain backward compatibility with existing command structure
- Update go.mod and go.sum with required dependencies - Add support for base58 encoding in endpoint generation - Include semaphore for concurrency control - Update wazero imports for WASM runtime support
- Remove cmd/ww/shell/ directory and all shell-related code - Remove examples/export/ directory and export examples - Remove examples/fd-demo/ directory and fd-demo examples - Remove examples/import/ directory and import examples - Clean up deprecated functionality to focus on core dual-mode design
- Remove system/socket.go (replaced by dual-mode design) - Remove system/system.capnp and system.capnp.go (capnp files) - Remove spec/cell.md (outdated specification) - Clean up deprecated system architecture components - Focus on streamlined dual-mode process design
- Remove cmd/ww/args/ directory (args handling) - Remove cmd/ww/run/README.md (outdated documentation) - Remove cmd/ww/run/fd.go and fd_test.go (file descriptor handling) - Remove cmd/ww/run/run_darwin.go and run_unix.go (platform-specific code) - Streamline run command to focus on dual-mode process execution - Remove platform-specific complexity in favor of unified approach
- Update README.md to reflect new dual-mode design - Remove references to deprecated shell and export functionality - Focus on core process execution capabilities - Document sync and async mode usage patterns - Align documentation with streamlined architecture
- String() method now returns just the endpoint name (e.Name) - Protocol() method returns the full protocol ID with /ww/0.1.0/ prefix - Updated TestEndpoint_String to match the expected behavior - All tests now pass
- Updated run.go and proc.go as part of cell removal feature - Changes align with feat/remove-cell branch goals
- Added nil check for c.Src before calling io.ReadAll - Prevents panic when Src is nil in error handling tests - Returns proper error message when source reader is nil
- Added syncIO type to handle stdin/stdout in sync mode - Fixed ProcConfig.New to set up ReadWriteCloser for sync mode - Added missing --async flag to ww run command - Echo example now works correctly in sync mode - Module closure in sync mode is now handled properly
- Replaced syncIO struct with anonymous struct composition - Uses os.Stdin as Reader and os.Stdout as WriteCloser directly - Cleaner and more direct approach for sync mode I/O
- Added LoadIPFSFile method to Env for loading WASM files from IPFS - Updated all test cases to use Src (io.ReadCloser) instead of Bytecode ([]byte) - Tests now properly reflect the new ProcConfig structure with Src field - Maintains compatibility with both local files and IPFS paths
- Added ww cat command to connect stdin/stdout to remote peer streams - Takes peer-id and base58-encoded endpoint as arguments - Constructs full protocol ID (/ww/0.1.0/<endpoint>) - Sets up bidirectional copying between local and remote streams - Includes comprehensive README with usage examples - Supports all standard wetware capabilities and options
- Refactored cat command to be client-only (no server setup) - Uses minimal libp2p host for direct peer connections - Added comprehensive logging for peer ID and endpoint information - Fixed protocol construction to use correct endpoint format - Removed IPFS dependency for simpler P2P connections - Added connection timeout and proper error handling
- Removed complex IPFS DHT integration (not available in RPC client) - Uses minimal libp2p host for direct peer connections - Simplified to use localhost:2020 for testing - Maintains IPFS environment for future enhancements - Clean client-only approach without server setup
- Add ww cat <peer> <proc> command for connecting to peers and executing procedures - Use libp2p host in client mode with DHT for peer discovery - Implement nc-like linger behavior where client waits for server to close after EOF - Remove IPFS subcommand approach in favor of direct libp2p stream handling - Add proper error handling for broken pipe and graceful closure detection Note: 'failed to find peer' error is expected when peer is not in DHT table, as the peer discovery requires proper DHT bootstrapping on both sides.
- Add util/dht.go with NewDHT() and WaitForDHTReady() functions - Add util/host.go with NewClient() and NewServer() functions - NewDHT() creates client-mode DHT seeded with IPFS peers - WaitForDHTReady() monitors WAN and LAN routing tables for readiness - Provides reusable utilities for DHT bootstrap and peer discovery
- Replace direct libp2p host creation with util.NewClient() - Use util.NewDHT() to create DHT client seeded with IPFS peers - Add DHT bootstrap and readiness monitoring before peer discovery - Improve error handling and connection flow - Remove hardcoded timeout, use context-based cancellation - Update IPFS endpoint default to use IP instead of DNS
- Add DHT client creation and bootstrap to run command - Add comprehensive libp2p event monitoring for peer identification - Update IPFS endpoint default to use IP instead of DNS - Add DHT field to IPFSEnv struct for proper cleanup - Improve error handling and logging for DHT operations
- Remove --with-mdns capability flag - Remove --discover and --namespace P2P flags - Simplify flag configuration by removing unused discovery options - These flags are no longer needed with DHT-based peer discovery
- Remove github.com/libp2p/zeroconf/v2 (mDNS dependency) - Remove unused golang.org/x/net and golang.org/x/sys versions - Clean up go.sum to remove orphaned dependency entries - Dependencies removed as part of switching from mDNS to DHT discovery
- Add defer os.Stdout.Sync() to ensure output is flushed - Improve reliability of echo example output - Minor improvement to example code quality
mikelsr
approved these changes
Sep 24, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds WASM-based process isolation and DHT-based message routing to transform Wetware into a distributed computing platform.
Key Changes
Technical Details
Breaking Changes
--with-mdns,--discover,--namespace)Files Changed
util/dht.go(new): DHT management utilitiesutil/host.go(new): Host creation utilitiescmd/ww/cat/: DHT-based peer discoverycmd/ww/run/: WASM execution with DHT routingcmd/internal/flags/: Updated capability flagsgo.mod: Cleaned up mDNS dependencies