Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,6 @@ node_modules/
test-results/
playwright-report/
.worktrees/

# Local dev data
.dev/
21 changes: 21 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,12 +70,33 @@ just serve-web # serve Leptos web app locally
just build-relay # build relay server (release)
just run # run native desktop app
just relay # run the relay server
just dev # start full local dev stack (relay + workers + web)
just dev-quick # same as dev, but skip cargo build
just dev-clean # remove .dev/ (keys, logs, storage DB)
```

**All code must pass `just check` (fmt + clippy + test + WASM) with zero
warnings before being committed.** Browser tests (`just test-browser`)
require Firefox and geckodriver installed.

### Local Development Stack

Run `just dev` to start all services locally:

| Service | Address | Description |
|---------|---------|-------------|
| Relay | `localhost:9090` (TCP), `localhost:9091` (WS) | Bridges peers |
| Replay node | connects via relay | In-memory state sync (max 1000 events/server) |
| Storage node | connects via relay | Archival SQLite storage |
| Web UI | `http://localhost:8080` | Leptos app via `trunk serve` |

All service logs are color-coded and interleaved in the terminal. Press
`Ctrl+C` to stop everything. Identity keys and data persist in `.dev/`
across restarts so peer IDs stay stable. Use `just dev-clean` to reset.

After the first run, use `just dev-quick` to skip the build step and
start services immediately.

### Dual-Target Support (Native + WASM)

All library crates must compile for both native and `wasm32-unknown-unknown`.
Expand Down
122 changes: 122 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
# Willow

A peer-to-peer Discord replacement built with Rust. No central servers, no
accounts, no middlemen. End-to-end encrypted by default.

## Features

- **Text chat** with channels, threads, reactions, pins, and emoji
- **End-to-end encryption** — ChaCha20-Poly1305 with X25519 key exchange
- **Peer-to-peer** — libp2p networking with GossipSub, Kademlia, and mDNS
- **File sharing** — content-addressed chunking, transferred peer-to-peer
- **Servers & permissions** — roles, fine-grained permissions, invites
- **Event-sourced state** — deterministic, mergeable, offline-friendly
- **Runs everywhere** — native desktop (Bevy) and web browser (Leptos/WASM)

## Architecture

```
Leptos Web UI or Bevy Desktop UI
│ │
└──── Client Library (willow-client) ────┐
│ │
State Machine Network Layer
(willow-state, pure) (willow-network, libp2p)
│ │
┌──────┴──────┐ ┌──────┴──────┐
│ Channels │ │ Relay │
│ Messaging │ │ Workers │
│ Crypto │ │ (replay, │
│ Files │ │ storage) │
└─────────────┘ └─────────────┘
```

**Crates:**

| Crate | Purpose |
|-------|---------|
| `willow-state` | Pure event-sourced state machine (zero I/O) |
| `willow-client` | UI-agnostic client library |
| `willow-transport` | Binary serialization & protocol framing |
| `willow-identity` | Ed25519 identity, message signing, profiles |
| `willow-messaging` | Chat messages, HLC ordering |
| `willow-crypto` | E2E encryption (ChaCha20-Poly1305, X25519) |
| `willow-channel` | Servers, channels, roles, permissions |
| `willow-files` | Content-addressed file chunking |
| `willow-network` | libp2p networking layer (native + WASM) |
| `willow-relay` | Relay server bridging TCP and WebSocket peers |
| `willow-web` | Leptos web UI |
| `willow-app` | Bevy desktop UI |

## Getting Started

### Prerequisites

- [Rust](https://rustup.rs/) (stable)
- [just](https://github.com/casey/just) (command runner)
- [trunk](https://trunkrs.dev/) (for the web UI)
- `wasm32-unknown-unknown` target: `rustup target add wasm32-unknown-unknown`

### Local Development

Start the full stack with a single command:

```bash
just dev
```

This launches all services:

| Service | Address | Description |
|---------|---------|-------------|
| Relay | `localhost:9090` (TCP), `localhost:9091` (WS) | Bridges peers |
| Replay node | connects via relay | In-memory state sync (max 1000 events/server) |
| Storage node | connects via relay | Archival SQLite storage |
| Web UI | `http://localhost:8080` | Leptos app via trunk serve |

All service logs are color-coded and interleaved in the terminal. Press
`Ctrl+C` to stop everything.

Identity keys and data persist in `.dev/` so peer IDs stay stable across
restarts. After the first run, use `just dev-quick` to skip the build step.

```bash
just dev-quick # skip build, start immediately
just dev-clean # reset all local dev data
```

### Running Individual Services

```bash
just relay # relay server only
just serve-web # web UI only (trunk serve)
just run # native desktop app
```

### Docker

```bash
just docker-build # build all images
just docker-up # start full stack
just docker-down # stop full stack
just docker-logs # tail all logs
```

## Testing

Willow has 420+ tests across multiple tiers:

```bash
just check # fmt + clippy + test + WASM check (run before committing)
just test # all cargo tests
just test-state # pure state machine (64 tests, instant)
just test-client # client library (93 tests)
just test-app # Bevy headless + network integration (113 tests)
just test-relay # relay history sync (3 tests)
just test-scale # scaling/performance tests
just test-browser # in-browser Leptos tests (requires Firefox + geckodriver)
```

## License

See [LICENSE](LICENSE) for details.
12 changes: 12 additions & 0 deletions justfile
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,18 @@ docker-ids:
@docker compose exec storage-1 willow-storage --print-peer-id 2>/dev/null || echo "storage-1: not running"
@docker compose exec storage-2 willow-storage --print-peer-id 2>/dev/null || echo "storage-2: not running"

# Start all services for local development (relay, workers, web UI)
dev:
./scripts/dev.sh

# Start all services, skipping the build step
dev-quick:
./scripts/dev.sh --skip-build

# Clean dev data (identity keys, logs, storage DB)
dev-clean:
rm -rf .dev

# Clean build artifacts
clean:
cargo clean
Expand Down
173 changes: 173 additions & 0 deletions scripts/dev.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,173 @@
#!/usr/bin/env bash
# scripts/dev.sh — Start all Willow services for local development.
#
# Usage:
# ./scripts/dev.sh # start all services
# ./scripts/dev.sh --skip-build # skip cargo build step
#
# Services started:
# - Relay (TCP 9090, WebSocket 9091)
# - Replay node (in-memory, max 1000 events/server)
# - Storage node (SQLite at .dev/storage.db)
# - Web UI (trunk serve on localhost:8080)

set -euo pipefail

ROOT="$(cd "$(dirname "$0")/.." && pwd)"
DEV_DIR="$ROOT/.dev"
LOG_DIR="$DEV_DIR/logs"

# Colors for service labels
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[0;33m'
BLUE='\033[0;34m'
MAGENTA='\033[0;35m'
NC='\033[0m' # No Color

SKIP_BUILD=false
for arg in "$@"; do
case "$arg" in
--skip-build) SKIP_BUILD=true ;;
esac
done

# Ensure dev directories exist
mkdir -p "$DEV_DIR" "$LOG_DIR"

# Track child PIDs for cleanup
PIDS=()

cleanup() {
echo ""
echo -e "${RED}Shutting down all services...${NC}"
for pid in "${PIDS[@]}"; do
if kill -0 "$pid" 2>/dev/null; then
kill "$pid" 2>/dev/null || true
fi
done
wait 2>/dev/null || true
echo -e "${GREEN}All services stopped.${NC}"
}
trap cleanup EXIT INT TERM

# Prefix each line of a command's output with a colored label
run_with_prefix() {
local color="$1" label="$2"
shift 2
"$@" 2>&1 | while IFS= read -r line; do
echo -e "${color}[${label}]${NC} $line"
done
}

# --- Build -------------------------------------------------------------------

if [ "$SKIP_BUILD" = false ]; then
echo -e "${GREEN}Building all services...${NC}"
cargo build -p willow-relay -p willow-replay -p willow-storage 2>&1 | \
while IFS= read -r line; do echo -e "${GREEN}[build]${NC} $line"; done
echo -e "${GREEN}Build complete.${NC}"
echo ""
fi

# --- Relay --------------------------------------------------------------------

RELAY_IDENTITY="$DEV_DIR/relay.key"
RELAY_LOG="$LOG_DIR/relay.log"

echo -e "${BLUE}Starting relay...${NC}"
cargo run -p willow-relay -- \
--tcp-port 9090 \
--ws-port 9091 \
--identity "$RELAY_IDENTITY" \
--name "Dev Relay" \
> "$RELAY_LOG" 2>&1 &
RELAY_PID=$!
PIDS+=("$RELAY_PID")

# Wait for the relay to log its peer ID (up to 10s)
RELAY_PEER_ID=""
for i in $(seq 1 50); do
if [ -f "$RELAY_LOG" ]; then
RELAY_PEER_ID=$(grep -oP 'peer_id[= ]+\K[A-Za-z0-9]+' "$RELAY_LOG" 2>/dev/null | head -1 || true)
if [ -n "$RELAY_PEER_ID" ]; then
break
fi
fi
sleep 0.2
done

if [ -z "$RELAY_PEER_ID" ]; then
echo -e "${RED}Failed to get relay peer ID. Check $RELAY_LOG${NC}"
exit 1
fi

RELAY_ADDR="/ip4/127.0.0.1/tcp/9091/ws/p2p/$RELAY_PEER_ID"
echo -e "${BLUE}Relay started:${NC} $RELAY_ADDR"
echo ""

# Tail relay logs with prefix
tail -f "$RELAY_LOG" 2>/dev/null | while IFS= read -r line; do
echo -e "${BLUE}[relay]${NC} $line"
done &
PIDS+=($!)

# --- Replay node --------------------------------------------------------------

REPLAY_IDENTITY="$DEV_DIR/replay.key"
echo -e "${YELLOW}Starting replay node...${NC}"
cargo run -p willow-replay -- \
--identity-path "$REPLAY_IDENTITY" \
--relay "$RELAY_ADDR" \
--max-events-per-server 1000 \
--sync-interval 10 \
> "$LOG_DIR/replay.log" 2>&1 &
PIDS+=($!)

tail -f "$LOG_DIR/replay.log" 2>/dev/null | while IFS= read -r line; do
echo -e "${YELLOW}[replay]${NC} $line"
done &
PIDS+=($!)

# --- Storage node -------------------------------------------------------------

STORAGE_IDENTITY="$DEV_DIR/storage.key"
STORAGE_DB="$DEV_DIR/storage.db"
echo -e "${MAGENTA}Starting storage node...${NC}"
cargo run -p willow-storage -- \
--identity-path "$STORAGE_IDENTITY" \
--relay "$RELAY_ADDR" \
--db-path "$STORAGE_DB" \
--sync-interval 15 \
> "$LOG_DIR/storage.log" 2>&1 &
PIDS+=($!)

tail -f "$LOG_DIR/storage.log" 2>/dev/null | while IFS= read -r line; do
echo -e "${MAGENTA}[storage]${NC} $line"
done &
PIDS+=($!)

# --- Web UI -------------------------------------------------------------------

echo -e "${GREEN}Starting web UI (trunk serve)...${NC}"
(cd "$ROOT/crates/web" && trunk serve) 2>&1 | while IFS= read -r line; do
echo -e "${GREEN}[web]${NC} $line"
done &
PIDS+=($!)

# --- Summary ------------------------------------------------------------------

echo ""
echo -e "${GREEN}═══════════════════════════════════════════════${NC}"
echo -e "${GREEN} Willow dev stack running${NC}"
echo -e "${GREEN}═══════════════════════════════════════════════${NC}"
echo -e " Relay: ${BLUE}localhost:9090${NC} (TCP) / ${BLUE}localhost:9091${NC} (WS)"
echo -e " Web UI: ${GREEN}http://localhost:8080${NC}"
echo -e " Relay ID: ${RELAY_PEER_ID}"
echo -e " Logs: ${LOG_DIR}/"
echo -e "${GREEN}═══════════════════════════════════════════════${NC}"
echo -e " Press ${RED}Ctrl+C${NC} to stop all services"
echo ""

# Wait forever (cleanup runs on signal)
wait