Development Guide

Development Guide

Contributing Workflow

1. Fork the repository -- Create your own fork on GitHub.
2. Create a feature branch -- Branch from main with a descriptive name (e.g., feature/add-new-watcher-type or fix/energy-decay-bug).
3. Make changes -- Follow the code style conventions described below.
4. Run linting and tests -- Ensure all checks pass before submitting.
5. Submit a pull request -- Open a PR against main with a clear description of your changes.

Testing

Framework

pytest with asyncio_mode = "auto". All async test functions are automatically detected and run with an event loop.

Running Tests

# Fast tests (no API calls, ~4 seconds)
uv run pytest tests/ -x -m "not slow and not integration"

# Full suite
uv run pytest tests/ -x

# With coverage (target: 80%+)
uv run pytest tests/ --cov=colloquip --cov-report=term-missing

# Run a specific test file
uv run pytest tests/test_energy.py -x

# Run a specific test by name
uv run pytest tests/ -x -k "test_energy_decay"

Test Markers

• @pytest.mark.slow -- Tests that make real LLM API calls. Skipped in fast runs.
• @pytest.mark.integration -- Full deliberation loop tests. Skipped in fast runs.

Test Categories

707 tests across 37 files, organized as follows:

• Unit tests -- Energy calculator, trigger evaluator, observer agent, agent prompts, config loading, Pydantic models
• API tests -- All REST endpoints, SSE streaming, WebSocket connections, error handling and validation
• Database tests -- ORM models, repository pattern, migration integrity
• Behavioral tests -- Emergent properties, cross-domain connections, red-team activation patterns
• Integration tests -- Full deliberation loops with mock LLM from start to synthesis
• Platform tests -- Communities, agent recruitment, institutional memory, watchers, feedback and calibration
• Infrastructure tests -- Docker health checks, logging configuration, Prometheus metrics

Fixtures

Shared test factories are defined in tests/conftest.py:

• create_post() -- Factory for building test posts with configurable stance, content, and metadata
• create_session() -- Factory for test deliberation sessions
• create_agent_config() -- Factory for agent configurations with persona, mandates, and keywords
• create_metrics() -- Factory for conversation metrics (novelty, disagreement, questions, staleness)

Mocking

• MockLLM -- Returns deterministic, structured responses for testing. Supports all the same interfaces as the real Anthropic provider.
• MockEmbedding -- Returns zero vectors for testing. Allows memory and similarity features to function without an API key.

Linting

Python

# Check for lint issues
uv run ruff check .

# Check formatting
uv run ruff format --check .

# Auto-fix lint issues and format
uv run ruff check --fix . && uv run ruff format .

Ruff configuration (from pyproject.toml):

• Line length: 100 characters
• Target version: Python 3.11+
• Enabled rules: E (pycodestyle errors), F (pyflakes), I (isort)
• Ignored: E402 -- module-level imports after load_dotenv() calls are intentional

Frontend

cd web && npm run lint

Pre-Commit Hook

Located at scripts/pre-commit, the hook runs three checks before each commit:

1. ruff check -- Lint check
2. ruff format --check -- Format check
3. Fast unit tests (pytest -m "not slow and not integration")

Installation

./scripts/install-hooks.sh

This symlinks the pre-commit script into .git/hooks/. All three checks must pass for the commit to proceed.

Common Development Tasks

Adding a New API Endpoint

1. Add the route handler in the appropriate src/colloquip/api/*_routes.py file.
2. If this is a new router module, register it in src/colloquip/api/__init__.py.
3. Add the corresponding API function in web/src/lib/api.ts.
4. Write tests in tests/.

Adding a New Database Table

1. Define the SQLAlchemy model in src/colloquip/db/tables.py.
2. Create an Alembic migration: uv run alembic revision --autogenerate -m "description".
3. Add repository methods in src/colloquip/db/repository.py.
4. Add a Pydantic model in src/colloquip/models.py if needed for API serialization.

Adding a New Agent Persona

1. Define the persona configuration in src/colloquip/agents/ with a full prompt, phase mandates, and domain keywords.
2. Each agent requires: agent_id, display_name, persona_prompt, phase_mandates, domain_keywords, knowledge_scope.
3. Register the agent in the agent pool / persona loader.

Adding a New Watcher Type

1. Implement the BaseWatcher interface in src/colloquip/watchers/.
2. Register the new type in WatcherManager.
3. Add a DB model if persistent state is needed.
4. Add API routes in watcher_routes.py.

CI/CD Pipeline

GitHub Actions workflows are defined in .github/workflows/:

ci.yml

Triggered on pushes to main and claude/* branches, and on pull requests targeting main.

• lint -- Runs ruff check and ruff format --check.
• test-unit -- Runs pytest across a Python 3.11 / 3.12 matrix with coverage reporting.
• test-integration -- Runs full deliberation loop tests. Depends on unit tests passing first.

deploy.yml

Triggered on tags matching v* (e.g., v1.0.0).

• Runs the full test suite.
• Builds the Docker image (multi-stage).
• Pushes to GitHub Container Registry (GHCR).

db-migration.yml

Triggered on pull requests that touch db/ or alembic/ files.

• Validates that all migrations apply cleanly against a fresh database.

Docker Development

Running with Hot-Reload

# Build and run with development overrides
docker compose -f docker-compose.yml -f docker-compose.dev.yml up

# With monitoring (Prometheus on :9090, Grafana on :3000)
docker compose -f docker-compose.yml -f docker-compose.monitoring.yml up -d

The development compose override provides:

• Source volume mounts -- src/, tests/, config/, web/, and alembic/ are mounted into the container so changes are reflected without rebuilding.
• Mock providers -- Uses EMBEDDING_PROVIDER=mock and MEMORY_STORE=in_memory for fast iteration without external API keys.
• Debug port -- Port 5678 is exposed for debugpy attachment.
• Exposed service ports -- PostgreSQL (5432) and Redis (6379) are accessible from the host for direct inspection.

Docker Services

Service	Image	Ports	Purpose
app	Custom (multi-stage)	8000	FastAPI + static SPA
postgres	pgvector/pgvector:pg16	5432 (dev only)	Database with vector extension
redis	redis:7-alpine	6379 (dev only)	Watcher job queue
prometheus	prom/prometheus (monitoring)	9090	Metrics collection
grafana	grafana/grafana (monitoring)	3000	Dashboards

Environment Variables

Copy .env.example to .env and configure as needed:

cp .env.example .env

Docker Compose reads .env automatically. For mock mode (no API keys needed), the defaults work out of the box.

Variable	Default	Description
ANTHROPIC_API_KEY	--	Required for real LLM mode
OPENAI_API_KEY	--	Required for OpenAI embeddings
DB_PASSWORD	colloquip	PostgreSQL password
EMBEDDING_PROVIDER	mock	mock or openai
MEMORY_STORE	in_memory	in_memory or pgvector
LOG_LEVEL	INFO	Standard Python log levels

Code Style Conventions

Python

• Line length: 100 characters.
• Target version: Python 3.11+.
• Async throughout: All DB operations, LLM calls, and API handlers are async. Use await consistently; do not mix sync and async database access.
• Type hints: Use Pydantic models for structured data and type annotations on all function signatures.
• Repository pattern: All database access goes through SessionRepository. Never query tables directly in route handlers or services.
• Interface pattern: Abstract base classes define contracts for LLM, Embedding, Tool, and Watcher implementations. Add new providers by implementing the appropriate ABC.
• Import order: Managed by ruff's isort rules (stdlib, third-party, local).

TypeScript / React

• Strict mode: Enabled with strict null checks. Do not use any unless absolutely necessary.
• Path alias: @/ maps to web/src/. Use this for all imports within the frontend codebase.
• Component pattern: Three layers -- Radix UI primitive, Tailwind + CVA styled wrapper (in components/ui/), then domain-specific component.
• State management: Zustand for client-side state (with persist middleware for theme, sidebar). TanStack React Query for all server state.
• Styling: Tailwind CSS utility classes exclusively. CSS custom properties in app.css drive the three-theme system (dark, light, pastel).
• Class merging: Use the cn() utility (clsx + tailwind-merge) for conditional class composition.
• Route file: web/src/routeTree.gen.ts is auto-generated by TanStack Router. Do not edit it manually.

File Organization

• Group components by domain: deliberation/, communities/, threads/, agents/, memories/.
• Shared and reusable components belong in components/shared/ or components/ui/.
• All API call functions are centralized in web/src/lib/api.ts.
• Query keys are centralized in web/src/lib/queryKeys.ts.
• TypeScript type definitions live in web/src/types/.