feat: library extraction (7 packages) + native agent harness by chazmaniandinkle · Pull Request #4 · cogos-dev/cogos

chazmaniandinkle · 2026-04-15T04:15:42Z

Summary

Major restructure extracting the monolithic kernel into an importable library ecosystem, plus a native Go agent harness that transforms the kernel from a passive daemon into an active homeostatic agent.

Library Extraction (Phase 1 complete)

7 stdlib-only packages extracted into pkg/ with go.work multi-module workspace:

Package	Layer	Files	Tests	What
`pkg/cogblock`	0	4	17	CogBlock struct, kind enum, provenance, trust, ledger (RFC 8785, hash chain, verify)
`pkg/coordination`	1	2	7	Claim/Handoff/Broadcast, 13 functions, AgentID()
`pkg/bep`	1	13	52	Wire protocol, TLS/DeviceID, index/version vectors, events, interfaces
`pkg/reconcile`	2	7	34	Reconcilable interface, State/Plan/Action, registry, topo sort, meta-orchestrator
`pkg/modality`	2	15	33	Module interface, Bus, wire protocol, events, salience, channels, supervisor
`pkg/cogfield`	3	15	21	Node/Edge/Graph, Block, BlockAdapter, conditions, signals, sessions, documents
`pkg/uri`	3	6	27	URI struct, Parse/Format, 35 namespaces, ExtractInlineRefs
Total		69	191	~10,200 lines, all stdlib-only

Kernel wired via type aliases and function wrappers — zero behavior change, full backward compatibility.

Native Agent Harness (Phase 2b partial)

agent_harness.go — goroutine in process loop, direct Ollama HTTP calls
agent_tools.go — tool dispatch as kernel-native Go functions (health, memory search, reconcile status, signal read/write)
agent_serve.go — HTTP endpoints for agent status and manual trigger
Adaptive interval — starts at 5m, backs off to 30m after consecutive no-ops, resets on drift detection
Ollama native API — switched from /v1/chat/completions to /api/chat with think:false for reliable structured output

Anthropic Messages API Proxy

serve_messages.go — /v1/messages endpoint proxying to Anthropic with kernel context injection
Enables cog claude command for kernel-mediated Claude Code sessions
Injects system prompt with kernel state, memory context, and tool definitions

Infrastructure

Port consolidation to 6931 (single kernel port for all HTTP)
cog claude command — launches Claude Code through the kernel proxy
All 191 package tests pass, go vet clean

Test plan

go vet ./... passes
All 7 pkg modules: go test ./... passes (191 tests)
internal/engine tests pass
Kernel builds and serves on :6931
Verify cog claude launches and proxies correctly
Verify agent loop triggers on reconciler drift

Set up Go workspace (go.work) with 7 new library modules under pkg/: cogblock, bep, coordination, reconcile, modality, cogfield, uri. Each module has a go.mod and doc.go stub. No code moved yet — this establishes the skeleton so `go work sync` resolves all modules. Implements ADR-074 layer 0-3 decomposition structure. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Move the canonical CogBlock type definitions (CogBlockKind, BlockProvenance, TrustContext, BlockArtifact) and the full ledger implementation (EventEnvelope, canonicalization, hash chaining, AppendEvent, VerifyLedger) into the importable pkg/cogblock library package with zero kernel dependencies. The engine retains its own CogBlock struct with the typed Messages field ([]ProviderMessage) since that couples to provider internals. Shared types are re-exported as type aliases for backward compatibility. The kernel's ledger_core.go becomes a thin delegation layer. 17 new tests cover block JSON round-trips, kind constants, canonicalization, hashing, hash chaining, and tamper detection. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Extracts the cog:// URI parsing, formatting, and validation logic into a standalone stdlib-only package. Includes URI struct, Parse/String functions, namespace registry, query helpers, inline reference extraction, and structured error types. Resolution (filesystem lookup) stays in the kernel. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…reconcile Moves the Reconcilable interface (7-method plan/apply contract), all type definitions (State, Plan, Action, Summary, Result, status enums), state management with lineage tracking, provider registry, event emission, and meta-reconciler with Kahn's topological sort into pkg/reconcile as a stdlib-only importable library. Kernel files become thin re-export layers with type aliases and function wrappers for backward compatibility. CLI commands, serve loop, watch mode, and MCP tool integration remain in the kernel since they depend on runtime internals (ResolveWorkspace, CogBus, field conditions). 34 tests in pkg/reconcile, all kernel reconcile tests continue to pass. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…kg/modality Moves the core modality abstractions (Module interface, Bus, WireMessage, SubprocessConn, ChannelRegistry, Salience, ProcessSupervisor, TextModule) into pkg/modality as a stdlib-only reusable package. Kernel files become thin re-export wrappers via type aliases, preserving backward compatibility. Layer A (types + interfaces) and Layer B (bus, wire, supervisor) are fully extracted. Layer C (ledger integration, HTTP modules, serve wiring, tools, HUD) remains in the kernel where it depends on kernel-specific types. 33 tests cover bus dispatch, wire serialization, channel registry, salience scoring, event validation, and text module lifecycle. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Selective extraction of the BEP (Block Exchange Protocol) types package from the kernel. Moves wire-compatible protocol types, TLS/DeviceID management, version vectors, index types, event types, and the framing layer into an importable package. Creates Engine and SyncProvider interfaces for decoupled consumption. Files left in kernel (coupled to bus, AgentCRD, fsnotify): - bep_engine.go, bep_model.go, bep_provider.go, bep_receiver.go 52 tests covering proto round-trips, TLS cert gen, wire framing, version vector logic, index scanning/diffing/persistence, and events. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Move all CogField data types, pure functions, and interfaces into a standalone package at pkg/cogfield/. The kernel retains HTTP handlers, adapter implementations, and constellation/reconcile integration, importing types from the package via aliases. Extracted: Node, Edge, Stats, Graph, Block, GraphBlock, BlockAdapter, AdapterNodeConfig, BlockTypeConfig, ExpandNodeResponse, BusDetail, BusRegistryEntry, SessionMessage, SessionDetail, DocumentDetail, DocRef, FieldCondition, TriggeredCondition, FieldConditionState, SignalFieldState, PersistedSignal, SessionJSONLEvent. Pure functions: NormalizeEntityType, InferSector, StrengthFromMetrics, ParseCSVSet, FilterNodes, BFSSubgraph, ComputeStats, FilterByMeta, GraphBlockToNode, ParseConditionQueryString, EvaluateFieldConditions, ComputeRelevance, SignalIsActive, ExtractTimestamp. 21 tests covering construction, validation, JSON round-trip, BFS, condition evaluation, and signal decay. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Thin observation-assessment-execution loop using a local model (Gemma E4B via Ollama) through the OpenAI chat completions wire protocol. Runs as a goroutine inside the kernel process with kernel-native tool dispatch. - AgentHarness: assess (JSON mode) + execute (tool loop) + RunCycle - 6 core tools: memory_search/read/write, coherence_check, bus_emit, workspace_status - Self-contained wire protocol types (no harness package import) - MaxTurns safety limit, context cancellation, tool error recovery - 10 tests with httptest mock servers Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…e Code sessions Adds a transparent proxy at POST /v1/messages that accepts Anthropic Messages API requests, forwards them to the upstream Anthropic API, and streams SSE responses back to the client. This enables Claude Code to route through the kernel via ANTHROPIC_BASE_URL=http://localhost:5100. - serve_messages.go: proxy handler with streaming/non-streaming support, header forwarding, API key resolution, bus event emission, and Anthropic-format error responses - serve_messages_test.go: 8 tests covering non-streaming, streaming, error passthrough, header forwarding, auth, bearer auth, unreachable upstream, and tool_use event counting - serve.go: mux registration and banner output - cog.go: `cog claude` command that checks kernel health and launches claude with ANTHROPIC_BASE_URL set, falling back to direct launch Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- agent_serve.go: ServeAgent runs E4B via native harness on 30-min ticker - Gathers workspace observations (git, memory, coherence) - Calls E4B for assessment (sleep/consolidate/repair/observe/escalate) - Executes actions through kernel-native tool functions - Emits cycle events to CogBus - Configurable via COG_AGENT_INTERVAL, COG_AGENT_MODEL env vars - serve_daemon.go: Wired into cog serve alongside reconciler - serve.go: defaultServePort changed from 5100 to 6931 (5100 decommed) - cog.go: cog claude uses single port check against 6931 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Agent loop starts at 5m, doubles toward 30m on consecutive "sleep" assessments, resets to 5m when action is needed (homeostatic backoff) - Strip <think>...</think> tags from model responses before JSON parsing (Gemma 4 sometimes wraps output in thinking blocks even in JSON mode) - Tighter system prompt: no markdown, no explanation, JSON only Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Use /api/chat instead of /v1/chat/completions (OpenAI compat) - Set think: false explicitly to control thinking at the source - Use format: "json" instead of response_format for structured output - Arguments field is now json.RawMessage (Ollama returns object, not string) - Remove stripThinkingTags — no longer needed when thinking is disabled properly - Update all tests for native response format Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…e via MCP and HTTP Closes Agent F gap #4 (chat history / turn replay API) by making the prompt → response pair a first-class, hash-chained ledger citizen. Also closes cogos#20 (RecordBlock data-loss) inline — RecordBlock stays untouched (its metadata-only ingest receipt is used by other call sites) and the new turn.completed event is the authoritative capture point for the conversation text that was previously dropped on the floor. Design: Agent R hybrid (recommendation C, survey cog://mem/semantic/surveys/2026-04-21-consolidation/agent-R-chat-history-design). What lands: - internal/engine/turn_storage.go — TurnRecord type, Process.RecordTurn method that (1) writes the full turn to a per-session sidecar JSONL at .cog/run/turns/<sessionID>.jsonl and (2) appends a truncated- preview turn.completed event to the hash-chained ledger. UTF-8-safe truncation; defaults: 8 KB prompt preview, 16 KB response preview. NextTurnIndex primes from the sidecar on first access so restarts don't reset the counter. - internal/engine/conversation_query.go — QueryConversation wrapper that scans the session ledger for turn.completed events and optionally hydrates each with the full sidecar row. Slots in cleanly where Agent L's QueryLedger will land; until then, does the O(N) session-scoped scan locally (cheap for typical sessions). - serve.go / serve_anthropic.go — handleChat and handleAnthropicMessages now prepare a TurnRecord at the top of the handler, thread it into completeChat/streamChat/completeAnthropicMessages/streamAnthropicMessages (streaming paths accumulate the response via strings.Builder) and RecordTurn after the response finishes flushing. GET /v1/conversation is a thin wrapper over QueryConversation with query-param filters (session_id, after_turn, before_turn, since, limit, include_full, include_tools, order). - mcp_server.go — new cog_read_conversation MCP tool delegates to QueryConversation. Completes the cog_read_* family (ledger, cogdoc, events, conversation). - tool_loop.go — new RunToolLoopWithTranscript variant captures a []ToolCallRecord transcript (kernel executions + rejections, with duration) so handlers can include per-turn kernel-tool detail in the sidecar. The original RunToolLoop signature is preserved as a 1-line delegator to avoid churn in existing call sites / tests. Integration notes: - The turn.completed event rides on the standard AppendEvent path, so Agent N's broker (PR #16) will fan it out on the live event bus for free the moment it lands — cog_tail_events event_type=turn.completed becomes live turn replay with no extra code. Agent L's hash chain automatically covers conversation history. - context.assembly events keep emitting as before; the cross-link is one-way by timestamp proximity and deferred to a follow-up (kept out of scope to keep this PR mechanical). - RecordBlock and cogblock_ledger.go are intentionally UNTOUCHED. cogblock.ingest stays as-is (block_id, kind, message_count) because other ingest paths depend on that shape. RecordTurn is the new parallel path for the full turn content. Tests (22 new, all pass under -race): - TestWriteTurnCompleted{HappyPath,PromptTruncation,ResponseTruncation, SidecarAppendDurable,Concurrent} — writer + sidecar + ledger round-trip. - TestTruncateUTF8Bytes_Boundary — never slices inside a multibyte rune. - TestNextTurnIndexPrimingFromSidecar — restart scenario. - TestRecordTurn{StatusDefault,NilGuards,TimestampRespected} — defensive. - TestQuery{EmptySession,SingleSessionTurnsOrdered,AfterTurnPagination, IncludeFullFalse,BadFilterCombo,BadOrder,LimitTruncation,SinceFilter, IncludeFullHydratesFromSidecar,ToolCallsIncludedByDefault, CountMatchesTruncationFlag,CrossSessionIgnored} — reader semantics. Full suite (380 tests) passes: go build ./..., go vet ./..., and go test ./internal/engine/... -short -count=1 -race all clean. Closes #20 Closes gap #4

- Invert write path: AppendEvent now runs under the registry lock before the in-memory mutation commits (register/heartbeat/end/offer/claim/ complete). Bus is ground truth; failed appends leave the registry untouched. - Hold lock across append in claim path so first-wins is enforced on the authoritative bus, not just the in-memory registry. - ApplyHeartbeat: check Ended before mutating LastSeen. 409 heartbeats no longer leave stray state. - Tighten session_id regex to require three hyphen-separated components (matches the design spec) and reject consecutive hyphens. - Sort events by Seq ascending on replay for determinism under out-of-order on-disk lines. - Reject caller-supplied handoff_id on offer; kernel always mints. - Reject ttl_seconds < 0 on offer. - Always include conflicting_session in handoff.claim_rejected payload (empty string when not applicable) — stable audit schema per amendment #4. - HandoffRegistry switched to sync.RWMutex; read paths use RLock. - HTTP offer unified to a single time.Now() for minted id + stored created_at. - MCP tool description restored to "3-component required." - 11 new tests: TestRegistryUnchangedOnBusAppendFailure (7 sub-tests covering each mutation path), TestHeartbeatOnEndedSessionDoesNotMutate, TestReplay_EmptyBuses, TestReplay_UnknownEventType, TestReplay_OutOfOrderSeq, TestReplay_DuplicateSeqWithConflictingPayload, TestReplay_CompleteWithoutClaim, TestSessionRegister_TwoComponentIDRejected, TestHandoffOffer_NegativeTTLRejected, TestHandoffOffer_CallerSuppliedIDRejected, TestRoutes_PresenceDoesNotShadowContext. - sessions_test.go header comment updated to 26 tests (was 14 while file had 15, per codex nit). Design calls: - Duplicate-seq replay policy: first-occurrence wins (codified in TestReplay_DuplicateSeqWithConflictingPayload). Matches what ReadEvents already does via its seen[Seq] check. - Orphaned handoff.complete with no prior claim: leave handoff in state Open, skip the transition (TestReplay_CompleteWithoutClaim). Matches the existing replay guard `if row.State == HandoffStateClaimed`. Codex review: /tmp/codex-pr43-review.md

… gap Closes Agent F gap #3 (session management, CRITICAL) — the last of the eight critical MCP surface gaps. Implements the hybrid design in cog://mem/semantic/surveys/2026-04-21-consolidation/ agent-P-session-management-evaluation with a few user-approved amendments (see below). Kernel changes -------------- - internal/engine/sessions.go: typed SessionState, SessionRegistry, HandoffState, HandoffRegistry with RWMutex / Mutex guards; session-id format validation; idempotent-register "update semantics"; atomic first-wins claim with TTL enforcement; replay-from-bus at startup so the in-memory view rebuilds from bus_sessions + bus_handoffs. - internal/engine/serve_sessions_mgmt.go: 8 HTTP handlers — POST /v1/sessions/register POST /v1/sessions/{id}/heartbeat POST /v1/sessions/{id}/end GET /v1/sessions/presence POST /v1/handoffs/offer GET /v1/handoffs POST /v1/handoffs/{id}/claim POST /v1/handoffs/{id}/complete The existing /v1/sessions and /v1/sessions/{id}[/context] routes (TAA inference context, regression-locked) are preserved untouched; the new specific patterns coexist thanks to Go 1.22 method-aware routing. - internal/engine/mcp_sessions.go: 8 cog_* MCP tools over the same registries so a future native client (Wave widget, desktop app, cog CLI) can use handoff without the Python bridge (amendment #5 — two MCP surfaces coexist by design). - internal/engine/sessions_test.go: 15 unit + integration tests (validation, lifecycle 404/409, active-window presence, task-field validation, 8-way concurrent claim atomicity, TTL expiry, phantom offer, complete-without-claim, replay rebuilds state, claim_rejected observability, end-to-end MCP round-trip). Amendments applied vs the survey -------------------------------- 1. No parallel coexistence. All four consumers are in-tree (this PR, the bridge on a local branch, the skill doc, and cmd_bus.go); migrated atomically. The survey's Open Question #1 was skipped. 2. Idempotent register = update-semantics (survey's Open Question #2 recommendation). Re-register during the active window updates the in-memory row; re-register after end is allowed if the prior row is ended or its heartbeat is outside the active window. 3. `handoff.claim_rejected` event emitted on every rejected claim, with reason ∈ {already_claimed, ttl_expired, offer_not_found, out_of_order}, attempting_session, and conflicting_session when relevant. Cheap; big audit value (amendment #4). 4. Two MCP surfaces coexist by design — 8 cogos_* bridge tools over the Python sandbox + 8 cog_* kernel-native tools via /mcp. Both hit the same kernel registries (amendment #5). Bridge migration ---------------- A paired local branch on cog-sandbox-mcp (`feat/sessions-kernel-native- bridge`, NOT pushed) refactors the 8 cogos_* tools to shim over the new kernel routes, removes client-side aggregation, and rewrites tests/test_session_handoff.py for the new wire shape. Bridge MCP signatures and the never-raise {"success": False, "error": ..., "bus_id"} envelope are preserved — no breaking change for agents using the bridge. Testing ------- - `go build ./...`, `go vet ./...`: silent. - `go test ./internal/engine/... -short -race -count=1`: green (pre-existing + new suite passes under race detector). - End-to-end smoke on port 6932 with a test workspace: register → heartbeat → offer → list → claim → second claim (→ 409 + claim_rejected event) → complete. bus_sessions chain: 3 events. bus_handoffs chain: 4 events (offer, claim, claim_rejected, complete). Bridge tools replayed the same flow against the live kernel with back-compat response shapes intact. - The running kernel at :6931 was NOT touched during this work. Survey reference: cog://mem/semantic/surveys/2026-04-21-consolidation/ agent-P-session-management-evaluation

chazmaniandinkle and others added 12 commits April 14, 2026 20:47

chazmaniandinkle merged commit 3e6f1e2 into cogos-dev:main Apr 15, 2026
5 checks passed

This was referenced Apr 21, 2026

bug: RecordBlock persists only metadata; chat prompt/response text is GC'd every turn #20

Closed

note: turn_metrics.jsonl is fossil data; no current writer in internal/engine/ #21

Closed

feat(engine): persist conversation turns via ledger + sidecar #24

Merged

chazmaniandinkle mentioned this pull request Apr 22, 2026

feat: kernel-native session management (hybrid) — closes last Agent F gap #43

Merged

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: library extraction (7 packages) + native agent harness#4

feat: library extraction (7 packages) + native agent harness#4
chazmaniandinkle merged 12 commits intocogos-dev:mainfrom
chazmaniandinkle:feat/library-extraction-agent-harness

chazmaniandinkle commented Apr 15, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant