Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,4 @@ coverage.out
.claude/settings.local.json
.private-journal/
a/
runs/
27 changes: 27 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,33 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Added

- **Library API hardening for v1.0** (#102, #103, #104, #106, #109):
- Typed enum-like strings for `CheckStatus` and `SuggestionKind` so consumers can switch-exhaust. Existing constants (`SuggestionRetryPattern`, etc.) retain their underlying string values.
- `tracker.WithVersionInfo(version, commit)` functional option replaces the CLI-only `DoctorConfig.TrackerVersion` / `TrackerCommit` fields.
- `DiagnoseConfig.LogWriter` / `AuditConfig.LogWriter` — optional `io.Writer` for non-fatal parse warnings. Nil is treated as `io.Discard` so library callers no longer see stray warnings on `os.Stderr`. The `tracker` CLI sets this to `io.Discard` for user-facing commands. `Doctor` has no warnings to suppress so it deliberately does not carry a `LogWriter` field.
- `Doctor`, `Diagnose`, `DiagnoseMostRecent`, `Audit`, `Simulate` now accept `context.Context`, honored by provider probes and binary version lookups. `getBinaryVersion` now uses `exec.CommandContext` with a 5-second timeout, matching `getDippinVersion`.
- Provider probe error bodies are now sanitized (API keys and bearer tokens stripped) before they land in `CheckDetail.Message`.
- `NDJSON` handler closures (pipeline, agent, LLM trace) now `recover()` from panics in the underlying writer so a misbehaving sink cannot crash the caller goroutine. Panic suppression is per-`NDJSONWriter` instance (not package-level), so one misbehaving sink cannot silence unrelated writers in the same process.
- `Diagnose` now streams `activity.jsonl` with `bufio.Scanner` instead of `os.ReadFile` → `strings.Split`, matching `LoadActivityLog` and avoiding a memory spike on large runs. Scanner errors (1 MB line-length overflow, I/O) and `ctx.Err()` now propagate out of `Diagnose` as a real error — partial reports are never returned as success, so automation with deadlines can distinguish complete from truncated analysis.

### Changed

- **BREAKING** (library):
- `tracker.Doctor(cfg)` → `tracker.Doctor(ctx, cfg, opts...)`.
- `tracker.Diagnose(runDir)` → `tracker.Diagnose(ctx, runDir, opts...)`.
- `tracker.DiagnoseMostRecent(workdir)` → `tracker.DiagnoseMostRecent(ctx, workdir, opts...)`.
- `tracker.Audit(runDir)` → `tracker.Audit(ctx, runDir)`. (No config struct — Audit emits no suppressible warnings. Use `ListRuns` + `AuditConfig{LogWriter}` for bulk enumeration.)
- `tracker.Simulate(source)` → `tracker.Simulate(ctx, source)`.
- `tracker.ListRuns(workdir)` now accepts optional `...AuditConfig`.
- `tracker.NDJSONEvent` → `tracker.StreamEvent`. Wire-format JSON tags unchanged.
- `NDJSONWriter.Write` now returns `error` so callers can detect a broken stream. First failure is still logged to `os.Stderr` once (unchanged behavior); subsequent failures are surfaced via the return value.
- `DoctorConfig.TrackerVersion` and `DoctorConfig.TrackerCommit` removed — use `tracker.WithVersionInfo(version, commit)` instead.
- `CheckResult.Status` and `CheckDetail.Status` are now typed as `tracker.CheckStatus` (underlying string). Untyped string literal comparisons (`status == "ok"`) keep working.
- `Suggestion.Kind` is now typed as `tracker.SuggestionKind` (underlying string).
- `tracker diagnose` suggestion order is now deterministic (alphabetical by node ID). Previously suggestions printed in Go map-iteration order, which varied between runs.

### Fixed

- **OpenAI Responses API: `function_call_output` and `function_call` items now always serialize required fields** (closes #114). Previously the shared `openaiInput` struct used `omitempty` on every field, so a tool returning an empty-string result produced `{"type":"function_call_output","call_id":"..."}` with no `output` field, and a no-argument tool call produced `function_call` with no `arguments`. OpenAI's endpoint tolerated this, but OpenRouter's strict Zod validator rejected the requests with `invalid_prompt` / `invalid_union` errors, symptomatic on GLM, Qwen, and Kimi via OpenRouter. Fixed by replacing the `omitempty`-tagged single struct with a `MarshalJSON` method that emits only fields valid per item type, with required fields always present. Reported by @Nopik.
Expand Down
11 changes: 8 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -574,9 +574,14 @@ result, _ := tracker.Run(ctx, source, tracker.Config{
### Analyzing past runs from code

```go
import tracker "github.com/2389-research/tracker"
import (
"context"

report, err := tracker.DiagnoseMostRecent(".")
tracker "github.com/2389-research/tracker"
)

ctx := context.Background()
report, err := tracker.DiagnoseMostRecent(ctx, ".")
if err != nil { log.Fatal(err) }

for _, f := range report.Failures {
Expand All @@ -588,7 +593,7 @@ for _, s := range report.Suggestions {
}
```

`tracker.Audit`, `tracker.Simulate`, and `tracker.Doctor` follow the same pattern and return JSON-serializable reports.
`tracker.Audit`, `tracker.DiagnoseMostRecent`, `tracker.Simulate`, and `tracker.Doctor` all accept `context.Context` as their first argument and return JSON-serializable reports. `tracker.ListRuns` and `DiagnoseMostRecent`/`Diagnose` accept an optional config (`AuditConfig`, `DiagnoseConfig`) with a `LogWriter` for non-fatal parse warnings — set it to `io.Discard` to silence warnings in embedded callers. `Audit` and `Simulate` currently take just `ctx` (plus their payload); `Doctor` takes a required `DoctorConfig` plus optional functional options (e.g., `tracker.WithVersionInfo`).

To stream events programmatically in the same NDJSON format as `tracker --json`, use `tracker.NewNDJSONWriter`:

Expand Down
6 changes: 4 additions & 2 deletions cmd/tracker/audit.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,17 @@
package main

import (
"context"
"fmt"
"io"
"time"

tracker "github.com/2389-research/tracker"
)

// listRuns shows all available runs with their status and node count.
func listRuns(workdir string) error {
runs, err := tracker.ListRuns(workdir)
runs, err := tracker.ListRuns(workdir, tracker.AuditConfig{LogWriter: io.Discard})
if err != nil {
return err
}
Expand Down Expand Up @@ -55,7 +57,7 @@ func runAudit(workdir, runID string) error {
if err != nil {
return err
}
report, err := tracker.Audit(runDir)
report, err := tracker.Audit(context.Background(), runDir)
if err != nil {
return err
}
Expand Down
6 changes: 4 additions & 2 deletions cmd/tracker/diagnose.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,9 @@
package main

import (
"context"
"fmt"
"io"
"os"
"sort"
"strings"
Expand All @@ -14,7 +16,7 @@ import (

// diagnoseMostRecent finds and diagnoses the most recent run.
func diagnoseMostRecent(workdir string) error {
report, err := tracker.DiagnoseMostRecent(workdir)
report, err := tracker.DiagnoseMostRecent(context.Background(), workdir, tracker.DiagnoseConfig{LogWriter: io.Discard})
if err != nil {
return err
}
Expand All @@ -28,7 +30,7 @@ func runDiagnose(workdir, runID string) error {
if err != nil {
return err
}
report, err := tracker.Diagnose(runDir)
report, err := tracker.Diagnose(context.Background(), runDir, tracker.DiagnoseConfig{LogWriter: io.Discard})
if err != nil {
return err
}
Expand Down
7 changes: 3 additions & 4 deletions cmd/tracker/doctor.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
package main

import (
"context"
"fmt"
"os"
"path/filepath"
Expand Down Expand Up @@ -73,14 +74,12 @@ func runDoctorWithConfig(workdir string, cfg DoctorConfig) error {
workdir = wd
}

report, err := tracker.Doctor(tracker.DoctorConfig{
report, err := tracker.Doctor(context.Background(), tracker.DoctorConfig{
WorkDir: workdir,
Backend: cfg.backend,
ProbeProviders: cfg.probe,
PipelineFile: cfg.pipelineFile,
TrackerVersion: version,
TrackerCommit: commit,
})
}, tracker.WithVersionInfo(version, commit))
if err != nil {
return err
}
Expand Down
3 changes: 2 additions & 1 deletion cmd/tracker/simulate.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
package main

import (
"context"
"fmt"
"io"
"os"
Expand Down Expand Up @@ -58,7 +59,7 @@ func runSimulateCmd(pipelineFile, formatOverride string, w io.Writer) error {
source = string(data)
}

report, err := tracker.Simulate(source)
report, err := tracker.Simulate(context.Background(), source)
if err != nil {
return err
}
Expand Down
4 changes: 2 additions & 2 deletions tracker_activity.go
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@ func ParseActivityLine(line string) (ActivityEntry, bool) {
if err := json.Unmarshal([]byte(line), &raw); err != nil {
return ActivityEntry{}, false
}
ts, ok := parseActivityTimestampLib(raw.Timestamp)
ts, ok := parseActivityTimestamp(raw.Timestamp)
if !ok {
return ActivityEntry{}, false
}
Expand All @@ -173,7 +173,7 @@ func ParseActivityLine(line string) (ActivityEntry, bool) {
}, true
}

func parseActivityTimestampLib(s string) (time.Time, bool) {
func parseActivityTimestamp(s string) (time.Time, bool) {
if ts, err := time.Parse(time.RFC3339Nano, s); err == nil {
return ts, true
}
Expand Down
47 changes: 42 additions & 5 deletions tracker_audit.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,9 @@
package tracker

import (
"context"
"fmt"
"io"
"os"
"path/filepath"
"sort"
Expand All @@ -12,6 +14,15 @@ import (
"github.com/2389-research/tracker/pipeline"
)

// AuditConfig configures an Audit() or ListRuns() call.
type AuditConfig struct {
// LogWriter receives non-fatal warnings (unreadable activity.jsonl
// in a run directory, etc.). Nil is treated as io.Discard so
// embedded library callers do not see warnings on os.Stderr. The
// tracker CLI sets this to io.Discard for user-facing commands.
LogWriter io.Writer
}

// AuditReport is the structured result of Audit().
type AuditReport struct {
RunID string `json:"run_id"`
Expand Down Expand Up @@ -68,7 +79,24 @@ type RunSummary struct {
// and activity.jsonl directly under it. For user-supplied input, resolve
// the path via ResolveRunDir or use MostRecentRunID first, which enforce
// the .tracker/runs/<runID> layout.
func Audit(runDir string) (*AuditReport, error) {
//
// ctx is checked at entry so a caller that passes an already-cancelled
// context gets an immediate error instead of silent work. Full
// cancellation mid-parse would require threading ctx through
// pipeline.LoadCheckpoint and LoadActivityLog, which is out of scope
// today (both are fast and bounded). Nil is coalesced to
// context.Background().
//
// Audit does not accept AuditConfig — it emits no warnings to suppress.
// Use ListRuns + AuditConfig{LogWriter} for bulk enumeration where the
// summary builder may skip unreadable activity logs.
func Audit(ctx context.Context, runDir string) (*AuditReport, error) {
if ctx == nil {
ctx = context.Background()
}
if err := ctx.Err(); err != nil {
return nil, err
}
cp, err := pipeline.LoadCheckpoint(filepath.Join(runDir, "checkpoint.json"))
if err != nil {
return nil, fmt.Errorf("load checkpoint: %w", err)
Expand Down Expand Up @@ -98,7 +126,9 @@ func Audit(runDir string) (*AuditReport, error) {
}

// ListRuns returns all runs under workdir/.tracker/runs, sorted newest first.
func ListRuns(workdir string) ([]RunSummary, error) {
func ListRuns(workdir string, opts ...AuditConfig) ([]RunSummary, error) {
cfg := firstAuditConfig(opts)
logW := logWriterOrDiscard(cfg.LogWriter)
runsDir := filepath.Join(workdir, ".tracker", "runs")
entries, err := os.ReadDir(runsDir)
if err != nil {
Expand All @@ -112,7 +142,7 @@ func ListRuns(workdir string) ([]RunSummary, error) {
if !e.IsDir() {
continue
}
rs, ok := buildLibRunSummary(runsDir, e.Name())
rs, ok := buildRunSummary(runsDir, e.Name(), logW)
if ok {
runs = append(runs, rs)
}
Expand All @@ -121,6 +151,13 @@ func ListRuns(workdir string) ([]RunSummary, error) {
return runs, nil
}

func firstAuditConfig(opts []AuditConfig) AuditConfig {
if len(opts) == 0 {
return AuditConfig{}
}
return opts[0]
}

func classifyStatus(cp *pipeline.Checkpoint, activity []ActivityEntry) string {
for i := len(activity) - 1; i >= 0; i-- {
switch activity[i].Type {
Expand Down Expand Up @@ -213,15 +250,15 @@ func buildAuditRecommendations(cp *pipeline.Checkpoint, status string, total tim
return recs
}

func buildLibRunSummary(runsDir, name string) (RunSummary, bool) {
func buildRunSummary(runsDir, name string, logW io.Writer) (RunSummary, bool) {
runDir := filepath.Join(runsDir, name)
cp, err := pipeline.LoadCheckpoint(filepath.Join(runDir, "checkpoint.json"))
if err != nil {
return RunSummary{}, false
}
activity, lerr := LoadActivityLog(runDir)
if lerr != nil {
fmt.Fprintf(os.Stderr, "warning: run %s: cannot read activity log: %v\n", name, lerr)
fmt.Fprintf(logW, "warning: run %s: cannot read activity log: %v\n", name, lerr)
activity = nil // continue with nil so the summary still builds
}
SortActivityByTime(activity)
Expand Down
43 changes: 41 additions & 2 deletions tracker_audit_test.go
Original file line number Diff line number Diff line change
@@ -1,13 +1,15 @@
package tracker

import (
"bytes"
"context"
"os"
"path/filepath"
"testing"
)

func TestAudit_CompletedRun(t *testing.T) {
r, err := Audit("testdata/runs/ok")
r, err := Audit(context.Background(), "testdata/runs/ok")
if err != nil {
t.Fatalf("Audit: %v", err)
}
Expand All @@ -23,7 +25,7 @@ func TestAudit_CompletedRun(t *testing.T) {
}

func TestAudit_FailedRun(t *testing.T) {
r, err := Audit("testdata/runs/failed")
r, err := Audit(context.Background(), "testdata/runs/failed")
if err != nil {
t.Fatalf("Audit: %v", err)
}
Expand Down Expand Up @@ -65,3 +67,40 @@ func TestListRuns_MultipleRuns(t *testing.T) {
t.Errorf("first = %q, want r2 (newest first)", runs[0].RunID)
}
}

func TestListRuns_LogWriterSilencesWarnings(t *testing.T) {
// Build a run directory whose checkpoint loads fine but whose activity.jsonl
// is unreadable (EISDIR). buildRunSummary should emit a warning to the
// LogWriter rather than os.Stderr.
workdir := t.TempDir()
runsDir := filepath.Join(workdir, ".tracker", "runs")
must(t, os.MkdirAll(filepath.Join(runsDir, "r1"), 0o755))
must(t, os.WriteFile(filepath.Join(runsDir, "r1", "checkpoint.json"),
[]byte(`{"run_id":"r1","completed_nodes":["A"],"timestamp":"2026-04-17T10:00:00Z"}`), 0o644))
// Make activity.jsonl a directory so os.ReadFile fails with EISDIR.
must(t, os.MkdirAll(filepath.Join(runsDir, "r1", "activity.jsonl"), 0o755))

var logBuf bytes.Buffer
runs, err := ListRuns(workdir, AuditConfig{LogWriter: &logBuf})
if err != nil {
t.Fatalf("ListRuns: %v", err)
}
if len(runs) != 1 {
t.Fatalf("got %d runs, want 1", len(runs))
}
if logBuf.Len() == 0 {
t.Error("expected log writer to receive a warning about activity.jsonl")
}
}

// TestAudit_CtxCancelledAtEntry verifies Audit returns the caller's
// cancellation error immediately rather than silently proceeding with the
// expensive checkpoint + activity log reads.
func TestAudit_CtxCancelledAtEntry(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
cancel()
_, err := Audit(ctx, "testdata/runs/ok")
if err != context.Canceled {
t.Errorf("err = %v, want context.Canceled", err)
}
}
Loading
Loading