Skip to content

feat(ci): daily LLM-backed upstream awareness digest #200

@randomm

Description

@randomm

Concept

A GitHub Action runs daily, compares anomalyco/dev to our last synced state, and uses an LLM to summarize what's new upstream — giving us awareness without automating the sync itself.

The sync process requires careful human + agent judgment and should remain manual. But knowing what upstream is doing is pure information that can be automated.

Why Not Full Automation

The /sync-upstream command handles the actual merge process well. Full automation of sync is risky because:

  • Conflict resolution requires judgment about our fork-specific features
  • Absorption decisions (should we adopt upstream's version?) need human input
  • The process is too consequential to run unattended

What This Would Do

Daily GitHub Action:

  1. Fetch anomalyco/dev
  2. Get commits we haven't absorbed: git log anomalyco/dev ^origin/dev --oneline
  3. Get changed file summary: git diff anomalyco/dev...origin/dev --stat
  4. Feed to LLM (Anthropic API) with a prompt asking it to:
    • Categorize changes (bug fixes, new features, breaking changes, docs)
    • Flag anything touching files we've modified in our fork (cross-reference .fork-features/manifest.json)
    • Highlight anything that looks like it could conflict with our fork features
    • Identify anything worth absorbing proactively
  5. Post digest as a comment on a pinned tracking issue (so it's searchable and notifies)

Open Questions

  • Output destination: Comment on a pinned 'upstream watch' issue vs. new issue per digest vs. something else?
  • LLM: Anthropic API directly in a Bun script (simplest, auditable) vs. opencode headless (dogfooding but complex in CI)
  • Prompt quality: How to get genuinely useful signal vs. noise — needs careful prompt engineering
  • Frequency: Daily may be too noisy if upstream is very active. Weekly?
  • ANTHROPIC_API_KEY secret: Needs to be added to repo secrets

Implementation Sketch

.github/workflows/upstream-watch.yml   # Daily cron workflow
script/upstream-digest.ts              # Bun script: git diff + LLM call + post to GitHub

The script would be a simple Bun script calling Anthropic API directly — no framework, fully auditable.

References

Status

Draft — concept agreed, implementation not yet designed. Resolve open questions before starting.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions