Skip to content

Fix memory pipeline quality: ranking, LLM gating, decay, noise suppression#22

Merged
CalebisGross merged 3 commits intomainfrom
fix/audit-pipeline-fixes
Mar 1, 2026
Merged

Fix memory pipeline quality: ranking, LLM gating, decay, noise suppression#22
CalebisGross merged 3 commits intomainfrom
fix/audit-pipeline-fixes

Conversation

@CalebisGross
Copy link
Copy Markdown
Collaborator

Summary

Addresses the core pipeline issues identified by the system audit — the memory system was amplifying noise instead of filtering it.

  • FTS5 BM25 ranking: SearchByFullText now uses ORDER BY rank instead of insertion order, so recall returns the most relevant matches first
  • Dreaming agent LLM gate: Skip the entire dream cycle when LLM is unavailable — without it, dreaming was blindly strengthening associations on junk data
  • Consolidation LLM gate: Skip merge and pattern extraction steps when LLM is down, but still run DB-only maintenance (decay, transitions, pruning, expiry)
  • Pattern dedup threshold: Lowered from 0.8 to 0.70 cosine similarity to catch near-duplicate patterns that were slipping through
  • Decay tuning: Reduced recency protection from 0.5x to 0.8x (24h window) and 0.8x to 0.9x (7d window) so low-quality memories fade in ~2 cycles instead of 4+
  • Heuristic noise suppression: Added path-aware zero-scoring for application-internal directories (Chrome/Chromium, LM Studio, GNOME dconf, Trash, browser storage dirs)
  • Hook bugfix: Fixed protect-git.sh false positive — regex matched -f in branch names like fix/audit-pipeline-fixes
  • go fmt: Reformatted 23 files that had drifted from standard formatting

Test plan

  • make build passes
  • make check (go fmt + go vet) passes
  • make test — all tests pass, including updated decay test expectations
  • Verify with live daemon: nuke DB, restart, confirm Chrome/LM Studio events get 0.0 score in logs
  • Verify dream cycle logs "skipping dream cycle: LLM unavailable" when LM Studio is stopped
  • Verify FTS recall returns more relevant results on real queries

🤖 Generated with Claude Code

CalebisGross and others added 3 commits February 28, 2026 08:32
…ppression

- Use FTS5 BM25 ranking in SearchByFullText instead of insertion order
- Gate dreaming agent on LLM availability to prevent blind association boosting
- Skip consolidation merge/pattern steps when LLM is unavailable
- Lower pattern dedup cosine threshold from 0.8 to 0.70 to catch near-duplicates
- Reduce recency protection from 0.5x to 0.8x so noise fades faster
- Add path-aware suppression for app-internal dirs (Chrome, LM Studio, GNOME, etc.)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The regex (-f|--force) matched -f anywhere in the command, including
branch names like fix/audit-pipeline-fixes. Added word boundaries.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@CalebisGross CalebisGross merged commit e2fa1d5 into main Mar 1, 2026
3 checks passed
@CalebisGross CalebisGross deleted the fix/audit-pipeline-fixes branch March 1, 2026 12:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant