Skip to content

chore(v5): drop orphan join tables + citation-tracking code (PR-v5-1, US1)#179

Merged
thebtf merged 3 commits into
mainfrom
feat/v5-us1-orphan-joins
Apr 17, 2026
Merged

chore(v5): drop orphan join tables + citation-tracking code (PR-v5-1, US1)#179
thebtf merged 3 commits into
mainfrom
feat/v5-us1-orphan-joins

Conversation

@thebtf
Copy link
Copy Markdown
Owner

@thebtf thebtf commented Apr 17, 2026

PR-v5-1: drop orphan join tables (US1, first destructive PR of v5.0.0)

First destructive PR of the engram v5.0.0 BREAKING cleanup sequence.
SAFEST starting point: 2 orphan tables, no static-entity impact, no vault impact.

Scope

Drop session_observation_injections + injection_log (both derived, orphan per audit) and all Go code referencing them.

What landed

Area Delta
Migrations 083_drop_session_observation_injections + 084_drop_injection_log (IDs in spec said 081/082 but those were already taken)
Go code 10 files · +83/-625 · net -542 LOC
Build / vet / test all green locally

Verification (local, run in worktree off e9c275d)

  • Ripgrep over Go sources for both table names returns only historical migration entries + CHANGELOG refs; zero live-code hits
  • go build ./... → clean
  • go vet ./... → clean
  • go test -short ./... → 40+ packages, all ok, 0 failures
  • Manual scripts/safety-gate.ps1 against live vault → OK (fingerprint=aa78e55cf896508c, credentials=13, mismatches=0) (unchanged baseline)

Known stubs (intentional, tracked for US7)

Two maintenance functions reduced to no-op return (0, nil):

  • Service.analyzeHitRate(ctx) in internal/maintenance/hit_rate.go
  • Service.adjustAdaptiveThresholds(ctx) in internal/maintenance/service.go

Rationale: maintenance subsystem drops in its entirety in US7 (PR-v5-7). Callers (scheduler) still run in the window between US1 and US7. Stubbing is cleaner than dangling references. Both have explanatory comments.

Invariants preserved (NFR-2)

Vault + content table + all static entities UNCHANGED. This PR modifies only derived/orphan storage + related code. No migrations touch any static entity table. Manual safety-gate run post-commit confirms aa78e55cf896508c / 13 / 0 baseline.

Known deviation from spec

Migration IDs 081/082 (per spec) were already taken by 081_project_identity_pure_hash and 082_projects_lifecycle. Landed as 083/084. Downstream planned IDs (084_drop_observations, 085_drop_user_prompts, etc.) need +2 renumbering — will be handled inline in those PRs' task briefs.

Next

After merge: PR-v5-2 (drop content_chunks + vector/embedding packages — US2).


Spec: .agent/specs/engram-v5-baseline/spec.md US1 · Phase 3 · Commit 0a858e0

Summary by CodeRabbit

  • Удалённые функции

    • Отключено логирование инъекций, подсчёт разнообразия и пометка цитирования; удалены связанные операции и фоновые задачи.
    • Удалено синхронное/асинхронное сохранение сеансовых инъекций и получение инъецированных наблюдений.
  • Изменения в обработчиках

    • Удалены API-эндпоинты для отметки цитирования и получения инъецированных наблюдений; сняты соответствующие вызовы и фоновые триггеры.
  • Поведение и аналитика

    • Аналитика hit-rate и адаптивные пороги теперь no-op; расчёт разнообразия стал no-op без внешних хуков.
  • Изменения базы данных

    • Добавлены миграции, удаляющие таблицы session_observation_injections и injection_log.

… US1)

Migrations:
- 083_drop_session_observation_injections (was 081 in spec; IDs 081/082 taken)
- 084_drop_injection_log (was 082 in spec)
Both: DROP TABLE IF EXISTS with Rollback re-CREATE to original schema.

Go code removal (10 files, -625/+83 = net -542 LOC):
- internal/db/gorm/injection_log_store.go: all 7 methods + struct gone
- internal/db/gorm/scoring_store.go: sessionObservationInjection + 2 methods gone
- internal/worker/handlers_context.go: async LogInjections goroutine gone
- internal/worker/handlers_maintenance.go: cleanupInjectionLog + "injection_log" from derivedTables gone
- internal/worker/handlers_scoring.go: RecordSessionInjections call + handleGetSessionInjectedObservations + handleMarkCited gone
- internal/worker/service.go: /mark-cited and /injected-observations routes gone
- internal/worker/retrieval_helpers.go: lookupDiversityScores production path returns nil; test hook preserved
- internal/maintenance/hit_rate.go: analyzeHitRate stubbed to (0, nil) — full maintenance drop in US7
- internal/maintenance/service.go: adjustAdaptiveThresholds stubbed to (0, nil) — full maintenance drop in US7

Known stubs (remove in US7 when maintenance subsystem drops):
- Service.analyzeHitRate(ctx) → (0, nil)
- Service.adjustAdaptiveThresholds(ctx) → (0, nil)
Both have explanatory comments; callers in maintenance scheduler are in
US7 scope. Cleaner than dangling references for the window between
US1 and US7.

Verification (run locally in worktree off e9c275d):
- rg 'session_observation_injections|injection_log' --type go: only historical
  migration entries + CHANGELOG references; zero live-code hits
- go build ./...: clean
- go vet ./...: clean
- go test -short ./...: 40+ packages, all ok, 0 failures

Invariants preserved (not touched by this PR):
- Vault: internal/crypto/*, handlers_vault.go, tools_credential.go, VaultView.vue
- content table, credentials observation rows, all 6 static entities
- scripts/safety-gate.* (just landed in #178)
- spec/plan/tasks artifacts under .agent/specs/engram-v5-baseline/
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 17, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: c769e1c7-1309-4df4-9a9b-4b1170a0b79a

📥 Commits

Reviewing files that changed from the base of the PR and between e292601 and 72dc0ec.

📒 Files selected for processing (5)
  • internal/db/gorm/injection_log_store.go
  • internal/maintenance/service.go
  • internal/mcp/tools_recall.go
  • internal/worker/handlers_data.go
  • internal/worker/reaper/reaper.go
💤 Files with no reviewable changes (1)
  • internal/db/gorm/injection_log_store.go
✅ Files skipped from review due to trivial changes (3)
  • internal/worker/handlers_data.go
  • internal/worker/reaper/reaper.go
  • internal/mcp/tools_recall.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • internal/maintenance/service.go

Walkthrough

Удалена подсистема учёта инъекций наблюдений: таблицы и модели БД, методы хранилища, HTTP-эндпоинты и связанная логика — всё поведение, читающее/пишущее в injection_log и session_observation_injections, удалено или заменено на no‑op.

Changes

Cohort / File(s) Summary
Удаление модуля логирования инъекций
internal/db/gorm/injection_log_store.go
Файл удалён: убраны методы LogInjection, LogInjections, GetDiversityScores, MarkCited, GetCitationRate, CleanupInjectionLog и тип injectionDiversityRow.
Миграции БД (удаление таблиц)
internal/db/gorm/migrations.go
Добавлены миграции 083 (DROP session_observation_injections) и 084 (DROP injection_log) с откатами, восстанавливающими соответствующие таблицы/индексы.
Удаление поддержки сессионных инъекций (GORM)
internal/db/gorm/scoring_store.go
Удалена модель sessionObservationInjection и методы RecordSessionInjections, GetSessionInjectedObservations (junction table удалён).
Отключение hit-rate и адаптивных порогов
internal/maintenance/hit_rate.go, internal/maintenance/service.go
analyzeHitRate и adjustAdaptiveThresholds заменены на no‑op, возвращают (0, nil), удалены DB‑операции и связанные вычисления.
Удаление вызовов логирования и очистки в воркере
internal/worker/handlers_context.go, internal/worker/handlers_maintenance.go
Убрана асинхронная отправка LogInjections из handleSearchByPrompt; удалён фоновый вызов очистки cleanupInjectionLog; injection_log исключён из purge/rebuild.
Удаление HTTP-эндпоинтов и связанного функционала
internal/worker/handlers_scoring.go, internal/worker/service.go
Удалены обработчики handleGetSessionInjectedObservations, handleMarkCited, тип InjectedObservationResponse и маршруты GET /api/sessions/{sessionId}/injected-observations, POST /api/sessions/{sessionId}/mark-cited; удалён импорт learning.
Изменение получения diversity‑оценок
internal/worker/retrieval_helpers.go
lookupDiversityScores больше не вызывает observationStore.GetDiversityScores по умолчанию: ids собираются только при наличии hook; иначе возвращает (nil, nil) (no‑op).
Текстовые и коммент-обновления
internal/mcp/tools_recall.go, internal/worker/handlers_data.go, internal/worker/reaper/reaper.go
Обновлены сообщения/комментарии, указывающие, что injection_log удалён в v5; без изменений логики выполнения.

Sequence Diagram(s)

(Пропущено — изменения в основном удаляют/отключают подсистему и не вводят новый многокомпонентный поток.)

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 минут

Possibly related PRs

Поэма

🐰 В логах инъекций тишь и гладь,
Таблицы в миграциях спят.
No‑op шепчет: «База чиста», — в ряд,
Я морковкой чиркаю взгляд.
Пусть ревью пройдёт легко и радостно.

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed Название чётко и ёмко отражает главное изменение: удаление orphan join таблиц и кода отслеживания цитирований в v5.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/v5-us1-orphan-joins

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.11.4)

Error: can't load config: unsupported version of the configuration: "" See https://golangci-lint.run/docs/product/migration-guide for migration instructions
The command is terminated due to an error: can't load config: unsupported version of the configuration: "" See https://golangci-lint.run/docs/product/migration-guide for migration instructions


Comment @coderabbitai help to get the list of available commands and usage tips.

@thebtf
Copy link
Copy Markdown
Owner Author

thebtf commented Apr 17, 2026

@coderabbitai review

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 17, 2026

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request performs a cleanup of the injection tracking system by dropping the injection_log and session_observation_injections tables and removing their associated logic across the codebase. Key changes include new database migrations to drop these tables, the removal of the injection_log_store.go file, and converting maintenance tasks like hit-rate analysis and adaptive threshold adjustments into no-ops. A significant issue was identified in the rollback logic for migration 084, which fails to restore the cited column and its corresponding index, potentially causing schema mismatches if the migration is reverted.

Comment thread internal/db/gorm/migrations.go
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
internal/maintenance/service.go (1)

629-631: Устаревший комментарий: ссылка на injection_log.

Комментарий для Task 20 всё ещё упоминает "Reads citation rates from injection_log", хотя функция теперь возвращает no-op. Рекомендуется обновить комментарий для согласованности с текущей реализацией.

📝 Предлагаемое исправление комментария
-	// Task 20: Adaptive threshold adjustment per project (Learning Memory v3 FR-6)
-	// Reads citation rates from injection_log, adjusts per-project thresholds.
-	// Bounds: [0.15, 0.60]. Window: last 50 sessions per project.
+	// Task 20: Adaptive threshold adjustment per project (Learning Memory v3 FR-6)
+	// NOTE: Currently a no-op — injection_log was dropped in v5 (US1).
+	// A follow-up task should replace this with an effectiveness_score-based approach.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@internal/maintenance/service.go` around lines 629 - 631, Update the outdated
Task 20 block comment to reflect the current no-op implementation: remove the
phrase "Reads citation rates from injection_log" and explicitly state that the
adaptive per-project threshold adjustment is currently disabled/no-op while
keeping the descriptive title "Adaptive threshold adjustment per project
(Learning Memory v3 FR-6)" and any notes about bounds [0.15, 0.60] and window
semantics for future reference; locate the comment block labeled "Task 20" in
internal/maintenance/service.go and edit that comment only so it accurately
documents the function's present behavior.
internal/db/gorm/injection_log_store.go (1)

1-3: Рассмотрите полное удаление файла вместо оставления пустого stub.

Файл содержит только объявление пакета и комментарий. Хотя комментарий полезен для контекста, git history уже предоставляет эту информацию.

Два варианта:

  1. Удалить файл полностью — более чистый подход, уменьшает количество файлов в репозитории
  2. Оставить как есть — комментарий даёт быстрый контекст без необходимости смотреть git blame

Оба варианта приемлемы. Это вопрос предпочтений команды.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@internal/db/gorm/injection_log_store.go` around lines 1 - 3, The file
injection_log_store.go is an empty stub containing only the package declaration
and a historical comment; remove this file entirely from the repository to keep
the codebase clean (or, if team prefers to retain context, explicitly state that
decision in a README—but default to deletion). Locate injection_log_store.go
(package symbol: package gorm) and delete it; ensure no imports or references to
injection_log_store.go remain elsewhere and run tests/build to confirm nothing
breaks.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@internal/db/gorm/migrations.go`:
- Around line 2743-2763: The rollback for migration 084 recreates injection_log
without the cited column added in migration 066; update the Rollback func for
migration 084 (the sqls slice inside Rollback) to include the cited BOOLEAN
DEFAULT false column in the CREATE TABLE IF NOT EXISTS injection_log statement
and also add the corresponding index creation for
idx_injection_log_session_cited so the restored table matches the schema
introduced by migration 066.

---

Nitpick comments:
In `@internal/db/gorm/injection_log_store.go`:
- Around line 1-3: The file injection_log_store.go is an empty stub containing
only the package declaration and a historical comment; remove this file entirely
from the repository to keep the codebase clean (or, if team prefers to retain
context, explicitly state that decision in a README—but default to deletion).
Locate injection_log_store.go (package symbol: package gorm) and delete it;
ensure no imports or references to injection_log_store.go remain elsewhere and
run tests/build to confirm nothing breaks.

In `@internal/maintenance/service.go`:
- Around line 629-631: Update the outdated Task 20 block comment to reflect the
current no-op implementation: remove the phrase "Reads citation rates from
injection_log" and explicitly state that the adaptive per-project threshold
adjustment is currently disabled/no-op while keeping the descriptive title
"Adaptive threshold adjustment per project (Learning Memory v3 FR-6)" and any
notes about bounds [0.15, 0.60] and window semantics for future reference;
locate the comment block labeled "Task 20" in internal/maintenance/service.go
and edit that comment only so it accurately documents the function's present
behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 46063481-3319-49f2-90a7-f06464530514

📥 Commits

Reviewing files that changed from the base of the PR and between e9c275d and 0a858e0.

📒 Files selected for processing (10)
  • internal/db/gorm/injection_log_store.go
  • internal/db/gorm/migrations.go
  • internal/db/gorm/scoring_store.go
  • internal/maintenance/hit_rate.go
  • internal/maintenance/service.go
  • internal/worker/handlers_context.go
  • internal/worker/handlers_maintenance.go
  • internal/worker/handlers_scoring.go
  • internal/worker/retrieval_helpers.go
  • internal/worker/service.go
💤 Files with no reviewable changes (4)
  • internal/worker/service.go
  • internal/worker/handlers_context.go
  • internal/db/gorm/scoring_store.go
  • internal/worker/handlers_scoring.go

Comment thread internal/db/gorm/migrations.go
Rollback was missing schema added by migration 066 (learning memory v3):
- cited BOOLEAN DEFAULT false column
- idx_injection_log_session_cited index on (session_id, cited)

Without these, a post-rollback database would lack schema that existed
immediately before DROP, breaking downstream migration replay.
CodeRabbit-flagged:
- Delete injection_log_store.go tombstone (was 3-line package stub referencing dropped table; G1 no-stubs compliance)
- internal/maintenance/service.go: Task 20 + Task 24 comments updated to reflect no-op state + US7 handoff pointer

Additional stale references found during review sweep:
- internal/mcp/tools_recall.go: user-facing 'no hit rate analytics data' message replaced (injection_log no longer exists)
- internal/worker/handlers_data.go: same constant message replaced
- internal/worker/reaper/reaper.go: FK audit comment updated to note v5 US1 drop
@thebtf thebtf merged commit 60250d8 into main Apr 17, 2026
8 checks passed
@thebtf thebtf deleted the feat/v5-us1-orphan-joins branch April 17, 2026 08:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant