feat(learning-engine): Q-Table & RL state not persisted across MCP restarts

[dmoellenbeck]

Is your feature request related to a problem?

The LearningEngine stores Q-values, algorithm configs, and RL state only in RAM. On MCP server restart, all trained Q-values are lost. hooks_learning_config settings vanish (causing epsilon error). Users must re-bootstrap every session.

Root Cause

1. export() incomplete — missing eligibilityTraces, actorWeights
2. import() incomplete — doesn't restore the above Maps
3. No dedicated persistence file — learning state mixed into intelligence.json
4. No auto-recovery on MCP startup
5. rewardHistory unbounded

Describe the solution you'd like

Phase 1: Fix export/import + dedicated learning-state.json

• Complete export() to serialize all internal Maps
• Complete import() to restore all Maps with graceful defaults
• Persist to .ruvector/learning-state.json (atomic write: tmp → rename)
• Auto-load on MCP startup
• Cap rewardHistory at 500

Phase 2 (future): RVF POLICY_KERNEL persistence

Store Q-tables in POLICY_KERNEL (0x31) segment, extending its semantics beyond Thompson Sampling to include tabular RL (Q-Learning, SARSA, Double-Q, Actor-Critic). This aligns with ADR-029 (RVF Canonical Format) and ADR-036 (AGI Cognitive Container).

PR

Phase 1 implementation: dmoellenbeck#2 (pending)

Environment

• ruvector 0.2.x (npm), Node.js 22.5+, macOS
• Real-world use case: endurance coaching with 200+ Q-learning experiences, daily training adaptation

[dmoellenbeck]

Closing — maintaining in fork. Will resubmit when ready for upstream.

[dmoellenbeck]

Reopening — bug confirmed still present in ruvector@0.2.19 (and likely 0.2.21)

The Bug (more precise than original report)

hooks_learn and hooks_batch_learn read/write Q-tables from different locations in the Intelligence class:

Handler	Reads from	Writes to	Persists via save()?
hooks_learn	intel.learning	intel.learning	NO — not inside this.data
hooks_batch_learn	intel.data.learning	intel.data.learning	YES — inside this.data

This means:

1. hooks_learn updates are lost on MCP restart (in-memory only)
2. hooks_batch_learn can never see hooks_learn's Q-tables (different path)
3. They maintain two separate, disconnected Q-tables

Fix (2-line change in mcp-server.js)

Change hooks_learn to use the same persisted path as hooks_batch_learn:

// hooks_learn handler — change intel.learning → intel.data.learning
// Read:
- if (intel.learning) engine.import(intel.learning);
+ if (intel.data.learning) engine.import(intel.data.learning);

// Write:
- intel.learning = engine.export();
+ intel.data.learning = engine.export();
+ intel.save();  // persist to disk

Same fix needed for hooks_learning_config, hooks_learning_stats, hooks_learning_update handlers.

Patch Available

We have a complete patch (103 insertions) that also adds:

• saveLearningState() with atomic write (tmp → rename)
• loadLearningState() with auto-recovery on startup
• Dedicated learning-state.json separate from intelligence.json

Happy to submit as PR if you point us to the right branch.

Impact

Without this fix, Q-Learning in any MCP-based application is effectively non-functional across restarts. Applications fall back to random recommendations (uniform 0.25 confidence) after every restart.