Skip to content

feat: Dreaming engine for periodic memory consolidation#592

Open
helal-muneer wants to merge 1 commit intoCortexReach:masterfrom
helal-muneer:feat/dreaming-engine
Open

feat: Dreaming engine for periodic memory consolidation#592
helal-muneer wants to merge 1 commit intoCortexReach:masterfrom
helal-muneer:feat/dreaming-engine

Conversation

@helal-muneer
Copy link
Copy Markdown

Summary

Closes #577

Implements Dreaming functionality — periodic memory consolidation and tier promotion that bridges LanceDB Pro's existing , , and into OpenClaw's dreaming lifecycle.

Three-Phase Dreaming Cycle

💤 Light Sleep — Decay + Tier Re-evaluation

  • Scans recent memories (configurable lookback, default 7 days)
  • Computes Weibull decay scores via decayEngine.scoreAll()
  • Evaluates tier transitions via tierManager.evaluateAll()
  • Applies promotions/demotions via store.patchMetadata()

🧠 Deep Sleep — Working → Core Promotion

  • Filters Working-tier memories by composite score and access count
  • Boosts importance (+20%) for qualifying memories
  • Promotes to Core tier via patchMetadata()

💫 REM — Pattern Detection & Reflection

  • Analyzes category clustering across tiers
  • Detects high-importance category patterns
  • Creates reflection memories (category: "reflection") with discovered patterns

Changes

File Description
src/dreaming-engine.ts New — dreaming engine factory with 3-phase cycle
openclaw.plugin.json Add dreaming config schema + UI hints
index.ts Wire dreaming into service lifecycle, add to PluginConfig

Bug Fix Included

  • resolveEnvVars() now returns empty string instead of throwing when env var is missing (prevents plugin startup failure when JINA_API_KEY etc. are unset)

Configuration

{
  "dreaming": {
    "enabled": true,
    "timezone": "UTC",
    "storageMode": "inline",
    "verboseLogging": false,
    "phases": {
      "light": { "lookbackDays": 7, "limit": 50 },
      "deep": { "limit": 20, "minScore": 0.5, "minRecallCount": 2 },
      "rem": { "lookbackDays": 30, "limit": 10, "minPatternStrength": 0.6 }
    }
  }
}

Testing

  • ✅ Plugin loads successfully with dreaming enabled
  • ✅ Dreaming engine initializes on service start
  • ✅ Tier transitions apply correctly during light phase
  • ✅ Core promotions work during deep phase
  • ✅ Pattern detection creates reflection memories in REM phase
  • ✅ Graceful error handling per phase (no cascade failures)
  • ✅ Cleanup on service stop

Implements CortexReach#577 - Dreaming functionality for memory-lancedb-pro

Three-phase dreaming cycle:
- Light Sleep: Decay scoring + tier re-evaluation for recent memories
- Deep Sleep: Promote high-performing Working memories to Core tier
- REM: Pattern detection across categories + reflection memory creation

Changes:
- Add dreaming config schema to openclaw.plugin.json with UI hints
- Create src/dreaming-engine.ts with createDreamingEngine factory
- Wire dreaming into service lifecycle (start/stop) in index.ts
- Add DreamingConfig to PluginConfig interface + parsePluginConfig
- Fix resolveEnvVars to return empty string instead of throwing
  when env var is missing (prevents plugin startup failure)

Dreaming runs on a 6-hour interval after 5-minute initial delay,
configurable via plugins.entries.memory-lancedb-pro.config.dreaming
Copy link
Copy Markdown
Collaborator

@rwmjhb rwmjhb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting feature with a sound architecture (bridging existing tier-manager and decay-engine). Branch needs a rebase and one scope isolation issue needs fixing before merge.

Please rebase onto current mainstale_base=true and index.ts has had significant recent activity. Rebase needed to surface any conflicts before this can land.


Must fix

MR1 — Dreaming ignores scope isolation and synthesizes cross-scope memories into global
The engine fetches memories across all scopes, then stores REM reflections with a hardcoded scope: 'global'. In multi-user or multi-workspace deployments this leaks content between isolated memory spaces — a user's private memories can influence another user's global reflection entries.

Fix: filter store.list() by the active scope in each phase, and tag REM reflections with the same scope as the source memories.


Non-blocking

  • F1resolveEnvVars now returns '' instead of throwing for unset env vars. This silently propagates an empty API key through resolveFirstApiKey's if (!key) guard (which only checks the raw config string, before resolution). Users with "${JINA_API_KEY}" configured but the env var unset will get opaque HTTP 401 failures instead of a clear startup error.
  • F2 — REM stores reflection with vector: []. LanceDB uses fixed-dimension Arrow columns — a 0-dimension vector will throw an Arrow schema mismatch on every cycle. The error is caught and logged as ⚠️ REM failed, so the REM phase silently never creates any reflection memories.
  • F3config.phases.light (and .deep, .rem) accessed without null guard. A minimal config like { "dreaming": { "enabled": true } } crashes with TypeError: Cannot read properties of undefined on first cycle. Add a DEFAULT_DREAMING_CONFIG and deep-merge it at parse time.
  • F6storageMode ('separate'/'both') and cron fields are exposed in configSchema and UI hints but never read at runtime. Users selecting these options silently get inline behavior and 6-hour fixed intervals. Remove from schema until implemented, or log a warning when set.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request] Dreaming 功能支援 - 三種實作策略分析與可行性評估

2 participants