[Bug] recallScoreThreshold bypassed in pickMemoriesForInjection function
Description
The recallScoreThreshold configuration is being bypassed during memory recall, causing low-scoring memories (below threshold) to be injected into context.
Evidence
- Config:
recallScoreThreshold: 0.25
- Observed: Retrieved memories include scores as low as
0.11 (below threshold)
- Expected: All returned memories should have score ≥ 0.25
Root Cause Analysis
The issue is in memory-ranking.ts, specifically in the pickMemoriesForInjection function.
Flow:
-
postProcessMemories correctly filters memories by threshold (line 49):
if (clampScore(item.score) < options.scoreThreshold) {
continue; // ✓ Correct filtering
}
-
However, pickMemoriesForInjection (line 221) bypasses this filtering when trying to fill up to the limit:
const leaves = deduped.filter((item) => isLeafLikeMemory(item));
if (leaves.length >= limit) {
return leaves.slice(0, limit); // ✗ No threshold check
}
const picked = [...leaves];
const used = new Set(leaves.map((item) => item.uri));
for (const item of deduped) {
if (picked.length >= limit) {
break;
}
if (used.has(item.uri)) {
continue;
}
picked.push(item); // ✗ Adds items without threshold check!
}
return picked;
Problem:
When there aren't enough "leaf-like" memories (level === 2), the function supplements from the general deduped array without re-checking the score threshold. This allows low-scoring memories to pass through.
Proposed Fix
Add threshold validation in pickMemoriesForInjection:
for (const item of deduped) {
if (picked.length >= limit) {
break;
}
if (used.has(item.uri)) {
continue;
}
// ADD: Re-check threshold before adding
if (clampScore(item.score) < options.scoreThreshold) {
continue;
}
picked.push(item);
}
Note: The function signature would need to accept scoreThreshold as a parameter, or use a default value.
Impact
- Low-quality/irrelevant memories are being injected into agent context
- Wastes token budget on low-scoring results
- Reduces overall recall quality
Environment
- OpenViking: v0.2.15
- Plugin installation: path-based (
/Users/neoshi/.openviking/plugins/openviking)
- Config mode: remote
Additional Context
This was discovered during investigation of recall quality after disabling rerank. Statistics from 14 queries:
- Average score: 0.2827
- Score range: 0.1100 - 0.7222 (minimum below 0.25 threshold)
- Zero-result rate: 0%
[Bug] recallScoreThreshold bypassed in pickMemoriesForInjection function
Description
The
recallScoreThresholdconfiguration is being bypassed during memory recall, causing low-scoring memories (below threshold) to be injected into context.Evidence
recallScoreThreshold: 0.250.11(below threshold)Root Cause Analysis
The issue is in
memory-ranking.ts, specifically in thepickMemoriesForInjectionfunction.Flow:
postProcessMemoriescorrectly filters memories by threshold (line 49):However,
pickMemoriesForInjection(line 221) bypasses this filtering when trying to fill up to the limit:Problem:
When there aren't enough "leaf-like" memories (level === 2), the function supplements from the general
dedupedarray without re-checking the score threshold. This allows low-scoring memories to pass through.Proposed Fix
Add threshold validation in
pickMemoriesForInjection:Note: The function signature would need to accept
scoreThresholdas a parameter, or use a default value.Impact
Environment
/Users/neoshi/.openviking/plugins/openviking)Additional Context
This was discovered during investigation of recall quality after disabling rerank. Statistics from 14 queries: