Skip to content

fix: harden memory tool input validation#5701

Open
MatthiasHowellYopp wants to merge 1 commit into
crewAIInc:mainfrom
MatthiasHowellYopp:feat/valkey-2-memory-tools-hardening
Open

fix: harden memory tool input validation#5701
MatthiasHowellYopp wants to merge 1 commit into
crewAIInc:mainfrom
MatthiasHowellYopp:feat/valkey-2-memory-tools-hardening

Conversation

@MatthiasHowellYopp
Copy link
Copy Markdown
Contributor

@MatthiasHowellYopp MatthiasHowellYopp commented May 4, 2026

Description:

Part 2/4 of adding Valkey as a storage backend for CrewAI. This PR hardens the memory tools against malformed inputs that surface more frequently with the new storage paths.

What changed:

  • memory_tools.py — RecallMemoryTool and RememberTool now handle None, empty lists, and lists of empty strings gracefully, returning descriptive error messages instead of crashing. String inputs are normalized to lists. The queries and contents fields are now optional with min_length=1 validation, and both schemas use extra="forbid" to reject unexpected parameters.

  • en.json — Updated tool descriptions for recall_memory and save_to_memory to include explicit parameter format examples (e.g. {"queries": ["search term"]}), reducing LLM misuse of the tool interface.

Testing:

test_memory_tools.py (15 tests) — Covers None input, empty lists, empty strings, string-to-list conversion, deduplication across queries, and single vs. batch remember paths.

Summary by CodeRabbit

  • New Features

    • Memory tools accept optional inputs and normalize single strings or lists; non-string items are ignored.
  • Bug Fixes

    • Improved handling of missing/empty/whitespace inputs with clearer error messages; previous recall/remember flows preserved.
  • Documentation

    • Instruction text updated to require explicit JSON parameters for recall and save operations.
  • Tests

    • Added tests covering non-string item handling and related edge cases.
  • Chores

    • Linting config adjusted to exclude JSON files.

Review Change Stack

@MatthiasHowellYopp MatthiasHowellYopp force-pushed the feat/valkey-2-memory-tools-hardening branch 3 times, most recently from 32ed03e to 0f0823f Compare May 7, 2026 19:59
@MatthiasHowellYopp MatthiasHowellYopp force-pushed the feat/valkey-2-memory-tools-hardening branch from 0f0823f to c35cd7e Compare May 11, 2026 19:53
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 11, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 7e9c5197-aa97-4f2b-82c7-14e4a5e366af

📥 Commits

Reviewing files that changed from the base of the PR and between bf40d62 and b8cdaf3.

📒 Files selected for processing (4)
  • lib/crewai/src/crewai/tools/memory_tools.py
  • lib/crewai/src/crewai/translations/en.json
  • lib/crewai/tests/tools/test_memory_tools.py
  • pyproject.toml
✅ Files skipped from review due to trivial changes (2)
  • lib/crewai/src/crewai/translations/en.json
  • pyproject.toml
🚧 Files skipped from review as they are similar to previous changes (2)
  • lib/crewai/src/crewai/tools/memory_tools.py
  • lib/crewai/tests/tools/test_memory_tools.py

📝 Walkthrough

Walkthrough

Memory tools' input schemas now permit optional queries and contents and forbid extra fields; tools normalize string inputs to lists, trim and filter entries (including non-strings), and return explicit errors when no usable items remain; translations and tests updated accordingly.

Changes

Memory Tools Input Validation

Layer / File(s) Summary
Input Schemas
lib/crewai/src/crewai/tools/memory_tools.py
RecallMemorySchema.queries and RememberSchema.contents change from required list[str] to optional list[str] | None with min-length validation and model_config = {"extra": "forbid"}.
Recall Tool Input Processing
lib/crewai/src/crewai/tools/memory_tools.py
RecallMemoryTool._run signature expands queries to accept list[str] | str | None. Adds validation for missing/empty inputs, string-to-list normalization, trimming/filtering blank and non-string queries, and targeted errors when no valid queries remain.
Remember Tool Input Processing
lib/crewai/src/crewai/tools/memory_tools.py
RememberTool._run signature expands contents to accept list[str] | str | None. Adds validation for missing/empty inputs, string-to-list normalization, trimming/filtering blank and non-string contents, and targeted errors when no valid contents remain.
Tool Instruction Translations
lib/crewai/src/crewai/translations/en.json
recall_memory and save_to_memory tool instructions now explicitly require queries and contents parameters respectively, with clearer JSON examples.
Test Module & Fixtures
lib/crewai/tests/tools/test_memory_tools.py
New pytest fixtures instantiate a mocked memory object and both tool instances; tests added/expanded to cover validation and behavior.
Recall Tool Tests
lib/crewai/tests/tools/test_memory_tools.py
TestRecallMemoryToolValidation covers None/empty/whitespace inputs, string-to-list normalization, per-query memory.recall invocation, no-match messaging, match formatting, and deduplication.
Remember Tool Tests
lib/crewai/tests/tools/test_memory_tools.py
TestRememberToolValidation covers None/empty/whitespace inputs, string-to-list normalization, routing to memory.remember vs memory.remember_many, whitespace filtering, and success messaging.
Non-String Item Tests
lib/crewai/tests/tools/test_memory_tools.py
TestNonStringItemHandling asserts filtering of non-string elements and errors when no valid strings remain for both recall and remember tools.
Ruff Exclude JSON
pyproject.toml
Added **/*.json to tool.ruff.extend-exclude.

Sequence Diagram

sequenceDiagram
  participant User
  participant RecallMemoryTool
  participant Memory
  User->>RecallMemoryTool: sends queries (None|string|list)
  RecallMemoryTool->>RecallMemoryTool: normalize, trim, filter, dedupe
  RecallMemoryTool->>Memory: memory.recall(valid_queries)
  Memory-->>RecallMemoryTool: returns matches
  RecallMemoryTool-->>User: formatted matches or error
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐰 I tidy queries and contents with care,
Trim the fluff, filter what isn't fair.
Strings become lists, blanks gently tossed,
Only real memories are kept — never lost.
Tests hop in place to guard what was crossed.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 12.50% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and specifically summarizes the primary change: hardening memory tool input validation to handle malformed inputs gracefully.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Warning

Review ran into problems

🔥 Problems

Git: Failed to clone repository. Please run the @coderabbitai full review command to re-trigger a full review. If the issue persists, set path_filters to include or exclude specific files.


Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
lib/crewai/tests/tools/test_memory_tools.py (1)

50-55: ⚡ Quick win

Add malformed mixed-type input tests to lock in hardening behavior.

Given the hardening objective, consider adding coverage for cases like queries=["ok", 123] and contents=["fact", None] to ensure the tools never crash and always return stable validation/success responses.

Also applies to: 109-113

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/crewai/tests/tools/test_memory_tools.py` around lines 50 - 55, Add tests
that exercise malformed mixed-type inputs so the memory tools never crash and
always return stable validation or error responses: extend
test_list_of_empty_strings_returns_error (and the analogous test around lines
109-113) to include cases like queries=["ok", 123] and contents=["fact", None],
calling RecallMemoryTool._run (and any corresponding tool methods used in the
other test) and asserting the call does not raise and returns a predictable
error/validation result (e.g., contains "Error" or a stable validation payload)
instead of crashing.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@lib/crewai/src/crewai/tools/memory_tools.py`:
- Around line 54-63: The current guard assumes every item in queries (and
similarly in contents at the other block) is a string and calls .strip(), which
will crash for non-string items like integers or booleans; update the
normalization to first handle non-list inputs, then coerce/filter items to
strings (or filter with isinstance(item, str)) before calling .strip() so
queries = [q for q in queries if isinstance(q, str) and q.strip()] (and apply
the same change to the contents handling around the 115-123 region) ensuring the
function that processes queries/contents rejects or converts non-string entries
rather than raising AttributeError.

---

Nitpick comments:
In `@lib/crewai/tests/tools/test_memory_tools.py`:
- Around line 50-55: Add tests that exercise malformed mixed-type inputs so the
memory tools never crash and always return stable validation or error responses:
extend test_list_of_empty_strings_returns_error (and the analogous test around
lines 109-113) to include cases like queries=["ok", 123] and contents=["fact",
None], calling RecallMemoryTool._run (and any corresponding tool methods used in
the other test) and asserting the call does not raise and returns a predictable
error/validation result (e.g., contains "Error" or a stable validation payload)
instead of crashing.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 4a1b25a8-04e6-464a-bfa4-5abfa951e4df

📥 Commits

Reviewing files that changed from the base of the PR and between 63a9e7e and c35cd7e.

📒 Files selected for processing (3)
  • lib/crewai/src/crewai/tools/memory_tools.py
  • lib/crewai/src/crewai/translations/en.json
  • lib/crewai/tests/tools/test_memory_tools.py

Comment thread lib/crewai/src/crewai/tools/memory_tools.py
@MatthiasHowellYopp MatthiasHowellYopp force-pushed the feat/valkey-2-memory-tools-hardening branch from c35cd7e to d9d3416 Compare May 11, 2026 20:32
@MatthiasHowellYopp MatthiasHowellYopp force-pushed the feat/valkey-2-memory-tools-hardening branch 2 times, most recently from 2f9d683 to bf40d62 Compare May 13, 2026 14:25
Add None/empty input handling to RecallMemoryTool and RememberTool.
Filter empty strings, convert string inputs to lists, and return
descriptive error messages. Update tool descriptions in en.json to
include explicit parameter examples.

Part 2/4 of Valkey storage implementation.
@MatthiasHowellYopp MatthiasHowellYopp force-pushed the feat/valkey-2-memory-tools-hardening branch from bf40d62 to b8cdaf3 Compare May 14, 2026 13:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant