feat: update flows with agent with debug logs for chat history#10204
Conversation
WalkthroughReplaces and expands AgentComponent implementations across multiple starter project JSONs, adds a LanguageModelComponent to one template, updates several code_hash metadata values, changes one JSON’s code string encoding, adds a new frontend E2E test, and adds a chat history debug log in the core agent module. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant UI as UI/Starter Template
participant Agent as AgentComponent
participant LMC as LanguageModelComponent
participant Prov as Model Provider
participant Mem as MemoryStore
participant Tools as Tools/CurrentDate
participant LLM as LLM
UI->>Agent: input_value, settings, schema?, tools?
Agent->>Mem: retrieve chat_history/session data
Mem-->>Agent: chat_history
Agent->>Prov: build_config (provider, model, params)
Prov-->>LMC: resolved model configuration
LMC-->>Agent: LLM factory / instance
Agent->>Tools: initialize toolkits (async)
Agent->>LLM: run (system + user + memory prompts)
alt structured JSON schema
Agent->>LLM: request structured output
LLM-->>Agent: candidate JSON
Agent->>Agent: validate/parse (pydantic)
else plain text
LLM-->>Agent: message text
end
Agent-->>UI: response (text/JSON)
opt error
Agent-->>UI: raise/log error
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Pre-merge checks and finishing touches❌ Failed checks (1 error, 2 warnings)
✅ Passed checks (4 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Frontend Unit Test Coverage ReportCoverage Summary
Unit Test Results
|
Codecov Report✅ All modified and coverable lines are covered by tests. ❌ Your project status has failed because the head coverage (48.71%) is below the target coverage (55.00%). You can increase the head coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## main #10204 +/- ##
=======================================
Coverage 24.76% 24.76%
=======================================
Files 1090 1090
Lines 40108 40108
Branches 5550 5550
=======================================
Hits 9934 9934
Misses 30003 30003
Partials 171 171
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
github-actions
Bot
added
enhancement
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (11)
src/backend/base/langflow/initial_setup/starter_projects/Youtube Analysis.json (1)
906-924: Avoid logging full chat history
logger.debug(f"Chat history: {self.chat_history}")will dump user conversations into backend logs. Even at debug level, this risks leaking PII / sensitive content and violates our usual privacy posture. Drop the log or sanitize/mask before emitting.src/backend/base/langflow/initial_setup/starter_projects/Pokédex Agent.json (1)
1327-1331: Avoid logging full chat history
logger.debug(f"Chat history: {self.chat_history}")will dump every prior user message into logs, which can contain sensitive/PII data. Even at debug level this is a compliance risk once debug logging is enabled in prod. Please remove the raw content from logs or mask/redact it (e.g., log counts/IDs only).src/backend/base/langflow/initial_setup/starter_projects/Social Media Agent.json (1)
1405-1420: Strip chat history dump from templateThis starter template now ships the same
logger.debug(f"Chat history: {self.chat_history}")line, so every generated flow will emit full user conversations to logs. Please remove or sanitize that statement here as well to stay compliant once the core component is fixed.src/backend/base/langflow/initial_setup/starter_projects/Price Deal Finder.json (1)
1723-1740: Remove chat transcript logging from this templateSame concern: the embedded AgentComponent code logs raw chat history. Please align this template with the sanitized implementation so flows derived from it don't leak conversation content.
src/backend/base/langflow/initial_setup/starter_projects/SaaS Pricing.json (1)
915-918: Do not log full chat history contents.
logger.debug(f"Chat history: {self.chat_history}")will dump entire conversations (potential PII) into logs. Even at debug level this is a compliance/privacy risk. Please remove it or replace with a safe, redacted metric (e.g., count of messages) gated behind an explicit opt-in.src/backend/base/langflow/initial_setup/starter_projects/News Aggregator.json (1)
1125-1128: Avoid logging raw chat transcripts.
logger.debug(f"Chat history: {self.chat_history}")emits the entire conversation to logs, which can include sensitive customer data. Replace this with a sanitized summary (e.g., length) or gate it behind a configurable opt-in before logging.src/backend/base/langflow/initial_setup/starter_projects/Nvidia Remix.json (1)
846-850: Redact chat history before logging
logger.debug(f"Chat history: {self.chat_history}")will dump user conversations—including PII—into application logs. That's a compliance/privacy risk and can violate retention policies. Please drop the raw log or redact it (e.g., log counts/message IDs only).src/backend/base/langflow/initial_setup/starter_projects/Invoice Summarizer.json (1)
1195-1210: Do not log full chat history content.
logger.debug(f"Chat history: {self.chat_history}")will emit every message in memory—including user prompts and potentially sensitive data—to our logs whenever debug logging is enabled. That’s a privacy/compliance risk and goes beyond “debug” utility. Please drop this line or replace it with a redacted/summary log that never prints raw conversation text.src/backend/base/langflow/initial_setup/starter_projects/Market Research.json (1)
1500-1515: Avoid dumping chat history into logs.Here as well,
logger.debug(f"Chat history: {self.chat_history}")prints every stored message verbatim. That exposes end-user conversations/PII in logs whenever debug is active. Please remove or sanitize this log before shipping.src/backend/base/langflow/initial_setup/starter_projects/Simple Agent.json (1)
1035-1037: Avoid logging full chat transcriptsDumping
self.chat_historyverbatim pushes every user message into our logs, which can easily include secrets or PII. That’s a compliance/privacy footgun even at debug level—debug pipelines often end up in central log storage. Please only log metadata (e.g., count) and keep the actual transcript out of the logs.- logger.debug(f"Chat history: {self.chat_history}") + history_count = len(self.chat_history) if isinstance(self.chat_history, list) else "unknown" + logger.debug("Chat history retrieved (count=%s)", history_count)src/backend/base/langflow/initial_setup/starter_projects/Instagram Copywriter.json (1)
2265-2326: Structured-output fallback fails for JSON arraysWhen the agent returns a list (common for multi-item schemas) with any leading text, the fallback regex (
r"\{.*\}") drops the entire payload, sobuild_structured_output_baseexits with"Try setting an output schema"instead of validating the list. Please allow both objects and arrays in the fallback scan.Apply this diff inside the embedded AgentComponent code:
- json_pattern = r"\{.*\}" + json_patterns = [r"\{.*?\}", r"\[.*?\]"] schema_error_msg = "Try setting an output schema" # Try to parse content as JSON first json_data = None try: json_data = json.loads(content) except json.JSONDecodeError: - json_match = re.search(json_pattern, content, re.DOTALL) - if json_match: - try: - json_data = json.loads(json_match.group()) - except json.JSONDecodeError: - return {"content": content, "error": schema_error_msg} - else: - return {"content": content, "error": schema_error_msg} + for pattern in json_patterns: + json_match = re.search(pattern, content, re.DOTALL) + if not json_match: + continue + try: + json_data = json.loads(json_match.group()) + break + except json.JSONDecodeError: + continue + if json_data is None: + return {"content": content, "error": schema_error_msg}
🧹 Nitpick comments (2)
src/backend/base/langflow/initial_setup/starter_projects/Search agent.json (1)
1068-1068: Consider log verbosity for large chat histories.The debug log
logger.debug(f"Chat history: {self.chat_history}")directly logs the entire chat history object. If the chat history contains many messages or long message contents, this could produce very verbose logs that are difficult to read and may impact performance.Consider one of these approaches:
- Log only the count and basic metadata:
logger.debug(f"Chat history: {len(self.chat_history)} messages")
- Use structured logging with truncation:
logger.debug( "Chat history retrieved", extra={ "message_count": len(self.chat_history), "sample": self.chat_history[:2] if self.chat_history else [] } )
- Add a helper method to summarize chat history for logging:
logger.debug(f"Chat history summary: {self._summarize_chat_history()}")src/frontend/tests/core/integrations/Simple Agent Memory.spec.ts (1)
59-60: Consider more specific assertion selector.The test uses
getByTestId("div-chat-message").innerText()which retrieves the text from all chat messages, not just the assistant's final response. While this may work, it could lead to false positives if "john doe" appears in any message (including the user's input message).Consider selecting specifically the last assistant message:
// Get all assistant messages and check the last one const assistantMessages = await page .getByTestId("div-chat-message") .filter({ hasText: /AI|Assistant|Machine/ }) // or whatever identifies assistant messages .all(); const lastAssistantText = await assistantMessages[assistantMessages.length - 1].innerText(); expect(lastAssistantText.toLowerCase()).toContain("john doe");Or if there's a specific test ID for assistant messages:
const finalText = await page .getByTestId("div-chat-message-ai") // or similar .last() .innerText(); expect(finalText.toLowerCase()).toContain("john doe");
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (15)
src/backend/base/langflow/initial_setup/starter_projects/Instagram Copywriter.json(2 hunks)src/backend/base/langflow/initial_setup/starter_projects/Invoice Summarizer.json(2 hunks)src/backend/base/langflow/initial_setup/starter_projects/Market Research.json(2 hunks)src/backend/base/langflow/initial_setup/starter_projects/News Aggregator.json(2 hunks)src/backend/base/langflow/initial_setup/starter_projects/Nvidia Remix.json(2 hunks)src/backend/base/langflow/initial_setup/starter_projects/Pokédex Agent.json(2 hunks)src/backend/base/langflow/initial_setup/starter_projects/Price Deal Finder.json(2 hunks)src/backend/base/langflow/initial_setup/starter_projects/Research Agent.json(2 hunks)src/backend/base/langflow/initial_setup/starter_projects/SaaS Pricing.json(2 hunks)src/backend/base/langflow/initial_setup/starter_projects/Search agent.json(2 hunks)src/backend/base/langflow/initial_setup/starter_projects/Simple Agent.json(2 hunks)src/backend/base/langflow/initial_setup/starter_projects/Social Media Agent.json(2 hunks)src/backend/base/langflow/initial_setup/starter_projects/Youtube Analysis.json(2 hunks)src/frontend/tests/core/integrations/Simple Agent Memory.spec.ts(1 hunks)src/lfx/src/lfx/components/agents/agent.py(1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
src/frontend/**/*.@(test|spec).{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (.cursor/rules/testing.mdc)
src/frontend/**/*.@(test|spec).{ts,tsx,js,jsx}: Frontend test files should be located in 'src/frontend/' and use '.test.{ts,tsx,js,jsx}' or '.spec.{ts,tsx,js,jsx}' extensions.
Test both sync and async code paths in frontend test files.
Mock external dependencies appropriately in frontend test files to isolate unit tests from external services.
Test error handling and edge cases in frontend test files.
Validate input/output behavior and test component initialization and configuration in frontend test files.
Each frontend test should have a clear description or comment explaining its purpose, especially for complex setups or mocks.
Files:
src/frontend/tests/core/integrations/Simple Agent Memory.spec.ts
🧬 Code graph analysis (1)
src/frontend/tests/core/integrations/Simple Agent Memory.spec.ts (2)
src/frontend/tests/utils/withEventDeliveryModes.ts (1)
withEventDeliveryModes(15-41)src/frontend/tests/utils/await-bootstrap-test.ts (1)
awaitBootstrapTest(4-49)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (52)
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 38/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 37/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 34/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 40/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 36/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 31/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 39/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 29/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 35/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 32/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 25/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 33/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 28/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 30/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 27/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 26/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 24/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 22/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 23/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 21/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 19/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 17/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 18/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 20/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 12/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 15/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 16/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 10/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 13/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 14/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 11/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 9/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 5/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 8/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 7/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 3/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 1/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 6/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 2/40
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 4/40
- GitHub Check: Lint Backend / Run Mypy (3.10)
- GitHub Check: Test Docker Images / Test docker images
- GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 4
- GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 3
- GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 2
- GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 5
- GitHub Check: Run Backend Tests / Integration Tests - Python 3.10
- GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 1
- GitHub Check: Test Starter Templates
- GitHub Check: test-starter-projects
- GitHub Check: Optimize new Python code in this PR
- GitHub Check: Update Starter Projects
🔇 Additional comments (3)
src/backend/base/langflow/initial_setup/starter_projects/Search agent.json (1)
899-899: LGTM!The code_hash update correctly reflects the updated AgentComponent code below.
src/frontend/tests/core/integrations/Simple Agent Memory.spec.ts (1)
17-19: Verify .env file path usageNo
.envfile was found in the repo tree—confirm where your.envactually resides and update thepath.resolve(__dirname, …)call to point to that location (or remove the explicitpathoption so dotenv loads the default.envfrom the working directory).src/backend/base/langflow/initial_setup/starter_projects/Research Agent.json (1)
2530-2532: Strip chat content from debug logsHere too we’re emitting the entire chat history into logs, exposing user conversations. That’s a privacy/compliance risk—please log only safe metadata (like message counts) rather than the raw transcript.
- logger.debug(f"Chat history: {self.chat_history}") + history_count = len(self.chat_history) if isinstance(self.chat_history, list) else "unknown" + logger.debug("Chat history retrieved (count=%s)", history_count)Likely an incorrect or invalid review comment.
| self.chat_history = await self.get_memory_data() | ||
| logger.debug(f"Chat history: {self.chat_history}") | ||
| if isinstance(self.chat_history, Message): | ||
| self.chat_history = [self.chat_history] |
There was a problem hiding this comment.
Remove sensitive chat history logging
self.chat_history contains end-user conversation data. Dumping it verbatim into logs—even at DEBUG—exposes PII/regulated content and breaks our privacy stance. Please drop this line or replace it with a redacted/aggregate signal (e.g., count of messages) before shipping.
🤖 Prompt for AI Agents
In src/lfx/src/lfx/components/agents/agent.py around lines 181 to 184, remove
the direct debug logging of self.chat_history because it may contain sensitive
end-user data; instead replace that line with a non-content-specific signal such
as logging the number of messages (and optionally metadata like message types or
timestamps) after you normalize Message -> list, ensuring no message text or PII
is written to logs.
There was a problem hiding this comment.
Agreed that this exposes user data in a sense, but not sure I necessarily agree it's PII.
More so, we shouldn't do this due to performance issues -- don't want to serialize the entire chat history in logs.
There was a problem hiding this comment.
@jordanrfrazier, excellent point about the performance impact! You're absolutely right that serializing the entire chat history in logs could be expensive, especially as conversations grow longer.
A lightweight alternative would be to log just the count or basic metadata instead:
logger.debug(f"Retrieved {len(self.chat_history)} chat history messages")This gives you the debugging signal you need (confirming messages were retrieved) without the serialization overhead.
✏️ Learnings added
Learnt from: jordanrfrazier
PR: langflow-ai/langflow#10204
File: src/lfx/src/lfx/components/agents/agent.py:181-184
Timestamp: 2025-10-10T16:02:39.849Z
Learning: In the langflow codebase, avoid logging entire chat history objects in debug statements due to performance concerns from serializing large data structures. Instead, log aggregate metadata like message counts (e.g., `logger.debug(f"Retrieved {len(chat_history)} messages")`).
Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.
There was a problem hiding this comment.
Agreed. but its debug logs right?
There was a problem hiding this comment.
I believe it logs only when loglevel is debug.
There was a problem hiding this comment.
Seems like the humans are having a chat. I'll hop back into my burrow for now. If you need me again, just tag @coderabbitai in a new comment, and I'll come hopping out!
github-actions
Bot
added
enhancement
github-actions
Bot
added
enhancement
…message element for assertion accuracy
github-actions
Bot
added
enhancement
edwinjosechittilappilly
changed the title
github-actions
Bot
added
enhancement
|
…low-ai#10204) * udpate flows * agent memory test * adding plauywright for memory * agent memory test * ✨ (Simple Agent Memory.spec.ts): update test to select the last chat message element for assertion accuracy --------- Co-authored-by: cristhianzl <cristhian.lousa@gmail.com>
3 participants
This pull request updates the metadata for the
Instagram Copywriter.jsonstarter project. The only change is an update to thecode_hashvalue, which likely reflects a new version of the project's code.code_hashfield in themetadatasection ofInstagram Copywriter.jsonto reflect the latest code version.