Combine all Cleanlab bundle components into one and update documentation#10067
Combine all Cleanlab bundle components into one and update documentation#10067cmauck10 wants to merge 9 commits into
Conversation
…ediator; update pyproject and lock
… linting; trim trailing whitespace
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the WalkthroughConsolidates Cleanlab integration by removing separate evaluator/remediator components and introducing a single Cleanlab component, updates docs to a unified guardrails/remediation narrative with Quickstart, deletes a legacy example JSON flow, and adds dependencies cleanlab-codex and langchain-voyageai. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
actor User
participant Langflow as Langflow Flow
participant CleanlabCmp as Cleanlab Component
participant CLAPI as Cleanlab API (Project)
User->>Langflow: Provide query/input
Langflow->>CleanlabCmp: response, query, context, system_message, api_key
CleanlabCmp->>CLAPI: Project.from_access_key(api_key)
CleanlabCmp->>CLAPI: validate(messages, response, query, context)
CLAPI-->>CleanlabCmp: result {is_bad_response, expert_answer?}
alt Acceptable response
CleanlabCmp-->>Langflow: final_response = original response (status: accepted)
else Bad response + expert answer
CleanlabCmp-->>Langflow: final_response = expert answer (status: remediated_expert)
else Bad response, no expert answer
CleanlabCmp-->>Langflow: final_response = fallback_text (status: remediated_fallback)
end
Langflow-->>User: Final response
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Pre-merge checks and finishing touches❌ Failed checks (1 error, 4 warnings)
✅ Passed checks (2 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
docs/docs/Components/bundles-cleanlab.mdx (1)
1-4: Add requireddescriptionfield to front matter.Per coding guidelines, all Markdown/MDX pages must include at least
titleanddescriptionin front matter. Consider also addingsidebar_positionif this page should appear in a specific order within the docs navigation.Apply this diff:
--- title: Cleanlab slug: /bundles-cleanlab +description: Integrate Cleanlab with Langflow to add trustworthy evaluation and remediation to your AI pipelines. +sidebar_position: X ---Replace
Xwith the appropriate position number for this page in the Components section.
🧹 Nitpick comments (1)
src/lfx/src/lfx/components/cleanlab/cleanlab.py (1)
114-122: Consider validating required input content is non-empty.The method uses
self.input,self.response, andself.querydirectly without checking if they contain meaningful content. While these inputs are marked asrequired=True, a Message object can have empty or None text.Consider adding validation at the start of the method to provide clearer error messages:
def get_final_response(self) -> Message: # Validate required inputs if not self.input or not str(self.input).strip(): raise ValueError("Input message cannot be empty") if not self.response or not str(self.response).strip(): raise ValueError("Model response cannot be empty") if not self.query or not str(self.query).strip(): raise ValueError("User query cannot be empty") project = Project.from_access_key(self.api_key) # ... rest of methodThis provides better user feedback if the component is wired incorrectly.
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (16)
docs/static/img/cleanlab-add-remediation.pngis excluded by!**/*.pngdocs/static/img/cleanlab-ai-platform.pngis excluded by!**/*.pngdocs/static/img/cleanlab-expert-answer.pngis excluded by!**/*.pngdocs/static/img/cleanlab-fallback.pngis excluded by!**/*.pngdocs/static/img/cleanlab-flow-example.pngis excluded by!**/*.pngdocs/static/img/cleanlab-politics-guardrail.pngis excluded by!**/*.pngdocs/static/img/cleanlab-politics-guardrailed-chat.pngis excluded by!**/*.pngdocs/static/img/cleanlab-politics-guardrailed.pngis excluded by!**/*.pngdocs/static/img/cleanlab-response-passed-validation.pngis excluded by!**/*.pngdocs/static/img/cleanlab-wrong-response.pngis excluded by!**/*.pngdocs/static/img/cleanlab_remediator_example.pngis excluded by!**/*.pngdocs/static/img/cleanlab_remediator_example_fallback.pngis excluded by!**/*.pngdocs/static/img/eval_rag.pngis excluded by!**/*.pngdocs/static/img/eval_response.pngis excluded by!**/*.pngdocs/static/img/eval_summary_rag.pngis excluded by!**/*.pnguv.lockis excluded by!**/*.lock
📒 Files selected for processing (8)
docs/docs/Components/bundles-cleanlab.mdx(1 hunks)docs/static/files/eval_and_remediate_cleanlab.json(0 hunks)pyproject.toml(1 hunks)src/lfx/src/lfx/components/cleanlab/__init__.py(1 hunks)src/lfx/src/lfx/components/cleanlab/cleanlab.py(1 hunks)src/lfx/src/lfx/components/cleanlab/cleanlab_evaluator.py(0 hunks)src/lfx/src/lfx/components/cleanlab/cleanlab_rag_evaluator.py(0 hunks)src/lfx/src/lfx/components/cleanlab/cleanlab_remediator.py(0 hunks)
💤 Files with no reviewable changes (4)
- docs/static/files/eval_and_remediate_cleanlab.json
- src/lfx/src/lfx/components/cleanlab/cleanlab_rag_evaluator.py
- src/lfx/src/lfx/components/cleanlab/cleanlab_evaluator.py
- src/lfx/src/lfx/components/cleanlab/cleanlab_remediator.py
🧰 Additional context used
📓 Path-based instructions (2)
docs/**/*.{md,mdx}
📄 CodeRabbit inference engine (.cursor/rules/docs_development.mdc)
docs/**/*.{md,mdx}: All Markdown/MDX pages must start with front matter including at least title and description; include sidebar_position for docs pages when applicable
Code blocks must specify a language and may include a title (```lang title="…")
Use sentence case for headings and keep paragraphs short and scannable
Write in second person, present tense, with a professional but approachable tone
Use inline code with backticks for code terms; use bold for UI elements and italics for emphasis; keep lists in parallel structure
Ensure internal links are functional and navigation works (update cross-references as needed)
Verify all code examples in docs and blog actually run as shown
Use correct terminology capitalization: Langflow, Component, Flow, API, JSON
Reference images with absolute paths under /img/... and provide descriptive alt text
Files:
docs/docs/Components/bundles-cleanlab.mdx
docs/docs/**/*.{md,mdx}
📄 CodeRabbit inference engine (.cursor/rules/docs_development.mdc)
Use Docusaurus admonitions (:::+tip|warning|danger) instead of custom callouts in docs pages
Files:
docs/docs/Components/bundles-cleanlab.mdx
🧬 Code graph analysis (2)
src/lfx/src/lfx/components/cleanlab/cleanlab.py (1)
src/lfx/src/lfx/schema/message.py (1)
Message(34-293)
src/lfx/src/lfx/components/cleanlab/__init__.py (1)
src/lfx/src/lfx/components/cleanlab/cleanlab.py (1)
Cleanlab(13-133)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)
- GitHub Check: Test Docs Build / Test Docs Build
- GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 5
- GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 4
- GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 3
- GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 2
- GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 1
- GitHub Check: Run Backend Tests / Integration Tests - Python 3.10
- GitHub Check: Test Starter Templates
🔇 Additional comments (3)
docs/docs/Components/bundles-cleanlab.mdx (1)
50-108: LGTM: Image references follow guidelines.All image paths use absolute
/img/...paths with descriptive alt text as required by the coding guidelines.src/lfx/src/lfx/components/cleanlab/__init__.py (1)
8-16: LGTM: API surface correctly consolidated.The public API has been properly updated to expose a single
Cleanlabcomponent instead of three separate components (CleanlabEvaluator, CleanlabRAGEvaluator, CleanlabRemediator). The lazy-loading pattern is maintained correctly.src/lfx/src/lfx/components/cleanlab/cleanlab.py (1)
109-133: Consider adding error handling for external API calls.The
get_final_responsemethod calls external Cleanlab API methods (Project.from_access_keyandproject.validate) without error handling. If these calls fail (e.g., invalid API key, network issues, service unavailable), the component will raise an unhandled exception.Consider whether error handling should be added to provide more graceful degradation. For example:
def get_final_response(self) -> Message: try: project = Project.from_access_key(self.api_key) # Build messages for validate() messages = [] if getattr(self, "system_message", None): messages.append({"role": "system", "content": self.system_message}) messages.append({"role": "user", "content": self.input}) result = project.validate( messages=messages, response=self.response, query=self.query, context=self.context if getattr(self, "context", None) else "", ) if result.is_bad_response: if result.expert_answer: self.status = "Bad response replaced with expert answer." return Message(text=result.expert_answer) self.status = "Bad response with no expert answer. Using fallback." return Message(text=self.fallback_text) self.status = "Original response accepted." return Message(text=self.response) except Exception as e: self.status = f"Cleanlab validation failed: {e}. Returning original response." return Message(text=self.response)Verify whether the Langflow framework already handles component exceptions appropriately, or if explicit error handling is expected.
… into cmauck10/cleanlab
- Add unified Cleanlab component replacing separate evaluator/remediator - Update pyproject.toml and uv.lock dependencies - Rewrite bundles-cleanlab.mdx with new flow examples and assets - Fix linting issues (docstring D205, line length E501)
|
| @@ -8,133 +8,145 @@ import PartialParams from '@site/docs/_partial-hidden-params.mdx'; | |||
|
|
|||
| [Cleanlab](https://www.cleanlab.ai/) adds automation and trust to every data point going in and every prediction coming out of AI and RAG solutions. | |||
|
|
|||
There was a problem hiding this comment.
I can edit this page after the PR is merged, but we use the Google Developer Documentation Style Guide as our primary style guide. This page doesn't follow a few of the main style points with respect to second person, present tense, and sentence case for headings.
Additionally, please look at some of the other bundle pages to see how we structure the information on these pages. Since you have a multi-purpose component here, you might use the Composio page for inspiration.
Finally, I would suggest linking to your own Cleanlab documentation (such as this and this) as much as possible for the explanations of the functionality. This document doesn't need to teach users how Cleanlab works; it should focus on configuring the Langflow Cleanlab component only. This future-proofs the page because you won't need to remember to update it if the Cleanlab UI or functionality change.
2 participants
Summary
Consolidates Cleanlab into a single component and refreshes docs/assets to match the new workflow.
What’s in this PR
src/lfx/src/lfx/components/cleanlab/cleanlab.py(unified component)cleanlab_evaluator.py,cleanlab_rag_evaluator.py,cleanlab_remediator.pypyproject.tomlanduv.lockdocs/docs/Components/bundles-cleanlab.mdxfor the new flowdocs/static/files/eval_and_remediate_cleanlab.jsonwithdocs/static/files/cleanlab_quickstart_rag_flow.jsondocs/static/img/*(validation, guardrails, remediation, flow)Breaking changes
Migration
Cleanlabcomponent.docs/static/files/cleanlab_quickstart_rag_flow.jsonas a wiring reference.Summary by CodeRabbit