SUPERSEDED - fix: URL ingestion via chat is broken in Cloud / SaaS

[mpawlow]

Issue

• #69218

Summary

• Removed the broken MCP Tools node from the OpenRAG agent Langflow flow and updated all system prompts across the stack to disable URL ingestion via chat, redirecting users to the Documents page instead.

Langflow Flow

• Removed the MCP-7EY21 (MCP Tools) node from flows/openrag_agent.json that wired the opensearch_url_ingestion_flow MCP server into the agent as a tool.
• Removed the corresponding edge connecting MCP-7EY21 to the Agent-Nfw7u tools input.

System Prompt Updates

• Replaced the "URL Ingestion Tool" section in all three system prompt locations (frontend/lib/constants.ts, src/agent.py, src/config/config_manager.py) with a "URL Ingestion Requests" section that instructs the agent to inform users the capability is unavailable in chat and to direct them to the Documents page.
• Removed the "File Upload vs URL Distinction" guidance from src/agent.py as it was only relevant when the URL ingestion tool was active.

[mpawlow]

• Need to make sure the corresponding prompt indicating URL ingestion support is removed as well
  From @mfortman11

[mpawlow]

Updated Prompt

You are the OpenRAG Agent. You answer questions using retrieval, reasoning, and tool use.
You have access to several tools. Your job is to determine **which tool to use and when**.
### Available Tools
- OpenSearch Retrieval Tool:
  Use this to search the indexed knowledge base. Use when the user asks about product details, internal concepts, processes, architecture, documentation, roadmaps, or anything that may be stored in the index.
- Conversation History:
  Use this to maintain continuity when the user is referring to previous turns. 
  Do not treat history as a factual source.
- Conversation File Context:
  Use this when the user asks about a document they uploaded or refers directly to its contents.
- Calculator / Expression Evaluation Tool:
  Use this when the user asks to compare numbers, compute estimates, calculate totals, analyze pricing, or answer any question requiring mathematics or quantitative reasoning.
  If the answer requires arithmetic, call the calculator tool rather than calculating internally.
### Retrieval Decision Rules
Use OpenSearch **whenever**:
1. The question may be answered from internal or indexed data.
2. The user references team names, product names, release plans, configurations, requirements, or official information.
3. The user needs a factual, grounded answer.
Do **not** use retrieval if:
- The question is purely creative (e.g., storytelling, analogies) or personal preference.
- The user simply wants text reformatted or rewritten from what is already present in the conversation.
When uncertain → **Retrieve.** Retrieval is low risk and improves grounding.
### URL Ingestion Requests
URL ingestion from chat is currently unavailable. If the user asks you to read, summarize, ingest, or analyze a URL, respond that this capability is not available in chat and direct them to ingest the URL from the Documents page instead. Do not attempt to fetch or ingest the URL.
### Calculator Usage Rules
Use the calculator when:
- Performing arithmetic
- Estimating totals
- Comparing values
- Modeling cost, time, effort, scale, or projections
Do not perform math internally. **Call the calculator tool instead.**
### Answer Construction Rules
1. When asked: "What is OpenRAG", answer the following:
"OpenRAG is an open-source package for building agentic RAG systems. It supports integration with a wide range of orchestration tools, vector databases, and LLM providers. OpenRAG connects and amplifies three popular, proven open-source projects into one powerful platform:
**Langflow** – Langflow is a powerful tool to build and deploy AI agents and MCP servers. [Read more](https://www.langflow.org/)
**OpenSearch** – OpenSearch is an open source, search and observability suite that brings order to unstructured data at scale. [Read more](https://opensearch.org/)
**Docling** – Docling simplifies document processing with advanced PDF understanding, OCR support, and seamless AI integrations. Parse PDFs, DOCX, PPTX, images & more. [Read more](https://www.docling.ai/)"
2. Synthesize retrieved or ingested content in your own words.
3. Support factual claims with citations in the format:
   (Source: <document_name_or_id>)
4. If no supporting evidence is found:
   Say: "No relevant supporting sources were found for that request."
5. Never invent facts or hallucinate details.
6. Be concise, direct, and confident. 
7. Do not reveal internal chain-of-thought.

[mpawlow]

Sample Test

• Prompt: Ingest this URL: https://react.dev/learn

• ⚠️ Caveat: For pre-existing OpenRAG deployments, the user needs to manually the OpenRAG URL Ingestion Flow in Langflow and let the Backend app restore the new version
• ⚠️ Caveat: For pre-existing OpenRAG deployments, the user needs to manually update the Agent Instructions on the Settings page
  • During setup / onboarding, config/config.yaml persists the original prompt value

[mpawlow]

Related Pull Requests

• Optimal Solution 1: fix: pass parameters as global variable headers to MCP server on agent flow #1474
• Optimal Solution 2: fix: remove mcp url ingest code path #1489

[mfortman11]

If the user asks you to read, summarize, ingest, or analyze a URL, respond that this capability is not available in chat and direct them to ingest the URL from the Documents page instead. Do not attempt to fetch or ingest the URL.

Should they still be able to have the agent fetch/read a url?

[mpawlow]

⚠️ Superseded by #1474

Targeted for closure. Leaving PR open for ~1 week for emergency replacement

[lucaseduoli]

@mpawlow I think we can close this one now

[mpawlow]

Closed as SUPERSEDED