Phase/langchain agent runtime

[Ryanakml]

Summary by CodeRabbit

• New Features

  • Added LLM message processing pipeline with support for OpenAI and Google Gemini providers.
  • Introduced intent classification to route messages (RAG queries, tool requests, escalations, clarifications).
  • Implemented built-in safety policy enforcement to detect and prevent inappropriate content.
  • Added structured output validation for consistent response formatting.
  • Enabled token usage tracking and cost estimation per LLM interaction.
  • Introduced versioned prompt management system.

• Tests

  • Added comprehensive test suites for prompt registry, pipeline orchestration, and cost calculation.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: df66476b-f5f4-4941-8312-9faec2d6f5a6

📥 Commits

Reviewing files that changed from the base of the PR and between abe71d7 and 683f3d7.

⛔ Files ignored due to path filters (1)

• package-lock.json is excluded by !**/package-lock.json

📒 Files selected for processing (22)

• packages/llm/package.json
• packages/llm/src/__tests__/prompts.test.ts
• packages/llm/src/index.ts
• packages/llm/src/langchain/__tests__/pipeline.test.ts
• packages/llm/src/langchain/chains/__tests__/composition.test.ts
• packages/llm/src/langchain/chains/classification.ts
• packages/llm/src/langchain/chains/composition.ts
• packages/llm/src/langchain/chains/confidence.ts
• packages/llm/src/langchain/chains/normalization.ts
• packages/llm/src/langchain/chains/policy.ts
• packages/llm/src/langchain/chains/retrieval.ts
• packages/llm/src/langchain/chains/router.ts
• packages/llm/src/langchain/pipeline.ts
• packages/llm/src/langchain/types.ts
• packages/llm/src/parsers/index.ts
• packages/llm/src/prompts/index.ts
• packages/llm/src/prompts/system.ts
• packages/llm/src/prompts/versioning.ts
• packages/llm/src/providers/gemini.ts
• packages/llm/src/providers/openai.ts
• packages/llm/src/router/__tests__/model-router.test.ts
• packages/llm/src/router/model-router.ts

---

📝 Walkthrough

Walkthrough

This pull request introduces a comprehensive LLM orchestration pipeline built on LangChain, adding configuration for OpenAI and Gemini providers, a prompt versioning system with safety invariants, semantic intent classification and routing, structured output parsing with Zod schemas, and an end-to-end message processing pipeline with multiple validation stages, alongside corresponding test coverage.

Changes

Cohort / File(s)	Summary
Package Configuration packages/llm/package.json	Added test script and dependencies: @langchain/core, @langchain/google-genai, @langchain/openai, zod, @types/node, and vitest dev dependency.
Type Definitions packages/llm/src/langchain/types.ts	New MessageContext and AgentState interfaces defining the state schema throughout the pipeline, including intent, confidence, routing, and safety fields.
Chain Components packages/llm/src/langchain/chains/normalization.ts, retrieval.ts, classification.ts, confidence.ts, policy.ts, router.ts, composition.ts	Seven independent RunnableLambda-based chain steps: input normalization, context retrieval, intent classification via keyword rules, confidence scoring by intent, policy safety checks, conditional routing via RunnableBranch, and composed response generation with structured output.
Prompt System packages/llm/src/prompts/versioning.ts, system.ts, index.ts	Semantic versioning registry for prompts with duplicate detection; mandatory SYSTEM_INVARIANTS embedded in two versioned base prompts (1.0.0, 1.1.0); factory functions for intent classification, RAG, and tool decision prompts.
Parser Definitions packages/llm/src/parsers/index.ts	Zod schemas and StructuredOutputParsers for intent classification (RAG/TOOL/CLARIFICATION/ESCALATION), standard responses, and structured output with content, confidence, and escalation flags.
Provider Adapters packages/llm/src/providers/openai.ts, gemini.ts	Configuration factories for ChatOpenAI and ChatGoogleGenerativeAI with environment fallback and sensible defaults (gpt-4o-mini, gemini-1.5-flash).
Model Router & Cost Estimation packages/llm/src/router/model-router.ts	Router combining OpenAI primary with Gemini fallback via withFallbacks; cost estimation from token usage metadata using provider-specific per-1K rates.
Pipeline Orchestration packages/llm/src/langchain/pipeline.ts, packages/llm/src/index.ts	ChainConfig interface, createStandardChain utility, processMessagePipeline sequence combining all seven chains, and processMessage entry point; re-exports of providers, prompts, parsers, and pipeline from main module index.
Test Suites packages/llm/src/__tests__/prompts.test.ts, packages/llm/src/langchain/__tests__/pipeline.test.ts, packages/llm/src/langchain/chains/__tests__/composition.test.ts, packages/llm/src/router/__tests__/model-router.test.ts	End-to-end pipeline tests with mocked model router validating normalization, intent detection, routing, composition, and safety; prompt versioning registry validation; composition chain error fallback; and cost calculation per provider.

Sequence Diagram(s)

sequenceDiagram
    actor Client
    participant NormChain as Normalization<br/>Chain
    participant RetrChain as Retrieval<br/>Chain
    participant ClassChain as Classification<br/>Chain
    participant ConfChain as Confidence<br/>Chain
    participant RouterChain as Router<br/>Chain
    participant CompChain as Composition<br/>Chain
    participant Model as Structured<br/>Model Router
    participant PolicyChain as Policy<br/>Chain

    Client->>NormChain: originalInput
    activate NormChain
    NormChain->>RetrChain: normalizedInput
    deactivate NormChain
    
    activate RetrChain
    RetrChain->>ClassChain: context (userId, conversationId, history)
    deactivate RetrChain
    
    activate ClassChain
    ClassChain->>ConfChain: intent (RAG/TOOL/CLARIFICATION/ESCALATION)
    deactivate ClassChain
    
    activate ConfChain
    ConfChain->>RouterChain: confidence (0.4-0.95)
    deactivate ConfChain
    
    activate RouterChain
    RouterChain->>CompChain: route (rag_path/tool_path/clarification_path/escalation_path)
    deactivate RouterChain
    
    activate CompChain
    CompChain->>Model: prompt + route + normalizedInput
    activate Model
    Model->>Model: apply StructuredOutputSchema
    Model->>CompChain: { content, confidence, escalate_flag }
    deactivate Model
    CompChain->>PolicyChain: composedResponse
    deactivate CompChain
    
    activate PolicyChain
    PolicyChain->>PolicyChain: check for 'inappropriate'
    PolicyChain->>Client: finalResponse + isSafe
    deactivate PolicyChain

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~50 minutes

Poem

🐰 Whiskers twitching with delight,
Chains of logic, links so tight,
From intent to routed path,
Safety checked and policy's wrath,
LLM flows like carrots bright! 🥕

✨ Finishing Touches

• 📝 Generate docstrings (stacked PR)
• 📝 Generate docstrings (commit on current branch)

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch phase/langchain-agent-runtime

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}