feat: add OCI Generative AI provider — basic text completion by fede-kamel · Pull Request #4959 · crewAIInc/crewAI

fede-kamel · 2026-03-19T17:41:52Z

Summary

Add native OCI Generative AI text completion provider (OCICompletion) supporting generic (Meta, Google, OpenAI, xAI) and Cohere model families
Shared OCI auth utilities (utilities/oci.py) for API key, security token, instance principal, and resource principal auth
OCI registered as optional dependency (crewai[oci])
15 unit tests (mocked SDK) + 2 parametrized integration tests (tested live against meta.llama-3.3-70b-instruct, cohere.command-r-plus-08-2024, google.gemini-2.5-flash, openai.gpt-5.2-chat-latest)

This is PR 1 of a series — follow-up PRs will add streaming, tool calling, structured output, multimodal, and embeddings support.

Supersedes #4885 (closed per reviewer feedback to split by scope).
Tracking issue: #4944

What's included

File	Lines	What
`utilities/oci.py`	72	Shared `get_oci_module()` + `create_oci_client_kwargs()`
`llms/providers/oci/completion.py`	505	`OCICompletion(BaseLLM)` — init, message building, basic call/acall
`llm.py`	+13	Provider registration (routing, pattern matching, import)
`pyproject.toml`	+3	`oci` optional dependency
Tests	491	conftest + 15 unit + 2 integration (parametrized across models)

What's NOT included (deferred to follow-up PRs)

Streaming (iter_stream, astream)
Tool calling / function calling
Structured output (response_model)
Multimodal (vision, documents, audio, video)
OCI embeddings provider
OCI tools (will be a standalone PyPI package per community tools guidelines)

Test plan

15 unit tests pass (mocked OCI SDK, no credentials needed)
Integration tests pass against 4 model families via OCI GenAI API:
- meta.llama-3.3-70b-instruct
- cohere.command-r-plus-08-2024
- google.gemini-2.5-flash
- openai.gpt-5.2-chat-latest
Both sync (call) and async (acall) paths verified

Note

Medium Risk
Adds a new native LLM provider with multiple OCI authentication modes and new routing/pattern matching, which could affect model selection and introduce integration/auth edge cases. Changes are mostly additive and isolated, but involve external SDK calls and credential handling.

Overview
Adds first-class support for Oracle Cloud Infrastructure (OCI) Generative AI as a native LLM provider, including provider routing (oci/...) and model-pattern validation for OCI model IDs and dedicated endpoint OCIDs.

Introduces OCICompletion for basic synchronous/async chat-based text completion (generic + Cohere formats), plus shared OCI SDK utilities for lazy importing and building client auth kwargs (API key, security token, instance/resource principals). Also registers oci as an optional dependency (crewai[oci]) and adds mocked unit tests plus live integration tests for the new provider.

^{Reviewed by Cursor Bugbot for commit 7ef6c02. Bugbot is set up for automated code reviews on this repo. Configure here.}

lib/crewai/src/crewai/llms/providers/oci/completion.py

fede-kamel · 2026-03-19T17:52:45Z

@greysonlalonde Hey! Following up on your feedback from #4885 — I split the original PR into smaller, scoped pieces. This is the first one: just the basic text completion provider for OCI GenAI (no streaming, no tools, no structured output, no multimodal, no embeddings).

I couldn't trim this one further — it's the minimal foundation (provider class + shared auth + registration) plus tests. The source code is ~600 lines and the rest is test fixtures/tests. Each follow-up PR will layer on one capability at a time.

Also, per the community tools guide you shared — the OCI tools (InvokeAgent, KnowledgeBase, ObjectStorage) will go into a standalone PyPI package (crewai-oci-tools), not into the crewAI repo.

Would appreciate a review whenever you get a chance. Thanks!

lib/crewai/src/crewai/llms/providers/oci/completion.py

lib/crewai/src/crewai/llm.py

Add streaming text completion via OCI SSE events: - stream=True in call() routes to _stream_call_impl with chunk events - iter_stream() yields raw text chunks (sync generator) - astream() wraps iter_stream via thread+queue for async callers - _stream_chat_events holds client lock for full stream duration - SSE event parsing handles both string and mapping payloads Tested live against meta.llama-3.3-70b-instruct, cohere.command-r-plus-08-2024, google.gemini-2.5-flash, and openai.gpt-5.2-chat-latest. Depends on: crewAIInc#4959 Tracking issue: crewAIInc#4944

Add native function calling for generic and Cohere model families: - _format_tools converts CrewAI tool specs to OCI SDK format - _extract_tool_calls normalizes responses back to CrewAI shape - _handle_tool_calls executes tools and recurses until model finishes - Cohere tool message handling with trailing tool results - Tool choice control (auto/none/required/function) - Passthrough parameter filtering via SDK introspection - Streaming tool call accumulation from SSE fragments - supports_function_calling() returns True Tested live against meta.llama-3.3-70b-instruct with raw tool call return and recursive tool execution. Depends on: crewAIInc#4961 (streaming), crewAIInc#4959 (basic text) Tracking issue: crewAIInc#4944

Add response_model (Pydantic) support for structured output: - _build_response_format converts Pydantic schema to OCI JsonSchemaResponseFormat (generic) or CohereResponseJsonFormat - _parse_structured_response validates and returns typed models - response_model threaded through call, _call_impl, _stream_call_impl, and _handle_tool_calls for full coverage - Handles JSON in markdown fences via base class _validate_structured_output Tested live against meta.llama-3.3-70b-instruct and google.gemini-2.5-flash. Depends on: crewAIInc#4962 (tool calling), crewAIInc#4961 (streaming), crewAIInc#4959 (basic text) Tracking issue: crewAIInc#4944

Add multimodal content handling for generic model families: - vision.py: model lists, data URI helpers, image encoding utilities - _build_generic_content handles image_url, document_url, video_url, audio_url content types mapped to OCI SDK content objects - _message_has_multimodal_content detects non-text payloads - Cohere models reject multimodal with clear error message - supports_multimodal() returns True Depends on: crewAIInc#4963, crewAIInc#4962, crewAIInc#4961, crewAIInc#4959 Tracking issue: crewAIInc#4944

Add OCI embedding support integrated with CrewAI's RAG pipeline: - OCIEmbeddingFunction: ChromaDB-compatible embedding callable with batching, config serialization, image embedding support - OCIProvider: Pydantic-based provider with alias validation for env vars and config keys - Factory registration in embeddings/factory.py + types.py - Supports text and image embeddings, output dimensions, custom endpoints, all 4 OCI auth modes Tested live against cohere.embed-english-v3.0 with API_KEY auth. Depends on: crewAIInc#4964, crewAIInc#4963, crewAIInc#4962, crewAIInc#4961, crewAIInc#4959 Tracking issue: crewAIInc#4944

Replace asyncio.to_thread wrappers with true async I/O using aiohttp for acall() and astream(). The OCI SDK is sync-only, so we bypass it for HTTP and use its signer for request authentication directly. - oci_async.py: OCIAsyncClient with aiohttp, OCI request signing, native SSE parsing, connection pooling - acall(): true async chat completion (no thread pool) - astream(): true async SSE streaming (no thread+queue bridge) - Graceful fallback to asyncio.to_thread when aiohttp unavailable or client is mocked (unit tests) - aiohttp + certifi added to crewai[oci] optional deps Temporary measure until OCI SDK ships native async support. Tested live: acall, astream, and concurrent acall against meta.llama-3.3-70b-instruct with API_KEY auth. Depends on: crewAIInc#4966, crewAIInc#4964, crewAIInc#4963, crewAIInc#4962, crewAIInc#4961, crewAIInc#4959 Tracking issue: crewAIInc#4944