Skip to content

Forward lora field on Cloudflare provider so consumers can use Workers AI fine-tunes #51

@stackbilt-admin

Description

@stackbilt-admin

Problem

CloudflareProvider.formatRequest (src/providers/cloudflare.ts) does not forward a lora field to env.AI.run(). This blocks any consumer of @stackbilt/llm-providers from using a Cloudflare Workers AI fine-tune (LoRA adapter) through the package.

Concrete example: codebeast (Stackbilt-dev/codebeast) ships a fine-tuned Qwen via LoRA UUID 6d028a43-759e-417f-83fb-fa9b681d81f4 applied at inference to @cf/qwen/qwen2.5-coder-32b-instruct. To use it from another worker, callers today must bypass LLMProviders entirely and call env.AI.run(model, { ..., lora }) directly — which violates Stackbilt's "no bolted-in LLM logic, route everything through @stackbilt/llm-providers" policy.

Discovered while

Building aegis-daemon's new /api/internal/cheap-llm delegation route (aegis v2.2.0). Wanted the code task type to use the codebeast Qwen tune; ended up shipping with the base model and filing this issue rather than carving out a bypass.

Suggested change

  1. Extend LLMRequest (src/types.ts) with an optional lora?: string field (LoRA name or UUID — Workers AI accepts either).
  2. In CloudflareProvider.formatRequest, pass lora through to the ai.run() options object when present.
  3. Other providers ignore the field (no-op).
  4. README example showing how to call a fine-tune through the package.

Why optional, not required

LoRAs are CF-account-scoped resources. The package can't validate the UUID — it just needs to forward what the caller passes. Caller is responsible for ensuring the binding's account hosts the adapter.

Downstream

Once shipped, aegis-daemon's cheap-llm route can switch the code task default from base @cf/qwen/qwen2.5-coder-32b-instruct to the codebeast-tuned variant without violating feedback_no_bolted_llm_logic.md.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions