OpenCode sends tools (and Jinja tool template) to llama.cpp results in 500 error unless --jinja; with --jinja, template crashes (reject filter)

[brianluby]

Environment

• Client: OpenCode via Homebrew on macOS — v0.4.41 (also seen with 0.4.40)
• Server: llama.cpp OpenAI‑compatible server (Windows)
• Model: Qwen3‑30B A3B Coder (GGUF), served via llama.cpp; alias qwen3-30b
• Endpoint: http://{server-ip}:8080/v1
• Auth header: Authorization: Bearer test-api-key

Summary
OpenCode includes a tools parameter and injects a Jinja <tools> prompt block even in “plain chat”.

• If llama.cpp runs without --jinja, it rejects the request with: tools param requires --jinja flag.
• If llama.cpp runs with --jinja, it attempts to render OpenCode’s Jinja tools block and fails with a 500 due to missing filters (e.g., reject), producing a long “Value is not callable” stack trace.

Reproduction (no --jinja)

1. Start llama.cpp:

   llama-server.exe -m C:\models\Qwen3-30B-A3B-Coder-480B-Distill-v2-Q8_0.gguf --alias qwen3-30b --host 0.0.0.0 --port 8080 --api-key test-api-key

2. OpenCode provider points to http://{server-ip}:8080/v1, agent set to “plain chat” (no tools in config).

3. Send “hi” in OpenCode.

Actual: Server 500 + log: tools param requires --jinja flag.
Expected: If no tools are configured, OpenCode should omit the tools key entirely.

Reproduction (with --jinja)

1. Start llama.cpp:

   llama-server.exe -m C:\models\Qwen3-30B-A3B-Coder-480B-Distill-v2-Q8_0.gguf --alias qwen3-30b --host 0.0.0.0 --port 8080 --api-key test-api-key --jinja

2. Same OpenCode config; send “hi”.

Actual: llama.cpp 500 with Jinja crash while rendering tool template. Excerpt:

Value is not callable: null at row 58, column 110:
{%- for json_key in param_fields.keys() | reject("in", handled_keys) %}
...
{{- "<tools>" }} {%- for tool in tools %} ...

Expected: Either don’t send Jinja in system prompt, or send OpenAI‑native tool schema without relying on server‑side Jinja filters.

Sanity checks (curl)

• With --jinja, server accepts tools: []:

  curl http://{server-ip}:8080/v1/chat/completions \
    -H "Content-Type: application/json" -H "Authorization: Bearer test-api-key" \
    -d '{"model":"qwen3-30b","messages":[{"role":"user","content":"Say hi"}],"tools":[]}'

Returns completion.

• Without --jinja, server accepts when no tools key is present:

  curl http://{server-ip}:8080/v1/chat/completions \
    -H "Content-Type: application/json" -H "Authorization: Bearer test-api-key" \
    -d '{"model":"qwen3-30b","messages":[{"role":"user","content":"Say hi"}]}'

Returns completion.

Why this seems client‑side
Even with a “plain chat” agent and no tools in my opencode.json, OpenCode still sends either:

• a tools field in the JSON payload, or
• a Jinja <tools> block in the system prompt.

This causes failures on llama.cpp depending on --jinja. Changelog 0.4.40 mentioned “disable todo tools for qwen models”, but the failures persist (they’re triggered by the prompt template/tool scaffolding, not the model).

Requested improvements

1. Conditional tools emission: If an agent has no tools configured or the provider is generic OpenAI‑compatible (e.g., llama.cpp), omit tools and tool_choice entirely. Consider auto‑retry without tools on 4xx/5xx indicating unsupported tools.
2. Config switch: A per‑agent flag like "sendTools": false that guarantees no tools are sent.
3. Jinja‑free prompting mode: Provide a “raw OpenAI” prompt path (no {% ... %} in system prompt).
4. Qwen defaults: Extend 0.4.40’s Qwen compatibility so all tool scaffolding is disabled by default for Qwen on OpenAI‑compatible providers, unless explicitly enabled.

Workarounds that fixed it for me

• Start llama.cpp with --jinja and supply a minimal chat template that ignores tools, or
• Run OpenCode through a tiny local proxy that strips tools/tool_choice from requests (works flawlessly with llama.cpp without --jinja).

Artifacts (can provide on request)

• Server logs with 500 traces (redacted)
• opencode.json (redacted)
• Minimal proxy script that strips tools and demonstrates successful operation

Thanks! Happy to test a dev build or provide more logs.

---

[shantur]

@thdxr - Any workaround or plan on this?

[rekram1-node]

@shantur so to confirm you want to be able to run opencode without any tools enabled at all? Or is it that opencode needs to send in different format for llama.cpp? Just trying to make sure i am following here because I haven't used llama.cpp yet

[hollandThomas]

Not entirely sure if related but I also can't seem to get OpenCode functioning properly with llama.cpp

OpenCode 0.15.29

# Start llama.cpp
~/llama.cpp/build/bin/llama-server -m ~/models/Q5_K_M/GLM-4.5-Air-Q5_K_M-merged.gguf --alias "GLM-4.5-Air-Q5_K_M" --host 0.0.0.0 --port 8080 --ctx-size 131072 --jinja

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "@ai-sdk/openai-compatible": {
      "name": "llama.cpp",
      "options": {
        "baseURL": "http://0.0.0.0:8080/v1"
      },
      "models": {
        "GLM-4.5": {
          "name": "GLM-4.5"
        }
      }
    }
  }
}

# Start an OpenCode session, Plan agent (this works). E.g.
> Please create a plan for a TypeScript project that is a Snake game that runs in the terminal. Use Bun as a runtime.

Here's a comprehensive plan for a TypeScript terminal Snake game using Bun:

## Project Structure

snake-game/
├── src/
│   ├── game/
│   │   ├── Game.ts         # Main game class
│   │   ├── Snake.ts        # Snake entity
│   │   ├── Food.ts         # Food entity
│   │   └── GameBoard.ts    # Game board and rendering
[…]

# Change to Build agent
> Please proceed with implementation

I'll implement the terminal Snake game in TypeScript using Bun. Let me start by creating the project structure and core
files. <tool_call>list <arg_key>path</arg_key> <arg_value>/home/tho/Projects/dummy</arg_value> </tool_call>

And there it just stops. I might be configuring it wrong tho, not sure 🤷‍♂️

Edit:
I hit it with "proceed" a couple of times and now it does something. It's all along these lines though. It managed to create a package.json but then failed to move on. Seems like I should figure out how to configure this better.

Terminal-Snake

The arguments provided to the tool are invalid: Model tried to call unavailable tool 'terminal-snake'. Available tools:
invalid, bash, edit, webfetch, glob, grep, list, read, write, todowrite, todoread, task.

<tool_call>write <arg_key>filePath</arg_key> <arg_value>/home/tho/Projects/dummy/tsconfig.json</arg_value>
<arg_key>content</arg_key> <arg_value>{ "compilerOptions": { "target": "ES2022", "module": "ESNext", "moduleResolution":
"bundler", "lib": ["ES2022"], "outDir": "./dist", "rootDir": "./src", "strict": true, "esModuleInterop": true,
"skipLibCheck": true, "forceConsistentCasingInFileNames": true, "declaration": true, "declarationMap": true, "sourceMap":
true, "removeComments": true, "noImplicitAny": true, "strictNullChecks": true, "strictFunctionTypes": true,
"noImplicitThis": true, "noImplicitReturns": true, "noFallthroughCasesInSwitch": true, "noUncheckedIndexedAccess": true,
"noImplicitOverride": true, "allowUnusedLabels": false, "allowUnreachableCode": false, "exactOptionalPropertyTypes": true,
"baseUrl": ".", "paths": { "@/": ["./src/"] } }, "include": ["src/**/*"], "exclude": ["node_modules", "dist"] }</arg_value>
</tool_call>

[rekram1-node]

@hollandThomas what is the context size you have llama cpp? This usually happens with local models when the llm doesnt get enough context, or the model may just suck at tool calling but it can happen in either case

[grigio]

@rekram1-node I've 32684

      /app/llama-server
      --model /models/Qwen3-Coder-30B-A3B-Instruct-IQ4_NL.gguf
      --rope-scaling yarn
      --rope-scale 3.2
      --ctx-size 32684
      --seed 3407
      --jinja
      --temp 0.7
      --top-p 0.9
      --top-k 20
      --min-p 0.0
      --repeat-penalty 1.05
      -ngl 99
      --port 9999

[grigio]

OK, this smaller model seems to work

  "Qwen3-0.6B-Q4_K_M":
    proxy: "http://127.0.0.1:9999"
    cmd: >
      /app/llama-server
      --model /models/Qwen3-0.6B-Q4_K_M.gguf
      --jinja
      --ctx-size 16384
      --seed 3407
      --prio 3
      --temp 0.6
      --min-p 0.0
      --top-p 0.95
      --top-k 20
      --port 9999

[marceldev89]

llama.cpp + Qwen3-Coder-* requires ggml-org/llama.cpp#16755 to work properly. Been using this since august-ish and it's working fine besides some minor model specific issues.