Deterministic synthetic two-party conversation corpus generator for testing AI scoring systems.
-
Updated
May 20, 2026 - Python
#
eval-harness
Here are 9 public repositories matching this topic...
Tolerant apply_patch for LLM-generated diffs, plus an eval harness for code agents.
-
Updated
May 6, 2026 - Python
Can Computer-Use Agents manage buy-side procurement operations? A benchmark across live e-commerce, with multi-model adapters (Northstar, OpenAI, Claude, Gemini), Kernel-hosted browsers, and Harbor/ATIF-v1.6 trajectory export.
benchmark kernel procurement cua lightcone computer-use-agent eval-harness tzafon-northstar openai-computer-use
-
Updated
May 10, 2026 - Python
Form ADV Part 2A intelligence + peer benchmarking — LangGraph, hybrid retrieval, eval harness
-
Updated
Apr 28, 2026 - Python
A lightweight workbench for dataset-driven agent and LLM evaluation.
-
Updated
May 1, 2026 - Python
YAML-driven evaluation harness for WhatsApp RAG bots
-
Updated
Apr 29, 2026 - Python
Slack Q&A bot template — ingests Slack threads + markdown docs into Pinecone, then an n8n workflow retrieves, re-ranks on metadata, and posts answers back in-thread. Includes contextual retrieval and an eval harness.
starter-template openai slack-bot knowledge-base pinecone rag n8n vector-database llm retrieval-augmented-generation llamaparse contextual-retrieval chatbot-template eval-harness
-
Updated
May 22, 2026 - Python
Compare OpenClaw setups against the same scenario suite. Run prompts across multiple configurations, capture answers, latency, token usage, tool calls, and file reads, then generate a single comparison report.
python cli benchmarking regression-testing llm-eval llm-evaluation ai-agent-evaluation agent-testing eval-harness openclaw
-
Updated
Mar 31, 2026 - Python
A boring, config-driven harness for evaluating AI systems. One YAML drives the run, the trace is the source of truth. Offline, backtesting, and online-eval modes — works with any agent, RAG, or code-modifying system.
-
Updated
May 20, 2026 - Python
Improve this page
Add a description, image, and links to the eval-harness topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the eval-harness topic, visit your repo's landing page and select "manage topics."