-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Description
Parent Issue
Part of #27
Description
Package the operator for deployment, integrate into local dev environment, and validate with a real showcase agent.
Scope
Helm chart
- CRD installed via Helm chart (following agent-runtime-operator pattern)
- Operator Deployment with RBAC (ServiceAccount, ClusterRole, ClusterRoleBinding)
- Configurable values: image, namespace, resource limits, testworkflow template names
- Webhook configuration for CRD validation (if implemented in Operator: Reconciler implementation, status reporting, and tests #29)
Docker image
- Dockerfile for the operator binary (multi-stage build)
make docker-buildandmake docker-pushtargets
Tilt integration
- Add operator to
Tiltfilefor local development (build image, deploy chart, live reload) - Operator starts automatically alongside existing infrastructure (testkube, agents, LGTM)
Update deploy/local/ examples
- Replace manual ConfigMap, TestWorkflow, and TestTrigger with
ExperimentCR - Keep existing examples as reference or remove if fully superseded
Showcase agent validation
- Create an
ExperimentCR targeting showcase-cross-selling agent - Deploy via Tilt and verify full pipeline executes end-to-end:
- Operator reconciles CR into ConfigMap + TestWorkflow + TestTrigger
- TestWorkflow runs all phases (run → evaluate → publish → visualize)
- Metrics published to OTLP endpoint and visible in Grafana
- Document any issues or adjustments needed for real-world agent compatibility
Acceptance Criteria
- Helm chart installs CRD + operator with correct RBAC
-
tilt updeploys operator alongside existing infrastructure -
deploy/local/updated to useExperimentCR -
ExperimentCR for showcase-cross-selling agent reconciles successfully - Full pipeline executes and metrics appear in Grafana
- Docker image builds via Makefile
References
- Issue Testbench Operator: Reconcile Experiment CRD to TestWorkflows #27 Helm chart and deployment sections
- agent-runtime-operator for Helm chart structure
- showcase-cross-selling for validation target
- Existing
chart/directory for template patterns
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels
Type
Projects
Status
No status