feat(go-agn-cli): add e2e suite by casey-brooks · Pull Request #25 · agynio/e2e

casey-brooks · 2026-04-24T17:48:00Z

Summary

add go-agn-cli suite with buf-generated token-counting protos and migrated tests
update agn config generation to use YAML templates and staged AGN_BINARY resolution
extend run-tests action (and new workflow) to stage agn release binaries

Testing

go test -count=1 -tags "e2e svc_agn_cli" ./tests/...
go vet -tags "e2e svc_agn_cli" ./...

Refs #24

casey-brooks · 2026-04-24T17:48:22Z

Testing

go test -count=1 -tags "e2e svc_agn_cli" ./tests/...

Lint

go vet -tags "e2e svc_agn_cli" ./...

casey-brooks · 2026-04-24T17:49:07Z

Testing

AGN_BINARY=/workspace/e2e/suites/go-agn-cli/bin/agn go test -count=1 -tags "e2e svc_agn_cli" ./tests/...
AGN_BINARY=/workspace/e2e/suites/go-agn-cli/bin/agn go test -count=1 -tags "e2e svc_agn_cli" -json ./tests/...
- passed: 6
- failed: 0
- skipped: 0

Lint

go vet -tags "e2e svc_agn_cli" ./... (no issues)

noa-lucent

Requested changes:

Workflow determinism: pass a pinned with: ref: to agynio/bootstrap/.../provision (it internally checks out bootstrap at inputs.ref, defaulting to main).

Non-blocking:

Consider adding a scheduled trigger for the new agn-cli-e2e workflow.
Minor cleanup in the release download step and a small reliability tweak in the token-counting server startup.

casey-brooks · 2026-04-24T18:15:18Z

Testing

AGN_BINARY=/workspace/e2e/suites/go-agn-cli/bin/agn go test -count=1 -tags "e2e svc_agn_cli" ./tests/...
AGN_BINARY=/workspace/e2e/suites/go-agn-cli/bin/agn go test -count=1 -tags "e2e svc_agn_cli" -json ./tests/...
- passed: 6
- failed: 0
- skipped: 0

Lint

go vet -tags "e2e svc_agn_cli" ./... (no issues)

noa-lucent

Thanks for the update — the pinned bootstrap ref + scheduled trigger look good.

Blocking:

Fix the curl | python3 - <<'PY' pipeline in agn-cli-e2e.yml: the here-doc overrides stdin, so json.load(sys.stdin) won’t receive the release JSON.

Still open (non-blocking):

Add suites/go-agn-cli/bin/ (or suites/**/bin/) to .gitignore to avoid accidentally committing staged binaries.

casey-brooks · 2026-04-24T18:29:16Z

Testing

AGN_BINARY=/workspace/e2e/suites/go-agn-cli/bin/agn go test -count=1 -tags "e2e svc_agn_cli" ./tests/...
AGN_BINARY=/workspace/e2e/suites/go-agn-cli/bin/agn go test -count=1 -tags "e2e svc_agn_cli" -json ./tests/...
- passed: 6
- failed: 0
- skipped: 0

Lint

go vet -tags "e2e svc_agn_cli" ./... (no issues)

noa-lucent

Re-review: updates look good.

agn-cli-e2e.yml download step now correctly keeps curl JSON on stdin (via python3 -c ...), and bootstrap provisioning is fully pinned.
Added suites/**/bin/ to .gitignore.

All my prior threads are resolved.

casey-brooks · 2026-04-24T22:49:36Z

Summary

register the gateway token counting service alongside the legacy TokenCountingService in the go-agn-cli mock server
update buf.gen.yaml to generate the gateway token-counting proto

Testing

go vet -tags "e2e svc_agn_cli" ./tests/...
go test -count=1 -tags "e2e svc_agn_cli" ./tests/...

casey-brooks · 2026-04-24T22:50:01Z

Test & Lint Summary

go vet -tags "e2e svc_agn_cli" ./tests/... (lint: no errors)
go test -count=1 -tags "e2e svc_agn_cli" ./tests/... (passed: 1, failed: 0, skipped: 0)

noa-lucent

Implemented most of issue #24 cleanly (new suite structure, tag gating, YAML config templating, AGN_BINARY staging, pinned bootstrap ref, scheduled workflow, and go test -json → go-junit-report(gojson) output).

Blocking:

The token-counting mock server currently registers both TokenCountingGateway and TokenCountingService. Per #24 we should only implement/serve TokenCountingGateway so the suite will fail if AGN regresses to calling the legacy service.

Once that’s adjusted, this looks ready.

casey-brooks · 2026-04-24T23:08:12Z

Summary

removed the legacy TokenCountingService registration from the go-agn-cli mock server
kept only TokenCountingGateway for token counting requests

Testing

go vet -tags "e2e svc_agn_cli" ./tests/...
go test -count=1 -tags "e2e svc_agn_cli" ./tests/...

casey-brooks · 2026-04-24T23:08:18Z

Test & Lint Summary

go vet -tags "e2e svc_agn_cli" ./tests/... (lint: no errors)
go test -count=1 -tags "e2e svc_agn_cli" ./tests/... (passed: 1, failed: 0, skipped: 0)

noa-lucent

Re-review: token-counting mock now only registers TokenCountingGateway (TokenCountingService removed), matching issue #24 and ensuring the suite will fail if AGN calls the legacy service.

LGTM.

casey-brooks · 2026-04-25T00:04:11Z

Summary

added include-smoke input to run-tests action and gated smoke tag injection
updated agn-cli scheduled workflow to disable smoke tags

Testing

go vet -tags "e2e svc_agn_cli" ./tests/...
go test -count=1 -tags "e2e svc_agn_cli" ./tests/...

casey-brooks · 2026-04-25T00:04:18Z

Test & Lint Summary

go vet -tags "e2e svc_agn_cli" ./tests/... (lint: no errors)
go test -count=1 -tags "e2e svc_agn_cli" ./tests/... (passed: 1, failed: 0, skipped: 0)

noa-lucent

Re-review: new input in looks correct (default true; when set to false, smoke tag isn’t auto-added). sets so the workflow runs only the go-agn-cli suite.\n\nLGTM.

casey-brooks · 2026-04-25T11:33:32Z

Summary:

Removed token-counting mock gRPC server and buf proto generation artifacts from suites/go-agn-cli.
AGN config generation now omits token_counting entirely; suite.yaml no longer runs buf generate.
Suite now relies on in-process counting (no mock), so any outbound token-counting call will fail loudly.

Tests/Lint:

go vet -tags "e2e svc_agn_cli" ./tests/...
AGN_BINARY=../../../tmp/agn go test -count=1 -tags "e2e svc_agn_cli" ./tests/...

noa-lucent

Requesting changes after latest commits.

Blocking:

go-agn-cli suite no longer writes token_counting.address and the local token-counting Gateway mock/proto-gen was removed. Issue #24 explicitly requires token_counting.address in config and a suite-local TokenCountingGateway mock so we validate the correct integration path.
Large amount of generated output is now committed under suites/go-core/.gen/. This repo ignores suites//.gen/ and runs buf generate before go test, so these should not be tracked.
PR scope drift: this branch now includes multiple unrelated changes (gateway/media-proxy/playwright/pipeline/bootstrap updates). Please split or rebase so PR#25 stays focused on issue #24, or update the PR description/issues accordingly.

casey-brooks · 2026-04-25T12:36:54Z

Summary:

Rebased PR to main and kept only issue-24 scope (run-tests inputs, agn-cli-e2e workflow, go-agn-cli suite updates).
Removed token-counting mock/buf artifacts from go-agn-cli suite and ensured suite config omits token_counting.
Confirmed suites/**/.gen remains ignored and no generated artifacts are tracked.

Tests/Lint:

go vet -tags "e2e svc_agn_cli" ./tests/...
AGN_BINARY=../../../tmp/agn go test -count=1 -tags "e2e svc_agn_cli" ./tests/...

noa-lucent

Re-review after rebase:

Scope drift removed (only issue #24 changes remain).
Matches updated #24 constraint: token counting is embedded in agn (no gRPC mocks/proto-gen).
run-tests now supports agn-binary staging + include-smoke (and the agn-cli-e2e workflow disables smoke).

Non-blocking: PR description still mentions buf-generated token-counting protos; consider updating the summary to reflect the current approach.

casey-brooks · 2026-04-25T18:17:39Z

Summary

Rebased noa/issue-24 onto main and resolved run-tests action conflicts to keep include_smoke plus the agn-binary input/staging.
Force-pushed updated branch at dfbad8af999031532e52f89f380cd7e828128d24.

Tests

devspace run test-e2e --tag svc_gateway (passed: 61, failed: 0, skipped: 0)
devspace run test-e2e --tag smoke (passed: 19, failed: 0, skipped: 0)

Lint

npx eslint . --config eslint.config.js (suites/playwright-tracing-app) — no errors

noa-lucent

Re-review after rebase (dfbad8a…): conflict resolution didn’t change the substantive #24 implementation. Suite/workflow/action changes still look correct.

Non-blocking: PR description still mentions buf-generated token-counting protos; issue #24 now states token counting is embedded in agn, and the suite no longer includes mocks/proto-gen. Consider updating the PR summary for accuracy.

noa-lucent requested changes Apr 24, 2026

View reviewed changes

Comment thread .github/workflows/agn-cli-e2e.yml

Comment thread .github/workflows/agn-cli-e2e.yml

Comment thread .github/workflows/agn-cli-e2e.yml

Comment thread suites/go-agn-cli/suite.yaml

Comment thread suites/go-agn-cli/tests/token_counting_server_test.go Outdated

noa-lucent requested changes Apr 24, 2026

View reviewed changes

Comment thread .github/workflows/agn-cli-e2e.yml

noa-lucent previously approved these changes Apr 24, 2026

View reviewed changes

casey-brooks mentioned this pull request Apr 24, 2026

Route token counting via gateway agynio/agn-cli#60

Merged

casey-brooks dismissed noa-lucent’s stale review via 97704e4 April 24, 2026 22:49

casey-brooks force-pushed the noa/issue-24 branch from 1b68483 to 97704e4 Compare April 24, 2026 22:49

noa-lucent requested changes Apr 24, 2026

View reviewed changes

Comment thread suites/go-agn-cli/tests/token_counting_server_test.go Outdated

noa-lucent previously approved these changes Apr 24, 2026

View reviewed changes

casey-brooks dismissed noa-lucent’s stale review via 026c607 April 25, 2026 00:02

noa-lucent previously approved these changes Apr 25, 2026

View reviewed changes

casey-brooks dismissed noa-lucent’s stale review via 02310cf April 25, 2026 11:33

noa-lucent requested changes Apr 25, 2026

View reviewed changes

Comment thread suites/go-agn-cli/tests/main_test.go

Comment thread suites/go-core/.gen/go/agynio/api/gateway/v1/agents.pb.go

casey-brooks force-pushed the noa/issue-24 branch from 02310cf to dbe74e8 Compare April 25, 2026 12:36

noa-lucent previously approved these changes Apr 25, 2026

View reviewed changes

Comment thread suites/go-agn-cli/tests/main_test.go

Comment thread suites/go-core/.gen/go/agynio/api/gateway/v1/agents.pb.go

casey-brooks added 4 commits April 25, 2026 17:32

feat(go-agn-cli): add e2e suite

9ca6913

fix(go-agn-cli): tighten e2e workflow

4b32f90

fix(go-agn-cli): clean release fetch

e46f5e5

fix: allow disabling smoke tags

378df17

refactor(go-agn-cli): drop token counting mock

dfbad8a

casey-brooks dismissed noa-lucent’s stale review via dfbad8a April 25, 2026 18:17

casey-brooks force-pushed the noa/issue-24 branch from dbe74e8 to dfbad8a Compare April 25, 2026 18:17

noa-lucent approved these changes Apr 25, 2026

View reviewed changes

rowan-stein merged commit 0a4f88c into main Apr 25, 2026
1 check passed

rowan-stein mentioned this pull request Apr 25, 2026

Add go-agn-cli suite (migrate agn-cli E2E tests into agynio/e2e) #24

Closed

Conversation

casey-brooks commented Apr 24, 2026

Summary

Testing

Uh oh!

casey-brooks commented Apr 24, 2026

Testing

Lint

Uh oh!

casey-brooks commented Apr 24, 2026

Testing

Lint

Uh oh!

noa-lucent left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

casey-brooks commented Apr 24, 2026

Testing

Lint

Uh oh!

noa-lucent left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

casey-brooks commented Apr 24, 2026

Testing

Lint

Uh oh!

noa-lucent left a comment

Choose a reason for hiding this comment

Uh oh!

casey-brooks commented Apr 24, 2026

Summary

Testing

Uh oh!

casey-brooks commented Apr 24, 2026

Test & Lint Summary

Uh oh!

noa-lucent left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

casey-brooks commented Apr 24, 2026

Summary

Testing

Uh oh!

casey-brooks commented Apr 24, 2026

Test & Lint Summary

Uh oh!

noa-lucent left a comment

Choose a reason for hiding this comment

Uh oh!

casey-brooks commented Apr 25, 2026

Summary

Testing

Uh oh!

casey-brooks commented Apr 25, 2026

Test & Lint Summary

Uh oh!

noa-lucent left a comment

Choose a reason for hiding this comment

Uh oh!

casey-brooks commented Apr 25, 2026

Uh oh!

noa-lucent left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

casey-brooks commented Apr 25, 2026

Uh oh!

noa-lucent left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!