Optimize CLI version checker workflow based on performance analysis

[Copilot]

Overview

This PR optimizes the CLI version checker workflow based on analysis of 13 recent runs (Oct 16-22, 2025) which revealed significant performance and reliability issues:

• High failure rate: 30% (4 out of 13 runs failed)
• Excessive iterations: Failed runs had 55-108 turns vs 23-24 for successful runs
• High token usage: Failed runs consumed 1.1-2.7M tokens vs 350-400k for successful runs
• High costs: Failed runs cost $0.87-1.52 vs $0.35-0.42 for successful runs

Changes Made

1. Added max-turns Limit

engine:
  id: claude
  max-turns: 30  # Prevents runaway loops

This directly addresses the runaway loop issue where failed runs consumed 55-108 turns. Setting a limit of 30 (based on successful runs averaging 23-24 turns) prevents excessive iteration while allowing sufficient time for legitimate updates.

2. Enabled Persistent Cache Storage

tools:
  cache-memory: true

Provides persistent storage at /tmp/gh-aw/cache-memory/ across workflow runs to:

• Store previous version checks and timestamps
• Cache CLI help outputs for version comparison
• Track failed attempts to avoid redundant work
• Resume from where previous runs left off

3. Added Efficient npm Commands

bash:
  - "npm view *"   # Query package metadata
  - "npm list *"   # List installed packages

Replaces web-fetch calls to NPM registry with native npm CLI commands for faster, more reliable package metadata queries.

4. Enhanced Efficiency Guidelines

Added comprehensive optimization instructions:

• Check cache first: Verify cached version checks before re-fetching
• Early exit: Exit successfully if no updates detected
• Parallel fetching: Fetch all versions in a single turn using multiple npm view calls
• Cache help outputs: Store CLI help text for version comparison
• Smart installation: Only install CLIs if version has changed

5. Improved Error Handling

• Save workflow state to cache-memory before exiting on errors
• Resume from cached state on workflow restart
• Better recovery from transient failures

Expected Improvements

Based on the performance analysis:

Metric	Before	After	Improvement
Failure Rate	30%	<10%	67% reduction
Avg Token Usage	1.45M	~600k	58% reduction
Avg Run Time	6.5min	2-3min	54-69% reduction
Avg Cost/Run	$0.96	$0.35	64% reduction

Testing

• ✅ Workflow compiles successfully (gh-aw compile cli-version-checker)
• ✅ All unit tests pass (make test-unit)
• ✅ Code formatted (make fmt)
• ✅ Configuration correctly applied in generated lock file

Related Issue

Fixes issue created by Q workflow run: https://github.com/githubnext/gh-aw/actions/runs/18738365042

Original prompt

---

This section details on the original issue you should resolve

<issue_title>[q] Optimize CLI version checker based on performance analysis</issue_title>
<issue_description># Q Workflow Optimization Report

Issues Found (from live data)

CLI Version Checker Workflow

• Log Analysis: Analyzed 13 recent runs from Oct 16-22, 2025
• Run IDs Analyzed: 18614220764, 18648770163, 18703293232, 18659303991, 18720599909, 18680370502, 18635159273, 18610602186, 18628881536, 18610287473, 18557777486, 18604473910, 18589489936
• Issues Identified:
  • High Failure Rate: 4 out of 13 runs failed (30% failure rate)
  • Performance Variability:
    • Successful runs: 2-3 minutes, ~350-400k tokens, $0.35-0.42 cost
    • Failed runs: 4-8 minutes, 1.1-2.7M tokens, $0.87-1.52 cost (3-7x more expensive)
  • Excessive Turns: Failed runs had 55-108 turns vs 23-24 turns for successful runs
  • Repetitive Operations:
    • TodoWrite: 106 total calls across runs
    • WebFetch: 65 total calls across runs
    • BashOutput: 27 total calls across runs
  • Missing Tools: npm view and npm list commands for efficient package metadata queries

Changes Made

cli-version-checker.md

Performance Optimization:

1. Added max-turns limit: Set to 30 to prevent runaway loops

   • Evidence: Failed runs had 55-108 turns, successful runs had 23-24
   • Expected: Reduce failed runs from 30% to <10%

2. Enabled cache-memory: Persistent state across runs

   • Purpose: Store previous version checks, help output, and failed attempts
   • Expected: Reduce redundant work by 40-60%

3. Added npm tools: npm view * and npm list * bash commands

   • Purpose: More efficient package metadata queries than web-fetch
   • Expected: Faster version checking, fewer network calls

Efficiency Guidelines Added:

• Fetch versions in parallel using multiple WebFetch calls in a single turn
• Check cache-memory first before re-analyzing same versions
• Skip CLI installation if no version change detected
• Use npm view instead of web-fetch for package metadata
• Save help output to cache-memory for future comparisons

Error Handling Improvements:

• Save progress to cache-memory before exiting on errors
• Check cache-memory on restart to resume from where left off
• Exit early if no updates found (check versions first)

Expected Improvements

Based on analysis of successful vs failed runs:

• Failure Rate: Reduce from 30% (4/13 runs) to <10% via max-turns limit
• Token Usage: Reduce average by 40-60% through caching and early exit
  • Current avg: ~1.45M tokens per run
  • Expected: ~600k tokens per run
• Run Time: Reduce average from 6.5min to 2-3min for repeat checks
• Cost: Reduce from avg $0.96/run to $0.35/run
• Efficiency: Eliminate redundant npm installs and version fetches

Validation

Workflow compiled successfully using the compile tool from gh-aw MCP server:

• ✅ cli-version-checker.md

Note: .lock.yml file will be generated automatically after merge.

References

• Logs Summary: 13 runs analyzed, 18.8M total tokens, $12.49 total cost
• Tool Usage Data: TodoWrite (106 calls), WebFetch (65 calls), BashOutput (27 calls)
• Performance Metrics:
  • Best run: 2.1min, 365k tokens, $0.35
  • Worst run: 8.4min, 2.7M tokens, $1.52
• Failure Pattern: All failures showed >50 turns and >1M tokens

AI generated by Q

---

[!NOTE]
This was originally intended as a pull request, but the git push operation failed.

Workflow Run: View run details and download patch artifact

The patch file is available as an artifact (aw.patch) in the workflow run linked above.
To apply the patch locally:

# Download the artifact from the workflow run https://github.com/githubnext/gh-aw/actions/runs/18738365042
# (Use GitHub MCP tools if gh CLI is not available)
gh run download 18738365042 -n aw.patch
# Apply the patch
git am aw.patch

Show patch preview (159 of 159 lines)

From 25e223610e9b9a15dda0e16ae8fa80359535e04f Mon Sep 17 00:00:00 2001
From: Q <github-actions[bot]@users.noreply.github.com>
Date: Thu, 23 Oct 2025 05:25:20 +0000
Subject: [PATCH] Optimize CLI version checker workflow

Based on analysis of 13 recent workflow runs showing 30% failure rate and high token usage in failed runs:

Performance improvements:
- Add max-turns: 30 limit to prevent runaway loops (failed runs had 55-108 turns vs 23-24 for successful)
- Enable cache-memory for persistent state across runs
- Add npm view and npm list to bash tools for more efficient package queries

Efficiency guidelines added:
- Fetch versions in parallel using multiple WebFetch calls
- Check cache-memory first to avoid re-analyzing same versions
- Skip CLI installation if no update needed
- Use npm view instead of web-fetch wh...

</details>

Fixes githubnext/gh-aw#2203

<!-- START COPILOT CODING AGENT TIPS -->
---

✨ Let Copilot coding agent [set things up for you](https://github.com/githubnext/gh-aw/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot) — coding agent works faster and does higher quality work when set up for your repo.

[github-actions]

Agentic Changeset Generator triggered by this pull request.