[agent-performance] Verify Metrics Collector workflow execution and data availability

[github-actions]

Problem

The Agent Performance Analyzer relies on historical metrics data from the Metrics Collector workflow, but during this run:

• Expected metrics directory not found: /tmp/gh-aw/repo-memory-default/memory/meta-orchestrators/metrics/
• No latest.json or daily/*.json files available
• Cannot perform trend analysis or compare to historical baselines

Expected Data Structure

According to the Agent Performance Analyzer instructions:

/tmp/gh-aw/repo-memory-default/memory/meta-orchestrators/
├── metrics/
│   ├── latest.json              # Most recent daily snapshot
│   └── daily/
│       ├── 2024-12-29.json     # Daily metrics for last 30 days
│       ├── 2024-12-28.json
│       └── ...

Metrics Expected

Each metrics file should contain:

1. Per-workflow metrics:

   • Safe output counts (issues, PRs, comments, discussions)
   • Workflow run statistics (total, successful, failed, success_rate)
   • Engagement metrics (reactions, comments, replies)
   • Quality indicators (merge rates, close times)

2. Ecosystem metrics:

   • Total workflow count
   • Engine distribution
   • Tool usage patterns
   • Overall success rates

Investigation Tasks

1. Verify Metrics Collector workflow exists and runs:

   • Check .github/workflows/metrics-collector.md
   • Verify schedule configuration (should be daily)
   • Check recent workflow runs

2. Verify repo-memory configuration:

   • Confirm branch: memory/meta-orchestrators
   • Check file glob patterns
   • Verify max file size limits

3. Test metrics collection:

   • Trigger manual workflow run
   • Verify data written to repo memory
   • Validate JSON structure

4. Debug permission issues:

   • Encountered "Permission denied" errors when accessing /tmp/gh-aw/repo-memory-default/
   • Verify directory permissions
   • Check workflow permissions configuration

Impact

Critical - Without metrics data, Agent Performance Analyzer cannot:

• Calculate actual quality scores
• Track effectiveness trends
• Compare current vs. historical performance
• Provide data-driven recommendations
• Measure improvement over time

Acceptance Criteria

• Metrics Collector workflow runs daily
• Latest metrics available at expected path
• Historical metrics (30 days) accessible
• JSON structure valid and complete
• Agent Performance Analyzer can read and parse data
• Directory permissions correct for workflow access

Priority

Critical - Core infrastructure for agent performance analysis

Estimated Effort

1-2 hours (investigation + fixes)

Potential Root Causes

1. Metrics Collector workflow not scheduled/running
2. Repo memory path incorrect or not initialized
3. Permission issues preventing writes
4. Branch name mismatch
5. File glob patterns not matching

Next Steps

1. Check if metrics-collector.md exists and review configuration
2. Verify last successful run of Metrics Collector
3. Test manual trigger to validate data collection
4. Review repo-memory configuration across workflows
5. Document metrics schema for validation

Related

• Blocks comprehensive agent performance analysis
• Complements #aw_enable_github_mcp (GitHub API access)
• Required for trend analysis and historical comparisons

AI generated by Agent Performance Analyzer - Meta-Orchestrator