Download agent file from GitHub on demand with version-aware URL patching #13612
Conversation
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
…rkdown files Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Added cleanup logic in commit 98c479d. The Cleanup includes:
|
|
@copilot The .github/agents/agentic-workflows.agent.md still needs to be written by the init command so it needs to be embedded by the go compiler. |
🔍 PR Triage ResultsCategory: refactor | Risk: high | Priority: 68/100 Scores Breakdown
📋 Recommended Action: fast_trackSignificant architectural improvement that simplifies the codebase by removing 5000+ lines of embedded template code. The PR makes Triaged by PR Triage Agent on 2026-02-04T00:35:56Z
|
|
@copilot remove the agent.md file embedded from the go compiler and installl download it on demand from GitHub. If the cli is a release build, use the release tag to get the version matching the current binary. |
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Fixed in commit 24b002a. The
The agent file is now properly embedded while prompt files remain unbundled. |
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Implemented in commit 6d583a8. The agent file is now downloaded on demand from GitHub:
The binary no longer embeds any markdown files. |
|
@copilot the downloaded agent file should patch the GitHub/gh-aw URLs to match the release tag |
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Implemented in commit a040410. Added
Example: For release v1.2.3, |
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR refactors the template file system to eliminate binary bloat by downloading the agent file from GitHub on demand rather than embedding markdown files in the binary. The agent file is version-aware, downloading from the appropriate release tag or main branch, and URLs within the file are automatically patched to match the installed version.
Changes:
- Removed all
go:embeddirectives and deleted 8 template markdown files (~5000 lines) frompkg/cli/templates/ - Added
downloadAgentFileFromGitHub()to fetch the agent file from GitHub with 30-second timeout - Implemented
patchAgentFileURLs()to convert local paths to versioned GitHub URLs - Modified
ensureAgenticWorkflowsDispatcher()to download and write agent file instead of using embedded template - Changed all other
ensure*functions to check file existence only (no creation from templates) - Added
deleteOldTemplateFiles()integrated intogh aw fix --writefor cleanup - Updated
.github/agents/agentic-workflows.agent.mdto reference prompts via GitHub URLs instead of local paths - Removed
sync-templatestarget from Makefile
Reviewed changes
Copilot reviewed 24 out of 24 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| pkg/cli/commands.go | Added download and URL patching functions with version detection |
| pkg/cli/copilot-agents.go | Refactored ensure functions to download agent file and check file existence only |
| pkg/cli/fix_command.go | Integrated template cleanup into fix command |
| pkg/cli/*_test.go | Updated tests to reflect new behavior (check existence vs create from template) |
| pkg/cli/templates/*.md | Deleted all embedded template files (8 files) |
| .github/agents/agentic-workflows.agent.md | Updated to use GitHub URLs for prompt references |
| Makefile | Removed sync-templates target and dependencies |
Comments suppressed due to low confidence (1)
pkg/cli/commands.go:115
- The URL patching at line 114 will replace ALL occurrences of
/blob/main/in the content, not just those in GitHub URLs. This could potentially modify unintended content if the string/blob/main/appears in:
- Code examples showing how to construct GitHub URLs
- Explanatory text or documentation
- Markdown code blocks
- Comments in the agent file
For example, if the agent file contains:
Example: `https://github.com/other-org/other-repo/blob/main/file.md` should not be changedThis would incorrectly become:
Example: `https://github.com/other-org/other-repo/blob/v1.2.3/file.md`Consider making the replacement more targeted by:
- Only replacing within
github.com/github/gh-aw/URLs - Using a regex pattern that matches the full URL structure
- Or documenting that the agent file should avoid this pattern in non-reference contexts
Example safer implementation:
if ref != "main" {
// Only replace in gh-aw repository URLs
content = strings.ReplaceAll(content,
"https://github.com/github/gh-aw/blob/main/",
fmt.Sprintf("https://github.com/github/gh-aw/blob/%s/", ref))
} if ref != "main" {
content = strings.ReplaceAll(content, "/blob/main/", fmt.Sprintf("/blob/%s/", ref))
}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| func downloadAgentFileFromGitHub(verbose bool) (string, error) { | ||
| commandsLog.Print("Downloading agentic-workflows.agent.md from GitHub") | ||
|
|
||
| // Determine the ref to use (tag for releases, main for dev builds) | ||
| ref := "main" | ||
| currentVersion := GetVersion() | ||
|
|
||
| // If version looks like a release tag (starts with v and contains dots), use it | ||
| isRelease := strings.HasPrefix(currentVersion, "v") && strings.Contains(currentVersion, ".") | ||
| if isRelease { | ||
| ref = currentVersion | ||
| commandsLog.Printf("Using release tag: %s", ref) | ||
| if verbose { | ||
| fmt.Fprintln(os.Stderr, console.FormatInfoMessage(fmt.Sprintf("Using release version: %s", ref))) | ||
| } | ||
| } else { | ||
| commandsLog.Print("Using main branch for dev build") | ||
| if verbose { | ||
| fmt.Fprintln(os.Stderr, console.FormatInfoMessage("Using main branch (dev build)")) | ||
| } | ||
| } | ||
|
|
||
| //go:embed templates/update-agentic-workflow.md | ||
| var updateWorkflowPromptTemplate string | ||
| // Construct the raw GitHub URL | ||
| url := fmt.Sprintf("https://raw.githubusercontent.com/github/gh-aw/%s/.github/agents/agentic-workflows.agent.md", ref) | ||
| commandsLog.Printf("Downloading from URL: %s", url) | ||
|
|
||
| //go:embed templates/create-shared-agentic-workflow.md | ||
| var createSharedAgenticWorkflowPromptTemplate string | ||
| // Create HTTP client with timeout | ||
| client := &http.Client{ | ||
| Timeout: 30 * time.Second, | ||
| } | ||
|
|
||
| //go:embed templates/debug-agentic-workflow.md | ||
| var debugWorkflowPromptTemplate string | ||
| // Download the file | ||
| resp, err := client.Get(url) | ||
| if err != nil { | ||
| return "", fmt.Errorf("failed to download agent file: %w", err) | ||
| } | ||
| defer resp.Body.Close() | ||
|
|
||
| //go:embed templates/upgrade-agentic-workflows.md | ||
| var upgradeAgenticWorkflowsPromptTemplate string | ||
| if resp.StatusCode != http.StatusOK { | ||
| return "", fmt.Errorf("failed to download agent file: HTTP %d", resp.StatusCode) | ||
| } | ||
|
|
||
| //go:embed templates/serena-tool.md | ||
| var serenaToolTemplate string | ||
| // Read the content | ||
| content, err := io.ReadAll(resp.Body) | ||
| if err != nil { | ||
| return "", fmt.Errorf("failed to read agent file content: %w", err) | ||
| } | ||
|
|
||
| // SetVersionInfo sets the version information for the CLI and workflow package | ||
| func SetVersionInfo(v string) { | ||
| version = v | ||
| workflow.SetDefaultVersion(v) // Keep workflow package in sync | ||
| contentStr := string(content) | ||
|
|
||
| // Patch URLs to match the current version/ref | ||
| patchedContent := patchAgentFileURLs(contentStr, ref) | ||
| if patchedContent != contentStr && verbose { | ||
| fmt.Fprintln(os.Stderr, console.FormatInfoMessage(fmt.Sprintf("Patched URLs to use ref: %s", ref))) | ||
| } | ||
|
|
||
| commandsLog.Printf("Successfully downloaded agent file (%d bytes)", len(patchedContent)) | ||
| return patchedContent, nil | ||
| } |
There was a problem hiding this comment.
The downloadAgentFileFromGitHub() function has no test coverage. While integration testing network calls can be complex, consider adding unit tests for:
- Error handling paths: Test that errors are properly formatted and returned
- URL construction logic: Test that the URL is correctly constructed for different version formats
- HTTP response handling: Mock HTTP responses to test status code handling (404, 500, etc.)
- Content patching: Test the integration between download and URL patching
Example test structure:
func TestDownloadAgentFileFromGitHub(t *testing.T) {
// Use httptest to mock GitHub responses
tests := []struct {
name string
version string
mockResponse string
mockStatus int
expectedError bool
expectedRef string
}{
{
name: "successful download for release version",
version: "v1.2.3",
mockResponse: "# Agent content with `.github/aw/file.md`",
mockStatus: 200,
expectedError: false,
expectedRef: "v1.2.3",
},
// ... more test cases
}
}This would improve confidence in the download logic without requiring actual network calls during testing.
| func patchAgentFileURLs(content, ref string) string { | ||
| // Pattern 1: Convert local paths to GitHub URLs | ||
| // `.github/aw/file.md` -> `https://github.com/github/gh-aw/blob/{ref}/.github/aw/file.md` | ||
| content = strings.ReplaceAll(content, "`.github/aw/", fmt.Sprintf("`https://github.com/github/gh-aw/blob/%s/.github/aw/", ref)) | ||
|
|
||
| // Pattern 2: Update existing GitHub URLs to use the correct ref | ||
| // https://github.com/github/gh-aw/blob/main/ -> https://github.com/github/gh-aw/blob/{ref}/ | ||
| if ref != "main" { | ||
| content = strings.ReplaceAll(content, "/blob/main/", fmt.Sprintf("/blob/%s/", ref)) | ||
| } | ||
|
|
||
| return content | ||
| } |
There was a problem hiding this comment.
The URL patching logic in patchAgentFileURLs() uses simple strings.ReplaceAll() which could potentially cause unintended replacements in edge cases.
Consider this scenario: if the agent file contains explanatory text like "Files stored at .github/aw/ are treated as..." or code examples showing how to write paths like ".github/aw/file.md", ALL occurrences will be patched, not just those in actual reference links.
While this may not be a critical issue in practice (the agent file likely only has reference links), the implementation could be more precise by:
- Using a more specific pattern that only matches actual markdown links
- Or documenting that the agent file should avoid using this exact pattern in non-reference contexts
However, given the scope of the PR and the controlled nature of the agent file content, this is a minor concern that could be addressed in a follow-up if issues arise.
| // Download the agent file from GitHub | ||
| agentContent, err := downloadAgentFileFromGitHub(verbose) | ||
| if err != nil { | ||
| copilotAgentsLog.Printf("Failed to download agent file from GitHub: %v", err) | ||
| return fmt.Errorf("failed to download agent file from GitHub: %w", err) | ||
| } |
There was a problem hiding this comment.
The ensureAgenticWorkflowsDispatcher() function now downloads the agent file from GitHub on demand, which introduces a dependency on network connectivity. If the download fails (no internet, GitHub outage, firewall blocking raw.githubusercontent.com), the function returns an error that will propagate up and fail operations like gh aw init and gh aw fix.
This is a significant change in behavior from the previous implementation where files were embedded in the binary and always available offline. Consider:
- Fallback handling: When download fails, log a warning but continue gracefully rather than failing the entire operation
- Caching: Cache the downloaded file locally and use it as fallback when network is unavailable
- Clear error messages: If the download must fail, provide actionable guidance (e.g., "Failed to download agent file. Check internet connectivity or try again later.")
Example improvement:
agentContent, err := downloadAgentFileFromGitHub(verbose)
if err != nil {
copilotAgentsLog.Printf("Failed to download agent file from GitHub: %v", err)
// Option 1: Try to use existing file if available
if existingContent != "" {
copilotAgentsLog.Print("Using existing agent file")
return nil // Continue with existing file
}
// Option 2: Make it non-fatal with warning
fmt.Fprintln(os.Stderr, console.FormatWarningMessage(fmt.Sprintf("Could not download agent file: %v. Skipping.", err)))
return nil
}This affects user experience in offline scenarios, corporate networks, and during GitHub outages.
| // If version looks like a release tag (starts with v and contains dots), use it | ||
| isRelease := strings.HasPrefix(currentVersion, "v") && strings.Contains(currentVersion, ".") |
There was a problem hiding this comment.
The version detection logic assumes that release versions always start with v and contain dots (e.g., v1.2.3). However, this could fail for valid version formats such as:
v1(single number version)v1.2.3-rc1(pre-release with suffix)1.2.3(withoutvprefix)
The current logic would incorrectly treat v1 as a dev build (missing dot) and use main branch instead of the v1 tag.
Consider using a more robust version detection approach:
- Check if version starts with
vfollowed by at least one digit - Or use semantic version parsing libraries
- Or document the exact version format requirements
Example improvement:
import "regexp"
// Match semantic version pattern: v1, v1.2, v1.2.3, v1.2.3-beta, etc.
isRelease := regexp.MustCompile(`^v\d`).MatchString(currentVersion)This is a minor edge case that may not affect most users, but it could cause confusion if uncommon versioning schemes are used.
|
Add @copilot, merge main, and recompile. |
The init command was embedding all instruction files from
.github/aw/and.github/agents/into the binary viago:embed, then syncing them during build. This created a dual source of truth and unnecessary binary bloat.Changes
Agent file (downloaded on demand with URL patching)
.github/agents/agentic-workflows.agent.mdfile is no longer embedded in the binarydownloadAgentFileFromGitHub()function that downloads the agent file from GitHub on demandv), uses the release tag to get version-matched filemainbranchhttps://raw.githubusercontent.com/github/gh-aw/{ref}/.github/agents/agentic-workflows.agent.mdpatchAgentFileURLs()function:.github/aw/file.md) to full GitHub URLs/blob/main/to/blob/{version}/for release buildsAgent file references
.github/agents/agentic-workflows.agent.mdto reference prompts via GitHub URLs:Removed all embedded templates
go:embeddirectives frompkg/cli/commands.go(no markdown files are embedded)pkg/cli/templates/directory completely (8 markdown files, ~5000 lines)sync-templatestarget from Makefilebuildandrecompiletargets to not depend on template syncingSimplified prompt file handling
copilot-agents.gofunctions for prompt files to check file existence only (no creation from templates)ensureFileMatchesTemplate()andensureAgentFromTemplate()- unused after template removalAdded cleanup for existing installations
deleteOldTemplateFiles()function to remove all old template files frompkg/cli/templates/directory (8 files)gh aw fix --writecommandtemplates/directory after cleanupResult
/blob/main//blob/v1.2.3/.github/aw/*.mdfiles are NOT embedded, serve as source of truth in gh-aw repo, and are referenced via GitHub URLsgh aw fix --writeremoves all old template files and the templates directoryUsers with existing installations can run
gh aw fix --writeto automatically clean up all old template files that are no longer needed.Example URL patching:
.github/aw/create-agentic-workflow.mdhttps://github.com/github/gh-aw/blob/v1.2.3/.github/aw/create-agentic-workflow.mdhttps://github.com/github/gh-aw/blob/main/.github/aw/create-agentic-workflow.mdOriginal prompt
✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.