🔍 Claude Code User Documentation Review - 2026-01-30 #12749
Closed
Replies: 1 comment
-
|
This discussion was automatically closed because it expired on 2026-02-06T13:58:12.375Z. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Executive Summary
As a Claude Code user reviewing the gh-aw documentation, I found that Claude users CAN successfully adopt gh-aw, but the documentation presents a Copilot-first experience that creates unnecessary friction and confusion for non-Copilot users. The good news: the technical foundation is solid and Claude is well-supported. The challenge: documentation assumptions and language make it harder than necessary for Claude users to get started.
Key Finding: Claude is a first-class supported engine with good technical documentation (engines.md), but the onboarding experience (README, Quick Start) assumes Copilot as the default, creating mental overhead for Claude users who must constantly translate "Copilot-first" instructions into "what does this mean for me?"
Persona Context
I reviewed this documentation as a developer who:
Question 1: Onboarding Experience
Can a Claude Code user understand and get started with gh-aw?
Answer: YES, but with significant mental overhead
The onboarding experience is functional but requires Claude users to mentally filter "Copilot-centric" language throughout. The Quick Start guide does mention Claude as an alternative, but the overall narrative positions Copilot as the default and Claude as "the other option."
Specific Issues Found:
Issue 1: README positions Copilot as the primary engine
README.md:25gh awcli converts this into a GitHub Actions Workflow (.yml) that runs an AI agent (Copilot, Claude, Codex, ...) in a containerized environment"Issue 2: Quick Start Prerequisites list Copilot first
docs/src/content/docs/setup/quick-start.md:16Issue 3: "How It Works" only shows Copilot usage
docs/src/content/docs/introduction/how-they-work.mdx:16Issue 4: Overview uses "Copilot" as the example engine
docs/src/content/docs/introduction/overview.mdx:13Recommended Fixes:
gh aw addinteractive mode handle engine selectionQuestion 2: Inaccessible Features for Non-Copilot Users
What features or steps don't work without Copilot?
Answer: NO critical blockers - Claude and Copilot have feature parity at the workflow level
Great news! After thorough analysis, Claude users have access to the same core functionality as Copilot users. The
engine:field makes the choice explicit, and all tools (GitHub MCP, safe-outputs, web-fetch, etc.) are engine-agnostic.Features That Require Copilot:
/agent agentic-workflowswhich requires Copilot integration with GitHub.com, but this is a workflow creation helper, not a core featureFeatures That Work Without Copilot:
Features That Are Copilot-Specific (by design):
docs/src/content/docs/setup/agentic-authoring.mdxMissing Documentation:
Verdict: The technical implementation is excellent - engines are truly pluggable. The documentation problem is clarity, not functionality.
Question 3: Documentation Gaps and Assumptions
Where does the documentation assume Copilot usage?
Analysis: The documentation has structural Copilot-centricity rather than explicit Copilot requirements. It's written from a "we built this with Copilot, oh and it works with others too" perspective.
Copilot-Centric Language Found In:
README.mdREADME.mddocs/src/content/docs/introduction/overview.mdx:13docs/src/content/docs/introduction/how-they-work.mdx:16docs/src/content/docs/setup/quick-start.mddocs/src/content/docs/setup/quick-start.md:46docs/src/content/docs/setup/agentic-authoring.mdxdocs/src/content/docs/reference/engines.mdMissing Alternative Instructions:
Severity-Categorized Findings
🚫 Critical Blockers (Score: 0/10 - None Found!)
Excellent news: There are NO critical blockers preventing Claude Code users from adopting gh-aw.
The technical architecture is sound, engines are truly pluggable, and all core functionality works with Claude. The issues are documentation clarity rather than technical limitations.
Obstacle 1: Copilot-First Onboarding Creates Mental Overhead
Impact: Significant friction in getting started - Claude users must constantly translate documentation
Current State:
Why It's Problematic:
Every mention of Copilot-first creates micro-friction where Claude users ask:
Cognitive load accumulates across dozens of small instances.
Suggested Fix:
Affected Files:
README.md,docs/src/content/docs/setup/quick-start.md,docs/src/content/docs/introduction/how-they-work.mdx,docs/src/content/docs/introduction/overview.mdxObstacle 2: "Agentic Authoring" Guide is Copilot-Only Without Disclaimer
Impact: Significant friction - Claude users waste time reading guide that doesn't apply
Current State:
docs/src/content/docs/setup/agentic-authoring.mdxdescribes using GitHub Copilot agent/agentcommandsWhy It's Problematic:
Claude users following the setup documentation path will read this guide thinking it applies to them. Only after reading instructions about "Agents tab" and
/agent agentic-workflowscommands do they realize it requires Copilot integration with GitHub.com.Suggested Fix:
Affected Files:
docs/src/content/docs/setup/agentic-authoring.mdxObstacle 3: No Dedicated "Claude Quick Start" Path
Impact: Moderate friction - Claude users must piece together instructions from multiple sources
Current State:
docs/src/content/docs/reference/engines.md(reference section, not setup)Why It's Problematic:
Claude users must navigate:
engine: claudein workflowSuggested Fix:
Affected Files:
docs/src/content/docs/setup/quick-start.md,docs/src/content/docs/reference/engines.mdObstacle 4: Engine Capabilities Not Explicitly Documented
Impact: Moderate friction - Claude users uncertain if features work with their engine
Current State:
Why It's Problematic:
Claude users encounter features and wonder:
Having to guess or test reduces confidence in adoption.
Suggested Fix:
Add "Engine Compatibility" section to reference/engines.md with matrix:
Add "All engines support the same core features" statement prominently
Document engine-specific differences (like web-search availability)
Affected Files:
docs/src/content/docs/reference/engines.md,docs/src/content/docs/reference/tools.md💡 Minor Confusion Points (Score: 4/10)
Issue 1: Architecture diagram (architecture.mdx:168) shows "Copilot CLI" as the agent example without mentioning this could be Claude or Codex - File:
docs/src/content/docs/introduction/architecture.mdx:168Issue 2: CLI reference (cli.md) uses Copilot in examples without showing Claude alternatives - File:
docs/src/content/docs/setup/cli.mdIssue 3: FAQ (faq.md) mentions "coding agents like Copilot" without listing other options in same sentence - File:
docs/src/content/docs/reference/faq.mdIssue 4: No "I'm coming from [X AI tool]" migration guides - missing comparison with Claude Code, Cursor, other AI assistants
Issue 5: Workflow examples in repo (144 total: 73 Copilot, 29 Claude, 9 Codex) aren't surfaced in docs - Claude users don't know examples exist
Issue 6: "Quick Example with Claude" in engines.md is buried in reference docs instead of being in Quick Start guide - File:
docs/src/content/docs/reference/engines.md:78-119Engine Comparison Analysis
Available Engines
Based on my review, gh-aw supports these engines:
Documentation Quality by Engine
Key Findings:
Tool Availability Analysis
Tools Review
Analyzed tool compatibility across engines by reviewing tools.md and testing against workflow examples:
Engine-Agnostic Tools:
Engine-Specific Notes:
Verdict: 99.9% feature parity across engines. The tools.md documentation correctly avoids mentioning specific engines, maintaining an engine-agnostic stance.
Authentication Requirements
Current Documentation
Quick Start guide covers authentication for:
All three engines have adequate authentication documentation in the engines reference.
Missing for Claude Users
While the authentication instructions exist, they're not where Claude users expect them:
/reference/engines.mdinstead of/setup/quick-start.mdSecret Names
Documentation clearly specifies:
COPILOT_GITHUB_TOKEN(documented in engines.md:40)ANTHROPIC_API_KEY(documented in engines.md:76)OPENAI_API_KEY(documented in engines.md:143)All clearly documented with gh aw secrets commands.
Recommendation
Move authentication setup into Quick Start as tabbed or expandable sections:
Quality of Examples
Copilot Examples:
Claude Examples:
smoke-claude.mdis a comprehensive test workflowKey Finding: The repository HAS extensive Claude examples proving feature parity, but the documentation doesn't leverage them to build confidence for Claude users.
Recommendation
Recommended Actions
Priority 1: Critical Documentation Fixes
Rewrite README and Quick Start to be engine-neutral
README.md,docs/src/content/docs/setup/quick-start.mdAdd "Choose Your Engine" section to Quick Start with tabbed setup instructions
docs/src/content/docs/setup/quick-start.mdAdd disclaimer to Agentic Authoring guide that it's Copilot-specific
docs/src/content/docs/setup/agentic-authoring.mdxCreate "Engine Feature Compatibility Matrix"
docs/src/content/docs/reference/engines.mdPriority 2: Major Improvements
Surface Claude examples in documentation
docs/src/content/docs/setup/quick-start.md,docs/src/content/docs/reference/engines.mdAdd "Quick Example with Claude" to Quick Start
docs/src/content/docs/setup/quick-start.mdUpdate conceptual documentation to use engine-neutral language
docs/src/content/docs/introduction/overview.mdx,docs/src/content/docs/introduction/how-they-work.mdxAdd "Authoring Workflows with Claude Code" guide
docs/src/content/docs/setup/authoring-with-claude.mdPriority 3: Nice-to-Have Enhancements
Add "Coming from [AI Tool]" migration guides
Create "Examples Gallery" page organized by engine
Add engine selection guidance
Positive Findings
What Works Well
The gh-aw team has done excellent work on the technical implementation:
engine: claudejust worksgh aw secrets set ANTHROPIC_API_KEYis well documentedgh aw add --engine claudeandgh aw secrets bootstrap --engine claudeexistgh aw initprompts for engine selection, doesn't assume CopilotKey Strength: The engineering team built a genuinely engine-agnostic system. The documentation just needs to catch up to reflect this reality.
Conclusion
Can Claude Code Users Successfully Adopt gh-aw?
Answer: YES - with moderate effort to filter Copilot-centric documentation
Reasoning:
The technical foundation is solid. Claude is a first-class supported engine with complete feature parity with Copilot (except web-search which Copilot lacks and Claude has!). All tools work, authentication is straightforward, and 29 working example workflows prove everything functions correctly.
The challenge is documentation positioning, not technical capability. Claude users face:
A Claude user who perseveres through the Copilot-centric onboarding will successfully deploy workflows. But many will bounce due to accumulated friction from dozens of "Is this for me?" moments.
Overall Assessment Score: 7/10
Breakdown:
Adjusted Score: 7/10 - The technical implementation is 10/10, but documentation reduces the experience to 7/10 for Claude users.
Next Steps
Immediate Actions (High Impact, Low Effort):
Short-term Actions (High Impact, Medium Effort):
Long-term Actions (Medium Impact, High Effort):
Appendix: Files Reviewed
Complete List of Documentation Files Analyzed
Core Documentation:
README.mddocs/src/content/docs/setup/quick-start.mddocs/src/content/docs/introduction/how-they-work.mdxdocs/src/content/docs/introduction/architecture.mdxdocs/src/content/docs/introduction/overview.mdxdocs/src/content/docs/reference/tools.mddocs/src/content/docs/reference/engines.mddocs/src/content/docs/setup/cli.mddocs/src/content/docs/reference/faq.mddocs/src/content/docs/setup/agentic-authoring.mdxExample Workflows Sampled:
.github/workflows/smoke-claude.mdAdditional Context:
Report Generated: 2026-01-30
Workflow Run: 21518070735
Engine Used: claude (eating our own dog food! 🐕)
Verdict: Claude users CAN adopt gh-aw successfully - documentation improvements will remove friction and accelerate adoption.
Beta Was this translation helpful? Give feedback.
All reactions