Skip to content

Add integration test for adding workflows from githubnext/agentics #16421

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pelikhan merged 4 commits into from
Feb 17, 2026
Merged

Add integration test for adding workflows from githubnext/agentics #16421

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
fe6c9e2
Initial plan
Copilot Feb 17, 2026
c69439c
Add agentics-workflows-integration test to ci.yml
Copilot Feb 17, 2026
e867020
Rename job to integration-add and update display name
Copilot Feb 17, 2026
f47d490
Add GH_TOKEN to integration-add step for authentication
Copilot Feb 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
145 changes: 145 additions & 0 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2073,3 +2073,148 @@ jobs:
name: safe-outputs-conformance-report
path: conformance-output.txt
retention-days: 7

integration-add:
name: Integration Add Workflows
runs-on: ubuntu-latest
permissions:
contents: read
concurrency:
group: ci-${{ github.ref }}-integration-add
cancel-in-progress: true
steps:
- name: Checkout code
uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5

- name: Set up Go
id: setup-go
uses: actions/setup-go@4dc6199c7b1a012772edbd06daecab0f50c9053c # v6
with:
go-version-file: go.mod
cache: true

- name: Report Go cache status
run: |
if [ "${{ steps.setup-go.outputs.cache-hit }}" == "true" ]; then
echo "✅ Go cache hit" >> $GITHUB_STEP_SUMMARY
else
echo "⚠️ Go cache miss" >> $GITHUB_STEP_SUMMARY
fi

- name: Download dependencies
run: go mod download

- name: Verify dependencies
run: go mod verify

- name: Build gh-aw binary
run: make build

- name: Verify gh-aw binary
run: |
./gh-aw --help
./gh-aw version

- name: Clone githubnext/agentics repository
run: |
echo "Cloning githubnext/agentics repository..."
cd /tmp
git clone --depth 1 --filter=blob:none https://github.com/githubnext/agentics.git
echo "✅ Repository cloned successfully"

- name: List workflows from agentics
id: list-workflows
run: |
echo "Listing workflow files from githubnext/agentics..."
cd /tmp/agentics/workflows

# Get list of all .md workflow files (just the names without .md extension)
WORKFLOWS=$(ls *.md | sed 's/\.md$//')

Comment on lines +2132 to +2133
Copy link

Copilot AI Feb 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The workflow list parsing doesn't handle edge cases properly. If the /tmp/agentics/workflows directory is empty or contains no .md files, the ls *.md command will fail with an error, causing the entire step to fail. Consider adding error handling: WORKFLOWS=$(ls *.md 2>/dev/null | sed 's/\.md$//' || echo "") and then check if WORKFLOWS is empty before proceeding. Additionally, add a conditional check after line 2143 to handle the case where no workflows are found.

This issue also appears on line 2138 of the same file.

Suggested change
WORKFLOWS=$(ls *.md | sed 's/\.md$//')
WORKFLOWS=$(ls *.md 2>/dev/null | sed 's/\.md$//' || echo "")
# Handle case where no workflows are found
if [ -z "$WORKFLOWS" ]; then
echo "No workflow files (.md) found in /tmp/agentics/workflows"
echo "workflow_count=0" >> $GITHUB_OUTPUT
exit 0
fi

Copilot uses AI. Check for mistakes.
echo "Found workflows:"
echo "$WORKFLOWS"

# Count workflows
WORKFLOW_COUNT=$(echo "$WORKFLOWS" | wc -l)
echo "Total workflows found: $WORKFLOW_COUNT"

# Save workflow list for next step
echo "$WORKFLOWS" > /tmp/workflow-list.txt
echo "workflow_count=$WORKFLOW_COUNT" >> $GITHUB_OUTPUT

- name: Add workflows one by one
id: add-workflows
env:
GH_TOKEN: ${{ github.token }}
run: |
cd /home/runner/work/gh-aw/gh-aw
Copy link

Copilot AI Feb 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The hardcoded path /home/runner/work/gh-aw/gh-aw is fragile and assumes a specific repository name and GitHub Actions runner environment. This will break if the repository is forked or the runner environment changes. Instead, use $GITHUB_WORKSPACE which is automatically set by GitHub Actions to the correct workspace directory, or use a relative path by staying in the workspace directory from the start.

Suggested change
cd /home/runner/work/gh-aw/gh-aw
cd "$GITHUB_WORKSPACE"

Copilot uses AI. Check for mistakes.

echo "## Adding Workflows from githubnext/agentics" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "| Workflow | Status | Details |" >> $GITHUB_STEP_SUMMARY
echo "|----------|--------|---------|" >> $GITHUB_STEP_SUMMARY

SUCCESS_COUNT=0
FAILURE_COUNT=0

# Read workflow list
while IFS= read -r workflow; do
Copy link

Copilot AI Feb 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The workflow list may contain empty lines if there are any empty entries from the sed command or if the file list has issues. When iterating with while IFS= read -r workflow, empty lines will still be processed, potentially causing gh-aw add to be called with an empty string or just the repository path. Add a check to skip empty lines: add [ -z "$workflow" ] && continue right after the do statement on line 2161.

Suggested change
while IFS= read -r workflow; do
while IFS= read -r workflow; do
[ -z "$workflow" ] && continue

Copilot uses AI. Check for mistakes.
echo "Processing workflow: $workflow"

# Try to add the workflow using gh aw add
if ./gh-aw add "githubnext/agentics/$workflow" --force 2>&1 | tee /tmp/add-${workflow}.log; then
Copy link

Copilot AI Feb 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tee command always returns 0 (success) regardless of the exit code of the piped command before it. This means the if condition will always evaluate to true, and failures won't be caught properly. To fix this, you should set PIPEFAIL before the loop: add set -o pipefail at the beginning of the script block (after the cd command). This ensures that the exit code of the piped command is preserved even when using tee.

Copilot uses AI. Check for mistakes.
echo "✅ Successfully added: $workflow"
echo "| $workflow | ✅ Success | Added successfully |" >> $GITHUB_STEP_SUMMARY
SUCCESS_COUNT=$((SUCCESS_COUNT + 1))
else
EXIT_CODE=$?
echo "❌ Failed to add: $workflow (exit code: $EXIT_CODE)"

# Extract error message from log
ERROR_MSG=$(tail -5 /tmp/add-${workflow}.log | tr '\n' ' ' | cut -c1-100)
Comment on lines +2165 to +2174
Copy link

Copilot AI Feb 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The workflow names are used directly in the log file path /tmp/add-${workflow}.log without sanitization. If a workflow name contains special characters like spaces, slashes, or other shell metacharacters, this could cause issues or security vulnerabilities. While the githubnext/agentics repository likely has well-formed workflow names, it's a best practice to sanitize the workflow name before using it in file paths. Consider using parameter expansion to remove or replace problematic characters: SAFE_NAME=$(echo "$workflow" | tr -c '[:alnum:]-_' '_').

Copilot uses AI. Check for mistakes.
echo "| $workflow | ❌ Failed | Exit code: $EXIT_CODE - ${ERROR_MSG}... |" >> $GITHUB_STEP_SUMMARY
FAILURE_COUNT=$((FAILURE_COUNT + 1))
fi

echo "---"
done < /tmp/workflow-list.txt

echo "" >> $GITHUB_STEP_SUMMARY
echo "### Summary" >> $GITHUB_STEP_SUMMARY
echo "- ✅ Successful: $SUCCESS_COUNT" >> $GITHUB_STEP_SUMMARY
echo "- ❌ Failed: $FAILURE_COUNT" >> $GITHUB_STEP_SUMMARY
echo "- Total: ${{ steps.list-workflows.outputs.workflow_count }}" >> $GITHUB_STEP_SUMMARY

echo "success_count=$SUCCESS_COUNT" >> $GITHUB_OUTPUT
echo "failure_count=$FAILURE_COUNT" >> $GITHUB_OUTPUT

# Report overall result
echo ""
echo "====================================="
echo "Integration Test Results"
echo "====================================="
echo "Successful additions: $SUCCESS_COUNT"
echo "Failed additions: $FAILURE_COUNT"
echo "Total workflows: ${{ steps.list-workflows.outputs.workflow_count }}"
echo "====================================="

- name: Check for added workflows
run: |
echo "Checking for added workflow files..."
if [ -d ".github/workflows" ]; then
echo "Found workflows directory"
ls -la .github/workflows/*.md 2>/dev/null | head -20 || echo "No .md files found"
else
echo "No .github/workflows directory found"
fi

- name: Test result summary
if: always()
run: |
echo "=== Agentics Workflows Integration Test Summary ==="
echo "This test validates that gh-aw can successfully add workflows"
echo "from the githubnext/agentics repository."
echo ""
echo "Test completed with:"
echo "- Success count: ${{ steps.add-workflows.outputs.success_count }}"
echo "- Failure count: ${{ steps.add-workflows.outputs.failure_count }}"
Loading