Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
111 changes: 111 additions & 0 deletions docs/src/content/docs/guides/web-search.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
---
title: Web Search with MCP
description: Learn how to add web search capabilities to GitHub Agentic Workflows using Tavily MCP server.
---

This guide covers how to add web search capabilities to workflows using the Tavily MCP server.

## Overview

Some AI engines (like Copilot) don't include built-in web search functionality. To add web search capabilities to these workflows, you can integrate third-party MCP servers that provide search functionality.

This guide focuses on Tavily, an AI-optimized search provider designed for LLM applications. Other alternatives include Exa (semantic search), SerpAPI (Google search access), and Brave Search (privacy-focused), though this guide only covers Tavily setup.

## Tavily Search

[Tavily](https://tavily.com/) provides AI-optimized search designed for LLM applications with structured results.

**MCP Server:** [@tavily/mcp-server](https://github.com/tavily-ai/tavily-mcp-server)

```aw
---
on: issues
engine: copilot
mcp-servers:
tavily:
command: npx
args: ["-y", "@tavily/mcp-server"]
env:
TAVILY_API_KEY: "${{ secrets.TAVILY_API_KEY }}"
allowed: ["search", "search_news"]
---

# Search and Respond

Search the web for information about: ${{ github.event.issue.title }}

Use the tavily search tool to find recent information.
```

**Features:**
- AI-optimized search results
- News search capability
- Structured JSON responses
- Fast response times

**Setup:**
1. Sign up at [tavily.com](https://tavily.com/)
2. Get your API key from the dashboard
3. Add as repository secret: `gh secret set TAVILY_API_KEY -a actions --body "<your-api-key>"`

**Terms of Service:** [Tavily Terms](https://tavily.com/terms)

## MCP Server Configuration

Tavily MCP server follows this basic pattern:

```yaml
mcp-servers:
tavily:
command: npx # Use npx for npm packages
args: ["-y", "@tavily/mcp-server"] # -y to auto-install
env:
TAVILY_API_KEY: "${{ secrets.TAVILY_API_KEY }}"
allowed: ["search", "search_news"] # Specific tools to allow
```

**Best Practices:**
1. Always use the `allowed` list to restrict which tools can be used
2. Store API keys in GitHub Secrets, never commit them
3. Use `-y` flag with npx to ensure automatic installation
4. Test MCP configuration with `gh aw mcp inspect <workflow-name>`

## Tool Discovery

To see available tools from the Tavily MCP server:

```bash
# Inspect the MCP server in your workflow
gh aw mcp inspect my-workflow --server tavily

# List tools with details
gh aw mcp list-tools tavily my-workflow --verbose
```

## Network Permissions

Some engines (like Claude) require explicit network permissions for MCP servers to access external APIs:

```yaml
engine: claude
network:
allowed:
- defaults # Basic infrastructure
- "*.tavily.com" # Tavily API
```

The Copilot engine doesn't require explicit network permissions as MCP servers run with network access by default.

## Related Documentation

- [MCP Integration](/gh-aw/guides/mcps/) - Complete MCP server guide
- [Tools Configuration](/gh-aw/reference/tools/) - Tool configuration reference
- [AI Engines](/gh-aw/reference/engines/) - Engine capabilities and limitations
- [CLI Commands](/gh-aw/tools/cli/) - CLI commands including `mcp inspect`

## External Resources

- [Model Context Protocol Specification](https://github.com/modelcontextprotocol/specification)
- [Tavily MCP Server](https://github.com/tavily-ai/tavily-mcp-server)
- [Tavily Documentation](https://tavily.com/)

4 changes: 3 additions & 1 deletion docs/src/content/docs/reference/engines.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,9 @@ engine:
- **`model`** (optional): AI model to use (`gpt-5` or defaults to `claude-sonnet-4`)
- **`version`** (optional): Version of the GitHub Copilot CLI to install (defaults to `latest`)

**Note:** The Copilot engine does not support the `web-search` tool. Use Claude or Codex engines if you need web search capabilities.
:::note
The Copilot engine does not have built-in `web-search` support. You can add web search capabilities using third-party MCP servers. See the [Web Search with MCP guide](/gh-aw/guides/web-search/) for available options and setup instructions.
:::

**Environment Variables:**
- **`COPILOT_MODEL`**: Alternative way to set the model (e.g., `gpt-5`)
Expand Down
4 changes: 4 additions & 0 deletions docs/src/content/docs/reference/tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,10 @@ tools:
# bash: ["echo", "ls", "git status"] # Or specify custom commands
```

:::note
Some engines (like Copilot) don't have built-in `web-search` support. You can add web search using third-party MCP servers instead. See the [Web Search with MCP guide](/gh-aw/guides/web-search/) for options.
:::

### Bash Command Configuration

The bash tool provides access to shell commands with different levels of control and security.
Expand Down
2 changes: 1 addition & 1 deletion pkg/workflow/compiler.go
Original file line number Diff line number Diff line change
Expand Up @@ -2881,7 +2881,7 @@ func (c *Compiler) validateWebSearchSupport(tools map[string]any, engine CodingA

// web-search is specified, check if the engine supports it
if !engine.SupportsWebSearch() {
fmt.Fprintln(os.Stderr, console.FormatWarningMessage(fmt.Sprintf("Engine '%s' does not support the web-search tool", engine.GetID())))
fmt.Fprintln(os.Stderr, console.FormatWarningMessage(fmt.Sprintf("Engine '%s' does not support the web-search tool. See https://githubnext.github.io/gh-aw/guides/web-search/ for alternatives.", engine.GetID())))
}
}

Expand Down
28 changes: 27 additions & 1 deletion pkg/workflow/search_integration_test.go
Original file line number Diff line number Diff line change
@@ -1,14 +1,16 @@
package workflow

import (
"bytes"
"io"
"os"
"path/filepath"
"strings"
"testing"
)

// TestWebSearchValidationForCopilot tests that when a Copilot workflow uses web-search,
// compilation succeeds but emits a warning
// compilation succeeds but emits a warning with documentation link
func TestWebSearchValidationForCopilot(t *testing.T) {
// Create a temporary directory for the test
tmpDir := t.TempDir()
Expand All @@ -33,15 +35,39 @@ Search the web for information.
t.Fatalf("Failed to write test workflow: %v", err)
}

// Capture stderr to verify warning message
oldStderr := os.Stderr
r, w, _ := os.Pipe()
os.Stderr = w

// Create a compiler
compiler := NewCompiler(false, "", "test")

// Compile the workflow - should succeed with a warning
err := compiler.CompileWorkflow(workflowPath)

// Restore stderr
w.Close()
os.Stderr = oldStderr

// Read captured stderr
var buf bytes.Buffer
io.Copy(&buf, r)
stderrOutput := buf.String()

if err != nil {
t.Fatalf("Expected compilation to succeed for Copilot engine with web-search tool (with warning), but got error: %v", err)
}

// Verify the warning message includes the documentation link
if !strings.Contains(stderrOutput, "does not support the web-search tool") {
t.Errorf("Expected warning about web-search not being supported, but got: %s", stderrOutput)
}

if !strings.Contains(stderrOutput, "https://githubnext.github.io/gh-aw/guides/web-search/") {
t.Errorf("Expected warning to include documentation link, but got: %s", stderrOutput)
}

// Verify the lock file was created
lockFile := strings.TrimSuffix(workflowPath, ".md") + ".lock.yml"
if _, err := os.Stat(lockFile); os.IsNotExist(err) {
Expand Down
Loading