PullGenius - AI Code Review Bot for Azure DevOps

PullGenius is an automated AI-powered code review bot that integrates with Azure DevOps pull requests. It uses Azure OpenAI to analyze code changes and provide structured, file-specific reviews directly in your pull requests.

Features

• 🤖 AI-Powered Reviews: Uses Azure OpenAI (GPT-4) to analyze code changes
• 📁 File-Based Comments: Posts reviews as file-specific comments in Azure DevOps
• ⚡ Parallel Processing: Reviews multiple files simultaneously for optimal performance
• 🎯 Smart Change Detection: Reviews only modified lines, not entire files
• 🔍 Method-Level Analysis: For C# files, uses Roslyn to extract and analyze changed methods
• 📊 Configurable Limits: Skip excessively large files to avoid timeouts
• 🌐 Multi-Language Support: Supports C#, JavaScript, TypeScript, Python, and many more

Architecture

Core Flow

1. Azure DevOps webhook triggers on PR created/updated events
2. Service processes each modified file in parallel
3. For each file:
   • Fetches diff from Azure DevOps
   • Analyzes with Azure OpenAI
   • Posts file-based review comment

Review Categories

• 🔒 Security: Vulnerabilities, data exposure, authentication flaws
• ⚡ Performance: Bottlenecks, memory leaks, inefficient algorithms
• 🔧 Quality: Maintainability, readability, design patterns
• 🐛 Logic & Bugs: Business logic errors, edge cases, potential bugs

Getting Started

Prerequisites

• .NET 8.0 SDK
• Azure DevOps Server (on-premise or cloud)
• Azure OpenAI Service with GPT-4 deployment
• Azure DevOps Personal Access Token (PAT) with Code Read & Write permissions

Installation

1. Clone the repository

git clone https://github.com/your-username/PullGenius.git
cd PullGenius

2. Configure settings

cd PullGenius
cp appsettings.Example.json appsettings.json

Edit appsettings.json and fill in your configuration:

{
  "AzureDevOps": {
    "BaseUrl": "https://dev.azure.com or your on-premise URL",
    "Organization": "YourOrganization",
    "Project": "YourProject",
    "PersonalAccessToken": "your-pat-token"
  },
  "OpenAI": {
    "Endpoint": "https://your-resource.openai.azure.com/",
    "ApiKey": "your-api-key",
    "Model": "gpt-4",
    "MaxTokens": 2000,
    "Temperature": 0.3,
    "MaxDiffLength": 8000
  }
}

3. Build and run

dotnet restore
dotnet build
dotnet run

The service will start on http://localhost:5093 and https://localhost:7050

Configure Azure DevOps Webhook

1. Go to your Azure DevOps project
2. Navigate to Project Settings → Service Hooks
3. Click + Create subscription
4. Select Web Hooks
5. Configure:
   • Event: Pull request created/updated
   • URL: http://your-server:5093/api/webhook/pullrequest
   • Method: POST
6. Click Finish

Configuration Options

AzureDevOps Section

• BaseUrl: Your Azure DevOps server URL
• Organization: Organization/Collection name
• Project: Project name
• PersonalAccessToken: PAT with Code Read/Write permissions

OpenAI Section

• Endpoint: Azure OpenAI resource endpoint
• ApiKey: Azure OpenAI API key
• Model: Model deployment name (default: gpt-4)
• MaxTokens: Maximum tokens per response (default: 2000)
• Temperature: Creativity level 0-1 (default: 0.3)
• MaxDiffLength: Max characters in diff before skipping file (default: 8000)

API Endpoints

• POST /api/webhook/pullrequest - Webhook endpoint for Azure DevOps
• GET /api/webhook/health - Health check endpoint

Development

Project Structure

PullGenius/
├── Controllers/         # API controllers
├── Services/           # Business logic
│   ├── PullRequestReviewService.cs
│   ├── AzureDevOpsService.cs
│   └── OpenAIService.cs
├── Models/             # Data models
└── appsettings.json    # Configuration

Key Dependencies

• Azure.AI.OpenAI (2.1.0) - Azure OpenAI client
• Microsoft.TeamFoundationServer.Client (19.225.1) - Azure DevOps API
• DiffPlex (1.9.0) - Text diffing
• Microsoft.CodeAnalysis.CSharp (4.14.0) - Roslyn for C# parsing

Security Notes

⚠️ Important: Never commit your appsettings.json with real credentials to source control!

• The .gitignore file is configured to exclude appsettings.json
• Use appsettings.Example.json as a template
• Store sensitive credentials in:
  • Azure Key Vault
  • Environment variables
  • User secrets (for development)

Troubleshooting

Webhook not triggering

• Verify webhook URL is accessible from Azure DevOps
• Check Azure DevOps Service Hooks history for errors
• Ensure firewall allows incoming connections

Reviews not appearing

• Check application logs for errors
• Verify PAT token has Code (Read & Write) permissions
• Confirm Azure OpenAI endpoint and API key are correct

Files being skipped

• Check logs for "diff too long" messages
• Increase MaxDiffLength if needed
• Consider splitting large changes into smaller PRs

Contributing

Contributions are welcome! Please feel free to submit issues or pull requests.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

• Uses Azure OpenAI for intelligent code analysis
• Built with ASP.NET Core 8.0
• Integrates with Azure DevOps REST API