feat: integrate pr-test-oracle for AI-powered test recommendations

CLAUDE.md

@@ -654,3 +654,21 @@ def mock_github_api():
 2. Run `uv run pytest webhook_server/tests/test_config_schema.py -v`
 3. Update examples in `examples/config.yaml`
 4. Test with `uv run webhook_server/tests/test_schema_validator.py examples/config.yaml`
+
+### PR Test Oracle Integration
+
+External AI service integration for test recommendations via [pr-test-oracle](https://github.com/myk-org/pr-test-oracle). Configured via `test-oracle` in config (global or per-repo).
+
+**Config keys:** `server-url` (required), `ai-provider` (required: claude/gemini/cursor), `ai-model` (required), `test-patterns` (optional), `triggers` (optional, default: [approved])
+
+**Trigger events:** `approved`, `pr-opened`, `pr-synchronized`
+
+**Comment command:** `/test-oracle` (always works when configured, no trigger needed)
+
+**Module:** `webhook_server/libs/test_oracle.py` - `call_test_oracle()` shared helper
+
+**Error handling:**
+
+- Health check failure: PR comment posted, continue flow
+- Analyze errors: log only, no PR comment
+- Never breaks webhook processing

README.md

@@ -85,6 +85,7 @@ GitHub Events → Webhook Server → Repository Management
 - **PyPI package publishing** for Python projects
 - **Tox testing integration** with configurable test environments
 - **Pre-commit hook validation** for code quality assurance
+- **PR Test Oracle** - AI-powered test recommendations based on PR diff analysis
 
 ### 👥 User Commands
 
@@ -584,6 +585,7 @@ uv run pytest webhook_server/tests/test_config_schema.py::TestConfigSchema::test
 | **Security** | `webhook-secret`, `verify-github-ips`, `verify-cloudflare-ips`, `disable-ssl-warnings`                              |
 | **GitHub**   | `github-app-id`, `github-tokens`, `webhook-ip`                                                                      |
 | **Defaults** | `docker`, `default-status-checks`, `auto-verified-and-merged-users`, `branch-protection`, `create-issue-for-new-pr` |
+| **AI**       | [`test-oracle`](https://github.com/myk-org/pr-test-oracle)                                                          |
 
 #### Repository Level Options
 
@@ -593,6 +595,7 @@ uv run pytest webhook_server/tests/test_config_schema.py::TestConfigSchema::test
 | **Features**      | `verified-job`, `pre-commit`, `pypi`, `tox`, `container`                                         |
 | **Pull Requests** | `minimum-lgtm`, `conventional-title`, `can-be-merged-required-labels`, `create-issue-for-new-pr` |
 | **Automation**    | `set-auto-merge-prs`, `auto-verified-and-merged-users`                                           |
+| **AI**            | [`test-oracle`](https://github.com/myk-org/pr-test-oracle) (`server-url`, `ai-provider`, `ai-model`, `test-patterns`, `triggers`) |
 | **Protection**    | `protected-branches`, `branch-protection`                                                        |
 
 ## Deployment
@@ -1237,6 +1240,7 @@ Users can interact with the webhook server through GitHub comments on pull reque
 | `/assign-reviewers` | Assign OWNERS-based reviewers                           | `/assign-reviewers` |
 | `/check-can-merge`  | Check merge readiness                                   | `/check-can-merge`  |
 | `/reprocess`        | Trigger complete PR workflow reprocessing (OWNERS only) | `/reprocess`        |
+| `/test-oracle`      | Request AI-powered test recommendations for PR changes ([pr-test-oracle](https://github.com/myk-org/pr-test-oracle)) | `/test-oracle`      |
 
 ### Workflow Management
 
@@ -1322,6 +1326,7 @@ auto-verify-cherry-picked-prs: true # Default: true (auto-verify). Set to false
 | Command               | Description                                   | Example               |
 | --------------------- | --------------------------------------------- | --------------------- |
 | `/retest <test-name>` | Run specific tests like `tox` or `pre-commit` | `/retest <test-name>` |
+| `/test-oracle`        | Request AI-powered test recommendations ([pr-test-oracle](https://github.com/myk-org/pr-test-oracle)) | `/test-oracle`        |
 
 ## OWNERS File Format
 

examples/.github-webhook-server.yaml

@@ -143,3 +143,15 @@ pr-size-thresholds:
   Extreme:
     threshold: inf     # PRs with 1000+ lines changed (unbounded largest category)
     color: black       # 'inf' means no upper limit - catches all PRs above 1000 lines
+
+# PR Test Oracle integration (overrides global config)
+# Analyzes PR diffs with AI and recommends which tests to run
+# See: https://github.com/myk-org/pr-test-oracle
+test-oracle:
+  server-url: "http://localhost:8000"
+  ai-provider: "claude"        # claude | gemini | cursor
+  ai-model: "sonnet"
+  test-patterns:
+    - "tests/**/*.py"
+  triggers:
+    - approved

examples/config.yaml

@@ -107,6 +107,20 @@ branch-protection:
   required_linear_history: True
   required_conversation_resolution: True
 
+# PR Test Oracle integration
+# Analyzes PR diffs with AI and recommends which tests to run
+# See: https://github.com/myk-org/pr-test-oracle
+test-oracle:
+  server-url: "http://localhost:8000"
+  ai-provider: "claude"        # claude | gemini | cursor
+  ai-model: "sonnet"
+  test-patterns:
+    - "tests/**/*.py"
+  triggers:                    # Default: [approved]
+    - approved                 # Run when PR gets approved
+    # - pr-opened             # Run when PR is opened
+    # - pr-synchronized       # Run when new commits pushed
+
 repositories:
   my-repository:
     name: my-org/my-repository
@@ -222,3 +236,14 @@ repositories:
 
     set-auto-merge-prs:
       - main
+
+    # PR Test Oracle (overrides global)
+    # See: https://github.com/myk-org/pr-test-oracle
+    test-oracle:
+      server-url: "http://localhost:8000"
+      ai-provider: "claude"
+      ai-model: "sonnet"
+      test-patterns:
+        - "tests/**/*.py"
+      triggers:
+        - approved

webhook_server/config/schema.yaml

@@ -92,6 +92,53 @@ properties:
       - Not set (default): commands blocked on draft PRs
       - Empty list []: all commands allowed on draft PRs
       - List with commands: only specified commands allowed (e.g., ["build-and-push-container", "retest"])
+  test-oracle:
+    type: object
+    description: |
+      PR Test Oracle integration. Sends PR data to an external AI service
+      that analyzes diffs and recommends which tests to run.
+      See https://github.com/myk-org/pr-test-oracle for server setup.
+      The /test-oracle comment command always works when configured (no trigger needed).
+    properties:
+      server-url:
+        type: string
+        format: uri
+        description: URL of the pr-test-oracle server (e.g., http://localhost:8000)
+      ai-provider:
+        type: string
+        enum:
+          - claude
+          - gemini
+          - cursor
+        description: AI provider for test analysis
+      ai-model:
+        type: string
+        description: AI model identifier (e.g., sonnet, gemini-2.5-pro)
+      test-patterns:
+        type: array
+        items:
+          type: string
+        description: Glob patterns for test files (e.g., ["tests/**/*.py"]). Optional - oracle has defaults.
+      triggers:
+        type: array
+        default:
+          - approved
+        items:
+          type: string
+          enum:
+            - approved
+            - pr-opened
+            - pr-synchronized
+        description: |
+          When to automatically run test oracle analysis. Default: [approved].
+          - approved: Run when a PR review is approved
+          - pr-opened: Run when a new PR is opened
+          - pr-synchronized: Run when new commits are pushed to a PR
+    required:
+      - server-url
+      - ai-provider
+      - ai-model
+    additionalProperties: false
   labels:
     type: object
     description: Configure which labels are enabled and their colors
@@ -402,6 +449,53 @@ properties:
             required:
               - threshold
             additionalProperties: false
+        test-oracle:
+          type: object
+          description: |
+            PR Test Oracle integration. Sends PR data to an external AI service
+            that analyzes diffs and recommends which tests to run.
+            See https://github.com/myk-org/pr-test-oracle for server setup.
+            The /test-oracle comment command always works when configured (no trigger needed).
+          properties:
+            server-url:
+              type: string
+              format: uri
+              description: URL of the pr-test-oracle server (e.g., http://localhost:8000)
+            ai-provider:
+              type: string
+              enum:
+                - claude
+                - gemini
+                - cursor
+              description: AI provider for test analysis
+            ai-model:
+              type: string
+              description: AI model identifier (e.g., sonnet, gemini-2.5-pro)
+            test-patterns:
+              type: array
+              items:
+                type: string
+              description: Glob patterns for test files (e.g., ["tests/**/*.py"]). Optional - oracle has defaults.
+            triggers:
+              type: array
+              default:
+                - approved
+              items:
+                type: string
+                enum:
+                  - approved
+                  - pr-opened
+                  - pr-synchronized
+              description: |
+                When to automatically run test oracle analysis. Default: [approved].
+                - approved: Run when a PR review is approved
+                - pr-opened: Run when a new PR is opened
+                - pr-synchronized: Run when new commits are pushed to a PR
+          required:
+            - server-url
+            - ai-provider
+            - ai-model
+          additionalProperties: false
         custom-check-runs:
           type: array
           description: |

webhook_server/libs/handlers/issue_comment_handler.py

@@ -14,7 +14,9 @@
 from webhook_server.libs.handlers.owners_files_handler import OwnersFileHandler
 from webhook_server.libs.handlers.pull_request_handler import PullRequestHandler
 from webhook_server.libs.handlers.runner_handler import RunnerHandler
+from webhook_server.libs.test_oracle import call_test_oracle
 from webhook_server.utils.constants import (
+    APPROVE_STR,
     AUTOMERGE_LABEL_STR,
     BUILD_AND_PUSH_CONTAINER_STR,
     CHERRY_PICK_LABEL_PREFIX,
@@ -26,6 +28,7 @@
     COMMAND_REGENERATE_WELCOME_STR,
     COMMAND_REPROCESS_STR,
     COMMAND_RETEST_STR,
+    COMMAND_TEST_ORACLE_STR,
     HOLD_LABEL_STR,
     REACTIONS,
     USER_LABELS_DICT,
@@ -37,6 +40,8 @@
     from webhook_server.libs.github_api import GithubWebhook
     from webhook_server.utils.context import WebhookContext
 
+_background_tasks: set[asyncio.Task[None]] = set()
+
 
 class IssueCommentHandler:
     def __init__(self, github_webhook: GithubWebhook, owners_file_handler: OwnersFileHandler):
@@ -159,14 +164,15 @@ async def user_commands(
             COMMAND_ASSIGN_REVIEWER_STR,
             COMMAND_ADD_ALLOWED_USER_STR,
             COMMAND_REGENERATE_WELCOME_STR,
+            COMMAND_TEST_ORACLE_STR,
         ]
 
         command_and_args: list[str] = command.split(" ", 1)
         _command = command_and_args[0]
         _args: str = command_and_args[1] if len(command_and_args) > 1 else ""
 
         # Check if command is allowed on draft PRs
-        if is_draft:
+        if is_draft and _command != COMMAND_TEST_ORACLE_STR:
             allow_commands_on_draft = self.github_webhook.config.get_value("allow-commands-on-draft-prs")
             if not isinstance(allow_commands_on_draft, list):
                 self.logger.debug(
@@ -272,6 +278,16 @@ async def user_commands(
             self.logger.info(f"{self.log_prefix} Regenerating welcome message")
             await self.pull_request_handler.regenerate_welcome_message(pull_request=pull_request)
 
+        elif _command == COMMAND_TEST_ORACLE_STR:
+            task = asyncio.create_task(
+                call_test_oracle(
+                    github_webhook=self.github_webhook,
+                    pull_request=pull_request,
+                )
+            )
+            _background_tasks.add(task)
+            task.add_done_callback(_background_tasks.discard)
+
         elif _command == BUILD_AND_PUSH_CONTAINER_STR:
             if self.github_webhook.build_and_push_container:
                 await self.runner_handler.run_build_container(
@@ -339,6 +355,16 @@ async def user_commands(
                 remove=remove,
                 reviewed_user=reviewed_user,
             )
+            if _command == APPROVE_STR:
+                task = asyncio.create_task(
+                    call_test_oracle(
+                        github_webhook=self.github_webhook,
+                        pull_request=pull_request,
+                        trigger="approved",
+                    )
+                )
+                _background_tasks.add(task)
+                task.add_done_callback(_background_tasks.discard)
 
     async def create_comment_reaction(self, pull_request: PullRequest, issue_comment_id: int, reaction: str) -> None:
         _comment = await asyncio.to_thread(pull_request.get_issue_comment, issue_comment_id)

webhook_server/libs/handlers/pull_request_handler.py

@@ -13,6 +13,7 @@
 from webhook_server.libs.handlers.labels_handler import LabelsHandler
 from webhook_server.libs.handlers.owners_files_handler import OwnersFileHandler
 from webhook_server.libs.handlers.runner_handler import RunnerHandler
+from webhook_server.libs.test_oracle import call_test_oracle
 from webhook_server.utils.constants import (
     APPROVED_BY_LABEL_PREFIX,
     AUTOMERGE_LABEL_STR,
@@ -42,6 +43,8 @@
     from webhook_server.libs.github_api import GithubWebhook
     from webhook_server.utils.context import WebhookContext
 
+_background_tasks: set[asyncio.Task[None]] = set()
+
 
 class PullRequestHandler:
     def __init__(self, github_webhook: GithubWebhook, owners_file_handler: OwnersFileHandler):
@@ -100,6 +103,18 @@ async def process_pull_request_webhook_data(self, pull_request: PullRequest) ->
 
             # Set auto merge only after all initialization of a new PR is done.
             await self.set_pull_request_automerge(pull_request=pull_request)
+
+            if hook_action == "opened":
+                task = asyncio.create_task(
+                    call_test_oracle(
+                        github_webhook=self.github_webhook,
+                        pull_request=pull_request,
+                        trigger="pr-opened",
+                    )
+                )
+                _background_tasks.add(task)
+                task.add_done_callback(_background_tasks.discard)
+
             if self.ctx:
                 self.ctx.complete_step("pr_handler", action=hook_action)
             return
@@ -115,6 +130,17 @@ async def process_pull_request_webhook_data(self, pull_request: PullRequest) ->
             for result in results:
                 if isinstance(result, Exception):
                     self.logger.error(f"{self.log_prefix} Async task failed: {result}")
+
+            task = asyncio.create_task(
+                call_test_oracle(
+                    github_webhook=self.github_webhook,
+                    pull_request=pull_request,
+                    trigger="pr-synchronized",
+                )
+            )
+            _background_tasks.add(task)
+            task.add_done_callback(_background_tasks.discard)
+
             if self.ctx:
                 self.ctx.complete_step("pr_handler", action=hook_action)
             return