Skip to content

fix: improve AI cherry-pick conflict resolution for modify/delete conflicts#1046

Merged
myakove merged 2 commits intomainfrom
fix/issue-1043-cherry-pick-ai-prompt
Mar 18, 2026
Merged

fix: improve AI cherry-pick conflict resolution for modify/delete conflicts#1046
myakove merged 2 commits intomainfrom
fix/issue-1043-cherry-pick-ai-prompt

Conversation

@myakove
Copy link
Copy Markdown
Collaborator

@myakove myakove commented Mar 18, 2026

Summary

  • Updated AI conflict resolution prompt with explicit rules for modify/delete conflicts
  • For modify/delete conflicts (file "deleted in HEAD"), the AI now keeps the file instead of deleting it
  • Removed --allow-empty fallback — empty cherry-picks now fall back to manual instructions since they indicate incorrect AI resolution

Context

Cherry-pick of PR RedHatQE/mtv-api-tests#349 to v2.10 resulted in an empty PR (#352) because the AI deleted the file (following the old prompt's "prefer target branch version" instruction). The file tools/bm-dns-setup.sh doesn't exist on v2.10 — git reports this as "deleted in HEAD" but it's actually a new file being brought to the target branch.

Test plan

  • Updated test_cherry_pick_ai_resolves_conflicts verifies prompt includes modify/delete guidance
  • Updated test_cherry_pick_ai_empty_result_falls_back_to_manual verifies empty cherry-picks fail correctly
  • Full test suite passes (1436 tests, 90.28% coverage)

Closes #1043

Summary by CodeRabbit

  • Improvements

    • Updated AI conflict-resolution behavior to handle modify/delete conflicts more explicitly and simplify post-resolution instructions.
  • Bug Fixes

    • Changed empty cherry-pick handling to report failure instead of creating empty commits or attempting auto-fallbacks.
  • Tests

    • Updated tests to expect a manual fallback path and to verify the AI prompt includes delete/modify conflict guidance.

…flicts

The AI prompt incorrectly told the AI to prefer the target branch
version for incompatible changes. For modify/delete conflicts (file
doesn't exist on target branch), this caused the AI to delete the
file, resulting in an empty cherry-pick with no changes.

Updated the prompt with explicit rules:
- For modify/delete conflicts, KEEP the file (it's new to the target)
- Removed --allow-empty fallback since empty cherry-picks indicate
  wrong AI resolution, not a valid state

Closes #1043
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Mar 18, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 2103736e-4a09-4c68-a87d-65610b8edc72

📥 Commits

Reviewing files that changed from the base of the PR and between f2258d7 and 037b327.

📒 Files selected for processing (2)
  • webhook_server/libs/handlers/runner_handler.py
  • webhook_server/tests/test_runner_handler.py

Walkthrough

Refactors AI-driven cherry-pick conflict resolution prompt to explicit per-conflict-type guidance and staging instructions; removes the automatic --allow-empty commit fallback — when AI resolution yields an empty cherry-pick the handler now logs an error and returns False. Tests updated to expect the manual-fallback path.

Changes

Cohort / File(s) Summary
Handler: AI conflict resolution
webhook_server/libs/handlers/runner_handler.py
Rewrote AI conflict-resolution prompt to explicit per-conflict-type rules (modify/delete handling, staging). Removed auto-empty-commit fallback; on empty-result or failed AI continuation the handler now logs an error and returns False unconditionally.
Tests: empty-cherry-pick and prompt checks
webhook_server/tests/test_runner_handler.py
Renamed test to test_cherry_pick_ai_empty_result_falls_back_to_manual. Updated assertions to expect failure/manual-comment path, removed expectations for git --allow-empty and automated PR creation, and added checks that AI prompt includes delete/modify conflict guidance.

Sequence Diagram(s)

(omitted)

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

Suggested labels

commented-myakove, commented-coderabbitai[bot], lgtm-coderabbitai[bot], Review effort 3/5

Suggested reviewers

  • rnetser
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Linked Issues check ⚠️ Warning The PR modifies AI prompt behavior to prevent empty cherry-picks but removes the --allow-empty fallback requested in issue #1043, creating a mismatch with the stated deliverable. The PR contradicts issue #1043's primary deliverable to handle empty cherry-picks with --allow-empty. Instead, it prevents empty results by changing the prompt and falls back to manual instructions, which conflicts with the expected behavior of committing with --allow-empty.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: improving AI cherry-pick conflict resolution specifically for modify/delete conflicts.
Out of Scope Changes check ✅ Passed All changes directly address cherry-pick conflict resolution and AI prompt improvements as outlined in the PR objectives, with no unrelated modifications detected.
Docstring Coverage ✅ Passed Docstring coverage is 83.33% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/issue-1043-cherry-pick-ai-prompt
📝 Coding Plan
  • Generate coding plan for human review comments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@myakove-bot
Copy link
Copy Markdown
Collaborator

Report bugs in Issues

Welcome! 🎉

This pull request will be automatically processed with the following features:

🔄 Automatic Actions

  • Reviewer Assignment: Reviewers are automatically assigned based on the OWNERS file in the repository root
  • Size Labeling: PR size labels (XS, S, M, L, XL, XXL) are automatically applied based on changes
  • Issue Creation: Disabled for this repository
  • Pre-commit Checks: pre-commit runs automatically if .pre-commit-config.yaml exists
  • Branch Labeling: Branch-specific labels are applied to track the target branch
  • Auto-verification: Auto-verified users have their PRs automatically marked as verified
  • Labels: All label categories are enabled (default configuration)

📋 Available Commands

PR Status Management

  • /wip - Mark PR as work in progress (adds WIP: prefix to title)
  • /wip cancel - Remove work in progress status
  • /hold - Block PR merging (approvers only)
  • /hold cancel - Unblock PR merging
  • /verified - Mark PR as verified
  • /verified cancel - Remove verification status
  • /reprocess - Trigger complete PR workflow reprocessing (useful if webhook failed or configuration changed)
  • /regenerate-welcome - Regenerate this welcome message

Review & Approval

  • /lgtm - Approve changes (looks good to me)
  • /approve - Approve PR (approvers only)
  • /automerge - Enable automatic merging when all requirements are met (maintainers and approvers only)
  • /assign-reviewers - Assign reviewers based on OWNERS file
  • /assign-reviewer @username - Assign specific reviewer
  • /check-can-merge - Check if PR meets merge requirements

Testing & Validation

  • /retest tox - Run Python test suite with tox
  • /retest build-container - Rebuild and test container image
  • /retest python-module-install - Test Python package installation
  • /retest pre-commit - Run pre-commit hooks and checks
  • /retest conventional-title - Validate commit message format
  • /retest all - Run all available tests

Container Operations

  • /build-and-push-container - Build and push container image (tagged with PR number)
    • Supports additional build arguments: /build-and-push-container --build-arg KEY=value

Cherry-pick Operations

  • /cherry-pick <branch> - Schedule cherry-pick to target branch when PR is merged
    • Multiple branches: /cherry-pick branch1 branch2 branch3

Label Management

  • /<label-name> - Add a label to the PR
  • /<label-name> cancel - Remove a label from the PR

✅ Merge Requirements

This PR will be automatically approved when the following conditions are met:

  1. Approval: /approve from at least one approver
  2. LGTM Count: Minimum 1 /lgtm from reviewers
  3. Status Checks: All required status checks must pass
  4. No Blockers: No wip, hold, has-conflicts labels and PR must be mergeable (no conflicts)
  5. Verified: PR must be marked as verified

📊 Review Process

Approvers and Reviewers

Approvers:

  • myakove
  • rnetser

Reviewers:

  • myakove
  • rnetser
Available Labels
  • hold
  • verified
  • wip
  • lgtm
  • approve
  • automerge

💡 Tips

  • WIP Status: Use /wip when your PR is not ready for review
  • Verification: The verified label is automatically removed on each new commit
  • Cherry-picking: Cherry-pick labels are processed when the PR is merged
  • Container Builds: Container images are automatically tagged with the PR number
  • Permission Levels: Some commands require approver permissions
  • Auto-verified Users: Certain users have automatic verification and merge privileges

For more information, please refer to the project documentation or contact the maintainers.

@myakove
Copy link
Copy Markdown
Collaborator Author

myakove commented Mar 18, 2026

@coderabbitai review

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Mar 18, 2026

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
webhook_server/libs/handlers/runner_handler.py (1)

802-810: ⚠️ Potential issue | 🟠 Major

HIGH: Missing empty cherry-pick recovery blocks valid operations.

When the target branch already contains equivalent changes, git cherry-pick --continue reports "The previous cherry-pick is now empty, possibly due to conflict resolution"—a legitimate condition, not an error. The current code returns False immediately, failing valid cherry-picks that need --allow-empty to complete.

The web search confirms git commit --allow-empty -C CHERRY_PICK_HEAD is the standard, supported approach to finish the cherry-pick while preserving the original commit message and author. This should be applied when cherry-pick --continue fails with an empty result.

Suggested fix
                if not rc:
-                    self.logger.error(f"{self.log_prefix} cherry-pick --continue failed after AI resolution: {err}")
-                    return False
+                    if "previous cherry-pick is now empty" in err.lower():
+                        rc, _, err = await run_command(
+                            command=f"{git_cmd} commit --allow-empty -C CHERRY_PICK_HEAD",
+                            log_prefix=self.log_prefix,
+                            redact_secrets=[github_token],
+                            mask_sensitive=self.github_webhook.mask_sensitive,
+                        )
+                        if rc:
+                            self.logger.info(
+                                f"{self.log_prefix} Cherry-pick became empty after AI resolution; preserved it with --allow-empty"
+                            )
+                            return True
+                    self.logger.error(f"{self.log_prefix} cherry-pick --continue failed after AI resolution: {err}")
+                    return False
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@webhook_server/libs/handlers/runner_handler.py` around lines 802 - 810, The
failure handler after calling run_command for "cherry-pick --continue" treats
any non-zero rc as fatal; change it to detect the empty-cherry-pick message
("The previous cherry-pick is now empty" / "possibly due to conflict
resolution") and, in that case, run a recovery command using git commit
--allow-empty -C CHERRY_PICK_HEAD (reusing git_cmd, github_token, and
mask_sensitive parameters passed to run_command) to finalize the cherry-pick and
return success; otherwise keep the existing error logging via self.logger.error
and return False. Ensure the recovery path logs a clear info/debug message and
still uses run_command with redact_secrets and mask_sensitive like the original
call.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@webhook_server/libs/handlers/runner_handler.py`:
- Around line 753-756: The current conflict-resolution branch that outputs the
message fragment "2. For modify/delete conflicts (file 'deleted in HEAD and
modified in ...'): the file does NOT exist on the target branch yet. KEEP the
file — use 'git add'..." must not unconditionally resurrect deleted files;
instead detect the specific conflict pattern "deleted in HEAD and modified in"
and treat the intent to delete in HEAD as authoritative: do not auto-create or
stage the file, mark the conflict resolved by removing the file (use 'git rm' or
equivalent in the code path) or leave it deleted and record resolution, and
update the user-facing message to reflect that deletion is preserved; locate the
code that emits that string and the associated resolution actions in
runner_handler.py and change the logic to prefer deletion for that exact
conflict pattern rather than always keeping the file.

---

Outside diff comments:
In `@webhook_server/libs/handlers/runner_handler.py`:
- Around line 802-810: The failure handler after calling run_command for
"cherry-pick --continue" treats any non-zero rc as fatal; change it to detect
the empty-cherry-pick message ("The previous cherry-pick is now empty" /
"possibly due to conflict resolution") and, in that case, run a recovery command
using git commit --allow-empty -C CHERRY_PICK_HEAD (reusing git_cmd,
github_token, and mask_sensitive parameters passed to run_command) to finalize
the cherry-pick and return success; otherwise keep the existing error logging
via self.logger.error and return False. Ensure the recovery path logs a clear
info/debug message and still uses run_command with redact_secrets and
mask_sensitive like the original call.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: a5b11842-561e-45df-baa8-4246d3b12543

📥 Commits

Reviewing files that changed from the base of the PR and between f392601 and f2258d7.

📒 Files selected for processing (2)
  • webhook_server/libs/handlers/runner_handler.py
  • webhook_server/tests/test_runner_handler.py

Comment thread webhook_server/libs/handlers/runner_handler.py Outdated
coderabbitai[bot]
coderabbitai Bot previously approved these changes Mar 18, 2026
Rewrite the AI prompt with clear natural language instructions for
each conflict type instead of vague rules. The AI now understands
when to keep vs remove files in modify/delete conflicts based on
whether the cherry-pick is introducing or backporting the file.
@myakove myakove merged commit 5a4bdd3 into main Mar 18, 2026
7 of 9 checks passed
@myakove myakove deleted the fix/issue-1043-cherry-pick-ai-prompt branch March 18, 2026 12:52
@myakove-bot
Copy link
Copy Markdown
Collaborator

New container for ghcr.io/myk-org/github-webhook-server:latest published

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: cherry-pick --continue fails when AI resolution produces empty commit

2 participants