MLH sometimes does not set the "mergeability" status

[joestringer]

Over the past month or two, I have noticed that PRs can sometimes get stuck with the only failing check being the "Mergeability" check that MLH is responsible to set.

Typically the workaround from a developer perspective is to set a label like dont-merge/bad-bot, wait a few seconds for MLH to receive the event, then remove the label again, and wait to see that MLH receives an event and sets the mergeability check to pass.

It seems like some time recently, MLH has begun to miss some events or does not complete event handling by setting the status for this check. It's unclear to me which is the case.

[joestringer]

Next steps would probably be to (a) Check MLH logs if there is any indication of an issue, (b) if not, improve MLH logging so we can gather information when this occurs, (c) perhaps do a manual review of the logic to see if any case might hit this problem, and (d) try to catch a PR in the act and track down what happens to the webhook, does MLH receive the request to update the PR, does MLH handle the event correctly, and are there appropriate retries if MLH cannot post the status successfully.

[joestringer]

cilium/cilium#40664 is exhibiting this behavior right now:

[joestringer]

There's another example here via a Renovate PR: cilium/cilium#40746 .