MIGRATION: [Developer] The merge_arrow_pr script should be able to handle issues on GitHub and JIRA

[raulcd]

Describe the feature requested.

As we are migrating from JIRA issues to GitHub issues and now that no new users can be created on JIRA we should modify the merge_pr script to be able to handle issues on GitHub.

I am happy to work on a PR but there are some things that might need to be decided first.

1. How do we link issues with PRs? Currently we add the JIRA ticket id to the title of the PRs ARROW-XXXX:. Do we want to do the same with GitHub issues: GH-XXXXX: or do we want to have a mandatory Closes XXXX on the PR description so once the PR is merged the issue gets automatically closed? We have to define a way for either the merge script or the PR being merge to close the GitHub issue.
2. We need to add Labels to GitHub to identify the different components. Currently the GitHub merge_pr shows the identified components when merging, we should keep that functionality.
3. We should create the initial milestones so the merge_pr script is able to add, prompt the correct milestone when merging the PR.
4. Do we want to add assignee to the issue as we do with JIRA?

The merge_pr script should continue working with JIRA as the parquet CPP project is not going to be move to GitHub as far as I understand.

Component

Developer Tools

[assignUser]

1. GitHub issues: GH-XXXXX ✖️ ✖️ The link between issue and PR should be made with the intended GH functionality, so either using a keyword like "closes ARROW-264: File format #123" or manually linking the PR and issue via the web gui. Adding the keyword to the PR body is basically the same as the ARROW-123 in the title, so replacing one with the other should be fine.
2. We already have the labeler workflow that could easily be adapted to label components by the sub dirs of the changed files (e.g. r/* -> R cpp/* -> C++ dev/archery/* -> Developer dev/tasks -> CI ...)
3. 👍
4. For issues it would make sense to assign them when you start working on them. We could have the bot add the PR creator as an assignee on the matching issue on opening the PR. That would keep things consistent and also mark the issues as WIP? I don't think we need to use the assignee field on PRs themselves though.

I am unsure on the details of parquet as a "sub-project"(?) but I really don't think it makes any sense to use two separate tools for issue tracking within the same repo.

[jorisvandenbossche]

2. We need to add Labels to GitHub to identify the different components. Currently the GitHub merge_pr shows the identified components when merging, we should keep that functionality.

This doesn't necessarily have to be in the merge script. For example, we could also have a bot that runs whenever a PR is merged, checks if there is a component label set, and if not adds a comment like "Hey merger, you forgot a component label!"

3. We should create the initial milestones so the merge_pr script is able to add, prompt the correct milestone when merging the PR.

I see that the 11.0.0 milestone already exists (the only relevant on the short term, I think, since for a potential 10.0.2 we still use JIRA?)

[pitrou]

I like the GH-1234 convention. CPython does this and it generates hypertext links, so there must be a way to make it work:
python/cpython@8dbe08e

We would probably still use the merge script that would handle the duty of autoclosing the GH issue if desired.

[jorisvandenbossche]

I think we should try to consistently use the GH-xxxx syntax to reference issues (that probably won't happen in general comments in discussions, but at least we could try to do this for the top-level comment that mentions the issue to close). That makes it clear it is a github issue, and is also more robust in case we have another migration in the future.

Whether we should include the "GH-xxx" in the title, I don't have a strong opinion (it makes it clearer that there it is a PR with an associated issue, but if we allow PRs without such associated issue, PRs would have a mix of titles with and without that prefix anyway ..)

[jorisvandenbossche]

I like the GH-1234 convention. CPython does this and it generates hypertext links, so there must be a way to make it work:
python/cpython@8dbe08e

I think that's just GitHub that recognizes such text and make it into a hyperlink when rendering?
From that link, it seems that the commit message there is just the PR title + top-post body

[pitrou]

I think that's just GitHub that recognizes such text and make it into a hyperlink when rendering?

Right, it just worked in your reply :-)

[assignUser]

https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/autolinked-references-and-urls#issues-and-pull-requests