MAINT: Use towncrier for release notes

[larsoner]

From the dev meeting today, we discussed a bit different options. @sappelhoff @agramfort you also commented in #11508 so I thought I'd bring your attention to this PR.

Towncrier was suggested by @drammock in #11508. Turns out that it's used by pip, pytest, and astropy, so should be well supported. Advantages relative to what we do now, all of which should make maintenance and contributing easier:

1. No more merge conflicts
2. More standard tool
3. Can automate "hey, please add a changelog entry" using an existing scientific-python action that checks for it
4. Can easily (I think) automate PR number addition (see below)

Minor arguable disadvantages:

1. Ordering is by PR number, so newcontrib no longer come first.
2. Automatic parenthetical at the end that links to the PR can only have a single PR number, so the workaround is to add any additional PRs as manual parentheticals like I do in this PR.
3. Notable changes must have bullet points and will have a PR number associated with them.

However, all three of these are pretty easy to fix manually at release time, so I think we can live with them for the benefits.

I think it will be easy at release time to use towncrier directly to create a doc/changes/v1.7.rst for example and remove the stubs from devel, which is great as well.

main	PR

towncrier wants to see files of the form <issue>.<type>.ext, so at the end of the day we need for example doc/changes/devel/12345.enhancement.rst. If you want to open a PR with an entry, currently you have to check to see what PR number you will get when you open by looking at the issues list, or open it then push your entry. With this PR, I'm going to write a little GitHub action (or improve the existing scientific-python one) so that you can just add doc/changes/devel/enhancement.rst and it will automatically git mv this to doc/changes/devel/{{ github.event.number}}.enhancement.rst for you!

Todo before merge I think:

• Break out current changes into stub files
• Use sphinxcontrib-towncrier to populate devel.rst on the fly
• Add action-towncrier-changelog
• Verify that the action raises an error for this PR

Separate PR:

• New action to add PR number to doc/changes/devel/*.rst files

Closes #12233
Closes #11508

[drammock]

looks good so far! One comment and one question:

• the screenshot says "Enhancements" rather than the "New Features" title we agreed on at the dev mtg.
• When do things get marked as "notable changes"? I'm guessing it could be done at PR time, or at release time, equally easily?
• How do things get added to "notable changes"? I can see 3 options, each with its own upsides/downsides:
  • PR only appears in "notable"
  • PR appears (copied verbatim) in "notable" and in its other natural category
  • PR appears in "notable" and in its natural category, but with different text in each place.

[larsoner]

Successfully failing!

the screenshot says "Enhancements" rather than the "New Features" title we agreed on at the dev mtg.

Pushing to fix

When do things get marked as "notable changes"?

Notable changes is a section just any any of the others (not special in any way). towncrier just takes all the changes/devel/NNNNN.type.rst files and creates the doc. Until we tell it to remove these and finalize the doc (which we'd do at release time), the doc/changes/devel/*.rst live in the repo, so they can be edited or appended to or whatever.

How do things get added to "notable changes"?

Whenever you add a 12345.notable.rst with whatever content you want. So it could be the same, different, etc. from any other entry to any other section. Nothing stops you from also adding a companion 12345.feature.rst for example with whatever content you want. towncrier will add the contents of 12345.notable.rst to Notable changes section and the 12345.feature.rst to the New features section. So it's up to us to decide how to split it.

[drammock]

So it's up to us to decide how to split it.

This is what I was actually trying to ask about (sorry that wasn't clear). So asking again: how do we want to populate the "notable changes" section?

1. with duplicates of entries from other sections
2. with entries from other sections, but reworded to be... better in some way (more high-level, less technical, more verbose, ...?)
3. as the sole entry for the PR to which it is related

[larsoner]

I think we can play it by ear / case-by-case. Generally I think (2) and (3) are better than (1). So far the case we do have is (2).

[cbrnr]

I'd prefer (3), (1) is the worst option IMO.

[sappelhoff]

I'd prefer (3), (1) is the worst option IMO.

agreed! (2) would be acceptable for me, and then less technical+more verbose and high level would be good.

[hoechenberger]

I'm a big fan of option 2

[larsoner]

@drammock I think we can do the auto-add-PR-number as a separate follow-up PR. Okay to merge this once I fix the conflicts so we can give it a shot (assuming consensus/decision is reached in #12233)?

[drammock]

@drammock I think we can do the auto-add-PR-number as a separate follow-up PR. Okay to merge this once I fix the conflicts so we can give it a shot (assuming consensus/decision is reached in #12233)?

I don't have time for a careful review of the code changes this morning, but I'm +1 to move ahead if CIs are green, you're satisfied with the config, and it follows #12233 (comment).