Automate breaking change detection

[rluvaton]

Is your feature request related to a problem or challenge?

I've just upgraded from datafusion 51 to 52 and now from 52 to 53, and in both cases not all breaking changes were listed in the upgrade guide which makes me dig the original change and understand it and what was before in order to update

Describe the solution you'd like

If we could have a GitHub action that track breaking changes (not behavior ones as this is much harder to do) and mark them with a label as a start.

Since clippy already know if the function is really public or not (like pub(crate) mod a { pub mod b { } } mod b is not really public we can use that somehow to check whether a change that was made in each pr is a breaking change regarding, and we can output a report of each breaking change in the pr, (like removed function a, b. added to trait function c with no default implementation, added argument to a function, changed visibility) and the author of the PR will have to address those with explanation and migration guide

It looks like there is a tool for this already:
https://crates.io/crates/cargo-semver-checks

Describe alternatives you've considered

Pain when I upgrade to encounter it myself

Additional context

No response

[rluvaton]

We could also force each breaking change PR to write a migration guide and in the release it is just a matter of collecting all of that migration guide files

[comphead]

I agree, LLM can help generate Upgrade Guide section, we already have a definition for breaking changes, as well as generate-changelog.py collects them by api-change Github tag. If we can feed those PRs to LLM asking for a concise upgrade summary, it might work

[rluvaton]

I think it should be up to the author of the breaking change

But to force PRs that make a breaking change to also include a couple of sentences is a good idea IMO

[Jefffrey]

fyi existing issue

• Use cargo semver-checks for release testing #13665