Some tools that create PRs for you (e.g. CLion) seem to create PRs and add labels to them in two separate steps. But it looks like the PR workflows are created with the metadata the PR had when the launch was triggered, not when the job was scheduled. So unless you use the GitHub UI (or perhaps the gh command line tool), you'll need to push an empty commit to cancel workflows. Bad UX, but I don't have the bandwidth to fix it right now.