[CI][Python] Investigate trusted publishing for uploading wheels to PyPI

[pitrou]

Describe the enhancement requested

"Trusted publishing" may help improve the security of wheel uploads, and also provide automated digital attestations.
See https://discuss.python.org/t/pypi-now-supports-digital-attestations/71158 and https://trailofbits.github.io/are-we-pep740-yet/ .

It might make it either easier or harder to publish releases, however :)

Component(s)

Continuous Integration, Packaging, Python

[pitrou]

cc @raulcd @assignUser

[assignUser]

We talked about trusted publishing and related GitHub features just yesterday at the infra roundtable!

[pitrou]

Ah, it's nice that you attend those. Thanks a lot :)

[raulcd]

I am curious on how the ASF release process wants to handle those. Currently if we follow the current release process we would be generating the binaries, uploading them to dist.apache.org, verifying them and once the vote passes, downloading them locally from dist and uploading them to PyPI.
If we still want to generate binaries and "vote" them, we still would have to upload them to dist.apache.org and after the vote passes create a GH action that downloads the "voted" binaries and uploads those to PyPI, while generating the short lived tokens for "Trusted publishing" this would remove the possibility of an ill intentioned PMC or committer with rights to upload wheels from uploading wrong wheels but there is still the case of someone uploading wrong wheels to dist.apache.org.

A different solution could be to just generate the final wheels once the source has been voted and released and upload those newly generated wheels on the same job that generates them without possibility of tampering (bear in mind source code is signed and could be validated that no modification has happened).
I am curious on what other projects do, are other projects not voting on wheels and just publishing afterwards or are they downloading the generated wheels and uploading them as we do?

[raulcd]

cc @kou for feedback too

[pitrou]

I've often said that I don't find our voting process useful. If the binaries are thoroughly verified on CI jobs, then it does not really make sense to have them verified manually, either (especially as the manual verification just consists in executing a script).

[kou]

RubyGems also has "Trusted Publishing" mechanism: https://guides.rubygems.org/trusted-publishing/

If we can use it, I want to use it for RubyGems too.

BTW, could you share what the INFRA said?

[ianmcook]

Review of a recent supply chain attack on a PyPI package that mentions the importance of using trusted publishing and revoking unused PyPI API tokens: https://blog.pypi.org/posts/2024-12-11-ultralytics-attack-analysis/

[pitrou]

Thanks for the reference @ianmcook . This looks scary.

[pitrou]

The following guidelines may be worth looking at by someone more competent than me at these things:
https://blog.pypi.org/posts/2024-12-11-ultralytics-attack-analysis/#what-can-you-do-as-a-publisher-to-the-python-package-index

[assignUser]

Here is a VERY WIP project that is currently in the initial phases that will change the releases process UX in the coming year (I hope, the ASF is hiring for it) https://artifacts.apache.org/docs/voting.html

Trusted publishing connections to the usual repositories have been mentioned as potential features, we will have to see where that goes.

I've often said that I don't find our voting process useful.

I think with reproducible builds the verification process will make much more sense, as it will test the actual reproducibility and not things that have already been tested. Though I haven't checked how other projects do it.

Airflow is likely the project with the largest scale release of python wheels for their providers: https://dist.apache.org/repos/dist/release/airflow/providers/ and they are currently working on setting up a comprehensive publishing workflow https://github.com/apache/airflow-publish