Shared pytest-dev PyPI account/token?

[The-Compiler]

To push releases, @nicoddemus added a PyPI token with 1d3f27cef076df028ef6434b2d3bd29358c421c3 (which is stored in the PYPI_TOKEN secret in this repo).

Is this your personal account, @nicoddemus? Wouldn't it make sense to create a pytest-dev PyPI user, and then have a token which we can configure as organisation-scoped token for the pytest-dev GitHub organisation? That way, all pytest-dev plugins could add the pytest-dev user to PyPI (which would then be the recommendation instead of "We recommend that each plugin has at least three people who have the right to release to PyPI.").

This way, it'd also be easier for repositories to set up automated deployment via GitHub Actions (which could be another recommendation with an example), as they can use the existing organization-wide token and pytest-dev account instead of using their personal account.

Thoughts?

[RonnyPfannschmidt]

Do the tokens isolate in the sense that plugin repos may not upload pytest releases, i don't want to be paranoid but I'd strongly prefer per project tokens scoped to the pypi distribution in question simply to prevent certain scenarios from larger impact

[The-Compiler]

Nope. I can see where you're coming from, but if someone got hold of your PyPI account (or mine, or Bruno's, or ...) there's probably a lot of damage to be done as well. I can see how the risk would be bigger for a token which can easily be accessed by anyone in the pytest-dev organization, though.

[Zac-HD]

Can we also require that everyone use a hardware token like a Yubikey for 2FA on PyPI?

If cost would make anyone hesitate I'm sure the PSF would look favorably on a grant application to cover it 🙂

[nicoddemus]

Is this your personal account, @nicoddemus?

Yep!

Wouldn't it make sense to create a pytest-dev PyPI user, and then have a token which we can configure as organisation-scoped token for the pytest-dev GitHub organisation? That way, all pytest-dev plugins could add the pytest-dev user to PyPI

I like the idea of making that more convenient, at the moment it indeed requires a bit of manual setup to get automatic releases going.

However I'm a bit worried about the security implications if such a token gets compromised: that token supposedly would be able upload any pytest-dev package to PyPI.

but if someone got hold of your PyPI account (or mine, or Bruno's, or ...) there's probably a lot of damage to be done as well.

Definitely, but a single token to access to the entire organization seems more problematic, I think.

which would then be the recommendation instead of "We recommend that each plugin has at least three people who have the right to release to PyPI."

I'm not sure having a single key shared by everyone would improve things much, because while right now the recommendation is "We recommend that each plugin has at least three people who have the right to release to PyPI.", we could update it to "We recommend that each plugin is setup for automatic deployment using a token belonging to one of the maintainers", and this is not much different than "We recommend that each plugin is setup for automatic deployment using the token PYTEST_PYPI_TOKEN shared by the organization".

Also, I don't think PyPI has a notion of "organization", where projects could share a token there: new projects joining pytest-dev would still need to give release rights to the owner of PYTEST_PYPI_TOKEN on PyPI.

Does anybody knows how other organizations use these shared secrets? I would be interested in see how that's used, and if it is used for releases.

If cost would make anyone hesitate I'm sure the PSF would look favorably on a grant application to cover it 🙂

If this is a direction we want to take, money shouldn't be a problem because we have funds on OpenCollective that could cover this. 😁

[Zac-HD]

right now the recommendation is "We recommend that each plugin has at least three people who have the right to release to PyPI.", we could update it to "We recommend that each plugin is setup for automatic deployment using a token belonging to one of the maintainers"

I like this as a development workflow, but it can still have problems if a maintainer vanishes.

[nicoddemus]

I like this as a development workflow, but it can still have problems if a maintainer vanishes.

Actually, in this setup, every developer which can push to the repository can make a release even if the owner vanishes: the release is triggered by a tag (pushed by anyone), using that maintainer's token.

[Zac-HD]

And if anything happens to the token - which might be invalidated following a PyPI incident, for example - the project can no longer make releases.

(in practice I think the pytest-dev core team is sufficiently well known to and trusted by the PSF and PyPA team that we could manually recover, but I'd rather model best practices for the rest of the ecosystem than rely on special treatment)

[ssbarnea]

The token itself should not be shared as it is a temporary secret. It is only picked from pypi and saved on github secrets. There are two sets needed, one for production and one of test.pypi.org

Still few org admins need to have control over the credentials of the bot account. Best way is to use a private mailing list as the pypi user address, so credentials can be reset. Be sure you record a nice gravatar for that email address.

There is no standard medium used by others, whatever the admins agreed on (sic). I know teams even using google docs for that (no comment).

Github have decent security measures for preventing secrets exposure in its pipelines. Keep in mind that pypi service account must be only allowed to publish, "maintainer", not admin.

Also setting a secret at org level does not force everyone to use it, you can still define overrides for specific repo if you want. It is a convenience for reducing maintenance over lots of repos, nobody likes have to update 100 repos if tokens need to be refreshed.

That is used by orgs with more sensitive stuff than pytest and I would say publishing using personal accounts pose a bigger risk than a service one (is more likely for one of personal secrets to leaked than the service account).

Keep also in mind that nobody is forcing projects under pytest-dev org to use the organization pypi service account, it is up to them to give it access and enable the publish pipeline, nothing is implicit. Still, I would be more than happy to see it adopted.

[The-Compiler]

Github have decent security measures for preventing secrets exposure in its pipelines.

Accidental exposure, yes. Deliberate exposure can't be prevented. In this scenario, if the GitHub account of anyone being able to push/merge code to master for any pytest-dev plugin gets compromised, it means that person will also be able to release malicious versions of any other plugin or pytest itself.

To be honest, I didn't think of that when opening this issue and I don't like the idea. Thus, I'm -0.5 on this myself.

[webknjaz]

I think that the best thing to have is a PyPI bot account with "maintainer" level of access to the projects. And then, have an API token scoped per project. I also considered having an org-wide secret but then understood that the security implications are unfortunate. I wouldn't even do this for TestPyPI.

[graingert]

Best way is to use a private mailing list as the pypi user address, so credentials can be reset.

One way is to store a private key in github actions and encrypt api keys with that key in git using password-store (pass) then anyone can roll the key or add new keys travis style

[webknjaz]

One way is to store a private key in github actions and encrypt api keys with that key in git using password-store (pass) then anyone can roll the key or add new keys travis style

Or use Keybase and mount it as fuse wherever necessary

[ssbarnea]

When the ticket was opened I was ready to bet some over engineered solution would be proposed that would likely negate all the other benefits... I hope we would evaluate the risk and possible damage before making a decision. There is no perfect security. I would also propose experimenting with less mainstream changes and plugins and adapt based on what we learn when we scale.

[RonnyPfannschmidt]

personally i'd prefer that a few of the core maintainers just set up the project specific tokens as part of the organization maintenance/take-in, and we have a pytest-dev-upoader user on pypi that would be maintainer on all the projects

a solution for sharing the login credential by the core maintainers has to be found tho
(pass sounds like something i'd like for that)

in essence

• each project/repo gets a own token
• tokens belong to a pytest-dev-uploader user (to be created/exact name to be chosen)
• that users should have 2factor auth and core maintainers should have a way to access it

[webknjaz]

@RonnyPfannschmidt

4. it'd be cool to also have some sort of a linter to ensure that this bot user does not have "owner" access to those packages, only "maintainer"