PEP 761: Deprecating PGP signatures for CPython artifacts

PEPs
1

If you’re a downstream user of PGP signatures, you can help feed SC context by answering these questions:

  • When do you start downloading and verifying Python pre-releases? In the fastest timeline for discontinuation, the first affected Python beta version would be released in May 2025. This PEP’s acceptance allows for the SC to change the timeline of discontinuation.
  • What blockers exist for your adoption of Sigstore? (not including offline verification, which is now supported by both Python and Go Sigstore clients)
  • What is the worse-case scenario for your use-case if PGP is not maintained and Sigstore can’t be adopted?
9 Likes
2

How about adding some links to the PEP to good, general Sigstore background information?

3 Likes
3

I can definitely add more links to Sigstore’s overview and documentation!

3 Likes
4

I can’t speak for all RMs, but as one of them, I have seen our adoption of Sigstore as a good thing. There were some minor teething problems that we’ve worked out quite a while ago, and we’re currently in a pretty good state.

I support this PEP.

One thing I would like Sigstore to allow us is for any release manager from the team to be able to issue a new release of any Python version. I did do this a few times over the years, and the last attempt (although generally successful) caused confusion as to why the Sigstore signatures are mine and not Pablo’s. But this is probably off-topic here.

I would also like to add that PGP’s future is somewhat unclear anyway given how the Web of Trust that it always explicitly listed as a crucial component of its security is now effectively dead due to denial-of-service attacks on key servers through signature spam and GDPR takedown requests. We’re not going back to physical key signing parties.

So I’m entirely fine with us dropping the PGP signatures for Python 3.14 and up.

8 Likes
5

Is there a way to engage with downstream distributors (conda, debian, fedora, BSDs…) to coordinate this change and be sure the dates chosen don’t cause avoidable trouble or ill-will?

On the debian side, a thread is ongoing: python devs are planning to stop signing with gpg
(link to first message, continued in Sept and Oct)

Some takeaways:

  • the tone is strangely «they are doing this! we are impacted!» with no suggestion of engaging in communication with python-dev [edit: that may be how I read it, based on previous discussions about supporting pyproject.toml, removing setup.py – no ill intent meant for the participants of this discussion]
  • the tool used to identify new versions of packages (uscan) can check gpg signatures; for python it wasn’t enabled until that thread
  • adding support for alternates to gpg, including sigstore, is of interest to debian developers for a few projects (including python)
5 Likes
6

Yeah, this is something that’s on my long-term roadmap as well: the adoption of “workload identities” instead of using release manager identity. In short, this would be possible if all of our artifacts were generated on platforms that are supported by Sigstore. Outstanding we have macOS artifacts that are generated outside of a hosted build platform and Azure Pipelines doesn’t support OpenID Connect identities last I checked (which wasn’t that long ago?)

I pinged a few folks from Red Hat and Fedora that I knew about in the pre-PEP discussion. As you mentioned Debian and Ubuntu are also aware, the discussion spawned from the pre-PEP discussion as well, I think there is alignment on adding support for Sigstore to tooling because there are becoming more-and-more Sigstore signatures for projects being repackaged (ie, from NPM and soon from PyPI with PEP 740).

My hopeful read of the situation is that if support is added to tooling then our decision isn’t as negatively impactful one as it might have been a few years ago where there wasn’t as much interest in adding Sigstore support to tooling.

I can try to find some more people, I’m not sure what other levers we have to spread the word about this upcoming change, maybe the Python blog?

2 Likes
7

As was pointed out in the prp-pep thread already, as well as touched on a bit here, pgp/gpg are inherently prone to error, and the web of trust is dead. No cryptographer today would design a signing scheme the way it is designed, and I’m glad to see the RMs using more modern, less error-prone options.

Announce it and let them be responsible as redistributors? (Which I believe is already in progress) I don’t want to start a fight on this about redistribution, but if CPython is providing a specific way because it is a better way, it’s up to them to use it or not or to come to the table with why it doesn’t meet their needs and an alternative[1]. Debian was already not verifying pgp signatures prior to this, which some might have a problem with, but others might find that how the artifacts were gotten already had a level of trust built in

  1. And that alternative shouldn’t just be keep using the fundamentally outdated pgp. ↩︎

2 Likes
8

On the debian side, a thread is ongoing: python devs are planning to stop signing with gpg
(link to first message, continued in Sept and Oct)

Some takeaways:

  • the tone is strangely «they are doing this! we are impacted!» with no suggestion of engaging in communication with python-dev

Somewhat of a misrepresentation I think? Both LtWorf and stefanor have been replying here too. The main takeaways from the Debian side have been:

  1. Oops! We should have been checking CPython release artifact signatures but weren’t. That’s being fixed now, though could regress back to the prior state with 3.14 releases next year.

  2. It would be nice if Debian’s tooling were updated to support additional release signature schemes besides OpenPGP (Signify, Sigstore, whatever else comes along).

8 Likes
9

My position hasn’t changed since the original thread, but I’ll copy it here for posterity/continuity: I’m a strong +1 on replacing PGP, especially in a natural context like this (where the RM succession would otherwise invoke a new RM having a make a PGP key for the first time).

3 Likes
10

The first 3.14 alpha release is scheduled for a week from now; is the plan to already apply this change for that release?

11

I’m not planning on including PGP signatures for the initial alphas. I hope we can have a decision well ahead of the beta feature freeze in May next year.

As the first release manager affected by this, I’m also a strong +1 on this.

Sigstore looks a much better solution, we already have it in place, and I’d much rather spend my RM time on things that are more useful for our users, such as including official binary releases for iOS and Android, or releasing installers during the security phase.

I’ve not needed to use PGP for two decades, since Nokia (remember them?) used to mail us Symbian (remember that?) SDKs encrypted on DVDs (remember those?).

10 Likes
12

This is pointing to a single blog post from 2019. Other than that, is there any consensus that the Web of Trust is effectively dead today?

(I otherwise agree it hasn’t been particularly effective, partly because of how obnoxious the tooling around it generally is)

13

That single post is pointing out gpg is ignoring key signatures coming from servers. self-sign-only is the default now. SKS servers shut down , and keys.openpgp.org only distributes emails and no third-party signatures.

6 Likes
14

I think there’s a general consensus in academic and corporate cryptography that the two things called the WoT in PGP are effectively dead:

  1. The SKS network has been effectively inoperative since 2019 (and officially inoperative since 2023, IIRC), per the original post. There are other keyservers, but they aren’t networked to the same degree and no equivalent alternative has arisen (in part because of the GDPR, as noted). Consequently, the dominant PGP implementation (GnuPG) removed SKS support a while back.
  2. The “strong set” is the other thing that people refer to as the WoT. It still exists (in the sense that some of the keys in it haven’t expired), but there’s no canonical view of the strong set anymore due to (1). In other words: there is still a “core” of trust encoded in some peoples’ local keyrings, but that trust is not meaningfully substantiated (and in a more fundamental sense never was, because trust relationships aren’t actually transitive).

Somewhat fittingly, there aren’t actually that many articles about the WoT dying, since people never really used it much in the first place :slight_smile: – the only other good ones I know of are:

8 Likes
15

This is pointing to a single blog post from 2019. Other than that, is there any consensus that the Web of Trust is effectively dead today?

Depends on what sort of consensus you’re looking for. The primary redistributor of WoT key signatures for many years was the SKS keyserver network, and the events surrounding CVE-2019-13050 along with an inability to effectively wipe specific entries (e.g. to handle GDPR-oriented requests) resulted in its maintainers taking it fully offline as of June 2021.

At this point, the most used successor (keys.openpgp.org) only distributes self-signatures on keys per its FAQ, so is effectively no longer supplying WoT relationships. In theory the software it runs on (Hagrid) should support third-party keysigs if attested to by the signee, but I haven’t found time to test that with my own key: database: serve first-party attested third-party certifications (!176) · Merge requests · keys.openpgp.org / hagrid · GitLab

Some communities run their own keyservers to reflect a smaller scope WoT among their members, but the idea of a global Web of Trust across a network of federated keyservers hasn’t been a reality for several years now.

3 Likes
16

I’m not sure why all this PGP/GPG bashing is needed to make an argument for not providing PGP/GPG sigs anymore.

Just to set the record straight: you are comparing apples to oranges when comparing PGP/GPG sigs to sigstore.

PGP/GPG works much like SSH keys, whereas sigstore works similarly to CA based TLS, with a central authority signing keys.

With sigstore, you have to trust the CA. With PGP/GPG you can choose to trust the person signing the data individually.

Now, PGP/GPG is not dead at all. It’s being used for email encryption in e.g. Thunderbird, and it’s the main authenticity provider in many Linux distributions. Just like with SSH keys you assign trust to one by one, the latter ask you whether you want to trust a particular key or not.

Of course, just like the CA based trust model, the PGP/GPG trust management model has its downsides.

Finding out whether to trust a key or not is left to the user and detemining the trust level can sometimes be difficult. OTOH, with the CA model, you simply pass this task on to third parties and have to trust those doing a good job.

In the end, there are pros and cons to both approaches.

Now, it’s well possible to work around the issues mentioned in the PEP and in some of the comments on this PEP topic, related to having to manage individual PGP/GPG keys for producing sigs: just like many other projects do, you simply use a single project key for all releases.

Signing can then be done automatically using a CI/CD process… in much the same way, PyPI is advocating cutting releases of packages today. This would eliviate the need for manual interaction by the RM and also removes the need for the RMs to maintain personal PGP/GPG keys - those can be managed in a central place and also easily rotated on a regular basis.

So taking all this into account, the PEP proposes a switch from putting trust into a handful of PGP/GPG keys to trusting a third party sigstore CA doing a good job.

I think the PEP ought to put more emphasis on this aspect, rather than using the new shiny thing is so much better and we should all use it approach.

I still believe that providing both sigs is a good service to the Python community, since both have their stengths and weaknesses.

7 Likes
17

FWIW, I happen to agree, but that’s part of the problem: secure cryptographic design dictates that cryptographic tools and protocols should do one thing and do it well, and PGP violates that design doctrine.

The problem with PGP (and GPG) is not the (manual) key or identity distribution model, per se: it’s that PGP is a kitchen sink ecosystem that doesn’t actually do signatures particularly well (whether in terms of broad/standardized support for good key/signature algorithms, or the packet design format itself, or various memory safety and cryptography bugs in dominant implementations like GPG).

I can go down the rabbithole on concrete examples of PGP being an unnecessarily dangerous ecosystem with bad defaults, but the point isn’t to bash here. It’s to evidence that the sober professional cryptographic opinion about PGP (and GnuPG) in particular is that it isn’t a good foundation for modern cryptographic tasks, and that the relative upside to supporting it is small given the RM transition.

(As an important side note: a shared CI-managed GPG key is a very different threat model for CPython releases than the current RM-held key model! It might be an acceptable one, but it changes the model from one where only the RM is trusted to sign to one where anybody with access to the key can sign, with those parties being indistinguishable at the identity layer. In practice, I’m also not sure that PGP key rotation practices are mature enough/easy enough to operationalize to prevent the normalization of deviance around key managment: consider that even GitHub has failed to properly rotate their signing key for gh twice now.)

3 Likes
18

Um, in this particular case it doesn’t seem all that difficult.
Hugo, who should I trust about proper handling of long-lived keys – you personally, or the third parties that you and Seth choose?

(That’s obviously a rhetorical question, as your name is on this PEP.)

I can’t help but notice that this would mean giving the keys to a third party – and in a way that’s not primarily designed for signing software.

2 Likes
19

You are stating the obvious, of course :slight_smile:

Well, you already trust those services with building your software distribution, so storing the keys in their vaults doesn’t appear to require much more trust in them.

At the end of the day, these signatures are mostly needed to verify whether the bundle you downloaded actually matches the one which was provided. Hashes already do a great job at this. Sigs are better, though, because you have to somehow know whether you hashes you used for verification are, in fact, the correct ones.

But as a user, you still have to trust the core devs, the RMs, the Linux distributions recompiling everything and the services being used by all of these people.

Note that in todays world, where we rather carelessly even put trust into random container images downloaded from docker hubs without further inspection, a lot of these distribution chain discussions appear a bit, well, theoretic. But hey, they are fun anyway :slight_smile:

3 Likes
20

In Gentoo, we’re not able to start evaluating or testing sigstore until CLI: `--offline` should mean fully offline · Issue #483 · sigstore/sigstore-python · GitHub is fixed (which it appears to be now) and it lands in a release, as it can’t work offline otherwise.

I would encourage both PGP + sigstore for at least one more release cycle to allow folks to both adapt and provide feedback, with a warning on the PGP sigs on the download page at least. We didn’t have sigstore packaged at all before now and had to package a lot of new dependencies for it which took time. Others will surely be in the same position. We also have a lot of tooling around PGP and none for sigstore yet.

1 Like