OCPBUGS-35528: keys: Update Red Hat keys to use SHA256 signatures #57
Conversation
This key was added in 1c626cb (Add Red Hat public keys and storage locations to signing, 2019-05-16, openshift#4), but never used. Drop it to reduce cruft.
The outgoing key content had SHA-1 self-signed signatures (digest algo 2 [1]). As SHA-1 is slowly deprecated and removed, this began causing failures like this Go 1.22 OpenShift 4.17 FIPS failed install [2,3]: $ curl -s [https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/logs/periodic-ci-openshift-release-master-nightly-4.17-e2e-aws-ovn-fips/1800906552731766784/artifacts/e2e-aws-ovn-fips/ipi-install-install/artifacts/log-bundle-20240612161314.tar] | tar -tvz | grep 'cluster-version.*log' -rw-r--r-- core/core 54653 2024-06-12 09:13 log-bundle-20240612161314/bootstrap/containers/cluster-version-operator-bd9f61984afa844dcd284f68006ffc9548377c045eff840096c74bcdcbe5cca3.log $ curl -s [https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/logs/periodic-ci-openshift-release-master-nightly-4.17-e2e-aws-ovn-fips/1800906552731766784/artifacts/e2e-aws-ovn-fips/ipi-install-install/artifacts/log-bundle-20240612161314.tar] | tar -xOz log-bundle-20240612161314/bootstrap/containers/cluster-version-operator-bd9f61984afa844dcd284f68006ffc9548377c045eff840096c74bcdcbe5cca3.log | grep GPG I0612 16:06:15.952567 1 start.go:256] Failed to initialize from payload; shutting down: the config map openshift-config-managed/release-verification has an invalid key "verifier-public-key-redhat" that must be a GPG public key: openpgp: invalid data: tag byte does not have MSB set: openpgp: invalid data: tag byte does not have MSB set E0612 16:06:15.952600 1 start.go:309] Collected payload initialization goroutine: the config map openshift-config-managed/release-verification has an invalid key "verifier-public-key-redhat" that must be a GPG public key: openpgp: invalid data: tag byte does not have MSB set: openpgp: invalid data: tag byte does not have MSB set That MSB complaint masked was from ReadKeyRing, masking an error like: openpgp: invalid data: user ID self-signature invalid: openpgp: invalid signature: RSA verification failure out of ReadArmoredKeyRing complaining about the SHA-1 self-signature. To address these issues, Red Hat has updated the self-signed signatures to use SHA-256 (digest algo 8 [1]): $ diff -u1 <(gpg --list-packets keys/verifier-public-key-redhat-release) <(curl -sL https://access.redhat.com/security/data/fd431d51.txt | gpg --list-packets) --- /dev/fd/63 2024-06-17 11:30:32.352577862 -0700 +++ /dev/fd/62 2024-06-17 11:30:32.355577862 -0700 @@ -10,5 +10,4 @@ :signature packet: algo 1, keyid 199E2F91FD431D51 - version 4, created 1256212795, md5len 0, sigclass 0x13 - digest algo 2, begin of digest 6c e9 - hashed subpkt 2 len 4 (sig created 2009-10-22) + version 4, created 1256212796, md5len 0, sigclass 0x13 + digest algo 8, begin of digest f2 24 hashed subpkt 27 len 1 (key flags: 03) @@ -19,2 +18,3 @@ hashed subpkt 23 len 1 (keyserver preferences: 80) + hashed subpkt 2 len 4 (sig created 2009-10-22) subpkt 16 len 8 (issuer key ID 199E2F91FD431D51) $ diff -u1 <(gpg --list-packets keys/verifier-public-key-redhat-beta-2) <(curl -s https://access.redhat.com/security/data/f21541eb.txt | gpg --list-packets) --- /dev/fd/63 2024-06-17 11:52:06.822506485 -0700 +++ /dev/fd/62 2024-06-17 11:52:06.824506485 -0700 @@ -10,5 +10,4 @@ :signature packet: algo 1, keyid 938A80CAF21541EB - version 4, created 1246901223, md5len 0, sigclass 0x13 - digest algo 2, begin of digest ff 6d - hashed subpkt 2 len 4 (sig created 2009-07-06) + version 4, created 1246901224, md5len 0, sigclass 0x13 + digest algo 8, begin of digest 14 74 hashed subpkt 27 len 1 (key flags: 03) @@ -19,3 +18,4 @@ hashed subpkt 23 len 1 (keyserver preferences: 80) + hashed subpkt 2 len 4 (sig created 2009-07-06) subpkt 16 len 8 (issuer key ID 938A80CAF21541EB) - data: [4096 bits] + data: [4093 bits] The public key data itself is unchanged and the SHA-256 self-signature has been backdated, so the resigned keys can still be used to verify existing signatures. OpenShift does not use the self-signed signatures to establish trust in the keys; that trust is established via the manifest in this repository being built into release images where the cluster-version operator can use it to validate signatures on requested update release targets. This commit updates this repository to bring in the new SHA-256 self-signatures. Generated with: $ curl -s https://access.redhat.com/security/data/fd431d51.txt >keys/verifier-public-key-redhat-release $ curl -s https://access.redhat.com/security/data/f21541eb.txt >keys/verifier-public-key-redhat-beta-2 $ make using: $ gpg --version gpg (GnuPG) 2.2.27 libgcrypt 1.8.8 Copyright (C) 2021 Free Software Foundation, Inc. License GNU GPL-3.0-or-later <https://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Home: /home/wking/.gnupg Supported algorithms: Pubkey: RSA, ELG, DSA, ECDH, ECDSA, EDDSA Cipher: IDEA, 3DES, CAST5, BLOWFISH, AES, AES192, AES256, TWOFISH, CAMELLIA128, CAMELLIA192, CAMELLIA256 Hash: SHA1, RIPEMD160, SHA256, SHA384, SHA512, SHA224 Compression: Uncompressed, ZIP, ZLIB, BZIP2 [1]: https://www.rfc-editor.org/rfc/rfc4880#section-9.4 [2]: https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-openshift-release-master-nightly-4.17-e2e-aws-ovn-fips/1800906552731766784 [3]: https://issues.redhat.com/browse/OCPBUGS-35528
openshift-ci-robot
added
jira/severity-moderate
|
@wking: This pull request references Jira Issue OCPBUGS-35528, which is invalid:
Comment The bug has been updated to refer to the pull request using the external bug tracker. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
/jira refresh |
openshift-ci-robot
added
the
jira/valid-bug
|
@wking: This pull request references Jira Issue OCPBUGS-35528, which is valid. The bug has been moved to the POST state. 3 validation(s) were run on this bug
Requesting review from QA contact: In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
openshift-ci-robot
removed
the
jira/invalid-bug
|
@wking: This pull request references Jira Issue OCPBUGS-35528, which is valid. 3 validation(s) were run on this bug
Requesting review from QA contact: In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
/lgtm |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: jupierce, wking The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
openshift-ci
bot
added
the
approved
@wking Is that bit expected or unexpected? Is this change a blocker for 1.22 based builds or something that will get changed in the 1.22 toolchain and has a bug tracking that? |
I hadn't personally expected this signature-algorithm issue. But golang/go#44226 has
It's possible that something could be changed in 1.22's Go to make this pull request unnecessary, but I'm not aware of any current trackers in this space. The |
|
I'm not clear on how okd-images could fail on $ curl -s https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/openshift_cluster-update-keys/57/pull-ci-openshift-cluster-update-keys-master-okd-images/1802791397808410624/artifacts/build-resources/imagestreams.json | jq '.items[].spec | .tags // [] | .[].name' | grep ovn
"ovn-kubernetes-base"
"ovn-kubernetes-base-rhel-9"
"ovn-kubernetes-microshift"
"ovn-kubernetes-microshift-rhel-9"
"ovn-kubernetes-rhel-9"
"ovn-kubernetes-base"
"ovn-kubernetes-base-rhel-9"
"ovn-kubernetes-microshift"
"ovn-kubernetes-microshift-rhel-9"
"ovn-kubernetes-rhel-9"Kicking off a retest, but I'll probably have to follow up about getting that OKD ImageStream modernized. /test okd-images |
|
/override ci/prow/okd-images |
|
@jupierce: Overrode contexts on behalf of jupierce: ci/prow/okd-e2e-aws, ci/prow/okd-e2e-aws-upgrade, ci/prow/okd-images In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
@wking: all tests passed! Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
@wking: Jira Issue OCPBUGS-35528: All pull requests linked via external trackers have merged: Jira Issue OCPBUGS-35528 has been moved to the MODIFIED state. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
[ART PR BUILD NOTIFIER] This PR has been included in build ose-cluster-update-keys-container-v4.17.0-202406180018.p0.g68c6f4a.assembly.stream.el9 for distgit ose-cluster-update-keys. |
|
/cherry-pick release-4.16 |
|
@rphillips: new pull request created: #64 In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
The outgoing key content had SHA-1 self-signed signatures (
digest algo 2). As SHA-1 is slowly deprecated and removed, this began causing failures like this Go 1.22 OpenShift 4.17 FIPS failed install:That MSB complaint masked was from ReadKeyRing, masking an error like:
out of
ReadArmoredKeyRingcomplaining about the SHA-1 self-signature. To address these issues, Red Hat has updated the self-signed signatures to use SHA-256 (digest algo 8):The public key data itself is unchanged and the SHA-256 self-signature has been backdated, so the resigned keys can still be used to verify existing signatures. OpenShift does not use the self-signed signatures to establish trust in the keys; that trust is established via the manifest in this repository being built into release images where the cluster-version operator can use it to validate signatures on requested update release targets.
This commit updates this repository to bring in the new SHA-256 self-signatures. Generated with:
using:
I'm also dropping an unused auxilliary key, as described in a3f5bae. With both commits, we no longer have any SHA-1 signatures: