Skip to content

Add SHAKE-128 KDF support to HPKE implementation#14429

Open
bitloi wants to merge 15 commits intopyca:mainfrom
bitloi:bitloi/hpke-shake128-14394
Open

Add SHAKE-128 KDF support to HPKE implementation#14429
bitloi wants to merge 15 commits intopyca:mainfrom
bitloi:bitloi/hpke-shake128-14394

Conversation

@bitloi
Copy link
Contributor

@bitloi bitloi commented Mar 5, 2026

Summary

This PR adds SHAKE-128 support as an HPKE KDF and keeps the change scoped to a single algorithm for easier review.

Fixes #14394

Changes

  • Add KDF.SHAKE128 in the Rust HPKE backend and Python stub bindings.
  • Implement SHAKE-128 one-stage HPKE key schedule handling in Suite::key_schedule.
  • Correct one-stage LabeledDerive input construction to match the spec-required encoding.
  • Update HPKE docs to include the new KDF.SHAKE128 option.
  • Add focused HPKE tests for SHAKE-128:
    • successful encrypt/decrypt roundtrips across supported AEADs
    • failure on mismatched info
    • API exposure checks

Why this scope

  • This is intentionally one algorithm per PR, per maintainer guidance.
  • #14394 tracks multiple remaining algorithms; this PR addresses only SHAKE-128.

Validation

  • cargo fmt --manifest-path src/rust/Cargo.toml --check
  • uv run --extra pep8test -m ruff check src/cryptography/hazmat/bindings/_rust/openssl/hpke.pyi tests/hazmat/primitives/test_hpke.py
  • uv run --extra test -m ensurepip
  • uv run --extra test -m pip install -e .
  • PYTHONPATH=vectors uv run --extra test -m pytest tests/hazmat/primitives/test_hpke.py -q (43 passed)
  • uv run --extra docs --extra docstest -m sphinx -T -W -b html -d /tmp/cryptography-docs-doctrees docs docs/_build/html

Checklist

  • Small, focused PR (single algorithm)
  • Unit tests added/updated
  • Documentation updated
  • Relevant lint/tests pass locally
  • No changelog entry added (HPKE module is new, consistent with maintainer feedback)

@alex
Copy link
Member

alex commented Mar 5, 2026

we're going to need some test vectors for these to ensure correctness, in the same way we have them for the other algorithms. (Unfortunately I don't think those HPKE vectors include SHAKE). I'd check what Go uses here.

@bitloi
Copy link
Contributor Author

bitloi commented Mar 5, 2026

I've added it. Let me know if this looks good.

Copy link
Member

@alex alex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm surprised this needed to be so invasive, will have to review more closely, but if you can also look if there's a cleaner abstraction, that'd be good.

(AES-128-GCM, AES-256-GCM, ChaCha20Poly1305).
* HPKE SHAKE-128 (X25519, mode Base) vectors generated with Go primitives
using the one-stage HPKE key schedule defined in
`Go's crypto/hpke package`_.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can use:

https://github.com/hpkewg/hpke-pq/blob/main/test-vectors.json

which I found in the go repo

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've updated it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit confused, you seem to have deleted the docs, but not switched to these vectors at all.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the heads-up. The SHAKE vectors are wired in code via vectors/cryptography_vectors/HPKE/go-shake128-vectors.json in test_hpke.py (test_shake128_vector_decryption). I also re-added a concise docs note for the SHAKE-128 vector source in docs/development/test-vectors.rst pointing to hpke-pq test vectors, so both implementation and docs now match.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The vectors checked in do not actually match the file from the pq repo.

You need to slow down and be more thoughtful in what you're doing. If something is unclear, ask a clarifying question, but right now I'm spending a ton of time pointing out very basic issues.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, just fixed again.

@bitloi bitloi requested a review from alex March 6, 2026 16:00
(X25519, X448, P-256, P-521), KDFs (HKDF-SHA256, HKDF-SHA512), and AEADs
(AES-128-GCM, AES-256-GCM, ChaCha20Poly1305).
* HPKE SHAKE-128 (X25519, Base mode) vectors are checked into
`go-shake128-vectors.json`_.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not understand what you are doing here.

I've said we want to use the vectors from https://github.com/hpkewg/hpke-pq/blob/main/test-vectors.json

I don't understand why you've got back and forth on this numerous times, and the link to the vectors should go to where they're from, not where in the repo they live.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, and I apologize for the back-and-forth. I removed the incorrect docs attribution.

I checked hpkewg/hpke-pq/test-vectors.json directly: there is no kem_id=0x0020 (X25519), kdf_id=0x0010 (SHAKE128) vector in that file. The SHAKE128 vector there is kem_id=0x0010 (DHKEM(P-256)).

This branch currently only supports KEM.X25519, so I cannot switch the SHAKE128 decrypt-vector test to that upstream entry without adding P-256 KEM support first.

Could you confirm the preferred path?

  1. wait for Add DHKEM(P-256, HKDF-SHA256) support to HPKE implementation #14398 to land and then switch this PR to the pq SHAKE128 vector, or
  2. keep this PR as SHAKE-only and drop the custom SHAKE decrypt vector test until P-256 support is available.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, waiting for DHKEM to land first and use the official vectors is preferred.

Sending a seperate PR that lands just the vector file, properly documented, would help keep this moving.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense. I’ll do this in two steps:

  1. keep this SHAKE PR without the custom vector fixture, and
  2. open a separate PR that only adds the official hpkewg/hpke-pq vector file with proper source documentation.

Then once DHKEM support lands, I’ll switch the SHAKE vector test over to those official vectors.

Copy link
Contributor Author

@bitloi bitloi Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bitloi bitloi requested a review from alex March 9, 2026 20:33
@bitloi
Copy link
Contributor Author

bitloi commented Mar 9, 2026

Hi @alex Do you have any feedback?

@alex
Copy link
Member

alex commented Mar 9, 2026 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Expand HPKE algorithms

3 participants