Support for prehashed ML-DSA variants

[bhess]

FIPS 204 specifies two features related to pre-hashing in ML-DSA that are currently not supported in liboqs: "externalMu" and "HashML-DSA".

externalMu

FIPS context

Defined in Algorithm 7, line 6:

message representative that may optionally be computed in a different cryptographic module

This means mu can be computed externally and passed into the signature process instead of the raw message.

Current upstream state

The pq-crystals upstream code does not provide an option to pass mu. It is always computed internally, see here.

Future upstream state when adopting mldsa-native

The upcoming mldsa-native implementation does support external mu, see here.

Integration options (once mldsa-native is released and integrated)

1. Add algorithm variants

   • Example: ML-DSA-44-externalMu
   • Reuse existing signing API and interpret the message input as mu for these variants.
   • Pros: No API changes; aligns with liboqs pattern of algorithm-agnostic API.
   • Cons: Slight ambiguity in API semantics (message parameter is overloaded).

2. Introduce a new API

   • Example: OQS_SIG_sign_with_external_mu(...)
   • Pros: Explicit interface.
   • Cons: Heavy API change for an algorithm-specific feature.

My personal preference: Option 1, as externalMu is algorithm-specific and adding a new global API is difficult to justify.

HashML-DSA

FIPS context

Defined in Algorithm 4.

Steps:

• Accept a pre-hashed message as input.
• Prepend:
  • A different domain separation byte than pure ML-DSA
  • The OID of the hash function.
• This makes HashML-DSA incompatible with standard ML-DSA.

Note: SLH-DSA (FIPS 205) has a similar pre-hash option, so this pattern is not unique to ML-DSA.

Upstream state

Neither pq-crystals nor mldsa-native plans to support HashML-DSA.
See README reference.

Integration options for liboqs

1. Expose the internal signing function (ML-DSA.Sign_internal)

• Let the application prepend the pre-hash data manually and call a variant like ML-DSA-44-internal.
  However, FIPS 204, Section 6 states:

  Other than for testing purposes, the [internal] interfaces should not be made available to applications.

• Pros: low maintainance cost
• Cons: not foreseen by FIPS204

2. Implement wrappers in liboqs

   • Prepend the required domain separation byte and hash OID internally in liboqs.
   • Expose variants like ML-DSA-44-SHA-512.
   • Pros: Doesn't expose the internal API.
   • Cons: More code complexity in liboqs for an algorithm variant not supported upstream.

3. Wait for potential future upstream support
   Lowest maintenance cost but delays functionality.

Discussion points

• Which prehash options should be added or prioritized: externalMu, HashML-DSA, or both?
• Which integration option is preferred (one outlined above, or others)?

[ashman-p]

@bess , Thanks for writing this up.
Regarding the "Discussion points": Priority my suggestion is for the order that you listed. externalMu, followed by HashML.

If we had a timeline for when MLDSA is native, that could help drive the decision.

[dstebila]

Pravek has started some drafts to integrate mldsa-native into liboqs.

• Draft integration into liboqs: Test mldsa-native integration #2222
• Draft of integration code to be added to mldsa-native to enable the integration: Add liboqs integration pq-code-package/mldsa-native#403

[ashman-p]

This Q&A writeup involving NIST may have some bearing on the decision we take to expose internal APIs for the purpose of supporting external Mu and prehashed variants.