Add module "musig" that implements MuSig2 multi-signatures (BIP 327)

[jonasnick]

EDIT: based on #1518. Closes #1452. Most of the code is a copy from libsecp256k1-zkp. The API added in this PR is identical with the exception of two modifications:

1. I removed the unused scratch_space argument from secp256k1_musig_pubkey_agg. This argument was intended to allow using ecmult_multi algorithms for key aggregation in the future. But at this point it's unclear whether the scratch_space object will remain in its current form (see Rework or get rid of scratch space  #1302).
2. Support for adaptor signatures was removed and therefore the adaptor argument of musig_nonce_process was also removed.

In contrast to the module in libsecp256k1-zkp, the module is non-experimental. I slightly cleaned up parts of the module, adjusted the code to the new definition of the VERIFY_CHECK macro and applied some simplifications that were possible because the module is now in the upstream repo (ge_from_bytes, ge_to_bytes). You can follow the changes I made to the libsecp256k1-zkp module at https://github.com/jonasnick/secp256k1-zkp/commits/musig2-upstream/.

[jonasnick]

Rebased on top of master to get #1480 which allowed dropping a commit. Old state is preserved at https://github.com/jonasnick/secp256k1/tree/musig2-module-backup.

[real-or-random]

needs rebase :)

[t-bast]

FWIW, we have JVM bindings on top of this branch in ACINQ/secp256k1-kmp#93 and an implementation of swap-in-potentiam (musig2 key-path with alternative delayed script path) in ACINQ/bitcoin-kmp#107 and everything is working fine, and the API is easy enough to use!

[jonasnick]

Rebased.

Thanks @t-bast, that's good to hear.

[siv2r]

Attaching a visualization for the API flow.

Edit: The above visualization is incorrect. I will update it with the correct one soon.

[siv2r]

Do we recommend that users sort their pubkeys before aggregating them? The musig_pubkey_agg API documentation simply says the user "can" do it.

If we recommend the sorting step, including it in the example file would be helpful.

[real-or-random]

I think there's no catch-all recommendation. BIP327 says "The aggregate public key produced by KeyAgg (regardless of the type) depends on the order of the individual public keys. If the application does not have a canonical order of the signers, the individual public keys can be sorted with the KeySort algorithm to ensure that the aggregate public key is independent of the order of signers."

In other words: If in your application, the collection of pubkeys (or signers represented by them) is conceptually an (ordered) list, then don't bother with sorting. If in your application, the collection of pubkeys is conceptually an (unordered) set, i.e., the application doesn't want to care about the order of pubkeys, then sort to make sure the set has a canonical serialization.

Perhaps we can explain this somewhere in more detail, either in the API docs or in the example.

[josibake]

nit: I think it would be nice to including sorting in the example for the following reasons:

• It provides a practical example on how to use the sorting API
• It's a good hook for adding a comment explaining what @real-or-random just explained above. Feels a bit nicer to have that comment in the example, vs the API docs

[jonasnick]

Added sorting and a comment.

[josibake]

I'm also using the heapsort commits in #1471. What do you think about splitting out the sort commits into their own PR? Also fine with cherry-picking for now, but figured I'd mention it since it might simplify both of our PRs.

[josibake]

in 26dde29 ("extrakeys: add secp256k1_pubkey_sort"):

Does it make sense to move this block into the secp256k1_sort function? I ended up copying these lines while writing a secp256k1_silentpayments_recipient_sort function, which made me realize anyone else would also need to copy these lines when writing a sort function for heapsort.

[jonasnick]

You mean into secp256k1_hsort? I'd guess the wrong warning is emitted when secp256k1_hsort is called and therefore it would be to late when the warning was disabled in secp256k1_hsort.

[theStack]

re-ACK 5e55f09

(as per $ git diff fdc09608036822afc1cebbe0c5b56cebf8ba508d 5e55f093118c2517f068841f5414f767fb35c7fe)

As far as I can tell, none of the unadressed questions/comments are critical.

[jonasnick]

@theStack

are there any musig API functions where the SECP256K1_WARN_UNUSED_RESULT attribute should be added?

That's a good point. I couldn't discover a consistent pattern we're currently following (e.g, secp256k1_ec_pubkey_parse has the warning, secp256k1_tagged_sha256 has the warning even though it always returns 1, and signature_parse_der, signature_parse_compact don't). I think this should be made more consistent treewide with a note to CONTRIBUTING.md.

There are different rules we could follow:

1. Just add the warning basically everywhere. Maybe that could be annoying sometimes if the user has data from a trusted source.
2. Add it to functions returning a boolean result that would be pointless to call without checking for the result (e.g., verify).
3. Add to all functions that can fail without calling a callback (e.g. they can fail because they are receiving data from untrusted sources -- at least in normal scenarios).
4. Add it to all functions you need to call before verify to make sure that verify doesn't succeed due to invalid values. However, this would be defense-in-depth because it shouldn't happen anyway.
5. Add it to all functions that operate on a seckey and produce a public output (e.g. sign) and all the functions the user needs to call before. This prevents that an invalid output (e.g., signature) accidentally leaks information about the seckey. However, users should already be protected from this because we zeroize the outputs if these functions fail.
6. Add it to functions which sound particularly scary (nonce generation).

I added it to a few more functions (following basically rules 2, 3, 4, 6), because removing the warning is backwards compatible.

[real-or-random]

reACK 352cd3b
nit: musig_example needs to be added to .gitignore

[real-or-random]

nit: you did this, but in the "wrong" commit

[jonasnick]

nit: musig_example needs to be added to .gitignore

fixed

[sipa]

reACK 168c920

[real-or-random]

reACK 168c920

[real-or-random]

We have an open issue for SECP256K1_WARN_UNUSED_RESULT, I copied @jonasnick's comment there.

[theStack]

re-ACK 168c920