feat: add non-persisted Async Payjoin support #200

mehmetefeumit · 2025-06-04T04:54:25Z

Description

This change implements the online commands for both sending and receiving a Payjoin. Note that this PR does not implement Payjoin persistence.

Notes to the reviewers

Tested on my local regtest with rpc using the OHTTP relays and directories in the payjoin-cli README. Feel free to do the same, and let me know if there are comments, documentation, etc. which should be changed with this. I was not able to find any...

Here are the sample command you can use to test on regtest:

Send

cargo run --features=rpc -- -n regtest wallet -a USERNAME:PASSWORD -w WALLET_NAME -e "DESCRIPTOR" send_payjoin --ohttp_relay "RELAY1" --ohttp_relay "RELAY2" --fee_rate 1 --uri "URI"

Receive

cargo run --features=rpc -- -n regtest wallet -a USERNAME:PASSWORD -w WALLET_NAME -e "DESCRIPTOR" receive_payjoin --max_fee_rate 1000 --directory "https://payjo.in"  --ohttp_relay "RELAY1" --ohttp_relay "RELAY2" --amount AMOUNT

Checklists

All Submissions:

I've signed all my commits
I followed the contribution guidelines
I ran cargo fmt and cargo clippy before committing

New Features:

[] I've added tests for the new feature <--- There is no testing suite for this.
I've added docs for the new feature
[N/A] I've updated CHANGELOG.md

Bugfixes:

This pull request breaks the existing API
I've added tests to reproduce the issue which are now passing
I'm linking the issue being fixed by this PR

src/handlers.rs

spacebear21

At a high level, V2 Payjoin should not fallback to V1. We should be able to tell whether the receiver is a v1 or v2 receiver based on the pj param in the BIP21 uri and use the appropriate version. If v2 send cannot happen due to ohttp_relay not being provided or some other issue, it should simply abort and print an error message. Generally bdk-cli should reproduce the behavior of the payjoin-cli reference implementation.

Persistence

Note that this change does not implement persistence.

I made the decision myself due to the UX complexity of adding the resume functionality. bdk-cli only has two layers of commands. For example, you can implement bitcoin-cli wallet send_payjoin, but you cannot do bitcoin-cli wallet send_payjoin resume. So there is a UX/interface decision to be made on how the resume would be implemented.

Here too we can look at payjoin-cli for guidance. There are three commands: send, receive and resume. The resume command resumes all pending send sessions and receive sessions in parallel.

That being said, it would be fine for this PR to only implement send functionality and add receive/resume as follow-ups. Especially as payjoin persistence is undergoing a major overhaul with the Session Event Log, persistence can be left as a NoOp for now.

src/handlers.rs

spacebear21 · 2025-06-04T17:01:49Z

src/commands.rs

+        fee_rate: u64,
+        /// URL of the OHTTP relay.
+        #[arg(env = "PAYJOIN_OHTTP_RELAY", long = "ohttp_relay")]
+        ohttp_relay: Option<String>,


payjoin-cli uses a list of ohttp relays so that it can fallback to another relay in case one is not working, it may be useful to follow the same pattern for bdk-cli instead of an Option.

spacebear21 · 2025-06-04T17:35:07Z

src/utils.rs

+#[cfg(any(
+    feature = "electrum",
+    feature = "esplora",
+    feature = "cbf",
+    feature = "rpc"
+))]


This feature gate seems unnecessary for this function

spacebear21 · 2025-06-04T17:52:12Z

src/handlers.rs

+            // We need to re-introduce the UTXO information of the original PSBT's inputs
+            // back into the Payjoin proposal we received from the receiver. The receiver strips
+            // the transaction of the information, so we need to add the `..._utxo` information back
+            // before we can re-sign our input(s) into the transaction.
+            let mut original_inputs_iter = input_pairs(&mut original_psbt).peekable();
+            for (proposal_txin, proposal_psbt_input) in input_pairs(&mut psbt) {
+                if let Some((orig_txin, orig_psbt_input)) = original_inputs_iter.peek() {
+                    if proposal_txin.previous_output == orig_txin.previous_output {
+                        proposal_psbt_input.witness_utxo = orig_psbt_input.witness_utxo.clone();
+                        proposal_psbt_input.non_witness_utxo =
+                            orig_psbt_input.non_witness_utxo.clone();
+                        original_inputs_iter.next();
+                    }
+                }
+            }


I don't think it's true that the receiver strips the utxo information? See here.

The only thing that gets stripped are the sender signatures afaict.

I'm not sure that a receiver necessarily strips it but I'm not sure you can depend on them not being stripped. Since BDK needs the info to find the keys associated with a script, you might want to add it back in as is done here.

coveralls · 2025-06-07T05:15:57Z

Pull Request Test Coverage Report for Build 15434070095

Details

0 of 231 (0.0%) changed or added relevant lines in 3 files are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage decreased (-0.4%) to 2.291%

Changes Missing Coverage	Changed/Added Lines	%
src/commands.rs	2	0.0%
src/utils.rs	7	0.0%
src/handlers.rs	222	0.0%

Files with Coverage Reduction	New Missed Lines	%
src/handlers.rs	1	3.04%

Totals
Change from base Build 15422078682:	-0.4%
Covered Lines:	25
Relevant Lines:	1091

💛 - Coveralls

mehmetefeumit · 2025-06-11T18:08:41Z

@spacebear21 Made all the changes regarding multiple OHTTP relays, no fallback, and commit organization. I was not able to find a built-in method for determining whether a URI is for Payjoin v1 or v2, so just looking for a '#' at the moment to differentiate, which is hacky.

I'll probably create an issue and work on that function if PDK does not have a way to tell the version of a URI at the moment.

DanGould · 2025-06-14T21:48:34Z

Cargo.toml

 thiserror = "2.0.11"
 tokio = { version = "1", features = ["full"] }
+payjoin = { version = "0.23.0", features = ["v1", "v2", "io"] }
+minreq = { version = "2.13.2", features = ["https"] }


If you're using the io feature, you're already depending on reqwest, so minreq is a second HTTP client. If you want to use minreq to reduce the dependency burden I recommend also writing the payjoin/io feature functions using minreq.

DanGould

I suppose this might work, but have specific concerns about how v1/v2 URIs are distinguished, which you brought up. I am going to propose a fix upstream to make this aspect more ergonomic, but for now, there are certainly improvements in the abstraction presented here, specifically separating the ohttp-relay selection logic from the payjoin protocol logic.

src/handlers.rs

src/utils.rs

src/handlers.rs

Cargo.toml

mehmetefeumit · 2025-06-24T21:48:56Z

@DanGould Made the changes in the comments. Major changes are (1) checking and determining a healthy OHTTP relay before we go ahead with everything else, (2) no failover from v2 to v1 based on any random failure, and instead using the v2 extraction to determine whether we should do one or the other.

Also added some additional error messaging for the user to understand why the cli may have failed over to v1 (no relays, URI being v1, etc.). Thanks for the catches and the recommendations.

mehmetefeumit · 2025-06-24T21:52:18Z

src/handlers.rs

+                async {
+                    for relay in ohttp_relays {
+                        if let Ok(_) =
+                            payjoin::io::fetch_ohttp_keys(relay.clone(), req_ctx.endpoint()).await


This is similar to what payjoin-cli is doing here. The main difference is that we are not really using the keys here since we are not storing them anywhere. Currently, the entire implementation is stateless -- there is no resume functionality, and we do not store information on the OHTTP relays either. So we're just using the fetch... function here to determine if a OHTTP relay passed in the arguments is healthy. And then scrap the keys, and leave everything to the extract_v2 call.

`{v1,v2,multiparty}::Sender{,Builder}` types have been built off of a heirarchy where multiparty:: has a v2:: and v2:: has a v1::, which was done for quick convenience and not because that relationship actually makes functional sense. This PR is the first in a few which I believe are necessary to rectify this distinction. It does so by drawing the separation between the sender's `PsbtContext` checks / response processing and the version-specific serialization for networked messaging. However, I don't think it goes far enough. For one, it only really rectifies this issue between v1 and v2. Multiparty is left abstract over v2. Second, There are still distinct SenderBuilders that can't tell whether or not they're handling a v1 or v2 URIs. Since the information necessary to distinguish between a v1/v2 URI is in the URI itself, it seems that ought to be the first order of business for the sender to do even before calling `SenderBuilder::new`. The lack of this distinction leads to a [problem](bitcoindevkit/bdk-cli#200 (comment)) with a hacky [solution](bitcoindevkit/bdk-cli#200 (comment)) where downstream users need to wait all the way until they attempt to create a v2 request and handle an error there in order to figure out the version. The `SenderBuilder` also ought to behave differently for each version, and I'm not sure our current fix of #847 does this completely (Does a v2 SenderBuilder sending to v1 URI honor pjos? it should). In order to do so I reckon we could create an actual `PjUri` type, rather than an alias, that enumerates over the versions when `check_pj_supported` or its replacement is called. In order to do *that* effectively by making sure the correct parameters are there and we're not just switching on the presence of a fragment, I think `UrlExt` also needs to check for the parameter presence and validity. The other issue with v1::Sender flow is that it doesn't use the generic typestate machine pattern to match v2, which would be nice as well but out of the scope of this PR. re: #809

DanGould · 2025-10-11T20:46:00Z

My concerns about distinguishing URIs and many other issues are fixed by the new payjoin-1.0.0-rc.0 release, it might be time for this PR to shine again

The reference payjoin-cli tool now features relay failover mechanisms which can be copied.

mehmetefeumit · 2025-10-11T22:07:21Z

My concerns about distinguishing URIs and many other issues are fixed by the new payjoin-1.0.0-rc.0 release, it might be time for this PR to shine again

The reference payjoin-cli tool now features relay failover mechanisms which can be copied.

Sounds good! I'll take a look.

This is a pre-requisite for adding Payjoin support. When the receiver sends the Payjoin proposal to the sender to be broadcasted, they need to sync the blockchain before checking if the Payjoin has indeed been broadcasted. To do that, the sync function will need to be shared between the two online commands.

Prior to the Payjoin integration, we need to have the broadcast logic outside the broadcast command so that it can be shared between the existing online command and the Payjoin sender.

Mshehu5

Awesome work Efe!

While testing this, I kept running into the error: "Failed to sign and finalize the Payjoin proposal PSBT." on the receiver side from what I observed, the issue comes from the wallet.sign() call inside the finalize_proposal function. The if statement expects a true value but it always evaluates to false because it can’t sign the sender’s input in the PSBT. This results in an error.

Once I removed that check, the Payjoin flow worked. We might want to consider a better or more appropriate way to perform this validation.

Feel free to test or confirm this I might be wrong and it could be something on my end.

src/payjoin/mod.rs

mehmetefeumit · 2025-12-01T00:22:09Z

Awesome work Efe!

While testing this, I kept running into the error: "Failed to sign and finalize the Payjoin proposal PSBT." on the receiver side from what I observed, the issue comes from the wallet.sign() call inside the finalize_proposal function. The if statement expects a true value but it always evaluates to false because it can’t sign the sender’s input in the PSBT. This results in an error.

Once I removed that check, the Payjoin flow worked. We might want to consider a better or more appropriate way to perform this validation.

Feel free to test or confirm this I might be wrong and it could be something on my end.

You are right. I was cleaning things up after making the receiver work, and I think I incorrectly left this. Thanks for the catch! Your comment is address on the PR now.

spacebear21

utACK 42eca12, this looks really good! I have a few minor comments but the core logic is sound and the commit history is clean and easy to read. It would be nice to follow up with persistence support for a more fully-featured integration, but this PR is a great start.

Cargo.toml

spacebear21 · 2025-12-02T17:30:06Z

Cargo.toml

+rpc = ["bdk_bitcoind_rpc", "_payjoin-dependencies"] 
+
+# Internal features
+_payjoin-dependencies = ["payjoin", "reqwest", "url"]


I believe the url dependency was removed from the public payjoin API for 1.0.0-rc, so it should no longer be required.

I am changing the function signatures to use payjoin::IntoUrl, but going to hold on to the Url dependency to parse the URLs before passing them to the Payjoin process functions. Let me know if there is an alternative approach 👍🏻

spacebear21 · 2025-12-02T17:30:49Z

src/commands.rs

+        /// URL of the Payjoin OHTTP relay. Can be repeated multiple times to attempt the
+        /// operation with multiple relays for redundancy.
+        #[arg(env = "PAYJOIN_OHTTP_RELAY", long = "ohttp_relay", required = true)]
+        ohttp_relay: Vec<String>,


nit: plural ohttp_relays

The reason I kept this singular here is that it works better from a CLI options perspective:

send_payjoin --fee_rate 1 --ohttp_relay "https://pj.bobspacebkk.com" --ohttp_relay "https://ohttp.achow101.com" --uri "..."

Having the options be --ohttp_relays would not be accurate. And to my knowledge there isn't another way to past that list in the command line.

Fair enough, I guess I was thinking about it from the perspective of a config file but I realize now this only supports command line argument parsing so it makes sense.

spacebear21 · 2025-12-02T17:33:26Z

src/commands.rs

+        /// URL of the Payjoin OHTTP relay. Can be repeated multiple times to attempt the
+        /// operation with multiple relays for redundancy.
+        #[arg(env = "PAYJOIN_OHTTP_RELAY", long = "ohttp_relay", required = true)]
+        ohttp_relay: Vec<String>,


nit: also plural here

Same comment as the above.

The reason I kept this singular here is that it works better from a CLI options perspective:

send_payjoin --fee_rate 1 --ohttp_relay "https://pj.bobspacebkk.com" --ohttp_relay "https://ohttp.achow101.com" --uri "..."

Having the options be --ohttp_relays would not be accurate. And to my knowledge there isn't another way to past that list in the command line.

src/payjoin/mod.rs

Mshehu5

Instead of limiting the mempool/blockchain check to a single sync, would a feature flag make sense allowing resync for all clients except Kyoto? As your comments stated and what I’ve seen, Kyoto is the only BlockchainClient that consumes the client while the others use reference based syncing (and even the BDK CLI defaults to Electrum).
This seems better than enforcing a one sync approach across all clients until a proper solution is available.
What do you think?

Mshehu5 · 2025-12-02T23:32:07Z

src/payjoin/mod.rs

+    /// Syncs the blockchain once and then checks whether the Payjoin was broadcasted by the
+    /// sender.
+    ///
+    /// The currenty implementation does not support checking for the PAyjoin broadcast in a loop
+    /// and returning only when it is detected or if a timeout is reached because the [`sync_wallet`]
+    /// function consumes the BlockchainClient. BDK CLI supports multiple blockchain clients, and
+    /// at the time of writing, Kyoto consumes the client since BDK CLI is not designed for long-running
+    /// tasks.


Is resyncing on other BlockchainClient options possible?
From the comments only kyoto is the consumes the client. Electrum, esplora and rpc are available and don't consume the client

See my comment below on the main discussion.

src/payjoin/mod.rs

mehmetefeumit · 2025-12-03T16:44:06Z

Instead of limiting the mempool/blockchain check to a single sync, would a feature flag make sense allowing resync for all clients except Kyoto? As your comments stated and what I’ve seen, Kyoto is the only BlockchainClient that consumes the client while the others use reference based syncing (and even the BDK CLI defaults to Electrum). This seems better than enforcing a one sync approach across all clients until a proper solution is available. What do you think?

Based on my understanding of the usage, using multiple blockchain client features is not an intended usage pattern. Online commands like broadcast or sync have an order in which they will match the client with the type, and this order is not based on anything and assumes that only a single blockchain feature is activated (@tvpeter Can you correct me if I am wrong on this? Also, your input for below would be highly appreciated)

If my assumption is true, this would eliminate the possibility of switching between different client types and add additional complexity. For example, what happens when only the Kyoto client is an active feature?

If we have different syncing behaviors for different clients, this creates a UX fork where users who are using the Kyoto client will get a difference experience than users using all of the other clients. I'd rather prefer consistency here since from a UX perspective, Payjoin should work the same no matter which client you want to use.

Instead, we can explore whether we can make Kyoto non-consuming in some way(?), but until and if that happens, I'd prefer keeping consistency between clients and have each Payjoin operation require a separate sync if the session takes longer than a single CLI command execution.

temp: move the send_payjoin out of the commit

src/payjoin/mod.rs

github-project-automation bot added this to BDK-CLI Jun 4, 2025

mehmetefeumit force-pushed the payjoin-send branch from 2d54a78 to 6932c53 Compare June 4, 2025 05:04

mehmetefeumit mentioned this pull request Jun 4, 2025

wip(payjoin-send): failing to sign the original PSBT #191

Closed

9 tasks

mehmetefeumit commented Jun 4, 2025

View reviewed changes

src/handlers.rs Show resolved Hide resolved

spacebear21 suggested changes Jun 4, 2025

View reviewed changes

mehmetefeumit force-pushed the payjoin-send branch 2 times, most recently from 89481cb to c02b679 Compare June 11, 2025 17:28

mehmetefeumit requested a review from spacebear21 June 11, 2025 18:08

DanGould reviewed Jun 14, 2025

View reviewed changes

Cargo.toml Outdated Show resolved Hide resolved

mehmetefeumit force-pushed the payjoin-send branch from c02b679 to d8f821d Compare June 24, 2025 21:44

mehmetefeumit commented Jun 24, 2025

View reviewed changes

mehmetefeumit force-pushed the payjoin-send branch from d8f821d to dd6a582 Compare June 24, 2025 21:53

DanGould mentioned this pull request Jul 16, 2025

Abstract common sender functionality, not heirarchy payjoin/rust-payjoin#884

Merged

mehmetefeumit force-pushed the payjoin-send branch from dd6a582 to cb8d0d5 Compare November 3, 2025 15:56

mehmetefeumit changed the title ~~feat: add Payjoin v2 send online command with fallback to v1~~ feat: add Payjoin v2 send online command Nov 3, 2025

mehmetefeumit marked this pull request as draft November 5, 2025 03:45

mehmetefeumit added 2 commits November 18, 2025 22:14

refactor: move broadcast to a helper function

f62287a

Prior to the Payjoin integration, we need to have the broadcast logic outside the broadcast command so that it can be shared between the existing online command and the Payjoin sender.

mehmetefeumit force-pushed the payjoin-send branch 2 times, most recently from 76ce4ba to 004bf4d Compare November 19, 2025 06:45

mehmetefeumit changed the title ~~feat: add Payjoin v2 send online command~~ feat: add non-persisted Async Payjoin support Nov 19, 2025

mehmetefeumit marked this pull request as ready for review November 19, 2025 06:50

Mshehu5 reviewed Nov 29, 2025

View reviewed changes

src/payjoin/mod.rs Outdated Show resolved Hide resolved

mehmetefeumit force-pushed the payjoin-send branch from 2af219c to 42eca12 Compare December 1, 2025 00:16

spacebear21 reviewed Dec 2, 2025

View reviewed changes

Mshehu5 reviewed Dec 3, 2025

View reviewed changes

feat: add payjoin send support

594dbcf

temp: move the send_payjoin out of the commit

mehmetefeumit force-pushed the payjoin-send branch from 42eca12 to bfd7162 Compare December 3, 2025 16:49

arminsabouri reviewed Dec 4, 2025

View reviewed changes

src/payjoin/mod.rs Outdated Show resolved Hide resolved

mehmetefeumit added 2 commits December 4, 2025 21:35

feat: add payjoin receive support

a48ca0a

docs: add Payjoin capability to the README

f14459c

mehmetefeumit force-pushed the payjoin-send branch from bfd7162 to f14459c Compare December 5, 2025 05:35

feat: add non-persisted Async Payjoin support #200

Are you sure you want to change the base?

feat: add non-persisted Async Payjoin support #200

Uh oh!

Conversation

mehmetefeumit commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Notes to the reviewers

Checklists

All Submissions:

New Features:

Bugfixes:

Uh oh!

Uh oh!

spacebear21 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

coveralls commented Jun 7, 2025

Pull Request Test Coverage Report for Build 15434070095

Details

💛 - Coveralls

Uh oh!

mehmetefeumit commented Jun 11, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

DanGould left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mehmetefeumit commented Jun 24, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

DanGould commented Oct 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mehmetefeumit commented Oct 11, 2025

Uh oh!

Mshehu5 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mehmetefeumit commented Dec 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

spacebear21 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mehmetefeumit commented Jun 4, 2025 •

edited

Loading

DanGould commented Oct 11, 2025 •

edited

Loading

Mshehu5 left a comment •

edited

Loading

mehmetefeumit commented Dec 1, 2025 •

edited

Loading