From b7cd7ccf76a8c8f70869045a1d15060545ab5da4 Mon Sep 17 00:00:00 2001
From: Tom Briar <tombriar08@Gmail.com>
Date: Fri, 20 Oct 2023 11:40:04 -0400
Subject: [PATCH 1/8] doc: Added Compressed Transaction Schema

---
 doc/compressed_transactions.md | 183 +++++++++++++++++++++++++++++++++
 1 file changed, 183 insertions(+)
 create mode 100644 doc/compressed_transactions.md

diff --git a/doc/compressed_transactions.md b/doc/compressed_transactions.md
new file mode 100644
index 0000000000000..c83651ce87eb7
--- /dev/null
+++ b/doc/compressed_transactions.md
@@ -0,0 +1,183 @@
+# Compressed Transaction Schema
+By (Tom Briar) and (Andrew Poelstra)
+
+## 1. Abstract
+
+With this Transaction Compression Schema we use several methods to compress transactions,
+including dropping data and recovering it on decompression by grinding until we obtain
+valid signatures.
+
+The bulk of our size savings come from replacing the prevout of each input by a block
+height and index. This requires the decompression to have access to the blockchain, and
+also means that compression is ineffective for transactions that spend unconfirmed or
+insufficiently confirmed outputs.
+
+Even without compression, Taproot keyspends are very small: as witness data they
+include only a single 64/65-byte signature and do not repeat the public key or
+any other metadata. By using pubkey recovery, we obtain Taproot-like compression
+for legacy and Segwit transactions.
+
+The main applications for this schema are for steganography, satellite/radio broadcast, and
+other low bandwidth channels with a high CPU availability on decompression. We
+assume users have some ability to shape their transactions to improve their
+compressibility, and therefore give special treatment to certain transaction forms.
+
+This schema is easily reversible except for compressing the Txid/Vout input pairs(Method 4). 
+Compressing the input Txid/Vout is optional, and without it still gleans 50% of the total compression. 
+This allows for the additional use case of P2P communication.
+
+## 2. Methods
+
+The four main methods to achieve a lower transactions size are:
+
+1. packing transaction metadata before the transaction and each of its inputs and
+outputs to determine the structure of the following data.
+2. replacing 32-bit numeric values with either variable-length integers (VarInts) or compact-integers (CompactSizes).
+3. using compressed signatures and public key recovery upon decompression.
+4. replacing the 36-byte txid/vout pair with a blockheight and output index.
+
+Method 4 will cause the compressed transaction to be undecompressable if a block
+reorg occurs at or before the block it's included in. Therefore, we'll only compress
+the Txid if the transaction input is at least one hundred blocks old.
+
+
+## 3 Schema
+
+### 3.1 Primitives
+
+| Name             | Width     | Description |
+|------------------|-----------|-------------|
+| CompactSize      | 1-5 Bytes | For 0-253, encode the value directly in one byte. For 254-65535, encode 254 followed by 2 little-endian bytes. For 65536-(2^32-1), encode 255 followed by 4 little-endian bytes. |
+| CompactSize flag | 2 Bits    | 1, 2 or 3 indicate literal values. 0 indicates that the value will be encoded in a later CompactInt. |
+| VarInt           | 1+ Bytes  | 7-bit little-endian encoding, with each 7-bit word encoded in a byte. The highest bit of each byte is 1 if more bytes follow, and 0 for the last byte. |
+| VLP-Bytestream   | 2+ Bytes  | A VarInt Length Prefixed Bytestream. Has a VarInt prefixed to determine the length. |
+
+### 3.2 General Schema
+
+| Name                           | Width           | Description |
+|--------------------------------|-----------------|-------------|
+| Transaction Metadata           | 1 Byte    | Information on the structure of the transaction. See Section 3.3. |
+| Version                        | 0-5 Bytes | An optional CompactSize containing the transactions version. |
+| Input Count                    | 0-5 Bytes | An optional CompactSize containing the transactions input count. |
+| Output Count                   | 0-5 Bytes | An optional CompactSize containing the transactions output count. |
+| LockTime                       | 0-5 Bytes | An optional CompactSize containing the transaction LockTime if its non zero. |
+| Minimum Blockheight            | 1-5 Bytes | A VarInt containing the Minimum Blockheight of which the transaction locktime and input blockheights are given as offsets. |
+| Input Metadata+Output Metadata | 1+ Bytes  | A Encoding containing metadata on all the inputs and then all the outputs of the transaction. For each input see Section 3.4, for each output see Section 3.5. |
+| Input Data                     | 66+ Bytes | See Section 3.6 for each input. |
+| Output Data                    | 3+ Bytes  | See Section 3.7 for each output. |
+
+For the four CompactSize listed above we could use a more compact bit encoding for these but they are already a fall back for the bit encoding of the Transaction Metadata.
+
+### 3.3 Transaction Metadata
+
+| Name         | Width  | Description |
+|--------------|--------|-------------|
+| Version      | 2 Bits | A CompactSize flag for the transaction version. |
+| Input Count  | 2 Bits | A CompactSize flag for the transaction input count. |
+| Output Count | 2 Bits | A CompactSize flag for the transaction output count. |
+| LockTime     | 1 Bit  | A Boolean to indicate if the transaction has a LockTime. |
+
+### 3.4 Input Metadata
+
+| Name                 | Width  | Description |
+|----------------------|--------|-------------|
+| Compressed Signature | 1 Bit  | Signature compression flag. For P2TR: 1 for keyspend, 0 for scriptspend; For P2SH: 0 for p2sh, 1 for p2sh-wpkh. |
+| Standard Hash        | 1 Bit  | A flag to determine if this Input's Signature Hash Type is standard (0x00 for Taproot, 0x01 for Legacy/Segwit). |
+| Standard Sequence    | 2 Bits | A CompactSize flag for the inputs sequence. Encode literal values as follows: 1 = 0x00000000, 2 = 0xFFFFFFFE, 3 = 0xFFFFFFFF. |
+
+### 3.5.1 Output Metadata
+
+| Name                | Width  | Description |
+|---------------------|--------|-------------|
+| Encoded Script Type | 3 Bits | Encoded Script Type. |
+
+#### 3.5.2 Script Type encoding
+
+| Script Type                | Value |
+|----------------------------|-------|
+| Uncompressed P2PK          | 0b000 |
+| Compressed P2PK            | 0b001 |
+| P2PKH                      | 0b010 |
+| P2SH                       | 0b011 |
+| P2WSH                      | 0b100 |
+| P2WPKH                     | 0b101 |
+| P2TR                       | 0b110 |
+| Uncompressed Custom Script | 0b111 |
+
+### 3.6 Input Data
+
+| Name                    | Width     | Description |
+|-------------------------|-----------|-------------|
+| Sequence                | 0-5 Bytes | An Optional VarInt containing the sequence if it was non-standard. |
+| Txid Blockheight        | 1-5 Bytes | A VarInt Either containing 0 if this an uncompressed input, or it contains the offset from Minimum Blockheight for this Txid. |
+| Txid/Signature Data     | 65+ Bytes | Txid/Signatures are determined to be uncompressed either by the output script of the previous transaction, or if the Txid Blockheight is zero. For each Compressed Txid/Signature See Section 3.6.1. For each Uncompressed Txid/Signature See Section 3.6.2. |
+
+### 3.6.1 Compressed Txid/Signature Data
+
+| Name              | Width     | Description |
+|-------------------|-----------|-------------|
+| Txid Block Index  | 1-5 Bytes | A VarInt containing the flattened index from the Txid Blockheight for the Vout. |
+| Signature         | 64 Bytes  | Contains the 64 byte signature. |
+| Hash Type         | 0-1 Bytes | An Optional Byte containing the Hash Type if it was non-standard.|
+
+### 3.6.2 Uncompressed Txid/Signature Data
+
+| Name      | Width     | Description |
+|-----------|-----------|-------------|
+| Txid      | 32 Bytes  | Contains the 32 byte Txid. |
+| Vout      | 1-5 Bytes | A CompactSize Containing the Vout of the Txid. |
+| Signature | 2+ Bytes  | A VLP-Bytestream containing the signature. |
+
+### 3.7 Output Data
+
+| Name          | Width     | Description |
+|---------------|-----------|-------------|
+| Output Script | 2+ Bytes  | A VLP-Bytestream containing the output script. |
+| Amount        | 1-9 Bytes | A VarInt containing the output amount. |
+
+## 4 Ideal Transaction
+
+The target transaction for the most optimal compression was chosen
+based off the most common transactions that are likely to be used
+for purposes that requires the best compression.
+
+| Field           | Requirements                      | Possible Savings                  |
+|-----------------|-----------------------------------|-----------------------------------|
+| Version         | Less than four                    | 30 Bits                           |
+| Input Count     | Less then four                    | 30 Bits                           |
+| Output Count    | Less then four                    | 30 Bits                           |
+| LockTime        | 0                                 | 30 Bits                           |
+| Input Sequence  | 0x00, 0xFFFFFFFE, or 0xFFFFFFFF   | 62 Bits For Each Input            |
+| Input Txid      | Compressed Outpoint               | 23-31 Bytes For Each Input        |
+| Input Vout      | Compressed Outpoint               | (-1)-3 Bytes For Each Input       |
+| Input Signature | Non-custom Script Signing         | 40-72 Bytes For Each Legacy Input |
+| Input Hash Type | 0x00 for Taproot, 0x01 for Legacy | 7 Bits For Each Input             |
+| Output Script   | Non-custom Scripts                | 2-5 Bytes For Each Output         |
+| Output Amount   | No Restrictions                   | (-1)-7 Bytes For Each Output      |
+
+## 5 Test Vectors
+
+| Transaction              | Before Compression | Possible Savings         | After Compression |
+|--------------------------|--------------------|--------------------------|-------------------|
+| 2-(input/output) Taproot | 312 Bytes          | 78-124 Bytes and 2 Bits  | 190-226 Bytes     |
+| 2-(input/output) Legacy  | 394 Bytes          | 118-196 Bytes and 2 Bits | 176-244 Bytes     |
+
+Taproot (Uncompressed)
+```
+020000000001028899af77861ede1ee384c333974722c96eabba8889506725b00735fc35ba41680000000000000000008899af77861ede1ee384c333974722c96eabba8889506725b00735fc35ba41680000000000000000000288130000000000002251206b10142cffb29e9d83f63a77a428be41f96bd9b6ccc9889e4ec74927058b41dda00f000000000000225120dd00ac641dc0f399e62a6ed6300aba1ec5fa4b3aeedf1717901e0d49d980efd20140f3d9bcc844eab7055a168a62f65b8625e3853fad8f834d5c82fdf23100b7b871cf48c2c956e7d76cdd367bbfefe496c426e64dcfeaef800ab9893142050714b6014081c15fe5ed6b8a0c0509e871dfbb7784ddb22dd33b47f3ad1a3b271d29acfe76b5152b53ed29a7f6ea27cb4f5882064da07e8430aacafab89a334b32780fcb2700000000
+```
+
+Taproot (Compressed)
+```
+2a81de3177d8019c2ef3d9bcc844eab7055a168a62f65b8625e3853fad8f834d5c82fdf23100b7b871cf48c2c956e7d76cdd367bbfefe496c426e64dcfeaef800ab9893142050714b6019c2e81c15fe5ed6b8a0c0509e871dfbb7784ddb22dd33b47f3ad1a3b271d29acfe76b5152b53ed29a7f6ea27cb4f5882064da07e8430aacafab89a334b32780fcb276b10142cffb29e9d83f63a77a428be41f96bd9b6ccc9889e4ec74927058b41dd8827dd00ac641dc0f399e62a6ed6300aba1ec5fa4b3aeedf1717901e0d49d980efd2a01f
+```
+
+Legacy (Uncompressed)
+```
+02000000000102c583fe4f934a0ed87e4d082cd52967cc774b943fbb2e21378ec18b926b8dc549000000000000000000c583fe4f934a0ed87e4d082cd52967cc774b943fbb2e21378ec18b926b8dc5490000000000000000000288130000000000002251206b10142cffb29e9d83f63a77a428be41f96bd9b6ccc9889e4ec74927058b41dda00f000000000000225120dd00ac641dc0f399e62a6ed6300aba1ec5fa4b3aeedf1717901e0d49d980efd202473044022000d1c81efcf6d20d87253749bcef8bf1be7ba51ccdf7a3b328174ea874226c3c02202d810c20f92d49c821eaa6e3a9ec7d764e0e71006e572d6ea96b631bd921767c0121037833d05665f3b21c479583ee12c6c573d1f25977dedfae12c70c18ec9dd4618702473044022000d1c81efcf6d20d87253749bcef8bf1be7ba51ccdf7a3b328174ea874226c3c02202d810c20f92d49c821eaa6e3a9ec7d764e0e71006e572d6ea96b631bd921767c0121037833d05665f3b21c479583ee12c6c573d1f25977dedfae12c70c18ec9dd4618700000000
+```
+
+Legacy (Compressed)
+```
+2ad1e53044d801ae276c0002473044022000d1c81efcf6d20d87253749bcef8bf1be7ba51ccdf7a3b328174ea874226c3c02202d810c20f92d49c821eaa6e3a9ec7d764e0e71006e572d6ea96b631bd921767c0121037833d05665f3b21c479583ee12c6c573d1f25977dedfae12c70c18ec9dd461870001ae276c0002473044022000d1c81efcf6d20d87253749bcef8bf1be7ba51ccdf7a3b328174ea874226c3c02202d810c20f92d49c821eaa6e3a9ec7d764e0e71006e572d6ea96b631bd921767c0121037833d05665f3b21c479583ee12c6c573d1f25977dedfae12c70c18ec9dd46187006b10142cffb29e9d83f63a77a428be41f96bd9b6ccc9889e4ec74927058b41dd8827dd00ac641dc0f399e62a6ed6300aba1ec5fa4b3aeedf1717901e0d49d980efd2a01f
+```

From 521848daeeecaeaaf823f31462e38f17bbecaf6f Mon Sep 17 00:00:00 2001
From: Tom Briar <tombriar08@Gmail.com>
Date: Fri, 20 Oct 2023 11:29:58 -0400
Subject: [PATCH 2/8] util: Added a variable length bitstream encoder

---
 src/util/strencodings.h | 42 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)

diff --git a/src/util/strencodings.h b/src/util/strencodings.h
index 439678c24a132..69bedddacdc7f 100644
--- a/src/util/strencodings.h
+++ b/src/util/strencodings.h
@@ -300,6 +300,48 @@ bool ConvertBits(O outfn, It it, It end, I infn = {}) {
     return true;
 }
 
+/** Convert from a set of power-of-2 number base to another set. */
+template<bool pad, typename O, typename It, typename I = IntIdentity>
+bool ConvertBitsVariable(const Span<int>& frombitsvec, const Span<int>& tobitsvec, O outfn, It it, It end, I infn = {}) {
+    size_t fromindex = 0;
+    size_t toindex = 0;
+    size_t acc = 0;
+    size_t bits = 0;
+    auto frombits = fromindex >= frombitsvec.size() ? frombitsvec[frombitsvec.size()-1] : frombitsvec[fromindex];
+    auto tobits = toindex >= tobitsvec.size() ? tobitsvec[tobitsvec.size()-1] : tobitsvec[toindex];
+    auto maxv = (1 << tobits) - 1;
+    auto max_acc = (1 << (frombits + tobits - 1)) - 1;
+
+    while (it != end) {
+       int v = infn(*it);
+        if (v < 0) return false;
+        acc = ((acc << frombits) | v) & max_acc;
+        bits += frombits;
+        while (bits >= (size_t)tobits) {
+            bits -= tobits;
+            outfn((acc >> bits) & maxv);
+            ++toindex;
+            tobits = toindex >= tobitsvec.size() ? tobitsvec[tobitsvec.size()-1] : tobitsvec[toindex];
+            maxv = (1 << tobits) - 1;
+            max_acc = (1 << (frombits + tobits - 1)) - 1;
+        }
+        ++it;
+        ++fromindex;
+        frombits = fromindex >= frombitsvec.size() ? frombitsvec[frombitsvec.size()-1] : frombitsvec[fromindex];
+        max_acc = (1 << (frombits + tobits - 1)) - 1;
+    }
+    frombits = fromindex >= frombitsvec.size() ? frombitsvec[frombitsvec.size()-1] : frombitsvec[fromindex];
+    tobits = toindex >= tobitsvec.size() ? tobitsvec[tobitsvec.size()-1] : tobitsvec[toindex];
+    maxv = (1 << tobits) - 1;
+    if (pad) {
+        if (bits) outfn((acc << (tobits - bits)) & maxv);
+    } else if (bits >= (size_t)frombits || ((acc << (tobits - bits)) & maxv)) {
+        return false;
+    }
+    if (fromindex < frombitsvec.size()-1 || toindex < tobitsvec.size()-1) return false;
+    return true;
+}
+
 /**
  * Converts the given character to its lowercase equivalent.
  * This function is locale independent. It only converts uppercase

From a30c5104e4ce718ab6e15a0fa8bd5b337d149f64 Mon Sep 17 00:00:00 2001
From: Tom Briar <tombriar08@Gmail.com>
Date: Fri, 20 Oct 2023 11:30:18 -0400
Subject: [PATCH 3/8] script: Added the rest of the IsPayTo functions

---
 src/script/script.cpp | 40 ++++++++++++++++++++++++++++++++++++++--
 src/script/script.h   |  4 ++++
 2 files changed, 42 insertions(+), 2 deletions(-)

diff --git a/src/script/script.cpp b/src/script/script.cpp
index 80e8d26bcfb97..5c30964abf059 100644
--- a/src/script/script.cpp
+++ b/src/script/script.cpp
@@ -204,20 +204,56 @@ unsigned int CScript::GetSigOpCount(const CScript& scriptSig) const
     return subscript.GetSigOpCount(true);
 }
 
+bool CScript::IsPayToPublicKey() const
+{
+    // Test for pay-to-public-key CScripts:
+    return ((this->size() == 0x43 && (*this)[0] == 0x41 && (*this)[66] == OP_CHECKSIG) ||
+            (this->size() == 0x23 && (*this)[0] == 0x21 && (*this)[34] == OP_CHECKSIG));
+}
+
 bool CScript::IsPayToScriptHash() const
 {
     // Extra-fast test for pay-to-script-hash CScripts:
-    return (this->size() == 23 &&
+    return (this->size() == 0x17 &&
             (*this)[0] == OP_HASH160 &&
             (*this)[1] == 0x14 &&
             (*this)[22] == OP_EQUAL);
 }
 
+bool CScript::IsPayToPublicKeyHash() const
+{
+    // Test for pay-to-public-key-hash CScripts:
+    return (this->size() == 0x19 &&
+            (*this)[0] == OP_DUP &&
+            (*this)[1] == OP_HASH160 &&
+            (*this)[2] == 0x14 &&
+            (*this)[23] == OP_EQUALVERIFY &&
+            (*this)[24] == OP_CHECKSIG);
+}
+
+
 bool CScript::IsPayToWitnessScriptHash() const
 {
     // Extra-fast test for pay-to-witness-script-hash CScripts:
-    return (this->size() == 34 &&
+    return (this->size() == 0x22 &&
+            (*this)[0] == OP_0 &&
+            (*this)[1] == 0x20);
+}
+
+bool CScript::IsPayToWitnessKeyHash() const
+{
+    // Test for pay-to-witness-public-key-hash CScripts:
+    return (this->size() == 0x16 &&
             (*this)[0] == OP_0 &&
+            (*this)[1] == 0x14);
+}
+
+
+bool CScript::IsPayToTaproot() const
+{
+    // Test for pay-to-taproot CScripts:
+    return (this->size() == 0x22 &&
+            (*this)[0] == OP_1 &&
             (*this)[1] == 0x20);
 }
 
diff --git a/src/script/script.h b/src/script/script.h
index 66d63fae89e12..22e3dd0ebd017 100644
--- a/src/script/script.h
+++ b/src/script/script.h
@@ -533,8 +533,12 @@ class CScript : public CScriptBase
      */
     unsigned int GetSigOpCount(const CScript& scriptSig) const;
 
+    bool IsPayToPublicKey() const;
     bool IsPayToScriptHash() const;
+    bool IsPayToPublicKeyHash() const;
     bool IsPayToWitnessScriptHash() const;
+    bool IsPayToWitnessKeyHash() const;
+    bool IsPayToTaproot() const;
     bool IsWitnessProgram(int& version, std::vector<unsigned char>& program) const;
 
     /** Called by IsStandardTx and P2SH/BIP62 VerifyScript (which makes it consensus-critical). */

From 574d0773fa05ef3f6fc75346a44a3ffcb6d6d71d Mon Sep 17 00:00:00 2001
From: Tom Briar <tombriar08@Gmail.com>
Date: Wed, 6 Dec 2023 14:02:27 -0500
Subject: [PATCH 4/8] addresstype: Split ExtractDestination into two functions

---
 src/addresstype.cpp | 24 ++++++++++++++----------
 src/addresstype.h   | 13 +++++++++++++
 2 files changed, 27 insertions(+), 10 deletions(-)

diff --git a/src/addresstype.cpp b/src/addresstype.cpp
index f199d1b479446..96c4541252984 100644
--- a/src/addresstype.cpp
+++ b/src/addresstype.cpp
@@ -8,7 +8,6 @@
 #include <hash.h>
 #include <pubkey.h>
 #include <script/script.h>
-#include <script/solver.h>
 #include <uint256.h>
 #include <util/hash_type.h>
 
@@ -46,17 +45,15 @@ WitnessV0ScriptHash::WitnessV0ScriptHash(const CScript& in)
     CSHA256().Write(in.data(), in.size()).Finalize(begin());
 }
 
-bool ExtractDestination(const CScript& scriptPubKey, CTxDestination& addressRet)
-{
-    std::vector<valtype> vSolutions;
-    TxoutType whichType = Solver(scriptPubKey, vSolutions);
+bool BuildDestination(const std::vector<std::vector<unsigned char>>& vSolutions, const TxoutType& scriptType, CTxDestination& addressRet) {
+    if (vSolutions.size() == 0) {
+        return false;
+    }
 
-    switch (whichType) {
+    switch (scriptType) {
     case TxoutType::PUBKEY: {
         CPubKey pubKey(vSolutions[0]);
-        if (!pubKey.IsValid()) {
-            addressRet = CNoDestination(scriptPubKey);
-        } else {
+        if (pubKey.IsValid()) {
             addressRet = PubKeyDestination(pubKey);
         }
         return false;
@@ -94,12 +91,19 @@ bool ExtractDestination(const CScript& scriptPubKey, CTxDestination& addressRet)
     case TxoutType::MULTISIG:
     case TxoutType::NULL_DATA:
     case TxoutType::NONSTANDARD:
-        addressRet = CNoDestination(scriptPubKey);
         return false;
     } // no default case, so the compiler can warn about missing cases
     assert(false);
 }
 
+bool ExtractDestination(const CScript& scriptPubKey, CTxDestination& addressRet)
+{
+    std::vector<valtype> vSolutions;
+    TxoutType scriptType = Solver(scriptPubKey, vSolutions);
+    addressRet = CNoDestination(scriptPubKey);
+    return BuildDestination(vSolutions, scriptType, addressRet);
+}
+
 namespace {
 class CScriptVisitor
 {
diff --git a/src/addresstype.h b/src/addresstype.h
index 0152858bad2a3..6f1964ed099b8 100644
--- a/src/addresstype.h
+++ b/src/addresstype.h
@@ -8,6 +8,7 @@
 #include <attributes.h>
 #include <pubkey.h>
 #include <script/script.h>
+#include <script/solver.h>
 #include <uint256.h>
 #include <util/hash_type.h>
 
@@ -133,6 +134,18 @@ using CTxDestination = std::variant<CNoDestination, PubKeyDestination, PKHash, S
 /** Check whether a CTxDestination corresponds to one with an address. */
 bool IsValidDestination(const CTxDestination& dest);
 
+/**
+ * Parse a vSolutions vector and script type into a destination address. Assigns result to
+ *
+ * For standard scripts that have addresses (and P2PK as an exception), a corresponding CTxDestination
+ * is assigned to addressRet.
+ * For all other scripts. addressRet is unassigned.
+ *
+ * Returns true for standard destinations with addresses - P2PKH, P2SH, P2WPKH, P2WSH, P2TR and P2W??? scripts.
+ * Returns false for non-standard destinations and those without addresses - P2PK, bare multisig, null data, and nonstandard scripts.
+ */
+bool BuildDestination(const std::vector<std::vector<unsigned char>>& vSolutions, const TxoutType& scriptType, CTxDestination& addressRet);
+
 /**
  * Parse a scriptPubKey for the destination.
  *

From 9482f266ebc1722c1464b21751bff590bfbd8aab Mon Sep 17 00:00:00 2001
From: Tom Briar <tombriar08@Gmail.com>
Date: Wed, 6 Dec 2023 15:55:30 -0500
Subject: [PATCH 5/8] node: Added vector based coin lookup

---
 src/node/coin.cpp | 19 +++++++++++++++++++
 src/node/coin.h   | 13 +++++++++++++
 2 files changed, 32 insertions(+)

diff --git a/src/node/coin.cpp b/src/node/coin.cpp
index 221854c5f67d5..056c68a9b39af 100644
--- a/src/node/coin.cpp
+++ b/src/node/coin.cpp
@@ -23,4 +23,23 @@ void FindCoins(const NodeContext& node, std::map<COutPoint, Coin>& coins)
         }
     }
 }
+
+std::vector<Coin> FindCoins(const node::NodeContext& node, const std::vector<COutPoint>& outpoints)
+{
+    assert(node.mempool);
+    assert(node.chainman);
+    LOCK2(cs_main, node.mempool->cs);
+    CCoinsViewCache& chain_view = node.chainman->ActiveChainstate().CoinsTip();
+    CCoinsViewMemPool mempool_view(&chain_view, *node.mempool);
+    std::vector<Coin> coins;
+    for (auto& outpoint : outpoints) {
+        Coin coin;
+        if (!mempool_view.GetCoin(outpoint, coin)) {
+            // Either the coin is not in the CCoinsViewCache or is spent. Clear it.
+            coin.Clear();
+        }
+        coins.push_back(coin);
+    }
+    return coins;
+}
 } // namespace node
diff --git a/src/node/coin.h b/src/node/coin.h
index b32e410e1cceb..9c3213b69efa9 100644
--- a/src/node/coin.h
+++ b/src/node/coin.h
@@ -6,6 +6,7 @@
 #define BITCOIN_NODE_COIN_H
 
 #include <map>
+#include <vector>
 
 class COutPoint;
 class Coin;
@@ -22,6 +23,18 @@ struct NodeContext;
  * @param[in,out] coins map to fill
  */
 void FindCoins(const node::NodeContext& node, std::map<COutPoint, Coin>& coins);
+
+/**
+ * Look up unspent output information. Returns coins in the mempool and in the
+ * current chain UTXO set. Iterates through all the outpoints and
+ * returns the corresponding coins.
+ *
+ * @param[in] node The node context to use for lookup
+ * @param[in] outpoints A Vector containing outpoints
+ */
+std::vector<Coin> FindCoins(const node::NodeContext& node, const std::vector<COutPoint>& outpoints);
 } // namespace node
 
+
+
 #endif // BITCOIN_NODE_COIN_H

From 84069f6a35f3157d44cf83af0c9516dffe39b6e7 Mon Sep 17 00:00:00 2001
From: Tom Briar <tombriar08@Gmail.com>
Date: Wed, 6 Dec 2023 15:56:17 -0500
Subject: [PATCH 6/8] primitives: Added Compressed Transaction

---
 src/Makefile.am                |   8 +
 src/primitives/compression.cpp | 291 ++++++++++++++++++
 src/primitives/compression.h   | 518 +++++++++++++++++++++++++++++++++
 src/primitives/transaction.h   |   9 +
 4 files changed, 826 insertions(+)
 create mode 100644 src/primitives/compression.cpp
 create mode 100644 src/primitives/compression.h

diff --git a/src/Makefile.am b/src/Makefile.am
index b6f0daaabadbd..734805687b411 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -626,6 +626,8 @@ crypto_libbitcoin_crypto_arm_shani_la_SOURCES = crypto/sha256_arm_shani.cpp
 libbitcoin_consensus_a_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES)
 libbitcoin_consensus_a_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS)
 libbitcoin_consensus_a_SOURCES = \
+  addresstype.cpp \
+  addresstype.h \
   arith_uint256.cpp \
   arith_uint256.h \
   consensus/amount.h \
@@ -641,6 +643,8 @@ libbitcoin_consensus_a_SOURCES = \
   primitives/block.h \
   primitives/transaction.cpp \
   primitives/transaction.h \
+  primitives/compression.cpp \
+  primitives/compression.h \
   pubkey.cpp \
   pubkey.h \
   script/bitcoinconsensus.cpp \
@@ -650,6 +654,8 @@ libbitcoin_consensus_a_SOURCES = \
   script/script.h \
   script/script_error.cpp \
   script/script_error.h \
+  script/solver.cpp \
+  script/solver.h \
   serialize.h \
   span.h \
   tinyformat.h \
@@ -921,6 +927,7 @@ libbitcoinkernel_la_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) -fvisibility=default
 #       shrink to only those which are absolutely necessary.
 libbitcoinkernel_la_SOURCES = \
   kernel/bitcoinkernel.cpp \
+  addresstype.cpp \
   arith_uint256.cpp \
   chain.cpp \
   clientversion.cpp \
@@ -957,6 +964,7 @@ libbitcoinkernel_la_SOURCES = \
   pow.cpp \
   primitives/block.cpp \
   primitives/transaction.cpp \
+  primitives/compression.cpp \
   pubkey.cpp \
   random.cpp \
   randomenv.cpp \
diff --git a/src/primitives/compression.cpp b/src/primitives/compression.cpp
new file mode 100644
index 0000000000000..c28d89f70da16
--- /dev/null
+++ b/src/primitives/compression.cpp
@@ -0,0 +1,291 @@
+#include <primitives/compression.h>
+
+CCompressedOutPoint::CCompressedOutPoint(const uint32_t block_height, const uint32_t block_index) : m_block_height(block_height), m_block_index(block_index), m_vout(0) {};
+
+CCompressedOutPoint::CCompressedOutPoint(const uint256& txid, const uint32_t vout) : m_block_height(0), m_block_index(0), m_txid(txid), m_vout(vout) {};
+
+std::string CCompressedOutPoint::ToString() const
+{
+    return strprintf("CCompressedOutPoint(block_height=%u, block_index=%u, txid=%s, vout=%u)",
+    m_block_height,
+    m_block_index,
+    HexStr(m_txid),
+    m_vout);
+}
+
+CCompressedTxIn::CCompressedTxIn(const CTxIn& txin, const CCompressedOutPoint& out, const CScript& script_pubkey) : m_prevout(out), m_compressedSignature(false), m_hashType(0), m_isHashStandard(false) {
+    if ((!txin.scriptSig.empty() || !txin.scriptWitness.stack.empty()) && !out.IsCoinbase()) {
+        CScript scriptSig;
+
+        //Use Solver to get script_type but ignore solutions vector, I can split out the Solver code into a subfunction so I don't waste this solutions vector.
+        std::vector<std::vector<unsigned char>> solutions;
+        TxoutType script_type = Solver(script_pubkey, solutions);
+        switch (script_type) {
+            case TxoutType::WITNESS_V1_TAPROOT:
+                if (txin.scriptWitness.stack.size() == 1 && (txin.scriptWitness.stack[0].size() == 64 || txin.scriptWitness.stack[0].size() == 65)) {
+                    m_signature = txin.scriptWitness.stack[0];
+                    if (m_signature.size() == 65) {
+                        m_hashType = m_signature[m_signature.size()-1];
+                        m_isHashStandard = m_hashType == 0x00;
+                        m_signature.pop_back();
+                    } else m_isHashStandard = true;
+                    m_compressedSignature = true;
+                }
+                break;
+            case TxoutType::PUBKEYHASH:
+                scriptSig = txin.scriptSig;
+                break;
+            case TxoutType::WITNESS_V0_KEYHASH:
+                scriptSig = CScript(txin.scriptWitness.stack[0].begin(), txin.scriptWitness.stack[0].end());
+                break;
+            case TxoutType::SCRIPTHASH: {
+                opcodetype opcodeRet;
+                CScriptBase::const_iterator pc = txin.scriptSig.begin();
+                std::vector<unsigned char> vec;
+                txin.scriptSig.GetOp(pc, opcodeRet, vec);
+                CScript redeemScript = CScript(vec.begin(), vec.end());
+                if (redeemScript.IsPayToWitnessKeyHash()) {
+                    scriptSig = CScript(txin.scriptWitness.stack[0].begin(), txin.scriptWitness.stack[0].end());
+                }
+                break;
+            }
+            default:
+                break;
+        }
+        if (!scriptSig.empty()) {
+            opcodetype opcodeRet;
+            CScriptBase::const_iterator pc = scriptSig.begin();
+            scriptSig.GetOp(pc, opcodeRet, m_signature);
+            if (!m_signature.empty()) {
+                uint8_t hashType = m_signature[m_signature.size()-1];
+                int length = m_signature.size()-1;
+                secp256k1_ecdsa_signature sig;
+                if (secp256k1_ecdsa_signature_parse_der(secp256k1_context_static, &sig, &m_signature[0], length)) {
+                    m_signature.resize(64);
+                    if (secp256k1_ecdsa_signature_serialize_compact(secp256k1_context_static, &m_signature[0], &sig)) {
+                        m_compressedSignature = true;
+                        m_isHashStandard = m_hashType == 0x01;
+                        m_hashType = m_isHashStandard ? 0 : hashType;
+                    }
+                }
+            }
+            if (!m_compressedSignature) m_warning = "Malformed Signature cannot be compressed.";
+        }
+
+        if (!m_compressedSignature) {
+            DataStream stream;
+            stream << VARINT(txin.scriptSig.size());
+            if (!txin.scriptSig.empty())
+                stream.write(MakeByteSpan(txin.scriptSig));
+            stream << VARINT(txin.scriptWitness.stack.size());
+            for (std::vector<unsigned char> stackitem : txin.scriptWitness.stack) {
+                stream << VARINT(stackitem.size());
+                stream.write(MakeByteSpan(stackitem));
+            }
+            m_signature.resize(stream.size());
+            stream.read(MakeWritableByteSpan(m_signature));
+        }
+    }
+    m_nSequence = txin.nSequence;
+}
+
+std::string CCompressedTxIn::ToString() const
+{
+    return strprintf("CCompressedTxIn(prevout=%s, signature=%s, compressed=%b, hashType=%u, isHashStandard=%b, nSequence=%u)",
+    m_prevout.ToString(),
+    HexStr(m_signature),
+    m_compressedSignature,
+    m_hashType,
+    m_isHashStandard,
+    m_nSequence);
+}
+
+CCompressedTxOut::CCompressedTxOut(const CTxOut& txout) : m_nValue(txout.nValue) {
+    //Uses Solver to parse a script into a TxoutType and a byte array containing the raw script without prefix or suffix. Script gets reassembled with BuildDestination
+    std::vector<std::vector<unsigned char>> solutions;
+    m_scriptType = Solver(txout.scriptPubKey, solutions);
+    switch (m_scriptType) {
+        case TxoutType::PUBKEY:
+        case TxoutType::PUBKEYHASH:
+        case TxoutType::SCRIPTHASH:
+        case TxoutType::WITNESS_V0_KEYHASH:
+        case TxoutType::WITNESS_V0_SCRIPTHASH:
+        case TxoutType::WITNESS_V1_TAPROOT:
+            assert(solutions.size() == 1);
+            m_scriptPubKey = solutions[0];
+            break;
+        default:
+            m_scriptType = TxoutType::NONSTANDARD;
+            m_scriptPubKey.resize(txout.scriptPubKey.size());
+            copy(txout.scriptPubKey.begin(), txout.scriptPubKey.end(), m_scriptPubKey.begin());
+    }
+}
+
+std::string CCompressedTxOut::ToString() const
+{
+    return strprintf("CCompressedTxOut(scriptPubKey=%s, scriptType=%s, nValue=%u)",
+    HexStr(m_scriptPubKey),
+    GetTxnOutputType(m_scriptType),
+    m_nValue);
+}
+
+CCompressedTransaction::CCompressedTransaction(const CTransaction tx, const uint32_t& minimumHeight, const std::vector<CCompressedInput>& cinputs) : m_minimumHeight(minimumHeight), m_nVersion(tx.nVersion), m_nLockTime(tx.nLockTime ? tx.nLockTime-minimumHeight : 0) {
+    assert(tx.vin.size() == cinputs.size());
+
+    for (auto const& txout : tx.vout) {
+        m_vout.emplace_back(txout);
+    }
+
+    for (size_t index = 0; index < tx.vin.size(); index++) {
+        m_vin.emplace_back(tx.vin[index], cinputs[index].outpoint, cinputs[index].scriptPubKey);
+    }
+}
+
+std::string CCompressedTransaction::ToString() const
+{
+    std::string str;
+    str += strprintf("CCompressedTransaction(nVersion=%u, nLockTime=%u, minimumHeight=%u)\n",
+    m_nVersion,
+    m_nLockTime,
+    m_minimumHeight);
+    for (const auto& txin : m_vin)
+        str += "    " + txin.ToString() + "\n";
+    for (const auto& txout : m_vout)
+        str += "    " + txout.ToString() + "\n";
+    return str;
+}
+
+CTxOut::CTxOut(const CCompressedTxOut& txout) {
+    if (txout.IsCompressed()) {
+        if (txout.scriptType() == TxoutType::PUBKEY) {
+            scriptPubKey << txout.scriptPubKey();
+            scriptPubKey << opcodetype::OP_CHECKSIG;
+        } else {
+            std::vector<std::vector<unsigned char>> vSolutions;
+            vSolutions.push_back(txout.scriptPubKey());
+            CTxDestination destination;
+            if(!BuildDestination(vSolutions, txout.scriptType(), destination)) throw std::ios_base::failure("Script Deseralization Failed, Either Invalid Payload or Script Type.");
+            scriptPubKey = GetScriptForDestination(destination);
+        }
+    } else {
+        scriptPubKey = CScript(txout.scriptPubKey().begin(), txout.scriptPubKey().end());
+    }
+    nValue = txout.nValue();
+}
+
+CMutableTransaction::CMutableTransaction(const CCompressedTransaction& tx, const std::vector<COutPoint>& prevouts, const std::vector<CTxOut>& outs) : nVersion(tx.nVersion()), nLockTime(tx.nLockTime()) {
+    assert(outs.size() == tx.vin().size() && prevouts.size() == tx.vin().size());
+    if (tx.nLockTime()) nLockTime += tx.minimumHeight();
+    for (const auto& txout : tx.vout()) {
+        vout.emplace_back(txout);
+    }
+    for (size_t index = 0; index < tx.vin().size(); index++) {
+        vin.emplace_back(prevouts[index], CScript(), tx.vin()[index].nSequence());
+    }
+
+    for (size_t index = 0; index < tx.vin().size(); index++) {
+        const CCompressedTxIn& input = tx.vin()[index];
+        const CTxOut& output = outs[index];
+        if (input.compressedSignature()) {
+            if (output.scriptPubKey.IsPayToTaproot()) {
+                std::vector<std::vector<unsigned char>> stack;
+                stack.push_back(input.signature());
+                uint8_t hashType = input.isHashStandard() ? 0 : input.hashType();
+                if (hashType) stack[0].push_back(input.hashType());
+                vin[index].scriptWitness.stack = stack;
+            } else {
+                uint8_t hashType = input.isHashStandard() ? 0x01 : input.hashType();
+                uint256 hash;
+
+                if (output.scriptPubKey.IsPayToPublicKeyHash()) {
+                    hash = SignatureHash(output.scriptPubKey, *this, index, hashType, output.nValue, SigVersion::BASE);
+                } else {
+                    std::vector<std::vector<unsigned char>> vSolutions;
+                    Solver(output.scriptPubKey, vSolutions);
+                    CTxDestination destination;
+                    BuildDestination(vSolutions, TxoutType::WITNESS_V0_KEYHASH, destination);
+                    CScript scriptCode = GetScriptForDestination(destination);
+                    hash = SignatureHash(scriptCode, *this, index, hashType, output.nValue, SigVersion::WITNESS_V0);
+                }
+
+                for (int recoveryId = 4; recoveryId < 8; recoveryId++) {
+                    std::vector<unsigned char> vch_sig(CPubKey::COMPACT_SIGNATURE_SIZE);
+                    vch_sig[0] = recoveryId+27;
+                    copy(input.signature().begin(), input.signature().end(), vch_sig.begin()+1);
+
+                    CPubKey pub;
+                    if (pub.RecoverCompact(hash, vch_sig)) {
+                        CScript script_pubkey;
+                        //Use Solver to get script_type but ignore solutions vector, I can split out the Solver code into a subfunction so I don't waste this solutions vector.
+                        std::vector<std::vector<unsigned char>> solutions;
+                        TxoutType script_type = Solver(output.scriptPubKey, solutions);
+                        switch (script_type) {
+                            case TxoutType::SCRIPTHASH:
+                                script_pubkey = GetScriptForDestination(ScriptHash(GetScriptForDestination(PKHash(pub))));
+                                break;
+                            case TxoutType::PUBKEYHASH:
+                                script_pubkey = GetScriptForDestination(PKHash(pub));
+                                if (script_pubkey != output.scriptPubKey) {
+                                    pub.Decompress();
+                                    script_pubkey = GetScriptForDestination(PKHash(pub));
+                                }
+                                break;
+                            default:
+                                script_pubkey = GetScriptForDestination(WitnessV0KeyHash(pub));
+                        }
+
+                        if (script_pubkey == output.scriptPubKey) {
+                            secp256k1_ecdsa_signature sig;
+                            secp256k1_ecdsa_signature_parse_compact(secp256k1_context_static, &sig, &input.signature()[0]);
+                            size_t sig_size = 71;
+                            std::vector<unsigned char> sig_vec(sig_size);
+                            secp256k1_ecdsa_signature_serialize_der(secp256k1_context_static, &sig_vec[0], &sig_size, &sig);
+
+                            size_t p_size = pub.size();
+                            std::vector<unsigned char> p_vec(p_size);
+                            copy(pub.begin(), pub.end(), p_vec.begin());
+
+                            if (output.scriptPubKey.IsPayToPublicKeyHash()) {
+                                sig_vec.resize(sig_size+1);
+                                sig_vec[sig_size] = hashType;
+                                vin[index].scriptSig << sig_vec;
+                                vin[index].scriptSig << p_vec;
+                            } else {
+                                sig_vec.resize(sig_size+1);
+                                sig_vec[sig_size] = hashType;
+                                std::vector<std::vector<unsigned char>> stack;
+                                stack.push_back(sig_vec);
+                                stack.push_back(p_vec);
+                                vin[index].scriptWitness.stack = stack;
+                            }
+                            break;
+                        }
+                    }
+                }
+            }
+            if (vin[index].scriptSig.empty() && vin[index].scriptWitness.stack.empty()) throw std::ios_base::failure("Malformed Signature For Input");
+        } else if (!input.signature().empty()) {
+            DataStream stream;
+            stream.write(MakeByteSpan(input.signature()));
+            uint64_t scriptSigLength = std::numeric_limits<uint64_t>::max();
+            stream >> VARINT(scriptSigLength);
+            if (scriptSigLength >= MAX_SCRIPT_SIZE) throw std::ios_base::failure("Script Signature Exceeds MAX_SCRIPT_SIZE");
+            if (scriptSigLength) {
+                std::vector<unsigned char> scriptSigBytes(scriptSigLength);
+                stream.read(MakeWritableByteSpan(scriptSigBytes));
+                vin[index].scriptSig = CScript(scriptSigBytes.begin(), scriptSigBytes.end());
+            }
+            uint64_t witnessCount = std::numeric_limits<uint64_t>::max();
+            stream >> VARINT(witnessCount);
+            std::vector<std::vector<unsigned char>> stack;
+            for (uint64_t i = 0; i < witnessCount; i++) {
+                uint64_t witnessLength = std::numeric_limits<uint64_t>::max();
+                stream >> VARINT(witnessLength);
+                std::vector<unsigned char> witness(witnessLength);
+                stream.read(MakeWritableByteSpan(witness));
+                stack.push_back(witness);
+            }
+            vin[index].scriptWitness.stack = stack;
+        }
+    }
+}
diff --git a/src/primitives/compression.h b/src/primitives/compression.h
new file mode 100644
index 0000000000000..daaaa2f80926a
--- /dev/null
+++ b/src/primitives/compression.h
@@ -0,0 +1,518 @@
+#ifndef BITCOIN_PRIMITIVES_COMPRESSION_H
+#define BITCOIN_PRIMITIVES_COMPRESSION_H
+
+#include <addresstype.h>
+#include <consensus/amount.h>
+#include <primitives/transaction.h>
+#include <script/interpreter.h>
+#include <script/script.h>
+#include <util/strencodings.h>
+#include <bitset>
+#include <cmath>
+#include <logging.h>
+#include <streams.h>
+#include <secp256k1.h>
+#include <secp256k1_extrakeys.h>
+#include <secp256k1_recovery.h>
+#include <secp256k1_schnorrsig.h>
+#include <serialize.h>
+#include <uint256.h>
+
+class CCompressedOutPoint
+{
+private:
+    uint32_t m_block_height;
+    uint32_t m_block_index;
+    uint256 m_txid;
+    uint32_t m_vout;
+public:
+    const uint32_t&  block_height() const  { return m_block_height; }
+    const uint32_t& block_index() const { return m_block_index; }
+    const uint256& txid() const { return m_txid; }
+    const uint32_t& vout() const { return m_vout; }
+
+    explicit CCompressedOutPoint(const uint32_t block_height, const uint32_t block_index);
+    explicit CCompressedOutPoint(const uint256& txid, const uint32_t vout);
+
+    bool IsCompressed() const {
+        return m_txid.IsNull();
+    }
+
+    bool IsCoinbase() const {
+        return !this->IsCompressed() && m_txid.IsNull();
+    }
+
+    friend bool operator==(const CCompressedOutPoint& a, const CCompressedOutPoint& b)
+    {
+        return (a.m_block_height == b.m_block_height && a.m_block_index == b.m_block_index && a.m_txid == b.m_txid && a.m_vout == b.m_vout);
+    }
+
+    friend bool operator!=(const CCompressedOutPoint& a, const CCompressedOutPoint& b)
+    {
+        return !(a == b);
+    }
+
+    template <typename Stream>
+    inline void Serialize(Stream& s) const {
+        /* BlockHeight: Section 3.6(1) */
+        s << VARINT(this->m_block_height);
+        if (this->IsCompressed()) {
+            /* Txid/Vout: Section 3.6.1(1) */
+            s << VARINT(this->m_block_index);
+        } else {
+            /* Txid: Section 3.6.2(1) */
+            s.write(MakeByteSpan(this->m_txid));
+            /* Vout: Section 3.6.2(2) */
+            s << VARINT(this->m_vout);
+
+        }
+    }
+
+    template <typename Stream>
+    inline void Unserialize(Stream& s, bool constructor=false) {
+        /* BlockHeight: Section 3.6(1) */
+        this->m_block_height = std::numeric_limits<uint32_t>::max();
+        s >> VARINT(this->m_block_height);
+        if (this->m_block_height > 0) {
+            /* BlockHeight: Section 3.6.1(1) */
+            this->m_block_index = std::numeric_limits<uint32_t>::max();
+            s >> VARINT(this->m_block_index);
+            this->m_vout = 0;
+        } else {
+            this->m_block_index = 0;
+            /* Txid: Section 3.6.2(1) */
+            std::vector<unsigned char> txid(32);
+            s.read(MakeWritableByteSpan(txid));
+            this->m_txid = uint256(txid);
+            /* Vout: Section 3.6.2(2) */
+            this->m_vout = std::numeric_limits<uint32_t>::max();
+            s >> VARINT(this->m_vout);
+        }
+    }
+
+    template <typename Stream>
+    explicit CCompressedOutPoint(deserialize_type, Stream& s) {
+        Unserialize(s, true);
+    }
+
+    std::string ToString() const;
+};
+
+class CCompressedTxIn
+{
+private:
+    CCompressedOutPoint m_prevout;
+    std::vector<unsigned char> m_signature;
+    bool m_compressedSignature;
+    uint8_t m_hashType;
+    bool m_isHashStandard;
+    uint32_t m_nSequence;
+    std::string m_warning;
+public:
+    const CCompressedOutPoint& prevout() const { return m_prevout; }
+    const std::vector<unsigned char>& signature() const { return m_signature; }
+    const bool& compressedSignature() const { return m_compressedSignature; }
+    const uint8_t& hashType() const { return m_hashType; }
+    const bool& isHashStandard() const { return m_isHashStandard; }
+    const uint32_t& nSequence() const { return m_nSequence; }
+    const std::string& warning() const { return m_warning; }
+
+    explicit CCompressedTxIn(const CTxIn& txin, const CCompressedOutPoint& out, const CScript& script_pubkey);
+
+    bool IsSigned() const {
+        return !this->m_signature.empty();
+    }
+
+    bool IsSequenceStandard() const {
+        return this->m_nSequence == 0x00000000 || this->m_nSequence == 0xFFFFFFFE || this->m_nSequence == 0xFFFFFFFF;
+    }
+
+    friend bool operator==(const CCompressedTxIn& a, const CCompressedTxIn& b)
+    {
+        return (a.m_prevout == b.m_prevout && a.m_signature == b.m_signature && a.m_compressedSignature == b.m_compressedSignature && a.m_hashType == b.m_hashType && a.m_isHashStandard == b.m_isHashStandard && a.m_nSequence == b.m_nSequence);
+    }
+
+    friend bool operator!=(const CCompressedTxIn& a, const CCompressedTxIn& b)
+    {
+        return !(a == b);
+    }
+
+    uint64_t GetMetadata() const {
+        uint64_t metadata = 0;
+        metadata |= this->m_compressedSignature;
+        metadata |= this->m_isHashStandard << 1;
+        switch (this->m_nSequence) {
+            case 0x00000000:
+                metadata |= (1 << 2);
+                break;
+            case 0xFFFFFFFE:
+                metadata |= (0b10 << 2);
+                break;
+            case 0xFFFFFFFF:
+                metadata |= (0b11 << 2);
+                break;
+        }
+        return metadata;
+    }
+
+    template <typename Stream>
+    inline void Serialize(Stream& s) const {
+        /* Txid/Vout: Section 3.6(1-2) */
+        this->m_prevout.Serialize(s);
+        /* Signature: Section 3.6(3) */
+        if (this->m_compressedSignature) {
+            /* Compressed Signature: Section 3.6.1(2) */
+            s.write(MakeByteSpan(this->m_signature));
+            /* HashType: Section 3.6.1(3) */
+            if (!this->m_isHashStandard) s << this->m_hashType;
+        } else {
+            s << COMPACTSIZE(this->m_signature.size());
+            if (!this->m_signature.empty()) {
+                /* Uncompressed Signature: Section 3.6.2(3) */
+                s.write(MakeByteSpan(this->m_signature));
+            }
+        }
+        /* Sequence: Section 3.6(4) */
+        if (!this->IsSequenceStandard()) {
+            s << VARINT(this->m_nSequence);
+        }
+    }
+
+    template <typename Stream>
+    inline void Unserialize(Stream& s, uint64_t& metadata, bool constructor=false) {
+        m_hashType = 0;
+        m_compressedSignature = metadata & 1;
+        m_isHashStandard = (metadata & (1 << 1)) >> 1;
+        uint8_t sequenceEncoding = (metadata & (0b11 << 2)) >> 2;
+
+        /* Txid/Vout: Section 3.6(1-2) */
+        if (!constructor) this->m_prevout.Unserialize(s);
+        /* Signature: Section 3.6(3) */
+        if (this->m_compressedSignature) {
+            /* Compressed Signature: Section 3.6.1(2) */
+            m_signature.resize(64);
+            s.read(MakeWritableByteSpan(m_signature));
+            /* HashType: Section 3.6.1(3) */
+            if (!m_isHashStandard) {
+                s >> this->m_hashType;
+            }
+        } else {
+            /* Uncompressed Signature: Section 3.6.2(3) */
+            uint64_t scriptLength = std::numeric_limits<uint64_t>::max();
+            s >> COMPACTSIZE(scriptLength);
+            if (scriptLength) {
+                m_signature.resize(scriptLength);
+                s.read(MakeWritableByteSpan(m_signature));
+            }
+        }
+
+        /* Sequence: Section 3.6(4) */
+        if (sequenceEncoding) {
+            if (sequenceEncoding == 0x01) m_nSequence = 0x00000000;
+            if (sequenceEncoding == 0x02) m_nSequence = 0xFFFFFFFE;
+            if (sequenceEncoding == 0x03) m_nSequence = 0xFFFFFFFF;
+        } else {
+            this->m_nSequence = std::numeric_limits<uint32_t>::max();
+            s >> VARINT(this->m_nSequence);
+        }
+    }
+
+    template <typename Stream>
+    explicit CCompressedTxIn(deserialize_type, Stream& s, uint64_t& metadata) : m_prevout(CCompressedOutPoint(deserialize, s)) {
+        Unserialize(s, metadata, true);
+    }
+
+    std::string ToString() const;
+};
+
+
+class CCompressedTxOut
+{
+private:
+    std::vector<unsigned char> m_scriptPubKey;
+    TxoutType m_scriptType;
+    int64_t m_nValue;
+public:
+    const std::vector<unsigned char>& scriptPubKey() const { return m_scriptPubKey; }
+    const TxoutType& scriptType() const { return m_scriptType; }
+    const int64_t& nValue() const { return m_nValue; }
+
+    explicit CCompressedTxOut(const CTxOut& txout);
+
+
+
+    friend bool operator==(const CCompressedTxOut& a, const CCompressedTxOut& b)
+    {
+        return (a.m_scriptPubKey == b.m_scriptPubKey && a.m_scriptType == b.m_scriptType && a.m_nValue == b.m_nValue);
+    }
+
+    friend bool operator!=(const CCompressedTxOut& a, const CCompressedTxOut& b)
+    {
+        return !(a == b);
+    }
+
+    uint8_t GetSerializedScriptType() const {
+        if (this->m_scriptType == TxoutType::PUBKEY && this->m_scriptPubKey.size() == 65) return 0;
+        if (this->m_scriptType == TxoutType::PUBKEY && this->m_scriptPubKey.size() == 33) return 1;
+        if (this->m_scriptType == TxoutType::PUBKEYHASH) return 2;
+        if (this->m_scriptType == TxoutType::SCRIPTHASH) return 3;
+        if (this->m_scriptType == TxoutType::WITNESS_V0_SCRIPTHASH) return 4;
+        if (this->m_scriptType == TxoutType::WITNESS_V0_KEYHASH) return 5;
+        if (this->m_scriptType == TxoutType::WITNESS_V1_TAPROOT) return 6;
+        return 7;
+    }
+
+    bool IsCompressed() const {
+        return this->GetSerializedScriptType() != 7;
+    }
+
+    template <typename Stream>
+    inline void Serialize(Stream& s) const {
+        if (!this->m_scriptPubKey.empty()) {
+            /* Output Script: Section 3.7(1) */
+            if (!this->IsCompressed()) {
+                s << COMPACTSIZE(this->m_scriptPubKey.size());
+            }
+            s.write(MakeByteSpan(this->m_scriptPubKey));
+        }
+        /* Amount: Section 3.7(2) */
+        s << VARINT(static_cast<uint64_t>(this->m_nValue));
+    }
+
+    template <typename Stream>
+    inline void Unserialize(Stream& s, uint8_t& serialized_script_type) {
+        /* Output Script: Section 3.7(1) */
+        uint64_t scriptLength = std::numeric_limits<uint64_t>::max();
+        switch (serialized_script_type) {
+            case 0:
+                scriptLength = 65;
+                this->m_scriptType = TxoutType::PUBKEY;
+                break;
+            case 1:
+                scriptLength = 33;
+                this->m_scriptType = TxoutType::PUBKEY;
+                break;
+            case 2:
+                scriptLength = 20;
+                this->m_scriptType = TxoutType::PUBKEYHASH;
+                break;
+            case 3:
+                scriptLength = 20;
+                this->m_scriptType = TxoutType::SCRIPTHASH;
+                break;
+            case 4:
+                scriptLength = 32;
+                this->m_scriptType = TxoutType::WITNESS_V0_SCRIPTHASH;
+                break;
+            case 5:
+                scriptLength = 20;
+                this->m_scriptType = TxoutType::WITNESS_V0_KEYHASH;
+                break;
+            case 6:
+                scriptLength = 32;
+                this->m_scriptType = TxoutType::WITNESS_V1_TAPROOT;
+                break;
+            case 7:
+                s >> COMPACTSIZE(scriptLength);
+                if (scriptLength >= MAX_SCRIPT_SIZE) throw std::ios_base::failure("Output Script Exceeds MAX_SCRIPT_SIZE");
+                this->m_scriptType = TxoutType::NONSTANDARD;
+                break;
+            default:
+                throw std::ios_base::failure(strprintf("Script Type Deseralization must be 0-7, %u is not a valid Script Type.", serialized_script_type));
+        }
+        this->m_scriptPubKey.resize(scriptLength);
+        s.read(MakeWritableByteSpan(this->m_scriptPubKey));
+
+        /* Amount: Section 3.7(2) */
+        uint64_t us_value = std::numeric_limits<uint64_t>::max();
+        s >> VARINT(us_value);
+        this->m_nValue = us_value;
+    }
+
+    template <typename Stream>
+    explicit CCompressedTxOut(deserialize_type, Stream& s, uint8_t& serialized_script_type) {
+        Unserialize(s, serialized_script_type);
+    }
+
+    std::string ToString() const;
+};
+
+struct CCompressedInput {
+    CCompressedOutPoint outpoint;
+    CScript scriptPubKey;
+};
+
+class CCompressedTransaction
+{
+private:
+    uint32_t m_minimumHeight;
+    int32_t m_nVersion;
+    uint32_t m_nLockTime;
+    std::vector<CCompressedTxIn> m_vin;
+    std::vector<CCompressedTxOut> m_vout;
+public:
+    const uint32_t& minimumHeight() const { return m_minimumHeight; }
+    const int32_t& nVersion() const { return m_nVersion; }
+    const uint32_t& nLockTime() const { return m_nLockTime; }
+    const std::vector<CCompressedTxIn>& vin() const { return m_vin; }
+    const std::vector<CCompressedTxOut>& vout() const { return m_vout; }
+
+    explicit CCompressedTransaction(const CTransaction tx, const uint32_t& minimumHeight, const std::vector<CCompressedInput>& cinputs);
+
+    friend bool operator==(const CCompressedTransaction& a, const CCompressedTransaction& b)
+    {
+        return (a.m_nVersion == b.m_nVersion && a.m_nLockTime == b.m_nLockTime && a.m_vin == b.m_vin && a.m_vout == b.m_vout);
+    }
+
+    friend bool operator!=(const CCompressedTransaction& a, const CCompressedTransaction& b)
+    {
+        return !(a == b);
+    }
+
+    template <typename Stream>
+    inline void Serialize(Stream& s) const {
+        if (this->m_vin.empty() || this->m_vout.empty()) throw std::ios_base::failure("Must have at least one input and output");
+
+        /* Transaction Metadata: Section 3.2(1) */
+        /* Version: Section 3.3(1) */
+        uint8_t version = 0;
+        if (this->m_nVersion < 4 && this->m_nVersion > 0) version = this->m_nVersion;
+        uint8_t control = version;
+        /* InputCount: Section 3.3(2) */
+        uint8_t inputCount = 0;
+        if (this->m_vin.size() < 4) inputCount = this->m_vin.size();
+        control |= inputCount << 2;
+        /* OutputCount: Section 3.3(3) */
+        uint8_t outputCount = 0;
+        if (this->m_vout.size() < 4) outputCount = this->m_vout.size();
+        control |= outputCount << 4;
+        /* LockTime: Section 3.3(4) */
+        control |= (this->m_nLockTime > 0) << 6;
+
+        s << control;
+
+        /* Version: Section 3.2(2) */
+        if (!version) s << VARINT(static_cast<uint32_t>(this->m_nVersion));
+        /* InputCount: Section 3.2(3) */
+        if (!inputCount) s << COMPACTSIZE(this->m_vin.size());
+        /* OutputCount: Section 3.2(4) */
+        if (!outputCount) s << COMPACTSIZE(this->m_vout.size());
+        /* LockTime: Section 3.2(5) */
+        if (this->m_nLockTime) s << VARINT(this->m_nLockTime);
+
+        /* Minimum BlockHeight: Section 3.2(6) */
+        s << VARINT(this->m_minimumHeight);
+
+        /* Input-Output Metadata: Section 3.4-3.5 */
+        std::vector<int> sizes;
+        std::vector<unsigned char> controlVec;
+        for (const auto &vin : this->m_vin) {
+            controlVec.push_back(vin.GetMetadata());
+            sizes.push_back(4);
+        }
+        for (const auto &vout : this->m_vout) {
+            controlVec.push_back(vout.GetSerializedScriptType());
+            sizes.push_back(3);
+        }
+        std::vector<int> resizes;
+        resizes.push_back(8);
+        std::vector<unsigned char> compressedControlVec;
+        CHECK_NONFATAL(ConvertBitsVariable<true>(sizes, resizes, [&](unsigned char c) { compressedControlVec.push_back(c); }, controlVec.begin(), controlVec.end()));
+        s.write(MakeByteSpan(compressedControlVec));
+
+        /* Input: Section 3.6 */
+        for (const auto &vin : this->m_vin) {
+            vin.Serialize(s);
+        }
+
+        /* Output: Section 3.7 */
+        for (const auto &vout : this->m_vout) {
+            vout.Serialize(s);
+        }
+    }
+
+    template <typename Stream>
+    inline void Unserialize(Stream& s) {
+        /* Transaction Metadata: Section 3.2 */
+        uint8_t control;
+        s >> control;
+        /* Version: Section 3.3(1) */
+        this->m_nVersion = control & 3;
+        /* InputCount: Section 3.3(2) */
+        uint32_t nInputCount = (control & (3 << 2)) >> 2;
+        /* OutputCount: Section 3.3(3) */
+        uint32_t nOutputCount = (control & (3 << 4)) >> 4;
+        /* LockTime: Section 3.3(4) */
+        this->m_nLockTime = (control & (3 << 6)) >> 6;
+
+        /* Version: Section 3.2(2) */
+        if (!this->m_nVersion) {
+            uint32_t us_version = std::numeric_limits<uint32_t>::max();
+            s >> VARINT(us_version);
+            this->m_nVersion = static_cast<int32_t>(us_version);
+        }
+
+        /* InputCount: Section 3.2(3) */
+        if (!nInputCount) {
+            nInputCount = std::numeric_limits<uint32_t>::max();
+            s >> COMPACTSIZE(nInputCount);
+        }
+
+        /* OutputCount: Section 3.2(4) */
+        if (!nOutputCount) {
+            nOutputCount = std::numeric_limits<uint32_t>::max();
+            s >> COMPACTSIZE(nOutputCount);
+        }
+
+        if (!nInputCount || !nOutputCount) throw std::ios_base::failure("Must have at least one input and output");
+
+        /* LockTime: Section 3.2(5) */
+        if (this->m_nLockTime) {
+            this->m_nLockTime = std::numeric_limits<uint32_t>::max();
+            s >> VARINT(this->m_nLockTime);
+        }
+
+        /* Minimum BlockHeight: Section 3.2(6) */
+        this->m_minimumHeight = std::numeric_limits<uint32_t>::max();
+        s >> VARINT(this->m_minimumHeight);
+
+        /* Input-Output Metadata: Section 3.4-3.5 */
+        std::vector<unsigned char> controlVec;
+        controlVec.resize(((nInputCount * 4) + (nOutputCount * 3) + 7) / 8);
+        s.read(MakeWritableByteSpan(controlVec));
+
+        std::vector<int> sizes;
+        sizes.push_back(8);
+        std::vector<int> resizes;
+        for (size_t index = 0; index < nInputCount; index++) {
+            resizes.push_back(4);
+        }
+        for (size_t index = 0; index < nOutputCount; index++) {
+            resizes.push_back(3);
+        }
+
+        std::vector<unsigned char> parsedControlVec;
+        CHECK_NONFATAL(ConvertBitsVariable<true>(sizes, resizes, [&](unsigned char c) { parsedControlVec.push_back(c); }, controlVec.begin(), controlVec.end()));
+
+        /* Input: Section 3.6 */
+        for (size_t index = 0; index < nInputCount; index++) {
+            uint64_t metadata = (uint64_t)parsedControlVec[index];
+            this->m_vin.push_back(CCompressedTxIn(deserialize, s, metadata));
+        }
+
+        /* Input: Section 3.7 */
+        for (size_t index = 0; index < nOutputCount; index++) {
+            uint8_t serializedScriptType = (uint8_t)parsedControlVec[index+nInputCount];
+            this->m_vout.push_back(CCompressedTxOut(deserialize, s, serializedScriptType));
+        }
+    }
+
+    template <typename Stream>
+    CCompressedTransaction(deserialize_type, Stream& s) {
+        Unserialize(s);
+    }
+
+    std::string ToString() const;
+};
+
+#endif // BITCOIN_PRIMITIVES_COMPRESSION_H
diff --git a/src/primitives/transaction.h b/src/primitives/transaction.h
index ccbeb3ec49bb6..c5f1b610c5acc 100644
--- a/src/primitives/transaction.h
+++ b/src/primitives/transaction.h
@@ -24,6 +24,8 @@
 #include <utility>
 #include <vector>
 
+class CCompressedOutPoint;
+
 /** An outpoint - a combination of a transaction hash and an index n into its vout */
 class COutPoint
 {
@@ -35,6 +37,7 @@ class COutPoint
 
     COutPoint(): n(NULL_INDEX) { }
     COutPoint(const Txid& hashIn, uint32_t nIn): hash(hashIn), n(nIn) { }
+    COutPoint(const CCompressedOutPoint& cout);
 
     SERIALIZE_METHODS(COutPoint, obj) { READWRITE(obj.hash, obj.n); }
 
@@ -143,6 +146,8 @@ class CTxIn
     std::string ToString() const;
 };
 
+class CCompressedTxOut;
+
 /** An output of a transaction.  It contains the public key that the next input
  * must be able to sign with to claim it.
  */
@@ -157,6 +162,7 @@ class CTxOut
         SetNull();
     }
 
+    CTxOut(const CCompressedTxOut& txout);
     CTxOut(const CAmount& nValueIn, CScript scriptPubKeyIn);
 
     SERIALIZE_METHODS(CTxOut, obj) { READWRITE(obj.nValue, obj.scriptPubKey); }
@@ -373,6 +379,8 @@ class CTransaction
     bool HasWitness() const { return m_has_witness; }
 };
 
+class CCompressedTransaction;
+
 /** A mutable version of CTransaction. */
 struct CMutableTransaction
 {
@@ -383,6 +391,7 @@ struct CMutableTransaction
 
     explicit CMutableTransaction();
     explicit CMutableTransaction(const CTransaction& tx);
+    explicit CMutableTransaction(const CCompressedTransaction& tx, const std::vector<COutPoint>& prevouts, const std::vector<CTxOut>& outs);
 
     template <typename Stream>
     inline void Serialize(Stream& s) const {

From ba03fc9602c06c2b3c4b4181ce981e2d2da0276a Mon Sep 17 00:00:00 2001
From: Tom Briar <tombriar08@Gmail.com>
Date: Fri, 20 Oct 2023 11:34:07 -0400
Subject: [PATCH 7/8] validation: Added OutPoint (De)Compression

---
 src/Makefile.am    |  1 +
 src/validation.cpp | 75 ++++++++++++++++++++++++++++++++++++++++++++++
 src/validation.h   |  6 ++++
 3 files changed, 82 insertions(+)

diff --git a/src/Makefile.am b/src/Makefile.am
index 734805687b411..68f52380357ab 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -443,6 +443,7 @@ libbitcoin_node_a_SOURCES = \
   policy/rbf.cpp \
   policy/settings.cpp \
   pow.cpp \
+  primitives/compression.cpp \
   rest.cpp \
   rpc/blockchain.cpp \
   rpc/fees.cpp \
diff --git a/src/validation.cpp b/src/validation.cpp
index 0f3d5d145414c..ad2daa20350e7 100644
--- a/src/validation.cpp
+++ b/src/validation.cpp
@@ -34,6 +34,7 @@
 #include <policy/settings.h>
 #include <pow.h>
 #include <primitives/block.h>
+#include <primitives/compression.h>
 #include <primitives/transaction.h>
 #include <random.h>
 #include <reverse_iterator.h>
@@ -4296,6 +4297,80 @@ bool Chainstate::LoadChainTip()
     return true;
 }
 
+std::tuple<uint32_t, std::vector<CCompressedInput>> Chainstate::CompressOutPoints(CTransaction& tx, std::vector<std::string>& warnings)
+{
+    std::vector<CCompressedInput> cinputs;
+    int barrier_height = CHECK_NONFATAL(this->m_chain.Tip())->nHeight - 100;
+
+    uint32_t minimumHeight = std::numeric_limits<uint32_t>::max();
+    if (tx.nLockTime > 0) minimumHeight = tx.nLockTime-1;
+
+    std::vector<Coin> coins;
+    for (size_t index = 0; index < tx.vin.size(); index++) {
+        Coin coin;
+        if (this->CoinsTip().GetCoin(tx.vin[index].prevout, coin)) {
+            if (coin.nHeight >= barrier_height) {
+                coin.nHeight = 0;
+                warnings.push_back(strprintf("UTXO(%u): %s", index, "Input must be older then 100 blocks to avoid a block reorg, Input will be uncompressed"));
+            } else if (coin.nHeight <= minimumHeight) {
+                minimumHeight = coin.nHeight-1;
+            }
+        } else {
+            warnings.push_back(strprintf("UTXO(%u): %s", index, "Input missing or spent"));
+        }
+        coins.push_back(coin);
+    }
+
+    for (size_t index = 0; index < tx.vin.size(); index++) {
+        Coin coin = coins[index];
+        if (!coin.nHeight) {
+            cinputs.push_back(CCompressedInput{CCompressedOutPoint(tx.vin[index].prevout.hash, tx.vin[index].prevout.n), coin.out.scriptPubKey});
+            continue;
+        }
+        CBlockIndex* pindex = this->m_chain[coin.nHeight];
+        CBlock block;
+        this->m_blockman.ReadBlockFromDisk(block, *pindex);
+        uint32_t flattendIndex = 0;
+        for (size_t vindex = 0; vindex < block.vtx.size(); vindex++) {
+            if ((block.vtx[vindex])->GetHash() == tx.vin[index].prevout.hash) {
+                cinputs.push_back(CCompressedInput{CCompressedOutPoint(coin.nHeight-minimumHeight, flattendIndex+tx.vin[index].prevout.n), coin.out.scriptPubKey});
+                break;
+            }
+            flattendIndex += (block.vtx[vindex])->vout.size();
+        }
+        if (index+1 > cinputs.size()) {
+            cinputs.push_back(CCompressedInput{CCompressedOutPoint(tx.vin[index].prevout.hash, tx.vin[index].prevout.n), coin.out.scriptPubKey});
+            warnings.push_back(strprintf("UTXO(%u): %s", index, "Coin found but block could not be read, Possibly a pruned node, Input will be uncompressed"));
+        }
+    }
+    return std::make_tuple(minimumHeight, cinputs);
+}
+
+std::vector<COutPoint> Chainstate::UncompressOutPoints(CCompressedTransaction& ctx) EXCLUSIVE_LOCKS_REQUIRED(::cs_main)
+{
+    std::vector<COutPoint> prevouts;
+
+    for (size_t index = 0; index < ctx.vin().size(); index++) {
+        if (ctx.vin()[index].prevout().IsCompressed()) {
+            CBlockIndex* pindex = this->m_chainman.ActiveChain()[ctx.vin()[index].prevout().block_height()+ctx.minimumHeight()];
+            if (pindex) {
+                CBlock block;
+                this->m_blockman.ReadBlockFromDisk(block, *pindex);
+                uint32_t vcounter = 0;
+                for (size_t vindex = 0; vindex < block.vtx.size(); vindex++) {
+                    if ((ctx.vin()[index].prevout().block_index()-vcounter) < (block.vtx[vindex])->vout.size()) {
+                        prevouts.emplace_back((block.vtx[vindex])->GetHash(), ctx.vin()[index].prevout().block_index()-vcounter);
+                    }
+                    vcounter += (block.vtx[vindex])->vout.size();
+                }
+            }
+        }
+        if (prevouts.size() != index+1) prevouts.emplace_back(Txid::FromUint256(ctx.vin()[index].prevout().txid()), ctx.vin()[index].prevout().vout());
+    }
+
+    return prevouts;
+}
+
 CVerifyDB::CVerifyDB(Notifications& notifications)
     : m_notifications{notifications}
 {
diff --git a/src/validation.h b/src/validation.h
index 093cecfcd104b..5345d38864ae4 100644
--- a/src/validation.h
+++ b/src/validation.h
@@ -24,6 +24,7 @@
 #include <policy/feerate.h>
 #include <policy/packages.h>
 #include <policy/policy.h>
+#include <primitives/compression.h>
 #include <script/script_error.h>
 #include <sync.h>
 #include <txdb.h>
@@ -743,6 +744,11 @@ class Chainstate
         size_t max_coins_cache_size_bytes,
         size_t max_mempool_size_bytes) EXCLUSIVE_LOCKS_REQUIRED(::cs_main);
 
+    /** Functions for indexing COutPoints for Compressed Transactions */
+    std::tuple<uint32_t, std::vector<CCompressedInput>> CompressOutPoints(CTransaction& tx, std::vector<std::string>& warnings) EXCLUSIVE_LOCKS_REQUIRED(::cs_main);
+
+    std::vector<COutPoint> UncompressOutPoints(CCompressedTransaction& ctx) EXCLUSIVE_LOCKS_REQUIRED(::cs_main);
+
     std::string ToString() EXCLUSIVE_LOCKS_REQUIRED(::cs_main);
 
     //! Indirection necessary to make lock annotations work with an optional mempool.

From 4205ec307e4918eb42da5d2d35ecb791eada79c8 Mon Sep 17 00:00:00 2001
From: Tom Briar <tombriar08@Gmail.com>
Date: Tue, 11 Jul 2023 16:55:45 -0400
Subject: [PATCH 8/8] rpc: Added RPC endpoints for Compressed Transactions

---
 src/rpc/rawtransaction.cpp | 101 +++++++++++++++++++++++++++++++++++++
 src/test/fuzz/rpc.cpp      |   2 +
 2 files changed, 103 insertions(+)

diff --git a/src/rpc/rawtransaction.cpp b/src/rpc/rawtransaction.cpp
index b1e6d677f8720..062b154535a2a 100644
--- a/src/rpc/rawtransaction.cpp
+++ b/src/rpc/rawtransaction.cpp
@@ -12,6 +12,7 @@
 #include <index/txindex.h>
 #include <key_io.h>
 #include <node/blockstorage.h>
+#include <node/chainstate.h>
 #include <node/coin.h>
 #include <node/context.h>
 #include <node/psbt.h>
@@ -20,6 +21,7 @@
 #include <policy/policy.h>
 #include <policy/rbf.h>
 #include <primitives/transaction.h>
+#include <primitives/compression.h>
 #include <psbt.h>
 #include <random.h>
 #include <rpc/blockchain.h>
@@ -494,6 +496,103 @@ static RPCHelpMan decoderawtransaction()
     };
 }
 
+static RPCHelpMan compressrawtransaction()
+{
+    return RPCHelpMan{"compressrawtransaction",
+                "Return a JSON object representing the compressed, serialized, hex-encoded transaction.",
+                {
+                    {"hexstring", RPCArg::Type::STR_HEX, RPCArg::Optional::NO, "The transaction hex string"},
+                },
+                RPCResult{
+                    RPCResult::Type::OBJ, "", "",
+                    {
+                        {RPCResult::Type::STR_HEX, "result", "The compressed transaction hex string"},
+                        {RPCResult::Type::ARR, "warnings", "",
+                        {
+                            {RPCResult::Type::STR_HEX, "warning", "Warnings given for each input"}
+                        }}
+                    }
+                },
+                RPCExamples{
+                    HelpExampleCli("compressrawtransaction", "\"hexstring\"")
+            + HelpExampleRpc("compressrawtransaction", "\"hexstring\"")
+                },
+        [&](const RPCHelpMan& self, const JSONRPCRequest& request) -> UniValue
+{
+    const NodeContext& node = EnsureAnyNodeContext(request.context);
+
+    UniValue r(UniValue::VOBJ);
+    UniValue warnings(UniValue::VARR);
+    CMutableTransaction mtx;
+    if (DecodeHexTx(mtx, request.params[0].get_str(), true, true)) {
+        CTransaction tx = CTransaction(mtx);
+        LOCK(cs_main);
+        std::vector<std::string> warning_strings;
+        std::tuple<uint32_t, std::vector<CCompressedInput>> cinputstup = EnsureChainman(node).ActiveChainstate().CompressOutPoints(tx, warning_strings);
+        CCompressedTransaction ctx = CCompressedTransaction(CTransaction(mtx), std::get<0>(cinputstup), std::get<1>(cinputstup));
+        for (const auto& warning : warning_strings) {
+            warnings.push_back(warning);
+        }
+        for (size_t index = 0; index < ctx.vin().size(); index++) {
+            if (ctx.vin()[index].warning() != "") warnings.push_back(strprintf("UTXO(%u): %s", index, ctx.vin()[index].warning()));
+        }
+
+        DataStream stream;
+        ctx.Serialize(stream);
+        std::vector<unsigned char> hex(stream.size());
+        stream.read(MakeWritableByteSpan(hex));
+
+        r.pushKV("result", HexStr(hex));
+        r.pushKV("warnings", warnings);
+        return r;
+    }
+    throw JSONRPCError(RPC_DESERIALIZATION_ERROR, "TX decode failed");
+},
+    };
+}
+
+static RPCHelpMan decompressrawtransaction()
+{
+    return RPCHelpMan{"decompressrawtransaction",
+                "Return a JSON object representing the decompressed, serialized, hex-encoded transaction.",
+                {
+                    {"hexstring", RPCArg::Type::STR_HEX, RPCArg::Optional::NO, "The compressed transaction hex string"},
+                },
+                RPCResult{
+                       RPCResult::Type::STR_HEX, "result", "The decompressed transaction hex string"
+                },
+                RPCExamples{
+                    HelpExampleCli("decompressrawtransaction", "\"hexstring\"")
+            + HelpExampleRpc("decompressrawtransaction", "\"hexstring\"")
+                },
+        [&](const RPCHelpMan& self, const JSONRPCRequest& request) -> UniValue
+{
+    const NodeContext& node = EnsureAnyNodeContext(request.context);
+    UniValue r(UniValue::VOBJ);
+    try {
+        DataStream ssData(ParseHex(request.params[0].get_str()));
+        CCompressedTransaction tx = CCompressedTransaction(deserialize, ssData);
+        LOCK(cs_main);
+
+        std::vector<COutPoint> prevouts = EnsureChainman(node).ActiveChainstate().UncompressOutPoints(tx);
+        std::vector<Coin> coins = FindCoins(node, prevouts);
+        std::vector<CTxOut> outs;
+        outs.reserve(coins.size());
+        for (auto& coin : coins) {
+            outs.push_back(coin.out);
+        }
+
+        std::tuple<std::vector<COutPoint>, std::vector<CTxOut>> Index = std::make_tuple(prevouts, outs);
+        if (std::get<0>(Index).size() != std::get<1>(Index).size() || std::get<1>(Index).size() != tx.vin().size()) throw JSONRPCError(RPC_DESERIALIZATION_ERROR, "Compressed TX decode failed");
+        return EncodeHexTx(CTransaction(CMutableTransaction(tx, std::get<0>(Index), std::get<1>(Index))));
+    } catch (const std::exception& exc) {
+        // Fall through.
+    }
+    throw JSONRPCError(RPC_DESERIALIZATION_ERROR, "Compressed TX decode failed");
+},
+    };
+}
+
 static RPCHelpMan decodescript()
 {
     return RPCHelpMan{
@@ -2007,6 +2106,8 @@ void RegisterRawTransactionRPCCommands(CRPCTable& t)
         {"rawtransactions", &getrawtransaction},
         {"rawtransactions", &createrawtransaction},
         {"rawtransactions", &decoderawtransaction},
+        {"rawtransactions", &compressrawtransaction},
+        {"rawtransactions", &decompressrawtransaction},
         {"rawtransactions", &decodescript},
         {"rawtransactions", &combinerawtransaction},
         {"rawtransactions", &signrawtransactionwithkey},
diff --git a/src/test/fuzz/rpc.cpp b/src/test/fuzz/rpc.cpp
index 2325bf094139f..5f0621f8c4ee0 100644
--- a/src/test/fuzz/rpc.cpp
+++ b/src/test/fuzz/rpc.cpp
@@ -93,6 +93,7 @@ const std::vector<std::string> RPC_COMMANDS_SAFE_FOR_FUZZING{
     "clearbanned",
     "combinepsbt",
     "combinerawtransaction",
+    "compressrawtransaction",
     "converttopsbt",
     "createmultisig",
     "createpsbt",
@@ -100,6 +101,7 @@ const std::vector<std::string> RPC_COMMANDS_SAFE_FOR_FUZZING{
     "decodepsbt",
     "decoderawtransaction",
     "decodescript",
+    "decompressrawtransaction",
     "deriveaddresses",
     "descriptorprocesspsbt",
     "disconnectnode",