feat(satp): sample implementation of SATP standard using relays #3001

outsidethecode · 2024-01-22T13:19:58Z

This is a first-of-a-kind sample PoC implementation of the Secure Asset Transfer Protocol (SATP), which is currently being standardized in an IETF Working Group, using network-owned Cacti relays.

The network relays provided by the Weaver submodule are augmented with the gateway functions specified in the SATP Core draft. A pair of networks, each using a relay for cross-network communication, effect the secure transfer of an asset from one to the other in an atomic operation. The protocol diagram that represents this implementation is illustrated here. (Note: a new version of this diagram, v20, was published very recently, after this PR was submitted, though it has just cosmetic changes from v19.)

Note: This is purely a sample reference implementation of the SATP protocol that requires more work in future PRs to be generally usable. It only supports a specific asset transfer scenario across two basic Fabric test networks used in Cacti for CI/CD, and is meant only for demonstrative purposes. Though all SATP endpoints are implemented in the relays, the precise asset transfer scenario is hardcoded within the backend functions (both in the relay and in the Fabric driver, a.k.a. connector). This hardcoded logic will be removed very soon, making the relay correspond to a generic "gateway" appliance of the kind described in the SATP specifications.

Implemented the two endpoints TransferProposalClaims and TransferProposalReceipt
Implemented the lock assertion method
Added the lock-assertion-receipt and lock-assertion-broadcast methods
Added the request to the driver to lock an asset
Corrected some variable names
Added the endpoints corresponding to the steps 3.1 to 3.9
Implemented the step 2.1B so that the driver call the gateway to update the asset status
Implemented the status check functionality required for 2.1B, 3.2B, 3.4B, and 3.6B
First iteration of switching to real driver (fabric)
Added the required functions to call back the gateway after the asset is locked.
Colorful log to show different stages and steps
Initial commit for create-asset, extinguish, and assign-asset
Added the resources required to assign an asset
Uncommented the perform lock part
Implemented the lock assertion method
Added the lock-assertion-receipt and lock-assertion-broadcast methods
Added the request to the driver to lock an asset
Corrected some variable names
Added the endpoints corresponding to the steps 3.1 to 3.9
Implemented the step 2.1B so that the driver call the gateway to update the asset status
Implemented the status check functionality required for 2.1B, 3.2B, 3.4B, and 3.6B
First iteration of switching to real driver (fabric)
Added the required functions to call back the gateway after the asset is locked.
Initial commit for create-asset, extinguish, and assign-asset
Added the resources required to assign an asset
Uncommented the perform lock part
Corrected compile error due to merging issue
Squashed commit: first implementation version of satp
Implemented the lock assertion method
Added the lock-assertion-receipt and lock-assertion-broadcast methods
Added the request to the driver to lock an asset
Corrected some variable names
Added the endpoints corresponding to the steps 3.1 to 3.9
Implemented the step 2.1B so that the driver call the gateway to update the asset status
Implemented the status check functionality required for 2.1B, 3.2B, 3.4B, and 3.6B
First iteration of switching to real driver (fabric)
Added the required functions to call back the gateway after the asset is locked.
Initial commit for create-asset, extinguish, and assign-asset
Added the resources required to assign an asset
Uncommented the perform lock part
Corrected compile error due to merging issue
Initial documentation on how to run the satp gateway and test it
feat(weaver): add application logs
Initial log functionality
feat: log statements are stored in sqlite db
feat: updated the .gitignore file
feat: added log entries to all requests
feat: removed unnecessary comments
feat: added the missing methods in driver
feat: remove the unnecessary log statements
feat: initial rfcs for satp
initial satp github action script
Corrected the github actions
Corrected the protoc version to be 3.17.3 and rebuild
Corrected the version of cacti_weaver_protos_rs and added the copyright statement
Removed the code copied from fabric-cli
feat: initial rfcs for satp
Initial satp github action script
Corrected the github actions
Corrected the protoc version to be 3.17.3 and rebuild
corrected the version of cacti_weaver_protos_rs and added the copyright statement
Removed the code copied from fabric-cli
Refactored the code in satp.ts
Corrected the satpsimpleasset smart contract

Pull Request Requirements

Rebased onto upstream/main branch and squashed into single commit to help maintainers review it more efficient and to avoid spaghetti git commit graphs that obfuscate which commit did exactly what change, when and, why.
Have git sign off at the end of commit message to avoid being marked red. You can add -s flag when using git commit command. You may refer to this link for more information.
Follow the Commit Linting specification. You may refer to this link for more information.

Character Limit

Pull Request Title and Commit Subject must not exceed 72 characters (including spaces and special characters).
Commit Message per line must not exceed 80 characters (including spaces and special characters).

A Must Read for Beginners
For rebasing and squashing, here's a must read guide for beginners.

weaver/core/drivers/fabric-driver/server/satp.ts

weaver/core/relay/config/Dummy_Relay.toml

petermetz

@outsidethecode I have a few nits (see above in the comments) but otherwise LGTM for the big picture. Please make sure to have @sandeepnRES or @VRamakrishna approve the PR because they are the primary maintainers for the weaver related code components.

.github/workflows/test_weaver-fabric-fabric-satp.yaml

package-lock.json

packages/cactus-plugin-ledger-connector-polkadot/src/test/rust/ink/flipper/Cargo.lock

outSH

Thank you very much for your work and this contribution! I've added some comments, please note that some of them applies to entire commit (I didn't want to repeat same message in multiple places), so please review and apply the changes accordingly :)

weaver/core/drivers/fabric-driver/server/satp.ts

weaver/samples/fabric/satpsimpleasset/assetmgmt_test.go

weaver/common/protos-go/common/access_control.pb.go

weaver/sdks/fabric/interoperation-node-sdk/src/SatpAssetManager.ts

petermetz · 2024-03-07T20:42:23Z

FYI:

Peter's changes:

I've removed the Cargo.lock file from the diff because it seemed
unrelated to the change at hand.
Applied some formatting (but not all because as I just realized we
don't yet have the linter working for the ./weaver/ subdirectories)
Removed dead code/unused variables
Adjusted var to let
Not perfect by any means but I'll work on some more incremental upgrades
later in follow-up pull requests.

petermetz · 2024-03-07T20:42:51Z

@RafaelAPB @outsidethecode Also I rebased onto upstream/main and then force pushed so please make sure not to overwrite the changes if you'll be doing any code adjustments of your own

petermetz

LGTM once @outSH's change requests were addressed.

VRamakrishna · 2024-03-08T10:42:18Z

@outsidethecode Left several suggestions for you in response to comments made by @outSH . Please fix these, and then, assuming the tests pass, we should be ready to merge.

outSH · 2024-03-11T10:27:56Z

@outsidethecode @VRamakrishna I've added one more comment in previous review items, please have a look :) Also please remember to re-request the review so I get a signal to review it sooner

RafaelAPB · 2024-04-09T13:59:01Z

@outsidethecode any updates on this? We would like to merge your contributions asap. Can you please solve conflicts, rebase, address the feedback, and push?

VRamakrishna · 2024-04-10T11:25:12Z

@RafaelAPB There's one outstanding comment from Michal that I didn't notice earlier, but have just asked Zakwan to fix.

@petermetz I think all your queries have been resolved. There's one item from Michal that's pending. I wonder why the tests aren't running though?

outSH

@outsidethecode LGTM, thank you very much! Remember to squash the commits and fix the conflicts before merge :)

weaver/core/relay/src/services/constants.rs

weaver/sdks/fabric/interoperation-node-sdk/src/SatpAssetManager.ts

weaver/core/relay/config/Corda_Relay.toml

sandeepnRES · 2024-05-08T06:18:00Z

@petermetz I've fixed the build issue, along with some other issues that I found during this.
I have a separate commit for the makefile fix, because I want to keep the commit separate, and not combine in the large commit, so that in future if it needs to be reverted it can be done easily. I can actually create separate PR, but it seemed logical to add it in this PR.

petermetz

@petermetz I've fixed the build issue, along with some other issues that I found during this.
I have a separate commit for the makefile fix, because I want to keep the commit separate, and not combine in the large commit, so that in future if it needs to be reverted it can be done easily. I can actually create separate PR, but it seemed logical to add it in this PR.

@sandeepnRES The guiding principle we have for commits on the main branch is that any commit on the main branch should compile successfully. Splitting the commit the way you are doing it now goes in the opposite direction by intentionally leaving a commit in on main that we know has the build broken. If you'd like to ensure that the commit can easily be reverted I would save the diff of that commit as a text file and attach it to the pull request (and/or save it anywhere else you know you'll find it) an the squash the commits together to satisfy the no build breaks on main principle.
Sometimes the rules can be broken too (as it have happened in the past) if there's some good reason to but the saving the diff to a text file is a one liner bash command so I'm thinking this should be OK as a workaround. With all that said, if there is a reason still after this to keep the commits separate then let us talk more about it.

sandeepnRES · 2024-05-09T08:42:08Z

@petermetz I've fixed the build issue, along with some other issues that I found during this.
I have a separate commit for the makefile fix, because I want to keep the commit separate, and not combine in the large commit, so that in future if it needs to be reverted it can be done easily. I can actually create separate PR, but it seemed logical to add it in this PR.

@sandeepnRES The guiding principle we have for commits on the main branch is that any commit on the main branch should compile successfully. Splitting the commit the way you are doing it now goes in the opposite direction by intentionally leaving a commit in on main that we know has the build broken. If you'd like to ensure that the commit can easily be reverted I would save the diff of that commit as a text file and attach it to the pull request (and/or save it anywhere else you know you'll find it) an the squash the commits together to satisfy the no build breaks on main principle. Sometimes the rules can be broken too (as it have happened in the past) if there's some good reason to but the saving the diff to a text file is a one liner bash command so I'm thinking this should be OK as a workaround. With all that said, if there is a reason still after this to keep the commits separate then let us talk more about it.

Hi @petermetz That build or the makefile isn't broken, in the sense the build works fine with the one big satp commit.
I think confusion is because of the build issue that also I fixed along with this extra commit, but that is part of the satp commit, so satp commit is stable and isn't broken.

In the separate commit about makefile, I just added a cleanup and a verbose message for the case where build fails, to tell that why the build has failed.
For e.g., the fabric-driver build will fail if its dependencies like protos-js aren't built before building fabric-driver, and my change is just doing some cleanup in this case and providing a message to build protos-js first.

This seemed to be unrelated to the big satp commit, that's why wanted to keep it separate. If you are okay, I can create a separate PR for it too, and remove the commit from this PR. Only reason I put it in this PR because it was a very small change related to components that are modified in this PR.

RafaelAPB · 2024-05-09T16:17:08Z

Looks good to me.

petermetz · 2024-05-09T19:52:51Z

@petermetz I've fixed the build issue, along with some other issues that I found during this.
I have a separate commit for the makefile fix, because I want to keep the commit separate, and not combine in the large commit, so that in future if it needs to be reverted it can be done easily. I can actually create separate PR, but it seemed logical to add it in this PR.

@sandeepnRES The guiding principle we have for commits on the main branch is that any commit on the main branch should compile successfully. Splitting the commit the way you are doing it now goes in the opposite direction by intentionally leaving a commit in on main that we know has the build broken. If you'd like to ensure that the commit can easily be reverted I would save the diff of that commit as a text file and attach it to the pull request (and/or save it anywhere else you know you'll find it) an the squash the commits together to satisfy the no build breaks on main principle. Sometimes the rules can be broken too (as it have happened in the past) if there's some good reason to but the saving the diff to a text file is a one liner bash command so I'm thinking this should be OK as a workaround. With all that said, if there is a reason still after this to keep the commits separate then let us talk more about it.

Hi @petermetz That build or the makefile isn't broken, in the sense the build works fine with the one big satp commit. I think confusion is because of the build issue that also I fixed along with this extra commit, but that is part of the satp commit, so satp commit is stable and isn't broken.

In the separate commit about makefile, I just added a cleanup and a verbose message for the case where build fails, to tell that why the build has failed. For e.g., the fabric-driver build will fail if its dependencies like protos-js aren't built before building fabric-driver, and my change is just doing some cleanup in this case and providing a message to build protos-js first.

This seemed to be unrelated to the big satp commit, that's why wanted to keep it separate. If you are okay, I can create a separate PR for it too, and remove the commit from this PR. Only reason I put it in this PR because it was a very small change related to components that are modified in this PR.

@sandeepnRES If the build wasn't broken then nevermind, I'm OK with it for the same reasons you explained. I just thought the build was not working at all.

petermetz · 2024-05-10T16:50:03Z

@sandeepnRES Now it's got seven commits though and the diff is much bigger as well. Maybe if it's this much I'd recommend starting a new PR for the house-cleaning/house-keeping changes separately if you can afford it time-wise. With that said, I'm also very strapped for time and so don't want to put up barriers that slow people down so this is strictly just an idea/suggestion.

sandeepnRES · 2024-05-10T17:51:40Z

@sandeepnRES Now it's got seven commits though and the diff is much bigger as well. Maybe if it's this much I'd recommend starting a new PR for the house-cleaning/house-keeping changes separately if you can afford it time-wise. With that said, I'm also very strapped for time and so don't want to put up barriers that slow people down so this is strictly just an idea/suggestion.

Hi @petermetz no actually we are seeing some CI tests failing, so Rama and me are trying to fix them. Once all tests passes, we will squash the commits into only 2 commits as we discussed.

petermetz · 2024-05-10T18:52:02Z

@sandeepnRES Ohhh, OK, fair enough! Good luck and let me know if I can help with anything!

* Implemented the two endpoints TransferProposalClaims and TransferProposalReceipt * implemented the lock assertion method * Added the lock-assertion-receipt and lock-assertion-broadcast methods * Added the request to the driver to lock an asset * corrected some variable names * Added the endpoints corresponding to the steps 3.1 to 3.9 * Implemented the step 2.1B so that the driver call the gateway to update the asset status * Implemented the status check functionality required for 2.1B, 3.2B, 3.4B, and 3.6B * First iteration of switching to real driver (fabric) * Added the required functions to call back the gateway after the asset is locked. * Colorful log to show different stages and steps * Initial commit for create-asset, extinguish, and assign-asset * Added the resources required to assign an asset * uncommented the perform lock part * implemented the lock assertion method * Added the lock-assertion-receipt and lock-assertion-broadcast methods * Added the request to the driver to lock an asset * corrected some variable names * Added the endpoints corresponding to the steps 3.1 to 3.9 * Implemented the step 2.1B so that the driver call the gateway to update the asset status * Implemented the status check functionality required for 2.1B, 3.2B, 3.4B, and 3.6B * First iteration of switching to real driver (fabric) * Added the required functions to call back the gateway after the asset is locked. * Initial commit for create-asset, extinguish, and assign-asset * Added the resources required to assign an asset * uncommented the perform lock part * corrected compile error due to merging issue * Squashed commit: first implementation version of satp * implemented the lock assertion method * Added the lock-assertion-receipt and lock-assertion-broadcast methods * Added the request to the driver to lock an asset * corrected some variable names * Added the endpoints corresponding to the steps 3.1 to 3.9 * Implemented the step 2.1B so that the driver call the gateway to update the asset status * Implemented the status check functionality required for 2.1B, 3.2B, 3.4B, and 3.6B * First iteration of switching to real driver (fabric) * Added the required functions to call back the gateway after the asset is locked. * Initial commit for create-asset, extinguish, and assign-asset * Added the resources required to assign an asset * uncommented the perform lock part * corrected compile error due to merging issue * Initial documentation on how to run the satp gateway and test it * feat(weaver): add application logs * Initial log functionality * feat: log statements are stored in sqlite db * feat: updated the .gitignore file * feat: added log entries to all requests * feat: removed the unnecessary * feat: removed unnecessary comments * feat: added the missing methods in driver * feat: remove the unnecessary log statements * feat: initial rfcs for satp * initial satp github action script * corrected the github actions * Corrected the protoc version to be 3.17.3 and rebuild * corrected the version of cacti_weaver_protos_rs and added the copyright statement * removed the code copied from fabric-cli * feat: initial rfcs for satp * initial satp github action script * corrected the github actions * Corrected the protoc version to be 3.17.3 and rebuild * corrected the version of cacti_weaver_protos_rs and added the copyright statement * removed the code copied from fabric-cli * refactored the code in satp.ts * corrected the satpsimpleasset smart contract * fix(relay-docker): change server to alpine image and add libsqlite * refactor: rename the service to sample_service due to hardcoded values --------- Peter's changes: 1. I've removed the Cargo.lock file from the diff because it seemed unrelated to the change at hand. 2. Applied some formatting (but not all because as I just realized we don't yet have the linter working for the ./weaver/ subdirectories) 3. Removed dead code/unused variables 4. Adjusted `var` to `let` 5. Not perfect by any means but I'll work on some more incremental upgrades later in follow-up pull requests. --------- Rama's changes * fix(weaver-common-rs): remove outdated Cargo.lock to ensure compilation * chore(weaver-common): upgraded build depenencies for Weaver Common Protobufs in Rust * fix(weaver-relay): upgraded relay Dockerfiles for SATP support Updated relay cargo dependencies. Updated instructions in docs. * fix(weaver): bug and linter fixes in relay, Fabric driver, and Fabric SDK Ensured that TLS endpoints in the relay module had the 'https' URL prefix. Removed unnecessary variable declarations. * chore(weaver-fabric-driver): removed unnecessary variables as flagged by Yarn Co-authored-by: Rafael Belchior <rafael.belchior@tecnico.ulisboa.pt> Co-authored-by: Peter Somogyvari <peter.somogyvari@accenture.com> Co-authored-by: Sandeep Nishad <sandeep.nishad1@ibm.com> Co-authored-by: VRamakrishna <vramakr2@in.ibm.com> Signed-off-by: outsidethecode <49871473+outsidethecode@users.noreply.github.com> Signed-off-by: outsidethecode <outsidethecode@gmail.com> Signed-off-by: Rafael Belchior <rafael.belchior@tecnico.ulisboa.pt> Signed-off-by: Peter Somogyvari <peter.somogyvari@accenture.com> Signed-off-by: Sandeep Nishad <sandeep.nishad1@ibm.com> Signed-off-by: VRamakrishna <vramakr2@in.ibm.com>

…ries on build fail Signed-off-by: Sandeep Nishad <sandeep.nishad1@ibm.com>

outsidethecode requested review from petermetz, takeutak, izuru0, jagpreetsinghsasan, VRamakrishna, sandeepnRES and outSH as code owners January 22, 2024 13:19

sandeepnRES reviewed Jan 23, 2024

View reviewed changes

weaver/core/drivers/fabric-driver/server/satp.ts Outdated Show resolved Hide resolved

sandeepnRES reviewed Jan 23, 2024

View reviewed changes

weaver/core/relay/config/Dummy_Relay.toml Outdated Show resolved Hide resolved

petermetz requested changes Jan 28, 2024

View reviewed changes

.github/workflows/test_weaver-fabric-fabric-satp.yaml Outdated Show resolved Hide resolved

package-lock.json Outdated Show resolved Hide resolved

packages/cactus-plugin-ledger-connector-polkadot/src/test/rust/ink/flipper/Cargo.lock Outdated Show resolved Hide resolved

outsidethecode force-pushed the satp-squashed branch 3 times, most recently from ec2f2d7 to 982cb09 Compare February 13, 2024 13:55

outSH requested changes Feb 20, 2024

View reviewed changes

petermetz force-pushed the satp-squashed branch from 982cb09 to 6610727 Compare March 7, 2024 20:41

petermetz self-requested a review March 7, 2024 20:42

petermetz reviewed Mar 7, 2024

View reviewed changes

outSH approved these changes Apr 15, 2024

View reviewed changes

sandeepnRES reviewed Apr 15, 2024

View reviewed changes

weaver/core/relay/src/services/constants.rs Outdated Show resolved Hide resolved

sandeepnRES reviewed Apr 15, 2024

View reviewed changes

weaver/sdks/fabric/interoperation-node-sdk/src/SatpAssetManager.ts Outdated Show resolved Hide resolved

sandeepnRES reviewed Apr 15, 2024

View reviewed changes

weaver/core/relay/config/Corda_Relay.toml Outdated Show resolved Hide resolved

outsidethecode force-pushed the satp-squashed branch 2 times, most recently from a629927 to a8ed48c Compare April 29, 2024 14:28

sandeepnRES requested a review from petermetz April 29, 2024 14:29

sandeepnRES requested a review from petermetz May 8, 2024 06:18

sandeepnRES force-pushed the satp-squashed branch from 9652562 to f08a941 Compare May 8, 2024 08:42

petermetz requested changes May 8, 2024

View reviewed changes

petermetz approved these changes May 9, 2024

View reviewed changes

sandeepnRES force-pushed the satp-squashed branch 5 times, most recently from c81b1d6 to ddb072f Compare May 14, 2024 07:24

sandeepnRES mentioned this pull request May 14, 2024

refactor(relay-server): statically linking libsqlite to relay binary and revert to using busybox as base image #3260

Open

sandeepnRES force-pushed the satp-squashed branch 6 times, most recently from 3d0d730 to 7ec6f32 Compare May 14, 2024 13:10

VRamakrishna changed the title ~~feat(satp): satp first implementation (#11)~~ feat(satp): Sample implementation of SATP standard using relays (#11) May 14, 2024

sandeepnRES changed the title ~~feat(satp): Sample implementation of SATP standard using relays (#11)~~ feat(satp): sample implementation of SATP standard using relays May 15, 2024

outsidethecode and others added 2 commits May 15, 2024 05:38

chore: fix JS modules makefile to correctly delete dependency directo…

3fd54a4

…ries on build fail Signed-off-by: Sandeep Nishad <sandeep.nishad1@ibm.com>

sandeepnRES force-pushed the satp-squashed branch from 7ec6f32 to 3fd54a4 Compare May 15, 2024 05:39

sandeepnRES enabled auto-merge (rebase) May 15, 2024 05:39

sandeepnRES merged commit 69c9bf3 into hyperledger-cacti:main May 15, 2024
129 of 144 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(satp): sample implementation of SATP standard using relays #3001

feat(satp): sample implementation of SATP standard using relays #3001

outsidethecode commented Jan 22, 2024 •

edited by sandeepnRES

Loading

petermetz left a comment

outSH left a comment

petermetz commented Mar 7, 2024

petermetz commented Mar 7, 2024

petermetz left a comment

VRamakrishna commented Mar 8, 2024

outSH commented Mar 11, 2024

RafaelAPB commented Apr 9, 2024

VRamakrishna commented Apr 10, 2024 •

edited

Loading

outSH left a comment

sandeepnRES commented May 8, 2024

petermetz left a comment

sandeepnRES commented May 9, 2024 •

edited

Loading

RafaelAPB commented May 9, 2024

petermetz commented May 9, 2024

petermetz commented May 10, 2024

sandeepnRES commented May 10, 2024

petermetz commented May 10, 2024

feat(satp): sample implementation of SATP standard using relays #3001

feat(satp): sample implementation of SATP standard using relays #3001

Conversation

outsidethecode commented Jan 22, 2024 • edited by sandeepnRES Loading

petermetz left a comment

Choose a reason for hiding this comment

outSH left a comment

Choose a reason for hiding this comment

petermetz commented Mar 7, 2024

petermetz commented Mar 7, 2024

petermetz left a comment

Choose a reason for hiding this comment

VRamakrishna commented Mar 8, 2024

outSH commented Mar 11, 2024

RafaelAPB commented Apr 9, 2024

VRamakrishna commented Apr 10, 2024 • edited Loading

outSH left a comment

Choose a reason for hiding this comment

sandeepnRES commented May 8, 2024

petermetz left a comment

Choose a reason for hiding this comment

sandeepnRES commented May 9, 2024 • edited Loading

RafaelAPB commented May 9, 2024

petermetz commented May 9, 2024

petermetz commented May 10, 2024

sandeepnRES commented May 10, 2024

petermetz commented May 10, 2024

outsidethecode commented Jan 22, 2024 •

edited by sandeepnRES

Loading

VRamakrishna commented Apr 10, 2024 •

edited

Loading

sandeepnRES commented May 9, 2024 •

edited

Loading