From 4e99a4bc1a4a9cd1c16c936f80eda2f0d5c87577 Mon Sep 17 00:00:00 2001 From: Brooklyn Zelenka Date: Sun, 10 Sep 2023 17:14:17 -0700 Subject: [PATCH 01/23] Update README --- .github/CODEOWNERS | 5 ++ .github/workflows/linkcheck.yml | 14 ++++ .github/workflows/spellcheck.yml | 12 +++ README.md | 138 ++++++++++++++++++++++++++++++- 4 files changed, 168 insertions(+), 1 deletion(-) create mode 100644 .github/CODEOWNERS create mode 100644 .github/workflows/linkcheck.yml create mode 100644 .github/workflows/spellcheck.yml diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS new file mode 100644 index 0000000..951a340 --- /dev/null +++ b/.github/CODEOWNERS @@ -0,0 +1,5 @@ +# These owners will be the default owners for everything in +# the repo. Unless a later match takes precedence, +# @global-owner1 and @global-owner2 will be requested for +# review when someone opens a pull request. +* @expede @zeeshanlakhani diff --git a/.github/workflows/linkcheck.yml b/.github/workflows/linkcheck.yml new file mode 100644 index 0000000..b25de1f --- /dev/null +++ b/.github/workflows/linkcheck.yml @@ -0,0 +1,14 @@ +name: Check Markdown links + +on: push + +jobs: + markdown-link-check: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@master + - uses: gaurav-nelson/github-action-markdown-link-check@v1 + with: + use-quiet-mode: 'yes' + check-modified-files-only: 'yes' + base-branch: 'main' diff --git a/.github/workflows/spellcheck.yml b/.github/workflows/spellcheck.yml new file mode 100644 index 0000000..06710ca --- /dev/null +++ b/.github/workflows/spellcheck.yml @@ -0,0 +1,12 @@ +name: Spellcheck Action +on: push + +jobs: + build: + name: Spellcheck + runs-on: ubuntu-latest + steps: + # The checkout step + - uses: actions/checkout@master + - uses: rojopolis/spellcheck-github-actions@0.24.0 + name: Spellcheck diff --git a/README.md b/README.md index 31ba6e9..8f2254f 100644 --- a/README.md +++ b/README.md @@ -1 +1,137 @@ -# northstar \ No newline at end of file +# 🌟 North Star + +# Meta + +> If your starting-point is unknown, and your end-point and intermediate stages are woven together out of unknown material, there may be coherence, but knowledge is completely out of the question. +> +> — Plato, The Republic + +This document is an informal description of IPVM's high-level goals. It forms the basis for reasoning about the project from first principles. It expected to only rarely change. + +This is meant to provide the basis for reasoning from first principles, but it is written in prose and not first-order logic. It "merely" provides a basis for aligning the people working on the project. Principles often contain some limited internal tension: use your best judgement and discuss with your peers when ambiguities inevitably arise. + +Thix text should be kept short, to the point, and as unambiguous as possible. Define any project-specific jargon, avoid editorializing, and limit subtle language. Use diagrams and imagery where appropriate. + +# Raison d'Être + +> Had the [Web] been proprietary, and in my total control, it would probably not have taken off. You can’t propose that something be a universal space and at the same time keep control of it. +> +> — Tim Berners-Lee + +IPVM intends to do nothing less than connect all of the world's users and services. It can be thought of as "the HTTP of compute": open, interoperable, and everywhere. Following the model of the Web being the network of linked documents, IPVM is the network of linked computation. It allows anyone to permissionlessly tie into the network without prenegotiation. + +TCP/IP and HTTP were successful for a number of reasons. Networking was always possible, but doing this from scratch for every application is infeasible. Hypertext provided a single mechanism for many applications to interconnect. Application agnosticism is a major pillar of such an approach: anything can run on this substrate, and it's easy to join. IPVM works equally well with traditional Cloud-based services as it does local-first compute and trustless arhcitectures. To be successful, IPVM must have a path towards becoming substancially better than a pure Cloud architecture. It does not stand in opposition to the Cloud, but rather extends and contains it. One could describe IPVM as a trojan horse forming a smooth upgrade path bridging exising services with new techniques and philosophies. + +The internet is no longer purely client/server. Comsumer devices are now significantly more powerful than when the LAMP stack and Cloud infrastrautcure were being codeveloped. Devices today are heterogeneous: there are powerful browsers, heavy servers, microservices, edge PoPs, tiny IoT devices, smartphones, and commons networks like Bittorrent, IPFS, and blockchains. IPVM provides a way to tie applications together with robust invocation, routing, and trust layers. The volume of data that modern applications requires leads to a situation known as data gravity. Given the speed of light, the amount of data that we produce and modify grows faster than is possible to sync to all locations, and . + +# Audience + +IPVM is intended for general distributed computation, but excels at tasks that would be found in systems like Lambda Step functions. IPVM itself is focused on machine-to-machine + +# Goals + +IPVM means to provide: + +- A task matchmaking network / marketplace +- Pluggable execution engine +- A trust layer (e.g. verifiable compute, encryption, PKI, capabilities) +- Seamlessly tie together external services +- Be open to extension +- First-class P2P payments + +It is application agnostic, but intends to abstract concerns of locality away from the programmer. + +## Antigoals + +IPVM will not attempt to develop into a: + +- replacement of IPFS internals +- high level language for distributed applications +- custom Wasm runtime +- blockchain or other global consensus mechanism + +IPVM's focus is on execution, networking, data flow, and safety. It is important to make the internals debuggable and friendly where possible, but only when it doens't conflict with the core aim at this layer: efficiency and abstracting away distirbuted systems concerns. + +IPVM's primary focus is on machines and efficiency. Human-friendliness is important, but many human-oriented features should be pushed into higher layers. These may include high-level APIs, language integration, etc. + +# Approach + +- Be network-efficient (lower bandwidth and latency) + - Efficiently move data-to-compute and compute-to-data + - Reuse both code and data +- Open to participation +- Enable integration of arbitrary services without prenegotiation +- Abstract away common distributed systems challenges + +## Design Principles + +> You can prove anything you want by coldly logical reason — if you pick the proper postulates. +> +> — Asimov, "I, Robot" + +1. Proliferate! +2. Open world / extension without prenegotiation +3. Participatory: anyone can consume, serve, or both +4. Learn into our strengths: IPVM is not k8s, so don't try to be +5. Common tasks should be simple, and complex things should be possible +6. Handle low-level details (networking, failure, trust) so that others can focus on business logic +7. Build in layers with clear audiences (machine-to-machine, end-user, etc) + +When in doubt, optimize for proliferation. Get implementations into many hands, find services that already tie-in at the RPC and trust layers (UCAN), and avoid having the core IPVM team become a bottleneck for extending the network. [Richard Gabriel] described Unix and C as being like (helpful) computer viruses due to how they spread and integrate with other systems. + +IPVM simplifies distributed dataflow. It's tempting to think of it as moving single threaded local code to a logically cetralized executor, but ultimately it's lowering the barrier of entry to distributed code. + +[Homestar] is the reference implementation for the [IPVM] network. It is intended to be largely self-contained, and to run on as many systems as possible. + +## Easy for Who? + +> Simplicity is a prerequisite for reliability +> +> — Dijkstra + +> Every application has an inherent amount of complexity that cannot be removed or hidden. Instead, it must be dealt with, either in product development or in user interaction. +> +> — [Tesler's Law] + +The word "simple" is often used to mean "to make a common use case easy". This is goal directed: "simple for who or what?" IPVM aims to make distributed computing easy by abstracting away the common challenges of that context. In the same way that TCP/IP handles machine-to-machine networking concerns in an application agnostic way, IPVM doesn't attempt to enforce what can be run on it. + +Per [Tesler's Law], solving these common problems for the distributed setting does mean that something has to be traded off. Running arbitrary code designed for local-only execution doesn't work in these settings. Much like depending on techniques like read-your-writes or CRDTs at the data layer, IPVM requires low-level implementations use certain tools and techniques. Following the tactic of "do one thing well", concrns of providing familiarity is pushed above the network layet to good librariers and tools. + +# Appendix + +## Eight Fallacies of Distribiuted Computing + +IPVM intends to make computing more assessible in a distributed context. Recall the [8 Fallacies of Distributed Computing]: + +1. The network is reliable +2. Latency is zero +3. Bandwidth is infinite +4. The network is secure +5. Topology doesn't change +6. There is one administrator +7. Transport cost is zero +8. The network is homogeneous + +Some strategies include: constraining expressivity to a safe subset (similar to [structured programming]), providing a robust standard library, and __________. + +## World Wide Web Foundation's Principles + +The early web was founded on the [following principles][W3F Principles]: + +> Decentralisation: No permission is needed from a central authority to post anything on the web, there is no central controlling node, and so no single point of failure … and no “kill switch”! This also implies freedom from indiscriminate censorship and surveillance. +> +> Non-discrimination: If I pay to connect to the internet with a certain quality of service, and you pay to connect with that or a greater quality of service, then we can both communicate at the same level. This principle of equity is also known as Net Neutrality. +> +> Bottom-up design: Instead of code being written and controlled by a small group of experts, it was developed in full view of everyone, encouraging maximum participation and experimentation. +> +> Universality: For anyone to be able to publish anything on the web, all the computers involved have to speak the same languages to each other, no matter what different hardware people are using; where they live; or what cultural and political beliefs they have. In this way, the web breaks down silos while still allowing diversity to flourish. +> +> Consensus: For universal standards to work, everyone had to agree to use them. Tim and others achieved this consensus by giving everyone a say in creating the standards, through a transparent, participatory process at W3C. + + + + + +[Richard Gabriel]: https://en.wikipedia.org/wiki/Richard_P._Gabriel +[W3F Principles]: https://webfoundation.org/about/vision/history-of-the-web/ +[Tesler's Law]: https://en.wikipedia.org/wiki/Law_of_conservation_of_complexity From d8b647fab67df772703f0cd651db1504516b41c2 Mon Sep 17 00:00:00 2001 From: Brooklyn Zelenka Date: Sun, 10 Sep 2023 22:07:37 -0700 Subject: [PATCH 02/23] Fix spellcheck config --- .github/workflows/dictionary.txt | 91 ++++++++++++++++++++++++++++ .github/workflows/linkcheck.cfg.json | 10 +++ .github/workflows/linkcheck.yml | 22 ++++--- .github/workflows/spellcheck.yml | 21 ++++--- README.md | 28 ++++++--- 5 files changed, 148 insertions(+), 24 deletions(-) create mode 100644 .github/workflows/dictionary.txt create mode 100644 .github/workflows/linkcheck.cfg.json diff --git a/.github/workflows/dictionary.txt b/.github/workflows/dictionary.txt new file mode 100644 index 0000000..ddcd7a6 --- /dev/null +++ b/.github/workflows/dictionary.txt @@ -0,0 +1,91 @@ +Acknowledgements +Agoric +Akiko +Akiko's +Bacalhau +CIDs +Cap'N +Cap'n +CapTP +DAG-CBOR +DAG-JSON +ERTP +Enum +Gozalishvili +Hardt +Holmgren +IPLD +IPVM +Invoker's +Invokers +Irakli +JSON +JSONPath +JWT +JWT-encoded +Krüger +Lakhani +Lemmer-Webber +Marsen +OAuth +OCapN +Philipp +Pre-Draft +Proto +RPC +SemVer +Spritely +TTL +UCAN +UCAN's +UCAN-IPLD +UCANTO +UCANs +URI +URIs +Vagg +Varsig +WebAssembly +Worthington +Zeeshan +Zelenka +atomicity +auth +backend +base64url +canonicalized +cryptographically +dataflow +de +delegator +dholms +expede +facto +implicits +interpretable +invariants +invoker +multiformat +outmost +parallelize +pipelined +pipelining +pre-vacation +referentially +requestor's +responder +signalling +simple-but-evolvable +struct +subdelegated +subdelegation +subtype +trustless +ucanto +unlabelled +unpadded +url +v0 +validator +cryptographic +CBOR diff --git a/.github/workflows/linkcheck.cfg.json b/.github/workflows/linkcheck.cfg.json new file mode 100644 index 0000000..1d3eb1e --- /dev/null +++ b/.github/workflows/linkcheck.cfg.json @@ -0,0 +1,10 @@ +{ + "ignorePatterns": [ + { + "pattern": "https://share.ansi.org/Shared%20Documents/Standards%20Activities/American%20National%20Standards/Procedures,%20Guides,%20and%20Forms/2020_ANSI_Essential_Requirements.pdf" + }, + { + "pattern": "https://share.ansi.org/Shared%20Documents/Standards%20Activities/American%20National%20Standards/Procedures,%20Guides,%20and%20Forms/2020_ANSI_Essential_Requirements.pdf" + } + ] +} diff --git a/.github/workflows/linkcheck.yml b/.github/workflows/linkcheck.yml index b25de1f..10fa532 100644 --- a/.github/workflows/linkcheck.yml +++ b/.github/workflows/linkcheck.yml @@ -1,14 +1,22 @@ name: Check Markdown links -on: push +on: + push: + branches: + - main + pull_request: jobs: markdown-link-check: runs-on: ubuntu-latest steps: - - uses: actions/checkout@master - - uses: gaurav-nelson/github-action-markdown-link-check@v1 - with: - use-quiet-mode: 'yes' - check-modified-files-only: 'yes' - base-branch: 'main' + - uses: actions/checkout@master + - uses: gaurav-nelson/github-action-markdown-link-check@v1 + with: + use-quiet-mode: "yes" + check-modified-files-only: "yes" + base-branch: "main" + config-file: "./.github/workflows/linkcheck.cfg.json" + - uses: DavidAnson/markdownlint-cli2-action@v9 + with: + globs: "README.md" diff --git a/.github/workflows/spellcheck.yml b/.github/workflows/spellcheck.yml index 06710ca..03bf811 100644 --- a/.github/workflows/spellcheck.yml +++ b/.github/workflows/spellcheck.yml @@ -1,12 +1,17 @@ -name: Spellcheck Action -on: push +name: "spellcheck" +on: + push: + branches: + - main + pull_request: jobs: - build: - name: Spellcheck + spellcheck: runs-on: ubuntu-latest steps: - # The checkout step - - uses: actions/checkout@master - - uses: rojopolis/spellcheck-github-actions@0.24.0 - name: Spellcheck + - uses: actions/checkout@v3 + - uses: matheus23/md-spellcheck-action@v4.2.2 + with: + files-to-check: "README.md" + files-to-exclude: "LICENSE.md" # optional + words-to-ignore-file: ".github/workflows/dictionary.txt" diff --git a/README.md b/README.md index 8f2254f..975d47e 100644 --- a/README.md +++ b/README.md @@ -118,15 +118,25 @@ Some strategies include: constraining expressivity to a safe subset (similar to The early web was founded on the [following principles][W3F Principles]: -> Decentralisation: No permission is needed from a central authority to post anything on the web, there is no central controlling node, and so no single point of failure … and no “kill switch”! This also implies freedom from indiscriminate censorship and surveillance. -> -> Non-discrimination: If I pay to connect to the internet with a certain quality of service, and you pay to connect with that or a greater quality of service, then we can both communicate at the same level. This principle of equity is also known as Net Neutrality. -> -> Bottom-up design: Instead of code being written and controlled by a small group of experts, it was developed in full view of everyone, encouraging maximum participation and experimentation. -> -> Universality: For anyone to be able to publish anything on the web, all the computers involved have to speak the same languages to each other, no matter what different hardware people are using; where they live; or what cultural and political beliefs they have. In this way, the web breaks down silos while still allowing diversity to flourish. -> -> Consensus: For universal standards to work, everyone had to agree to use them. Tim and others achieved this consensus by giving everyone a say in creating the standards, through a transparent, participatory process at W3C. +### Decentralisation + +No permission is needed from a central authority to post anything on the web, there is no central controlling node, and so no single point of failure … and no “kill switch”! This also implies freedom from indiscriminate censorship and surveillance. + +### Non-discrimination + +If I pay to connect to the internet with a certain quality of service, and you pay to connect with that or a greater quality of service, then we can both communicate at the same level. This principle of equity is also known as Net Neutrality. + +### Bottom-up design + +Instead of code being written and controlled by a small group of experts, it was developed in full view of everyone, encouraging maximum participation and experimentation. + +### Universality + +For anyone to be able to publish anything on the web, all the computers involved have to speak the same languages to each other, no matter what different hardware people are using; where they live; or what cultural and political beliefs they have. In this way, the web breaks down silos while still allowing diversity to flourish. + +### Consensus + +For universal standards to work, everyone had to agree to use them. Tim and others achieved this consensus by giving everyone a say in creating the standards, through a transparent, participatory process at W3C. From aa096d6eb81f062f241afcf6971405dd5f9fd333 Mon Sep 17 00:00:00 2001 From: Brooklyn Zelenka Date: Sun, 10 Sep 2023 22:14:34 -0700 Subject: [PATCH 03/23] Remove line lenghth checks --- .github/workflows/linkcheck.yml | 3 --- 1 file changed, 3 deletions(-) diff --git a/.github/workflows/linkcheck.yml b/.github/workflows/linkcheck.yml index 10fa532..b5a0a32 100644 --- a/.github/workflows/linkcheck.yml +++ b/.github/workflows/linkcheck.yml @@ -17,6 +17,3 @@ jobs: check-modified-files-only: "yes" base-branch: "main" config-file: "./.github/workflows/linkcheck.cfg.json" - - uses: DavidAnson/markdownlint-cli2-action@v9 - with: - globs: "README.md" From 324c8fd0cc8a4980de70da5931243b846bc15326 Mon Sep 17 00:00:00 2001 From: Brooklyn Zelenka Date: Sun, 10 Sep 2023 22:21:00 -0700 Subject: [PATCH 04/23] Spelling --- .github/workflows/dictionary.txt | 23 +++++++++++++++++++++-- README.md | 18 +++++++++--------- 2 files changed, 30 insertions(+), 11 deletions(-) diff --git a/.github/workflows/dictionary.txt b/.github/workflows/dictionary.txt index ddcd7a6..2bb3efc 100644 --- a/.github/workflows/dictionary.txt +++ b/.github/workflows/dictionary.txt @@ -1,9 +1,13 @@ +APIs Acknowledgements Agoric Akiko Akiko's Bacalhau +Berners-Lee +CBOR CIDs +CRDTs Cap'N Cap'n CapTP @@ -14,10 +18,14 @@ Enum Gozalishvili Hardt Holmgren +Homestar +IPFS IPLD IPVM +IPVM's Invoker's Invokers +IoT Irakli JSON JSONPath @@ -29,13 +37,18 @@ Lemmer-Webber Marsen OAuth OCapN +PKI Philipp +PoPs Pre-Draft Proto RPC +Raison SemVer Spritely +TCP TTL +Tesler's UCAN UCAN's UCAN-IPLD @@ -45,6 +58,7 @@ URI URIs Vagg Varsig +Wasm WebAssembly Worthington Zeeshan @@ -54,23 +68,30 @@ auth backend base64url canonicalized +cryptographic cryptographically +d'Être dataflow de +debuggable delegator dholms expede +expressivity facto implicits interpretable invariants invoker +microservices multiformat outmost parallelize +permissionlessly pipelined pipelining pre-vacation +prenegotiation referentially requestor's responder @@ -87,5 +108,3 @@ unpadded url v0 validator -cryptographic -CBOR diff --git a/README.md b/README.md index 975d47e..cd2b658 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ This document is an informal description of IPVM's high-level goals. It forms th This is meant to provide the basis for reasoning from first principles, but it is written in prose and not first-order logic. It "merely" provides a basis for aligning the people working on the project. Principles often contain some limited internal tension: use your best judgement and discuss with your peers when ambiguities inevitably arise. -Thix text should be kept short, to the point, and as unambiguous as possible. Define any project-specific jargon, avoid editorializing, and limit subtle language. Use diagrams and imagery where appropriate. +This text should be kept short, to the point, and as unambiguous as possible. Define any project-specific jargon, avoid editorializing, and limit subtle language. Use diagrams and imagery where appropriate. # Raison d'Être @@ -20,9 +20,9 @@ Thix text should be kept short, to the point, and as unambiguous as possible. De IPVM intends to do nothing less than connect all of the world's users and services. It can be thought of as "the HTTP of compute": open, interoperable, and everywhere. Following the model of the Web being the network of linked documents, IPVM is the network of linked computation. It allows anyone to permissionlessly tie into the network without prenegotiation. -TCP/IP and HTTP were successful for a number of reasons. Networking was always possible, but doing this from scratch for every application is infeasible. Hypertext provided a single mechanism for many applications to interconnect. Application agnosticism is a major pillar of such an approach: anything can run on this substrate, and it's easy to join. IPVM works equally well with traditional Cloud-based services as it does local-first compute and trustless arhcitectures. To be successful, IPVM must have a path towards becoming substancially better than a pure Cloud architecture. It does not stand in opposition to the Cloud, but rather extends and contains it. One could describe IPVM as a trojan horse forming a smooth upgrade path bridging exising services with new techniques and philosophies. +TCP/IP and HTTP were successful for a number of reasons. Networking was always possible, but doing this from scratch for every application is infeasible. Hypertext provided a single mechanism for many applications to interconnect. Application agnosticism is a major pillar of such an approach: anything can run on this substrate, and it's easy to join. IPVM works equally well with traditional Cloud-based services as it does local-first compute and trustless architectures. To be successful, IPVM must have a path towards becoming substancially better than a pure Cloud architecture. It does not stand in opposition to the Cloud, but rather extends and contains it. -The internet is no longer purely client/server. Comsumer devices are now significantly more powerful than when the LAMP stack and Cloud infrastrautcure were being codeveloped. Devices today are heterogeneous: there are powerful browsers, heavy servers, microservices, edge PoPs, tiny IoT devices, smartphones, and commons networks like Bittorrent, IPFS, and blockchains. IPVM provides a way to tie applications together with robust invocation, routing, and trust layers. The volume of data that modern applications requires leads to a situation known as data gravity. Given the speed of light, the amount of data that we produce and modify grows faster than is possible to sync to all locations, and . +The internet is no longer purely client/server. Consumer devices are now significantly more powerful than when the LAMP stack and Cloud infrastrautcure were being codeveloped. Devices today are heterogeneous: there are powerful browsers, heavy servers, microservices, edge PoPs, tiny IoT devices, smartphones, and commons networks like Bittorrent, IPFS, and blockchains. IPVM provides a way to tie applications together with robust invocation, routing, and trust layers. The volume of data that modern applications requires leads to a situation known as data gravity. Given the speed of light, the amount of data that we produce and modify grows faster than is possible to sync to all locations, and . # Audience @@ -50,7 +50,7 @@ IPVM will not attempt to develop into a: - custom Wasm runtime - blockchain or other global consensus mechanism -IPVM's focus is on execution, networking, data flow, and safety. It is important to make the internals debuggable and friendly where possible, but only when it doens't conflict with the core aim at this layer: efficiency and abstracting away distirbuted systems concerns. +IPVM's focus is on execution, networking, data flow, and safety. It is important to make the internals debuggable and friendly where possible, but only when it doesn't conflict with the core aim at this layer: efficiency and abstracting away distributed systems concerns. IPVM's primary focus is on machines and efficiency. Human-friendliness is important, but many human-oriented features should be pushed into higher layers. These may include high-level APIs, language integration, etc. @@ -79,7 +79,7 @@ IPVM's primary focus is on machines and efficiency. Human-friendliness is import When in doubt, optimize for proliferation. Get implementations into many hands, find services that already tie-in at the RPC and trust layers (UCAN), and avoid having the core IPVM team become a bottleneck for extending the network. [Richard Gabriel] described Unix and C as being like (helpful) computer viruses due to how they spread and integrate with other systems. -IPVM simplifies distributed dataflow. It's tempting to think of it as moving single threaded local code to a logically cetralized executor, but ultimately it's lowering the barrier of entry to distributed code. +IPVM simplifies distributed dataflow. It's tempting to think of it as moving single threaded local code to a logically centralized executor, but ultimately it's lowering the barrier of entry to distributed code. [Homestar] is the reference implementation for the [IPVM] network. It is intended to be largely self-contained, and to run on as many systems as possible. @@ -95,13 +95,13 @@ IPVM simplifies distributed dataflow. It's tempting to think of it as moving sin The word "simple" is often used to mean "to make a common use case easy". This is goal directed: "simple for who or what?" IPVM aims to make distributed computing easy by abstracting away the common challenges of that context. In the same way that TCP/IP handles machine-to-machine networking concerns in an application agnostic way, IPVM doesn't attempt to enforce what can be run on it. -Per [Tesler's Law], solving these common problems for the distributed setting does mean that something has to be traded off. Running arbitrary code designed for local-only execution doesn't work in these settings. Much like depending on techniques like read-your-writes or CRDTs at the data layer, IPVM requires low-level implementations use certain tools and techniques. Following the tactic of "do one thing well", concrns of providing familiarity is pushed above the network layet to good librariers and tools. +Per [Tesler's Law], solving these common problems for the distributed setting does mean that something has to be traded off. Running arbitrary code designed for local-only execution doesn't work in these settings. Much like depending on techniques like read-your-writes or CRDTs at the data layer, IPVM requires low-level implementations use certain tools and techniques. Following the tactic of "do one thing well", concerns of providing familiarity is pushed above the network layer to good librariers and tools. # Appendix -## Eight Fallacies of Distribiuted Computing +## Eight Fallacies of Distributed Computing -IPVM intends to make computing more assessible in a distributed context. Recall the [8 Fallacies of Distributed Computing]: +IPVM intends to make computing more accessible in a distributed context. Recall the [8 Fallacies of Distributed Computing]: 1. The network is reliable 2. Latency is zero @@ -118,7 +118,7 @@ Some strategies include: constraining expressivity to a safe subset (similar to The early web was founded on the [following principles][W3F Principles]: -### Decentralisation +### Decentralization No permission is needed from a central authority to post anything on the web, there is no central controlling node, and so no single point of failure … and no “kill switch”! This also implies freedom from indiscriminate censorship and surveillance. From 3a5a65dc252701fc8962134c46543a7e7897ffd8 Mon Sep 17 00:00:00 2001 From: Brooklyn Zelenka Date: Sun, 10 Sep 2023 22:23:05 -0700 Subject: [PATCH 05/23] A few more --- .github/workflows/dictionary.txt | 2 ++ README.md | 8 ++++---- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/.github/workflows/dictionary.txt b/.github/workflows/dictionary.txt index 2bb3efc..e45442a 100644 --- a/.github/workflows/dictionary.txt +++ b/.github/workflows/dictionary.txt @@ -3,6 +3,7 @@ Acknowledgements Agoric Akiko Akiko's +Antigoals Bacalhau Berners-Lee CBOR @@ -39,6 +40,7 @@ OAuth OCapN PKI Philipp +Pluggable PoPs Pre-Draft Proto diff --git a/README.md b/README.md index cd2b658..eb7152f 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,7 @@ This document is an informal description of IPVM's high-level goals. It forms the basis for reasoning about the project from first principles. It expected to only rarely change. -This is meant to provide the basis for reasoning from first principles, but it is written in prose and not first-order logic. It "merely" provides a basis for aligning the people working on the project. Principles often contain some limited internal tension: use your best judgement and discuss with your peers when ambiguities inevitably arise. +This is meant to provide the basis for reasoning from first principles, but it is written in prose and not first-order logic. It "merely" provides a basis for aligning the people working on the project. Principles often contain some limited internal tension: use your best judgment and discuss with your peers when ambiguities inevitably arise. This text should be kept short, to the point, and as unambiguous as possible. Define any project-specific jargon, avoid editorializing, and limit subtle language. Use diagrams and imagery where appropriate. @@ -20,9 +20,9 @@ This text should be kept short, to the point, and as unambiguous as possible. De IPVM intends to do nothing less than connect all of the world's users and services. It can be thought of as "the HTTP of compute": open, interoperable, and everywhere. Following the model of the Web being the network of linked documents, IPVM is the network of linked computation. It allows anyone to permissionlessly tie into the network without prenegotiation. -TCP/IP and HTTP were successful for a number of reasons. Networking was always possible, but doing this from scratch for every application is infeasible. Hypertext provided a single mechanism for many applications to interconnect. Application agnosticism is a major pillar of such an approach: anything can run on this substrate, and it's easy to join. IPVM works equally well with traditional Cloud-based services as it does local-first compute and trustless architectures. To be successful, IPVM must have a path towards becoming substancially better than a pure Cloud architecture. It does not stand in opposition to the Cloud, but rather extends and contains it. +TCP/IP and HTTP were successful for a number of reasons. Networking was always possible, but doing this from scratch for every application is infeasible. Hypertext provided a single mechanism for many applications to interconnect. Application agnosticism is a major pillar of such an approach: anything can run on this substrate, and it's easy to join. IPVM works equally well with traditional Cloud-based services as it does local-first compute and trustless architectures. To be successful, IPVM must have a path towards becoming substantially better than a pure Cloud architecture. It does not stand in opposition to the Cloud, but rather extends and contains it. -The internet is no longer purely client/server. Consumer devices are now significantly more powerful than when the LAMP stack and Cloud infrastrautcure were being codeveloped. Devices today are heterogeneous: there are powerful browsers, heavy servers, microservices, edge PoPs, tiny IoT devices, smartphones, and commons networks like Bittorrent, IPFS, and blockchains. IPVM provides a way to tie applications together with robust invocation, routing, and trust layers. The volume of data that modern applications requires leads to a situation known as data gravity. Given the speed of light, the amount of data that we produce and modify grows faster than is possible to sync to all locations, and . +The internet is no longer purely client/server. Consumer devices are now significantly more powerful than when the LAMP stack and Cloud infrastrautcure were being co-developed. Devices today are heterogeneous: there are powerful browsers, heavy servers, microservices, edge PoPs, tiny IoT devices, smartphones, and commons networks like BitTorrent, IPFS, and blockchains. IPVM provides a way to tie applications together with robust invocation, routing, and trust layers. The volume of data that modern applications requires leads to a situation known as data gravity. Given the speed of light, the amount of data that we produce and modify grows faster than is possible to sync to all locations, and . # Audience @@ -95,7 +95,7 @@ IPVM simplifies distributed dataflow. It's tempting to think of it as moving sin The word "simple" is often used to mean "to make a common use case easy". This is goal directed: "simple for who or what?" IPVM aims to make distributed computing easy by abstracting away the common challenges of that context. In the same way that TCP/IP handles machine-to-machine networking concerns in an application agnostic way, IPVM doesn't attempt to enforce what can be run on it. -Per [Tesler's Law], solving these common problems for the distributed setting does mean that something has to be traded off. Running arbitrary code designed for local-only execution doesn't work in these settings. Much like depending on techniques like read-your-writes or CRDTs at the data layer, IPVM requires low-level implementations use certain tools and techniques. Following the tactic of "do one thing well", concerns of providing familiarity is pushed above the network layer to good librariers and tools. +Per [Tesler's Law], solving these common problems for the distributed setting does mean that something has to be traded off. Running arbitrary code designed for local-only execution doesn't work in these settings. Much like depending on techniques like read-your-writes or CRDTs at the data layer, IPVM requires low-level implementations use certain tools and techniques. Following the tactic of "do one thing well", concerns of providing familiarity is pushed above the network layer to good libraries and tools. # Appendix From 70918670ca1bc5d8b462e20fd1cfeefb2387216b Mon Sep 17 00:00:00 2001 From: Brooklyn Zelenka Date: Sun, 10 Sep 2023 22:35:10 -0700 Subject: [PATCH 06/23] Remove redundancies --- README.md | 24 ++++++++---------------- 1 file changed, 8 insertions(+), 16 deletions(-) diff --git a/README.md b/README.md index eb7152f..67970f5 100644 --- a/README.md +++ b/README.md @@ -22,21 +22,21 @@ IPVM intends to do nothing less than connect all of the world's users and servic TCP/IP and HTTP were successful for a number of reasons. Networking was always possible, but doing this from scratch for every application is infeasible. Hypertext provided a single mechanism for many applications to interconnect. Application agnosticism is a major pillar of such an approach: anything can run on this substrate, and it's easy to join. IPVM works equally well with traditional Cloud-based services as it does local-first compute and trustless architectures. To be successful, IPVM must have a path towards becoming substantially better than a pure Cloud architecture. It does not stand in opposition to the Cloud, but rather extends and contains it. -The internet is no longer purely client/server. Consumer devices are now significantly more powerful than when the LAMP stack and Cloud infrastrautcure were being co-developed. Devices today are heterogeneous: there are powerful browsers, heavy servers, microservices, edge PoPs, tiny IoT devices, smartphones, and commons networks like BitTorrent, IPFS, and blockchains. IPVM provides a way to tie applications together with robust invocation, routing, and trust layers. The volume of data that modern applications requires leads to a situation known as data gravity. Given the speed of light, the amount of data that we produce and modify grows faster than is possible to sync to all locations, and . +The internet is no longer purely client/server. Consumer devices are now significantly more powerful than when the LAMP stack and Cloud infrastructure were being co-developed. Devices today are heterogeneous: there are powerful browsers, heavy servers, microservices, edge PoPs, tiny IoT devices, smartphones, and commons networks like BitTorrent, IPFS, and blockchains. IPVM provides a way to tie applications together with robust invocation, routing, and trust layers. The volume of data that modern applications requires leads to a situation known as data gravity. Given the speed of light, the amount of data that we produce and modify grows faster than is possible to sync to all locations, and . # Audience -IPVM is intended for general distributed computation, but excels at tasks that would be found in systems like Lambda Step functions. IPVM itself is focused on machine-to-machine +IPVM is intended for general distributed computation, but excels at tasks that would be found in systems like Lambda Step functions. The IPVM network itself is focused on machine-to-machine interaction — where each node acts as the user's agent — but is not expected to be interacted with directly when on the happy path. # Goals IPVM means to provide: -- A task matchmaking network / marketplace +- Task matchmaking network / marketplace - Pluggable execution engine - A trust layer (e.g. verifiable compute, encryption, PKI, capabilities) - Seamlessly tie together external services -- Be open to extension +- Interfaces open to extension - First-class P2P payments It is application agnostic, but intends to abstract concerns of locality away from the programmer. @@ -54,16 +54,7 @@ IPVM's focus is on execution, networking, data flow, and safety. It is important IPVM's primary focus is on machines and efficiency. Human-friendliness is important, but many human-oriented features should be pushed into higher layers. These may include high-level APIs, language integration, etc. -# Approach - -- Be network-efficient (lower bandwidth and latency) - - Efficiently move data-to-compute and compute-to-data - - Reuse both code and data -- Open to participation -- Enable integration of arbitrary services without prenegotiation -- Abstract away common distributed systems challenges - -## Design Principles +# Design Principles > You can prove anything you want by coldly logical reason — if you pick the proper postulates. > @@ -112,7 +103,7 @@ IPVM intends to make computing more accessible in a distributed context. Recall 7. Transport cost is zero 8. The network is homogeneous -Some strategies include: constraining expressivity to a safe subset (similar to [structured programming]), providing a robust standard library, and __________. +Some strategies include: constraining expressivity to a safe subset (similar to the approach taken by [structured programming]), providing a robust standard library, and static analysis. ## World Wide Web Foundation's Principles @@ -142,6 +133,7 @@ For universal standards to work, everyone had to agree to use them. Tim and othe +[Homestar]: https://github.com/ipvm-wg/homestar/ [Richard Gabriel]: https://en.wikipedia.org/wiki/Richard_P._Gabriel -[W3F Principles]: https://webfoundation.org/about/vision/history-of-the-web/ [Tesler's Law]: https://en.wikipedia.org/wiki/Law_of_conservation_of_complexity +[W3F Principles]: https://webfoundation.org/about/vision/history-of-the-web/ From db07128585532fedadd6b820fa04f49f17f1770c Mon Sep 17 00:00:00 2001 From: Brooklyn Zelenka Date: Sun, 10 Sep 2023 22:48:33 -0700 Subject: [PATCH 07/23] Tighten up a few sections --- README.md | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index 67970f5..acf2b3a 100644 --- a/README.md +++ b/README.md @@ -16,7 +16,7 @@ This text should be kept short, to the point, and as unambiguous as possible. De > Had the [Web] been proprietary, and in my total control, it would probably not have taken off. You can’t propose that something be a universal space and at the same time keep control of it. > -> — Tim Berners-Lee +> — Tim Berners-Lee, [FAQ][TBL FAQ] IPVM intends to do nothing less than connect all of the world's users and services. It can be thought of as "the HTTP of compute": open, interoperable, and everywhere. Following the model of the Web being the network of linked documents, IPVM is the network of linked computation. It allows anyone to permissionlessly tie into the network without prenegotiation. @@ -39,18 +39,18 @@ IPVM means to provide: - Interfaces open to extension - First-class P2P payments -It is application agnostic, but intends to abstract concerns of locality away from the programmer. +IPVM is application agnostic, but intends to abstract concerns of locality away from the programmer. ## Antigoals -IPVM will not attempt to develop into a: +IPVM will _not_ attempt to develop into a: -- replacement of IPFS internals +- replacement or upgrade of IPFS internals - high level language for distributed applications - custom Wasm runtime - blockchain or other global consensus mechanism -IPVM's focus is on execution, networking, data flow, and safety. It is important to make the internals debuggable and friendly where possible, but only when it doesn't conflict with the core aim at this layer: efficiency and abstracting away distributed systems concerns. +IPVM's focus is on execution, networking, data flow, and trust. It is important to make the internals debuggable and friendly where possible, but only when it doesn't conflict with the core aim at this layer: efficiency and abstracting away distributed systems concerns. IPVM's primary focus is on machines and efficiency. Human-friendliness is important, but many human-oriented features should be pushed into higher layers. These may include high-level APIs, language integration, etc. @@ -64,15 +64,17 @@ IPVM's primary focus is on machines and efficiency. Human-friendliness is import 2. Open world / extension without prenegotiation 3. Participatory: anyone can consume, serve, or both 4. Learn into our strengths: IPVM is not k8s, so don't try to be -5. Common tasks should be simple, and complex things should be possible +5. Common things should be simple, and complex things should be possible 6. Handle low-level details (networking, failure, trust) so that others can focus on business logic 7. Build in layers with clear audiences (machine-to-machine, end-user, etc) -When in doubt, optimize for proliferation. Get implementations into many hands, find services that already tie-in at the RPC and trust layers (UCAN), and avoid having the core IPVM team become a bottleneck for extending the network. [Richard Gabriel] described Unix and C as being like (helpful) computer viruses due to how they spread and integrate with other systems. +When in doubt, optimize for proliferation. Get implementations into many hands, find services that already tie-in at the RPC and trust layers ([UCAN]), and avoid having the core IPVM team become a bottleneck for extending the network. [Richard Gabriel] described Unix and C as being like (helpful) computer viruses due to how they spread and integrate with other systems. -IPVM simplifies distributed dataflow. It's tempting to think of it as moving single threaded local code to a logically centralized executor, but ultimately it's lowering the barrier of entry to distributed code. +IPVM simplifies distributed dataflow. It's tempting to think of it as moving unconstrained centralized code to a logically centralized executor, but IPVM actually lowers the barrier of entry to distributed code. A disrtibuted context is fundamentally different from writing code for a single machine[^Lamport's Problem]. -[Homestar] is the reference implementation for the [IPVM] network. It is intended to be largely self-contained, and to run on as many systems as possible. +[Homestar] is the reference implementation of the IPVM protocol. Homestar is intended to be largely self-contained, and to run on as many systems as possible. + +[^Lamport's Problem]: As Leslie Lamport says, "a distributed system is one in which the failure of a machine you have never heard of can cause your own machine to become unusable". ## Easy for Who? @@ -135,5 +137,7 @@ For universal standards to work, everyone had to agree to use them. Tim and othe [Homestar]: https://github.com/ipvm-wg/homestar/ [Richard Gabriel]: https://en.wikipedia.org/wiki/Richard_P._Gabriel +[TBL FAQ]:https://www.w3.org/People/Berners-Lee/FAQ.html [Tesler's Law]: https://en.wikipedia.org/wiki/Law_of_conservation_of_complexity +[UCAN]: https://github.com/ucan-wg [W3F Principles]: https://webfoundation.org/about/vision/history-of-the-web/ From aeb347b3aa7955ab5f72a52adb33bf24a2beed80 Mon Sep 17 00:00:00 2001 From: Brooklyn Zelenka Date: Mon, 11 Sep 2023 15:48:11 -0700 Subject: [PATCH 08/23] Fix footnote --- README.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index acf2b3a..8efc72a 100644 --- a/README.md +++ b/README.md @@ -70,11 +70,11 @@ IPVM's primary focus is on machines and efficiency. Human-friendliness is import When in doubt, optimize for proliferation. Get implementations into many hands, find services that already tie-in at the RPC and trust layers ([UCAN]), and avoid having the core IPVM team become a bottleneck for extending the network. [Richard Gabriel] described Unix and C as being like (helpful) computer viruses due to how they spread and integrate with other systems. -IPVM simplifies distributed dataflow. It's tempting to think of it as moving unconstrained centralized code to a logically centralized executor, but IPVM actually lowers the barrier of entry to distributed code. A disrtibuted context is fundamentally different from writing code for a single machine[^Lamport's Problem]. +IPVM simplifies distributed dataflow. It's tempting to think of it as moving unconstrained centralized code to a logically centralized executor, but IPVM actually lowers the barrier of entry to distributed code. A disrtibuted context is fundamentally different from writing code for a single machine[^LamportsProblem]. [Homestar] is the reference implementation of the IPVM protocol. Homestar is intended to be largely self-contained, and to run on as many systems as possible. -[^Lamport's Problem]: As Leslie Lamport says, "a distributed system is one in which the failure of a machine you have never heard of can cause your own machine to become unusable". +[^LamportsProblem]: As Leslie Lamport says, "a distributed system is one in which the failure of a machine you have never heard of can cause your own machine to become unusable". ## Easy for Who? @@ -141,3 +141,4 @@ For universal standards to work, everyone had to agree to use them. Tim and othe [Tesler's Law]: https://en.wikipedia.org/wiki/Law_of_conservation_of_complexity [UCAN]: https://github.com/ucan-wg [W3F Principles]: https://webfoundation.org/about/vision/history-of-the-web/ +[structured programming]: https://en.wikipedia.org/wiki/Structured_programming From 6fc1ee93f60f936d5b74252d4ed5c605254e5cb9 Mon Sep 17 00:00:00 2001 From: Brooklyn Zelenka Date: Mon, 11 Sep 2023 15:52:32 -0700 Subject: [PATCH 09/23] Fix a couple links --- README.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 8efc72a..43e805b 100644 --- a/README.md +++ b/README.md @@ -20,7 +20,7 @@ This text should be kept short, to the point, and as unambiguous as possible. De IPVM intends to do nothing less than connect all of the world's users and services. It can be thought of as "the HTTP of compute": open, interoperable, and everywhere. Following the model of the Web being the network of linked documents, IPVM is the network of linked computation. It allows anyone to permissionlessly tie into the network without prenegotiation. -TCP/IP and HTTP were successful for a number of reasons. Networking was always possible, but doing this from scratch for every application is infeasible. Hypertext provided a single mechanism for many applications to interconnect. Application agnosticism is a major pillar of such an approach: anything can run on this substrate, and it's easy to join. IPVM works equally well with traditional Cloud-based services as it does local-first compute and trustless architectures. To be successful, IPVM must have a path towards becoming substantially better than a pure Cloud architecture. It does not stand in opposition to the Cloud, but rather extends and contains it. +TCP/IP and HTTP were successful for a number of reasons. Networking was always possible, but doing this from scratch for every application is infeasible. Hypertext provided a single mechanism for many applications to interconnect. Application agnosticism is a major pillar of [such an approach][WWW Foundation Principles]: anything can run on this substrate, and it's easy to join. IPVM works equally well with traditional Cloud-based services as it does local-first compute and trustless architectures. To be successful, IPVM must have a path towards becoming substantially better than a pure Cloud architecture. It does not stand in opposition to the Cloud, but rather extends and contains it. The internet is no longer purely client/server. Consumer devices are now significantly more powerful than when the LAMP stack and Cloud infrastructure were being co-developed. Devices today are heterogeneous: there are powerful browsers, heavy servers, microservices, edge PoPs, tiny IoT devices, smartphones, and commons networks like BitTorrent, IPFS, and blockchains. IPVM provides a way to tie applications together with robust invocation, routing, and trust layers. The volume of data that modern applications requires leads to a situation known as data gravity. Given the speed of light, the amount of data that we produce and modify grows faster than is possible to sync to all locations, and . @@ -92,9 +92,9 @@ Per [Tesler's Law], solving these common problems for the distributed setting do # Appendix -## Eight Fallacies of Distributed Computing +## Fallacies of Distributed Computing -IPVM intends to make computing more accessible in a distributed context. Recall the [8 Fallacies of Distributed Computing]: +IPVM intends to make computing more accessible in a distributed context. Recall the [Fallacies of Distributed Computing]: 1. The network is reliable 2. Latency is zero @@ -133,8 +133,11 @@ For universal standards to work, everyone had to agree to use them. Tim and othe +[WWW Foundation Principles]: #world-wide-web-foundations-principles + +[Fallacies of Distributed Computing]: https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing [Homestar]: https://github.com/ipvm-wg/homestar/ [Richard Gabriel]: https://en.wikipedia.org/wiki/Richard_P._Gabriel [TBL FAQ]:https://www.w3.org/People/Berners-Lee/FAQ.html From cd996edfdd1b3bde7cc0f8608c8a886250c0cec1 Mon Sep 17 00:00:00 2001 From: Brooklyn Zelenka Date: Mon, 11 Sep 2023 15:55:45 -0700 Subject: [PATCH 10/23] Remove appendix, move to links --- README.md | 47 ++--------------------------------------------- 1 file changed, 2 insertions(+), 45 deletions(-) diff --git a/README.md b/README.md index 43e805b..2810545 100644 --- a/README.md +++ b/README.md @@ -20,7 +20,7 @@ This text should be kept short, to the point, and as unambiguous as possible. De IPVM intends to do nothing less than connect all of the world's users and services. It can be thought of as "the HTTP of compute": open, interoperable, and everywhere. Following the model of the Web being the network of linked documents, IPVM is the network of linked computation. It allows anyone to permissionlessly tie into the network without prenegotiation. -TCP/IP and HTTP were successful for a number of reasons. Networking was always possible, but doing this from scratch for every application is infeasible. Hypertext provided a single mechanism for many applications to interconnect. Application agnosticism is a major pillar of [such an approach][WWW Foundation Principles]: anything can run on this substrate, and it's easy to join. IPVM works equally well with traditional Cloud-based services as it does local-first compute and trustless architectures. To be successful, IPVM must have a path towards becoming substantially better than a pure Cloud architecture. It does not stand in opposition to the Cloud, but rather extends and contains it. +TCP/IP and HTTP were successful for a number of reasons. Networking was always possible, but doing this from scratch for every application is infeasible. Hypertext provided a single mechanism for many applications to interconnect. Application agnosticism is a major pillar of [such an approach][W3F Principles]: anything can run on this substrate, and it's easy to join. IPVM works equally well with traditional Cloud-based services as it does local-first compute and trustless architectures. To be successful, IPVM must have a path towards becoming substantially better than a pure Cloud architecture. It does not stand in opposition to the Cloud, but rather extends and contains it. The internet is no longer purely client/server. Consumer devices are now significantly more powerful than when the LAMP stack and Cloud infrastructure were being co-developed. Devices today are heterogeneous: there are powerful browsers, heavy servers, microservices, edge PoPs, tiny IoT devices, smartphones, and commons networks like BitTorrent, IPFS, and blockchains. IPVM provides a way to tie applications together with robust invocation, routing, and trust layers. The volume of data that modern applications requires leads to a situation known as data gravity. Given the speed of light, the amount of data that we produce and modify grows faster than is possible to sync to all locations, and . @@ -88,53 +88,10 @@ IPVM simplifies distributed dataflow. It's tempting to think of it as moving unc The word "simple" is often used to mean "to make a common use case easy". This is goal directed: "simple for who or what?" IPVM aims to make distributed computing easy by abstracting away the common challenges of that context. In the same way that TCP/IP handles machine-to-machine networking concerns in an application agnostic way, IPVM doesn't attempt to enforce what can be run on it. -Per [Tesler's Law], solving these common problems for the distributed setting does mean that something has to be traded off. Running arbitrary code designed for local-only execution doesn't work in these settings. Much like depending on techniques like read-your-writes or CRDTs at the data layer, IPVM requires low-level implementations use certain tools and techniques. Following the tactic of "do one thing well", concerns of providing familiarity is pushed above the network layer to good libraries and tools. - -# Appendix - -## Fallacies of Distributed Computing - -IPVM intends to make computing more accessible in a distributed context. Recall the [Fallacies of Distributed Computing]: - -1. The network is reliable -2. Latency is zero -3. Bandwidth is infinite -4. The network is secure -5. Topology doesn't change -6. There is one administrator -7. Transport cost is zero -8. The network is homogeneous - -Some strategies include: constraining expressivity to a safe subset (similar to the approach taken by [structured programming]), providing a robust standard library, and static analysis. - -## World Wide Web Foundation's Principles - -The early web was founded on the [following principles][W3F Principles]: - -### Decentralization - -No permission is needed from a central authority to post anything on the web, there is no central controlling node, and so no single point of failure … and no “kill switch”! This also implies freedom from indiscriminate censorship and surveillance. - -### Non-discrimination - -If I pay to connect to the internet with a certain quality of service, and you pay to connect with that or a greater quality of service, then we can both communicate at the same level. This principle of equity is also known as Net Neutrality. - -### Bottom-up design - -Instead of code being written and controlled by a small group of experts, it was developed in full view of everyone, encouraging maximum participation and experimentation. - -### Universality - -For anyone to be able to publish anything on the web, all the computers involved have to speak the same languages to each other, no matter what different hardware people are using; where they live; or what cultural and political beliefs they have. In this way, the web breaks down silos while still allowing diversity to flourish. - -### Consensus - -For universal standards to work, everyone had to agree to use them. Tim and others achieved this consensus by giving everyone a say in creating the standards, through a transparent, participatory process at W3C. +Per [Tesler's Law], solving these [common problems for the distributed setting][Fallacies of Distributed Computing] does mean that something has to be traded off. Running arbitrary code designed for local-only execution doesn't work in these settings. Much like depending on techniques like read-your-writes or CRDTs at the data layer, IPVM requires low-level implementations use certain tools and techniques. Following the tactic of "do one thing well", concerns of providing familiarity is pushed above the network layer to good libraries and tools. -[WWW Foundation Principles]: #world-wide-web-foundations-principles - [Fallacies of Distributed Computing]: https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing From 7d9502eb603a38248cf7b64a8203b4bff69abc40 Mon Sep 17 00:00:00 2001 From: Brooklyn Zelenka Date: Mon, 11 Sep 2023 15:57:52 -0700 Subject: [PATCH 11/23] Update dictionary --- .github/workflows/dictionary.txt | 2 ++ README.md | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/.github/workflows/dictionary.txt b/.github/workflows/dictionary.txt index e45442a..1572c9a 100644 --- a/.github/workflows/dictionary.txt +++ b/.github/workflows/dictionary.txt @@ -34,6 +34,8 @@ JWT JWT-encoded Krüger Lakhani +Lamport +LamportsProblem Lemmer-Webber Marsen OAuth diff --git a/README.md b/README.md index 2810545..3d0f625 100644 --- a/README.md +++ b/README.md @@ -70,7 +70,7 @@ IPVM's primary focus is on machines and efficiency. Human-friendliness is import When in doubt, optimize for proliferation. Get implementations into many hands, find services that already tie-in at the RPC and trust layers ([UCAN]), and avoid having the core IPVM team become a bottleneck for extending the network. [Richard Gabriel] described Unix and C as being like (helpful) computer viruses due to how they spread and integrate with other systems. -IPVM simplifies distributed dataflow. It's tempting to think of it as moving unconstrained centralized code to a logically centralized executor, but IPVM actually lowers the barrier of entry to distributed code. A disrtibuted context is fundamentally different from writing code for a single machine[^LamportsProblem]. +IPVM simplifies distributed dataflow. It's tempting to think of it as moving unconstrained centralized code to a logically centralized executor, but IPVM actually lowers the barrier of entry to distributed code. A distributed context is fundamentally different from writing code for a single machine[^LamportsProblem]. [Homestar] is the reference implementation of the IPVM protocol. Homestar is intended to be largely self-contained, and to run on as many systems as possible. From fc5dbfbe7d307d036685137bbc083fd6fc5861e2 Mon Sep 17 00:00:00 2001 From: Brooklyn Zelenka Date: Mon, 11 Sep 2023 16:00:06 -0700 Subject: [PATCH 12/23] Cite source --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 3d0f625..9874a37 100644 --- a/README.md +++ b/README.md @@ -80,7 +80,7 @@ IPVM simplifies distributed dataflow. It's tempting to think of it as moving unc > Simplicity is a prerequisite for reliability > -> — Dijkstra +> — Dijkstra, [How do we tell truths that might hurt?] > Every application has an inherent amount of complexity that cannot be removed or hidden. Instead, it must be dealt with, either in product development or in user interaction. > @@ -96,6 +96,7 @@ Per [Tesler's Law], solving these [common problems for the distributed setting][ [Fallacies of Distributed Computing]: https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing [Homestar]: https://github.com/ipvm-wg/homestar/ +[How do we tell truths that might hurt?]:https://www.cs.virginia.edu/~evans/cs655/readings/ewd498.html [Richard Gabriel]: https://en.wikipedia.org/wiki/Richard_P._Gabriel [TBL FAQ]:https://www.w3.org/People/Berners-Lee/FAQ.html [Tesler's Law]: https://en.wikipedia.org/wiki/Law_of_conservation_of_complexity From bff135726455c1099f836e1ceaa0177902fd996d Mon Sep 17 00:00:00 2001 From: Brooklyn Zelenka Date: Mon, 11 Sep 2023 23:47:40 -0700 Subject: [PATCH 13/23] Apply a few of ZL's suggestions --- README.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 9874a37..7e586ee 100644 --- a/README.md +++ b/README.md @@ -49,8 +49,9 @@ IPVM will _not_ attempt to develop into a: - high level language for distributed applications - custom Wasm runtime - blockchain or other global consensus mechanism +- server orchestration -IPVM's focus is on execution, networking, data flow, and trust. It is important to make the internals debuggable and friendly where possible, but only when it doesn't conflict with the core aim at this layer: efficiency and abstracting away distributed systems concerns. +IPVM's focus is on execution, networking, dataflow, and trust. It is important to make the internals debuggable and friendly where possible, but only when it doesn't conflict with the core aim at this layer: efficiency and abstracting away distributed systems concerns. IPVM's primary focus is on machines and efficiency. Human-friendliness is important, but many human-oriented features should be pushed into higher layers. These may include high-level APIs, language integration, etc. @@ -68,7 +69,7 @@ IPVM's primary focus is on machines and efficiency. Human-friendliness is import 6. Handle low-level details (networking, failure, trust) so that others can focus on business logic 7. Build in layers with clear audiences (machine-to-machine, end-user, etc) -When in doubt, optimize for proliferation. Get implementations into many hands, find services that already tie-in at the RPC and trust layers ([UCAN]), and avoid having the core IPVM team become a bottleneck for extending the network. [Richard Gabriel] described Unix and C as being like (helpful) computer viruses due to how they spread and integrate with other systems. +When in doubt, optimize for proliferation. Get implementations into many hands, find services that already tie-in at the RPC and trust layers ([UCAN]), and avoid having the core IPVM team become a bottleneck for extending the network. [Richard Gabriel] described [Unix and C as being like (helpful) computer viruses][Modles of Software Acceptance] due to how they spread and integrate with other systems. IPVM simplifies distributed dataflow. It's tempting to think of it as moving unconstrained centralized code to a logically centralized executor, but IPVM actually lowers the barrier of entry to distributed code. A distributed context is fundamentally different from writing code for a single machine[^LamportsProblem]. @@ -86,7 +87,7 @@ IPVM simplifies distributed dataflow. It's tempting to think of it as moving unc > > — [Tesler's Law] -The word "simple" is often used to mean "to make a common use case easy". This is goal directed: "simple for who or what?" IPVM aims to make distributed computing easy by abstracting away the common challenges of that context. In the same way that TCP/IP handles machine-to-machine networking concerns in an application agnostic way, IPVM doesn't attempt to enforce what can be run on it. +The word "simple" is often used to mean "to make a common use case easy". This is goal directed: "simple for who or what?" IPVM aims to make distributed computing easy by [abstracting away][Fault Oblivious] the common challenges of that context. In the same way that TCP/IP handles machine-to-machine networking concerns in an application agnostic way, IPVM doesn't attempt to enforce what can be run on it. Per [Tesler's Law], solving these [common problems for the distributed setting][Fallacies of Distributed Computing] does mean that something has to be traded off. Running arbitrary code designed for local-only execution doesn't work in these settings. Much like depending on techniques like read-your-writes or CRDTs at the data layer, IPVM requires low-level implementations use certain tools and techniques. Following the tactic of "do one thing well", concerns of providing familiarity is pushed above the network layer to good libraries and tools. @@ -95,8 +96,10 @@ Per [Tesler's Law], solving these [common problems for the distributed setting][ [Fallacies of Distributed Computing]: https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing +[Fault Oblivious]: https://rebelsky.cs.grinnell.edu/~rebelsky/Courses/CS302/Fun/fault-oblivious.html [Homestar]: https://github.com/ipvm-wg/homestar/ [How do we tell truths that might hurt?]:https://www.cs.virginia.edu/~evans/cs655/readings/ewd498.html +[Models of Software Acceptance]: https://dreamsongs.com/Files/AcceptanceModels.pdf [Richard Gabriel]: https://en.wikipedia.org/wiki/Richard_P._Gabriel [TBL FAQ]:https://www.w3.org/People/Berners-Lee/FAQ.html [Tesler's Law]: https://en.wikipedia.org/wiki/Law_of_conservation_of_complexity From 6ec54c50d27db5c49057fd0bb0fe00342cf9bf27 Mon Sep 17 00:00:00 2001 From: Brooklyn Zelenka Date: Mon, 11 Sep 2023 23:52:36 -0700 Subject: [PATCH 14/23] Remove redundant bullet point --- README.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/README.md b/README.md index 7e586ee..3e25aad 100644 --- a/README.md +++ b/README.md @@ -33,7 +33,6 @@ IPVM is intended for general distributed computation, but excels at tasks that w IPVM means to provide: - Task matchmaking network / marketplace -- Pluggable execution engine - A trust layer (e.g. verifiable compute, encryption, PKI, capabilities) - Seamlessly tie together external services - Interfaces open to extension @@ -69,7 +68,7 @@ IPVM's primary focus is on machines and efficiency. Human-friendliness is import 6. Handle low-level details (networking, failure, trust) so that others can focus on business logic 7. Build in layers with clear audiences (machine-to-machine, end-user, etc) -When in doubt, optimize for proliferation. Get implementations into many hands, find services that already tie-in at the RPC and trust layers ([UCAN]), and avoid having the core IPVM team become a bottleneck for extending the network. [Richard Gabriel] described [Unix and C as being like (helpful) computer viruses][Modles of Software Acceptance] due to how they spread and integrate with other systems. +When in doubt, optimize for proliferation. Get implementations into many hands, find services that already tie-in at the RPC and trust layers ([UCAN]), and avoid having the core IPVM team become a bottleneck for extending the network. [Richard Gabriel] described [Unix and C as being like (helpful) computer viruses][Models of Software Acceptance] due to how they spread and integrate with other systems. IPVM simplifies distributed dataflow. It's tempting to think of it as moving unconstrained centralized code to a logically centralized executor, but IPVM actually lowers the barrier of entry to distributed code. A distributed context is fundamentally different from writing code for a single machine[^LamportsProblem]. From 631b5e61ed74dc2039df695b9bf50d285ca8dee2 Mon Sep 17 00:00:00 2001 From: Brooklyn Zelenka Date: Tue, 12 Sep 2023 00:04:13 -0700 Subject: [PATCH 15/23] Fix where it . --- README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 3e25aad..a6785f8 100644 --- a/README.md +++ b/README.md @@ -22,7 +22,9 @@ IPVM intends to do nothing less than connect all of the world's users and servic TCP/IP and HTTP were successful for a number of reasons. Networking was always possible, but doing this from scratch for every application is infeasible. Hypertext provided a single mechanism for many applications to interconnect. Application agnosticism is a major pillar of [such an approach][W3F Principles]: anything can run on this substrate, and it's easy to join. IPVM works equally well with traditional Cloud-based services as it does local-first compute and trustless architectures. To be successful, IPVM must have a path towards becoming substantially better than a pure Cloud architecture. It does not stand in opposition to the Cloud, but rather extends and contains it. -The internet is no longer purely client/server. Consumer devices are now significantly more powerful than when the LAMP stack and Cloud infrastructure were being co-developed. Devices today are heterogeneous: there are powerful browsers, heavy servers, microservices, edge PoPs, tiny IoT devices, smartphones, and commons networks like BitTorrent, IPFS, and blockchains. IPVM provides a way to tie applications together with robust invocation, routing, and trust layers. The volume of data that modern applications requires leads to a situation known as data gravity. Given the speed of light, the amount of data that we produce and modify grows faster than is possible to sync to all locations, and . +The internet is no longer purely client/server. Consumer devices are now significantly more powerful than when the LAMP stack and Cloud infrastructure were being co-developed. Devices today are heterogeneous: there are powerful browsers, heavy servers, microservices, edge PoPs, tiny IoT devices, smartphones, and commons networks like BitTorrent, IPFS, and blockchains. IPVM provides a way to tie applications together with robust invocation, routing, and trust layers. + +Finally: the volume of data that modern applications requires leads to a situation known as data gravity. Given the speed of light, the amount of data that we produce and modify grows faster than is possible to sync to all locations, and the rate of change makes it physically ipossible to ever catch up. There are also still many cases where the data is small and.or infrequent enough to be practical to move around on the network. To meet the needs of such a heterogeneous network, IPVM provides a flexible model that takes into account the [costs and tradeoffs of distributed computing][Fallacies of Distributed Computing] by ad hoc moving data to where the code lives, as well as ad hoc moving code to where the data lives. # Audience From d27203c55a6a5a0014c6f31ebd881b5c4759af15 Mon Sep 17 00:00:00 2001 From: Brooklyn Zelenka Date: Tue, 12 Sep 2023 00:07:27 -0700 Subject: [PATCH 16/23] Link to AWS Lambda --- README.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index a6785f8..daf5bf1 100644 --- a/README.md +++ b/README.md @@ -28,7 +28,7 @@ Finally: the volume of data that modern applications requires leads to a situati # Audience -IPVM is intended for general distributed computation, but excels at tasks that would be found in systems like Lambda Step functions. The IPVM network itself is focused on machine-to-machine interaction — where each node acts as the user's agent — but is not expected to be interacted with directly when on the happy path. +IPVM is intended for general distributed computation, but excels at tasks that would be found in systems like [Lambda Step functions]. The IPVM network itself is focused on machine-to-machine interaction — where each node acts as the user's agent — but is not expected to be interacted with directly when on the happy path. # Goals @@ -50,7 +50,7 @@ IPVM will _not_ attempt to develop into a: - high level language for distributed applications - custom Wasm runtime - blockchain or other global consensus mechanism -- server orchestration +- a web server framework IPVM's focus is on execution, networking, dataflow, and trust. It is important to make the internals debuggable and friendly where possible, but only when it doesn't conflict with the core aim at this layer: efficiency and abstracting away distributed systems concerns. @@ -100,6 +100,7 @@ Per [Tesler's Law], solving these [common problems for the distributed setting][ [Fault Oblivious]: https://rebelsky.cs.grinnell.edu/~rebelsky/Courses/CS302/Fun/fault-oblivious.html [Homestar]: https://github.com/ipvm-wg/homestar/ [How do we tell truths that might hurt?]:https://www.cs.virginia.edu/~evans/cs655/readings/ewd498.html +[Lambda Step functions]: https://docs.aws.amazon.com/lambda/latest/dg/lambda-stepfunctions.html [Models of Software Acceptance]: https://dreamsongs.com/Files/AcceptanceModels.pdf [Richard Gabriel]: https://en.wikipedia.org/wiki/Richard_P._Gabriel [TBL FAQ]:https://www.w3.org/People/Berners-Lee/FAQ.html From c7ee1395e4c511ad393e559c2a500518e60f6dd9 Mon Sep 17 00:00:00 2001 From: Brooklyn Zelenka Date: Tue, 12 Sep 2023 00:10:24 -0700 Subject: [PATCH 17/23] And I would have gotten away with it too, if it weren't for that meddling spellcheck --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index daf5bf1..64b79e4 100644 --- a/README.md +++ b/README.md @@ -24,7 +24,7 @@ TCP/IP and HTTP were successful for a number of reasons. Networking was always p The internet is no longer purely client/server. Consumer devices are now significantly more powerful than when the LAMP stack and Cloud infrastructure were being co-developed. Devices today are heterogeneous: there are powerful browsers, heavy servers, microservices, edge PoPs, tiny IoT devices, smartphones, and commons networks like BitTorrent, IPFS, and blockchains. IPVM provides a way to tie applications together with robust invocation, routing, and trust layers. -Finally: the volume of data that modern applications requires leads to a situation known as data gravity. Given the speed of light, the amount of data that we produce and modify grows faster than is possible to sync to all locations, and the rate of change makes it physically ipossible to ever catch up. There are also still many cases where the data is small and.or infrequent enough to be practical to move around on the network. To meet the needs of such a heterogeneous network, IPVM provides a flexible model that takes into account the [costs and tradeoffs of distributed computing][Fallacies of Distributed Computing] by ad hoc moving data to where the code lives, as well as ad hoc moving code to where the data lives. +Finally: the volume of data that modern applications requires leads to a situation known as data gravity. Given the speed of light, the amount of data that we produce and modify grows faster than is possible to sync to all locations, and the rate of change makes it physically impossible to ever catch up. There are also still many cases where the data is small and.or infrequent enough to be practical to move around on the network. To meet the needs of such a heterogeneous network, IPVM provides a flexible model that takes into account the [costs and trade-offs of distributed computing][Fallacies of Distributed Computing] by ad hoc moving data to where the code lives, as well as ad hoc moving code to where the data lives. # Audience From 59b4d2b04b94bb1023aff8afeabacff2ef3952b6 Mon Sep 17 00:00:00 2001 From: Brooklyn Zelenka Date: Tue, 12 Sep 2023 09:47:23 -0700 Subject: [PATCH 18/23] Add more than Lambda --- README.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 64b79e4..874b452 100644 --- a/README.md +++ b/README.md @@ -28,7 +28,7 @@ Finally: the volume of data that modern applications requires leads to a situati # Audience -IPVM is intended for general distributed computation, but excels at tasks that would be found in systems like [Lambda Step functions]. The IPVM network itself is focused on machine-to-machine interaction — where each node acts as the user's agent — but is not expected to be interacted with directly when on the happy path. +IPVM is intended for general distributed computation, but excels at tasks that would be found in systems like [AWS Lambda Step functions], [Fastly Compute@Edge], and [Temporal Cloud]. The IPVM network itself is focused on machine-to-machine interaction — where each node acts as the user's agent — but is not expected to be interacted with directly when on the happy path. # Goals @@ -96,11 +96,13 @@ Per [Tesler's Law], solving these [common problems for the distributed setting][ +[Temporal Cloud]: https://temporal.io/ +[AWS Lambda Step functions]: https://docs.aws.amazon.com/lambda/latest/dg/lambda-stepfunctions.html [Fallacies of Distributed Computing]: https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing +[Fastly Compute@Edge]: https://docs.fastly.com/products/compute-at-edge [Fault Oblivious]: https://rebelsky.cs.grinnell.edu/~rebelsky/Courses/CS302/Fun/fault-oblivious.html [Homestar]: https://github.com/ipvm-wg/homestar/ [How do we tell truths that might hurt?]:https://www.cs.virginia.edu/~evans/cs655/readings/ewd498.html -[Lambda Step functions]: https://docs.aws.amazon.com/lambda/latest/dg/lambda-stepfunctions.html [Models of Software Acceptance]: https://dreamsongs.com/Files/AcceptanceModels.pdf [Richard Gabriel]: https://en.wikipedia.org/wiki/Richard_P._Gabriel [TBL FAQ]:https://www.w3.org/People/Berners-Lee/FAQ.html From b45c4ff1613c2586203edb87cbe7bce2d35d9161 Mon Sep 17 00:00:00 2001 From: Brooklyn Zelenka Date: Tue, 12 Sep 2023 10:25:29 -0700 Subject: [PATCH 19/23] WIP --- README.md | 55 ++++++++++++++++++++++++++++++++++--------------------- 1 file changed, 34 insertions(+), 21 deletions(-) diff --git a/README.md b/README.md index 874b452..c3227d3 100644 --- a/README.md +++ b/README.md @@ -16,7 +16,7 @@ This text should be kept short, to the point, and as unambiguous as possible. De > Had the [Web] been proprietary, and in my total control, it would probably not have taken off. You can’t propose that something be a universal space and at the same time keep control of it. > -> — Tim Berners-Lee, [FAQ][TBL FAQ] +> — Berners-Lee, [FAQ][TBL FAQ] IPVM intends to do nothing less than connect all of the world's users and services. It can be thought of as "the HTTP of compute": open, interoperable, and everywhere. Following the model of the Web being the network of linked documents, IPVM is the network of linked computation. It allows anyone to permissionlessly tie into the network without prenegotiation. @@ -30,6 +30,33 @@ Finally: the volume of data that modern applications requires leads to a situati IPVM is intended for general distributed computation, but excels at tasks that would be found in systems like [AWS Lambda Step functions], [Fastly Compute@Edge], and [Temporal Cloud]. The IPVM network itself is focused on machine-to-machine interaction — where each node acts as the user's agent — but is not expected to be interacted with directly when on the happy path. +A natural first step is to run on servers (similar to a FaaS), but + +## Easy for Whom? + +> Every application has an inherent amount of complexity that cannot be removed or hidden. Instead, it must be dealt with, either in product development or in user interaction. +> +> — [Tesler's Law] + +> I think this notion of nearness is really, really cool. [...] There's sort of the physical notion of being near. Right? Is something, you know, right there. And I think that's where the root of the word came from. You know, this is easy to obtain because it's nearby. It's not in the next town. [...] we do sort of have, you know, our own hard drive or our own toolset, or it's sort of the ability to make things physically near by getting them through things like installers and stuff like that. +> +> The second notion of nearness is something being near to our understanding, right, or in our current skill set. And I don't mean in this case near to our understanding meaning a capability. I mean literally near something that we already know. So the word in this case is about being familiar. +> +> I think that, collectively, we are infatuated with these two notions of easy. We are just so self-involved in these two aspects; it's hurting us tremendously. Right? All we care about is, you know, can I get this instantly and start running it in five seconds? It could be this giant hairball that you got, but all you care is, you know, can you get it. +> +> Hickey, [Simple Made Easy] + +IPVM + + + + + + +The word "simple" is often used to mean "to make a common use case easy". This is goal directed: "simple for who or what?" IPVM aims to make distributed computing easy by [abstracting away][Fault Oblivious] the common challenges of that context. In the same way that TCP/IP handles machine-to-machine networking concerns in an application agnostic way, IPVM doesn't attempt to enforce what can be run on it. + +Per [Tesler's Law], solving these [common problems for the distributed setting][Fallacies of Distributed Computing] does mean that something has to be traded off. Running arbitrary code designed for local-only execution doesn't work in these settings. Much like depending on techniques like read-your-writes or CRDTs at the data layer, IPVM requires low-level implementations use certain tools and techniques. Following the tactic of "do one thing well", concerns of providing familiarity is pushed above the network layer to good libraries and tools. + # Goals IPVM means to provide: @@ -39,8 +66,9 @@ IPVM means to provide: - Seamlessly tie together external services - Interfaces open to extension - First-class P2P payments - -IPVM is application agnostic, but intends to abstract concerns of locality away from the programmer. +- Run everywhere (servers, browsers, mobile applications, etc) +- Application agnostic +- Abstract concerns of locality away from the programmer ## Antigoals @@ -52,9 +80,7 @@ IPVM will _not_ attempt to develop into a: - blockchain or other global consensus mechanism - a web server framework -IPVM's focus is on execution, networking, dataflow, and trust. It is important to make the internals debuggable and friendly where possible, but only when it doesn't conflict with the core aim at this layer: efficiency and abstracting away distributed systems concerns. - -IPVM's primary focus is on machines and efficiency. Human-friendliness is important, but many human-oriented features should be pushed into higher layers. These may include high-level APIs, language integration, etc. +IPVM focuses on execution, networking, dataflow, and trust. It is important to make the internals debuggable and friendly where possible, but only when it doesn't conflict with the core aim at this layer: efficiency and abstracting away distributed systems concerns. Human-friendliness is important, but many human-oriented features should be pushed into higher layers. These may include high-level APIs, language integration, etc. # Design Principles @@ -78,25 +104,10 @@ IPVM simplifies distributed dataflow. It's tempting to think of it as moving unc [^LamportsProblem]: As Leslie Lamport says, "a distributed system is one in which the failure of a machine you have never heard of can cause your own machine to become unusable". -## Easy for Who? - -> Simplicity is a prerequisite for reliability -> -> — Dijkstra, [How do we tell truths that might hurt?] - -> Every application has an inherent amount of complexity that cannot be removed or hidden. Instead, it must be dealt with, either in product development or in user interaction. -> -> — [Tesler's Law] - -The word "simple" is often used to mean "to make a common use case easy". This is goal directed: "simple for who or what?" IPVM aims to make distributed computing easy by [abstracting away][Fault Oblivious] the common challenges of that context. In the same way that TCP/IP handles machine-to-machine networking concerns in an application agnostic way, IPVM doesn't attempt to enforce what can be run on it. - -Per [Tesler's Law], solving these [common problems for the distributed setting][Fallacies of Distributed Computing] does mean that something has to be traded off. Running arbitrary code designed for local-only execution doesn't work in these settings. Much like depending on techniques like read-your-writes or CRDTs at the data layer, IPVM requires low-level implementations use certain tools and techniques. Following the tactic of "do one thing well", concerns of providing familiarity is pushed above the network layer to good libraries and tools. - -[Temporal Cloud]: https://temporal.io/ [AWS Lambda Step functions]: https://docs.aws.amazon.com/lambda/latest/dg/lambda-stepfunctions.html [Fallacies of Distributed Computing]: https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing [Fastly Compute@Edge]: https://docs.fastly.com/products/compute-at-edge @@ -105,7 +116,9 @@ Per [Tesler's Law], solving these [common problems for the distributed setting][ [How do we tell truths that might hurt?]:https://www.cs.virginia.edu/~evans/cs655/readings/ewd498.html [Models of Software Acceptance]: https://dreamsongs.com/Files/AcceptanceModels.pdf [Richard Gabriel]: https://en.wikipedia.org/wiki/Richard_P._Gabriel +[Simple Made Easy]: https://www.infoq.com/presentations/Simple-Made-Easy/ [TBL FAQ]:https://www.w3.org/People/Berners-Lee/FAQ.html +[Temporal Cloud]: https://temporal.io/ [Tesler's Law]: https://en.wikipedia.org/wiki/Law_of_conservation_of_complexity [UCAN]: https://github.com/ucan-wg [W3F Principles]: https://webfoundation.org/about/vision/history-of-the-web/ From dec0a916e982aa926d4b8e5686283e21a3fb7d14 Mon Sep 17 00:00:00 2001 From: Brooklyn Zelenka Date: Tue, 12 Sep 2023 12:32:21 -0700 Subject: [PATCH 20/23] Cut and rearrange a bunch of audience text --- README.md | 25 +++++-------------------- 1 file changed, 5 insertions(+), 20 deletions(-) diff --git a/README.md b/README.md index c3227d3..0563e6f 100644 --- a/README.md +++ b/README.md @@ -28,34 +28,19 @@ Finally: the volume of data that modern applications requires leads to a situati # Audience -IPVM is intended for general distributed computation, but excels at tasks that would be found in systems like [AWS Lambda Step functions], [Fastly Compute@Edge], and [Temporal Cloud]. The IPVM network itself is focused on machine-to-machine interaction — where each node acts as the user's agent — but is not expected to be interacted with directly when on the happy path. - -A natural first step is to run on servers (similar to a FaaS), but - -## Easy for Whom? - > Every application has an inherent amount of complexity that cannot be removed or hidden. Instead, it must be dealt with, either in product development or in user interaction. > > — [Tesler's Law] -> I think this notion of nearness is really, really cool. [...] There's sort of the physical notion of being near. Right? Is something, you know, right there. And I think that's where the root of the word came from. You know, this is easy to obtain because it's nearby. It's not in the next town. [...] we do sort of have, you know, our own hard drive or our own toolset, or it's sort of the ability to make things physically near by getting them through things like installers and stuff like that. -> -> The second notion of nearness is something being near to our understanding, right, or in our current skill set. And I don't mean in this case near to our understanding meaning a capability. I mean literally near something that we already know. So the word in this case is about being familiar. -> -> I think that, collectively, we are infatuated with these two notions of easy. We are just so self-involved in these two aspects; it's hurting us tremendously. Right? All we care about is, you know, can I get this instantly and start running it in five seconds? It could be this giant hairball that you got, but all you care is, you know, can you get it. +> I think that, collectively, we are infatuated with these two notions of easy ["near-at-hand" and "in your current skill set"]. We are just so self-involved in these two aspects; it's hurting us tremendously. > -> Hickey, [Simple Made Easy] - -IPVM - - - - +> — Hickey, [Simple Made Easy] +IPVM is intended for general distributed computation, but excels at tasks that would be found in systems like [AWS Lambda Step functions], [Fastly Compute@Edge], and [Temporal Cloud]. The IPVM network itself is focused on machine-to-machine interaction — where each node acts as the user's agent — but is not expected to be interacted with directly when on the happy path. -The word "simple" is often used to mean "to make a common use case easy". This is goal directed: "simple for who or what?" IPVM aims to make distributed computing easy by [abstracting away][Fault Oblivious] the common challenges of that context. In the same way that TCP/IP handles machine-to-machine networking concerns in an application agnostic way, IPVM doesn't attempt to enforce what can be run on it. +The word "simple" is often used interchangably with "easy" to mean making a common use case require little effort. This is goal directed: "easy for who or what?" IPVM aims to make distributed computing easy by [abstracting away][Fault Oblivious] the common challenges of that context. In the same way that TCP/IP handles machine-to-machine networking concerns in an application agnostic way, IPVM doesn't attempt to enforce which kinds of services can be interconnected. -Per [Tesler's Law], solving these [common problems for the distributed setting][Fallacies of Distributed Computing] does mean that something has to be traded off. Running arbitrary code designed for local-only execution doesn't work in these settings. Much like depending on techniques like read-your-writes or CRDTs at the data layer, IPVM requires low-level implementations use certain tools and techniques. Following the tactic of "do one thing well", concerns of providing familiarity is pushed above the network layer to good libraries and tools. +Standardizing on Wasm also highlights IPVM's core audience. At time of writing, there is comparatively little Wasm written; but the belief is that will change quickly. IPVM uses Wasm in its emphasis to decompose compute into simpler units that are composable, cachable, reusable, and tractable for a distributed setting. Per [Tesler's Law], solving these [common problems for the distributed setting][Fallacies of Distributed Computing] means that something else has to be traded off. Running arbitrary code designed for local-only execution doesn't work in these settings. Much like depending on techniques like read-your-writes or CRDTs at the data layer, IPVM requires low-level implementations use certain tools and techniques. Following the tactic of "do one thing well", concerns of providing familiarity is pushed above the network layer to good libraries and tools. # Goals From 90c9eeead39d008da842de8b83de37572fabd409 Mon Sep 17 00:00:00 2001 From: Brooklyn Zelenka Date: Tue, 12 Sep 2023 12:35:21 -0700 Subject: [PATCH 21/23] Fix much spelling --- .github/workflows/dictionary.txt | 4 +++- README.md | 6 +++--- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/.github/workflows/dictionary.txt b/.github/workflows/dictionary.txt index 1572c9a..6bd6d70 100644 --- a/.github/workflows/dictionary.txt +++ b/.github/workflows/dictionary.txt @@ -1,3 +1,5 @@ +composable +dataflow APIs Acknowledgements Agoric @@ -16,6 +18,7 @@ DAG-CBOR DAG-JSON ERTP Enum +Fastly Gozalishvili Hardt Holmgren @@ -75,7 +78,6 @@ canonicalized cryptographic cryptographically d'Être -dataflow de debuggable delegator diff --git a/README.md b/README.md index 0563e6f..d7a4660 100644 --- a/README.md +++ b/README.md @@ -38,9 +38,9 @@ Finally: the volume of data that modern applications requires leads to a situati IPVM is intended for general distributed computation, but excels at tasks that would be found in systems like [AWS Lambda Step functions], [Fastly Compute@Edge], and [Temporal Cloud]. The IPVM network itself is focused on machine-to-machine interaction — where each node acts as the user's agent — but is not expected to be interacted with directly when on the happy path. -The word "simple" is often used interchangably with "easy" to mean making a common use case require little effort. This is goal directed: "easy for who or what?" IPVM aims to make distributed computing easy by [abstracting away][Fault Oblivious] the common challenges of that context. In the same way that TCP/IP handles machine-to-machine networking concerns in an application agnostic way, IPVM doesn't attempt to enforce which kinds of services can be interconnected. +The word "simple" is often used interchangeably with "easy" to mean making a common use case require little effort. This is goal directed: "easy for who or what?" IPVM aims to make distributed computing easy by [abstracting away][Fault Oblivious] the common challenges of that context. In the same way that TCP/IP handles machine-to-machine networking concerns in an application agnostic way, IPVM doesn't attempt to enforce which kinds of services can be interconnected. -Standardizing on Wasm also highlights IPVM's core audience. At time of writing, there is comparatively little Wasm written; but the belief is that will change quickly. IPVM uses Wasm in its emphasis to decompose compute into simpler units that are composable, cachable, reusable, and tractable for a distributed setting. Per [Tesler's Law], solving these [common problems for the distributed setting][Fallacies of Distributed Computing] means that something else has to be traded off. Running arbitrary code designed for local-only execution doesn't work in these settings. Much like depending on techniques like read-your-writes or CRDTs at the data layer, IPVM requires low-level implementations use certain tools and techniques. Following the tactic of "do one thing well", concerns of providing familiarity is pushed above the network layer to good libraries and tools. +Standardizing on Wasm also highlights IPVM's core audience. At time of writing, there is comparatively little Wasm written; but the belief is that will change quickly. IPVM uses Wasm in its emphasis to decompose compute into simpler units that are composable, cacheable, reusable, and tractable for a distributed setting. Per [Tesler's Law], solving these [common problems for the distributed setting][Fallacies of Distributed Computing] means that something else has to be traded off. Running arbitrary code designed for local-only execution doesn't work in these settings. Much like depending on techniques like read-your-writes or CRDTs at the data layer, IPVM requires low-level implementations use certain tools and techniques. Following the tactic of "do one thing well", concerns of providing familiarity is pushed above the network layer to good libraries and tools. # Goals @@ -81,7 +81,7 @@ IPVM focuses on execution, networking, dataflow, and trust. It is important to m 6. Handle low-level details (networking, failure, trust) so that others can focus on business logic 7. Build in layers with clear audiences (machine-to-machine, end-user, etc) -When in doubt, optimize for proliferation. Get implementations into many hands, find services that already tie-in at the RPC and trust layers ([UCAN]), and avoid having the core IPVM team become a bottleneck for extending the network. [Richard Gabriel] described [Unix and C as being like (helpful) computer viruses][Models of Software Acceptance] due to how they spread and integrate with other systems. +When in doubt, optimize for proliferation. Get implementations into many hands, find services that already tie-in at the RPC and trust layers ([UCAN]), and avoid having the core IPVM team become a bottleneck for extending the network. Unix and C have been describes as being like [(helpful) computer viruses][Models of Software Acceptance] due to how they spread and integrate with other systems. IPVM simplifies distributed dataflow. It's tempting to think of it as moving unconstrained centralized code to a logically centralized executor, but IPVM actually lowers the barrier of entry to distributed code. A distributed context is fundamentally different from writing code for a single machine[^LamportsProblem]. From 6206f49f3368a22c12d4a88a42ac62bab58e7a90 Mon Sep 17 00:00:00 2001 From: Brooklyn Zelenka Date: Tue, 12 Sep 2023 12:37:28 -0700 Subject: [PATCH 22/23] Add to dictionary --- .github/workflows/dictionary.txt | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/.github/workflows/dictionary.txt b/.github/workflows/dictionary.txt index 6bd6d70..d72e53a 100644 --- a/.github/workflows/dictionary.txt +++ b/.github/workflows/dictionary.txt @@ -1,5 +1,3 @@ -composable -dataflow APIs Acknowledgements Agoric @@ -74,10 +72,13 @@ atomicity auth backend base64url +cacheable canonicalized +composable cryptographic cryptographically d'Être +dataflow de debuggable delegator From 016bae3e1ba72a1979802e260597b531df89dc3c Mon Sep 17 00:00:00 2001 From: Brooklyn Zelenka Date: Tue, 12 Sep 2023 15:54:59 -0700 Subject: [PATCH 23/23] ZL's nits --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index d7a4660..60f3ab8 100644 --- a/README.md +++ b/README.md @@ -48,10 +48,10 @@ IPVM means to provide: - Task matchmaking network / marketplace - A trust layer (e.g. verifiable compute, encryption, PKI, capabilities) -- Seamlessly tie together external services +- The ability to tie together external services seamlessly - Interfaces open to extension - First-class P2P payments -- Run everywhere (servers, browsers, mobile applications, etc) +- The ability to run everywhere (servers, browsers, mobile applications, etc) - Application agnostic - Abstract concerns of locality away from the programmer