diff --git a/docs/charon/charon-cli-reference.md b/docs/charon/charon-cli-reference.md index df2b760710..020e207413 100644 --- a/docs/charon/charon-cli-reference.md +++ b/docs/charon/charon-cli-reference.md @@ -11,7 +11,7 @@ The `charon` client is under heavy development, interfaces are subject to change ::: -The following is a reference for charon version [`v0.16.0`](https://github.com/ObolNetwork/charon/releases/tag/v0.16.0). Find the latest release on [our Github](https://github.com/ObolNetwork/charon/releases). +The following is a reference for charon version [`v0.17.0`](https://github.com/ObolNetwork/charon/releases/tag/v0.17.0). Find the latest release on [our Github](https://github.com/ObolNetwork/charon/releases). The following are the top-level commands available to use. @@ -81,15 +81,13 @@ Flags: `charon create cluster` creates a set of distributed validators locally, including the private keys, a `cluster-lock.json` file, and deposit and exit data. However, this command should only be used for solo use of distributed validators. To run a Distributed Validator with a group of operators, it is preferable to create these artifacts using the `charon dkg` command. That way, no single operator custodies all of the private keys to a distributed validator. ```markdown -charon create cluster --help Creates a local charon cluster configuration including validator keys, charon p2p keys, cluster-lock.json and a deposit-data.json. See flags for supported features. Usage: charon create cluster [flags] Flags: - --clean Delete the cluster directory before generating it. - --cluster-dir string The target folder to create the cluster in. (default ".charon/cluster") + --cluster-dir string The target folder to create the cluster in. (default "./") --definition-file string Optional path to a cluster definition file or an HTTP URL. This overrides all other configuration flags. --fee-recipient-addresses strings Comma separated list of Ethereum addresses of the fee recipient for each validator. Either provide a single fee recipient address or fee recipient addresses for each validator. -h, --help Help for cluster @@ -97,9 +95,9 @@ Flags: --keymanager-addresses strings Comma separated list of keymanager URLs to import validator key shares to. Note that multiple addresses are required, one for each node in the cluster, with node0's keyshares being imported to the first address, node1's keyshares to the second, and so on. --keymanager-auth-tokens strings Authentication bearer tokens to interact with the keymanager URLs. Don't include the "Bearer" symbol, only include the api-token. --name string The cluster name - --network string Ethereum network to create validators for. Options: mainnet, gnosis, goerli, kiln, ropsten, sepolia. (default "mainnet") - --nodes int The number of charon nodes in the cluster. Minimum is 3. (default 4) - --num-validators int The number of distributed validators needed in the cluster. (default 1) + --network string Ethereum network to create validators for. Options: mainnet, goerli, gnosis, sepolia. + --nodes int The number of charon nodes in the cluster. Minimum is 3. + --num-validators int The number of distributed validators needed in the cluster. --publish Publish lock file to obol-api. --publish-address string The URL to publish the lock file to. (default "https://api.obol.tech") --split-existing-keys Split an existing validator's private key into a set of distributed validator private key shares. Does not re-create deposit data for this key. diff --git a/docs/charon/cluster-configuration.md b/docs/charon/cluster-configuration.md index 56565254a1..aab4104033 100644 --- a/docs/charon/cluster-configuration.md +++ b/docs/charon/cluster-configuration.md @@ -157,8 +157,5 @@ BFT #: max number of faulty (byzantine) nodes given size n f(n) = floor((n-1)/3) CFT #: max number of unavailable (crashed) nodes given size n -crashed(n) = n - Quarom(n) +crashed(n) = n - Quorum(n) ``` - - - diff --git a/docs/int/faq/errors.mdx b/docs/int/faq/errors.mdx index 8a89a39607..f7ac92f8f1 100644 --- a/docs/int/faq/errors.mdx +++ b/docs/int/faq/errors.mdx @@ -277,6 +277,17 @@ docker compose logs msgSigAgg indicates that BLS threshold aggregation of sufficient partial signatures failed. This indicates inconsistent signed data. This indicates a bug in charon as it is unexpected. +
+ +

+ Existing private key lock file found, another charon instance may be running on your machine error +

+ + When you turn on the --private-key-file-lock option in Charon, it checks for a special file called the private key lock file. This file has the same name as the ENR private key file but with a .lock extension. + If the private key lock file exists and is not older than 5 seconds, Charon won't run. It doesn't allow running multiple Charon instances with the same ENR private key. + If the private key lock file has a timestamp older than 5 seconds, Charon will replace it and continue with its work. + If you're sure that no other Charon instances are running, you can delete the private key lock file. +
diff --git a/docs/int/faq/risks.md b/docs/int/faq/risks.md index ef6f988108..e2af412935 100644 --- a/docs/int/faq/risks.md +++ b/docs/int/faq/risks.md @@ -5,17 +5,17 @@ description: Centralization Risks and mitigation # Centralization risks and mitigation -# Risk: Obol hosting the relay infrastructure +## Risk: Obol hosting the relay infrastructure **Mitigation**: Self-host a relay One of the risks associated with Obol hosting the [LibP2P relays](docs/charon/networking.md) infrastructure allowing peer discovery is that if Obol-hosted relays go down, peers won't be able to discover each other and perform the DKG. To mitigate this risk, external organizations and node operators can consider self-hosting a relay. This way, if Obol's relays go down, the clusters can still operate through other relays in the network. -# Risk: Obol being able to update Charon code -**Mitigation**: Pin specific versions +## Risk: Obol being able to update Charon code +**Mitigation**: Pin specific docker versions or compile from source on a trusted commit -Another risk associated with Obol is having the ability to update the [Charon code](https://github.com/ObolNetwork/charon) running on the network which could introduce vulnerabilities or malicious code. To mitigate this risk, Obol can consider pinning specific versions of the code that have been thoroughly tested and accepted by the network. This would ensure that any updates are carefully vetted and reviewed by the community. +Another risk associated with Obol is having the ability to update the [Charon code](https://github.com/ObolNetwork/charon) running on the network which could introduce vulnerabilities or malicious code. To mitigate this risk, operators can consider pinning specific versions of the code that have been thoroughly tested and accepted by the network. This would ensure that any updates are carefully vetted and reviewed by the community. -# Risk: Obol hosting the DV Launchpad +## Risk: Obol hosting the DV Launchpad **Mitigation**: Use [`create cluster`](docs/charon/charon-cli-reference.md) or [`create dkg`](docs/charon/charon-cli-reference.md) locally and distribute the files manually Hosting the first Charon frontend, the [DV Launchpad](docs/dvl/intro.md), on a centralized server could create a single point of failure, as users would have to rely on Obol's server to access the protocol. This could limit the decentralization of the protocol and could make it vulnerable to attacks or downtime. Obol hosting the launchpad on a decentralized network, such as IPFS is a first step but not enough. This is why the Charon code is open-source and contains a CLI interface to interact with the protocol locally. @@ -23,7 +23,7 @@ Hosting the first Charon frontend, the [DV Launchpad](docs/dvl/intro.md), on a c To mitigate the risk of launchpad failure, consider using the `create cluster` or `create dkg` commands locally and distributing the key shares files manually. -# Risk: Obol going bust/rogue +## Risk: Obol going bust/rogue **Mitigation**: Use key recovery The final centralization risk associated with Obol is the possibility of the company going bankrupt or acting maliciously, which would lead to a loss of control over the network and potentially cause damage to the ecosystem. To mitigate this risk, Obol has implemented a key recovery mechanism. This would allow the clusters to continue operating and to retrieve full private keys even if Obol is no longer able to provide support. diff --git a/docs/int/faq/threat_model.md b/docs/int/faq/threat_model.md new file mode 100644 index 0000000000..0fc2a5931d --- /dev/null +++ b/docs/int/faq/threat_model.md @@ -0,0 +1,155 @@ +--- +sidebar_position: 4 +description: Threat model for a Distributed Validator +--- + +# Charon threat model + +This page outlines a threat model for Charon, in the context of it being a Distributed Validator middleware for Ethereum validator clients. + +## Actors + +- Node owner (NO) +- Cluster node operators (CNO) +- Rogue node operator (RNO) +- Outside attacker (OA) + +## General observations + +This page describes some considerations the Obol core team made about the security of a distributed validator in the context of its deployment and interaction with outside actors. + +The goal of this threat model is to provide transparency, but it is by no means a comprehensive audit or complete security reference. It’s a sharing of the experiences and thoughts we gained during the last few years building distributed validator technologies. + +While to the Beacon Chain, a distributed validator is seen in much the same way as a regular validator, and thus retains some of the same security considerations, Charon’s threat model is different from a validator client’s threat model because of its general design. + +While a validator client owns and operates on a set of validator private keys, the design of Charon allows its node operators to rarely (if ever) see the complete validator private keys, relying instead on modern cryptography to generate partial private key shares. + +An Ethereum distributed validator employs advanced signature primitives such that no operator ever handles the full validator private key in any standard lifecycle step: the [BLS digital signature scheme](https://en.wikipedia.org/wiki/BLS_digital_signature) employed by the Ethereum network allows distributed validators to individually sign a blob of data and then aggregate the resulting signatures in a transparent manner, never requiring any of the participating parties to know the full private key to do so. + +If the subset of the available Charon nodes is lower than a given threshold, the cluster is not able to continue with its duties. + +Given the collaborative nature of a Distributed Validator cluster, every operator must prioritize the liveliness and well-being of the cluster. Charon, at the moment of writing this page cannot reward and penalize operators within a cluster independently. + +This implies that Charon’s threat model can’t quite be equated to that of a single validator client, since they work on a different - albeit similar - set of security concepts. + +## Identity private key + +A distributed validator cluster is made up of a number of nodes, often run by a number of independent operators. For each DV cluster there’s a set of Ethereum validator private keys on which they want to validate on behalf of. + +Alongside those, each node (henceforth ‘operator’) holds an SECP256K1 identity private key, referred to as an ENR, that identifies their node to the other cluster operators’ nodes. + +Exfiltration of said private key could lead to possible impersonation from an outside attacker, possibly leading to intra-cluster peering issues, eclipse attack risks, and degraded validator performance. + +Charon client communication is handled via BFT consensus, which is able to tolerate a given number of misbehaving nodes up to a certain threshold: once this threshold is reached, the cluster is not able to continue with its lifecycle and loses liveness guarantees (the validator goes offline). If more than two-thirds of nodes in a cluster are malicious, a cluster also loses safety guarantees (enough bad actors could collude to come to consensus on something slashable). + +Identity private key theft and the subsequent execution of a rogue cluster node is equivalent in the context of BFT consensus to a misbehaving node, hence the cluster can survive and continue with its duties up to what’s specified by the cluster’s BFT protocol’s parameters. + +The likelihood of this happening is low: an OA with enough knowledge of the topology of the operator’s network must steal `fault tolerance of the cluster + 1` identity private keys and run Charon nodes to subvert the distributed validator BFT consensus to push the validator offline. + +## Ethereum validator private key access + +A distributed validator cluster executes Ethereum validator duties by acting as a middleman between the beacon chain and a validator client. + +To do so, the cluster must have knowledge of the Ethereum validator’s private key. + +The design and implementation of Charon minimizes the chances of this by splitting the Ethereum validator private keys into parts, which are then assigned to each node operator. +A [distributed key generation](https://en.wikipedia.org/wiki/Distributed_key_generation) (DKG) process is used in order to evenly and safely create the private key shares without any central party having access to the full private key. + +The cryptography primitives employed in Charon can allow a threshold of the node operator’s private key shares to be reconstructed into the whole validator private key if needed. + +While the facilities to do this are present in the form of CLI commands, as stated before Charon never reconstructs the key in normal operations since BLS digital signature system allows for signature aggregation. + +A distributed validator cluster can be started in two ways: + +1. An existing Ethereum validator private key is split by the private key holder, and distributed in a trusted manner among the operators. +2. The operators participate in a distributed key generation (DKG) process, to create private key shares that collectively can be used to sign validation duties as an Ethereum distributed validator. The full private key for the cluster never exists in one place during or after the DKG. + +In case 1, one of the node operators K has direct access to the Ethereum validator key and is tasked with the generation of other operator’s identity keys and key shards. + +It is clear that in this case the entirety of the sensitive material set is as secure as K’s environment; if K is compromised or malicious, the distributed validator could be slashed. + +Case 2 is different, because there’s no pre-existing Ethereum validator key in a single operator's hands: it will be generated using the FROST DKG algorithm. + +Assuming a successful DKG process, each operator will only ever handle its own key shares instead of the full Ethereum validator private key. + +A set of rogue operators composed of enough members to reconstruct the original Ethereum private keys might pose the risk of slashing for a distributed validator by colluding to produce slashable messages together. + +We deem this scenario’s likelihood as low, as it would mean that node operators decided to willfully slash the stake that they should be being rewarded for staking. + +Still, in the context of an outside attack, purposefully slashing a validator would mean stealing multiple operator key shares, which in turn means violating many cluster operator’s security almost at the same time. This scenario may occur if there is a 0-day vulnerability in a piece of software they all run or in case of node misconfiguration. + +## Rogue node operator + +Nodes are connected by means of either relay nodes, or directly to one another. + +Each node operator is at risk of being impeded by other nodes or by the relay operator in the execution of their duties. + +Nodes need to expose a set of TCP ports to be able to work, and the mere fact of doing that opens up the opportunity for rogue parties to execute DDoS attacks. + +Another attack surface for the cluster exists in rogue nodes purposefully filling the various inter-state databases with meaningless data, or more generally submitting bogus information to the other parties to slow down the processing or, in the case of a sybil attack, bring the cluster to a halt. + +The likelihood of this scenario is medium, because there’s no active threat hunting part: there’s no need for the rogue node operator to penetrate and compromise other nodes to disturb the cluster’s lifecycle. + +## Outside attackers interfering with a cluster + +There are two levels of sophistication in an OA: + +1. No knowledge of the topology of the cluster: The attacker doesn’t know where each cluster node is located and so can’t force fault tolerance +1 nodes offline if it can’t find them. +2. Knowledge of the topology of the network (or part of it) is possessed: the OA can operate DDoS attacks or try breaking into node’s servers - at that point, the “rogue node operator” scenario applies. + +The likelihood of this scenario is low: an OA needs extensive capabilities and sufficient incentive to be able to carry out an attack of this size. + +An outside attacker could also find and use vulnerabilities in the underlying cryptosystems and cryptography libraries used by Charon and other Ethereum clients. Forging signatures that fool Charon’s cryptographic library or other dependencies may be feasible, but forging signatures or otherwise finding a vulnerability in either the SECP256K1+ECDSA or BLS12-381+BLS cryptosystems we deem to be a low likelihood risk. + +## Malicious beacon nodes + +A malicious beacon node (BN) could prevent the distributed validator from operating its validation duties, and could plausibly increase the likelihood of slashing by serving charon illegitimate information. + +If the amount of nodes configured with the malicious BN are equal to the byzantine threshold for the Charon BFT consensus protocol, the validation process can potentially halt since the BFT parameter threshold is reached - most of the nodes are byzantine - the system will reach consensus on a set of data that isn’t valid. + +We deem the likelihood of this scenario to be medium depending on the trust model associated with the BNs deployment (cloud, self-hosted, SaaS product): node operators should always host or at least trust their own beacon nodes. + +## Malicious charon relays + +A Charon relay is used as a communication bridge between nodes that aren’t directly exposed on the Internet. It also acts as the peer discovery mechanism for a cluster. + +Once a peer’s IP address has been discovered via the relay, a direct connection can be attempted. Nodes can either communicate by exchanging data through a relay, or by using the relay as a means to establish a direct TCP connection to one another. + +A malicious relay owned by a OA could lead to: + +- Network topology discovery, facilitating the “outside attackers interactions with a cluster” scenario +- Impeding node communication, potentially impacting the BFT consensus protocol liveness (not security) and distributed validator duties +- DKG process disruption leading to frustration and potential abandonment by node operators: could lead to the usage of a standard Ethereum validator setup, which implies weaker security overall + +We note that BFT consensus liveness disruption can only happen if the number of nodes using the malicious relay for communication is equal to the byzantine nodes amount defined in the consensus parameters. + +This risk can be mitigated by configuring nodes with multiple relay URLs from only [trusted entities](../quickstart/advanced/self-relay.md). + +The likelihood of this scenario is medium: Charon nodes are configured with a default set of relay nodes, so if an OA were to compromise those, it would lead to many cluster topologies getting discovered and potentially attacked and disrupted. + +## Compromised runtime files + +Charon operates with two runtime files: + +- A lock file used to address operator’s nodes, define the Ethereum validator public keys and the public key shares associated with it +- A cluster definition file used to define the operator’s addresses and identities during the DKG process + +The lock file is signed and validated by all the nodes participating in the cluster: assuming good security practices on the node operator side, and no bugs in Charon or its dependencies’ implementations, this scenario is unlikely. + +If one or more node operators are using less than ideal security practices an OA could rewire the Charon CLI flags to include the `--no-verify` flags, which disables lock file signature and hash verification (usually intended only for development purposes). + +By doing that, the OA can edit the lock file as it sees fit, leading to the “rogue node operator” scenario. An OA or RNO might also manage to social engineer their way into convincing other operators into running their malicious lock file with verification disabled. + +The likelihood of this scenario is low: an OA would need to compromise every node operator through social engineering to both use a different set of files, and to run its cluster with `--no-verify`. + +## Conclusions + +Distributed Validator Technology (DVT) helps maintain a high-assurance environment for Ethereum validators by leveraging modern cryptography to ensure no single point of failure is easily found in the system. + +As with any computing system, security considerations are to be expected in order to keep the environment safe. + +From the point of view of an Ethereum validator entity, running their services with a DV client can help greatly with availability, minimizing slashing risks, and maximizing participation in the network. + +On the other hand, one must take into consideration the risks involved with dishonest cluster operators, as well as rogue third-party beacon nodes or relay providers. + +In the end, we believe the benefits of DVT greatly outweigh the potential threats described in this overview. diff --git a/docs/int/quickstart/activate-dv.md b/docs/int/quickstart/activate-dv.md index e77b18667d..e4b778f254 100644 --- a/docs/int/quickstart/activate-dv.md +++ b/docs/int/quickstart/activate-dv.md @@ -11,23 +11,41 @@ import TabItem from '@theme/TabItem'; Charon is in an alpha state and should be used with caution according to its [Terms of Use](https://obol.tech/terms.pdf). ::: -If you have successfully created a distributed validator and you are ready to activate it, congratulations. +If you have successfully created a distributed validator and you are ready to activate it, congratulations! 🎉 -Once you have connected all of your charon clients together, synced all of your ethereum nodes such that the monitoring indicates that they are all healthy and ready to operate, **ONE operator** may proceed to activate the validators using one of the following websites: +Once you have connected all of your charon clients together, synced all of your ethereum nodes such that the monitoring indicates that they are all healthy and ready to operate, **ONE operator** may proceed to deposit and activate the validator(s). + +The `deposit-data.json` to be used to deposit will be located in each operator's `.charon` folder. The copies across every node should be identical and any of them can be uploaded. + +:::warning +If you are being given a `deposit-data.json` file that you didn't generate yourself, please take extreme care to ensure this operator has not given you a malicious `deposit-data.json` file that is not the one you expect. Cross reference the files from multiple operators if there is any doubt. Activating the wrong validator or an invalid deposit could result in complete theft or loss of funds. +::: + +Use any of the following tools to deposit. Please use the third-party tools at your own risk and always double check the staking contract address. - Obol Distributed Validator Launchpad +
  • Obol Distributed Validator Launchpad
    • +
  • ethereum.org Staking Launchpad
    •  - ethereum.org Staking Launchpad +
  • Obol Distributed Validator Launchpad (Soon)
    • +
  • ethereum.org Staking Launchpad
    • +
  • From a SAFE Multisig (Repeat these steps for every validator to deposit in your cluster) +
      +
    • From the SAFE UI, click on New Transaction then Transaction Builder to create a new custom transaction
      • +
    • Enter the beacon chain contract for Deposit on mainnet - you can find it here
      • +
    • Fill the transaction information
      • +
        +
      • Set amount to 32 in ETH
        • +
      • Use your deposit-data.json to fill the required data : pubkey,withdrawal credentials,signature,deposit_data_root. Make sure to prefix the input with 0x to format them in bytes
        • +
      +
    • Click on Add transaction
      • +
    • Click on Create Batch
      • +
    • Click on Send Batch, you can click on Simulate to check if the transaction will execute successfully
      • +
    • Get the minimum threshold of signatures from the other addresses and execute the custom transaction
      • +
    • -The `deposit-data.json` to be uploaded to the launchpad will be located in each operator's `.charon` folder. The copies across every node should be identical and any of them can be uploaded. Only one, 32 ether deposit, needs to be made for each validator operated by the cluster. - -:::warning -If you are being given a `deposit-data.json` file that you didn't generate yourself, please take extreme care to ensure this operator has not given you a malicious `deposit-data.json` file that is not the one you expect. Cross reference the files from multiple operators if there is any doubt. Activating the wrong validator or an invalid deposit could result in complete theft or loss of funds. -::: - The activation process can take a minimum of 16 hours, with the maximum time to activation being dictated by the length of the activation queue, which can be weeks. \ No newline at end of file diff --git a/docs/int/quickstart/advanced/push-metrics.md b/docs/int/quickstart/advanced/push-metrics.md new file mode 100644 index 0000000000..8d9e0ceca1 --- /dev/null +++ b/docs/int/quickstart/advanced/push-metrics.md @@ -0,0 +1,40 @@ +--- +sidebar_position: 5 +description: Add monitoring credentials to help the Obol Team monitor the health of your cluster +--- + +# Push Metrics to Obol Monitoring + +:::info +This is **optional** and does not confer any special privileges within the Obol Network. +::: + +You may have been provided with **Monitoring Credentials** used to push distributed validator metrics to Obol's central prometheus cluster to monitor, analyze, and improve your Distributed Validator Cluster's performance. + +The provided credentials needs to be added in `prometheus/prometheus.yml` replacing `$PROM_REMOTE_WRITE_TOKEN` and will look like: +``` +obol20!tnt8U!C... +``` + +The updated `prometheus/prometheus.yml` file should look like: +``` +global: + scrape_interval: 30s # Set the scrape interval to every 30 seconds. + evaluation_interval: 30s # Evaluate rules every 30 seconds. + +remote_write: + - url: https://vm.monitoring.gcp.obol.tech/write + authorization: + credentials: obol20!tnt8U!C... + +scrape_configs: + - job_name: 'charon' + static_configs: + - targets: ['charon:3620'] + - job_name: "lodestar" + static_configs: + - targets: [ "lodestar:5064" ] + - job_name: 'node-exporter' + static_configs: + - targets: ['node-exporter:9100'] +``` \ No newline at end of file diff --git a/docs/int/quickstart/advanced/quickstart-combine.md b/docs/int/quickstart/advanced/quickstart-combine.md index 8d1025c641..19743adf1e 100644 --- a/docs/int/quickstart/advanced/quickstart-combine.md +++ b/docs/int/quickstart/advanced/quickstart-combine.md @@ -82,7 +82,7 @@ Run the following command: ```sh # Combine a clusters private keys -docker run --rm -v "$(pwd):/opt/charon" obolnetwork/charon:v0.16.0 combine --cluster-dir /opt/charon/validators-to-be-combined +docker run --rm -v "$(pwd):/opt/charon" obolnetwork/charon:v0.17.0 combine --cluster-dir /opt/charon/validators-to-be-combined ``` This command will create one subdirectory for each validator private key that has been combined, named after its public key. diff --git a/docs/int/quickstart/advanced/quickstart-sdk.md b/docs/int/quickstart/advanced/quickstart-sdk.md index dc1fe707c6..9c6cd3491c 100644 --- a/docs/int/quickstart/advanced/quickstart-sdk.md +++ b/docs/int/quickstart/advanced/quickstart-sdk.md @@ -3,6 +3,9 @@ sidebar_position: 1 description: Create a DV cluster using the Obol Typescript SDK --- +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; + # Create a DV using the SDK :::caution @@ -20,13 +23,18 @@ This is a walkthrough of using the [Obol-SDK](https://www.npmjs.com/package/@obo Install the Obol-SDK package into your development environment -```bash -# NPM -npm install --save @obolnetwork/obol-sdk - -# Yarn -yarn add @obolnetwork/obol-sdk -``` + + + 
    +      npm install --save @obolnetwork/obol-sdk
+
    +     + + 
    +      yarn add @obolnetwork/obol-sdk
+
    +     +     ## Instantiate the client @@ -94,4 +102,8 @@ Reference lock files can be found [here](https://github.com/ObolNetwork/charon/t ## Activate the DVs using the deposit contract -This guide will in future cover activating the deposit data from a lock file. \ No newline at end of file +This guide will in future cover activating the deposit data from a lock file. + +## Usage Examples + +Examples of how our SDK can be used are found [here](https://github.com/ObolNetwork/obol-sdk-examples). diff --git a/docs/int/quickstart/advanced/quickstart-split.md b/docs/int/quickstart/advanced/quickstart-split.md index 0bf7bfdc9c..7731aafd33 100644 --- a/docs/int/quickstart/advanced/quickstart-split.md +++ b/docs/int/quickstart/advanced/quickstart-split.md @@ -56,13 +56,13 @@ At the end of this process, you should have a tree like this: Run the following docker command to split the keys: ```shell -CHARON_VERSION= # E.g. v0.16.0 +CHARON_VERSION= # E.g. v0.17.0 CLUSTER_NAME= # The name of the cluster you want to create. WITHDRAWAL_ADDRESS= # The address you want to use for withdrawals. FEE_RECIPIENT_ADDRESS= # The address you want to use for fee payments. NODES= # The number of nodes in the cluster. -docker run --rm -v $(pwd):/opt/charon obolnetwork/charon:${CHARON_VERSION} create cluster --name="${CLUSTER_NAME}" --withdrawal-addresses="${WITHDRAWAL_ADDRESS}" --fee-recipient-addresses="${FEE_RECIPIENT_ADDRESS}" --split-existing-keys --split-keys-dir=/opt/charon/split_keys --nodes ${NODES} +docker run --rm -v $(pwd):/opt/charon obolnetwork/charon:${CHARON_VERSION} create cluster --name="${CLUSTER_NAME}" --withdrawal-addresses="${WITHDRAWAL_ADDRESS}" --fee-recipient-addresses="${FEE_RECIPIENT_ADDRESS}" --split-existing-keys --split-keys-dir=/opt/charon/split_keys --nodes ${NODES} --network goerli ``` The above command will create `validator_keys` along with `cluster-lock.json` in `./.charon/cluster` for each node. diff --git a/docs/int/quickstart/alone/create-keys.md b/docs/int/quickstart/alone/create-keys.md index e7f75360d9..2e37e71ac7 100644 --- a/docs/int/quickstart/alone/create-keys.md +++ b/docs/int/quickstart/alone/create-keys.md @@ -42,7 +42,7 @@ Alternatively, the private key shares can be created in a lower-trust manner wit Then, run this command to create all the key shares and cluster artifacts locally:

           
-      docker run --rm -v "$(pwd):/opt/charon" obolnetwork/charon:v0.16.0 create cluster --name="mycluster" --withdrawal-addresses="{'${WITHDRAWAL_ADDR}'}" --fee-recipient-addresses="{'${FEE_RECIPIENT_ADDR}'}" --nodes="{'${NB_NODES}'}"
+      docker run --rm -v "$(pwd):/opt/charon" obolnetwork/charon:v0.17.0 create cluster --name="mycluster" --withdrawal-addresses="{'${WITHDRAWAL_ADDR}'}" --fee-recipient-addresses="{'${FEE_RECIPIENT_ADDR}'}" --nodes="{'${NB_NODES}'}" --network goerli --num-validators=1
    diff --git a/docs/int/quickstart/alone/test-locally.md b/docs/int/quickstart/alone/test-locally.md index f1c309d439..b7fd4ecce5 100644 --- a/docs/int/quickstart/alone/test-locally.md +++ b/docs/int/quickstart/alone/test-locally.md @@ -60,10 +60,10 @@ The default cluster consists of: FEE_RECIPIENT_ADDR= # Create a distributed validator cluster - docker run --rm -v "$(pwd):/opt/charon" obolnetwork/charon:v0.16.0 create cluster --name="mycluster" --withdrawal-addresses="${WITHDRAWAL_ADDR}" --fee-recipient-addresses="${FEE_RECIPIENT_ADDR}" --nodes 6 --threshold 5 + docker run --rm -v "$(pwd):/opt/charon" obolnetwork/charon:v0.17.0 create cluster --name="mycluster" --cluster-dir=".charon/cluster/" --withdrawal-addresses="${WITHDRAWAL_ADDR}" --fee-recipient-addresses="${FEE_RECIPIENT_ADDR}" --nodes 6 --network goerli --num-validators=1 ``` -These commands will create a subdirectory `.charon/cluster`. In it are six folders, one for each charon node created. Each folder contains partial private keys that together make up the distributed validator described in `.charon/cluster/cluster-lock.json`. +These commands will create six folders within `.charon/cluster`, one for each node created. You will need to rename `node*` to `.charon` for each folder to be found by the default `charon run` command, or you can use `charon run --private-key-file=".charon/cluster/node0/charon-enr-private-key" --lock-file=".charon/cluster/node0/cluster-lock.json"` for each instance of charon you start. ## Start the cluster diff --git a/docs/int/quickstart/group/quickstart-cli.md b/docs/int/quickstart/group/quickstart-cli.md index 28e1f476ea..9f4b233053 100644 --- a/docs/int/quickstart/group/quickstart-cli.md +++ b/docs/int/quickstart/group/quickstart-cli.md @@ -32,7 +32,7 @@ git clone https://github.com/ObolNetwork/charon-distributed-validator-node.git cd charon-distributed-validator-node # Create your charon ENR private key, this will create a charon-enr-private-key file in the .charon directory -docker run --rm -v "$(pwd):/opt/charon" obolnetwork/charon:v0.16.0 create enr +docker run --rm -v "$(pwd):/opt/charon" obolnetwork/charon:v0.17.0 create enr ``` You should expect to see a console output like @@ -59,7 +59,7 @@ Finally, share your ENR with the leader or creator so that he/she can proceed to 3. Run the `charon create dkg` command that generates DKG cluster-definition.json file. ``` - docker run --rm -v "$(pwd):/opt/charon" --env-file .env.create_dkg obolnetwork/charon:v0.16.0 create dkg + docker run --rm -v "$(pwd):/opt/charon" --env-file .env.create_dkg obolnetwork/charon:v0.17.0 create dkg ``` This command should output a file at `.charon/cluster-definition.json`. This file needs to be shared with the other operators in a cluster. @@ -72,7 +72,7 @@ Every cluster member then participates in the DKG ceremony. For Charon v1, this ``` # Participate in DKG ceremony, this will create .charon/cluster-lock.json, .charon/deposit-data.json and .charon/validator_keys -docker run --rm -v "$(pwd):/opt/charon" obolnetwork/charon:v0.16.0 dkg +docker run --rm -v "$(pwd):/opt/charon" obolnetwork/charon:v0.17.0 dkg ``` >This is a helpful [video walkthrough](https://www.youtube.com/watch?v=94Pkovp5zoQ&ab_channel=ObolNetwork). diff --git a/docs/int/quickstart/group/quickstart-group-leader-creator.md b/docs/int/quickstart/group/quickstart-group-leader-creator.md index a79ede60cd..d0b7481c94 100644 --- a/docs/int/quickstart/group/quickstart-group-leader-creator.md +++ b/docs/int/quickstart/group/quickstart-group-leader-creator.md @@ -52,7 +52,7 @@ Before starting the cluster creation, you will need to collect one Ethereum addr cd charon-distributed-validator-node # Create your charon ENR private key, this will create a charon-enr-private-key file in the .charon directory - docker run --rm -v "$(pwd):/opt/charon" obolnetwork/charon:v0.16.0 create enr + docker run --rm -v "$(pwd):/opt/charon" obolnetwork/charon:v0.17.0 create enr ``` You should expect to see a console output like diff --git a/docs/int/quickstart/group/quickstart-group-operator.md b/docs/int/quickstart/group/quickstart-group-operator.md index 3394e64282..3c56d24c8f 100644 --- a/docs/int/quickstart/group/quickstart-group-operator.md +++ b/docs/int/quickstart/group/quickstart-group-operator.md @@ -35,7 +35,7 @@ git clone https://github.com/ObolNetwork/charon-distributed-validator-node.git cd charon-distributed-validator-node # Create your charon ENR private key, this will create a charon-enr-private-key file in the .charon directory -docker run --rm -v "$(pwd):/opt/charon" obolnetwork/charon:v0.16.0 create enr +docker run --rm -v "$(pwd):/opt/charon" obolnetwork/charon:v0.17.0 create enr ``` You should expect to see a console output like diff --git a/docs/sc/01_introducing-obol-managers.md b/docs/sc/01_introducing-obol-managers.md index 2cc857f2c9..3209b873f5 100644 --- a/docs/sc/01_introducing-obol-managers.md +++ b/docs/sc/01_introducing-obol-managers.md @@ -1,16 +1,27 @@ --- -description: How does the Obol Network look on-chain? +description: Smart contracts for managing Distributed Validators --- # Obol Manager Contracts -Obol develops and maintains a suite of smart contracts for use with Distributed Validators. +Obol develops and maintains a suite of smart contracts for use with Distributed Validators. These contracts include: -## Withdrawal Recipients +- Withdrawal Recipients: Contracts used for a validator's withdrawal address. +- Split contracts: Contracts to split ether across multiple entities. Developed by [0xSplits](https://0xsplits.xyz) +- Split controllers: Contracts that can mutate a splitter's configuration. + +Two key goals of validator reward management are: + +1. To be able to differentiate reward ether from principal ether such that node operators can be paid a percentage the _reward_ they accrue for the principal provider rather than a percentage of _principal+reward_. +2. To be able to withdraw the rewards in an ongoing manner without exiting the validator. + +Without access to the consensus layer state in the EVM to check a validator's status or balance, and due to the incoming ether being from an irregular state transition, neither of these requirements are easily satisfiable. -The key to a distributed validator is understanding how a withdrawal is processed. The most common way to handle a withdrawal of a validator operated by a number of different people is to use an immutable withdrawal recipient contract, with the distribution rules hardcoded into it. +The following sections outline different contracts that can be composed to form a solution for one or both goals. -For the time being Obol uses `0x01` withdrawal credentials, and intends to upgrade to [0x03 withdrawal credentials](https://ethresear.ch/t/0x03-withdrawal-credentials-simple-eth1-triggerable-withdrawals/10021) when smart contract initiated exits are enabled. +## Withdrawal Recipients + +Validators have two streams of revenue, the consensus layer rewards and the execution layer rewards. Withdrawal Recipients focus on the former, receiving the balance skimming from a validator with >32 ether in an ongoing manner, and receiving the principal of the validator upon exit. ### Ownable Withdrawal Recipient @@ -31,29 +42,48 @@ contract WithdrawalRecipientOwnable is Ownable { ``` -An Ownable Withdrawal Recipient is the most basic type of withdrawal recipient contract. It implements Open Zeppelin's `Ownable` interface and allows one address to call the `withdraw()` function, which pulls all ether from the address into the owners address (or another address specified). Calling withdraw could also fund a fee split to the Obol Network, and/or the protocol that has deployed and instantiated this DV. +An Ownable Withdrawal Recipient is the most basic example of a withdrawal recipient contract. It implements Open Zeppelin's `Ownable` interface and allows one address to call the `withdraw()` function, which pulls all ether from the address into the owner's address (or another address specified). This contract does no accounting on the amount of ether that is withdrawn, nor does it differentiate reward from principal. -### Immutable Withdrawal Recipient +### Optimistic Withdrawal Recipient -An immutable withdrawal recipient is similar to an ownable recipient except the owner is hardcoded during construction and the ability to change ownership is removed. This contract should only be used as part of a larger smart contract system, for example a yearn vault strategy might use an immutable recipient contract as its vault address should never change. +This is the primary withdrawal recipient Obol uses, as it allows for the separation of reward from principal, as well as permitting the ongoing withdrawal of accruing rewards. -## Registries +An Optimistic Withdrawal Recipient contract takes three inputs when deployed: -### Deposit Registry +- A _principal_ address: The address that controls where the principal ether will be transferred post-exit. +- A _reward_ address: The address where the accruing reward ether is transferred to. +- The amount of ether that makes up the principal. -The Deposit Registry is a way for the deposit and activation of distributed validators to be two separate processes. In the simple case for DVs, a registry of deposits is not required. However when the person depositing the ether is not the same entity as the operators producing the deposits, a coordination mechanism is needed to make sure only one 32 eth deposit is submitted per DV. A deposit registry can prevent double deposits by ordering the allocation of ether to validator deposits. +This contract **assumes that any ether that has appeared in it's address since it was last able to do balance accounting is reward from a successful validator** (or number of validators) unless the change is > 16 ether. This means balance skimming is immediately claimable as reward, while an inflow of e.g. 31 ether is tracked as a return of principal (despite being slashed in this example). -### Operator Registry +:::warning -If the submission of deposits to a deposit registry needs to be gated to only whitelisted addresses, a simple operator registry may serve as a way to control who can submit deposits to the deposit registry. +Worst-case mass slashings can theoretically exceed 16 ether, if this were to occur, the returned principal would be misclassified as a reward, and distributed to the wrong address. This risk is the drawback that makes this contract variant 'optimistic'. If you intend to use this contract type, **it is important you understand and accept this risk**, however minute. -### Validator Registry +The alternative is to use an 0xSplits waterfall contract, which won't allow the claiming of rewards until all principal ether has been pulled, meaning validators need to be exited for operators to claim their CL rewards. -If validators need to be managed on chain programmatically rather than manually with humans triggering exits, a validator registry can be used. Deposits getting activated get an entry into the validator registry, and validators using 0x03 exits get staged for removal from the registry. This registry can be used to coordinate many validators with similar operators and configuration. +::: -:::note +This contract fits both design goals and can be used with thousands of validators. If you deploy an Optimistic Withdrawal Recipient with a principal higher than you actually end up using, nothing goes wrong. If you activate more validators than you specified in your contract deployment, you will record too much ether as reward and will overpay your reward address with ether that was principal ether, not earned ether. Current iterations of this contract are not designed for editing the amount of principal set. -Validator registries depend on the as of yet unimplemented `0x03` validator exit feature. +### Exitable Withdrawal Recipient -::: +A much awaited feature for proof of stake Ethereum is the ability to trigger the exit of a validator with only the withdrawal address. This is tracked in [EIP-7002](https://eips.ethereum.org/EIPS/eip-7002). Support for this feature will be inheritable in all other withdrawal recipient contracts. This will mitigate the risk to a principal provider of funds being stuck, or a validator being irrecoverably offline. + +## Split Contracts + +A split, or splitter, is a set of contracts that can divide ether or an ERC20 across a number of addresses. Splits are used in conjunction with withdrawal recipients. Execution Layer rewards for a DV are directed to a split address through the use of a `fee recipient` address. Splits can be either immutable, or mutable by way of an admin address capable of updating them. + +Further information about splits can be found on the 0xSplits team's [docs site](https://docs.0xsplits.xyz/). + +## Split Controllers + +Splits can be completely edited through the use of the `controller` address, however, total editability of a split is not always wanted. A permissive controller and a restrictive controller are given as examples below. + +### (Gnosis) SAFE wallet + +A [SAFE](https://safe.global/) is a common method to administrate a mutable split. The most well-known deployment of this pattern is the [protocol guild](https://protocol-guild.readthedocs.io/en/latest/3-smart-contract.html). The SAFE can arbitrarily update the split to any set of addresses with any valid set of percentages. + +### Immutable Split Controller +This is a contract that updates one split configuration with another, exactly once. Only a permissioned address can trigger the change. This contract is suitable for changing a split at an unknown point in future to a configuration pre-defined at deployment. diff --git a/docs/testnet.md b/docs/testnet.md index 28141b3de8..f430d00a0b 100644 --- a/docs/testnet.md +++ b/docs/testnet.md @@ -114,4 +114,3 @@ This testnet is also important for learning the conditions Charon will be subjec - Distributed Validator returns are competitive versus single validator clients. - Make deploying Ethereum validator nodes accessible using the DV Launchpad. - Build comprehensive guides for various profiles to spin up DVs with minimal supervision from the core team. - diff --git a/docusaurus.config.js b/docusaurus.config.js index 684dcf3d4c..96b4c3e9b6 100644 --- a/docusaurus.config.js +++ b/docusaurus.config.js @@ -123,6 +123,7 @@ const config = { label: 'API', }, { to: "https://blog.obol.tech", label: "Blog", position: "left" }, + { to: "https://obolnetwork.github.io/obol-packages", label: "SDK", position: "left" }, { href: "https://github.com/obolnetwork/obol-docs", label: "GitHub", diff --git a/versioned_docs/version-v0.16.0/int/faq/errors.mdx b/versioned_docs/version-v0.16.0/int/faq/errors.mdx index 8a89a39607..90f539037b 100644 --- a/versioned_docs/version-v0.16.0/int/faq/errors.mdx +++ b/versioned_docs/version-v0.16.0/int/faq/errors.mdx @@ -89,7 +89,7 @@ docker compose logs

    Not enough time for a discovery seach error

    -     This could be linked to a internet connection being to slow or relying on a slow third-party service such as Infura. + This could be linked to an internet connection being to slow or relying on a slow third-party service such as Infura.
    @@ -167,7 +167,7 @@ docker compose logs Error opening relay circuit: NO_RESERVATION error - Error opening relay circuit NO_RESERVATION (204) indicates the peer isn't connected to the relay, so the the charon + Error opening relay circuit NO_RESERVATION (204) indicates the peer isn't connected to the relay, so the charon client cannot connect to the peer via the relay. That might be because the peer is offline or the peer is configured to connect to a different relay.

    @@ -295,7 +295,7 @@ docker compose logs

    How to fix the grafana dashboard?

    -     Sometimes, grafana dashboard doesn't load any data first time around.You can solve this by following the steps below:
    -
    \ No newline at end of file + diff --git a/versioned_docs/version-v0.16.0/sec/overview.md b/versioned_docs/version-v0.16.0/sec/overview.md index 257dec6368..35c877fe1d 100644 --- a/versioned_docs/version-v0.16.0/sec/overview.md +++ b/versioned_docs/version-v0.16.0/sec/overview.md @@ -43,7 +43,7 @@ The Obol Network consists of four core public goods: - The Distributed Validator [Launchpad](https://docs.obol.tech/docs/dvl/intro), a [User Interface](https://goerli.launchpad.obol.tech/) for bootstrapping Distributed Validators - [Charon](https://docs.obol.tech/docs/charon/intro), a middleware client that enables validators to run in a fault-tolerant, distributed manner - [Obol Managers](https://docs.obol.tech/docs/sc/introducing-obol-managers), a set of solidity smart contracts for the formation of Distributed Validators -- [Obol Testnets])(https://docs.obol.tech/docs/testnet), a set of on-going public incentivized testnets that enable any sized operator to test their deployment before serving for the mainnet Obol Network +- [Obol Testnets](https://docs.obol.tech/docs/testnet), a set of on-going public incentivized testnets that enable any sized operator to test their deployment before serving for the mainnet Obol Network ## List of Security Audits diff --git a/versioned_docs/version-v0.17.0/cg/_category_.json b/versioned_docs/version-v0.17.0/cg/_category_.json new file mode 100644 index 0000000000..6658367ce5 --- /dev/null +++ b/versioned_docs/version-v0.17.0/cg/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "Contribution & Feedback", + "position": 10, + "collapsed": true +} diff --git a/versioned_docs/version-v0.17.0/cg/bug-report.md b/versioned_docs/version-v0.17.0/cg/bug-report.md new file mode 100644 index 0000000000..9a10b3b553 --- /dev/null +++ b/versioned_docs/version-v0.17.0/cg/bug-report.md @@ -0,0 +1,57 @@ +# Filing a bug report + +Bug reports are critical to the rapid development of Obol. In order to make the process quick and efficient for all parties, it is best to follow some common reporting etiquette when filing to avoid double issues or miscommunications. + +## Checking if your issue exists + +Duplicate tickets are a hindrance to the development process and, as such, it is crucial to first check through Charon's existing issues to see if what you are experiencing has already been indexed. + +To do so, head over to the [issue page](https://github.com/ObolNetwork/charon/issues) and enter some related keywords into the search bar. This may include a sample from the output or specific components it affects. + +If searches have shown the issue in question has not been reported yet, feel free to open up a new issue ticket. + +## Writing quality bug reports + +A good bug report is structured to help the developers and contributors visualize the issue in the clearest way possible. It's important to be concise and use comprehensive language, while also providing all relevant information on-hand. Use short and accurate sentences without any unnecessary additions, and include all existing specifications with a list of steps to reproduce the expected problem. Issues that cannot be reproduced **cannot be solved**. + +If you are experiencing multiple issues, it is best to open each as a separate ticket. This allows them to be closed individually as they are resolved. + +An original bug report will very likely be preserved and used as a record and sounding board for users that have similar experiences in the future. Because of this, it is a great service to the community to ensure that reports meet these standards and follow the template closely. + + +## The bug report template + +Below is the standard bug report template used by all of Obol's official repositories. + +```sh + + +## Expected Behavior + + +## Current Behavior + + +## Steps to Reproduce + +1. +2. +3. +4. +5. + +## Detailed Description + + +## Specifications + +Operating system: +Version(s) used: + +## Possible Solution + + +## Further Information + + + ## What is Charon? + + + + ## Charon explained + ``` + +#### Bold text + +Double asterisks `**` are used to define **boldface** text. Use bold text when the reader must interact with something displayed as text: buttons, hyperlinks, images with text in them, window names, and icons. + +```markdown +In the **Login** window, enter your email into the **Username** field and click **Sign in**. +``` + +#### Italics + +Underscores `_` are used to define _italic_ text. Style the names of things in italics, except input fields or buttons: + +```markdown +Here are some American things: + +- The _Spirit of St Louis_. +- The _White House_. +- The United States _Declaration of Independence_. + +``` + +Quotes or sections of quoted text are styled in italics and surrounded by double quotes `"`: + +```markdown +In the wise words of Winnie the Pooh _"People say nothing is impossible, but I do nothing every day."_ +``` + +#### Code blocks + +Tag code blocks with the syntax of the core they are presenting: + +````markdown + ```javascript + console.log(error); + ``` +```` + +#### List items + +All list items follow sentence structure. Only _names_ and _places_ are capitalized, along with the first letter of the list item. All other letters are lowercase: + +1. Never leave Nottingham without a sandwich. +2. Brian May played guitar for Queen. +3. Oranges. + +List items end with a period `.`, or a colon `:` if the list item has a sub-list: + +1. Charles Dickens novels: + 1. Oliver Twist. + 2. Nicholas Nickelby. + 3. David Copperfield. +2. J.R.R Tolkien non-fiction books: + 1. The Hobbit. + 2. Silmarillion. + 3. Letters from Father Christmas. + +##### Unordered lists + +Use the dash character `-` for un-numbered list items: + +```markdown +- An apple. +- Three oranges. +- As many lemons as you can carry. +- Half a lime. +``` + +#### Special characters + +Whenever possible, spell out the name of the special character, followed by an example of the character itself within a code block. + +```markdown +Use the dollar sign `$` to enter debug-mode. +``` + +#### Keyboard shortcuts + +When instructing the reader to use a keyboard shortcut, surround individual keys in code tags: + +```bash +Press `ctrl` + `c` to copy the highlighted text. +``` + +The plus symbol `+` stays outside of the code tags. + +### Images + +The following rules and guidelines define how to use and store images. + +#### Storage location + +All images must be placed in the `/static/img` folder. For multiple images attributed to a single topic, a new folder within `/img/` may be needed. + +#### File names + +All file names are lower-case with dashes `-` between words, including image files: + +```text +concepts/ +├── content-addressed-data.md +├── images +│   └── proof-of-spacetime +│   └── post-diagram.png +└── proof-of-replication.md +└── proof-of-spacetime.md +``` + +_The framework and some information for this was forked from the original found on the [Filecoin documentation portal](https://docs.filecoin.io)_ + diff --git a/versioned_docs/version-v0.17.0/cg/feedback.md b/versioned_docs/version-v0.17.0/cg/feedback.md new file mode 100644 index 0000000000..76042e28aa --- /dev/null +++ b/versioned_docs/version-v0.17.0/cg/feedback.md @@ -0,0 +1,5 @@ +# Feedback + +If you have followed our quickstart guides, and whether you succeeded or failed at running the distributed validator successfully, we would like to hear your feedback on the process and where you encountered difficulties. +- Please let us know by joining and posting on our [Discord](https://discord.gg/n6ebKsX46w). +- Also, feel free to add issues to our [GitHub repos](https://github.com/ObolNetwork). \ No newline at end of file diff --git a/versioned_docs/version-v0.17.0/charon/_category_.json b/versioned_docs/version-v0.17.0/charon/_category_.json new file mode 100644 index 0000000000..5ed247b0e5 --- /dev/null +++ b/versioned_docs/version-v0.17.0/charon/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "Charon", + "position": 3, + "collapsed": false +} diff --git a/versioned_docs/version-v0.17.0/charon/charon-cli-reference.md b/versioned_docs/version-v0.17.0/charon/charon-cli-reference.md new file mode 100644 index 0000000000..020e207413 --- /dev/null +++ b/versioned_docs/version-v0.17.0/charon/charon-cli-reference.md @@ -0,0 +1,379 @@ +--- +description: A go-based middleware client for taking part in Distributed Validator clusters. +sidebar_position: 5 +--- + +# CLI reference + +:::caution + +The `charon` client is under heavy development, interfaces are subject to change until a first major version is published. + +::: + +The following is a reference for charon version [`v0.17.0`](https://github.com/ObolNetwork/charon/releases/tag/v0.17.0). Find the latest release on [our Github](https://github.com/ObolNetwork/charon/releases). + +The following are the top-level commands available to use. + +```markdown +Charon enables the operation of Ethereum validators in a fault tolerant manner by splitting the validating keys across a group of trusted parties using threshold cryptography. + +Usage: + charon [command] + +Available Commands: + combine Combines the private key shares of a distributed validator cluster into a set of standard validator private keys. + completion Generate the autocompletion script for the specified shell + create Create artifacts for a distributed validator cluster + dkg Participate in a Distributed Key Generation ceremony + enr Prints a new ENR for this node + help Help about any command + relay Start a libp2p relay server + run Run the charon middleware client + version Print version and exit + +Flags: + -h, --help Help for charon + +Use "charon [command] --help" for more information about a command. +``` + +## The `create` subcommand + +The `create` subcommand handles the creation of artifacts needed by charon to operate. + +```markdown +charon create --help +Create artifacts for a distributed validator cluster. These commands can be used to facilitate the creation of a distributed validator cluster between a group of operators by performing a distributed key generation ceremony, or they can be used to create a local cluster for single operator use cases. + +Usage: + charon create [command] + +Available Commands: + cluster Create private keys and configuration files needed to run a distributed validator cluster locally + dkg Create the configuration for a new Distributed Key Generation ceremony using charon dkg + enr Create an Ethereum Node Record (ENR) private key to identify this charon client + +Flags: + -h, --help Help for create + +Use "charon create [command] --help" for more information about a command. +``` + +### Creating an ENR for charon + +An `enr` is an Ethereum Node Record. It is used to identify this charon client to its other counterparty charon clients across the internet. + +```markdown +charon create enr --help +Create an Ethereum Node Record (ENR) private key to identify this charon client + +Usage: + charon create enr [flags] + +Flags: + --data-dir string The directory where charon will store all its internal data (default ".charon") + -h, --help Help for enr +``` + +### Create a full cluster locally + +`charon create cluster` creates a set of distributed validators locally, including the private keys, a `cluster-lock.json` file, and deposit and exit data. However, this command should only be used for solo use of distributed validators. To run a Distributed Validator with a group of operators, it is preferable to create these artifacts using the `charon dkg` command. That way, no single operator custodies all of the private keys to a distributed validator. + +```markdown +Creates a local charon cluster configuration including validator keys, charon p2p keys, cluster-lock.json and a deposit-data.json. See flags for supported features. + +Usage: + charon create cluster [flags] + +Flags: + --cluster-dir string The target folder to create the cluster in. (default "./") + --definition-file string Optional path to a cluster definition file or an HTTP URL. This overrides all other configuration flags. + --fee-recipient-addresses strings Comma separated list of Ethereum addresses of the fee recipient for each validator. Either provide a single fee recipient address or fee recipient addresses for each validator. + -h, --help Help for cluster + --insecure-keys Generates insecure keystore files. This should never be used. It is not supported on mainnet. + --keymanager-addresses strings Comma separated list of keymanager URLs to import validator key shares to. Note that multiple addresses are required, one for each node in the cluster, with node0's keyshares being imported to the first address, node1's keyshares to the second, and so on. + --keymanager-auth-tokens strings Authentication bearer tokens to interact with the keymanager URLs. Don't include the "Bearer" symbol, only include the api-token. + --name string The cluster name + --network string Ethereum network to create validators for. Options: mainnet, goerli, gnosis, sepolia. + --nodes int The number of charon nodes in the cluster. Minimum is 3. + --num-validators int The number of distributed validators needed in the cluster. + --publish Publish lock file to obol-api. + --publish-address string The URL to publish the lock file to. (default "https://api.obol.tech") + --split-existing-keys Split an existing validator's private key into a set of distributed validator private key shares. Does not re-create deposit data for this key. + --split-keys-dir string Directory containing keys to split. Expects keys in keystore-*.json and passwords in keystore-*.txt. Requires --split-existing-keys. + --threshold int Optional override of threshold required for signature reconstruction. Defaults to ceil(n*2/3) if zero. Warning, non-default values decrease security. + --withdrawal-addresses strings Comma separated list of Ethereum addresses to receive the returned stake and accrued rewards for each validator. Either provide a single withdrawal address or withdrawal addresses for each validator. +``` + +### Creating the configuration for a DKG Ceremony + +This `charon create dkg` command creates a cluster_definition file used for the `charon dkg` command. + +```markdown +charon create dkg --help +Create a cluster definition file that will be used by all participants of a DKG. + +Usage: + charon create dkg [flags] + +Flags: + --dkg-algorithm string DKG algorithm to use; default, keycast, frost (default "default") + --fee-recipient-addresses strings Comma separated list of Ethereum addresses of the fee recipient for each validator. Either provide a single fee recipient address or fee recipient addresses for each validator. + -h, --help Help for dkg + --name string Optional cosmetic cluster name + --network string Ethereum network to create validators for. Options: mainnet, gnosis, goerli, kiln, ropsten, sepolia. (default "mainnet") + --num-validators int The number of distributed validators the cluster will manage (32ETH staked for each). (default 1) + --operator-enrs strings [REQUIRED] Comma-separated list of each operator's Charon ENR address. + --output-dir string The folder to write the output cluster-definition.json file to. (default ".charon") + -t, --threshold int Optional override of threshold required for signature reconstruction. Defaults to ceil(n*2/3) if zero. Warning, non-default values decrease security. + --withdrawal-addresses strings Comma separated list of Ethereum addresses to receive the returned stake and accrued rewards for each validator. Either provide a single withdrawal address or withdrawal addresses for each validator. +``` + +## The `dkg` subcommand + +### Performing a DKG Ceremony + +The `charon dkg` command takes a `cluster_definition.json` file that instructs charon on the terms of a new distributed validator cluster to be created. Charon establishes communication with the other nodes identified in the file, performs a distributed key generation ceremony to create the required threshold private keys, and signs deposit data for each new distributed validator. The command outputs the `cluster-lock.json` file and key shares for each Distributed Validator created. + +```markdown +charon dkg --help +Participate in a distributed key generation ceremony for a specific cluster definition that creates +distributed validator key shares and a final cluster lock configuration. Note that all other cluster operators should run +this command at the same time. + +Usage: + charon dkg [flags] + +Flags: + --data-dir string The directory where charon will store all its internal data (default ".charon") + --definition-file string The path to the cluster definition file or an HTTP URL. (default ".charon/cluster-definition.json") + -h, --help Help for dkg + --keymanager-address string The keymanager URL to import validator keyshares. + --keymanager-auth-token string Authentication bearer token to interact with keymanager API. Don't include the "Bearer" symbol, only include the api-token. + --log-color string Log color; auto, force, disable. (default "auto") + --log-format string Log format; console, logfmt or json (default "console") + --log-level string Log level; debug, info, warn or error (default "info") + --no-verify Disables cluster definition and lock file verification. + --p2p-allowlist string Comma-separated list of CIDR subnets for allowing only certain peer connections. Example: 192.168.0.0/16 would permit connections to peers on your local network only. The default is to accept all connections. + --p2p-denylist string Comma-separated list of CIDR subnets for disallowing certain peer connections. Example: 192.168.0.0/16 would disallow connections to peers on your local network. The default is to accept all connections. + --p2p-disable-reuseport Disables TCP port reuse for outgoing libp2p connections. + --p2p-external-hostname string The DNS hostname advertised by libp2p. This may be used to advertise an external DNS. + --p2p-external-ip string The IP address advertised by libp2p. This may be used to advertise an external IP. + --p2p-relays strings Comma-separated list of libp2p relay URLs or multiaddrs. (default [https://0.relay.obol.tech]) + --p2p-tcp-address strings Comma-separated list of listening TCP addresses (ip and port) for libP2P traffic. Empty default doesn't bind to local port therefore only supports outgoing connections. + --publish Publish lock file to obol-api. + --publish-address string The URL to publish the lock file to. (default "https://api.obol.tech") + --shutdown-delay duration Graceful shutdown delay. (default 1s) +``` + +## The `run` subcommand + +### Run the Charon middleware + +This `run` command accepts a `cluster-lock.json` file that was created either via a `charon create cluster` command or `charon dkg`. This lock file outlines the nodes in the cluster and the distributed validators they operate on behalf of. + +```markdown +charon run --help +Starts the long-running Charon middleware process to perform distributed validator duties. + +Usage: + charon run [flags] + +Flags: + --beacon-node-endpoints strings Comma separated list of one or more beacon node endpoint URLs. + --builder-api Enables the builder api. Will only produce builder blocks. Builder API must also be enabled on the validator client. Beacon node must be connected to a builder-relay to access the builder network. + --feature-set string Minimum feature set to enable by default: alpha, beta, or stable. Warning: modify at own risk. (default "stable") + --feature-set-disable strings Comma-separated list of features to disable, overriding the default minimum feature set. + --feature-set-enable strings Comma-separated list of features to enable, overriding the default minimum feature set. + -h, --help Help for run + --jaeger-address string Listening address for jaeger tracing. + --jaeger-service string Service name used for jaeger tracing. (default "charon") + --lock-file string The path to the cluster lock file defining distributed validator cluster. (default ".charon/cluster-lock.json") + --log-color string Log color; auto, force, disable. (default "auto") + --log-format string Log format; console, logfmt or json (default "console") + --log-level string Log level; debug, info, warn or error (default "info") + --loki-addresses strings Enables sending of logfmt structured logs to these Loki log aggregation server addresses. This is in addition to normal stderr logs. + --loki-service string Service label sent with logs to Loki. (default "charon") + --monitoring-address string Listening address (ip and port) for the monitoring API (prometheus, pprof). (default "127.0.0.1:3620") + --no-verify Disables cluster definition and lock file verification. + --p2p-allowlist string Comma-separated list of CIDR subnets for allowing only certain peer connections. Example: 192.168.0.0/16 would permit connections to peers on your local network only. The default is to accept all connections. + --p2p-denylist string Comma-separated list of CIDR subnets for disallowing certain peer connections. Example: 192.168.0.0/16 would disallow connections to peers on your local network. The default is to accept all connections. + --p2p-disable-reuseport Disables TCP port reuse for outgoing libp2p connections. + --p2p-external-hostname string The DNS hostname advertised by libp2p. This may be used to advertise an external DNS. + --p2p-external-ip string The IP address advertised by libp2p. This may be used to advertise an external IP. + --p2p-relays strings Comma-separated list of libp2p relay URLs or multiaddrs. (default [https://0.relay.obol.tech]) + --p2p-tcp-address strings Comma-separated list of listening TCP addresses (ip and port) for libP2P traffic. Empty default doesn't bind to local port therefore only supports outgoing connections. + --private-key-file string The path to the charon enr private key file. (default ".charon/charon-enr-private-key") + --private-key-file-lock Enables private key locking to prevent multiple instances using the same key. + --simnet-beacon-mock Enables an internal mock beacon node for running a simnet. + --simnet-beacon-mock-fuzz Configures simnet beaconmock to return fuzzed responses. + --simnet-slot-duration duration Configures slot duration in simnet beacon mock. (default 1s) + --simnet-validator-keys-dir string The directory containing the simnet validator key shares. (default ".charon/validator_keys") + --simnet-validator-mock Enables an internal mock validator client when running a simnet. Requires simnet-beacon-mock. + --synthetic-block-proposals Enables additional synthetic block proposal duties. Used for testing of rare duties. + --validator-api-address string Listening address (ip and port) for validator-facing traffic proxying the beacon-node API. (default "127.0.0.1:3600") +``` + +## The `combine` subcommand + +### Combine distributed validator keyshares into a single Validator key + +The `combine` command combines many validator keyshares into a single Ethereum validator key. + +To run this command, one needs all the node operator's `.charon` directories, which need to be organized in the following way: + +```shell +validators-to-be-combined/ +├── node0 +│   ├── charon-enr-private-key +│   ├── cluster-lock.json +│   ├── deposit-data.json +│   └── validator_keys +│   ├── keystore-0.json +│   ├── keystore-0.txt +│   ├── keystore-1.json +│   └── keystore-1.txt +├── node1 +│   ├── charon-enr-private-key +│   ├── cluster-lock.json +│   ├── deposit-data.json +│   └── validator_keys +│   ├── keystore-0.json +│   ├── keystore-0.txt +│   ├── keystore-1.json +│   └── keystore-1.txt +├── node2 +│   ├── charon-enr-private-key +│   ├── cluster-lock.json +│   ├── deposit-data.json +│   └── validator_keys +│   ├── keystore-0.json +│   ├── keystore-0.txt +│   ├── keystore-1.json +│   └── keystore-1.txt +└── node3 + ├── charon-enr-private-key + ├── cluster-lock.json + ├── deposit-data.json + └── validator_keys + ├── keystore-0.json + ├── keystore-0.txt + ├── keystore-1.json + └── keystore-1.txt +``` + +That is, each operator '.charon' directory must be placed in a parent directory, and renamed. + +Note that all validator keys are required for the successful execution of this command. + +If for example the lock file defines 2 validators, each `validator_keys` directory must contain exactly 4 files, a JSON and TXT file for each validator. + +Those files must be named with an increasing index associated with the validator in the lock file, starting from 0. + +The chosen name doesn't matter, as long as it's different from `.charon`. + +At the end of the process `combine` will create a new set of directories containing one validator key each, named after its public key: + +```shell +validators-to-be-combined/ +├── 0x822c5310674f4fc4ec595642d0eab73d01c62b588f467da6f98564f292a975a0ac4c3a10f1b3a00ccc166a28093c2dcd # contains private key +│   └── validator_keys +│   ├── keystore-0.json +│   └── keystore-0.txt +├── 0x8929b4c8af2d2eb222d377cac2aa7be950e71d2b247507d19b5fdec838f0fb045ea8910075f191fd468da4be29690106 # contains private key +│   └── validator_keys +│   ├── keystore-0.json +│   └── keystore-0.txt +├── node0 +│   ├── charon-enr-private-key +│   ├── cluster-lock.json +│   ├── deposit-data.json +│   └── validator_keys +│   ├── keystore-0.json +│   ├── keystore-0.txt +│   ├── keystore-1.json +│   └── keystore-1.txt +├── node1 +│   ├── charon-enr-private-key +│   ├── cluster-lock.json +│   ├── deposit-data.json +│   └── validator_keys +│   ├── keystore-0.json +│   ├── keystore-0.txt +│   ├── keystore-1.json +│   └── keystore-1.txt +├── node2 +│   ├── charon-enr-private-key +│   ├── cluster-lock.json +│   ├── deposit-data.json +│   └── validator_keys +│   ├── keystore-0.json +│   ├── keystore-0.txt +│   ├── keystore-1.json +│   └── keystore-1.txt +└── node3 + ├── charon-enr-private-key + ├── cluster-lock.json + ├── deposit-data.json + └── validator_keys + ├── keystore-0.json + ├── keystore-0.txt + ├── keystore-1.json + └── keystore-1.txt +``` + +By default, the `combine` command will refuse to overwrite any private key that is already present in the destination directory. + +To force the process, use the `--force` flag. + +```markdown +charon combine --help +Combines the private key shares from a threshold of operators in a distributed validator cluster into a set of validator private keys that can be imported into a standard Ethereum validator client. + +Warning: running the resulting private keys in a validator alongside the original distributed validator cluster *will* result in slashing. + +Usage: + charon combine [flags] + +Flags: + --cluster-dir string Parent directory containing a number of .charon subdirectories from the required threshold of nodes in the cluster. (default ".charon/cluster") + --force Overwrites private keys with the same name if present. + -h, --help Help for combine + --output-dir string Directory to output the combined private keys to. (default "./validator_keys") +``` + +## Host a relay + +Relays run a libp2p [circuit relay](https://docs.libp2p.io/concepts/nat/circuit-relay/) server that allows charon clusters to perform peer discovery and for charon clients behind NAT gateways to be communicated with. If you want to self-host a relay for your cluster(s) the following command will start one. + +```markdown +charon relay --help +Starts a libp2p relay that charon nodes can use to bootstrap their p2p cluster + +Usage: + charon relay [flags] + +Flags: + --auto-p2pkey Automatically create a p2pkey (secp256k1 private key used for p2p authentication and ENR) if none found in data directory. (default true) + --data-dir string The directory where charon will store all its internal data (default ".charon") + -h, --help Help for relay + --http-address string Listening address (ip and port) for the relay http server serving runtime ENR. (default "127.0.0.1:3640") + --log-color string Log color; auto, force, disable. (default "auto") + --log-format string Log format; console, logfmt or json (default "console") + --log-level string Log level; debug, info, warn or error (default "info") + --loki-addresses strings Enables sending of logfmt structured logs to these Loki log aggregation server addresses. This is in addition to normal stderr logs. + --loki-service string Service label sent with logs to Loki. (default "charon") + --monitoring-address string Listening address (ip and port) for the prometheus and pprof monitoring http server. (default "127.0.0.1:3620") + --p2p-advertise-private-addresses Enable advertising of libp2p auto-detected private addresses. This doesn't affect manually provided p2p-external-ip/hostname. + --p2p-allowlist string Comma-separated list of CIDR subnets for allowing only certain peer connections. Example: 192.168.0.0/16 would permit connections to peers on your local network only. The default is to accept all connections. + --p2p-denylist string Comma-separated list of CIDR subnets for disallowing certain peer connections. Example: 192.168.0.0/16 would disallow connections to peers on your local network. The default is to accept all connections. + --p2p-disable-reuseport Disables TCP port reuse for outgoing libp2p connections. + --p2p-external-hostname string The DNS hostname advertised by libp2p. This may be used to advertise an external DNS. + --p2p-external-ip string The IP address advertised by libp2p. This may be used to advertise an external IP. + --p2p-max-connections int Libp2p maximum number of peers that can connect to this relay. (default 16384) + --p2p-max-reservations int Updates max circuit reservations per peer (each valid for 30min) (default 512) + --p2p-relay-loglevel string Libp2p circuit relay log level. E.g., debug, info, warn, error. + --p2p-relays strings Comma-separated list of libp2p relay URLs or multiaddrs. (default [https://0.relay.obol.tech]) + --p2p-tcp-address strings Comma-separated list of listening TCP addresses (ip and port) for libP2P traffic. Empty default doesn't bind to local port therefore only supports outgoing connections. +``` diff --git a/versioned_docs/version-v0.17.0/charon/cluster-configuration.md b/versioned_docs/version-v0.17.0/charon/cluster-configuration.md new file mode 100644 index 0000000000..aab4104033 --- /dev/null +++ b/versioned_docs/version-v0.17.0/charon/cluster-configuration.md @@ -0,0 +1,161 @@ +--- +description: Documenting a Distributed Validator Cluster in a standardised file format +sidebar_position: 3 +--- + +# Cluster configuration + +:::caution +These cluster definition and cluster lock files are a work in progress. The intention is for the files to be standardised for operating distributed validators via the [EIP process](https://eips.ethereum.org/) when appropriate. +::: + +This document describes the configuration options for running a charon client or cluster. + +A charon cluster is configured in two steps: + +- `cluster-definition.json` which defines the intended cluster configuration before keys have been created in a distributed key generation ceremony. +- `cluster-lock.json` which includes and extends `cluster-definition.json` with distributed validator BLS public key shares. + +In the case of a solo operator running a cluster, the [`charon create cluster`](./charon-cli-reference.md#create-a-full-cluster-locally) command combines both steps into one and just outputs the final `cluster-lock.json` without a DKG step. + +## Cluster Definition File + +The `cluster-definition.json` is provided as input to the DKG which generates keys and the `cluster-lock.json` file. + +### Using the CLI + +The [`charon create dkg`](./charon-cli-reference.md#creating-the-configuration-for-a-dkg-ceremony) command is used to create the `cluster-definition.json` file which is used as input to `charon dkg`. + +The schema of the `cluster-definition.json` is defined as: + +```json +{ + "name": "best cluster", // Optional cosmetic identifier + "creator": { + "address": "0x123..abfc", //ETH1 address of the creator + "config_signature": "0x123654...abcedf" // EIP712 Signature of config_hash using creator privkey + }, + "operators": [ + { + "address": "0x123..abfc", // ETH1 address of the operator + "enr": "enr://abcdef...12345", // Charon node ENR + "config_signature": "0x123456...abcdef", // EIP712 Signature of config_hash by ETH1 address priv key + "enr_signature": "0x123654...abcedf" // EIP712 Signature of ENR by ETH1 address priv key + } + ], + "uuid": "1234-abcdef-1234-abcdef", // Random unique identifier. + "version": "v1.2.0", // Schema version + "timestamp": "2022-01-01T12:00:00+00:00", // Creation timestamp + "num_validators": 2, // Number of distributed validators to be created in cluster-lock.json + "threshold": 3, // Optional threshold required for signature reconstruction + "validators": [ + { + "fee_recipient_address": "0x123..abfc", // ETH1 fee_recipient address of validator + "withdrawal_address": "0x123..abfc" // ETH1 withdrawal address of validator + }, + { + "fee_recipient_address": "0x123..abfc", // ETH1 fee_recipient address of validator + "withdrawal_address": "0x123..abfc" // ETH1 withdrawal address of validator + } + ], + "dkg_algorithm": "foo_dkg_v1", // Optional DKG algorithm for key generation + "fork_version": "0x00112233", // Chain/Network identifier + "config_hash": "0xabcfde...acbfed", // Hash of the static (non-changing) fields + "definition_hash": "0xabcdef...abcedef" // Final hash of all fields +} +``` + +### Using the DV Launchpad + +- A [`leader/creator`](docs/int/quickstart/group/index.md), that wishes to coordinate the creation of a new Distributed Validator Cluster navigates to the launchpad and selects "Create new Cluster" +- The `leader/creator` uses the user interface to configure all of the important details about the cluster including: + - The `Withdrawal Address` for the created validators + - The `Fee Recipient Address` for block proposals if it differs from the withdrawal address + - The number of distributed validators to create + - The list of participants in the cluster specified by Ethereum address(/ENS) + - The threshold of fault tolerance required +- These key pieces of information form the basis of the cluster configuration. These fields (and some technical fields like DKG algorithm to use) are serialized and merklized to produce the definition's `cluster_definition_hash`. This merkle root will be used to confirm that there is no ambiguity or deviation between definitions when they are provided to charon nodes. +- Once the `leader/creator` is satisfied with the configuration they publish it to the launchpad's data availability layer for the other participants to access. (For early development the launchpad will use a centralized backend db to store the cluster configuration. Near production, solutions like IPFS or arweave may be more suitable for the long term decentralization of the launchpad.) + +## Cluster Lock File + +The `cluster-lock.json` has the following schema: + +```json +{ + "cluster_definition": {...}, // Cluster definiition json, identical schema to above, + "distributed_validators": [ // Length equal to cluster_definition.num_validators. + { + "distributed_public_key": "0x123..abfc", // DV root pubkey + "public_shares": [ "abc...fed", "cfd...bfe"], // Length equal to cluster_definition.operators + "fee_recipient": "0x123..abfc" // Defaults to withdrawal address if not set, can be edited manually + } + ], + "lock_hash": "abcdef...abcedef", // Config_hash plus distributed_validators + "signature_aggregate": "abcdef...abcedef" // BLS aggregate signature of the lock hash signed by each DV pubkey. +} +``` + +## Cluster Size and Resilience + +The cluster size (the number of nodes/operators in the cluster) determines the resilience of the cluster; its ability remain operational under diverse failure scenarios. +Larger clusters can tolerate more faulty nodes. +However, increased cluster size implies higher operational costs and potential network latency, which may negatively affect performance + +Optimal cluster size is therefore trade-off between resilience (larger is better) vs cost-efficiency and performance (smaller is better). + +Cluster resilience can be broadly classified into two categories: + - **[Byzantine Fault Tolerance (BFT)](https://en.wikipedia.org/wiki/Byzantine_fault)** - the ability to tolerate nodes that are actively trying to disrupt the cluster. + - **[Crash Fault Tolerance (CFT)](https://en.wikipedia.org/wiki/Fault_tolerance)** - the ability to tolerate nodes that have crashed or are otherwise unavailable. + +Different cluster sizes tolerate different counts of byzantine vs crash nodes. +In practice, hardware and software crash relatively frequently, while byzantine behaviour is relatively uncommon. +However, Byzantine Fault Tolerance is crucial for trust minimised systems like distributed validators. +Thus, cluster size can be chosen to optimise for either BFT or CFT. + +The table below lists different cluster sizes and their characteristics: + - `Cluster Size` - the number of nodes in the cluster. + - `Threshold` - the minimum number of nodes that must collaborate to reach consensus quorum and to create signatures. + - `BFT #` - the maximum number of byzantine nodes that can be tolerated. + - `CFT #` - the maximum number of crashed nodes that can be tolerated. + +| Cluster Size | Threshold | BFT # | CFT # | Note | +|--------------|-----------|-------|-------|------------------------------------| +| 1 | 1 | 0 | 0 | ❌ Invalid: Not CFT nor BFT! | +| 2 | 2 | 0 | 0 | ❌ Invalid: Not CFT nor BFT! | +| 3 | 2 | 0 | 1 | ⚠️ Warning: CFT but not BFT! | +| 4 | 3 | 1 | 1 | ✅ CFT and BFT optimal for 1 faulty | +| 5 | 4 | 1 | 1 | | +| 6 | 4 | 1 | 2 | ✅ CFT optimal for 2 crashed | +| 7 | 5 | 2 | 2 | ✅ BFT optimal for 2 byzantine | +| 8 | 6 | 2 | 2 | | +| 9 | 6 | 2 | 3 | ✅ CFT optimal for 3 crashed | +| 10 | 7 | 3 | 3 | ✅ BFT optimal for 3 byzantine | +| 11 | 8 | 3 | 3 | | +| 12 | 8 | 3 | 4 | ✅ CFT optimal for 4 crashed | +| 13 | 9 | 4 | 4 | ✅ BFT optimal for 4 byzantine | +| 14 | 10 | 4 | 4 | | +| 15 | 10 | 4 | 5 | ✅ CFT optimal for 5 crashed | +| 16 | 11 | 5 | 5 | ✅ BFT optimal for 5 byzantine | +| 17 | 12 | 5 | 5 | | +| 18 | 12 | 5 | 6 | ✅ CFT optimal for 6 crashed | +| 19 | 13 | 6 | 6 | ✅ BFT optimal for 6 byzantine | +| 20 | 14 | 6 | 6 | | +| 21 | 14 | 6 | 7 | ✅ CFT optimal for 7 crashed | +| 22 | 15 | 7 | 7 | ✅ BFT optimal for 7 byzantine | + +The table above is determined by the QBFT consensus algorithm with the +following formulas from [this](https://arxiv.org/pdf/1909.10194.pdf) paper: + +``` +n = cluster size + +Threshold: min number of honest nodes required to reach quorum given size n +Quarom(n) = ceiling(2n/3) + +BFT #: max number of faulty (byzantine) nodes given size n +f(n) = floor((n-1)/3) + +CFT #: max number of unavailable (crashed) nodes given size n +crashed(n) = n - Quorum(n) +``` diff --git a/versioned_docs/version-v0.17.0/charon/dkg.md b/versioned_docs/version-v0.17.0/charon/dkg.md new file mode 100644 index 0000000000..86b0e28d2d --- /dev/null +++ b/versioned_docs/version-v0.17.0/charon/dkg.md @@ -0,0 +1,73 @@ +--- +description: Generating private keys for a Distributed Validator requires a Distributed Key Generation (DKG) Ceremony. +sidebar_position: 2 +--- + +# Distributed Key Generation + +## Overview + +A [**distributed validator key**](docs/int/key-concepts.md#distributed-validator-key) is a group of BLS private keys that together operate as a threshold key for participating in proof-of-stake consensus. + +To make a distributed validator with no fault-tolerance (i.e. all nodes need to be online to sign every message), due to the BLS signature scheme used by Proof of Stake Ethereum, each key share could be chosen by operators independently. However, to create a distributed validator that can stay online despite a subset of its nodes going offline, the key shares need to be generated together (4 randomly chosen points on a graph don't all necessarily sit on the same order three curve). To do this in a secure manner with no one party being trusted to distribute the keys requires what is known as a [**distributed key generation ceremony**](docs/int/key-concepts.md#distributed-validator-key-generation-ceremony). + +The charon client has the responsibility of securely completing a distributed key generation ceremony with its counterparty nodes. The ceremony configuration is outlined in a [cluster definition](../charon/cluster-configuration). + +## Actors Involved + +A distributed key generation ceremony involves `Operators` and their `Charon clients`. + +- An `Operator` is identified by their Ethereum address. They will sign a message with this address to authorize their charon client to take part in the DKG ceremony. + +- A `Charon client` is also identified by a public/private key pair, in this instance, the public key is represented as an [Ethereum Node Record](https://eips.ethereum.org/EIPS/eip-778) (ENR). This is a standard identity format for both EL and CL clients. These ENRs are used by each charon node to identify its cluster peers over the internet, and to communicate with one another in an [end to end encrypted manner](https://github.com/libp2p/go-libp2p/tree/master/p2p/security/noise). These keys need to be created (and backed up) by each operator before they can participate in a cluster creation. + +## Cluster Definition Creation + +This cluster definition specifies the intended cluster configuration before keys have been created in a distributed key generation ceremony. The `cluster-definition.json` file can be created with the help of the [Distributed Validator Launchpad](./cluster-configuration.md#using-the-dv-launchpad) or via the [CLI](./cluster-configuration.md#using-the-cli). + +## Carrying out the DKG ceremony + +Once all participants have signed the cluster definition, they can load the `cluster-definition` file into their charon client, and the client will attempt to complete the DKG. + +Charon will read the ENRs in the definition, confirm that its ENR is present, and then will reach out to relays that are deployed to find the other ENRs on the network. (Fresh ENRs just have a public key and an IP address of 0.0.0.0 until they are loaded into a live charon client, which will update the IP address and increment the ENRs nonce and resign with the clients private key. If an ENR with a higher nonce is seen by a charon client, they will update the IP address of that ENR in their address book.) + +Once all clients in the cluster can establish a connection with one another and they each complete a handshake (confirm everyone has a matching `cluster_definition_hash`), the ceremony begins. + +No user input is required, charon does the work and outputs the following files to each machine and then exits. + +## Backing up the ceremony artifacts + +At the end of a DKG ceremony, each operator will have a number of files outputted by their charon client based on how many distributed validators the group chose to generate together. + +These files are: + +- **Validator keystore(s):** These files will be loaded into the operator's validator client and each file represents one share of a Distributed Validator. +- **A distributed validator cluster lock file:** This `cluster-lock.json` file contains the configuration a distributed validator client like charon needs to join a cluster capable of operating a number of distributed validators. +- **Validator deposit data:** This file is used to activate one or more distributed validators on the Ethereum network. + +Once the ceremony is complete, all participants should take a backup of the created files. In future versions of charon, if a participant loses access to these key shares, it will be possible to use a key re-sharing protocol to swap the participants old keys out of a distributed validator in favor of new keys, allowing the rest of a cluster to recover from a set of lost key shares. However for now, without a backup, the safest thing to do would be to exit the validator. + +## DKG Verification + +For many use cases of distributed validators, the funder/depositor of the validator may not be the same person as the key creators/node operators, as (outside of the base protocol) stake delegation is a common phenomenon. This handover of information introduces a point of trust. How does someone verify that a proposed validator `deposit data` corresponds to a real, fair, DKG with participants the depositor expects? + +There are a number of aspects to this trust surface that can be mitigated with a "Don't trust, verify" model. Verification for the time being is easier off chain, until things like a [BLS precompile](https://eips.ethereum.org/EIPS/eip-2537) are brought into the EVM, along with cheap ZKP verification on chain. Some of the questions that can be asked of Distributed Validator Key Generation Ceremonies include: + +- Do the public key shares combine together to form the group public key? + - This can be checked on chain as it does not require a pairing operation + - This can give confidence that a BLS pubkey represents a Distributed Validator, but does not say anything about the custody of the keys. (e.g. Was the ceremony sybil attacked, did they collude to reconstitute the group private key etc.) +- Do the created BLS public keys attest to their `cluster_definition_hash`? + - This is to create a backwards link between newly created BLS public keys and the operator's eth1 addresses that took part in their creation. + - If a proposed distributed validator BLS group public key can produce a signature of the `cluster_definition_hash`, it can be inferred that at least a threshold of the operators signed this data. + - As the `cluster_definition_hash` is the same for all distributed validators created in the ceremony, the signatures can be aggregated into a group signature that verifies all created group keys at once. This makes it cheaper to verify a number of validators at once on chain. +- Is there either a VSS or PVSS proof of a fair DKG ceremony? + - VSS (Verifiable Secret Sharing) means only operators can verify fairness, as the proof requires knowledge of one of the secrets. + - PVSS (Publicly Verifiable Secret Sharing) means anyone can verify fairness, as the proof is usually a Zero Knowledge Proof. + - A PVSS of a fair DKG would make it more difficult for operators to collude and undermine the security of the Distributed Validator. + - Zero Knowledge Proof verification on chain is currently expensive, but is becoming achievable through the hard work and research of the many ZK based teams in the industry. + +## Appendix + +### Sample Configuration and Lock Files + +Refer to the details [here](../charon/cluster-configuration). diff --git a/versioned_docs/version-v0.17.0/charon/intro.md b/versioned_docs/version-v0.17.0/charon/intro.md new file mode 100644 index 0000000000..767a6a3ded --- /dev/null +++ b/versioned_docs/version-v0.17.0/charon/intro.md @@ -0,0 +1,82 @@ +--- +description: Charon - The Distributed Validator Client +sidebar_position: 1 +--- + +# Introduction + +This section introduces and outlines the Charon *[kharon]* middleware, Obol's implementation of DVT. Please see the [key concepts](/docs/int/key-concepts) section as background and context. + +## What is Charon? + +Charon is a GoLang-based, HTTP middleware built by Obol to enable any existing Ethereum validator clients to operate together as part of a distributed validator. + +Charon sits as a middleware between a normal validating client and its connected beacon node, intercepting and proxying API traffic. Multiple Charon clients are configured to communicate together to come to consensus on validator duties and behave as a single unified proof-of-stake validator together. The nodes form a cluster that is _byzantine-fault tolerant_ and continues to progress assuming a supermajority of working/honest nodes is met. + +![Charon Cluster](/img/DVCluster.png) + +## Charon Architecture +Charon is an Ethereum proof of stake distributed validator (DV) client. Like any validator client, its main purpose is to perform validation duties for the Beacon Chain, primarily attestations and block proposals. The beacon client handles a lot of the heavy lifting, leaving the validator client to focus on fetching duty data, signing that data, and submitting it back to the beacon client. + +Charon is designed as a generic event-driven workflow with different components coordinating to perform validation duties. All duties follow the same flow, the only difference being the signed data. The workflow can be divided into phases consisting of one or more components: + +![Charon Workflow](/img/workflow.jpg) + +### Determine **when** duties need to be performed +The beacon chain is divided into [slots](https://eth2book.info/bellatrix/part3/config/types/#slot) and [epochs](https://eth2book.info/bellatrix/part3/config/types/#epoch), which divides it into deterministically fixed-size time chunks. +The first step is to determine when (which slot/epoch) duties need to be performed. This is done by the `scheduler` component. +It queries the beacon node to detect which validators defined in the cluster lock are active, and what duties they need to perform for +the upcoming epoch and slots. When such a slot starts, the `scheduler` emits an event indicating which validator needs to perform what duty. + +### Fetch and come to consensus on **what** data to sign +A DV cluster consists of multiple operators each provided with one of the M-of-N threshold BLS private key shares per validator. +The key shares are imported into the validator clients which produce partial signatures. +Charon threshold aggregates these partial signatures before broadcasting them to the Beacon Chain. +*But to threshold aggregate partial signatures, each validator must sign the same data.* +The cluster must therefore coordinate and come to a consensus on what data to sign. + +`Fetcher` fetches the unsigned duty data from the beacon node upon receiving an event from `Scheduler`. +For attestations, this is the unsigned attestation, for block proposals, this is the unsigned block. + +The `Consensus` component listens to events from Fetcher and starts a [QBFT](https://docs.goquorum.consensys.net/configure-and-manage/configure/consensus-protocols/qbft/) consensus game with the other +Charon nodes in the cluster for that specific duty and slot. +When consensus is reached, the resulting unsigned duty data is stored in the `DutyDB`. + +### **Wait** for the VC to sign +Charon is a **middleware** distributed validator client. That means Charon doesn’t have access to the +validator private key shares and cannot sign anything on demand. +Instead, operators import the key shares into industry-standard validator clients (VC) +that are configured to connect to their local Charon client instead of their local Beacon node directly. + +Charon, therefore, serves the [Ethereum Beacon Node API](https://ethereum.github.io/beacon-APIs/#/) from the `ValidatorAPI` component and +intercepts some endpoints while proxying other endpoints directly to the upstream Beacon node. + +The VC queries the `ValidatorAPI` for unsigned data which is retrieved from the `DutyDB`. It then signs it and submits it +back to the `ValidatorAPI` which stores it in the `PartialSignatureDB`. + +### **Share** partial signatures +The `PartialSignatureDB` stores the partially signed data submitted by the local Charon client’s VC. +But it also stores all the partial signatures submitted by the VCs of other peers in the cluster. +This is achieved by the `PartialSignatureExchange` component that exchanges partial signatures between all peers in the cluster. +All charon clients, therefore, store all partial signatures the cluster generates. + +### **Threshold Aggregate** partial signatures +The `SignatureAggregator` is invoked as soon as sufficient (any M of N) partial signatures are stored in the `PartialSignatureDB`. +It performs BLS threshold aggregation of the partial signatures resulting in a final signature that is valid for the beacon chain. + +### **Broadcast** final signature +Finally, the `Broadcaster` component broadcasts the final threshold aggregated signature to the Beacon client, thereby completing the duty. + +### Ports + +The following is an outline of the services that can be exposed by charon. + +- **:3600** - The validator REST API. This is the port that serves the consensus layer's [beacon node API](https://ethereum.github.io/beacon-APIs/). This is the port validator clients should talk to instead of their standard consensus client REST API port. Charon subsequently proxies these requests to the upstream consensus client specified by `--beacon-node-endpoints`. + +- **:3610** - Charon P2P port. This is the port that charon clients use to communicate with one another via TCP. This endpoint should be port-forwarded on your router and exposed publicly, preferably on a static IP address. This IP address should then be set on the charon run command with `--p2p-external-ip` or `CHARON_P2P_EXTERNAL_IP`. + +- **:3620** - Monitoring port. This port hosts a webserver that serves prometheus metrics on `/metrics`, a readiness endpoint on `/readyz` and a liveness endpoint on `/livez`, and a pprof server on `/debug/pprof`. This port should not be exposed publicly. + +## Getting started + +For more information on running charon, take a look at our [Quickstart Guides](docs/int/quickstart/index.md). diff --git a/versioned_docs/version-v0.17.0/charon/networking.md b/versioned_docs/version-v0.17.0/charon/networking.md new file mode 100644 index 0000000000..5b56a09dcc --- /dev/null +++ b/versioned_docs/version-v0.17.0/charon/networking.md @@ -0,0 +1,99 @@ +--- +description: Networking +sidebar_position: 4 +--- + +# Charon networking + +## Overview + +This document describes Charon's networking model which can be divided into two parts: the [*internal validator stack*](#internal-validator-stack) and the [*external p2p network*](#external-p2p-network). + +## Internal Validator Stack + +Internal Validator Stack
    + +Charon is a middleware DVT client and is therefore connected to an upstream beacon node and a downstream validator client is connected to it. +Each operator should run the whole validator stack (all 4 client software types), either on the same machine or on different machines. The networking between +the nodes should be private and not exposed to the public internet. + +Related Charon configuration flags: +- `--beacon-node-endpoints`: Connects Charon to one or more beacon nodes. +- `--validator-api-address`: Address for Charon to listen on and serve requests from the validator client. + +## External P2P Network + +![External P2P Network](/img/ExternalP2PNetwork.png) +The Charon clients in a DV cluster are connected to each other via a small p2p network consisting of only the clients in the cluster. Peer IP addresses are +discovered via an external "relay" server. The p2p connections are over the public internet so the charon p2p port must be publicly accessible. Charon leverages +the popular [libp2p](https://libp2p.io/) protocol. + +Related [Charon configuration flags](docs/charon/charon-cli-reference.md): +- `--p2p-tcp-addresses`: Addresses for Charon to listen on and serve p2p requests. +- `--p2p-relays`: Connect charon to one or more relay servers. +- `--private-key-file`: Private key identifying the charon client. + +### LibP2P Authentication and Security + +Each charon client has a secp256k1 private key. The associated public key is encoded into the [cluster lock file](cluster-configuration.md#Cluster-Lock-File) to identify the nodes in the cluster. +For ease of use and to align with the Ethereum ecosystem, Charon encodes these public keys in the [ENR format](https://eips.ethereum.org/EIPS/eip-778), +not in [libp2p’s Peer ID format](https://docs.libp2p.io/concepts/fundamentals/peers/). + +:::caution +Each Charon node's secp256k1 private key is critical for authentication and must be kept secure to prevent cluster compromise. + +Do not use the same key across multiple clusters, as this can lead to security issues. + +For more on p2p security, refer to [libp2p's article](https://docs.libp2p.io/concepts/security/security-considerations). +::: + +Charon currently only supports libp2p tcp connections with [noise](https://noiseprotocol.org/) security and only accepts incoming libp2p connections from peers defined in the cluster lock. + +### LibP2P Relays and Peer Discovery + +Relays are simple libp2p servers that are publicly accessible supporting the [circuit-relay](https://docs.libp2p.io/concepts/nat/circuit-relay/) protocol. +Circuit-relay is a libp2p transport protocol that routes traffic between two peers over a third-party “relay” peer. + +Obol hosts a publicly accessible relay at https://0.relay.obol.tech and will work with other organisations in the community to host alternatives Anyone can host their own relay server for their DV cluster. + +Each charon node knows which peers are in the cluster from the ENRs in the cluster lock file, but their IP addresses are unknown. By connecting to the same relay, +nodes establish “relay connections” to each other. Once connected via relay they exchange their known public addresses via libp2p’s [identify](https://docs.libp2p.io/concepts/fundamentals/protocols/#identify) +protocol. The relay connection is then upgraded to a direct connection. If a node’s public IP changes, nodes once again connect via relay, exchange the new IP, and then connect directly once again. + +Note that in order for two peers to discover each other, they must connect to the same relay. Cluster operators should therefore coordinate which relays to use. + +Libp2p’s [identify](https://docs.libp2p.io/concepts/fundamentals/protocols/#identify) protocol attempts to automatically detect the public IP address of a charon +client without the need to explicitly configure it. If this however fails, the following two configuration flags can be used to explicitly set the publicly advertised +address: +- `--p2p-external-ip`: Explicitly sets the external IP address. +- `--p2p-external-hostname`: Explicitly sets the external DNS host name. + +:::caution +If a pair of charon clients are not publicly accessible, due to being behind a NAT, they will not be able to upgrade their relay connections to a direct connection. +Even though this is supported, it isn’t recommended as relay connections introduce additional latency and reduced throughput and will result in decreased validator effectiveness +and possible missed block proposals and attestations. +::: + +Libp2p’s circuit-relay connections are end-to-end encrypted, even though relay servers accept connections between nodes from multiple different clusters, relays are merely +routing opaque connections. And since Charon only accepts incoming connections from other peers in its cluster, the use of a relay doesn’t allow connections between clusters. + +Only the following three libp2p protocols are established between a charon node and a relay itself: +- [circuit-relay](https://docs.libp2p.io/concepts/nat/circuit-relay/): To establish relay e2e encrypted connections between two peers in a cluster.
    +- [identify](https://docs.libp2p.io/concepts/fundamentals/protocols/#identify): Auto-detection of public IP addresses to share with other peers in the cluster.
    +- [peerinfo](https://github.com/ObolNetwork/charon/blob/main/app/peerinfo/peerinfo.go): Exchanges basic application [metadata](https://github.com/ObolNetwork/charon/blob/main/app/peerinfo/peerinfopb/v1/peerinfo.proto) for improved operational metrics and observability.
    + +All other charon protocols are only established between nodes in the same cluster. + +### Scalable Relay Clusters + +In order for a charon client to connect to a relay, it needs the relay's [multiaddr](https://docs.libp2p.io/concepts/fundamentals/addressing/) (containing its public key and IP address). +But a single multiaddr can only point to a single relay server which can easily be overloaded if too many clusters connect to it. Charon therefore supports resolving a relay’s multiaddr +via HTTP GET request. Since charon also includes the unique `cluster-hash` header in this request, the relay provider can use +[consistent header-based load-balancing](https://cloud.google.com/load-balancing/docs/https/traffic-management-global#traffic_steering_header-based_routing) to map clusters to one of many relays using a single HTTP address. + +The relay supports serving its runtime public multiaddrs via its `--http-address` flag. + +E.g., https://0.relay.obol.tech is actually a load-balancer that routes HTTP requests to one of many relays based on the `cluster-hash` header returning the target relay’s multiaddr +which the charon client then uses to connect to that relay. + +The charon `--p2p-relays` flag therefore supports both multiaddrs as well as HTTP URls. diff --git a/versioned_docs/version-v0.17.0/dvl/_category_.json b/versioned_docs/version-v0.17.0/dvl/_category_.json new file mode 100644 index 0000000000..b7a2cf3a69 --- /dev/null +++ b/versioned_docs/version-v0.17.0/dvl/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "DV Launchpad", + "position": 4, + "collapsed": true +} diff --git a/versioned_docs/version-v0.17.0/dvl/intro.md b/versioned_docs/version-v0.17.0/dvl/intro.md new file mode 100644 index 0000000000..f227f32130 --- /dev/null +++ b/versioned_docs/version-v0.17.0/dvl/intro.md @@ -0,0 +1,18 @@ +--- +description: A dapp to securely create Distributed Validators alone or with a group. +sidebar_position: 1 +--- + +# Introduction + +![DV Launchpad Promo Image](/img/DistributeYourValidators.svg) + +In order to activate an Ethereum validator, 32 ETH must be deposited into the official deposit contract. + +The vast majority of users that created validators to date have used the **[~~Eth2~~ Staking Launchpad](https://launchpad.ethereum.org/)**, a public good open source website built by the Ethereum Foundation alongside participants that later went on to found Obol. This tool has been wildly successful in the safe and educational creation of a significant number of validators on the Ethereum mainnet. + +To facilitate the generation of distributed validator keys amongst remote users with high trust, the Obol Network developed and maintains a website that enables a group of users to come together and create these threshold keys: [**The DV Launchpad**](https://goerli.launchpad.obol.tech/). + +## Getting started + +For more information on running charon in a UI friendly way through the DV Launchpad, take a look at our [Quickstart Guides](docs/int/quickstart/index.md). \ No newline at end of file diff --git a/versioned_docs/version-v0.17.0/fr/_category_.json b/versioned_docs/version-v0.17.0/fr/_category_.json new file mode 100644 index 0000000000..72d8434c21 --- /dev/null +++ b/versioned_docs/version-v0.17.0/fr/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "Further reading", + "position": 11, + "collapsed": true +} diff --git a/versioned_docs/version-v0.17.0/fr/eth.md b/versioned_docs/version-v0.17.0/fr/eth.md new file mode 100644 index 0000000000..9de166df23 --- /dev/null +++ b/versioned_docs/version-v0.17.0/fr/eth.md @@ -0,0 +1,131 @@ +# Ethereum resources + +This page serves material necessary to catch up with the current state of Ethereum proof-of-stake development and provides readers with the base knowledge required to assist with the growth of Obol. Whether you are an expert on all things Ethereum or are new to the blockchain world entirely, there are appropriate resources here that will help you get up to speed. + +## **Ethereum fundamentals** + +### Introduction + +* [What is Ethereum?](http://ethdocs.org/en/latest/introduction/what-is-ethereum.html) +* [How Does Ethereum Work Anyway?](https://medium.com/@preethikasireddy/how-does-ethereum-work-anyway-22d1df506369) +* [Ethereum Introduction](https://ethereum.org/en/what-is-ethereum/) +* [Ethereum Foundation](https://ethereum.org/en/foundation/) +* [Ethereum Wiki](https://eth.wiki/) +* [Ethereum Research](https://ethresear.ch/) +* [Ethereum White Paper](https://github.com/ethereum/wiki/wiki/White-Paper) +* [What is Hashing?](https://blockgeeks.com/guides/what-is-hashing/) +* [Hashing Algorithms and Security](https://www.youtube.com/watch?v=b4b8ktEV4Bg) +* [Understanding Merkle Trees](https://www.codeproject.com/Articles/1176140/Understanding-Merkle-Trees-Why-use-them-who-uses-t) +* [Ethereum Block Architecture](https://ethereum.stackexchange.com/questions/268/ethereum-block-architecture/6413#6413) +* [What is an Ethereum Token?](https://blockgeeks.com/guides/ethereum-token/) +* [What is Ethereum Gas?](https://blockgeeks.com/guides/ethereum-gas-step-by-step-guide/) +* [Client Implementations](https://eth.wiki/eth1/clients) + +## **ETH2 fundamentals** + +*Disclaimer: Because some parts of Ethereum consensus are still an active area of research and/or development, some resources may be outdated.* + +### Introduction and specifications + +* [The Explainer You Need to Read First](https://ethos.dev/beacon-chain/) +* [Official Specifications](https://github.com/ethereum/eth2.0-specs) +* [Annotated Spec](https://benjaminion.xyz/eth2-annotated-spec/) +* [Another Annotated Spec](https://notes.ethereum.org/@djrtwo/Bkn3zpwxB) +* [Rollup-Centric Roadmap](https://ethereum-magicians.org/t/a-rollup-centric-ethereum-roadmap/4698) + +### Sharding + +* [Blockchain Scalability: Why?](https://blockgeeks.com/guides/blockchain-scalability/) +* [What Are Ethereum Nodes and Sharding](https://blockgeeks.com/guides/what-are-ethereum-nodes-and-sharding/) +* [How to Scale Ethereum: Sharding Explained](https://medium.com/prysmatic-labs/how-to-scale-ethereum-sharding-explained-ba2e283b7fce) +* [Sharding FAQ](https://eth.wiki/sharding/Sharding-FAQs) +* [Sharding Introduction: R&D Compendium](https://eth.wiki/en/sharding/sharding-introduction-r-d-compendium) + +### Peer-to-peer networking + +* [Ethereum Peer to Peer Networking](https://geth.ethereum.org/docs/interface/peer-to-peer) +* [P2P Library](https://libp2p.io/) +* [Discovery Protocol](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md) + +### Latest News + +* [Ethereum Blog](https://blog.ethereum.org/) +* [News from Ben Edgington](https://hackmd.io/@benjaminion/eth2_news) + +### Prater Testnet Blockchain + +* [Launchpad](https://goerli.launchpad.ethereum.org/en/) +* [Beacon Chain Explorer](https://prater.beaconcha.in/) + +### Mainnet Blockchain + +* [Launchpad](https://launchpad.ethereum.org/en/) +* [Beacon Chain Explorer](https://beaconcha.in/) +* [Another Beacon Chain Explorer](https://explorer.bitquery.io/eth2) +* [Validator Queue Statistics](https://eth2-validator-queue.web.app/index.html) +* [Slashing Detector](https://twitter.com/eth2slasher) + +### Client Implementations + +* [Prysm](https://github.com/prysmaticlabs/prysm) developed in Golang and maintained by [Prysmatic Labs](https://prysmaticlabs.com/) +* [Lighthouse](https://github.com/sigp/lighthouse) developed in Rust and maintained by [Sigma Prime](https://sigmaprime.io/) +* [Lodestar](https://github.com/ChainSafe/lodestar) developed in TypeScript and maintained by [ChainSafe Systems](https://chainsafe.io/) +* [Nimbus](https://github.com/status-im/nimbus-eth2) developed in Nim and maintained by [status](https://status.im/) +* [Teku](https://github.com/ConsenSys/teku) developed in Java and maintained by [ConsenSys](https://consensys.net/) + +## Other + +### Serenity concepts + +* [Sharding Concepts Mental Map](https://www.mindomo.com/zh/mindmap/sharding-d7cf8b6dee714d01a77388cb5d9d2a01) +* [Taiwan Sharding Workshop Notes](https://hackmd.io/s/HJ_BbgCFz#%E2%9F%A0-General-Introduction) +* [Sharding Research Compendium](http://notes.ethereum.org/s/BJc_eGVFM) +* [Torus Shaped Sharding Network](https://ethresear.ch/t/torus-shaped-sharding-network/1720/8) +* [General Theory of Sharding](https://ethresear.ch/t/a-general-theory-of-what-quadratically-sharded-validation-is/1730/10) +* [Sharding Design Compendium](https://ethresear.ch/t/sharding-designs-compendium/1888/25) + +### Serenity research posts + +* [Sharding v2.1 Spec](https://notes.ethereum.org/SCIg8AH5SA-O4C1G1LYZHQ) +* [Casper/Sharding/Beacon Chain FAQs](https://notes.ethereum.org/9MMuzWeFTTSg-3Tz_YeiBA?view) +* [RETIRED! Sharding Phase 1 Spec](https://ethresear.ch/t/sharding-phase-1-spec-retired/1407/92) +* [Exploring the Proposer/Collator Spec and Why it Was Retired](https://ethresear.ch/t/exploring-the-proposer-collator-split/1632/24) +* [The Stateless Client Concept](https://ethresear.ch/t/the-stateless-client-concept/172/4) +* [Shard Chain Blocks vs. Collators](https://ethresear.ch/t/shard-chain-blocks-vs-collators/429) +* [Ethereum Concurrency Actors and Per Contract Sharding](https://ethresear.ch/t/ethereum-concurrency-actors-and-per-contract-sharding/375) +* [Future Compatibility for Sharding](https://ethresear.ch/t/future-compatibility-for-sharding/386) +* [Fork Choice Rule for Collation Proposal Mechanisms](https://ethresear.ch/t/fork-choice-rule-for-collation-proposal-mechanisms/922/8) +* [State Execution](https://ethresear.ch/t/state-execution-scalability-and-cost-under-dos-attacks/1048) +* [Fast Shard Chains With Notarization](https://ethresear.ch/t/as-fast-as-possible-shard-chains-with-notarization/1806/2) +* [RANDAO Notary Committees](https://ethresear.ch/t/fork-free-randao/1835/3) +* [Safe Notary Pool Size](https://ethresear.ch/t/safe-notary-pool-size/1728/3) +* [Cross Links Between Main and Shard Chains](https://ethresear.ch/t/cross-links-between-main-chain-and-shards/1860/2) + +### Serenity-related conference talks + +* [Sharding Presentation by Vitalik from IC3-ETH Bootcamp](https://vod.video.cornell.edu/media/Sharding+-+Vitalik+Buterin/1_1xezsfb4/97851101) +* [Latest Research and Sharding by Justin Drake from Tech Crunch](https://www.youtube.com/watch?v=J6xO7DH20Js) +* [Beacon Casper Chain by Vitalik and Justin Drake](https://www.youtube.com/watch?v=GAywmwGToUI) +* [Proofs of Custody by Vitalik and Justin Drake](https://www.youtube.com/watch?v=jRcS9D_gw_o) +* [So You Want To Be a Casper Validator by Vitalik](https://www.youtube.com/watch?v=rl63S6kCKbA) +* [Ethereum Sharding from EDCon by Justin Drake](https://www.youtube.com/watch?v=J4rylD6w2S4) +* [Casper CBC and Sharding by Vlad Zamfir](https://www.youtube.com/watch?v=qDa4xjQq1RE&t=1951s) +* [Casper FFG in Depth by Carl](https://www.youtube.com/watch?v=uQ3IqLDf-oo) +* [Ethereum & Scalability Technology from Asia Pacific ETH meet up by Hsiao Wei](https://www.youtube.com/watch?v=GhuWWShfqBI) + +### Ethereum Virtual Machine + +* [What is the Ethereum Virtual Machine?](https://themerkle.com/what-is-the-ethereum-virtual-machine/) +* [Ethereum VM](https://medium.com/@jeff.ethereum/go-ethereums-jit-evm-27ef88277520) +* [Ethereum Protocol Subtleties](https://github.com/ethereum/wiki/wiki/Subtleties) +* [Awesome Ethereum Virtual Machine](https://github.com/ethereum/wiki/wiki/Ethereum-Virtual-Machine-%28EVM%29-Awesome-List) + +### Ethereum-flavoured WebAssembly + +* [eWASM background, motivation, goals, and design](https://github.com/ewasm/design) +* [The current eWASM spec](https://github.com/ewasm/design/blob/master/eth_interface.md) +* [Latest eWASM community call including live demo of the testnet](https://www.youtube.com/watch?v=apIHpBSdBio) +* [Why eWASM? by Alex Beregszaszi](https://www.youtube.com/watch?v=VF7f_s2P3U0) +* [Panel: entire eWASM team discussion and Q&A](https://youtu.be/ThvForkdPyc?t=119) +* [Ewasm community meetup at ETHBuenosAires](https://www.youtube.com/watch?v=qDzrbj7dtyU) + diff --git a/versioned_docs/version-v0.17.0/fr/golang.md b/versioned_docs/version-v0.17.0/fr/golang.md new file mode 100644 index 0000000000..5bc5686805 --- /dev/null +++ b/versioned_docs/version-v0.17.0/fr/golang.md @@ -0,0 +1,9 @@ +# Golang resources + +* [The Go Programming Language](https://www.amazon.com/Programming-Language-Addison-Wesley-Professional-Computing/dp/0134190440) \(Only recommended book\) +* [Ethereum Development with Go](https://goethereumbook.org) +* [How to Write Go Code](http://golang.org/doc/code.html) +* [The Go Programming Language Tour](http://tour.golang.org/) +* [Getting Started With Go](http://www.youtube.com/watch?v=2KmHtgtEZ1s) +* [Go Official Website](https://golang.org/) + diff --git a/versioned_docs/version-v0.17.0/int/Overview.md b/versioned_docs/version-v0.17.0/int/Overview.md new file mode 100644 index 0000000000..fea46425b8 --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/Overview.md @@ -0,0 +1,47 @@ +--- +sidebar_position: 2 +description: An overview of the Obol network +--- + +## The Network + +The network can be best visualized as a work layer that sits directly on top of base layer consensus. This work layer is designed to provide the base layer with more resiliency and promote decentralization as it scales. As Ethereum matures over the coming years, the community will move onto the next great scaling challenge, which is stake centralization. To effectively mitigate these risks, community building and credible neutrality must be used as primary design principles. + +Obol is focused on scaling consensus by providing permissionless access to Distributed Validators (DV's). We believe that distributed validators will and should make up a large portion of mainnet validator configurations. In preparation for the first wave of adoption, the network currently utilizes a middleware implementation of Distributed Validator Technology (DVT), to enable the operation of distributed validator clusters that can preserve validators current client and remote signing infrastructure. + +Similar to how roll-up technology laid the foundation for L2 scaling implementations, we believe DVT will do the same for scaling consensus while preserving decentralization. Staking infrastructure is entering its protocol phase of evolution, which must include trust-minimized staking networks that can be plugged into at scale. Layers like Obol are critical to the long term viability and resiliency of public networks, especially networks like Ethereum. We believe DVT will evolve into a widely used primitive and will ensure the security, resiliency, and decentralization of the public blockchain networks that adopt it. + +The Obol Network consists of four core public goods: + +- The [Distributed Validator Launchpad](../dvl/intro), a [User Interface](https://goerli.launchpad.obol.tech/) for bootstrapping Distributed Validators +- [Charon](../charon/intro), a middleware client that enables validators to run in a fault-tolerant, distributed manner +- [Obol Managers](../sc/01_introducing-obol-managers.md), a set of solidity smart contracts for the formation of Distributed Validators +- [Obol Testnets](../testnet.md), a set of on-going public incentivized testnets that enable any sized operator to test their deployment before serving for the mainnet Obol Network + +### Sustainable Public Goods + +The Obol Ecosystem is inspired by previous work on Ethereum public goods and experimenting with circular economics. We believe that to unlock innovation in staking use cases, a credibly neutral layer must exist for innovation to flow and evolve vertically. Without this layer highly available uptime will continue to be a moat and stake will accumulate amongst a few products. + +The Obol Network will become an open, community governed, self-sustaining project over the coming months and years. Together we will incentivize, build, and maintain distributed validator technology that makes public networks a more secure and resilient foundation to build on top of. + +## The Vision + +The road to decentralizing stake is a long one. At Obol we have divided our vision into two key versions of distributed validators. + +### V1 - Trusted Distributed Validators + +![Multi Operator DV Cluster](/img/MultiOperator7.png) + +The first version of distributed validators will have dispute resolution out of band. Meaning you need to know and communicate with your counterparty operators if there is an issue with your shared cluster. + +A DV without in-band dispute resolution/incentivization is still extremely valuable. Individuals and staking as a service providers can deploy DVs on their own to make their validators fault tolerant. Groups can run DVs together, but need to bring their own dispute resolution to the table, whether that be a smart contract of their own, a traditional legal service agreement, or simply high trust between the group. + +Obol V1 will utilize retroactive public goods principles to lay the foundation of its economic ecosystem. The Obol Community will responsibly allocate the collected ETH as grants to projects in the staking ecosystem for the entirety of V1. + +### V2 - Trustless Distributed Validators + +V1 of charon serves a small by count, large by stake-weight group of individuals. The long tail of home and small stakers also deserve to have access to fault tolerant validation, but they may not know enough other operators personally to a sufficient level of trust to run a DV cluster together. + +Version 2 of charon will layer in an incentivization scheme to ensure any operator not online and taking part in validation is not earning any rewards. Further incentivization alignment can be achieved with operator bonding requirements that can be slashed for unacceptable performance. + +To add an un-gameable incentivization layer to threshold validation requires complex interactive cryptography schemes, secure off-chain dispute resolution, and EVM support for proofs of the consensus layer state, as a result, this will be a longer and more complex undertaking than V1, hence the delineation between the two. diff --git a/versioned_docs/version-v0.17.0/int/_category_.json b/versioned_docs/version-v0.17.0/int/_category_.json new file mode 100644 index 0000000000..f23f9bc2d0 --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "Getting started", + "position": 2, + "collapsed": false +} diff --git a/versioned_docs/version-v0.17.0/int/faq/_category_.json b/versioned_docs/version-v0.17.0/int/faq/_category_.json new file mode 100644 index 0000000000..c5279cb839 --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/faq/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "FAQ", + "position": 10, + "collapsed": true +} \ No newline at end of file diff --git a/versioned_docs/version-v0.17.0/int/faq/errors.mdx b/versioned_docs/version-v0.17.0/int/faq/errors.mdx new file mode 100644 index 0000000000..f7ac92f8f1 --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/faq/errors.mdx @@ -0,0 +1,389 @@ +--- +sidebar_position: 2 +description: Errors & Resolutions +--- + +# Errors & Resolutions + +All operators should try to restart their nodes and should check if they are on the latest stable version before attempting anything other configuration change as we are still in beta and frequently releasing fixes. You can restart and update with the following commands: + +``` +docker compose down +git pull +docker compose up +``` + +You can check your logs using + +``` +docker compose logs +``` +
    + +

    ENRs & Keys

    +     +
    + +

    What is an ENR?

    +     + An ENR is shorthand for an Ethereum Node Record. + It is a way to represent a node on a public network, with a reliable mechanism to update its information. +

    + At Obol we use ENRs to identify charon nodes to one another such that they can form clusters with the right charon nodes and not impostors. + ENRs have private keys they use to sign updates to the data contained in their ENR. + This private key is by default found at .charon/charon-enr-private-key, and should be kept secure, and not checked into version control. +

    + An ENR looks something like this: 
    enr:-JG4QAgAOXjGFcTIkXBO30aUMzg2YSo1CYV0OH8Sf2s7zA2kFjVC9ZQ_jZZItdE8gA-tUXW-rWGDqEcoQkeJ98Pw7GaGAYFI7eoegmlkgnY0gmlwhCKNyGGJc2VjcDI1NmsxoQI6SQlzw3WGZ_VxFHLhawQFhCK8Aw7Z0zq8IABksuJEJIN0Y3CCPoODdWRwgj6E
    +
    +
    + +

    How do I get my ENR if I want to generate it again?

    +     + +
    +
    + +

    What do I do if lose my charon-enr-private-key?

    +     + +
    +
    + +

    I can't find the keys anywhere

    +     + +
    +
    +
    + +

    Lighthouse

    +     +
    + +

    Downloading historical blocks

    +     This means that Lighthouse is still syncing which will throw a lot of errors down the line. Wait for the sync before moving further. +
    +
    + +

    + Failed to request attester duties error +

    +     Indicates there is something wrong with your lighthouse beacon node. This might be because the request buffer is full as your node is never starting consensus since it never gets the duties. +
    +
    + +

    + Not enough time for a discovery seach error +

    +     This could be linked to a internet connection being to slow or relying on a slow third-party service such as Infura. +
    +
    +
    + +

    Beacon Node

    +     +
    + +

    + Error communicating with Beacon Node API & Error while connecting to beacon node event stream +

    +     This is likely due to lighthouse not done syncing, wait and try again once synced. Can also be linked to Teku keystore issue. +
    +
    + +

    Clock sync issues

    +     Either your clock server time is off, or you are talking to a remote beacon client that is super slow (this is why we advise against using services like infura). +
    +
    + +

    My beacon node API is flaky with lots of errors and timeouts

    +     + A good quality beacon node API is critical to validator performance. + It is always advised to run your own beacon node to ensure low latencies to boost validator performance. +

    + Using 3rd party services like Infura's beacon node API has significant disadvantages since the quality is often low. + Requests often return 500s or timeout (Charon times out after 2s). + This results in lots of warnings and errors and failed duties. + We are working on an issue to mitigate against this, but running a local beacon node is still always preferred. We are not yet considering increasing the 2s timeout since that can have knock-on effects. +
    +
    +
    + +

    Charon

    +     +
    + +

    + Attester failed in consensus component error +

    +     The required number of operators defined in your cluster-lock file is probably not online to sign successfully. Make sure all operators are running the latest version of charon. To check if some peers are not online: docker logs charon-distributed-validator-node-charon-1 2>&1 | grep 'absent' +
    +
    + +

    + Load private key error +

    +     Make sure you have successfully run a DKG before running the node. The key should be created and placed in the right directory during the ceremony Also, make sure you are working in the right directory: charon-distributed-validator-node +
    +
    + +

    + Failed to confirm node connection error +

    +     Wait for Teku & Lighthouse sync to be complete. +
    +
    + +

    + Reserve relay circuit: reservation failed error +

    +     + RESERVATION_REFUSED is returned by the libp2p relay when some maximum limit has been reached. + This is most often due to "maximum reservations per IP/peer". + This is when your charon node is restarting or in some error loop and constantly attempting to create new relay reservations reaching the maximum. +

    + To fix this error, stop your charon node for 30mins before restarting it. + This should allow the relay enough time to reset your ip/peer limits and should then allow new reservations. + This could also be due to the relay being overloaded in general, so reaching a server wide "maximum connections" limit. + This is an issue with relay scalability and we are working in a long term fix for this. +
    +
    + +

    + Error opening relay circuit: NO_RESERVATION error +

    +     + Error opening relay circuit NO_RESERVATION (204) indicates the peer isn't connected to the relay, so the the charon + client cannot connect to the peer via the relay. That might be because the peer is offline or the + peer is configured to connect to a different relay. +

    + To fix this error, ensure the peer is online and configured with the exact same `--p2p-relays` flag. +
    +
    + +

    + Couldn't fetch duty data from the beacon node error +

    +     + msgFetcher indicates a duty failed in the fetcher component when it failed to fetch the required data from the beacon node API. This indicates a problem with the upstream beacon node. +
    +
    + +

    + Couldn't aggregate attestation due to failed attester duty error +

    +     + msgFetcherAggregatorNoAttData indicates an attestation aggregation duty failed in the fetcher component since it couldn't fetch the prerequisite attestation data. This indicates the associated attestation duty failed to obtain a cluster agreed upon value. +
    +
    + +

    + Couldn't aggregate attestation due to insufficient partial v2 committee subscriptions error +

    +     + msgFetcherAggregatorZeroPrepares indicates an attestation aggregation duty failed in the fetcher component since it couldn't fetch the prerequisite aggregated v2 committee subscription. This indicates the associated prepare aggregation duty failed due to no partial v2 committee subscription submitted by the cluster validator clients. +
    +
    + +

    + Couldn't aggregate attestation due to failed prepare aggregator duty error +

    +     + msgFetcherAggregatorFailedPrepare indicates an attestation aggregation duty failed in the fetcher component since it couldn't fetch the prerequisite aggregated v2 committee subscription. This indicates the associated prepare aggregation duty failed. +
    +
    + +

    + Couldn't propose block due to insufficient partial randao signatures error +

    +     + msgFetcherProposerFewRandaos indicates a block proposer duty failed in the fetcher component since it couldn't fetch the prerequisite aggregated RANDAO. This indicates the associated randao duty failed due to insufficient partial randao signatures submitted by the cluster validator clients. +
    +
    + +

    + Couldn't propose block due to zero partial randao signatures error +

    +     + msgFetcherProposerZeroRandaos indicates a block proposer duty failed in the fetcher component since it couldn't fetch the prerequisite aggregated RANDAO. This indicates the associated randao duty failed due to no partial randao signatures submitted by the cluster validator clients. +
    +
    + +

    + Couldn't propose block due to failed randao duty error +

    +     + msgFetcherProposerZeroRandaos indicates a block proposer duty failed in the fetcher component since it couldn't fetch the prerequisite aggregated RANDAO. This indicates the associated randao duty failed. +
    +
    + +

    + Consensus algorithm didn't complete error +

    +     + msgConsensus indicates a duty failed in consensus component. This could indicate that insufficient honest peers participated in consensus or p2p network connection problems. +
    +
    + +

    + Signed duty not submitted by local validator client error +

    +     + msgValidatorAPI indicates that partial signature were never submitted by the local validator client. This could indicate that the local validator client is offline, or has connection problems with charon, or has some other problem. See validator client logs for more details. +
    +
    + +

    + Bug: partial signature database didn't trigger partial signature exchange error +

    +     + msgParSigDBInternal indicates a bug in the partial signature database as it is unexpected. +
    +
    + +

    + No partial signatures received from peers error +

    +     + msgParSigEx indicates that no partial signature for the duty was received from any peer. This indicates all peers are offline or p2p network connection problems. +
    +
    + +

    + Insufficient partial signatures received, minimum required threshold not reached error +

    +     + msgParSigDBThreshold indicates that insufficient partial signatures for the duty was received from peers. This indicates problems with peers or p2p network connection problems. +
    +
    + +

    + Bug: threshold aggregation of partial signatures failed due to inconsistent signed data error +

    +     + msgSigAgg indicates that BLS threshold aggregation of sufficient partial signatures failed. This indicates inconsistent signed data. This indicates a bug in charon as it is unexpected. +
    +
    + +

    + Existing private key lock file found, another charon instance may be running on your machine error +

    +     + When you turn on the --private-key-file-lock option in Charon, it checks for a special file called the private key lock file. This file has the same name as the ENR private key file but with a .lock extension. + If the private key lock file exists and is not older than 5 seconds, Charon won't run. It doesn't allow running multiple Charon instances with the same ENR private key. + If the private key lock file has a timestamp older than 5 seconds, Charon will replace it and continue with its work. + If you're sure that no other Charon instances are running, you can delete the private key lock file. +
    +
    +
    + +

    Teku

    +     +
    + +

    Teku keystore file error

    +     Teku sometimes logs an error which looks like Keystore file /opt/charon/validator_keys/keystore-0.json.lock already in use. This can be solved by deleting the file(s) ending with .lock in the folder .charon/validator_keys. It is caused by an unsafe shut down of Teku (usually by double pressing `Ctrl+C` to shutdown containers faster). +
    +
    +
    + +

    Grafana

    +     +
    + +

    How to fix the grafana dashboard?

    +     Sometimes, grafana dashboard doesn't load any data first time around.You can solve this by following the steps below: +
    +
    + +

    + N/A & No data in validator info panel +

    +     Can be linked to the Teku Keystore issue. +
    +
    +
    + +

    Prometheus

    +     +
    + +

    + Unauthorized: authentication error: invalid token +

    +     You can ignore this error unless you have been contacted by the Obol Team with monitoring credentials. In that case, follow Step 6 of the quickstart. It does not affect cluster performance or prevent the cluster from running. +
    +
    +
    + +

    Docker

    +     +
    + +

    How to fix permission denied errors?

    +     Permission denied errors can come up in a variety of manners, particularly on Linux and WSL for Windows systems. In the interest of security, the charon docker image runs as a non-root user, and this user often does not have the permissions to write in the directory you have checked out the code to. This can be generally be fixed with some of the following: +
    +
    + +

    I see a lot of errors after running docker compose up +

    +     It's because both geth and lighthouse start syncing and so there's connectivity issues among the containers. Simply let the containers run for a while. You won't observe frequent errors when geth finishes syncing. You can also add a second beacon node endpoint for something like infura by adding a comma separated API URL to the end of CHARON_BEACON_NODE_ENDPOINTS in the docker-compose(./docker-compose.yml#84). +
    +
    + +

    How do I fix the plugin "loki" not found error? +

    +     If you get the following error when calling `docker compose up`:
    + Error response from daemon: error looking up logging plugin loki: plugin "loki" not found.
    Then it probably means that the Loki docker driver isn't installed. In that case, run the following command to install loki:
    + docker plugin install grafana/loki-docker-driver:latest --alias loki --grant-all-permissions +
    +
    +
    + +

    Relay

    +     +
    + +

    + Resolve IP of p2p external host flag: lookup replace.with.public.ip.or.hostname: no such host {" "} error +

    +     Replace replace.with.public.ip.or.hostname in the relay/docker-compose.yml with your real public IP or DNS hostname. +
    +
    + +

    + Timeout resolving bootnode ENR: context deadline exceeded {" "} error +

    +     The relay you are trying to connect to your peers via is offline or unreachable. +
    +
    \ No newline at end of file diff --git a/versioned_docs/version-v0.17.0/int/faq/general.md b/versioned_docs/version-v0.17.0/int/faq/general.md new file mode 100644 index 0000000000..8022256a30 --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/faq/general.md @@ -0,0 +1,64 @@ +--- +sidebar_position: 1 +description: Frequently asked questions +--- + +# Frequently asked questions + +## General + +### Does Obol have a token? + +No. Distributed validators use only Ether. + +### Where can I learn more about Distributed Validators? + +Have you checked out our [blog site](https://blog.obol.tech) and [twitter](https://twitter.com/ObolNetwork) yet? Maybe join our [discord](https://discord.gg/n6ebKsX46w) too. + +### Where does the name Charon come from? + +[Charon](https://www.theoi.com/Khthonios/Kharon.html) [kharon] is the Ancient Greek Ferryman of the Dead. He was tasked with bringing people across the Acheron river to the underworld. His fee was one Obol coin, placed in the mouth of the deceased. This tradition of placing a coin or Obol in the mouth of the deceased continues to this day across the Greek world. + +### What are the hardware requirements for running a Charon node? +Charon alone uses negligible disk space of not more than a few MBs. However, if you are running your consensus client and execution client on the same server as charon, then you will typically need the same hardware as running a full Ethereum node: + +At minimum: +- A CPU with 2+ physical cores (or 4 vCPUs) +- 8GB RAM +- 1.5TB+ free SSD disk space (for mainnet) +- 10mb/s internet bandwidth + +Recommended specifications: +- A CPU with 4+ physical cores +- 16GB+ RAM +- 2TB+ free disk on a high performance SSD (e.g. NVMe) +- 25mb/s internet bandwidth + +For more hardware considerations, check out the [ethereum.org guides](https://ethereum.org/en/developers/docs/nodes-and-clients/run-a-node/#environment-and-hardware) which explores various setups and trade-offs, such as running the node locally or in the cloud. + +For now, Geth, Teku & Lighthouse clients are packaged within the docker compose file provided in the [quickstart guides](../quickstart/group), so you don't have to install anything else to run a cluster. Just make sure you give them some time to sync once you start running your node. + +### What is the difference between a node, a validator and a cluster? +A node is a single instance of Ethereum EL+CL clients that can communicate with other nodes to maintain the Ethereum blockchain. + +A validator is a node that participates in the consensus process by verifying transactions and creating new blocks. Multiple validators can run from the same node. + +A cluster is a group of nodes that act together as one or several validators, which allows for a more efficient use of resources, reduces operational costs, and provides better reliability and fault tolerance. + +### Can I migrate an existing Charon node to a new machine? + +It is possible to migrate your Charon node to another machine running the same config by moving the `.charon` folder with its contents to your new machine. Make sure the EL and CL on the new machine are synced before proceeding to the move to minimize downtime. + +## Distributed Key Generation + +### What are the min and max numbers of operators for a Distributed Validator? + +Currently, the minimum is 4 operators with a threshold of 3. + +The threshold (aka quorum) corresponds to the minimum numbers of operators that need to be active for the validator(s) to be able to perform its duties. It is defined by the following formula `n-(ceil(n/3)-1)`. We strongly recommend using this default threshold in your DKG as it maximises liveness while maintaining BFT safety. Setting a 4 out of 4 cluster for example, would make your validator more vulnerable to going offline instead of less vulnerable. You can check the recommended threshold values for a cluster [here](../key-concepts.md#distributed-validator-threshold). + +## Debugging Errors in Logs + +You can check if the containers on your node are outputting errors by running `docker compose logs` on a machine with a running cluster. + +Diagnose some common errors and view their resolutions [here](./errors.mdx). \ No newline at end of file diff --git a/versioned_docs/version-v0.17.0/int/faq/risks.md b/versioned_docs/version-v0.17.0/int/faq/risks.md new file mode 100644 index 0000000000..e2af412935 --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/faq/risks.md @@ -0,0 +1,31 @@ +--- +sidebar_position: 3 +description: Centralization Risks and mitigation +--- + +# Centralization risks and mitigation + +## Risk: Obol hosting the relay infrastructure +**Mitigation**: Self-host a relay + +One of the risks associated with Obol hosting the [LibP2P relays](docs/charon/networking.md) infrastructure allowing peer discovery is that if Obol-hosted relays go down, peers won't be able to discover each other and perform the DKG. To mitigate this risk, external organizations and node operators can consider self-hosting a relay. This way, if Obol's relays go down, the clusters can still operate through other relays in the network. + +## Risk: Obol being able to update Charon code +**Mitigation**: Pin specific docker versions or compile from source on a trusted commit + +Another risk associated with Obol is having the ability to update the [Charon code](https://github.com/ObolNetwork/charon) running on the network which could introduce vulnerabilities or malicious code. To mitigate this risk, operators can consider pinning specific versions of the code that have been thoroughly tested and accepted by the network. This would ensure that any updates are carefully vetted and reviewed by the community. + +## Risk: Obol hosting the DV Launchpad +**Mitigation**: Use [`create cluster`](docs/charon/charon-cli-reference.md) or [`create dkg`](docs/charon/charon-cli-reference.md) locally and distribute the files manually + +Hosting the first Charon frontend, the [DV Launchpad](docs/dvl/intro.md), on a centralized server could create a single point of failure, as users would have to rely on Obol's server to access the protocol. This could limit the decentralization of the protocol and could make it vulnerable to attacks or downtime. Obol hosting the launchpad on a decentralized network, such as IPFS is a first step but not enough. This is why the Charon code is open-source and contains a CLI interface to interact with the protocol locally. + +To mitigate the risk of launchpad failure, consider using the `create cluster` or `create dkg` commands locally and distributing the key shares files manually. + + +## Risk: Obol going bust/rogue +**Mitigation**: Use key recovery + +The final centralization risk associated with Obol is the possibility of the company going bankrupt or acting maliciously, which would lead to a loss of control over the network and potentially cause damage to the ecosystem. To mitigate this risk, Obol has implemented a key recovery mechanism. This would allow the clusters to continue operating and to retrieve full private keys even if Obol is no longer able to provide support. + +A guide to recombine key shares into a single private key can be accessed [here](../quickstart/advanced/quickstart-combine.md). \ No newline at end of file diff --git a/versioned_docs/version-v0.17.0/int/faq/threat_model.md b/versioned_docs/version-v0.17.0/int/faq/threat_model.md new file mode 100644 index 0000000000..0fc2a5931d --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/faq/threat_model.md @@ -0,0 +1,155 @@ +--- +sidebar_position: 4 +description: Threat model for a Distributed Validator +--- + +# Charon threat model + +This page outlines a threat model for Charon, in the context of it being a Distributed Validator middleware for Ethereum validator clients. + +## Actors + +- Node owner (NO) +- Cluster node operators (CNO) +- Rogue node operator (RNO) +- Outside attacker (OA) + +## General observations + +This page describes some considerations the Obol core team made about the security of a distributed validator in the context of its deployment and interaction with outside actors. + +The goal of this threat model is to provide transparency, but it is by no means a comprehensive audit or complete security reference. It’s a sharing of the experiences and thoughts we gained during the last few years building distributed validator technologies. + +While to the Beacon Chain, a distributed validator is seen in much the same way as a regular validator, and thus retains some of the same security considerations, Charon’s threat model is different from a validator client’s threat model because of its general design. + +While a validator client owns and operates on a set of validator private keys, the design of Charon allows its node operators to rarely (if ever) see the complete validator private keys, relying instead on modern cryptography to generate partial private key shares. + +An Ethereum distributed validator employs advanced signature primitives such that no operator ever handles the full validator private key in any standard lifecycle step: the [BLS digital signature scheme](https://en.wikipedia.org/wiki/BLS_digital_signature) employed by the Ethereum network allows distributed validators to individually sign a blob of data and then aggregate the resulting signatures in a transparent manner, never requiring any of the participating parties to know the full private key to do so. + +If the subset of the available Charon nodes is lower than a given threshold, the cluster is not able to continue with its duties. + +Given the collaborative nature of a Distributed Validator cluster, every operator must prioritize the liveliness and well-being of the cluster. Charon, at the moment of writing this page cannot reward and penalize operators within a cluster independently. + +This implies that Charon’s threat model can’t quite be equated to that of a single validator client, since they work on a different - albeit similar - set of security concepts. + +## Identity private key + +A distributed validator cluster is made up of a number of nodes, often run by a number of independent operators. For each DV cluster there’s a set of Ethereum validator private keys on which they want to validate on behalf of. + +Alongside those, each node (henceforth ‘operator’) holds an SECP256K1 identity private key, referred to as an ENR, that identifies their node to the other cluster operators’ nodes. + +Exfiltration of said private key could lead to possible impersonation from an outside attacker, possibly leading to intra-cluster peering issues, eclipse attack risks, and degraded validator performance. + +Charon client communication is handled via BFT consensus, which is able to tolerate a given number of misbehaving nodes up to a certain threshold: once this threshold is reached, the cluster is not able to continue with its lifecycle and loses liveness guarantees (the validator goes offline). If more than two-thirds of nodes in a cluster are malicious, a cluster also loses safety guarantees (enough bad actors could collude to come to consensus on something slashable). + +Identity private key theft and the subsequent execution of a rogue cluster node is equivalent in the context of BFT consensus to a misbehaving node, hence the cluster can survive and continue with its duties up to what’s specified by the cluster’s BFT protocol’s parameters. + +The likelihood of this happening is low: an OA with enough knowledge of the topology of the operator’s network must steal `fault tolerance of the cluster + 1` identity private keys and run Charon nodes to subvert the distributed validator BFT consensus to push the validator offline. + +## Ethereum validator private key access + +A distributed validator cluster executes Ethereum validator duties by acting as a middleman between the beacon chain and a validator client. + +To do so, the cluster must have knowledge of the Ethereum validator’s private key. + +The design and implementation of Charon minimizes the chances of this by splitting the Ethereum validator private keys into parts, which are then assigned to each node operator. +A [distributed key generation](https://en.wikipedia.org/wiki/Distributed_key_generation) (DKG) process is used in order to evenly and safely create the private key shares without any central party having access to the full private key. + +The cryptography primitives employed in Charon can allow a threshold of the node operator’s private key shares to be reconstructed into the whole validator private key if needed. + +While the facilities to do this are present in the form of CLI commands, as stated before Charon never reconstructs the key in normal operations since BLS digital signature system allows for signature aggregation. + +A distributed validator cluster can be started in two ways: + +1. An existing Ethereum validator private key is split by the private key holder, and distributed in a trusted manner among the operators. +2. The operators participate in a distributed key generation (DKG) process, to create private key shares that collectively can be used to sign validation duties as an Ethereum distributed validator. The full private key for the cluster never exists in one place during or after the DKG. + +In case 1, one of the node operators K has direct access to the Ethereum validator key and is tasked with the generation of other operator’s identity keys and key shards. + +It is clear that in this case the entirety of the sensitive material set is as secure as K’s environment; if K is compromised or malicious, the distributed validator could be slashed. + +Case 2 is different, because there’s no pre-existing Ethereum validator key in a single operator's hands: it will be generated using the FROST DKG algorithm. + +Assuming a successful DKG process, each operator will only ever handle its own key shares instead of the full Ethereum validator private key. + +A set of rogue operators composed of enough members to reconstruct the original Ethereum private keys might pose the risk of slashing for a distributed validator by colluding to produce slashable messages together. + +We deem this scenario’s likelihood as low, as it would mean that node operators decided to willfully slash the stake that they should be being rewarded for staking. + +Still, in the context of an outside attack, purposefully slashing a validator would mean stealing multiple operator key shares, which in turn means violating many cluster operator’s security almost at the same time. This scenario may occur if there is a 0-day vulnerability in a piece of software they all run or in case of node misconfiguration. + +## Rogue node operator + +Nodes are connected by means of either relay nodes, or directly to one another. + +Each node operator is at risk of being impeded by other nodes or by the relay operator in the execution of their duties. + +Nodes need to expose a set of TCP ports to be able to work, and the mere fact of doing that opens up the opportunity for rogue parties to execute DDoS attacks. + +Another attack surface for the cluster exists in rogue nodes purposefully filling the various inter-state databases with meaningless data, or more generally submitting bogus information to the other parties to slow down the processing or, in the case of a sybil attack, bring the cluster to a halt. + +The likelihood of this scenario is medium, because there’s no active threat hunting part: there’s no need for the rogue node operator to penetrate and compromise other nodes to disturb the cluster’s lifecycle. + +## Outside attackers interfering with a cluster + +There are two levels of sophistication in an OA: + +1. No knowledge of the topology of the cluster: The attacker doesn’t know where each cluster node is located and so can’t force fault tolerance +1 nodes offline if it can’t find them. +2. Knowledge of the topology of the network (or part of it) is possessed: the OA can operate DDoS attacks or try breaking into node’s servers - at that point, the “rogue node operator” scenario applies. + +The likelihood of this scenario is low: an OA needs extensive capabilities and sufficient incentive to be able to carry out an attack of this size. + +An outside attacker could also find and use vulnerabilities in the underlying cryptosystems and cryptography libraries used by Charon and other Ethereum clients. Forging signatures that fool Charon’s cryptographic library or other dependencies may be feasible, but forging signatures or otherwise finding a vulnerability in either the SECP256K1+ECDSA or BLS12-381+BLS cryptosystems we deem to be a low likelihood risk. + +## Malicious beacon nodes + +A malicious beacon node (BN) could prevent the distributed validator from operating its validation duties, and could plausibly increase the likelihood of slashing by serving charon illegitimate information. + +If the amount of nodes configured with the malicious BN are equal to the byzantine threshold for the Charon BFT consensus protocol, the validation process can potentially halt since the BFT parameter threshold is reached - most of the nodes are byzantine - the system will reach consensus on a set of data that isn’t valid. + +We deem the likelihood of this scenario to be medium depending on the trust model associated with the BNs deployment (cloud, self-hosted, SaaS product): node operators should always host or at least trust their own beacon nodes. + +## Malicious charon relays + +A Charon relay is used as a communication bridge between nodes that aren’t directly exposed on the Internet. It also acts as the peer discovery mechanism for a cluster. + +Once a peer’s IP address has been discovered via the relay, a direct connection can be attempted. Nodes can either communicate by exchanging data through a relay, or by using the relay as a means to establish a direct TCP connection to one another. + +A malicious relay owned by a OA could lead to: + +- Network topology discovery, facilitating the “outside attackers interactions with a cluster” scenario +- Impeding node communication, potentially impacting the BFT consensus protocol liveness (not security) and distributed validator duties +- DKG process disruption leading to frustration and potential abandonment by node operators: could lead to the usage of a standard Ethereum validator setup, which implies weaker security overall + +We note that BFT consensus liveness disruption can only happen if the number of nodes using the malicious relay for communication is equal to the byzantine nodes amount defined in the consensus parameters. + +This risk can be mitigated by configuring nodes with multiple relay URLs from only [trusted entities](../quickstart/advanced/self-relay.md). + +The likelihood of this scenario is medium: Charon nodes are configured with a default set of relay nodes, so if an OA were to compromise those, it would lead to many cluster topologies getting discovered and potentially attacked and disrupted. + +## Compromised runtime files + +Charon operates with two runtime files: + +- A lock file used to address operator’s nodes, define the Ethereum validator public keys and the public key shares associated with it +- A cluster definition file used to define the operator’s addresses and identities during the DKG process + +The lock file is signed and validated by all the nodes participating in the cluster: assuming good security practices on the node operator side, and no bugs in Charon or its dependencies’ implementations, this scenario is unlikely. + +If one or more node operators are using less than ideal security practices an OA could rewire the Charon CLI flags to include the `--no-verify` flags, which disables lock file signature and hash verification (usually intended only for development purposes). + +By doing that, the OA can edit the lock file as it sees fit, leading to the “rogue node operator” scenario. An OA or RNO might also manage to social engineer their way into convincing other operators into running their malicious lock file with verification disabled. + +The likelihood of this scenario is low: an OA would need to compromise every node operator through social engineering to both use a different set of files, and to run its cluster with `--no-verify`. + +## Conclusions + +Distributed Validator Technology (DVT) helps maintain a high-assurance environment for Ethereum validators by leveraging modern cryptography to ensure no single point of failure is easily found in the system. + +As with any computing system, security considerations are to be expected in order to keep the environment safe. + +From the point of view of an Ethereum validator entity, running their services with a DV client can help greatly with availability, minimizing slashing risks, and maximizing participation in the network. + +On the other hand, one must take into consideration the risks involved with dishonest cluster operators, as well as rogue third-party beacon nodes or relay providers. + +In the end, we believe the benefits of DVT greatly outweigh the potential threats described in this overview. diff --git a/versioned_docs/version-v0.17.0/int/key-concepts.md b/versioned_docs/version-v0.17.0/int/key-concepts.md new file mode 100644 index 0000000000..c6014ba322 --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/key-concepts.md @@ -0,0 +1,110 @@ +--- +sidebar_position: 3 +description: Some of the key terms in the field of Distributed Validator Technology +--- + +# Key concepts +This page outlines a number of the key concepts behind the various technologies that Obol is developing. + +## Distributed validator + +![A Distributed Validator](/img/32Eth.png) + +A distributed validator is an Ethereum proof-of-stake validator that runs on more than one node/machine. This functionality is possible with the use of **Distributed Validator Technology** (DVT). + +Distributed validator technology removes some of the single points of failure in validation. Should <33% of the participating nodes in a DV cluster go offline, the remaining active nodes can still come to consensus on what to sign and can produce valid signatures for their staking duties. This is known as Active/Active redundancy, a common pattern for minimizing downtime in mission critical systems. +​ + +## Distributed Validator Node + +![A Distributed Validator Node](/img/DVNode.png) + +A distributed validator node is the set of clients an operator needs to configure and run to fulfil the duties of a Distributed Validator Operator. An operator may also run redundant execution and consensus clients, an execution payload relayer like [mev-boost](https://github.com/flashbots/mev-boost), or other monitoring or telemetry services on the same hardware to ensure optimal performance. + +In the above example, the stack includes Geth, Lighthouse, Charon and Teku. + +### Execution Client + +![A Geth Client](/img/POWNodeV2.png) + +An execution client (formerly known as an Eth1 client) specializes in running the EVM and managing the transaction pool for the Ethereum network. These clients provide execution payloads to consensus clients for inclusion into blocks. + +Examples of execution clients include: + +- [Go-Ethereum](https://geth.ethereum.org/) +- [Nethermind](https://docs.nethermind.io/nethermind/) +- [Erigon](https://github.com/ledgerwatch/erigon) + +### Consensus Client + +![A Geth Client](/img/POSClient.png) + +A consensus client's duty is to run the proof of stake consensus layer of Ethereum, often referred to as the beacon chain. + +Examples of Consensus clients include: + +- [Prysm](https://docs.prylabs.network/docs/how-prysm-works/beacon-node) +- [Teku](https://docs.teku.consensys.net/en/stable/) +- [Lighthouse](https://lighthouse-book.sigmaprime.io/api-bn.html) +- [Nimbus](https://nimbus.guide/) +- [Lodestar](https://github.com/ChainSafe/lodestar) + +### Distributed Validator Client + +![A Charon Client](/img/CharonBrick.png) + +A distributed validator client intercepts the validator client ↔ consensus client communication flow over the [standardised REST API](https://ethereum.github.io/beacon-APIs/#/ValidatorRequiredApi), and focuses on two core duties. + +- Coming to consensus on a candidate duty for all validators to sign +- Combining signatures from all validators into a distributed validator signature + +The only example of a distributed validator client built with a non-custodial middleware architecture to date is [charon](../charon/intro). + +### Validator Client + +![A Lighthouse Client](/img/ValidatorBrick.png) + +A validator client is a piece of code that operates one or more Ethereum validators. + +Examples of validator clients include: + +- [Vouch](https://www.attestant.io/posts/introducing-vouch/) +- [Prysm](https://docs.prylabs.network/docs/how-prysm-works/prysm-validator-client/) +- [Teku](https://docs.teku.consensys.net/en/stable/) +- [Lighthouse](https://lighthouse-book.sigmaprime.io/api-bn.html) + +## Distributed Validator Cluster + +![A Distributed Validator Cluster](/img/DVCluster.png) + +A distributed validator cluster is a collection of distributed validator nodes connected together to service a set of distributed validators generated during a DVK ceremony. + +### Distributed Validator Key + +![A Distributed Validator Key](/img/ThresholdSigning.png) + +A distributed validator key is a group of BLS private keys, that together operate as a threshold key for participating in proof of stake consensus. + +### Distributed Validator Key Share + +One piece of the distributed validator private key. + +### Distributed Validator Threshold + +The number of nodes in a cluster that needs to be online and honest for their distributed validators to be online is outlined in the following table. + +| Cluster Size | Threshold | Note | +|:------------:|:---------:|:------------------| +| 4 | 3/4 | Minimum threshold | +| 5 | 4/5 | | +| 6 | 4/6 | Minimum to tolerate two offline nodes| +| 7 | 5/7 | Minimum to tolerate two **malicious** nodes | +| 8 | 6/8 | | +| 9 | 6/9 | Minimum to tolerate three offline nodes | +| 10 | 7/10 | Minimum to tolerate three **malicious** nodes | + +### Distributed Validator Key Generation Ceremony + +To achieve fault tolerance in a distributed validator, the individual private key shares need to be generated together. Rather than have a trusted dealer produce a private key, split it and distribute it, the preferred approach is to never construct the full private key at any point, by having each operator in the distributed validator cluster participate in what is known as a Distributed Key Generation ceremony. + +A distributed validator key generation ceremony is a type of DKG ceremony. A ceremony produces signed validator deposit and exit data, along with all of the validator key shares and their associated metadata. Read more about these ceremonies [here](../charon/dkg). diff --git a/versioned_docs/version-v0.17.0/int/quickstart/_category_.json b/versioned_docs/version-v0.17.0/int/quickstart/_category_.json new file mode 100644 index 0000000000..3ef52ab186 --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/quickstart/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "Quickstart Guides", + "position": 3, + "collapsed": false +} diff --git a/versioned_docs/version-v0.17.0/int/quickstart/activate-dv.md b/versioned_docs/version-v0.17.0/int/quickstart/activate-dv.md new file mode 100644 index 0000000000..e77b18667d --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/quickstart/activate-dv.md @@ -0,0 +1,33 @@ +--- +sidebar_position: 4 +description: Activate the Distributed Validator using the deposit contract +--- +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; + +# Activate a DV + +:::caution +Charon is in an alpha state and should be used with caution according to its [Terms of Use](https://obol.tech/terms.pdf). +::: + +If you have successfully created a distributed validator and you are ready to activate it, congratulations. + +Once you have connected all of your charon clients together, synced all of your ethereum nodes such that the monitoring indicates that they are all healthy and ready to operate, **ONE operator** may proceed to activate the validators using one of the following websites: + + + + Obol Distributed Validator Launchpad + + + ethereum.org Staking Launchpad + + + +The `deposit-data.json` to be uploaded to the launchpad will be located in each operator's `.charon` folder. The copies across every node should be identical and any of them can be uploaded. Only one, 32 ether deposit, needs to be made for each validator operated by the cluster. + +:::warning +If you are being given a `deposit-data.json` file that you didn't generate yourself, please take extreme care to ensure this operator has not given you a malicious `deposit-data.json` file that is not the one you expect. Cross reference the files from multiple operators if there is any doubt. Activating the wrong validator or an invalid deposit could result in complete theft or loss of funds. +::: + +The activation process can take a minimum of 16 hours, with the maximum time to activation being dictated by the length of the activation queue, which can be weeks. \ No newline at end of file diff --git a/versioned_docs/version-v0.17.0/int/quickstart/advanced/_category_.json b/versioned_docs/version-v0.17.0/int/quickstart/advanced/_category_.json new file mode 100644 index 0000000000..c606c5656f --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/quickstart/advanced/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "Advanced Guides", + "position": 10, + "collapsed": true + } \ No newline at end of file diff --git a/versioned_docs/version-v0.17.0/int/quickstart/advanced/adv-docker-configs.md b/versioned_docs/version-v0.17.0/int/quickstart/advanced/adv-docker-configs.md new file mode 100644 index 0000000000..d14de53e8b --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/quickstart/advanced/adv-docker-configs.md @@ -0,0 +1,38 @@ +--- +sidebar_position: 8 +description: Use advanced docker-compose features to have more flexibility and power to change the default configuration. +--- + +# Advanced Docker Configs + +:::info +This section is intended for *docker power users*, i.e., for those who are familiar with working with `docker-compose` and want to have more flexibility and power to change the default configuration. +::: + +We use the "Multiple Compose File" feature which provides a very powerful way to override any configuration in `docker-compose.yml` without needing to modify git-checked-in files since that results in conflicts when upgrading this repo. +See [this](https://docs.docker.com/compose/extends/#multiple-compose-files) for more details. + +There are some additional compose files in [this repository](https://github.com/ObolNetwork/charon-distributed-validator-node/), `compose-debug.yml` and `docker-compose.override.yml.sample`, along-with the default `docker-compose.yml` file that you can use for this purpose. + +- `compose-debug.yml` contains some additional containers that developers can use for debugging, like `jaeger`. To achieve this, you can run: + +``` +docker compose -f docker-compose.yml -f compose-debug.yml up +``` + +- `docker-compose.override.yml.sample` is intended to override the default configuration provided in `docker-compose.yml`. This is useful when, for example, you wish to add port mappings or want to disable a container. + +- To use it, just copy the sample file to `docker-compose.override.yml` and customise it to your liking. Please create this file ONLY when you want to tweak something. This is because the default override file is empty and docker errors if you provide an empty compose file. + +``` +cp docker-compose.override.yml.sample docker-compose.override.yml + +# Tweak docker-compose.override.yml and then run docker compose up +docker compose up +``` + +- You can also run all these compose files together. This is desirable when you want to use both the features. For example, you may want to have some debugging containers AND also want to override some defaults. To achieve this, you can run: + +``` +docker compose -f docker-compose.yml -f docker-compose.override.yml -f compose-debug.yml up +``` diff --git a/versioned_docs/version-v0.17.0/int/quickstart/advanced/monitoring-credentials.md b/versioned_docs/version-v0.17.0/int/quickstart/advanced/monitoring-credentials.md new file mode 100644 index 0000000000..fdbec169b9 --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/quickstart/advanced/monitoring-credentials.md @@ -0,0 +1,100 @@ +--- +sidebar_position: 4 +description: Add monitoring credentials to help the Obol Team monitor the health of your cluster +--- +# Getting Started Monitoring your Node + +Welcome to this comprehensive guide, designed to assist you in effectively monitoring your Charon cluster and nodes, and setting up alerts based on specified parameters. + +## Pre-requisites + +Ensure the following software are installed: + +- Docker: Find the installation guide for Ubuntu **[here](https://docs.docker.com/engine/install/ubuntu/)** +- Prometheus: You can install it using the guide available **[here](https://prometheus.io/docs/prometheus/latest/installation/)** +- Grafana: Follow this **[link](https://grafana.com/docs/grafana/latest/setup-grafana/installation/)** to install Grafana + +## Import Pre-Configured Charon Dashboards + +- Navigate to the **[repository](https://github.com/ObolNetwork/monitoring/tree/main/dashboards)** that contains a variety of Grafana dashboards. For this demonstration, we will utilize the Charon Dashboard json. + +- In your Grafana interface, create a new dashboard and select the import option. + +- Copy the content of the Charon Dashboard json from the repository and paste it into the import box in Grafana. Click "Load" to proceed. + +- Finalize the import by clicking on the "Import" button. At this point, your dashboard should begin displaying metrics. Ensure your Charon client and Prometheus are operational for this to occur. + +## Example Alerting Rules + +To create alerts for Node-Exporter, follow these steps based on the sample rules provided on the "Awesome Prometheus alerts" page: + +1. Visit the **[Awesome Prometheus alerts](https://samber.github.io/awesome-prometheus-alerts/rules.html#host-and-hardware)** page. Here, you will find lists of Prometheus alerting rules categorized by hardware, system, and services. + +2. Depending on your need, select the category of alerts. For example, if you want to set up alerts for your system's CPU usage, click on the 'CPU' under the 'Host & Hardware' category. + +3. On the selected page, you'll find specific alert rules like 'High CPU Usage'. Each rule will provide the PromQL expression, alert name, and a brief description of what the alert does. You can copy these rules. + +4. Paste the copied rules into your Prometheus configuration file under the `rules` section. Make sure you understand each rule before adding it to avoid unnecessary alerts. + +5. Finally, save and apply the configuration file. Prometheus should now trigger alerts based on these rules. + + +For alerts specific to Charon/Alpha, refer to the alerting rules available on this [ObolNetwork/monitoring](https://github.com/ObolNetwork/monitoring/tree/main/alerting-rules). + +## Understanding Alert Rules + +1. `ClusterBeaconNodeDown`This alert is activated when the beacon node in a specified Alpha cluster is offline. The beacon node is crucial for validating transactions and producing new blocks. Its unavailability could disrupt the overall functionality of the cluster. +2. `ClusterBeaconNodeSyncing`This alert indicates that the beacon node in a specified Alpha cluster is synchronizing, i.e., catching up with the latest blocks in the cluster. +3. `ClusterNodeDown`This alert is activated when a node in a specified Alpha cluster is offline. +4. `ClusterMissedAttestations`:This alert indicates that there have been missed attestations in a specified Alpha cluster. Missed attestations may suggest that validators are not operating correctly, compromising the security and efficiency of the cluster. +5. `ClusterInUnknownStatus`: This alert is designed to activate when a node within the cluster is detected to be in an unknown state. The condition is evaluated by checking whether the maximum of the app_monitoring_readyz metric is 0. +6. `ClusterInsufficientPeers`:This alert is set to activate when the number of peers for a node in the Alpha M1 Cluster #1 is insufficient. The condition is evaluated by checking whether the maximum of the **`app_monitoring_readyz`** equals 4. +7. `ClusterFailureRate`: This alert is activated when the failure rate of the Alpha M1 Cluster #1 exceeds a certain threshold. +8. `ClusterVCMissingValidators`: This alert is activated if any validators in the Alpha M1 Cluster #1 are missing. +9. `ClusterHighPctFailedSyncMsgDuty`: This alert is activated if a high percentage of sync message duties failed in the cluster. The alert is activated if the sum of the increase in failed duties tagged with "sync_message" in the last hour divided by the sum of the increase in total duties tagged with "sync_message" in the last hour is greater than 0.1. +10. `ClusterNumConnectedRelays`: This alert is activated if the number of connected relays in the cluster falls to 0. +11. PeerPingLatency: 1. This alert is activated if the 90th percentile of the ping latency to the peers in a cluster exceeds 500ms within 2 minutes. + +## Best Practices for Monitoring Charon Nodes & Cluster + +- **Establish Baselines**: Familiarize yourself with the normal operation metrics like CPU, memory, and network usage. This will help you detect anomalies. +- **Define Key Metrics**: Set up alerts for essential metrics, encompassing both system-level and Charon-specific ones. +- **Configure Alerts**: Based on these metrics, set up actionable alerts. +- **Monitor Network**: Regularly assess the connectivity between nodes and the network. +- **Perform Regular Health Checks**: Consistently evaluate the status of your nodes and clusters. +- **Monitor System Logs**: Keep an eye on logs for error messages or unusual activities. +- **Assess Resource Usage**: Ensure your nodes are neither over- nor under-utilized. +- **Automate Monitoring**: Use automation to ensure no issues go undetected. +- **Conduct Drills**: Regularly simulate failure scenarios to fine-tune your setup. +- **Update Regularly**: Keep your nodes and clusters updated with the latest software versions. + +## Third-Party Services for Uptime Testing + +- [updown.io](https://updown.io/) +- [Grafana synthetic Monitoring](https://grafana.com/grafana/plugins/grafana-synthetic-monitoring-app/) + +## Key metrics to watch to verify node health based on jobs + +- Node Exporter: + +**CPU Usage**: High or spiking CPU usage can be a sign of a process demanding more resources than it should. + +**Memory Usage**: If a node is consistently running out of memory, it could be due to a memory leak or simply under-provisioning. + +**Disk I/O**: Slow disk operations can cause applications to hang or delay responses. High disk I/O can indicate storage performance issues or a sign of high load on the system. + +**Network Usage**: High network traffic or packet loss can signal network configuration issues, or that a service is being overwhelmed by requests. + +**Disk Space**: Running out of disk space can lead to application errors and data loss. + +**Uptime**: The amount of time a system has been up without any restarts. Frequent restarts can indicate instability in the system. + +**Error Rates**: The number of errors encountered by your application. This could be 4xx/5xx HTTP errors, exceptions, or any other kind of error your application may log. + +**Latency**: The delay before a transfer of data begins following an instruction for its transfer. + +It is also important to check: + +- NTP clock skew +- Process restarts and failures (eg. through `node_systemd`) +- alert on high error and panic log counts. \ No newline at end of file diff --git a/versioned_docs/version-v0.17.0/int/quickstart/advanced/prysm-vc.md b/versioned_docs/version-v0.17.0/int/quickstart/advanced/prysm-vc.md new file mode 100644 index 0000000000..50e9b349fe --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/quickstart/advanced/prysm-vc.md @@ -0,0 +1,26 @@ +--- +sidebar_position: 6 +description: Run Prysm VCs in a DV +--- + +# Run Prysm validators as part of a DV Cluster + + +:::info +Support for prysm validator clients is in an experimental phase as prysm doesn't provide [complete support](https://github.com/prysmaticlabs/prysm/issues/11580) +for running their validator client on a beacon node REST API yet. Stay up to date on the latest support information for all validator clients at [dvt.obol.tech](https://dvt.obol.tech). +::: + +We have provided experimental support of prysm validator client through [compose-prysm.yml](https://github.com/ObolNetwork/charon-distributed-validator-cluster/blob/main/compose-prysm.yml) +which needs a prysm beacon node to work alongside a REST based beacon node. Here are the steps to test prysm in a DV Cluster: + +1. Run the following command: + ```sh + cp docker-compose.override.yml.sample docker-compose.override.yml + ``` +2. **Disable** the VCs in `docker-compose.override.yml` that are to be replaced by corresponding prysm VCs by un-commenting the `profiles` field +([more details on profiles](https://docs.docker.com/compose/profiles/)). Also disable the prysm VCs that are not required in the cluster in `docker-compose.override.yml`. +3. Run the following command: + ```sh + docker compose -f docker-compose.yml -f compose-prysm.yml -f docker-compose.override.yml up --build + ``` diff --git a/versioned_docs/version-v0.17.0/int/quickstart/advanced/push-metrics.md b/versioned_docs/version-v0.17.0/int/quickstart/advanced/push-metrics.md new file mode 100644 index 0000000000..8d9e0ceca1 --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/quickstart/advanced/push-metrics.md @@ -0,0 +1,40 @@ +--- +sidebar_position: 5 +description: Add monitoring credentials to help the Obol Team monitor the health of your cluster +--- + +# Push Metrics to Obol Monitoring + +:::info +This is **optional** and does not confer any special privileges within the Obol Network. +::: + +You may have been provided with **Monitoring Credentials** used to push distributed validator metrics to Obol's central prometheus cluster to monitor, analyze, and improve your Distributed Validator Cluster's performance. + +The provided credentials needs to be added in `prometheus/prometheus.yml` replacing `$PROM_REMOTE_WRITE_TOKEN` and will look like: +``` +obol20!tnt8U!C... +``` + +The updated `prometheus/prometheus.yml` file should look like: +``` +global: + scrape_interval: 30s # Set the scrape interval to every 30 seconds. + evaluation_interval: 30s # Evaluate rules every 30 seconds. + +remote_write: + - url: https://vm.monitoring.gcp.obol.tech/write + authorization: + credentials: obol20!tnt8U!C... + +scrape_configs: + - job_name: 'charon' + static_configs: + - targets: ['charon:3620'] + - job_name: "lodestar" + static_configs: + - targets: [ "lodestar:5064" ] + - job_name: 'node-exporter' + static_configs: + - targets: ['node-exporter:9100'] +``` \ No newline at end of file diff --git a/versioned_docs/version-v0.17.0/int/quickstart/advanced/quickstart-builder-api.md b/versioned_docs/version-v0.17.0/int/quickstart/advanced/quickstart-builder-api.md new file mode 100644 index 0000000000..f72776e0df --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/quickstart/advanced/quickstart-builder-api.md @@ -0,0 +1,79 @@ +--- +sidebar_position: 2 +description: Run a distributed validator cluster with the builder API (MEV-Boost) +--- + +# Run a cluster with MEV-Boost + +:::caution +Charon is in an alpha state and should be used with caution according to its [Terms of Use](https://obol.tech/terms.pdf). + +Charon's integration with MEV-Boost is also in an alpha state and requires a non-trivial amount of configuration to get working successfully. In the future, this process aims to be much more automated and seamless from a user's perspective. +::: + +This quickstart guide focuses on configuring the builder API for Charon and supported Validator clients. + +## Getting started with Charon & the Builder API + +Running a distributed validator cluster with the builder API enabled will give the validators in the cluster access to the builder network. This builder network is a network of "Block Builders" +who work with MEV searchers to produce the most valuable blocks a validator can propose. + +[MEV-Boost](https://boost.flashbots.net/) is one such product from flashbots that enables you to ask multiple +block relays (who communicate with the "Block Builders") for blocks to propose. The block that pays the largest reward to the validator will be signed and returned to the relay for broadcasting to the wider +network. The end result for the validator is generally an increased APY as they receive some share of the MEV. + +## Run MEV Boost + +Currently, we don't support [mev-boost](https://boost.flashbots.net/) if you are running your charon cluster using [this repo](https://github.com/ObolNetwork/charon-distributed-validator-node) since the repo +uses lodestar validator client which is not `builder API` compatible with Charon. + +We are working on a solution that enables the builder API feature irrespective of the choice of the Validator Client. + +## Builder API + +If you want to configure Charon and supported Validator clients to support the builder API feature, you can do that as well. + +Currently, Charon with the builder API enabled, is compatible only with [Teku](https://github.com/ConsenSys/teku). Work is underway [to support](https://dvt.obol.tech/) all validator client implementations in a mev-enabled distributed validator cluster seamlessly. + +### Charon + +Charon supports builder API with the `--builder-api` flag. To use builder API, one simply needs to add this flag to the `charon run` command: + +``` +charon run --builder-api +``` + +### Validator Clients + +#### Teku Validator Client + +Configuring the Teku validator client with Charon can be done by following the same process outlined in the [Teku official guide](https://docs.teku.consensys.net/how-to/configure/use-proposer-config-file). + +The validator client must be set up to use the `--validators-proposer-config` [flag](https://docs.teku.consensys.net/reference/cli#validators-proposer-config) with a value equal to `http://$CHARON_ENDPOINT:3600/teku_proposer_config`. + +Once the flag is set up, Obol distributed validators will be able to register to the builder network, submit blinded beacon blocks and gain a share of the MEV profits. + +#### Lighthouse Validator Client + +For Lighthouse, we are currently waiting on the following [PR](https://github.com/sigp/lighthouse/pull/4306) to be merged into their unstable branch to enable compatability, please review the PR's status. + +## Verify MEV Boost is functional + +Once you have executed the above steps, you can verify if your setup is functional by reviewing your proposed blocks on [beaconcha.in](https://beaconcha.in) dashboards or via the Relay API endpoints. + +:::caution +Note that the mainnet block in the below description is taken only for representation, and not actually proposed by a distributed validator. +:::caution + +As an example, if my validator was the block proposer for block `17370078` on mainnet, I can review the following resources: + +* [Beaconcha.in](https://beaconcha.in): + * Consider this [Mainnet block 17370078](https://beaconcha.in/block/17370078). + * If we check the `Block Extra Data` field under `Execution Payload`, we will see the tag `Illuminate Dmocratize Dstribute` (Hex:`0x496c6c756d696e61746520446d6f63726174697a6520447374726962757465`). + * Relays will generally add a tag to the block. Since this block was submitted via the Flashbots Relay, as a result it has the tag. + * All mainnet flashbots blocks have this tag `Illuminate Dmocratize Dstribute`. +* [Relay API](https://flashbots.github.io/relay-specs/): + * If you navigate to the `Data API` section on the Relay API page, you will see an endpoint labeled `/relay/v1/data/bidtraces/proposer_payload_delivered`. + * You can add a query argument of `block_number` to this call to see if a block was submitted via that Relay. + * [Here](https://boost-relay.flashbots.net/relay/v1/data/bidtraces/proposer_payload_delivered?block_number=17370078) is the query for the example block 17370078. + * Blocks that have not been submitted to the Relay will return an empty array `[]`. diff --git a/versioned_docs/version-v0.17.0/int/quickstart/advanced/quickstart-combine.md b/versioned_docs/version-v0.17.0/int/quickstart/advanced/quickstart-combine.md new file mode 100644 index 0000000000..19743adf1e --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/quickstart/advanced/quickstart-combine.md @@ -0,0 +1,153 @@ +--- +sidebar_position: 9 +description: Combine distributed validator private key shares to recover the validator private key. +--- + +# Combine DV private key shares + +:::warning +Reconstituting Distributed Validator private key shares into a standard validator private key is a security risk, and can potentially cause your validator to be slashed. + +Only combine private keys as a last resort and do so with extreme caution. +::: + +Combine distributed validator private key shares into an Ethereum validator private key. + +## Pre-requisites + +- Ensure you have the `.charon` directories of at least a threshold of the cluster's node operators. +- Ensure you have [docker](https://docs.docker.com/engine/install/) installed. +- Make sure `docker` is running before executing the commands below. + +## Step 1. Set up the key combination directory tree + +Rename each cluster node operator `.charon` directory in a different way to avoid folder name conflicts. + +We suggest naming them clearly and distinctly, to avoid confusion. + +At the end of this process, you should have a tree like this: + +```shell +$ tree ./validators-to-be-combined + +validators-to-be-combined/ +├── node0 +│   ├── charon-enr-private-key +│   ├── cluster-lock.json +│   ├── deposit-data.json +│   └── validator_keys +│   ├── keystore-0.json +│   ├── keystore-0.txt +│   ├── keystore-1.json +│   └── keystore-1.txt +├── node1 +│   ├── charon-enr-private-key +│   ├── cluster-lock.json +│   ├── deposit-data.json +│   └── validator_keys +│   ├── keystore-0.json +│   ├── keystore-0.txt +│   ├── keystore-1.json +│   └── keystore-1.txt +├── node2 +│   ├── charon-enr-private-key +│   ├── cluster-lock.json +│   ├── deposit-data.json +│   └── validator_keys +│   ├── keystore-0.json +│   ├── keystore-0.txt +│   ├── keystore-1.json +│   └── keystore-1.txt +... +└── node* + ├── charon-enr-private-key + ├── cluster-lock.json + ├── deposit-data.json + └── validator_keys + ├── keystore-0.json + ├── keystore-0.txt + ├── keystore-1.json + └── keystore-1.txt +``` + +:::caution +Make sure to never mix the various `.charon` directories with one another. + +Doing so can potentially cause the combination process to fail. +::: + +## Step 2. Combine the key shares + +Run the following command: + +```sh +# Combine a clusters private keys +docker run --rm -v "$(pwd):/opt/charon" obolnetwork/charon:v0.17.0 combine --cluster-dir /opt/charon/validators-to-be-combined +``` + +This command will create one subdirectory for each validator private key that has been combined, named after its public key. + +```shell +$ tree ./validators-to-be-combined + +validators-to-be-combined/ +├── 0x822c5310674f4fc4ec595642d0eab73d01c62b588f467da6f98564f292a975a0ac4c3a10f1b3a00ccc166a28093c2dcd +│   └── validator_keys +│   ├── keystore-0.json +│   └── keystore-0.txt +├── 0x8929b4c8af2d2eb222d377cac2aa7be950e71d2b247507d19b5fdec838f0fb045ea8910075f191fd468da4be29690106 +│   └── validator_keys +│   ├── keystore-0.json +│   └── keystore-0.txt +├── node0 +│   ├── charon-enr-private-key +│   ├── cluster-lock.json +│   ├── deposit-data.json +│   └── validator_keys +│   ├── keystore-0.json +│   ├── keystore-0.txt +│   ├── keystore-1.json +│   └── keystore-1.txt +├── node1 +│   ├── charon-enr-private-key +│   ├── cluster-lock.json +│   ├── deposit-data.json +│   └── validator_keys +│   ├── keystore-0.json +│   ├── keystore-0.txt +│   ├── keystore-1.json +│   └── keystore-1.txt +├── node2 +│   ├── charon-enr-private-key +│   ├── cluster-lock.json +│   ├── deposit-data.json +│   └── validator_keys +│   ├── keystore-0.json +│   ├── keystore-0.txt +│   ├── keystore-1.json +│   └── keystore-1.txt +└── node3 + ├── charon-enr-private-key + ├── cluster-lock.json + ├── deposit-data.json + └── validator_keys + ├── keystore-0.json + ├── keystore-0.txt + ├── keystore-1.json + └── keystore-1.txt +``` + +We can verify that the directory names are correct by looking at the lock file: + +```shell +$ jq .distributed_validators[].distributed_public_key validators-to-be-combined/node0/cluster-lock.json +"0x822c5310674f4fc4ec595642d0eab73d01c62b588f467da6f98564f292a975a0ac4c3a10f1b3a00ccc166a28093c2dcd" +"0x8929b4c8af2d2eb222d377cac2aa7be950e71d2b247507d19b5fdec838f0fb045ea8910075f191fd468da4be29690106" +``` + +:::info + +The generated private keys are in the standard [EIP-2335](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-2335.md) format, and can be imported in any Ethereum validator client that supports it. + +Ensure your distributed validator cluster is completely shut down before starting a replacement validator or you are likely to be slashed. +::: diff --git a/versioned_docs/version-v0.17.0/int/quickstart/advanced/quickstart-sdk.md b/versioned_docs/version-v0.17.0/int/quickstart/advanced/quickstart-sdk.md new file mode 100644 index 0000000000..9c6cd3491c --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/quickstart/advanced/quickstart-sdk.md @@ -0,0 +1,109 @@ +--- +sidebar_position: 1 +description: Create a DV cluster using the Obol Typescript SDK +--- + +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; + +# Create a DV using the SDK + +:::caution + +The Obol-SDK is in an alpha state and should be used with caution. +::: + +This is a walkthrough of using the [Obol-SDK](https://www.npmjs.com/package/@obolnetwork/obol-sdk) to propose a four-node distributed validator cluster for creation using the [DV Launchpad](../../../dvl/intro.md). + +## Pre-requisites + +- You have [node.js](https://nodejs.org/en) installed. + +## Install the package + +Install the Obol-SDK package into your development environment + + + + 
    +      npm install --save @obolnetwork/obol-sdk
+
    +     + + 
    +      yarn add @obolnetwork/obol-sdk
+
    +     +     + +## Instantiate the client + +The first thing you need to do is create a instance of the Obol SDK client. The client takes two constructor parameters: + +- The `chainID` for the chain you intend to use. +- An ethers.js [signer](https://docs.ethers.org/v6/api/providers/#Signer-signTypedData) object. + + +```ts +import { Client } from "@obolnetwork/obol-sdk"; +import { ethers } from "ethers"; + +// Create a dummy ethers signer object with a throwaway private key +const mnemonic = ethers.Wallet.createRandom().mnemonic?.phrase || ""; +const privateKey = ethers.Wallet.fromPhrase(mnemonic).privateKey; +const wallet = new ethers.Wallet(privateKey); +const signer = wallet.connect(null); + +// Instantiate the Obol Client for goerli +const obol = new Client({chainId: 5}, signer); +``` + +## Propose the cluster + +List the Ethereum addresses of participating operators, along with withdrawal and fee recipient address data for each validator you intend for the operators to create. + +```ts +// A config hash is a deterministic hash of the proposed DV cluster configuration +const configHash = await obol.createClusterDefinition({ + name: "SDK Demo Cluster", + operators: + [ + { address: "0xC35CfCd67b9C27345a54EDEcC1033F2284148c81" }, + { address: "0x33807D6F1DCe44b9C599fFE03640762A6F08C496" }, + { address: "0xc6e76F72Ea672FAe05C357157CfC37720F0aF26f" }, + { address: "0x86B8145c98e5BD25BA722645b15eD65f024a87EC" } + ], + validators: [{ + fee_recipient_address: "0x3CD4958e76C317abcEA19faDd076348808424F99", + withdrawal_address: "0xE0C5ceA4D3869F156717C66E188Ae81C80914a6e" + }], +}); + +console.log(`Direct the operators to https://goerli.launchpad.obol.tech/dv?configHash=${configHash} to complete the key generation process`) +``` + +## Invite the Operators to complete the DKG + +Once the Obol-API returns a `configHash` string from the `createClusterDefinition` method, you can use this identifier to invite the operators to the [Launchpad](../../../dvl/intro.md) to complete the process + +1. Operators navigate to `https://.launchpad.obol.tech/dv?configHash=` and complete the [run a DV with others](../group/quickstart-group-operator.md) flow. +1. Once the DKG is complete, and operators are using the `--publish` flag, the created cluster details will be posted to the Obol API +1. The creator will be able to retrieve this data with `obol.getClusterLock(configHash)`, to use for activating the newly created validator. + +## Retrieve the created Distributed Validators using the SDK + +Once the DKG is complete, the proposer of the cluster can retrieve key data such as the validator public keys and their associated deposit data messages. + +```js +const clusterLock = await obol.getClusterLock(configHash); +``` + +Reference lock files can be found [here](https://github.com/ObolNetwork/charon/tree/main/cluster/testdata). + +## Activate the DVs using the deposit contract + +This guide will in future cover activating the deposit data from a lock file. + +## Usage Examples + +Examples of how our SDK can be used are found [here](https://github.com/ObolNetwork/obol-sdk-examples). diff --git a/versioned_docs/version-v0.17.0/int/quickstart/advanced/quickstart-split.md b/versioned_docs/version-v0.17.0/int/quickstart/advanced/quickstart-split.md new file mode 100644 index 0000000000..7731aafd33 --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/quickstart/advanced/quickstart-split.md @@ -0,0 +1,91 @@ +--- +sidebar_position: 3 +description: Split existing validator keys +--- + +# Split existing validator private keys + +:::caution +Charon is in an alpha state and should be used with caution according to its [Terms of Use](https://obol.tech/terms.pdf). + +This process should only be used if you want to split an *existing validator private key* into multiple private key shares for use in a Distributed Validator Cluster. If your existing validator is not properly shut down before the Distributed Validator starts, your validator may be slashed. + +If you are starting a new validator, you should follow a [quickstart guide](../index.md) instead. +::: + +Split an existing Ethereum validator key into multiple key shares for use in an [Obol Distributed Validator Cluster](../../key-concepts#distributed-validator-cluster). + + +## Pre-requisites + +- Ensure you have the existing validator keystores (the ones to split) and passwords. +- Ensure you have [docker](https://docs.docker.com/engine/install/) installed. +- Make sure `docker` is running before executing the commands below. + +## Step 1. Clone the charon repo and copy existing keystore files + +Clone the [charon](https://github.com/ObolNetwork/charon) repo. + + ```sh + # Clone the repo + git clone https://github.com/ObolNetwork/charon.git + + # Change directory + cd charon/ + + # Create a folder within this checked out repo + mkdir split_keys + ``` + +Copy the existing validator `keystore.json` files into this new folder. Alongside them, with a matching filename but ending with `.txt` should be the password to the keystore. E.g., `keystore-0.json` `keystore-0.txt` + +At the end of this process, you should have a tree like this: +```shell +├── split_keys +│   ├── keystore-0.json +│   ├── keystore-0.txt +│   ├── keystore-1.json +│   ├── keystore-1.txt +│ ... +│   ├── keystore-*.json +│   ├── keystore-*.txt +``` + +## Step 2. Split the keys using the charon docker command + +Run the following docker command to split the keys: + +```shell +CHARON_VERSION= # E.g. v0.17.0 +CLUSTER_NAME= # The name of the cluster you want to create. +WITHDRAWAL_ADDRESS= # The address you want to use for withdrawals. +FEE_RECIPIENT_ADDRESS= # The address you want to use for fee payments. +NODES= # The number of nodes in the cluster. + +docker run --rm -v $(pwd):/opt/charon obolnetwork/charon:${CHARON_VERSION} create cluster --name="${CLUSTER_NAME}" --withdrawal-addresses="${WITHDRAWAL_ADDRESS}" --fee-recipient-addresses="${FEE_RECIPIENT_ADDRESS}" --split-existing-keys --split-keys-dir=/opt/charon/split_keys --nodes ${NODES} --network goerli +``` + +The above command will create `validator_keys` along with `cluster-lock.json` in `./.charon/cluster` for each node. + +Command output: + +```shell +***************** WARNING: Splitting keys ********************** + Please make sure any existing validator has been shut down for + at least 2 finalised epochs before starting the charon cluster, + otherwise slashing could occur. +**************************************************************** + +Created charon cluster: + --split-existing-keys=true + +.charon/cluster/ +├─ node[0-*]/ Directory for each node +│ ├─ charon-enr-private-key Charon networking private key for node authentication +│ ├─ cluster-lock.json Cluster lock defines the cluster lock file which is signed by all nodes +│ ├─ validator_keys Validator keystores and password +│ │ ├─ keystore-*.json Validator private share key for duty signing +│ │ ├─ keystore-*.txt Keystore password files for keystore-*.json +``` + +These split keys can now be used to start a charon cluster. diff --git a/versioned_docs/version-v0.17.0/int/quickstart/advanced/self-relay.md b/versioned_docs/version-v0.17.0/int/quickstart/advanced/self-relay.md new file mode 100644 index 0000000000..ae157214b7 --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/quickstart/advanced/self-relay.md @@ -0,0 +1,36 @@ +--- +sidebar_position: 7 +description: Self-host a relay +--- + +# Self-Host a Relay + +If you are experiencing connectivity issues with the Obol hosted relays, or you want to improve your clusters latency and decentralization, you can opt to host your own relay on a separate open and static internet port. + +``` +# Figure out your public IP +curl v4.ident.me + +# Clone the repo and cd into it. +git clone https://github.com/ObolNetwork/charon-distributed-validator-node.git + +cd charon-distributed-validator-node + +# Replace 'replace.with.public.ip.or.hostname' in relay/docker-compose.yml with your public IPv4 or DNS hostname # Replace 'replace.with.public.ip.or.hostname' in relay/docker-compose.yml with your public IPv4 or DNS hostname + +nano relay/docker-compose.yml + +docker compose -f relay/docker-compose.yml up +``` + +Test whether the relay is publicly accessible. This should return an ENR: +`curl http://replace.with.public.ip.or.hostname:3640/enr` + +Ensure the ENR returned by the relay contains the correct public IP and port by decoding it with https://enr-viewer.com/. + +Configure **ALL** charon nodes in your cluster to use this relay: + +- Either by adding a flag: `--p2p-relays=http://replace.with.public.ip.or.hostname:3640/enr` +- Or by setting the environment variable: `CHARON_P2P_RELAYS=http://replace.with.public.ip.or.hostname:3640/enr` + +Note that a local `relay/.charon/charon-enr-private-key` file will be created next to `relay/docker-compose.yml` to ensure a persisted relay ENR across restarts. diff --git a/versioned_docs/version-v0.17.0/int/quickstart/alone/_category_.json b/versioned_docs/version-v0.17.0/int/quickstart/alone/_category_.json new file mode 100644 index 0000000000..9f98f73841 --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/quickstart/alone/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "Create a DV alone", + "position": 1, + "collapsed": true +} \ No newline at end of file diff --git a/versioned_docs/version-v0.17.0/int/quickstart/alone/create-keys.md b/versioned_docs/version-v0.17.0/int/quickstart/alone/create-keys.md new file mode 100644 index 0000000000..2e37e71ac7 --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/quickstart/alone/create-keys.md @@ -0,0 +1,56 @@ +--- +sidebar_position: 2 +description: Run all nodes in a distributed validator cluster +--- + +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; + +# Create the private key shares + +:::caution +Charon is in an alpha state and should be used with caution according to its [Terms of Use](https://obol.tech/terms.pdf). +::: + +:::info +Running a Distributed Validator alone means that a single operator manages all of the nodes of the DV. Depending on the operators security preferences, the private key shares can be created centrally, and distributed securely to each node. This is the focus of the below guide. + +Alternatively, the private key shares can be created in a lower-trust manner with a [Distributed Key Generation](../../key-concepts.md#distributed-validator-key-generation-ceremony) process, which avoids the validator private key being stored in full anywhere, at any point in its lifecycle. Follow the [group quickstart](./../group/index.md) instead for this latter case. +::: + +## Pre-requisites + +- Ensure you have [docker](https://docs.docker.com/engine/install/) installed. +- Make sure `docker` is running before executing the commands below. + +## Create the key shares locally + + + + Create the artifacts needed to run a DV cluster by running the following command to setup the inputs for the DV. + Check the Charon CLI reference for additional optional flags to set. +

    + 
    +      
+      WITHDRAWAL_ADDR=[ENTER YOUR WITHDRAWAL ADDRESS HERE]
+      
    
+      FEE_RECIPIENT_ADDR=[ENTER YOUR FEE RECIPIENT ADDRESS HERE]
+      
    
+      NB_NODES=[ENTER AMOUNT OF DESIRED NODES]
+          
+
    + Then, run this command to create all the key shares and cluster artifacts locally:

    + 
    +      
+      docker run --rm -v "$(pwd):/opt/charon" obolnetwork/charon:v0.17.0 create cluster --name="mycluster" --withdrawal-addresses="{'${WITHDRAWAL_ADDR}'}" --fee-recipient-addresses="{'${FEE_RECIPIENT_ADDR}'}" --nodes="{'${NB_NODES}'}" --network goerli --num-validators=1
+      
+
    +     + + Go to the Obol Launchpad and select Create a distributed validator alone. Follow the steps to configure your DV cluster. + +     + +After successful completion, a subdirectory `.charon/cluster` should be created. In it are as many folders as nodes of the cluster. Each folder contains partial private keys that together make up the distributed validator described in `.charon/cluster/cluster-lock.json`. + +Once ready, you can move to [deploying this cluster physically](./deploy.md). \ No newline at end of file diff --git a/versioned_docs/version-v0.17.0/int/quickstart/alone/deploy.md b/versioned_docs/version-v0.17.0/int/quickstart/alone/deploy.md new file mode 100644 index 0000000000..bdd5d3af8c --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/quickstart/alone/deploy.md @@ -0,0 +1,26 @@ +--- +sidebar_position: 3 +description: Move the private key shares to the nodes and run the cluster +--- + +# Deploy the cluster +To distribute your cluster physically and start the DV, each node needs a directory called `.charon` with one (or several) private key shares within it as per the structure below. + +``` +├── .charon +│   ├── charon-enr-private-key +│   ├── cluster-lock.json +│   ├── deposit-data.json +│   └── validator_keys +│   ├── keystore-0.json +│   ├── keystore-0.txt +│   ├── ... +│   ├── keystore-N.json +│   └── keystore-N.txt +``` + +:point_right: Use the single node [docker compose](https://github.com/ObolNetwork/charon-distributed-validator-node), the kubernetes [manifests](https://github.com/ObolNetwork/charon-k8s-distributed-validator-node), or the [helm chart](https://github.com/ObolNetwork/helm-charts) example repos to get your nodes up and connected after loading the `.charon` folder artifacts into them appropriately. + +:::warning +Right now, the `charon-distributed-node-cluster` repo [used earlier to create the private keys](./create-keys) outputs a folder structure like `.charon/ cluster/node0/validator_keys`. Make sure to grab the `./node0/*` folder, RENAME it to `.charon` and then move it to one of the single node repo above to have a working cluster as per the folder structure shown above. +::: \ No newline at end of file diff --git a/versioned_docs/version-v0.17.0/int/quickstart/alone/test-locally.md b/versioned_docs/version-v0.17.0/int/quickstart/alone/test-locally.md new file mode 100644 index 0000000000..b7fd4ecce5 --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/quickstart/alone/test-locally.md @@ -0,0 +1,81 @@ +--- +sidebar_position: 1 +description: Test the solo cluster locally +--- + +# Run a test cluster locally +:::warning +This is a demo repo to understand how Distributed Validators work and is not suitable for a production deployment. + +This guide only runs one Execution Client, one Consensus Client, and 6 Distributed Validator Charon Client + Validator Client pairs on a single docker instance. As a consequence, if this machine fails, there will not be fault tolerance. + +Follow these two guides sequentially instead for production deployment: [create keys centrally](./create-keys.md) and [how to deploy them](./deploy.md). +::: + +The [`charon-distributed-validator-cluster`](https://github.com/ObolNetwork/charon-distributed-validator-cluster) repo contains six charon clients in separate docker containers along with an execution client and consensus client, simulating a Distributed Validator cluster running. + +The default cluster consists of: +- [Nethermind](https://github.com/NethermindEth/nethermind), an execution layer client +- [Lighthouse](https://github.com/sigp/lighthouse), a consensus layer client +- Six [charon](https://github.com/ObolNetwork/charon) nodes +- A mixture of validator clients: + - VC0: [Lighthouse](https://github.com/sigp/lighthouse) + - vc1: [Teku](https://github.com/ConsenSys/teku) + - vc2: [Nimbus](https://github.com/status-im/nimbus-eth2) + - vc3: [Lighthouse](https://github.com/sigp/lighthouse) + - vc4: [Teku](https://github.com/ConsenSys/teku) + - vc5: [Nimbus](https://github.com/status-im/nimbus-eth2) + +## Pre-requisites + +- Ensure you have [docker](https://docs.docker.com/engine/install/) installed. +- Ensure you have [git](https://git-scm.com/downloads) installed. +- Make sure `docker` is running before executing the commands below. + +## Create the key shares locally + +1. Clone the [charon-distributed-validator-cluster](https://github.com/ObolNetwork/charon-distributed-validator-cluster) repo and `cd` into the directory. + + ```sh + # Clone the repo + git clone https://github.com/ObolNetwork/charon-distributed-validator-cluster.git + + # Change directory + cd charon-distributed-validator-cluster/ + ``` + +2. Prepare the environment variables + + ```sh + # Copy the sample environment variables + cp .env.sample .env + ``` + `.env.sample` is a sample environment file that allows overriding default configuration defined in `docker-compose.yml`. Uncomment and set any variable to override its value. + +3. Create the artifacts needed to run a DV cluster by running the following command: + + ```sh + # Enter required validator addresses + WITHDRAWAL_ADDR= + FEE_RECIPIENT_ADDR= + + # Create a distributed validator cluster + docker run --rm -v "$(pwd):/opt/charon" obolnetwork/charon:v0.17.0 create cluster --name="mycluster" --cluster-dir=".charon/cluster/" --withdrawal-addresses="${WITHDRAWAL_ADDR}" --fee-recipient-addresses="${FEE_RECIPIENT_ADDR}" --nodes 6 --network goerli --num-validators=1 + ``` + +These commands will create six folders within `.charon/cluster`, one for each node created. You will need to rename `node*` to `.charon` for each folder to be found by the default `charon run` command, or you can use `charon run --private-key-file=".charon/cluster/node0/charon-enr-private-key" --lock-file=".charon/cluster/node0/cluster-lock.json"` for each instance of charon you start. + +## Start the cluster + +Run this command to start your cluster containers + +```sh +# Start the distributed validator cluster +docker compose up --build +``` +Check the monitoring dashboard and see if things look all right + +```sh +# Open Grafana +open http://localhost:3000/d/laEp8vupp +``` \ No newline at end of file diff --git a/versioned_docs/version-v0.17.0/int/quickstart/group/_category_.json b/versioned_docs/version-v0.17.0/int/quickstart/group/_category_.json new file mode 100644 index 0000000000..e4c14eb202 --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/quickstart/group/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "Create a DV as a group", + "position": 2, + "collapsed": true +} \ No newline at end of file diff --git a/versioned_docs/version-v0.17.0/int/quickstart/group/index.md b/versioned_docs/version-v0.17.0/int/quickstart/group/index.md new file mode 100644 index 0000000000..96ee831c51 --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/quickstart/group/index.md @@ -0,0 +1,18 @@ +# Run a cluster as a group + +:::caution +Charon is in an alpha state and should be used with caution according to its [Terms of Use](https://obol.tech/terms.pdf). +::: + +:::info +Running a Distributed Validator with others typically means that several operators run the various nodes of the cluster. In such a case, the key shares should be created with a [distributed key generation process](../../key-concepts.md#distributed-validator-key-generation-ceremony), avoiding the private key being stored in full, anywhere. +::: + +There are two sequential user journeys when setting up a DV cluster with others. Each comes with its own quickstart: + +1. The [**Creator** (**Leader**) Journey](./group/quickstart-group-leader-creator), which outlines the steps to propose a Distributed Validator Cluster. + - In the **Creator** case, the person creating the cluster *will NOT* be a node operator in the cluster. + - In the **Leader** case, the person creating the cluster *will* be a node operator in the cluster. + + +2. The [**Operator** Journey](./group/quickstart-group-operator) which outlines the steps to create a Distributed Validator Cluster proposed by a leader or creator using the above process. \ No newline at end of file diff --git a/versioned_docs/version-v0.17.0/int/quickstart/group/quickstart-cli.md b/versioned_docs/version-v0.17.0/int/quickstart/group/quickstart-cli.md new file mode 100644 index 0000000000..9f4b233053 --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/quickstart/group/quickstart-cli.md @@ -0,0 +1,129 @@ +--- +sidebar_position: 3 +description: Run one node in a multi-operator distributed validator cluster using the CLI +--- + +# Using the CLI + +:::caution +Charon is in an alpha state and should be used with caution according to its [Terms of Use](https://obol.tech/terms.pdf). +::: + +The following instructions aim to assist a group of operators coordinating together to create a distributed validator cluster via the CLI. + +## Pre-requisites + +- Ensure you have [docker](https://docs.docker.com/engine/install/) installed. +- Ensure you have [git](https://git-scm.com/downloads) installed. +- Make sure `docker` is running before executing the commands below. +- Decide who the Leader or Creator of your cluster will be. Only them have to perform [step 2](#step-2-leader-creates-the-dkg-configuration-file-and-distributes-it-to-everyone-else) and [step 5](#step-5-activate-the-deposit-data) in this quickstart. They do not get any special privilege. + - In the **Leader** case, the operator creating the cluster will also operate a node in the cluster. + - In the **Creator** case, the cluster is created by an external party to the cluster. + +## Step 1. Create and back up a private key for charon + +In order to prepare for a distributed key generation ceremony, all operators (including the leader but NOT a creator) need to create an [ENR](../../faq/errors.mdx) for their charon client. This ENR is a public/private key pair, and allows the other charon clients in the DKG to identify and connect to your node. + +```sh +# Clone this repo +git clone https://github.com/ObolNetwork/charon-distributed-validator-node.git + +# Change directory +cd charon-distributed-validator-node + +# Create your charon ENR private key, this will create a charon-enr-private-key file in the .charon directory +docker run --rm -v "$(pwd):/opt/charon" obolnetwork/charon:v0.17.0 create enr +``` + +You should expect to see a console output like + + Created ENR private key: .charon/charon-enr-private-key + enr:-JG4QGQpV4qYe32QFUAbY1UyGNtNcrVMip83cvJRhw1brMslPeyELIz3q6dsZ7GblVaCjL_8FKQhF6Syg-O_kIWztimGAYHY5EvPgmlkgnY0gmlwhH8AAAGJc2VjcDI1NmsxoQKzMe_GFPpSqtnYl-mJr8uZAUtmkqccsAx7ojGmFy-FY4N0Y3CCDhqDdWRwgg4u + +:::caution +Please make sure to create a backup of the private key at `.charon/charon-enr-private-key`. Be careful not to commit it to git! **If you lose this file you won't be able to take part in the DKG ceremony and start the DV cluster successfully.** +::: + +Finally, share your ENR with the leader or creator so that he/she can proceed to Step 2. + +## Step 2. Leader or Creator creates the DKG configuration file and distribute it to cluster operators + +1. The leader or creator of the cluster will prepare the `cluster-definition.json` file for the Distributed Key Generation ceremony using the `charon create dkg` command. + + ``` + # Prepare an environment variable file + cp .env.create_dkg.sample .env.create_dkg + ``` +2. Populate the `.env.create_dkg` file created with the `cluster name`, the `fee recipient` and `withdrawal Ethereum addresses`, and the `ENRs` of all the operators participating in the cluster. + - The file generated is hidden by default. To view it, run `ls -al` in your terminal. Else, if you are on `macOS`, press `Cmd + Shift + .` to view all hidden files in the finder application. + +3. Run the `charon create dkg` command that generates DKG cluster-definition.json file. + ``` + docker run --rm -v "$(pwd):/opt/charon" --env-file .env.create_dkg obolnetwork/charon:v0.17.0 create dkg + ``` + + This command should output a file at `.charon/cluster-definition.json`. This file needs to be shared with the other operators in a cluster. + +## Step 3. Run the DKG + +After receiving the `cluster-definition.json` file created by the leader, cluster operators should ideally save it in the `.charon/` folder that was created during step 1, alternatively the `--definition-file` flag can override the default expected location for this file. + +Every cluster member then participates in the DKG ceremony. For Charon v1, this needs to happen relatively synchronously between participants at an agreed time. + +``` +# Participate in DKG ceremony, this will create .charon/cluster-lock.json, .charon/deposit-data.json and .charon/validator_keys +docker run --rm -v "$(pwd):/opt/charon" obolnetwork/charon:v0.17.0 dkg +``` + +>This is a helpful [video walkthrough](https://www.youtube.com/watch?v=94Pkovp5zoQ&ab_channel=ObolNetwork). + +Assuming the DKG is successful, a number of artefacts will be created in the `.charon` folder. These include: + +- A `deposit-data.json` file. This contains the information needed to activate the validator on the Ethereum network. +- A `cluster-lock.json` file. This contains the information needed by charon to operate the distributed validator cluster with its peers. +- A `validator_keys/` folder. This folder contains the private key shares and passwords for the created distributed validators. + +:::caution +Please make sure to create a backup of `.charon/validator_keys`. **If you lose your keys you won't be able to start the DV cluster successfully.** +::: + +:::info +The `cluster-lock` and `deposit-data` files are identical for each operator and can be copied if lost. +::: + +## Step 4. Start your Distributed Validator Node + +With the DKG ceremony over, the last phase before activation is to prepare your node for validating over the long term. This repo is configured to sync an execution layer client (`geth`) and a consensus layer client (`lighthouse`). + +Before completing these instructions, you should assign a static local IP address to your device (extending the DHCP reservation indefinitely or removing the device from the DCHP pool entirely if you prefer), and port forward the TCP protocol on the public port `:3610` on your router to your device's local IP address on the same port. This step is different for every person's home internet, and can be complicated by the presence of dynamic public IP addresses. We are currently working on making this as easy as possible, but for the time being, a distributed validator cluster isn't going to work very resiliently if all charon nodes cannot talk directly to one another and instead need to have an intermediary node forwarding traffic to them. + +**Caution**: If you manually update `docker-compose` to mount `lighthouse` from your locally synced `~/.lighthouse`, the whole chain database may get deleted. It'd be best not to manually update as `lighthouse` checkpoint-syncs so the syncing doesn't take much time. + +**Note**: If you have a `geth` node already synced, you can simply copy over the directory. For ex: `cp -r ~/.ethereum/goerli data/geth`. This makes everything faster since you start from a synced geth node. + +``` +# Delete lighthouse data if it exists +rm -r ./data/lighthouse + +# Spin up a Distributed Validator Node with a Validator Client +docker compose up + +# Open Grafana dashboard +open http://localhost:3000/d/singlenode/ +``` + +You should use the grafana dashboard to infer whether your cluster is healthy. In particular you should check: + +- That your charon client can connect to the configured beacon client. +- That your charon client can connect to all peers. + +Most components in the dashboard have some help text there to assist you in understanding your cluster performance. + +You might notice that there are logs indicating that a validator cannot be found and that APIs are returning 404. This is to be expected at this point, as the validator public keys listed in the lock file have not been deposited and acknowledged on the consensus layer yet (usually ~16 hours after the deposit is made). + +If at any point you need to turn off your node, you can run: + +``` +# Shut down the currently running distributed validator node +docker compose down +``` \ No newline at end of file diff --git a/versioned_docs/version-v0.17.0/int/quickstart/group/quickstart-group-leader-creator.md b/versioned_docs/version-v0.17.0/int/quickstart/group/quickstart-group-leader-creator.md new file mode 100644 index 0000000000..d0b7481c94 --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/quickstart/group/quickstart-group-leader-creator.md @@ -0,0 +1,194 @@ +--- +sidebar_position: 1 +description: A leader/creator creates a cluster configuration to be shared with operators +--- +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; + +# Creator & Leader Journey + +:::caution +Charon is in an alpha state and should be used with caution according to its [Terms of Use](https://obol.tech/terms.pdf). +::: + +The following instructions aim to assist with the preparation of a distributed validator key generation ceremony. Select the *Leader* tab if you **will** be an operator participating in the cluster, and select the *Creator* tab if you **will NOT** be an operator in the cluster. + +These roles hold no position of privilege in the cluster, they only set the initial terms of the cluster that the other operators agree to. + + + + The person creating the cluster will be a node operator in the cluster.

    +

    Pre-requisites

    +
      +
    • Ensure you have docker installed.
      • +
    • Ensure you have git installed.
      • +
    • Make sure docker is running before executing the commands below.
      • +
    +     + + The person creating the cluster will not be a node operator in the cluster. + +     + +## Overview Video + +

    + +## Step 1. Collect Ethereum addresses of the cluster operators +Before starting the cluster creation, you will need to collect one Ethereum address per operator in the cluster. They will need to be able to sign messages through metamask with this address. Broader wallet support will be added in future. + +## Step 2. Create and back up a private key for charon + + + + + In order to prepare for a distributed key generation ceremony, you need to create an [ENR](docs/int/faq/errors.mdx#enrs-keys) for your charon client. Operators in your cluster will also need to do this step, as per their [quickstart](./quickstart-group-operator#step-2-create-and-back-up-a-private-key-for-charon). This ENR is a public/private key pair, and allows the other charon clients in the DKG to identify and connect to your node. + + ```sh + # Clone this repo + git clone https://github.com/ObolNetwork/charon-distributed-validator-node.git + + # Change directory + cd charon-distributed-validator-node + + # Create your charon ENR private key, this will create a charon-enr-private-key file in the .charon directory + docker run --rm -v "$(pwd):/opt/charon" obolnetwork/charon:v0.17.0 create enr + ``` + +You should expect to see a console output like + + Created ENR private key: .charon/charon-enr-private-key + enr:-JG4QGQpV4qYe32QFUAbY1UyGNtNcrVMip83cvJRhw1brMslPeyELIz3q6dsZ7GblVaCjL_8FKQhF6Syg-O_kIWztimGAYHY5EvPgmlkgnY0gmlwhH8AAAGJc2VjcDI1NmsxoQKzMe_GFPpSqtnYl-mJr8uZAUtmkqccsAx7ojGmFy-FY4N0Y3CCDhqDdWRwgg4u + +If instead of being shown your `enr` you see an error saying `permission denied` then you may need to [update docker permissions](/docs/int/faq/errors#docker-permission-denied-error) to allow the command to run successfully. + +:::caution +Please make sure to create a backup of the private key at `.charon/charon-enr-private-key`. Be careful not to commit it to git! **If you lose this file you won't be able to take part in the DKG ceremony and start the DV cluster successfully.** +::: + + + + + This step is not needed and you can move on to [Step 3](#step-3-create-the-dkg-configuration-file-and-distribute-it-to-cluster-operators). + + + + +## Step 3. Create the DKG configuration file and distribute it to cluster operators + +You will prepare the configuration file for the distributed key generation ceremony using the launchpad. + +1. Go to the [DV Launchpad](https://goerli.launchpad.obol.tech) +2. Connect your wallet + + ![Connect your Wallet](/img/Guide01.png) + +3. Select `Create a Cluster with a group` then `Get Started`. + + ![Get Started](/img/Guide02.png) + +4. Follow the flow and accept the advisories. +5. Configure the Cluster + - Input the `Cluster Name` & `Cluster Size` (i.e. number of operators in the cluster). The threshold for the cluster to operate sucessfully will update automatically. + + + +
    • + + ⚠️ Leave the `Non-Operator` toggle OFF. + +
    +     + +
    • + + ⚠️ Turn the `Non-Operator` toggle ON. + +
    +     +     + + - Input the Ethereum addresses for each operator collected during [step 1](#step-1-collect-ethereum-addresses-of-the-cluster-operators). + - Select the desired amount of validators (32 ETH each) the cluster will run. + - Paste your `ENR` generated at [Step 2](#step-2-create-and-back-up-a-private-key-for-charon). + - Select the `Withdrawal Addresses` method. Use `Single address` to receive the principal and fees to a single address or `Splitter Contracts` to share them among operators. + + + +
      +
    • Enter the Withdrawal Address that will receive the validator effective balance at exit and when balance skimming occurs.
      • +
    • Enter the Fee Recipient Address to receive MEV rewards (if enabled), and block proposal priority fees.
      • +
    + You can set them to be the same as your connected wallet address in one click. +

    + + ![Create Group](/img/Guide03.png) + +     + +
      +
    • Enter the Ethereum address to claim the validator principal (32 ether) at exit.
      • +
    • Enter the Ethereum addresses and their percentage split of the validator's rewards. Validator rewards include consensus rewards, MEV rewards and proposal priority fees.
      • +

    + + ![Create Group](/img/Guide03-splitter.png) + +     +     + + - Click `Create Cluster Configuration` + + + + +
      + 6. Review the cluster configuration +
    +     + +
      6. Deploy the withdrawal manager contracts by signing the two transactions with your wallet.
    +     +     + + + +
      + 7. You will be asked to confirm your configuration and to sign: +
    +
      +
        +
      • The config_hash. This is a hashed representation of the details of this cluster, to ensure everyone is agreeing to an identical setup.
        • +
      • The operator_config_hash. This is your acceptance of the terms as a participating node operator.
        • +
      • Your ENR. Signing your ENR authorises the corresponding private key to act on your behalf in the cluster.
        • +
      +
    +     + +
      + 7. You will be asked to confirm your configuration and to sign: +
    +
      +
        +
      • The config_hash. This is a hashed representation of the details of this cluster, to ensure everyone is agreeing to an identical setup.
        • +
      +
    +     +     + +8. Share your cluster invite link with the operators. Following the link will show you a screen waiting for other operators to accept the configuration you created. + + ![Invite Operators](/img/Guide04.png) + + + + + 👉 Once every participating operator has signed their approval to the terms, you will continue the [**Operator** journey](./quickstart-group-operator#step-3-run-the-dkg) by completing the distributed key generation step. + + + + + Your journey ends here and you can monitor with the link whether the operators confirm their agreement to the cluster by signing their approval. Future versions of the launchpad will allow a creator to track a distributed validator's lifecycle in its entirety. + + + + diff --git a/versioned_docs/version-v0.17.0/int/quickstart/group/quickstart-group-operator.md b/versioned_docs/version-v0.17.0/int/quickstart/group/quickstart-group-operator.md new file mode 100644 index 0000000000..3c56d24c8f --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/quickstart/group/quickstart-group-operator.md @@ -0,0 +1,144 @@ +--- +sidebar_position: 1 +description: A node operator joins a DV cluster +--- + +# Operator Journey + +:::caution +Charon is in an alpha state and should be used with caution according to its [Terms of Use](https://obol.tech/terms.pdf). +::: + +The following instructions aim to assist a group of operators coordinating together to create a distributed validator cluster after receiving an cluster invite link from a leader or creator. + +## Overview Video +

    + +## Pre-requisites + +- Ensure you have [docker](https://docs.docker.com/engine/install/) installed. +- Ensure you have [git](https://git-scm.com/downloads) installed. +- Make sure `docker` is running before executing the commands below. + +## Step 1. Share an Ethereum address with your Leader or Creator +Before starting the cluster creation, make sure you have shared an Ethereum address with your cluster **Leader** or **Creator**. If you haven't chosen someone as a Leader or Creator yet, please go back to the [Quickstart intro](./index.md) and define one person to go through the [Leader & Creator Journey](./quickstart-group-leader-creator) before moving forward. + +## Step 2. Create and back up a private key for charon + +In order to prepare for a distributed key generation ceremony, you need to create an [ENR](docs/int/faq/errors.mdx#enrs-keys) for your charon client. This ENR is a public/private key pair, and allows the other charon clients in the DKG to identify and connect to your node. + +```sh +# Clone this repo +git clone https://github.com/ObolNetwork/charon-distributed-validator-node.git + +# Change directory +cd charon-distributed-validator-node + +# Create your charon ENR private key, this will create a charon-enr-private-key file in the .charon directory +docker run --rm -v "$(pwd):/opt/charon" obolnetwork/charon:v0.17.0 create enr +``` + +You should expect to see a console output like + + Created ENR private key: .charon/charon-enr-private-key + enr:-JG4QGQpV4qYe32QFUAbY1UyGNtNcrVMip83cvJRhw1brMslPeyELIz3q6dsZ7GblVaCjL_8FKQhF6Syg-O_kIWztimGAYHY5EvPgmlkgnY0gmlwhH8AAAGJc2VjcDI1NmsxoQKzMe_GFPpSqtnYl-mJr8uZAUtmkqccsAx7ojGmFy-FY4N0Y3CCDhqDdWRwgg4u + +If instead of being shown your `enr` you see an error saying `permission denied` then you may need to [update docker permissions](/docs/int/faq/errors#docker-permission-denied-error) to allow the command to run successfully. + +:::caution +Please make sure to create a backup of the private key at `.charon/charon-enr-private-key`. Be careful not to commit it to git! **If you lose this file you won't be able to take part in the DKG ceremony and start the DV cluster successfully.** +::: + +## Step 3. Join and sign the cluster configuration + +After receiving the invite link created by the **Leader** or **Creator**, you will be able to join and sign the cluster configuration created. + +1. Go to the DV launchpad link provided by the leader or creator. +2. Connect your wallet using the Ethereum address provided to the leader in [step 1](#step-1-share-an-ethereum-address-with-your-leader-or-creator). + + ![Connect Wallet](/img/Guide05.png) + +3. Review the operators addresses submitted and click `Get Started` to continue. + + ![Get Started](/img/Guide06.png) + +4. Review and accept the advisories. +5. Review the configuration created by the leader or creator and add your `ENR` generated in [step 2](#step-2-create-and-back-up-a-private-key-for-charon). + + ![Review Config](/img/Guide07.png) + +6. Sign the following with your wallet + - The config hash. This is a hashed representation of all of the details for this cluster. + - Your own `ENR`. This signature authorises the key represented by this ENR to act on your behalf in the cluster. + +7. Wait for all the other operators in your cluster to do the same. + +## Step 4. Run the DKG +:::info +For the [DKG](docs/charon/dkg.md) to complete, all operators need to be running the command simultaneously. It helps to coordinate an agreed upon time amongst operators at which to run the command. +::: + +### Overview +

    + +1. Once all operators successfully signed, your screen will automatically advance to the next step and look like this. Click `Continue`. If you closed the tab, just go back to the invite link shared by the leader and connect your wallet. + + ![Config Signing Success](/img/Guide08.png) + +2. You have two options to perform the DKG. + 1. **Option 1** and default is to copy and run the `docker` command on the screen into your terminal. It will retrieve the remote cluster details and begin the DKG process. + + 2. **Option 2** (Manual DKG) is to download the `cluster-definition` file manually and move it to the hidden `.charon` folder. Then, every cluster member participates in the DKG ceremony by running the command displayed. + + ![Run the DKG](/img/Guide10.png) + +3. Assuming the DKG is successful, a number of artefacts will be created in the `.charon` folder. These include: + + - A `deposit-data.json` file. This contains the information needed to activate the validator on the Ethereum network. + - A `cluster-lock.json` file. This contains the information needed by charon to operate the distributed validator cluster with its peers. + - A `validator_keys/` folder. This folder contains the private key shares and passwords for the created distributed validators. + +:::caution +Please make sure to create a backup of `.charon/validator_keys`. **If you lose your keys you won't be able to start the DV cluster successfully.** +::: + +:::info +The `cluster-lock` and `deposit-data` files are identical for each operator and can be copied if lost. +::: + +## Step 5. Start your Distributed Validator Node + +With the DKG ceremony over, the last phase before activation is to prepare your node for validating over the long term. This repo is configured to sync an execution layer client (`geth`) and a consensus layer client (`lighthouse`). + +Before completing these instructions, you should assign a static local IP address to your device (extending the DHCP reservation indefinitely or removing the device from the DCHP pool entirely if you prefer), and port forward the TCP protocol on the public port `:3610` on your router to your device's local IP address on the same port. This step is different for every person's home internet, and can be complicated by the presence of dynamic public IP addresses. We are currently working on making this as easy as possible, but for the time being, a distributed validator cluster isn't going to work very resiliently if all charon nodes cannot talk directly to one another and instead need to have an intermediary node forwarding traffic to them. + +**Caution**: If you manually update `docker compose` to mount `lighthouse` from your locally synced `~/.lighthouse`, the whole chain database may get deleted. It'd be best not to manually update as `lighthouse` checkpoint-syncs so the syncing doesn't take much time. + +**Note**: If you have a `geth` node already synced, you can simply copy over the directory. For ex: `cp -r ~/.ethereum/goerli data/geth`. This makes everything faster since you start from a synced geth node. + +``` +# Delete lighthouse data if it exists +rm -r ./data/lighthouse + +# Spin up a Distributed Validator Node with a Validator Client +docker compose up + +# Open Grafana dashboard +open http://localhost:3000/d/singlenode/ +``` + +You should use the grafana dashboard to infer whether your cluster is healthy. In particular you should check: + +- That your charon client can connect to the configured beacon client. +- That your charon client can connect to all peers. + +Most components in the dashboard have some help text there to assist you in understanding your cluster performance. + +You might notice that there are logs indicating that a validator cannot be found and that APIs are returning 404. This is to be expected at this point, as the validator public keys listed in the lock file have not been deposited and acknowledged on the consensus layer yet (usually ~16 hours after the deposit is made). + +If at any point you need to turn off your node, you can run: + +``` +# Shut down the currently running distributed validator node +docker compose down +``` \ No newline at end of file diff --git a/versioned_docs/version-v0.17.0/int/quickstart/index.md b/versioned_docs/version-v0.17.0/int/quickstart/index.md new file mode 100644 index 0000000000..5d4cb15900 --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/quickstart/index.md @@ -0,0 +1,11 @@ +# Quickstart Guides + +:::caution +Charon is in an alpha state and should be used with caution according to its [Terms of Use](https://obol.tech/terms.pdf). +::: + +There are two ways to set up a distributed validator and each comes with its own quickstart +1. [Run a DV cluster as a **group**](./group/index.md), where several operators run the nodes that make up the cluster. In this setup, the key shares are created using a distributed key generation process, avoiding the full private keys being stored in full in any one place. +This approach can also be used by single operators looking to manage all nodes of a cluster but wanting to create the key shares in a trust-minimised fashion. + +2. [Run a DV cluster **alone**](./quickstart/alone/create-keys), where a single operator runs all the nodes of the DV. Depending on trust assumptions, there is not necessarily the need to create the key shares via a DKG process. Instead the key shares can be created in a centralised manner, and distributed securely to the nodes. \ No newline at end of file diff --git a/versioned_docs/version-v0.17.0/int/quickstart/quickstart-exit.md b/versioned_docs/version-v0.17.0/int/quickstart/quickstart-exit.md new file mode 100644 index 0000000000..875b844b8f --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/quickstart/quickstart-exit.md @@ -0,0 +1,128 @@ +--- +sidebar_position: 5 +description: Exit a validator +--- + +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; + +# Exit a DV + +:::caution +Charon is in an alpha state and should be used with caution according to its [Terms of Use](https://obol.tech/terms.pdf). +::: + +Users looking to exit staking entirely and withdraw their full balance back must also sign and broadcast a "voluntary exit" message with validator keys which will start the process of exiting from staking. This is done with your validator client and submitted to your beacon node, and does not require gas. In the case of a DV, each charon node needs to broadcast a partial exit to the other nodes of the cluster. Once a threshold of partial exits has been received by any node, the full voluntary exit will be sent to the beacon chain. + +This process will take 27 hours or longer depending on the current length of the exit queue. + +:::info +- A threshold of operators needs to run the exit command for the exit to succeed. +- If a charon client restarts after the exit command is run but before the threshold is reached, it will lose the partial exits it has received from the other nodes. If all charon clients restart and thus all partial exits are lost before the required threshold of exit messages are received, operators will have to rebroadcast their partial exit messages. +::: + +## Run the `voluntary-exit` command on your validator client + +Run the appropriate command on your validator client to broadcast an exit message from your validator client to its upstream charon client. + +It needs to be the validator client that is connected to your charon client taking part in the DV, as you are only signing a partial exit message, with a partial private key share, which your charon client will combine with the other partial exit messages from the other operators. + +:::info +- All operators need to use the same `EXIT_EPOCH` for the exit to be successful. Assuming you want to exit as soon as possible, the default epoch of `162304` included in the below commands should be sufficient. +- Partial exits can be broadcasted by any validator client as long as the sum reaches the threshold for the cluster. +::: + + + + 
    +      
+    {String.raw`docker exec -ti charon-distributed-validator-node-teku-1 /opt/teku/bin/teku voluntary-exit \
+    --beacon-node-api-endpoint="http://charon:3600/" \
+    --confirmation-enabled=false \
+    --validator-keys="/opt/charon/validator_keys:/opt/charon/validator_keys" \
+    --epoch=162304`}
+      
+
    +     + + The following executes an interactive command inside the Nimbus VC container. It copies all files and directories from the Keystore path /home/user/data/charon to the newly created /home/user/data/wd directory. +

    + For each file in the /home/user/data/wd/secrets directory, it: +
  • Extracts the filename without the extension as the file name is the public key
    • +
  • Appends {String.raw`--validator=`} to the command variable.
    • +
  • Executes a program called nimbus_beacon_node with the following arguments:
    • +
      +
    • deposits exit : Exits validators
      • +
    • $command : The generated command string from the loop.
      • +
    • --epoch=162304 : The epoch upon which to submit the voluntary exit.
      • +
    • --rest-url=http://charon:3600/ : Specifies the Charon host:port
      • +
    • --data-dir=/home/user/charon/ : Specifies the Keystore path which has all the validator keys. There will be a secrets and a validators folder inside it.
      • +
    + 
    +      
+        {String.raw`docker exec -it charon-distributed-validator-node-nimbus-1 /bin/bash -c '\
+    
+        mkdir /home/user/data/wd
+        cp -r /home/user/data/charon/ /home/user/data/wd
+    
+        command=""; \
+        for file in /home/user/data/wd/secrets/*; do \
+            filename=$(basename "$file" | cut -d. -f1); \
+            command+=" --validator=$filename"; \
+        done; \
+    
+/home/user/nimbus_beacon_node deposits exit $command --epoch=162304 --rest-url=http://charon:3600/ --data-dir=/home/user/data/wd/'`}
+      
+
    +     + + The following executes an interactive command inside the Loestar VC container to exit all validators. It executes + node /usr/app/packages/cli/bin/lodestar validator voluntary-exit with the arguments: +
      +
    • --beaconNodes="http://charon:3600" : Specifies the Charon host:port.
      • +
    • --data-dir=/opt/data : Specifies the folder where the key stores were imported.
      • +
    • --exitEpoch=162304 : The epoch upon which to submit the voluntary exit.
      • +
    • --network=goerli : Specifies the network.
      • +
    • --yes : Skips confirmation prompt.
      • +
    + 
    +      
+    {String.raw`docker exec -it charon-distributed-validator-node-lodestar-1 /bin/sh -c 'node /usr/app/packages/cli/bin/lodestar validator voluntary-exit --beaconNodes="http://charon:3600" --dataDir=/opt/data --exitEpoch=162304 --network=goerli --yes'`}
+      
+
    +     +     + +Once a threshold of exit signatures has been received by any single charon client, it will craft a valid voluntary exit message and will submit it to the beacon chain for inclusion. You can monitor partial exits stored by each node in the [Grafana Dashboard](https://github.com/ObolNetwork/charon-distributed-validator-node). + +## Exit epoch and withdrawable epoch +The process of a validator exiting from staking takes variable amounts of time, depending on how many others are exiting at the same time. + +Immediately upon broadcasting a signed voluntary exit message, the exit epoch and withdrawable epoch values are calculated based off the current epoch number. These values determine exactly when the validator will no longer be required to be online performing validation, and when the validator is eligible for a full withdrawal respectively. +1. Exit epoch - epoch at which your validator is no longer active, no longer earning rewards, and is no longer subject to slashing rules. + :::caution + Up until this epoch (while "in the queue") your validator is expected to be online and is held to the same slashing rules as always. Do not turn your DV node off until this epoch is reached. + ::: +2. Withdrawable epoch - epoch at which your validator funds are eligible for a full withdrawal during the next validator sweep. +This occurs 256 epochs after the exit epoch, which takes ~27.3 hours. + +## How to verify a validator exit + +Consult the examples below and compare them to your validator's monitoring to verify that exits from each operator in the cluster are being received. This example is a cluster of 4 nodes with 2 validators and threshold of 3 nodes broadcasting exits are needed. + +1. Operator 1 broadcasts an exit on validator client 1. + ![Verify in Grafana Exit panel](/img/ExitPromQuery-01.png) + ![Verify in Grafana Exit panel](/img/DutyExit-01.png) +2. Operator 2 broadcasts an exit on validator client 2. + ![Verify in Grafana Exit panel](/img/ExitPromQuery-02.png) + ![Verify in Grafana Exit panel](/img/DutyExit-02.png) +3. Operator 3 broadcasts an exit on validator client 3. + ![Verify in Grafana Exit panel](/img/ExitPromQuery-03.png) + ![Verify in Grafana Exit panel](/img/DutyExit-03.png) + +At this point, the threshold of 3 has been reached and the validator exit process will start. The logs will show the following: + ![Verify in Grafana Exit panel](/img/ExitLogs.png) + +:::tip +Once a validator has broadcasted an exit message, it must continue to validate for at least 27 hours or longer. Do not shut off your distributed validator nodes until your validator is fully exited. +::: \ No newline at end of file diff --git a/versioned_docs/version-v0.17.0/int/quickstart/quickstart-mainnet.md b/versioned_docs/version-v0.17.0/int/quickstart/quickstart-mainnet.md new file mode 100644 index 0000000000..c5ebab2674 --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/quickstart/quickstart-mainnet.md @@ -0,0 +1,107 @@ +--- +sidebar_position: 7 +description: Run a cluster on mainnet +--- + +# Run a DV on mainnet + +:::caution +Charon is in an alpha state, and you should proceed only if you accept the risk, the [terms of use](https://obol.tech/terms.pdf), and have tested running a Distributed Validator on a testnet first. + +Distributed Validators created for goerli cannot be used on mainnet and vice versa, please take caution when creating, backing up, and activating mainnet validators. +::: + +This section is intended for users who wish to run their Distributed Validator on Ethereum mainnet. + +### Pre-requisites + +- Ensure you have [docker](https://docs.docker.com/engine/install/) installed. +- Ensure you have [git](https://git-scm.com/downloads) installed. +- Make sure `docker` is running before executing the commands below. + +### Steps + +1. Clone the [charon-distributed-validator-node](https://github.com/ObolNetwork/charon-distributed-validator-node) repo and `cd` into the directory. + +```sh +# Clone this repo +git clone https://github.com/ObolNetwork/charon-distributed-validator-node.git + +# Change directory +cd charon-distributed-validator-node +``` + +2. If you have already cloned the repo, make sure that it is [up-to-date](./update). + +3. Copy the `.env.sample` file to `.env` +``` +cp -n .env.sample .env +``` + +4. In your `.env` file, uncomment and set values for `NETWORK` & `LIGHTHOUSE_CHECKPOINT_SYNC_URL` +``` +... +# Overrides network for all the relevant services. +NETWORK=mainnet +... +# Checkpoint sync url used by lighthouse to fast sync. +LIGHTHOUSE_CHECKPOINT_SYNC_URL=https://mainnet.checkpoint.sigp.io/https://eth-clients.github.io/checkpoint-sync-endpoints/#mainnet +... +``` +Note that you can choose any checkpoint sync url from https://eth-clients.github.io/checkpoint-sync-endpoints/#mainnet. + +Your DV stack is now mainnet ready 🎉 + +#### Remote mainnet beacon node + +:::caution +Using a remote beacon node will impact the performance of your Distributed Validator and should be used sparingly. +::: + +If you already have a mainnet beacon node running somewhere and you want to use that instead of running EL (`geth`) & CL (`lighthouse`) as part of the repo, you can disable these images. To do so, follow these steps: + +1. Copy the `docker-compose.override.yml.sample` file +``` +cp -n docker-compose.override.yml.sample docker-compose.override.yml +``` +2. Uncomment the `profiles: [disable]` section for both `geth` and `lighthouse`. The override file should now look like this +``` +services: + geth: + # Disable geth + profiles: [disable] + # Bind geth internal ports to host ports + #ports: + #- 8545:8545 # JSON-RPC + #- 8551:8551 # AUTH-RPC + #- 6060:6060 # Metrics + + lighthouse: + # Disable lighthouse + profiles: [disable] + # Bind lighthouse internal ports to host ports + #ports: + #- 5052:5052 # HTTP + #- 5054:5054 # Metrics +... +``` +3. Then, uncomment and set the `CHARON_BEACON_NODE_ENDPOINTS` variable in the `.env` file to your mainnet beacon node's URL +``` +... +# Connect to one or more external beacon nodes. Use a comma separated list excluding spaces. +CHARON_BEACON_NODE_ENDPOINTS= +... +``` + +#### Exit a mainnet distributed validator + +If you want to exit your mainnet validator, you need to uncomment and set the `EXIT_EPOCH` variable in the `.env` file + +``` +... +# Cluster wide consistent exit epoch. Set to latest for fork version, see `curl $BEACON_NODE/eth/v1/config/fork_schedule` +# Currently, the latest fork is capella (epoch: 194048) +EXIT_EPOCH=194048 +... +``` +Note that `EXIT_EPOCH` should be `194048` after the [shapella fork](https://blog.ethereum.org/2023/03/28/shapella-mainnet-announcement). diff --git a/versioned_docs/version-v0.17.0/int/quickstart/update.md b/versioned_docs/version-v0.17.0/int/quickstart/update.md new file mode 100644 index 0000000000..3187bbc0bf --- /dev/null +++ b/versioned_docs/version-v0.17.0/int/quickstart/update.md @@ -0,0 +1,76 @@ +--- +sidebar_position: 6 +description: Update your DV cluster with the latest Charon release +--- +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; + +# Update a DV + +It is highly recommended to upgrade your DV stack from time to time. This ensures that your node is secure, performant, up-to-date and you don't miss important hard forks. + +To do this, follow these steps: + +### Navigate to the node directory + + + + 
    +      
+        cd charon-distributed-validator-node
+      
+
    +     + + + 
    +      
+        cd charon-distributed-validator-cluster
+      
+
    +     +     + +### Pull latest changes to the repo +``` +git pull +``` + +### Create (or recreate) your DV stack +``` +docker compose up -d --build +``` +:::warning +If you run more than one node in a DV Cluster, please take caution upgrading them simultaneously. Particularly if you are updating or changing the validator client used or recreating disks. It is recommended to update nodes on a sequential basis to minimse liveness and safety risks. +::: + +### Conflicts + +:::info +You may get a `git conflict` error similar to this: +::: +```markdown +error: Your local changes to the following files would be overwritten by merge: +prometheus/prometheus.yml +... +Please commit your changes or stash them before you merge. +``` +This is probably because you have made some changes to some of the files, for example to the `prometheus/prometheus.yml` file. + +To resolve this error, you can either: + +- Stash and reapply changes if you want to keep your custom changes: + ``` + git stash # Stash your local changes + git pull # Pull the latest changes + git stash apply # Reapply your changes from the stash + ``` + After reapplying your changes, manually resolve any conflicts that may arise between your changes and the pulled changes using a text editor or Git's conflict resolution tools. + +- Override changes and recreate configuration if you don't need to preserve your local changes and want to discard them entirely: + ``` + git reset --hard # Discard all local changes and override with the pulled changes + docker-compose up -d --build # Recreate your DV stack + ``` + After overriding the changes, you will need to recreate your DV stack using the updated files. + By following one of these approaches, you should be able to handle Git conflicts when pulling the latest changes to your repository, either preserving your changes or overriding them as per your requirements. \ No newline at end of file diff --git a/versioned_docs/version-v0.17.0/intro.md b/versioned_docs/version-v0.17.0/intro.md new file mode 100644 index 0000000000..10a81b9143 --- /dev/null +++ b/versioned_docs/version-v0.17.0/intro.md @@ -0,0 +1,20 @@ +--- +sidebar_position: 1 +description: Welcome to the Multi-Operator Validator Network +--- + +# Introduction + +## What is Obol? + +Obol Labs is a research and software development team focused on proof-of-stake infrastructure for public blockchain networks. Specific topics of focus are Internet Bonds, Distributed Validator Technology and Multi-Operator Validation. The team currently includes 20 members that are spread across the world. + +The core team is building the Obol Network, a protocol to foster trust minimized staking through multi-operator validation. This will enable low-trust access to Ethereum staking yield, which can be used as a core building block in a variety of Web3 products. + +## About this documentation + +This manual is aimed at developers and stakers looking to utilize the Obol Network for multi-party staking. To contribute to this documentation, head over to our [Github repository](https://github.com/ObolNetwork/obol-docs) and file a pull request. + +## Need assistance? + +If you have any questions about this documentation or are experiencing technical problems with any Obol-related projects, head on over to our [Discord](https://discord.gg/n6ebKsX46w) where a member of our team or the community will be happy to assist you. diff --git a/versioned_docs/version-v0.17.0/sc/01_introducing-obol-managers.md b/versioned_docs/version-v0.17.0/sc/01_introducing-obol-managers.md new file mode 100644 index 0000000000..2cc857f2c9 --- /dev/null +++ b/versioned_docs/version-v0.17.0/sc/01_introducing-obol-managers.md @@ -0,0 +1,59 @@ +--- +description: How does the Obol Network look on-chain? +--- + +# Obol Manager Contracts + +Obol develops and maintains a suite of smart contracts for use with Distributed Validators. + +## Withdrawal Recipients + +The key to a distributed validator is understanding how a withdrawal is processed. The most common way to handle a withdrawal of a validator operated by a number of different people is to use an immutable withdrawal recipient contract, with the distribution rules hardcoded into it. + +For the time being Obol uses `0x01` withdrawal credentials, and intends to upgrade to [0x03 withdrawal credentials](https://ethresear.ch/t/0x03-withdrawal-credentials-simple-eth1-triggerable-withdrawals/10021) when smart contract initiated exits are enabled. + +### Ownable Withdrawal Recipient + +```solidity title="WithdrawalRecipientOwnable.sol" +// SPDX-License-Identifier: MIT + +pragma solidity ^0.8.0; + +import "@openzeppelin/contracts/access/Ownable.sol"; + +contract WithdrawalRecipientOwnable is Ownable { + receive() external payable {} + + function withdraw(address payable recipient) public onlyOwner { + recipient.transfer(address(this).balance); + } +} + +``` + +An Ownable Withdrawal Recipient is the most basic type of withdrawal recipient contract. It implements Open Zeppelin's `Ownable` interface and allows one address to call the `withdraw()` function, which pulls all ether from the address into the owners address (or another address specified). Calling withdraw could also fund a fee split to the Obol Network, and/or the protocol that has deployed and instantiated this DV. + +### Immutable Withdrawal Recipient + +An immutable withdrawal recipient is similar to an ownable recipient except the owner is hardcoded during construction and the ability to change ownership is removed. This contract should only be used as part of a larger smart contract system, for example a yearn vault strategy might use an immutable recipient contract as its vault address should never change. + +## Registries + +### Deposit Registry + +The Deposit Registry is a way for the deposit and activation of distributed validators to be two separate processes. In the simple case for DVs, a registry of deposits is not required. However when the person depositing the ether is not the same entity as the operators producing the deposits, a coordination mechanism is needed to make sure only one 32 eth deposit is submitted per DV. A deposit registry can prevent double deposits by ordering the allocation of ether to validator deposits. + +### Operator Registry + +If the submission of deposits to a deposit registry needs to be gated to only whitelisted addresses, a simple operator registry may serve as a way to control who can submit deposits to the deposit registry. + +### Validator Registry + +If validators need to be managed on chain programmatically rather than manually with humans triggering exits, a validator registry can be used. Deposits getting activated get an entry into the validator registry, and validators using 0x03 exits get staged for removal from the registry. This registry can be used to coordinate many validators with similar operators and configuration. + +:::note + +Validator registries depend on the as of yet unimplemented `0x03` validator exit feature. + +::: + diff --git a/versioned_docs/version-v0.17.0/sc/_category_.json b/versioned_docs/version-v0.17.0/sc/_category_.json new file mode 100644 index 0000000000..c740383c52 --- /dev/null +++ b/versioned_docs/version-v0.17.0/sc/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "Smart contracts", + "position": 5, + "collapsed": true +} diff --git a/versioned_docs/version-v0.17.0/sec/_category_.json b/versioned_docs/version-v0.17.0/sec/_category_.json new file mode 100644 index 0000000000..2c9d0b38b7 --- /dev/null +++ b/versioned_docs/version-v0.17.0/sec/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "Security", + "position": 7, + "collapsed": true +} diff --git a/versioned_docs/version-v0.17.0/sec/bug-bounty.md b/versioned_docs/version-v0.17.0/sec/bug-bounty.md new file mode 100644 index 0000000000..528a9c871a --- /dev/null +++ b/versioned_docs/version-v0.17.0/sec/bug-bounty.md @@ -0,0 +1,95 @@ +--- +description: Bug Bounty Policy +sidebar_position: 3 +--- + +# Obol Bug Bounty + +## Overview +Obol Labs is committed to ensuring the security of our distributed validator software and services. As part of our commitment to security, we have established a bug bounty program to encourage security researchers to report vulnerabilities in our software and services to us so that we can quickly address them. + +## Eligibility +To participate in the Bug Bounty Program you must: +- Not be a resident of any country that does not allow participation in these types of programs +- Be at least 14 years old and have legal capacity to agree to these terms and participate in the Bug Bounty Program +- Have permission from your employer to participate +- Not be (for the previous 12 months) an Obol Labs employee, immediate family member of an Obol employee, Obol contractor, or Obol service provider. + +## Scope +The bug bounty program applies to software and services that are built by Obol. Only submissions under the following domains are eligible for rewards: +- Charon DVT Middleware +- DV Launchpad +- Obol’s Public API +- Obol’s Smart Contracts and the contracts they depend on. +- Obol’s Public Relay + +Additionally, all vulnerabilities that require or are related to the following are out of scope: +- Social engineering +- Rate Limiting (Non-critical issues) +- Physical security +- Non-security-impacting UX issues +- Vulnerabilities or weaknesses in third party applications that integrate with Obol +- The Obol website or the Obol infrastructure in general is NOT part of this bug bounty program. + +## Rules +- Bug has not been publicly disclosed +- Vulnerabilities that have been previously submitted by another contributor or already known by the Obol development team are not eligible for rewards +- The size of the bounty payout depends on the assessment of the severity of the exploit. Please refer to the rewards section below for additional details +- Bugs must be reproducible in order for us to verify the vulnerability. Submissions with a working proof of concept is necessary +- Rewards and the validity of bugs are determined by the Obol security team and any payouts are made at their sole discretion +- Terms and conditions of the Bug Bounty program can be changed at any time at the discretion of Obol +- Details of any valid bugs may be shared with complementary protocols utilised in the Obol ecosystem in order to promote ecosystem cohesion and safety. + +## Rewards +The rewards for participating in our bug bounty program will be based on the severity and impact of the vulnerability discovered. We will evaluate each submission on a case-by-case basis, and the rewards will be at Obol’s sole discretion. + +### Low: up to $500 +A Low-level vulnerability is one that has a limited impact and can be easily fixed. Unlikely to have a meaningful impact on availability, integrity, and/or loss of funds. +- Low impact, medium likelihood +- Medium impact, low likelihood +Examples: +- Attacker can sometimes put a charon node in a state that causes it to drop one out of every one hundred attestations made by a validator + +### Medium: up to $1,000 +A Medium-level vulnerability is one that has a moderate impact and requires a more significant effort to fix. Possible to have an impact on availability, integrity, and/or loss of funds. +- High impact, low likelihood +- Medium impact, medium likelihood +- Low impact, high likelihood +Examples: +- Attacker can successfully conduct eclipse attacks on the cluster nodes with peer-ids with 4 leading zero bytes. + +### High: up to $2,500 +A High-level vulnerability is one that has a significant impact on the security of the system and requires a significant effort to fix. Likely to have impact on availability, integrity, and/or loss of funds. +- High impact, medium likelihood +- Medium impact, high likelihood +Examples: +- Attacker can successfully partition the cluster and exceeding its threshold. + +### Critical: up to $5,000 +A Critical-level vulnerability is one that has a severe impact on the security of the system and requires immediate attention to fix. Highly likely to have a material impact on availability, integrity, and/or loss of funds. +- High impact, high likelihood +Examples: +- Attacker can successfully conduct remote code execution in charon client. + +We may offer rewards in the form of cash, merchandise, or recognition. We will only award one reward per vulnerability discovered, and we reserve the right to deny a reward if we determine that the researcher has violated the terms and conditions of this policy. + +## Submission process +Please email security@obol.tech + +Your report should include the following information: +- Description of the vulnerability and its potential impact +- Steps to reproduce the vulnerability +- Proof of concept code, screenshots, or other supporting documentation +- Your name, email address, and any contact information you would like to provide. +Reports that do not include sufficient detail will not be eligible for rewards. + +## Disclosure Policy +Obol Labs will disclose the details of the vulnerability and the researcher’s identity (with their consent) only after we have remediated the vulnerability and issued a fix. Researchers must keep the details of the vulnerability confidential until Obol Labs has acknowledged and remediated the issue. + +## Legal Compliance +All participants in the bug bounty program must comply with all applicable laws, regulations, and policy terms and conditions. Obol will not be held liable for any unlawful or unauthorised activities performed by participants in the bug bounty program. + +We will not take any legal action against security researchers who discover and report security vulnerabilities in accordance with this bug bounty policy. We do, however, reserve the right to take legal action against anyone who violates the terms and conditions of this policy. + +## Non-Disclosure Agreement +All participants in the bug bounty program will be required to sign a non-disclosure agreement (NDA) before they are given access to our software and services for testing purposes. diff --git a/versioned_docs/version-v0.17.0/sec/contact.md b/versioned_docs/version-v0.17.0/sec/contact.md new file mode 100644 index 0000000000..b4925395c2 --- /dev/null +++ b/versioned_docs/version-v0.17.0/sec/contact.md @@ -0,0 +1,9 @@ +--- +description: Security Contacts +sidebar_position: 4 +--- + +# Contacts +Please email security@obol.tech to report a security incident, vulnerability, bug or inquire about Obol's security. + +Also, visit the [obol security repo](https://github.com/ObolNetwork/obol-security) for more details. diff --git a/versioned_docs/version-v0.17.0/sec/overview.md b/versioned_docs/version-v0.17.0/sec/overview.md new file mode 100644 index 0000000000..35c877fe1d --- /dev/null +++ b/versioned_docs/version-v0.17.0/sec/overview.md @@ -0,0 +1,51 @@ +--- +description: Security Overview +sidebar_position: 1 +--- + +# Overview +This page serves as an overview of the Obol Network from a security auditor’s point of view. It lists all of the projects that are intended to fall under the scope of the Obol Network project, as well as past audit reports, notable security bugs, and open security/privacy challenges in the Obol Network. You can think of this page as “a security auditor’s guide to Obol.” + +This page is updated quarterly. The last update was on 2023-03-21. + +## Table of Contents +1. [Open Challenges](#open-challenges) +2. [Core Public Goods](#core-public-goods) +3. [List of Security Audits](#list-of-security-audits) + +## Open Challenges +These are the “big picture” security challenges for Obol Network that are on our radar. + +### Validation of Contract Deployment and Deposit Data Relies Heavily on Launchpad + +The risks identified include the possibility of malicious contracts being deployed by attackers who compromise the Launchpad or an underlying dependency. + +Key concerns raised by the auditor: +1. How does the group creator know the Launchpad deployed the correct contracts? +2. How does the rest of the group know the creator deployed the contracts through the Launchpad? +The current verification process relies on the independent verification performed by each group member during and after the cluster's setup phase. However, this process may not be sufficient, as most users lack the necessary expertise to verify the source code accurately. + +The primary risk is that users may deposit with malicious withdrawal or fee recipient credentials, potentially allowing an attacker to steal the entire withdrawal amount once the cluster exits. + +The audit also mentions similar risks in validating deposit data but lacks clarity on the Obol stack's specific part that generates the deposit data/transaction. + +The auditor suggests that the mitigation for these risks would involve a more thorough and reliable verification process, although further details are not provided in the summary. + +### Social Consensus, aka “Who sends the 32 ETH?” + +Obol allows multiple operators to act as a single validator, requiring a total of 32 ETH for depositing to the beacon chain. Currently, the process relies on trust and social consensus, where the group decides on individual contributions and trusts someone to complete the deposit process correctly without misusing the funds. + +While the initial launch of Obol is limited to a small group, for a future public release, the deposit process should be simpler and less reliant on trust to ensure security and user confidence. + +## Core Public Goods +The Obol Network consists of four core public goods: + +- The Distributed Validator [Launchpad](https://docs.obol.tech/docs/dvl/intro), a [User Interface](https://goerli.launchpad.obol.tech/) for bootstrapping Distributed Validators +- [Charon](https://docs.obol.tech/docs/charon/intro), a middleware client that enables validators to run in a fault-tolerant, distributed manner +- [Obol Managers](https://docs.obol.tech/docs/sc/introducing-obol-managers), a set of solidity smart contracts for the formation of Distributed Validators +- [Obol Testnets](https://docs.obol.tech/docs/testnet), a set of on-going public incentivized testnets that enable any sized operator to test their deployment before serving for the mainnet Obol Network + +## List of Security Audits + +### 2023 +The completed audits reports are linked [here](https://github.com/ObolNetwork/obol-security/tree/main/audits). diff --git a/versioned_docs/version-v0.17.0/sec/roadmap.md b/versioned_docs/version-v0.17.0/sec/roadmap.md new file mode 100644 index 0000000000..03dbf33df2 --- /dev/null +++ b/versioned_docs/version-v0.17.0/sec/roadmap.md @@ -0,0 +1,11 @@ +--- +description: Security Roadmap +sidebar_position: 2 +--- + +# Roadmap + +## Upcoming Audits +1. Solidity smart contracts audit - Date TBD +## Penetration Test +1. Launchpad and public APIs - September 2023 diff --git a/versioned_docs/version-v0.17.0/testnet.md b/versioned_docs/version-v0.17.0/testnet.md new file mode 100644 index 0000000000..28141b3de8 --- /dev/null +++ b/versioned_docs/version-v0.17.0/testnet.md @@ -0,0 +1,117 @@ +--- +sidebar_position: 6 +description: Obol testnets roadmap +--- + +# Testnets + +Over the coming quarters, Obol Labs has and will continue to coordinate and host a number of progressively larger testnets to help harden the Charon client and iterate on the key generation tooling. + +The following is a breakdown of the intended testnet roadmap, the features that are to be completed by each testnet, and their target start date and duration. + +# Testnets + +- [x] [Dev Net 1](#devnet-1) +- [x] [Dev Net 2](#devnet-2) +- [x] [Athena Public Testnet 1](#athena-public-testnet-1) +- [x] [Bia Public Testnet 2](#bia-public-testnet-2) + +## Devnet 1 + +The first devnet aimed to have a number of trusted operators test out our earliest tutorial flows. The aim was for a single user to complete these tutorials alone, using `docker compose` to spin up 4 Charon clients and 4 different validator clients on a single machine, with a remote consensus client. The keys were created locally in Charon and activated with the existing launchpad. + +**Participants:** Obol Dev Team, Client team advisors. + +**State:** Pre-product + +**Network:** Kiln + +**Completed Date:** June 2022 + +**Duration:** 1 week + +**Goals:** + +- A single user completes a first tutorial alone, using `docker compose` to spin up 4 Charon clients on a single machine, with a remote consensus client. The keys are created locally in Charon and activated with the existing launchpad. +- Prove that the distributed validator paradigm with 4 separate VC implementations together operating as one logical validator works. +- Get the basics of monitoring in place, for the following testnet where accurate monitoring will be important due to Charon running across a network. + +## Devnet 2 + +The second devnet aimed to have a number of trusted operators test out our earliest tutorial flows **together** for the first time. + +The aim was for groups of 4 testers to complete a group onboarding tutorial, using `docker compose` to spin up 4 Charon clients and 4 different validator clients (lighthouse, teku, lodestar and vouch), each on their own machine at each operator's home or their place of choosing, running at least a kiln consensus client. + +This devnet was the first time `charon dkg` was tested with users. A core focus of this devnet was to collect network performance data. + +This was also the first time Charon was run in variable, non-virtual networks (i.e. the real internet). + +**Participants:** Obol Dev Team, Client team advisors. + +**State:** Pre-product + +**Network:** Kiln + +**Completed Date:** July 2022 + +**Duration:** 2 weeks + +**Goals:** + +- Groups of 4 testers complete a group onboarding tutorial, using `docker compose` to spin up 4 Charon clients, each on their own machine at each operator's home or their place of choosing, running at least a kiln consensus client. +- Operators avoid exposing Charon to the public internet on a static IP address through the use of Obol-hosted relay nodes. +- Users test `charon dkg`. The launchpad is not used, and this dkg is triggered by a manifest config file created locally by a single operator using the `charon create dkg` command. +- Effective collection of network performance data, to enable gathering even higher signal performance data at scale during public testnets. +- Block proposals are in place. + +## Athena Public Testnet 1 + +With tutorials for solo and group flows having been developed and refined. The goal for public testnet 1 was to get distributed validators into the hands of the wider Obol Community for the first time. The core focus of this testnet was the onboarding experience. + +The core output from this testnet was a significant number of public cluster running and public feedback collected. + +This was an unincentivized testnet and formed the basis for us to figure out a Sybil resistance mechanism. + +**Participants:** Obol Community + +**State:** Bare Minimum + +**Network:** Görli + +**Completed date:** October 2022 + +**Duration:** 2 weeks cluster setup, 8 weeks operation + +**Goals:** + +- Get distributed validators into the hands of the Obol Early Community for the first time. +- Create the first public onboarding experience and gather feedback. This is the first time we need to provide comprehensive instructions for as many platforms (Unix, Mac, Windows) as possible. +- Make deploying Ethereum validator nodes accessible using the CLI. +- Generate a backlog of bugs, feature requests, platform requests and integration requests. + +## Bia Public Testnet 2 + +This second public testnet intends to take the learning from Athena and scale the network by engaging both the wider at-home validator community and professional operators. This is the first time users are setting up DVs using the DV launchpad. + +This testnet is also important for learning the conditions Charon will be subjected to in production. A core output of this testnet is a large number of autonomous public DV clusters running and building up the Obol community with technical ambassadors. + +**Participants:** Obol Community, Ethereum staking community + +**State:** MVP + +**Network:** Görli + +**Target Completed date:** March 2023 + +**Duration:** 2 weeks cluster setup, 4-8 weeks operation + +**Goals:** + +- Engage the wider Solo and Professional Ethereum Staking Community. +- Get integration feedback. +- Build confidence in Charon after running DVs on an Ethereum testnet. +- Learn about the conditions Charon will be subjected to in production. +- Distributed Validator returns are competitive versus single validator clients. +- Make deploying Ethereum validator nodes accessible using the DV Launchpad. +- Build comprehensive guides for various profiles to spin up DVs with minimal supervision from the core team. + diff --git a/versioned_sidebars/version-v0.17.0-sidebars.json b/versioned_sidebars/version-v0.17.0-sidebars.json new file mode 100644 index 0000000000..caea0c03ba --- /dev/null +++ b/versioned_sidebars/version-v0.17.0-sidebars.json @@ -0,0 +1,8 @@ +{ + "tutorialSidebar": [ + { + "type": "autogenerated", + "dirName": "." + } + ] +} diff --git a/versions.json b/versions.json index d9276d7fd5..b2ed9be0d7 100644 --- a/versions.json +++ b/versions.json @@ -1,4 +1,5 @@ [ + "v0.17.0", "v0.16.0", "v0.15.0", "v0.14.4",