From f8dc25c1fe84e2399f4a65d8fab1be5bbed10fb7 Mon Sep 17 00:00:00 2001 From: Giulio Date: Wed, 30 Oct 2024 15:24:54 +0100 Subject: [PATCH 01/10] save --- EIPS/eip-xxxx.md | 71 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 71 insertions(+) create mode 100644 EIPS/eip-xxxx.md diff --git a/EIPS/eip-xxxx.md b/EIPS/eip-xxxx.md new file mode 100644 index 0000000000000..23d76e79c1049 --- /dev/null +++ b/EIPS/eip-xxxx.md @@ -0,0 +1,71 @@ +--- +eip: xxxx +title: eth/70 - Sharded Blocks with Extended Protocol +description: Replaces block range with a bitlist representing 1-million-block spans in the handshake, with probabilistic shard retention +author: Ahmad Bitar (@smartprogrammer93) , Giulio Rebuffo (@Giulio2002) +discussions-to: https://ethereum-magicians.org/t/eip-eth-70-available-blocks-extended-protocol-handshake/16188 +status: Draft +type: Standards Track +category: Networking +created: 2023-10-21 +requires: 7642 +--- + +## Abstract + +This EIP introduces a method enabling an Ethereum node to communicate its available block spans via a bitlist, where each bit represents a 1-million-block span. Nodes use this bitlist to signal which spans of historical data they store, enabling peers to make informed decisions about data availability when requesting blocks. This aims to improve network efficiency by providing a probabilistic snapshot of data locality across nodes. + +The proposal extends the Ethereum wire protocol (`eth`) with version `eth/70`, introducing a `blockBitlist` field in the handshake. Nodes probabilistically retain certain block spans to support data locality across the network. + +## Motivation + +With [EIP-4444](./eip-4444.md), nodes may begin pruning historical data while others continue to serve it. The current approach of connecting and requesting specific blocks to determine data availability is inefficient, consuming unnecessary bandwidth. This EIP addresses this by requiring nodes to retain at least one 1-million-block shard and retain additional shards probabilistically, enabling better-informed and targeted sync processes. + +By using a bitlist in place of a range, nodes provide a snapshot of data locality across historical epochs, supporting efficient data requests. + +## Specification + +- Advertise a new `eth` protocol capability (version) at `eth/70`. + - The existing `eth/69` protocol will continue alongside `eth/70` until sufficient adoption. +- Modify the `Status (0x00)` message for `eth/70` to add a `blockBitlist` field after the `forkid`: + - Current packet for `eth/69`: `[version: P, networkid: P, blockhash: B_32, genesis: B_32, forkid]` + - New packet for `eth/70`: `[version: P, networkid: P, blockhash: B_32, genesis: B_32, forkid, blockBitlist]`, + where `blockBitlist` is a bitlist, with each bit representing a 1-million-block span. + +- Define node behavior based on the bitlist as follows: + - **Bitlist Initialization**: Upon startup, nodes **MUST** set at least one bit in the `blockBitlist` to `on` and backfill that 1-million-block span to ensure data availability for that shard. + - **Shard Retention Probability**: Nodes **SHOULD** probabilistically retain new block spans, with an `M%` probability (e.g., 5–10%) to maintain network data locality and avoid frequent pruning of new ranges. + - **Active Shard Maintenance**: Nodes **MUST** retain all blocks from shards that are still actively being created to support synchronization. + +Upon connecting using `eth/70`, nodes exchange the `Status` message, including the `blockBitlist`. This single handshake message, which includes the bitlist, eliminates the need for additional message types. + +Nodes should maintain connections regardless of a peer’s bitlist state, except when peer slots are full. In this case, nodes may prioritize connections with peers that serve more relevant shards. + +If a node is interested in querying a specific block span, it can use the bitlist to determine which peers are more likely to have the data. If the peer they are connected to does not have the data, they can disconnect and connect to another peer, until they find what they have been looking for. + +### ENR variation + +This EIP could be implemented without a new protocol version by adding a new field to the Ethereum Node Record (ENR) to advertise the bitlist. This would allow nodes to advertise their available block spans without requiring a handshake. However, this approach would not provide the same level of assurance as the handshake, as the ENR is not authenticated. Also, it would make the discovery process much simpler, as nodes just would need to take a look at the ENR to determine if a peer has the data they are looking for. + +## Rationale + +The bitlist approach provides a flexible means to represent and retain block data with support for data locality, especially as nodes probabilistically retain certain shards. This mechanism aligns with the pruning proposed in EIP-4444, while ensuring that historical data spans remain available across the network. + +The bitlist approach is already used in the Consensus Layer for attestation subnets, making it a familiar and efficient method for representing data spans. Additionally, communicating shards this way is broadly used in many distributed systems, making it a natural fit for Ethereum. + +## Backwards Compatibility + +This EIP introduces `eth/70`, extending the handshake in a backward-incompatible manner. However, `devp2p` allows concurrent versions of the same protocol, so nodes can continue using older protocol versions (e.g., `eth/69`, `eth/68`, `eth/67`) if they lack support for `eth/70`. + +This EIP does not affect the consensus engine or require a hard fork. + +## Security Considerations + +There are some considerations: + +* The size of the network is not taken into account, so for an extended history, shard sizes may need to increase. However, this is not a major issue for Ethereum, as many nodes are active on the network at all times. +* The more blocks in the network, the more shards there will be, potentially diluting peers across multiple shards. This could make querying specific shards more challenging in the future. However, with sufficiently large shard sizes, this should not pose a significant problem. For instance, if Ethereum has 200,000,000 blocks and a shard size of 1,000,000 blocks, and a node has 32 peers, then the probability that a peer has a specific shard is around 15%, which is manageable. + +## Copyright + +This document is CC0-licensed; rights are waived through [CC0](../LICENSE.md). From da6d349b0a5b4d6eef83d538aa5a2cd1fca0228c Mon Sep 17 00:00:00 2001 From: Giulio Date: Wed, 30 Oct 2024 15:35:03 +0100 Subject: [PATCH 02/10] save --- EIPS/eip-xxxx.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/EIPS/eip-xxxx.md b/EIPS/eip-xxxx.md index 23d76e79c1049..d79bb8041bf0a 100644 --- a/EIPS/eip-xxxx.md +++ b/EIPS/eip-xxxx.md @@ -1,13 +1,13 @@ --- eip: xxxx -title: eth/70 - Sharded Blocks with Extended Protocol +title: eth/70 - Sharded Blocks Protocol description: Replaces block range with a bitlist representing 1-million-block spans in the handshake, with probabilistic shard retention author: Ahmad Bitar (@smartprogrammer93) , Giulio Rebuffo (@Giulio2002) -discussions-to: https://ethereum-magicians.org/t/eip-eth-70-available-blocks-extended-protocol-handshake/16188 +discussions-to: status: Draft type: Standards Track category: Networking -created: 2023-10-21 +created: 2024-10-30 requires: 7642 --- From 2f008c7c77c0367f9e5baee24f9e74b66edff971 Mon Sep 17 00:00:00 2001 From: Giulio Date: Wed, 30 Oct 2024 15:50:15 +0100 Subject: [PATCH 03/10] save --- EIPS/{eip-xxxx.md => eip-7793.md} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename EIPS/{eip-xxxx.md => eip-7793.md} (99%) diff --git a/EIPS/eip-xxxx.md b/EIPS/eip-7793.md similarity index 99% rename from EIPS/eip-xxxx.md rename to EIPS/eip-7793.md index d79bb8041bf0a..98ef9f9fd0cff 100644 --- a/EIPS/eip-xxxx.md +++ b/EIPS/eip-7793.md @@ -1,5 +1,5 @@ --- -eip: xxxx +eip: 7793 title: eth/70 - Sharded Blocks Protocol description: Replaces block range with a bitlist representing 1-million-block spans in the handshake, with probabilistic shard retention author: Ahmad Bitar (@smartprogrammer93) , Giulio Rebuffo (@Giulio2002) From b994f42c5d662d54c317aa017b9580ad495c45e2 Mon Sep 17 00:00:00 2001 From: Giulio Date: Wed, 30 Oct 2024 15:50:58 +0100 Subject: [PATCH 04/10] save --- EIPS/{eip-7793.md => eip-7800.md} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename EIPS/{eip-7793.md => eip-7800.md} (99%) diff --git a/EIPS/eip-7793.md b/EIPS/eip-7800.md similarity index 99% rename from EIPS/eip-7793.md rename to EIPS/eip-7800.md index 98ef9f9fd0cff..560066e07054c 100644 --- a/EIPS/eip-7793.md +++ b/EIPS/eip-7800.md @@ -1,5 +1,5 @@ --- -eip: 7793 +eip: 7800 title: eth/70 - Sharded Blocks Protocol description: Replaces block range with a bitlist representing 1-million-block spans in the handshake, with probabilistic shard retention author: Ahmad Bitar (@smartprogrammer93) , Giulio Rebuffo (@Giulio2002) From 66241657831828f7cf72566cec4e9931d63e6ecc Mon Sep 17 00:00:00 2001 From: Giulio Date: Wed, 30 Oct 2024 18:24:27 +0100 Subject: [PATCH 05/10] save --- EIPS/eip-7800.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EIPS/eip-7800.md b/EIPS/eip-7800.md index 560066e07054c..61a7d013dcbaf 100644 --- a/EIPS/eip-7800.md +++ b/EIPS/eip-7800.md @@ -3,7 +3,7 @@ eip: 7800 title: eth/70 - Sharded Blocks Protocol description: Replaces block range with a bitlist representing 1-million-block spans in the handshake, with probabilistic shard retention author: Ahmad Bitar (@smartprogrammer93) , Giulio Rebuffo (@Giulio2002) -discussions-to: +discussions-to: https://ethereum-magicians.org/t/eip-7790-eth-70-sharded-blocks-protocol/21507 status: Draft type: Standards Track category: Networking From b214ebc3bc6b16d875c2f4fca1e724b2c8772343 Mon Sep 17 00:00:00 2001 From: Giulio Date: Wed, 30 Oct 2024 18:25:44 +0100 Subject: [PATCH 06/10] save --- EIPS/eip-7800.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/EIPS/eip-7800.md b/EIPS/eip-7800.md index 61a7d013dcbaf..c6d56cf340115 100644 --- a/EIPS/eip-7800.md +++ b/EIPS/eip-7800.md @@ -63,8 +63,8 @@ This EIP does not affect the consensus engine or require a hard fork. There are some considerations: -* The size of the network is not taken into account, so for an extended history, shard sizes may need to increase. However, this is not a major issue for Ethereum, as many nodes are active on the network at all times. -* The more blocks in the network, the more shards there will be, potentially diluting peers across multiple shards. This could make querying specific shards more challenging in the future. However, with sufficiently large shard sizes, this should not pose a significant problem. For instance, if Ethereum has 200,000,000 blocks and a shard size of 1,000,000 blocks, and a node has 32 peers, then the probability that a peer has a specific shard is around 15%, which is manageable. +- The size of the network is not taken into account, so for an extended history, shard sizes may need to increase. However, this is not a major issue for Ethereum, as many nodes are active on the network at all times. +- The more blocks in the network, the more shards there will be, potentially diluting peers across multiple shards. This could make querying specific shards more challenging in the future. However, with sufficiently large shard sizes, this should not pose a significant problem. For instance, if Ethereum has 200,000,000 blocks and a shard size of 1,000,000 blocks, and a node has 32 peers, then the probability that a peer has a specific shard is around 15%, which is manageable. ## Copyright From b10e92811a3c2304cbadce88518f5f85be9e3988 Mon Sep 17 00:00:00 2001 From: Giulio Date: Wed, 30 Oct 2024 19:52:30 +0100 Subject: [PATCH 07/10] save --- EIPS/eip-7800.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EIPS/eip-7800.md b/EIPS/eip-7800.md index c6d56cf340115..c5539bdf516d8 100644 --- a/EIPS/eip-7800.md +++ b/EIPS/eip-7800.md @@ -3,7 +3,7 @@ eip: 7800 title: eth/70 - Sharded Blocks Protocol description: Replaces block range with a bitlist representing 1-million-block spans in the handshake, with probabilistic shard retention author: Ahmad Bitar (@smartprogrammer93) , Giulio Rebuffo (@Giulio2002) -discussions-to: https://ethereum-magicians.org/t/eip-7790-eth-70-sharded-blocks-protocol/21507 +discussions-to: https://ethereum-magicians.org/t/eip-7800-eth-70-sharded-blocks-protocol/21507 status: Draft type: Standards Track category: Networking From 8b4c0cb816558edeabe26916c493d9fd2c1ff903 Mon Sep 17 00:00:00 2001 From: Giulio Date: Thu, 31 Oct 2024 00:40:05 +0100 Subject: [PATCH 08/10] save --- EIPS/eip-7800.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EIPS/eip-7800.md b/EIPS/eip-7800.md index c5539bdf516d8..e5799e609c4e9 100644 --- a/EIPS/eip-7800.md +++ b/EIPS/eip-7800.md @@ -1,5 +1,5 @@ --- -eip: 7800 +eip: 7801 title: eth/70 - Sharded Blocks Protocol description: Replaces block range with a bitlist representing 1-million-block spans in the handshake, with probabilistic shard retention author: Ahmad Bitar (@smartprogrammer93) , Giulio Rebuffo (@Giulio2002) From dcd136fe494c1daffc3cfccb0b33ee284d30c813 Mon Sep 17 00:00:00 2001 From: Giulio Date: Thu, 31 Oct 2024 00:40:33 +0100 Subject: [PATCH 09/10] save --- EIPS/{eip-7800.md => eip-7801.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename EIPS/{eip-7800.md => eip-7801.md} (100%) diff --git a/EIPS/eip-7800.md b/EIPS/eip-7801.md similarity index 100% rename from EIPS/eip-7800.md rename to EIPS/eip-7801.md From be540ee005635b9a131a424d08babdc477382811 Mon Sep 17 00:00:00 2001 From: Giulio Date: Thu, 31 Oct 2024 00:40:58 +0100 Subject: [PATCH 10/10] save --- EIPS/eip-7801.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EIPS/eip-7801.md b/EIPS/eip-7801.md index e5799e609c4e9..73ae5cd1b18c5 100644 --- a/EIPS/eip-7801.md +++ b/EIPS/eip-7801.md @@ -3,7 +3,7 @@ eip: 7801 title: eth/70 - Sharded Blocks Protocol description: Replaces block range with a bitlist representing 1-million-block spans in the handshake, with probabilistic shard retention author: Ahmad Bitar (@smartprogrammer93) , Giulio Rebuffo (@Giulio2002) -discussions-to: https://ethereum-magicians.org/t/eip-7800-eth-70-sharded-blocks-protocol/21507 +discussions-to: https://ethereum-magicians.org/t/eip-7801-eth-70-sharded-blocks-protocol/21507 status: Draft type: Standards Track category: Networking