| Harmony | Prysm | Lighthouse | Artemis | Trinity | Lodestar | Nimbus | Shasper | |
|---|---|---|---|---|---|---|---|---|
| 0. Link to raw answers | Link | Link | Link | Link | Link | Link | Link | Link |
| 1. General | ||||||||
| 1.1 What is the latest version of the specification which your client currently supports? | v0.8.2 | v0.8.1 | v0.8.2 | v0.8.1 | v0.8.2 | v0.8.1 | v0.8.2 | v0.8.1 |
| 1.2 Is v0.8.2+ targeted as interop version? If not, which version do you suggest? | Yes | No | Yes | Yes | Yes | Yes | Yes | Yes |
| 1.3 Are any features or components particularly difficult to update, and on what versions are these currently? | test suite update | test suite update | merklization. | SSZ 8.1 update | No | No | No | No |
| 1.4 In terms of interop, do you have any suggestions? | time/place to work on code | - | network conformance | multi-client integration test network SEE DOC for full answer |
Start small | A best-effort for Lodestar, see if latest speed improvements make interop possible | Start with libp2p and work our way up the stack all the way to application layer / consensus | N/A |
| 1.5 What has been your primary bottleneck in development? | network stack and attestation pool | runtime bugs, libp2p breaking or poor design | Spec updates -- re-doing work or coding defensively in expectation of a spec update. | SSZ and (more recently) test format changes | Optimizing python, and taking on libp2p in python | resource allocation | Dependencies whose code needs writing | Networking |
| 1.6 What do you anticipate to be the primary bottleneck in the future? | sync process and consensus | runtime bugs, and following design changes | Perhaps spec updates during future phases, or friction trying to collaborate with other teams. | Keeping 9 clients interoperable. | Performant state transition function. Attestations workload | resource allocation | Testing / polish while community is pushing for release | Haven't been doing much |
| 2. Networking Essentials | ||||||||
| 2.1 Does your client currently implement libp2p? If not, what subset is working? | No, but we are going to integrate JVM libp2p. Currently using TCP wire protocol. | Yes | Yes | Yes, Mothra is handling our libp2p side. | Yes | Yes | Yes, libp2p Go deamon | Yes |
| 2.2 Do you make use of a libp2p daemon approach? | No | No | No | No, we have native bindings to rust libp2p. No gRPC required | No | No | Yes | No |
| 2.3 How does your client become aware of its peers? Static node list, DHT/discv5, etc. | Static node list | Kademlia now, wrapping up discv5 | Discv5 - Although we can support libp2p kademlia if needed. | We use discv5 and static nodes (via mothra) | Static node list | Static node list | Static node list | mDNS discovery and Kademlia |
| 2.4 By which process does your client establish a handshake with its peers? | unencrypted TCP | libp2p spec | discv5 session handshake, libp2p secio* | Network spec | Networking spec | req/resp hello messages | static nodes, networking spec | status message starts sync |
| 2.5 Which wire-level encryption methods does your client implement or support? Secio? TLS? Other? If your client supports multiple encryption methods, please indicate which ones. | none now. secio and noise in future. | SecIO supported, could add TLS | Secio (although we could upgrade to noise relatively easily). | Secio currently | Secio | Secio | Everything supported by the Go daemon | tcp-ws-secio-mplex-yamux |
| 2.6 Does your client conform to the specified wire protocol? If not, please provide a link to the appropriate code snippet or repo which defines these message types. | Yes, except for libp2p | Yes | Yes | Yes, GOSSIP as defined in the network spec. Working on RPC | Yes | WIP, old wire protocol with 1 stream/peer works | Yes | simple status sync |
| 2.7 Can you run a stable testnet with multiple nodes of your single client? | Yes | Yes* | Yes | Yes* | Yes | Yes | ||
| 3. Syncing | ||||||||
| 3.1 Do you use the /eth2/beacon_chain/req/beacon_blocks/1/ proposed in the Network Spec for syncing? | Not yet, using custom protocol. | Yes | Yes | Not yet | Yes | Not yet | WIP | No |
| 3.2 Do you support a full sync from genesis after the network is running for a longer period of time? | Yes | Yes | Yes | Not yet | Not yet | No | Yes | Yes |
| 3.3 Can you bootstrap syncing with a copy of sync data? | No, but will before event | No | No | No | No | No | Yes | No |
| 3.4 Do you make use of batch-requests for blocks? If so, what does your batched block request look like? | Yes, 128 chunks. | Yes | Yes, it's based on the BeaconBlocks req. We batch lookup with fixed count parameters. | Not yet | Not yet | No | Not yet, we plan to use batch requests for both | No |
| 3.5 Do you have any particular sync strategy? (Full sequential, skip-ahead, or some hybrid approach?) | Randomly requesting peers* | Full sequential sync from genesis with multiple peers | Primarily full sequential if needed, with optimizations | No | No | No | full sequential sync* | simple full sequential |
| 3.6 Do you implement any pruning mechanism? (not necessary for initial interop) | No | For attestations | Not in master, but we are prototyping in a PR (its a part of the hot/cold DB we're working on). | No | No | No | Not yet | No |
| 4. State Storage | ||||||||
| (Out of interest, no hard requirements) | ||||||||
| 4.1 Do you minimize storage? Are you supporting any of the following approaches? | No | WIP, should be ready in 1-2 weeks. | No minimization. Currently store all blocks and states for all slots | No | No | No | ||
| Immutable state data structure | No | No | No | No | Yes | No | No | No |
| Segmented/chunkfied state | No | No | No | No | No | No | No | No |
| Full copies | Yes | No | No | Yes | Yes | Yes | Yes | Yes |
| Other | No | Yes | No | No | No | No | No | No |
| 4.2 What is your storage approach like? | ||||||||
| Store on X interval | No | No | No | No | No | No | No | No |
| Store every slot | No | No | No | Yes | Yes | No | No | No |
| Store every block | No | Yes | Yes, but to be replaced | Yes | No | Yes | No | Yes |
| Other | write-buffer that is flushed when its reaches pre-defined threshold (64Mb for now) | No | WIP, hot/cold database where hot is space-inefficient but fast & cold uses a highly optimized layout which assumes no forking (relies upon finality). | No | WIP, Epoch boundaries | No | Yes, Epoch boundaries | No |
| 5. Attestation Aggregation | ||||||||
| 5.1 Do you follow the current basic interop aggregate strategy? (parse anything, but publish just minimal what you have to aggregate later). | No | Yes | Yes | Yes* | Yes | No | Yes* | No |
| 5.2 Do you support alternative (more advanced) aggregation strategies? | No | No | We can support strategies but none are built/employed. | No | No | No | No | No |
| 6. Fork Choice | ||||||||
| 6.1 Do you test fork choice? In what kind of context(s) do you test it? | No | Yes | No | Yes | No | No | No | |
| 6.2 What is your implementation type like: | ||||||||
| Unoptimized spec | Yes | Yes | Yes | Yes | Yes | No | Yes | Yes |
| Cached spec-like | No | WIP | Yes | No | No | No | No | No |
| Reduced form (shortcut 1-child nodes) | No | Yes | Yes | No | No | No | No | No |
| Stateful structure | No | No | No | No | No | Yes | No | No |
| Other | No | No | No | No | No | No | No | No |
| 7. Spec-Tests / Transition Consensus | ||||||||
| 7.1 Do you pass the following spec tests? If not, in which configuration and what tests?: | ||||||||
| Block operations | Yes | Yes | Yes | Not yet | Yes | Yes | WIP | Yes |
| Epoch processing | Yes | Yes | Yes | Not yet | Yes | Yes | WIP | Yes |
| Sanity tests | Yes | Yes | Yes | Not yet | Yes | Yes | WIP | Yes |
| BLS integration | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
| SSZ_static | Yes | Yes | Yes | Yes | Not yet | Yes | Yes | Yes |
| Shuffling | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
| 7.2 What spec version are you currently targetting for tests? | v0.8.2 | v0.8.1 | v0.8.2 | v0.8.2 | v0.8.3 | v0.8.1 | v0.8.3 | v0.8.1 |
| 8. Block Propagation (Strategy) | ||||||||
| 8.1 Do you follow the network spec: verify the proposer signature before relaying a block? | No | Yes | No, Currently no gossipsub validation - it's on the TODO list | No | Yes | WIP | No | No, import block first |
| 8.2 And if so, do you transition state completely, or just enough to know the proposer index? | N/A | Just enough to know proposer and verify signature | Full state transition | Just enough to know proposer and verify signature | Just enough to know proposer and verify signature | Transition state completely | Our block propagation is immediate and fully naive | Transition state completely |
| 8.3 Do you use any different approaches, like: | N/A | |||||||
| 8.3.1 Do you at any time randomly (or always) relay blocks without verification of the signature? | always rely on unverified blocks | No | Relay all blocks currently - planned to change | We receive the block and propogate before checking it. | No | Yes | Yes | N/A |
| 8.3.2 Which blocks do you process first (i.e. what gets priority)? | No priority | No | We request needed blocks during syncing and they are processed sequentially. We process each block from gossip prioritised by time of arrival | FIFO | No priority | FIFO | FIFO | N/A |
| 8.3.3 Do you detect spam? (i.e. drop peers with high amounts of invalid blocks) | No | No | We detect this, but do not drop peers | No | Not yet | Save block root of invalid blocks and skip them if they are received repeatedly. | No | N/A |
| 8.3.4 Is there any security check for double voting on the same block height before propagation? | No | No | No | No | No | No | No | N/A |
| 9. Attestation Propagation (strategy) | ||||||||
| 9.1 Do you follow the network spec: verify a voted-for block fully before relaying an attestation? (option 1), or do you take another approach, such as relaying first, or with certain peers only? | queued until block is known | Like spec | blindly relay | blindly relay | Verify attestation, but not if block root is known | blindly relay | blindly relay | No |
| 10. Block Proposals | ||||||||
| 10.1 Do you fill grafitti with any debug data? - What do you think of thus grafitti debug format proposal, described here? | No, lgtm | No, lgtm | Yes, but only tag the client version. | No, lgtm | No, lgtm | No, lgtm | No, lgtm | No, lgtm |
| 10.2 Do you implement the latest Validator API? | Yes | Yes, but JSON only with b64 | WIP | No, we have our own gRPC communication with our Validators | Not yet | Yes | No | Yes |
| 10.3 What is your clock syncing approach? | local machine time NTP | roughtime servers/sytem time | NTP | system clock | system clock | system clock | system clock | system clock |
| 11. Monitoring | ||||||||
| 11.1 Do you implement the proposed Metrics | No | Yes | Yes* | Yes | No | Yes | No | No |
| 11.2 Do you provide a API endpoint for: - Sync status - Current chain head (from node perspective) - A series of blocks |
No | Yes | Yes | No | No | Yes | No* | No |
| 11.3 What do you use for logging? (e.g. custom JSON, library XYZ) | log4j2 library | Logrus, JSON | JSON | Log4J, JSON/CSV | Python native | custom text, winston logger | JSON | No |
| 11.4 Provide links to any misc. API implemented by the beacon node. | N/A | Link | N/A | N/A | N/A | N/A | N/A | N/A |
| 12. Keystore | ||||||||
| 12.1 Do you use anything like the Eth1 keystore format? | Yes | Yes | No, currently store raw keys. | Yes | Not yet | Yes | Not yet. | No |
| 12.2 Do you see any problems with the latest proposed keystore format, for interop purposes specifically? | No | No | No | No | No | No | Yes* | No |
| 13. SSZ | ||||||||
| 13.1 Do you have SSZ v0.8 (hash-tree-roots with stable depth, bitlists/vectors) implemented currently? | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
| 13.2 Do you experience any particular delays with hash-tree-root? (If not already for the minimal configuration, does it apply to mainnet state sizes for your ssz implementation?) | No | No | No | No | Yes | No | No | No |
| 14. BLS | ||||||||
| 14.1 Do tests pass for version v0.8.2+ (little endian domain bytes) | Yes | No | Not sure. Will be easy to fix if not. | Yes | Yes | TBD | Yes | Passed for v0.8.1, haven't upgraded yet |
| 14.2 What BLS library do you use? (provide link) | Milagro | BLS12-381 | Apache Milagro | Milagro | Milagro | Milagro | BLS12-381 | Milagro |
| 14.3 Do you implement a BLS wrapper? (provide link) | Milagro | Custom | Milagro | Mikuli | Milagro | Custom | Milagro | N/A |
| 14.4 Do you have a benchmark of verify-aggregate bench speed of 128 participants, same message being signed. Called from client. | Yes | Yes | Yes | No | Yes | Yes | No | No |
| 15. Chain Start (reference doc) | ||||||||
| 15.1 Do you support loading: | only support loading genesis spec | |||||||
| A kickstart (plain (balance, pubkey, witdraw_credentials) tuple) | No | No | Yes* | No | Yes | Yes | No | No |
| A list of deposits, with incremental proofs (genesis spec) | No | No | WIP | Yes | No | No | No, (?) | No |
| A list of deposits, with proofs all to the same deposit root. | Yes | No | WIP | No | No | No | No, (?) | No |
| A series of deposit contract logs from an Eth 1.0 oracle, from a mock/test service | Yes | Yes | WIP | Yes | No | Yes | No | No |
| A series of deposit contract logs from a real Eth 1.0 node? | Yes | Yes | WIP | No | No | Yes | No | No |
| A genesis constructed from a (slow and long) stream of deposit log events? | No | Yes | Not yet | Yes | No | No | No | No |
| A plain prepared genesis BeaconState object from SSZ? | No | No | Yes | Yes | Yes, yaml | Yes | Yes | No |
| 15.2 For testing genesis, do you generate keys in advance? And/or in a predictable reproducible manner for debugging? | Advance | Advance | Reproducible | Reproducible | Both | Reproducible | Reproducible | Advance |
| 16. Configuration & Performance | ||||||||
| 16.1 Do you meet minimal configuration in respect to processing performance; 6 seconds with 8-slot epochs? | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
| 16.2 Are there alternative variations / easier parameters that work particularly well for your testnet(s)? | No | No | 1s slot times | Not sure | Yes, 4 slots/epoch, 4 shards | No | 3 seconds slots in 16 shards, 1000 validators network | No |
| 16.3 Do you load configuration on compile-time or run-time? | Both | Comile time presets, switch on runtime | Compile time presets, switch on runtime. Some TOML configurables | Both | Compile-time | Runtime | Compile-time | Compile-time |
| 17. Building & deploying | ||||||||
| 17.1 Do you provide a build script for the client? | Gradle | Bazel | Cargo | Yes | No | Yes | Yes | No |
| 17.2 Do you use a form of containerization? E.g. Docker? | No | Yes | Yes | Yes | No | Yes | Yes | No |
| 17.3 Have you automated testnet deployments? | No | Yes | Yes | No | No | No | Yes | No |
| 17.4 What platforms/architectures are supported? | x86_64 | x86_64 & ARM | OSX, Linux | OSX, Linux | OSX | Linux | OSX, Linux, Windows | Linux |
| 18. Conclusion | ||||||||
| 18.1 Is there anything that this questionnaire did not cover? | No | No | No | No | No | No | No | No |
| 18.2 Any miscellaneous suggestions? | No | No | No | No | No | No | No | No |
| 18.3 Are there any bottlenecks into which we may not currently have visibility? | No | Libp2p design causing problems | No | No | Phase 1 to phase 2, BLS standard | Naive implementation, slow code, brittle cli | No | No |
| 18.4 What can we do to provide you with adequate support? In other words, how can we make your life easier? | No test format changes | No test format changes | - | - | - | list optimizations, create “slow network” configuration | - | - |
| 18.5 In terms of tooling, what are we currently lacking? | Network/chain monitors | - | Interop network tests, large scale testing, differential fuzzing | Decode encrypted payloads with wireshark or tcpdump | Network monitor/stats, hive for eth2. A stable eth1 light client | No | Network monitoring tools (libp2p, Eth2) | No |