diff --git a/README.md b/README.md index 118a821..bb779b3 100644 --- a/README.md +++ b/README.md @@ -82,6 +82,19 @@ Grafana is started with the two pre-provisioned dashboards from [leanMetrics](ht > **Note:** The `--metrics` flag only affects local deployments. When using Ansible deployment mode, this flag is ignored. Metrics ports are always exposed by clients regardless of this flag. +### Aggregator Selection + +```sh +# Let the system randomly select an aggregator (default behavior) +NETWORK_DIR=local-devnet ./spin-node.sh --node all --generateGenesis + +# Manually specify which node should be the aggregator +NETWORK_DIR=local-devnet ./spin-node.sh --node all --generateGenesis --aggregator zeam_0 + +# The aggregator selection is applied automatically and the isAggregator flag +# is updated in validator-config.yaml before nodes are started +``` + ## Args 1. `NETWORK_DIR` is an env to specify the network directory. Should have a `genesis` directory with genesis config. A `data` folder will be created inside this `NETWORK_DIR` if not already there. @@ -139,6 +152,12 @@ Grafana is started with the two pre-provisioned dashboards from [leanMetrics](ht - On Ctrl+C cleanup, the metrics stack is stopped automatically Note: Client metrics endpoints are always enabled regardless of this flag. +12. `--aggregator` specifies which node should act as the aggregator (1 aggregator per subnet). + - If not provided, one node will be randomly selected as the aggregator + - If provided, the specified node will be set as the aggregator + - The aggregator selection updates the `isAggregator` flag in `validator-config.yaml` + - Example: `--aggregator zeam_0` to make zeam_0 the aggregator + - Example: Without flag, a random node will be selected automatically ### Clients supported diff --git a/TESTING_DEVNET3.md b/TESTING_DEVNET3.md new file mode 100644 index 0000000..3b948b2 --- /dev/null +++ b/TESTING_DEVNET3.md @@ -0,0 +1,159 @@ +# Testing devnet3 Configuration + +## Setup + +The local-devnet has been configured to test devnet3 features with the local zeam image. + +### Changes Made: + +1. **zeam-cmd.sh**: Updated to use `0xpartha/zeam:local` image +2. **local-devnet/validator-config.yaml**: Added commented attestation_committee_count parameter + +## devnet3 Features Being Tested + +### 1. Automatic Aggregator Selection +- One node will be randomly selected as aggregator on startup +- The `isAggregator` flag in validator-config.yaml will be automatically updated +- Only the selected aggregator will receive `--is-aggregator` flag + +### 2. Optional Attestation Committee Count +- Currently commented out (clients use hardcoded default) +- Can be enabled by uncommenting: `attestation_committee_count: 1` +- When set, all clients receive `--attestation-committee-count ` flag + +## Test Commands + +### Basic Test - Single Node (zeam_0) +```bash +# Run the test script +./test-local-zeam.sh + +# Or manually: +NETWORK_DIR=local-devnet ./spin-node.sh --node zeam_0 --generateGenesis --cleanData +``` + +### Test with Manual Aggregator Selection +```bash +# Specify zeam_0 as aggregator +NETWORK_DIR=local-devnet ./spin-node.sh --node zeam_0 --generateGenesis --cleanData --aggregator zeam_0 +``` + +### Test Multiple Nodes +```bash +# Run all nodes (zeam_0 will be randomly selected as aggregator) +NETWORK_DIR=local-devnet ./spin-node.sh --node all --generateGenesis --cleanData + +# Run specific nodes with zeam_0 as aggregator +NETWORK_DIR=local-devnet ./spin-node.sh --node "zeam_0 ream_0" --generateGenesis --cleanData --aggregator zeam_0 +``` + +### Test with Attestation Committee Count Override +```bash +# 1. Uncomment attestation_committee_count in local-devnet/genesis/validator-config.yaml +# 2. Set desired value (e.g., attestation_committee_count: 4) +# 3. Run: +NETWORK_DIR=local-devnet ./spin-node.sh --node zeam_0 --generateGenesis --cleanData +``` + +## Expected Behavior + +### Aggregator Selection +1. Script displays: "Randomly selected aggregator: zeam_0 (index 0 out of 7 nodes)" OR "Using user-specified aggregator: zeam_0" +2. Script updates validator-config.yaml: sets `isAggregator: true` for selected node +3. parse-vc.sh output shows: "Is Aggregator: true" for aggregator, "false" for others +4. zeam command includes `--is-aggregator` flag for aggregator only + +### Attestation Committee Count +**When NOT set (default):** +- parse-vc.sh does NOT display "Attestation Committee Count" +- zeam command does NOT include `--attestation-committee-count` flag +- Client uses its hardcoded default + +**When set (e.g., to 4):** +- parse-vc.sh displays: "Attestation Committee Count: 4" +- zeam command includes: `--attestation-committee-count 4` +- Client uses the specified value + +## Verification + +### Check Docker Container +```bash +# Inspect the running container +docker inspect zeam_0 + +# Check container logs +docker logs zeam_0 + +# Verify command-line arguments +docker inspect zeam_0 | grep -A20 Args +``` + +### Check Configuration +```bash +# Verify aggregator selection in validator-config.yaml +yq eval '.validators[] | select(.name == "zeam_0") | .isAggregator' local-devnet/genesis/validator-config.yaml + +# Check all aggregators +yq eval '.validators[] | select(.isAggregator == true) | .name' local-devnet/genesis/validator-config.yaml +``` + +### Monitor Node Output +```bash +# Watch zeam_0 logs in real-time +docker logs -f zeam_0 + +# Check for aggregator-related messages +docker logs zeam_0 2>&1 | grep -i aggregat +``` + +## Cleanup + +```bash +# Stop and remove zeam_0 container +docker rm -f zeam_0 + +# Stop all nodes +NETWORK_DIR=local-devnet ./spin-node.sh --node all --stop + +# Clean data directories +rm -rf local-devnet/data/* +``` + +## Troubleshooting + +### Image Not Found +If you get "image not found" error: +```bash +# Check if image exists +docker images | grep zeam + +# Pull/build the image if needed +# (build instructions depend on your zeam setup) +``` + +### Port Conflicts +If ports are already in use: +```bash +# Check what's using the port +lsof -i :8081 # zeam_0 metrics port +lsof -i :9001 # zeam_0 QUIC port + +# Kill conflicting processes or change ports in validator-config.yaml +``` + +### Genesis Generation Fails +```bash +# Ensure yq is installed +brew install yq # macOS +# or follow: https://github.com/mikefarah/yq#install + +# Check validator-config.yaml syntax +yq eval . local-devnet/genesis/validator-config.yaml +``` + +## Notes + +- The `0xpartha/zeam:local` image should have the latest devnet3 changes +- All devnet3 features (aggregator selection, optional attestation_committee_count) are enabled +- The configuration is set up for local testing with 127.0.0.1 IPs +- Hash-sig keys will be generated automatically on first run with `--generateGenesis` diff --git a/ansible-devnet/genesis/validator-config.yaml b/ansible-devnet/genesis/validator-config.yaml index 81e80f5..091275a 100644 --- a/ansible-devnet/genesis/validator-config.yaml +++ b/ansible-devnet/genesis/validator-config.yaml @@ -14,6 +14,7 @@ validators: ip: "46.224.123.223" quic: 9001 metricsPort: 9095 + isAggregator: false count: 1 # number of indices for this node - name: "ream_0" @@ -25,6 +26,7 @@ validators: ip: "77.42.27.219" quic: 9001 metricsPort: 9095 + isAggregator: false devnet: 1 count: 1 @@ -36,6 +38,7 @@ validators: ip: "46.224.123.220" quic: 9001 metricsPort: 9095 + isAggregator: false count: 1 - name: "lantern_0" @@ -47,6 +50,7 @@ validators: ip: "46.224.135.177" quic: 9001 metricsPort: 9095 + isAggregator: false count: 1 - name: "lighthouse_0" @@ -58,6 +62,7 @@ validators: ip: "46.224.135.169" quic: 9001 metricsPort: 9095 + isAggregator: false count: 1 - name: "grandine_0" @@ -66,6 +71,7 @@ validators: ip: "37.27.250.20" quic: 9001 metricsPort: 9095 + isAggregator: false count: 1 - name: "ethlambda_0" @@ -74,4 +80,5 @@ validators: ip: "78.47.44.215" quic: 9001 metricsPort: 9095 + isAggregator: false count: 1 \ No newline at end of file diff --git a/ansible/roles/ethlambda/tasks/main.yml b/ansible/roles/ethlambda/tasks/main.yml index 3fda6f8..4aa3b1a 100644 --- a/ansible/roles/ethlambda/tasks/main.yml +++ b/ansible/roles/ethlambda/tasks/main.yml @@ -38,12 +38,14 @@ loop: - enrFields.quic - metricsPort + - isAggregator when: node_name is defined -- name: Set node ports +- name: Set node ports and aggregator flag set_fact: ethlambda_quic_port: "{{ ethlambda_node_config.results[0].stdout }}" ethlambda_metrics_port: "{{ ethlambda_node_config.results[1].stdout }}" + ethlambda_is_aggregator: "{{ 'true' if (ethlambda_node_config.results[2].stdout | default('') | trim) == 'true' else 'false' }}" when: ethlambda_node_config is defined - name: Ensure node key file exists @@ -96,6 +98,7 @@ --node-key /config/{{ node_name }}.key --metrics-address 0.0.0.0 --metrics-port {{ ethlambda_metrics_port }} + {{ '--is-aggregator' if (ethlambda_is_aggregator | default('false')) == 'true' else '' }} register: ethlambda_container changed_when: ethlambda_container.rc == 0 when: deployment_mode == 'docker' diff --git a/ansible/roles/grandine/tasks/main.yml b/ansible/roles/grandine/tasks/main.yml index 7679d50..ff1f472 100644 --- a/ansible/roles/grandine/tasks/main.yml +++ b/ansible/roles/grandine/tasks/main.yml @@ -39,13 +39,15 @@ - enrFields.quic - metricsPort - privkey + - isAggregator when: node_name is defined -- name: Set node ports +- name: Set node ports and aggregator flag set_fact: grandine_quic_port: "{{ grandine_node_config.results[0].stdout }}" grandine_metrics_port: "{{ grandine_node_config.results[1].stdout }}" grandine_privkey: "{{ grandine_node_config.results[2].stdout }}" + grandine_is_aggregator: "{{ 'true' if (grandine_node_config.results[3].stdout | default('') | trim) == 'true' else 'false' }}" when: grandine_node_config is defined - name: Ensure node key file exists @@ -103,6 +105,7 @@ --metrics --http-address 0.0.0.0 --http-port {{ grandine_metrics_port }} + {{ '--is-aggregator' if (grandine_is_aggregator | default('false')) == 'true' else '' }} register: grandine_container changed_when: grandine_container.rc == 0 when: deployment_mode == 'docker' diff --git a/ansible/roles/lantern/defaults/main.yml b/ansible/roles/lantern/defaults/main.yml index 768e9ad..abd2e73 100644 --- a/ansible/roles/lantern/defaults/main.yml +++ b/ansible/roles/lantern/defaults/main.yml @@ -3,6 +3,7 @@ # Note: These are fallback defaults. Actual values are extracted from client-cmds/lantern-cmd.sh # in the tasks/main.yml file. These defaults are used if extraction fails. -lantern_docker_image: "piertwo/lantern:v0.0.2" +# Platform-specific - will be set by tasks based on architecture +lantern_docker_image: "piertwo/lantern:v0.0.3-test-amd64" deployment_mode: docker # docker or binary diff --git a/ansible/roles/lantern/tasks/main.yml b/ansible/roles/lantern/tasks/main.yml index 8f18357..c58ed06 100644 --- a/ansible/roles/lantern/tasks/main.yml +++ b/ansible/roles/lantern/tasks/main.yml @@ -2,16 +2,14 @@ # Lantern role: Deploy and manage Lantern nodes # Converts client-cmds/lantern-cmd.sh logic to Ansible tasks -- name: Extract docker image from client-cmd.sh - shell: | - # Extract the docker image from LANTERN_IMAGE line - # playbook_dir points to ansible/playbooks, go up two levels to reach project root - project_root="$(cd '{{ playbook_dir }}/../..' && pwd)" - grep -E '^LANTERN_IMAGE=' "$project_root/client-cmds/lantern-cmd.sh" | head -1 | sed -E 's/.*LANTERN_IMAGE="([^"]+)".*/\1/' - register: lantern_docker_image_raw +- name: Detect target host architecture + command: uname -m + register: target_arch changed_when: false - delegate_to: localhost - run_once: true + +- name: Set lantern docker image based on architecture + set_fact: + lantern_docker_image: "{{ 'piertwo/lantern:v0.0.3-test-amd64' if target_arch.stdout == 'x86_64' else 'piertwo/lantern:v0.0.3-test-arm64' }}" - name: Extract deployment mode from client-cmd.sh shell: | @@ -23,9 +21,8 @@ delegate_to: localhost run_once: true -- name: Set docker image and deployment mode from client-cmd.sh +- name: Set deployment mode from client-cmd.sh set_fact: - lantern_docker_image: "{{ lantern_docker_image_raw.stdout | trim | default('piertwo/lantern:v0.0.1') }}" deployment_mode: "{{ lantern_deployment_mode_raw.stdout | trim | default('docker') }}" - name: Extract node configuration from validator-config.yaml @@ -38,13 +35,15 @@ - enrFields.quic - metricsPort - privkey + - isAggregator when: node_name is defined -- name: Set node ports +- name: Set node ports and aggregator flag set_fact: lantern_quic_port: "{{ lantern_node_config.results[0].stdout }}" lantern_metrics_port: "{{ lantern_node_config.results[1].stdout }}" lantern_privkey: "{{ lantern_node_config.results[2].stdout }}" + lantern_is_aggregator: "{{ 'true' if (lantern_node_config.results[3].stdout | default('') | trim) == 'true' else 'false' }}" when: lantern_node_config is defined - name: Ensure node key file exists @@ -97,6 +96,7 @@ --metrics-port {{ lantern_metrics_port }} --http-port 5055 --hash-sig-key-dir /config/hash-sig-keys + {{ '--is-aggregator' if (lantern_is_aggregator | default('false')) == 'true' else '' }} register: lantern_container changed_when: lantern_container.rc == 0 when: deployment_mode == 'docker' diff --git a/ansible/roles/lighthouse/tasks/main.yml b/ansible/roles/lighthouse/tasks/main.yml index 7385715..f932153 100644 --- a/ansible/roles/lighthouse/tasks/main.yml +++ b/ansible/roles/lighthouse/tasks/main.yml @@ -38,13 +38,15 @@ - enrFields.quic - metricsPort - privkey + - isAggregator when: node_name is defined -- name: Set node ports +- name: Set node ports and aggregator flag set_fact: lighthouse_quic_port: "{{ lighthouse_node_config.results[0].stdout }}" lighthouse_metrics_port: "{{ lighthouse_node_config.results[1].stdout }}" lighthouse_privkey: "{{ lighthouse_node_config.results[2].stdout }}" + lighthouse_is_aggregator: "{{ 'true' if (lighthouse_node_config.results[3].stdout | default('') | trim) == 'true' else 'false' }}" when: lighthouse_node_config is defined - name: Ensure node key file exists @@ -101,6 +103,7 @@ --metrics --metrics-address 0.0.0.0 --metrics-port {{ lighthouse_metrics_port }} + {{ '--is-aggregator' if (lighthouse_is_aggregator | default('false')) == 'true' else '' }} register: lighthouse_container changed_when: lighthouse_container.rc == 0 when: deployment_mode == 'docker' diff --git a/ansible/roles/qlean/tasks/main.yml b/ansible/roles/qlean/tasks/main.yml index 07579b7..0f49bcb 100644 --- a/ansible/roles/qlean/tasks/main.yml +++ b/ansible/roles/qlean/tasks/main.yml @@ -38,13 +38,15 @@ - enrFields.quic - metricsPort - privkey + - isAggregator when: node_name is defined -- name: Set node ports +- name: Set node ports and aggregator flag set_fact: qlean_quic_port: "{{ qlean_node_config.results[0].stdout }}" qlean_metrics_port: "{{ qlean_node_config.results[1].stdout }}" qlean_privkey: "{{ qlean_node_config.results[2].stdout }}" + qlean_is_aggregator: "{{ 'true' if (qlean_node_config.results[3].stdout | default('') | trim) == 'true' else 'false' }}" when: qlean_node_config is defined - name: Extract validator index from validators.yaml @@ -110,6 +112,7 @@ --node-key /config/{{ node_name }}.key --listen-addr /ip4/0.0.0.0/udp/{{ qlean_quic_port }}/quic-v1 --prometheus-port {{ qlean_metrics_port }} + {{ '--is-aggregator' if (qlean_is_aggregator | default('false')) == 'true' else '' }} -ldebug register: qlean_container changed_when: qlean_container.rc == 0 diff --git a/ansible/roles/ream/tasks/main.yml b/ansible/roles/ream/tasks/main.yml index 1810958..eb26f34 100644 --- a/ansible/roles/ream/tasks/main.yml +++ b/ansible/roles/ream/tasks/main.yml @@ -38,13 +38,15 @@ - enrFields.quic - metricsPort - privkey + - isAggregator when: node_name is defined -- name: Set node ports +- name: Set node ports and aggregator flag set_fact: ream_quic_port: "{{ ream_node_config.results[0].stdout }}" ream_metrics_port: "{{ ream_node_config.results[1].stdout }}" ream_privkey: "{{ ream_node_config.results[2].stdout }}" + ream_is_aggregator: "{{ 'true' if (ream_node_config.results[3].stdout | default('') | trim) == 'true' else 'false' }}" when: ream_node_config is defined - name: Ensure node key file exists @@ -97,6 +99,7 @@ --metrics-address 0.0.0.0 --metrics-port {{ ream_metrics_port }} --http-address 0.0.0.0 + {{ '--is-aggregator' if (ream_is_aggregator | default('false')) == 'true' else '' }} register: ream_container changed_when: ream_container.rc == 0 when: deployment_mode == 'docker' diff --git a/ansible/roles/zeam/defaults/main.yml b/ansible/roles/zeam/defaults/main.yml index f9e66ac..674b4d0 100644 --- a/ansible/roles/zeam/defaults/main.yml +++ b/ansible/roles/zeam/defaults/main.yml @@ -3,7 +3,7 @@ # Note: These are fallback defaults. Actual values are extracted from client-cmds/zeam-cmd.sh # in the tasks/main.yml file. These defaults are used if extraction fails. -zeam_docker_image: "blockblaz/zeam:devnet2" +zeam_docker_image: "0xpartha/zeam:devnet3" zeam_binary_path: "{{ playbook_dir }}/../zig-out/bin/zeam" deployment_mode: docker # docker or binary diff --git a/ansible/roles/zeam/tasks/main.yml b/ansible/roles/zeam/tasks/main.yml index 74c0473..66f2c62 100644 --- a/ansible/roles/zeam/tasks/main.yml +++ b/ansible/roles/zeam/tasks/main.yml @@ -45,11 +45,13 @@ loop: - metricsPort - privkey + - isAggregator when: node_name is defined -- name: Set node metrics port +- name: Set node metrics port and aggregator flag set_fact: zeam_metrics_port: "{{ node_config.results[0].stdout }}" + zeam_is_aggregator: "{{ 'true' if (node_config.results[2].stdout | default('') | trim) == 'true' else 'false' }}" when: node_config is defined - name: Ensure node key file exists @@ -106,6 +108,7 @@ --node-key /config/{{ node_name }}.key --metrics_enable --api-port {{ zeam_metrics_port }} + {{ '--is-aggregator' if (zeam_is_aggregator | default('false')) == 'true' else '' }} register: zeam_container_result changed_when: zeam_container_result.rc == 0 diff --git a/client-cmds/ethlambda-cmd.sh b/client-cmds/ethlambda-cmd.sh index 50bd63c..d02954e 100644 --- a/client-cmds/ethlambda-cmd.sh +++ b/client-cmds/ethlambda-cmd.sh @@ -4,6 +4,18 @@ binary_path="$scriptDir/../ethlambda/target/release/ethlambda" +# Set aggregator flag based on isAggregator value +aggregator_flag="" +if [ "$isAggregator" == "true" ]; then + aggregator_flag="--is-aggregator" +fi + +# Set attestation committee count flag if explicitly configured +attestation_committee_flag="" +if [ -n "$attestationCommitteeCount" ]; then + attestation_committee_flag="--attestation-committee-count $attestationCommitteeCount" +fi + # Command when running as binary node_binary="$binary_path \ --custom-network-config-dir $configDir \ @@ -11,15 +23,19 @@ node_binary="$binary_path \ --node-id $item \ --node-key $configDir/$item.key \ --metrics-address 0.0.0.0 \ - --metrics-port $metricsPort" + --metrics-port $metricsPort \ + $attestation_committee_flag \ + $aggregator_flag" # Command when running as docker container -node_docker="ghcr.io/lambdaclass/ethlambda:devnet2 \ +node_docker="ghcr.io/lambdaclass/ethlambda:devnet3 \ --custom-network-config-dir /config \ --gossipsub-port $quicPort \ --node-id $item \ --node-key /config/$item.key \ --metrics-address 0.0.0.0 \ - --metrics-port $metricsPort" + --metrics-port $metricsPort \ + $attestation_committee_flag \ + $aggregator_flag" node_setup="docker" diff --git a/client-cmds/grandine-cmd.sh b/client-cmds/grandine-cmd.sh index cd92472..0ab2232 100644 --- a/client-cmds/grandine-cmd.sh +++ b/client-cmds/grandine-cmd.sh @@ -1,5 +1,17 @@ #!/bin/bash +# Set aggregator flag based on isAggregator value +aggregator_flag="" +if [ "$isAggregator" == "true" ]; then + aggregator_flag="--is-aggregator" +fi + +# Set attestation committee count flag if explicitly configured +attestation_committee_flag="" +if [ -n "$attestationCommitteeCount" ]; then + attestation_committee_flag="--attestation-committee-count $attestationCommitteeCount" +fi + node_binary="$grandine_bin \ --genesis $configDir/config.yaml \ --validator-registry-path $configDir/validators.yaml \ @@ -11,7 +23,9 @@ node_binary="$grandine_bin \ --metrics \ --http-address 0.0.0.0 \ --http-port $metricsPort \ - --hash-sig-key-dir $configDir/hash-sig-keys" + --hash-sig-key-dir $configDir/hash-sig-keys \ + $attestation_committee_flag \ + $aggregator_flag" node_docker="sifrai/lean:devnet-2 \ --genesis /config/config.yaml \ @@ -24,7 +38,9 @@ node_docker="sifrai/lean:devnet-2 \ --metrics \ --http-address 0.0.0.0 \ --http-port $metricsPort \ - --hash-sig-key-dir /config/hash-sig-keys" + --hash-sig-key-dir /config/hash-sig-keys \ + $attestation_committee_flag \ + $aggregator_flag" # choose either binary or docker node_setup="docker" diff --git a/client-cmds/lantern-cmd.sh b/client-cmds/lantern-cmd.sh index b918355..8d8b5a2 100755 --- a/client-cmds/lantern-cmd.sh +++ b/client-cmds/lantern-cmd.sh @@ -1,13 +1,38 @@ #!/bin/bash #-----------------------lantern setup---------------------- -LANTERN_IMAGE="piertwo/lantern:v0.0.2" +# Platform-specific lantern image +ARCH=$(uname -m) +if [ "$ARCH" = "x86_64" ]; then + LANTERN_IMAGE="piertwo/lantern:v0.0.3-test-amd64" +elif [ "$ARCH" = "aarch64" ] || [ "$ARCH" = "arm64" ]; then + LANTERN_IMAGE="piertwo/lantern:v0.0.3-test-arm64" +else + echo "Unsupported architecture: $ARCH" + exit 1 +fi devnet_flag="" if [ -n "$devnet" ]; then devnet_flag="--devnet $devnet" fi +# Lantern does not support --is-aggregator flag (unlike zeam) +# The aggregator role is determined by the validator-config.yaml isAggregator field +# which lantern reads directly from the config file +aggregator_flag="" + +# Set attestation committee count flag if explicitly configured +attestation_committee_flag="" +if [ -n "$attestationCommitteeCount" ]; then + attestation_committee_flag="--attestation-committee-count $attestationCommitteeCount" +fi + +# Set HTTP port (default to 5055 if not specified in validator-config.yaml) +if [ -z "$httpPort" ]; then + httpPort="5055" +fi + # Lantern's repo: https://github.com/Pier-Two/lantern node_binary="$scriptDir/lantern/build/lantern_cli \ --data-dir $dataDir/$item \ @@ -20,9 +45,10 @@ node_binary="$scriptDir/lantern/build/lantern_cli \ --node-id $item --node-key-path $configDir/$privKeyPath \ --listen-address /ip4/0.0.0.0/udp/$quicPort/quic-v1 \ --metrics-port $metricsPort \ - --http-port 5055 \ - --log-level debug \ - --hash-sig-key-dir $configDir/hash-sig-keys" + --http-port $httpPort \ + --log-level info \ + --hash-sig-key-dir $configDir/hash-sig-keys \ + $attestation_committee_flag" node_docker="$LANTERN_IMAGE --data-dir /data \ --genesis-config /config/config.yaml \ @@ -34,9 +60,10 @@ node_docker="$LANTERN_IMAGE --data-dir /data \ --node-id $item --node-key-path /config/$privKeyPath \ --listen-address /ip4/0.0.0.0/udp/$quicPort/quic-v1 \ --metrics-port $metricsPort \ - --http-port 5055 \ - --log-level debug \ - --hash-sig-key-dir /config/hash-sig-keys" + --http-port $httpPort \ + --log-level info \ + --hash-sig-key-dir /config/hash-sig-keys \ + $attestation_committee_flag" # choose either binary or docker node_setup="docker" diff --git a/client-cmds/lighthouse-cmd.sh b/client-cmds/lighthouse-cmd.sh index 1e129c2..0f94f66 100644 --- a/client-cmds/lighthouse-cmd.sh +++ b/client-cmds/lighthouse-cmd.sh @@ -3,6 +3,18 @@ # Metrics enabled by default metrics_flag="--metrics" +# Set aggregator flag based on isAggregator value +aggregator_flag="" +if [ "$isAggregator" == "true" ]; then + aggregator_flag="--is-aggregator" +fi + +# Set attestation committee count flag if explicitly configured +attestation_committee_flag="" +if [ -n "$attestationCommitteeCount" ]; then + attestation_committee_flag="--attestation-committee-count $attestationCommitteeCount" +fi + node_binary="$lighthouse_bin lean_node \ --datadir \"$dataDir/$item\" \ --config \"$configDir/config.yaml\" \ @@ -14,7 +26,9 @@ node_binary="$lighthouse_bin lean_node \ --socket-port $quicPort\ $metrics_flag \ --metrics-address 0.0.0.0 \ - --metrics-port $metricsPort" + --metrics-port $metricsPort \ + $attestation_committee_flag \ + $aggregator_flag" node_docker="hopinheimer/lighthouse:latest lighthouse lean_node \ --datadir /data \ @@ -27,6 +41,8 @@ node_docker="hopinheimer/lighthouse:latest lighthouse lean_node \ --socket-port $quicPort\ $metrics_flag \ --metrics-address 0.0.0.0 \ - --metrics-port $metricsPort" + --metrics-port $metricsPort \ + $attestation_committee_flag \ + $aggregator_flag" node_setup="docker" diff --git a/client-cmds/qlean-cmd.sh b/client-cmds/qlean-cmd.sh index 28de40b..10812f6 100644 --- a/client-cmds/qlean-cmd.sh +++ b/client-cmds/qlean-cmd.sh @@ -3,6 +3,19 @@ #-----------------------qlean setup---------------------- # expects "qlean" submodule or symlink inside "lean-quickstart" root directory # https://github.com/qdrvm/qlean-mini + +# Set aggregator flag based on isAggregator value +aggregator_flag="" +if [ "$isAggregator" == "true" ]; then + aggregator_flag="--is-aggregator" +fi + +# Set attestation committee count flag if explicitly configured +attestation_committee_flag="" +if [ -n "$attestationCommitteeCount" ]; then + attestation_committee_flag="--attestation-committee-count $attestationCommitteeCount" +fi + node_binary="$scriptDir/qlean/build/src/executable/qlean \ --modules-dir $scriptDir/qlean/build/src/modules \ --genesis $configDir/config.yaml \ @@ -15,6 +28,8 @@ node_binary="$scriptDir/qlean/build/src/executable/qlean \ --node-id $item --node-key $configDir/$privKeyPath \ --listen-addr /ip4/0.0.0.0/udp/$quicPort/quic-v1 \ --prometheus-port $metricsPort \ + $attestation_committee_flag \ + $aggregator_flag \ -ldebug \ -ltrace" @@ -29,6 +44,8 @@ node_docker="qdrvm/qlean-mini:devnet-2 \ --node-id $item --node-key /config/$privKeyPath \ --listen-addr /ip4/0.0.0.0/udp/$quicPort/quic-v1 \ --prometheus-port $metricsPort \ + $attestation_committee_flag \ + $aggregator_flag \ -ldebug \ -ltrace" diff --git a/client-cmds/ream-cmd.sh b/client-cmds/ream-cmd.sh index 9985c92..3ddd55c 100755 --- a/client-cmds/ream-cmd.sh +++ b/client-cmds/ream-cmd.sh @@ -4,6 +4,18 @@ # Metrics enabled by default metrics_flag="--metrics" +# Set aggregator flag based on isAggregator value +aggregator_flag="" +if [ "$isAggregator" == "true" ]; then + aggregator_flag="--is-aggregator" +fi + +# Set attestation committee count flag if explicitly configured +attestation_committee_flag="" +if [ -n "$attestationCommitteeCount" ]; then + attestation_committee_flag="--attestation-committee-count $attestationCommitteeCount" +fi + # modify the path to the ream binary as per your system node_binary="$scriptDir/../ream/target/release/ream --data-dir $dataDir/$item \ lean_node \ @@ -15,7 +27,9 @@ node_binary="$scriptDir/../ream/target/release/ream --data-dir $dataDir/$item \ $metrics_flag \ --metrics-address 0.0.0.0 \ --metrics-port $metricsPort \ - --http-address 0.0.0.0" + --http-address 0.0.0.0 \ + $attestation_committee_flag \ + $aggregator_flag" node_docker="ghcr.io/reamlabs/ream:latest-devnet2 --data-dir /data \ lean_node \ @@ -27,7 +41,9 @@ node_docker="ghcr.io/reamlabs/ream:latest-devnet2 --data-dir /data \ $metrics_flag \ --metrics-address 0.0.0.0 \ --metrics-port $metricsPort \ - --http-address 0.0.0.0" + --http-address 0.0.0.0 \ + $attestation_committee_flag \ + $aggregator_flag" # choose either binary or docker node_setup="docker" diff --git a/client-cmds/zeam-cmd.sh b/client-cmds/zeam-cmd.sh index 4b56d7a..f19cdfb 100644 --- a/client-cmds/zeam-cmd.sh +++ b/client-cmds/zeam-cmd.sh @@ -6,21 +6,37 @@ # Metrics enabled by default metrics_flag="--metrics_enable" +# Set aggregator flag based on isAggregator value +aggregator_flag="" +if [ "$isAggregator" == "true" ]; then + aggregator_flag="--is-aggregator" +fi + +# Set attestation committee count flag if explicitly configured +attestation_committee_flag="" +if [ -n "$attestationCommitteeCount" ]; then + attestation_committee_flag="--attestation-committee-count $attestationCommitteeCount" +fi + node_binary="$scriptDir/../zig-out/bin/zeam node \ --custom_genesis $configDir \ --validator_config $validatorConfig \ --data-dir $dataDir/$item \ --node-id $item --node-key $configDir/$item.key \ $metrics_flag \ - --api-port $metricsPort" + --api-port $metricsPort \ + $attestation_committee_flag \ + $aggregator_flag" -node_docker="--security-opt seccomp=unconfined blockblaz/zeam:devnet2 node \ +node_docker="--security-opt seccomp=unconfined 0xpartha/zeam:devnet3 node \ --custom_genesis /config \ --validator_config $validatorConfig \ --data-dir /data \ --node-id $item --node-key /config/$item.key \ $metrics_flag \ - --api-port $metricsPort" + --api-port $metricsPort \ + $attestation_committee_flag \ + $aggregator_flag" # choose either binary or docker node_setup="docker" \ No newline at end of file diff --git a/local-devnet/genesis/validator-config.yaml b/local-devnet/genesis/validator-config.yaml index 7f99d48..597b36c 100644 --- a/local-devnet/genesis/validator-config.yaml +++ b/local-devnet/genesis/validator-config.yaml @@ -4,6 +4,8 @@ deployment_mode: local config: activeEpoch: 18 keyType: "hash-sig" + # Optional: Override client's hardcoded attestation_committee_count + # attestation_committee_count: 1 validators: - name: "zeam_0" # node id 7d0904dc6d8d7130e0e68d5d3175d0c3cf470f8725f67bd8320882f5b9753cc0 @@ -14,67 +16,72 @@ validators: ip: "127.0.0.1" quic: 9001 metricsPort: 8081 + isAggregator: false count: 1 # number of indices for this node - - - name: "ream_0" + - name: "zeam_1" # node id bc531fc1a99a896acb45603f28a32f81ae607480af46435009de4609370cb7bb # peer id 16Uiu2HAmPQhkD6Zg5Co2ee8ShshkiY4tDePKFARPpCS2oKSLj1E1 privkey: "af27950128b49cda7e7bc9fcb7b0270f7a3945aa7543326f3bfdbd57d2a97a32" enrFields: - #verify /ip4/127.0.0.1/udp/9001/quic-v1/p2p/16Uiu2HAmPQhkD6Zg5Co2ee8ShshkiY4tDePKFARPpCS2oKSLj1E1 + #verify /ip4/127.0.0.1/udp/9002/quic-v1/p2p/16Uiu2HAmPQhkD6Zg5Co2ee8ShshkiY4tDePKFARPpCS2oKSLj1E1 ip: "127.0.0.1" quic: 9002 metricsPort: 8082 - devnet: 1 + isAggregator: true count: 1 - - - name: "qlean_0" - # TODO: add a third entry in nodes.yaml corresponding to this - privkey: "c2bbdac5e876b3e9d4b8b6b8c2bbdac5e876b3e9d4b8b6b8c2bbdac5e876b3e9" - enrFields: - #verify /ip4/127.0.0.1/udp/9001/quic-v1/p2p/16Uiu2HAmPQhkD6Zg5Co2ee8ShshkiY4tDePKFARPpCS2oKSLj1E1 - ip: "127.0.0.1" - quic: 9003 - metricsPort: 8083 - count: 1 - - - name: "lantern_0" - # node id a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2 - # peer id 16Uiu2HAm7TYVs6qvDKnrovd9m4vvRikc4HPXm1WyLumKSe5fHxBv + # - name: "qlean_0" + # # TODO: add a third entry in nodes.yaml corresponding to this + # privkey: "c2bbdac5e876b3e9d4b8b6b8c2bbdac5e876b3e9d4b8b6b8c2bbdac5e876b3e9" + # enrFields: + # #verify /ip4/127.0.0.1/udp/9001/quic-v1/p2p/16Uiu2HAmPQhkD6Zg5Co2ee8ShshkiY4tDePKFARPpCS2oKSLj1E1 + # ip: "127.0.0.1" + # quic: 9003 + # metricsPort: 8083 + # isAggregator: false + # count: 1 + - name: "ethlambda_0" privkey: "d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2c3d4e5" - # verify /ip4/127.0.0.1/udp/9004/quic-v1/p2p/16Uiu2HAm7TYVs6qvDKnrovd9m4vvRikc4HPXm1WyLumKSe5fHxBv enrFields: ip: "127.0.0.1" quic: 9004 metricsPort: 8084 + isAggregator: false count: 1 - - - name: "lighthouse_0" - # node id a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2 - # peer id 16Uiu2HAm7TYVs6qvDKnrovd9m4vvRikc4HPXm1WyLumKSe5fHxBv + - name: "ethlambda_1" privkey: "4fd22cf461fbeae4947a3fdaef8d533fc7fd1ef1ce4cd98e993210c18234df3f" - # verify /ip4/127.0.0.1/udp/9004/quic-v1/p2p/16Uiu2HAm7TYVs6qvDKnrovd9m4vvRikc4HPXm1WyLumKSe5fHxBv enrFields: ip: "127.0.0.1" quic: 9005 metricsPort: 8085 + isAggregator: false count: 1 - - - name: "grandine_0" - privkey: "64a7f5ab53907966374ca23af36392910af682eec82c12e3abbb6c2ccdf39a72" - enrFields: - ip: "127.0.0.1" - quic: 9006 - metricsPort: 8086 - count: 1 - - - name: "ethlambda_0" - # node id a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2 - # peer id 16Uiu2HAmPV5jU62WtmDkCEmfq1jzbBDkGbHNsDN78gJyvmv2TuC5 - privkey: "299550529a79bc2dce003747c52fb0639465c893e00b0440ac66144d625e066a" - # verify /ip4/127.0.0.1/udp/9007/quic-v1/p2p/16Uiu2HAmPV5jU62WtmDkCEmfq1jzbBDkGbHNsDN78gJyvmv2TuC5 - enrFields: - ip: "127.0.0.1" - quic: 9007 - metricsPort: 8087 - count: 1 +# - name: "lighthouse_0" +# # node id a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2 +# # peer id 16Uiu2HAm7TYVs6qvDKnrovd9m4vvRikc4HPXm1WyLumKSe5fHxBv +# privkey: "4fd22cf461fbeae4947a3fdaef8d533fc7fd1ef1ce4cd98e993210c18234df3f" +# # verify /ip4/127.0.0.1/udp/9004/quic-v1/p2p/16Uiu2HAm7TYVs6qvDKnrovd9m4vvRikc4HPXm1WyLumKSe5fHxBv +# enrFields: +# ip: "127.0.0.1" +# quic: 9005 +# metricsPort: 8085 +# isAggregator: false +# count: 1 +# - name: "grandine_0" +# privkey: "64a7f5ab53907966374ca23af36392910af682eec82c12e3abbb6c2ccdf39a72" +# enrFields: +# ip: "127.0.0.1" +# quic: 9006 +# metricsPort: 8086 +# isAggregator: false +# count: 1 +# - name: "ethlambda_0" +# # node id a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2 +# # peer id 16Uiu2HAmPV5jU62WtmDkCEmfq1jzbBDkGbHNsDN78gJyvmv2TuC5 +# privkey: "299550529a79bc2dce003747c52fb0639465c893e00b0440ac66144d625e066a" +# # verify /ip4/127.0.0.1/udp/9007/quic-v1/p2p/16Uiu2HAmPV5jU62WtmDkCEmfq1jzbBDkGbHNsDN78gJyvmv2TuC5 +# enrFields: +# ip: "127.0.0.1" +# quic: 9007 +# metricsPort: 8087 +# isAggregator: false +# count: 1 diff --git a/parse-env.sh b/parse-env.sh index 91b2a6a..112feae 100755 --- a/parse-env.sh +++ b/parse-env.sh @@ -80,6 +80,11 @@ while [[ $# -gt 0 ]]; do stopNodes=true shift ;; + --aggregator) + aggregatorNode="$2" + shift # past argument + shift # past value + ;; *) # unknown option shift # past argument ;; @@ -110,3 +115,4 @@ echo "cleanData = $cleanData" echo "popupTerminal = $popupTerminal" echo "dockerTag = ${dockerTag:-latest}" echo "enableMetrics = $enableMetrics" +echo "aggregatorNode = ${aggregatorNode:-}" \ No newline at end of file diff --git a/parse-vc.sh b/parse-vc.sh index 8b50e82..abfa24c 100644 --- a/parse-vc.sh +++ b/parse-vc.sh @@ -45,12 +45,30 @@ if [ -z "$metricsPort" ] || [ "$metricsPort" == "null" ]; then exit 1 fi +# Automatically extract HTTP port using yq (optional - only some clients use it) +httpPort=$(yq eval ".validators[] | select(.name == \"$item\") | .httpPort" "$validator_config_file") +if [ -z "$httpPort" ] || [ "$httpPort" == "null" ]; then + httpPort="" +fi + # Automatically extract devnet using yq (optional - only ream uses it) devnet=$(yq eval ".validators[] | select(.name == \"$item\") | .devnet" "$validator_config_file") if [ -z "$devnet" ] || [ "$devnet" == "null" ]; then devnet="" fi +# Automatically extract isAggregator flag using yq (defaults to false if not set) +isAggregator=$(yq eval ".validators[] | select(.name == \"$item\") | .isAggregator // false" "$validator_config_file") +if [ -z "$isAggregator" ] || [ "$isAggregator" == "null" ]; then + isAggregator="false" +fi + +# Extract attestation_committee_count from config section (optional - only if explicitly set) +attestationCommitteeCount=$(yq eval ".config.attestation_committee_count" "$validator_config_file") +if [ -z "$attestationCommitteeCount" ] || [ "$attestationCommitteeCount" == "null" ]; then + attestationCommitteeCount="" +fi + # Automatically extract private key using yq privKey=$(yq eval ".validators[] | select(.name == \"$item\") | .privkey" "$validator_config_file") @@ -99,10 +117,18 @@ if [ "$keyType" == "hash-sig" ] && [ "$hashSigKeyIndex" != "null" ] && [ -n "$ha echo "Hash-Sig Key Index: $hashSigKeyIndex" echo "Hash-Sig Public Key: $hashSigPkPath" echo "Hash-Sig Secret Key: $hashSigSkPath" + echo "Is Aggregator: $isAggregator" + if [ -n "$attestationCommitteeCount" ]; then + echo "Attestation Committee Count: $attestationCommitteeCount" + fi else echo "Node: $item" echo "QUIC Port: $quicPort" echo "Metrics Port: $metricsPort" echo "Devnet: ${devnet:-}" echo "Private Key File: $privKeyPath" + echo "Is Aggregator: $isAggregator" + if [ -n "$attestationCommitteeCount" ]; then + echo "Attestation Committee Count: $attestationCommitteeCount" + fi fi diff --git a/spin-node.sh b/spin-node.sh index d0f50d5..9f91470 100755 --- a/spin-node.sh +++ b/spin-node.sh @@ -81,6 +81,43 @@ echo "Detected nodes: ${nodes[@]}" # nodes=("zeam_0" "ream_0" "qlean_0") spin_nodes=() +# Aggregator selection logic (1 aggregator per subnet) +# If user specified --aggregator, use that; otherwise randomly select one +if [ -n "$aggregatorNode" ]; then + # Validate that the specified aggregator exists in the validator list + aggregator_found=false + for available_node in "${nodes[@]}"; do + if [[ "$aggregatorNode" == "$available_node" ]]; then + selected_aggregator="$aggregatorNode" + aggregator_found=true + echo "Using user-specified aggregator: $selected_aggregator" + break + fi + done + + if [[ "$aggregator_found" == false ]]; then + echo "Error: Specified aggregator '$aggregatorNode' not found in validator config" + echo "Available nodes: ${nodes[@]}" + exit 1 + fi +else + # Randomly select one node as aggregator + # Get the number of nodes + num_nodes=${#nodes[@]} + # Generate random index (0 to num_nodes-1) + random_index=$((RANDOM % num_nodes)) + selected_aggregator="${nodes[$random_index]}" + echo "Randomly selected aggregator: $selected_aggregator (index $random_index out of $num_nodes nodes)" +fi + +# Update the validator-config.yaml to set isAggregator flag +# First, reset all nodes to isAggregator: false +yq eval -i '.validators[].isAggregator = false' "$validator_config_file" + +# Then set the selected aggregator to isAggregator: true +yq eval -i "(.validators[] | select(.name == \"$selected_aggregator\") | .isAggregator) = true" "$validator_config_file" +echo "Set $selected_aggregator as aggregator in $validator_config_file" + # Parse comma-separated or space-separated node names or handle single node/all if [[ "$node" == "all" ]]; then # Spin all nodes @@ -241,6 +278,10 @@ elif [[ "$OSTYPE" == "linux"* ]]; then fi spinned_pids=() for item in "${spin_nodes[@]}"; do + # extract client config FIRST before printing + IFS='_' read -r -a elements <<< "$item" + client="${elements[0]}" + echo -e "\n\nspining $item: client=$client (mode=$node_setup)" printf '%*s' $(tput cols) | tr ' ' '-' echo @@ -260,10 +301,6 @@ for item in "${spin_nodes[@]}"; do # parse validator-config.yaml for $item to load args values source parse-vc.sh - # extract client config - IFS='_' read -r -a elements <<< "$item" - client="${elements[0]}" - # get client specific cmd and its mode (docker, binary) sourceCmd="source client-cmds/$client-cmd.sh" echo "$sourceCmd" diff --git a/test-local-zeam.sh b/test-local-zeam.sh new file mode 100755 index 0000000..82d4fb5 --- /dev/null +++ b/test-local-zeam.sh @@ -0,0 +1,21 @@ +#!/bin/bash + +# Test script for zeam with local image +echo "=========================================" +echo "Testing zeam with 0xpartha/zeam:local" +echo "=========================================" + +# Clean up any existing containers +echo "Cleaning up existing zeam_0 container..." +docker rm -f zeam_0 2>/dev/null || true + +# Run with local-devnet configuration +echo "" +echo "Starting zeam_0 with devnet3 configuration..." +echo "- Image: 0xpartha/zeam:local" +echo "- Aggregator: Will be randomly selected" +echo "- Attestation committee count: Using client default (not overridden)" +echo "" + +NETWORK_DIR=local-devnet ./spin-node.sh --node zeam_0 --generateGenesis --cleanData +