Skip to content

Commit

Permalink
Use local SRS points, remove operator-setup submodule
Browse files Browse the repository at this point in the history
  • Loading branch information
teddyknox committed Jun 21, 2024
1 parent ceff6e8 commit 8c1857d
Show file tree
Hide file tree
Showing 15 changed files with 38 additions and 47 deletions.
8 changes: 4 additions & 4 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,16 @@ EIGENDA_PROXY_ADDR=127.0.0.1
EIGENDA_PROXY_PORT=3100

# Directory path to SRS tables
EIGENDA_PROXY_TARGET_CACHE_PATH=
EIGENDA_PROXY_TARGET_CACHE_PATH=resources/SRSTables

# Directory path to g1.point file
EIGENDA_PROXY_TARGET_KZG_G1_PATH=
EIGENDA_PROXY_TARGET_KZG_G1_PATH=resources/g1.point

# Directory path to g2.point.powerOf2 file
EIGENDA_PROXY_TARGET_G2_TAU_PATH=
EIGENDA_PROXY_TARGET_G2_TAU_PATH=resources/g2.point.powerOf2

# RPC endpoint of the EigenDA disperser
EIGENDA_PROXY_RPC=
EIGENDA_PROXY_EIGENDA_DISPERSER_RPC=

# Wait time between retries of EigenDA blob status queries (default: 5s)
EIGENDA_PROXY_STATUS_QUERY_INTERVAL=5s
Expand Down
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ go.work
.env

## kzg cache
e2e/resources/kzg/SRSTables/
resources/SRSTables/

## Vscode
/.vscode
Expand Down
3 changes: 0 additions & 3 deletions .gitmodules

This file was deleted.

5 changes: 0 additions & 5 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -54,11 +54,6 @@ gosec:
submodules:
git submodule update --init --recursive

srs:
if ! test -f /operator-setup/resources/g1.point; then \
cd operator-setup && ./srs_setup.sh; \
fi

op-devnet-allocs:
@echo "Generating devnet allocs..."
@./scripts/op-devnet-allocs.sh
Expand Down
25 changes: 11 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ In order to disperse to the EigenDA network in production, or at high throughput

Additional CLI args are provided for targeting an EigenDA network backend:

* `--eigenda-rpc`: RPC host of disperser service. (e.g, on holesky this is `disperser-holesky.eigenda.xyz:443`, full network list [here](https://docs.eigenlayer.xyz/eigenda/networks/))
* `--eigenda-disperser-rpc`: RPC host of disperser service. (e.g, on holesky this is `disperser-holesky.eigenda.xyz:443`, full network list [here](https://docs.eigenlayer.xyz/eigenda/networks/))
* `--eigenda-status-query-timeout`: (default: 30m) Duration for which a client will wait for a blob to finalize after being sent for dispersal.
* `--eigenda-status-query-retry-interval`: (default: 5s) How often a client will attempt a retry when awaiting network blob finalization.
* `--eigenda-disable-tls`: (default: false) Whether to disable TLS for grpc communication with disperser.
Expand All @@ -28,33 +28,35 @@ Additional CLI args are provided for targeting an EigenDA network backend:
* `--eigenda-cache-path`: Directory path to dump cached SRS tables
* `--eigenda-max-blob-length`: The maximum blob length that this EigenDA sidecar proxy should expect to be written or read from EigenDA. This configuration setting is used to determine how many SRS points should be loaded into memory for generating/verifying KZG commitments returned by the EigenDA disperser. Valid byte units are either base-2 or base-10 byte amounts (not bits), e.g. `30 MiB`, `4Kb`, `30MB`. The maximum blob size is a little more than `1GB`.


### Certificate verification

For additional security, there is a cert verification feature which verifies the blob metadata read from the disperser to ensure that:

1. The respective batch hash can be computed locally and matches the one persisted on-chain in the `ServiceManager` contract
2. The blob inclusion proof can be merkalized to generate the proper batch root
3. All quorum params are adequately defined and expressed when compared to their on-chain counterparts

To target this feature, the following CLI args should be provided:

* `--eigenda-svc-manager-addr`: The deployed EigenDA service manager address. The list can be found [here](https://github.com/Layr-Labs/eigenlayer-middleware/?tab=readme-ov-file#current-mainnet-deployment).
* `--eigenda-eth-rpc` : JSON RPC node endpoint for the Ethereum network used for finalizing DA blobs. See available list [here](https://docs.eigenlayer.xyz/eigenda/networks/).


### In-Memory Storage

An ephemeral memory store backend can be used for faster feedback testing when performing rollup integrations. The following cli args can be used to target the feature:

* `--memstore.enabled`: Boolean feature flag
* `--memstore.expiration`: Duration for which a blob will exist

## Metrics
## Metrics

To the see list of available metrics, run `./bin/eigenda-proxy doc metrics`

## Running Locally

1. Compile binary: `make eigenda-proxy`
2. Generate a new private key and save it as `EIGENDA_PROXY_SIGNER_PRIVATE_KEY_HEX` env var without the `0x` prefix.
3. Run binary; e.g: `./bin/eigenda-proxy --addr 127.0.0.1 --port 5050 --eigenda-rpc 127.0.0.1:443 --eigenda-status-query-timeout 45m --eigenda-g1-path e2e/resources/kzg/g1.point --eigenda-g2-tau-path e2e/resources/kzg/g2.point.powerOf2 --eigenda-max-blob-length='90Kib'`
3. Run binary; e.g: `./bin/eigenda-proxy --addr 127.0.0.1 --port 5050 --eigenda-disperser-rpc disperser-holesky.eigenda.xyz:443 --eigenda-status-query-timeout 45m --eigenda-max-blob-length='90Kib'`

**Env File**
An env file can be provided to the binary for runtime process ingestion; e.g:
Expand Down Expand Up @@ -84,23 +86,18 @@ The `raw commitment` for EigenDA is encoding certificate and kzg fields.
## Testing

### Unit

Unit tests can be ran via invoking `make test`.

### Holesky
A holesky integration test can be ran using `make holesky-test` to assert proper dispersal/retrieval against a public network. Please **note** that EigenDA Holesky network which is subject to rate-limiting and slow confirmation times *(i.e, >10 minutes per blob confirmation)*. Please advise EigenDA's [inabox](https://github.com/Layr-Labs/eigenda/tree/master/inabox#readme) if you'd like to spin-up a local DA network for faster iteration testing.

A holesky integration test can be ran using `make holesky-test` to assert proper dispersal/retrieval against a public network. Please **note** that EigenDA Holesky network which is subject to rate-limiting and slow confirmation times *(i.e, >10 minutes per blob confirmation)*. Please advise EigenDA's [inabox](https://github.com/Layr-Labs/eigenda/tree/master/inabox#readme) if you'd like to spin-up a local DA network for faster iteration testing.

### Optimism
An E2E test exists which spins up a local OP sequencer instance using the [op-e2e](https://github.com/ethereum-optimism/optimism/tree/develop/op-e2e) framework for asserting correct interaction behaviors with batch submission and state derivation. These tests can be ran via `make optimism-test`.

**NOTE:**

## Downloading Mainnet SRS

KZG commitment verification requires constructing the SRS string from the proper trusted setup values (g1, g2, g2.power_of_tau). These values can be downloaded locally using the [operator-setup](https://github.com/Layr-Labs/eigenda-operator-setup) submodule via the following commands.
An E2E test exists which spins up a local OP sequencer instance using the [op-e2e](https://github.com/ethereum-optimism/optimism/tree/develop/op-e2e) framework for asserting correct interaction behaviors with batch submission and state derivation. These tests can be ran via `make optimism-test`.

1. `make submodules`
2. `make srs`
**NOTE:**

## Hardware Requirements

Expand Down
Binary file removed e2e/resources/kzg/g1.point
Binary file not shown.
Binary file removed e2e/resources/kzg/g2.point
Binary file not shown.
Binary file removed e2e/resources/kzg/g2.point.powerOf2
Binary file not shown.
8 changes: 4 additions & 4 deletions e2e/setup.go
Original file line number Diff line number Diff line change
Expand Up @@ -67,11 +67,11 @@ func CreateTestSuite(t *testing.T, useMemory bool) (TestSuite, func()) {
},
EthRPC: ethRPC,
SvcManagerAddr: "0xD4A7E1Bd8015057293f0D0A557088c286942e84b", // incompatible with non holeskly networks
CacheDir: "../operator-setup/resources/SRSTables",
G1Path: "../operator-setup/resources/g1_abbr.point",
G2Path: "../test/resources/kzg/g2.point", // do we need this?
CacheDir: "../resources/SRSTables",
G1Path: "../resources/g1.point",
G2Path: "",
MaxBlobLength: "90kib",
G2PowerOfTauPath: "../operator-setup/resources/kzg/g2_abbr.point.powerOf2",
G2PowerOfTauPath: "../resources/g2.point.powerOf2",
PutBlobEncodingVersion: 0x00,
}

Expand Down
11 changes: 7 additions & 4 deletions eigenda/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ import (
)

const (
RPCFlagName = "eigenda-rpc"
EigenDADisperserRPCFlagName = "eigenda-disperser-rpc"
EthRPCFlagName = "eigenda-eth-rpc"
SvcManagerAddrFlagName = "eigenda-svc-manager-addr"
StatusQueryRetryIntervalFlagName = "eigenda-status-query-retry-interval"
Expand Down Expand Up @@ -118,7 +118,7 @@ func ReadConfig(ctx *cli.Context) Config {
cfg := Config{
ClientConfig: clients.EigenDAClientConfig{
/* Required Flags */
RPC: ctx.String(RPCFlagName),
RPC: ctx.String(EigenDADisperserRPCFlagName),
StatusQueryRetryInterval: ctx.Duration(StatusQueryRetryIntervalFlagName),
StatusQueryTimeout: ctx.Duration(StatusQueryTimeoutFlagName),
DisableTLS: ctx.Bool(DisableTlsFlagName),
Expand Down Expand Up @@ -152,9 +152,9 @@ func CLIFlags(envPrefix string) []cli.Flag {
}
return []cli.Flag{
&cli.StringFlag{
Name: RPCFlagName,
Name: EigenDADisperserRPCFlagName,
Usage: "RPC endpoint of the EigenDA disperser.",
EnvVars: prefixEnvVars("RPC"),
EnvVars: prefixEnvVars("EIGENDA_DISPERSER_RPC"),
},
&cli.DurationFlag{
Name: StatusQueryTimeoutFlagName,
Expand Down Expand Up @@ -213,16 +213,19 @@ func CLIFlags(envPrefix string) []cli.Flag {
Name: G1PathFlagName,
Usage: "Directory path to g1.point file",
EnvVars: prefixEnvVars("TARGET_KZG_G1_PATH"),
Value: "resources/g1.point",
},
&cli.StringFlag{
Name: G2TauFlagName,
Usage: "Directory path to g2.point.powerOf2 file",
EnvVars: prefixEnvVars("TARGET_G2_TAU_PATH"),
Value: "resources/g2.point.powerOf2",
},
&cli.StringFlag{
Name: CachePathFlagName,
Usage: "Directory path to SRS tables",
EnvVars: prefixEnvVars("TARGET_CACHE_PATH"),
Value: "resources/SRSTables/",
},
&cli.StringFlag{
Name: EthRPCFlagName,
Expand Down
1 change: 0 additions & 1 deletion operator-setup
Submodule operator-setup deleted from 2a5a91
Binary file added resources/g1.point
Binary file not shown.
Binary file added resources/g2.point.powerOf2
Binary file not shown.
12 changes: 6 additions & 6 deletions store/memory_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,9 @@ func TestGetSet(t *testing.T) {
defer cancel()

kzgConfig := &kzg.KzgConfig{
G1Path: "../e2e/resources/kzg/g1.point",
G2PowerOf2Path: "../e2e/resources/kzg/g2.point.powerOf2",
CacheDir: "../e2e/resources/kzg/SRSTables",
G1Path: "../resources/g1.point",
G2PowerOf2Path: "../resources/g2.point.powerOf2",
CacheDir: "../resources/SRSTables",
SRSOrder: 3000,
SRSNumberToLoad: 3000,
NumWorker: uint64(runtime.GOMAXPROCS(0)),
Expand Down Expand Up @@ -67,9 +67,9 @@ func TestExpiration(t *testing.T) {
defer cancel()

kzgConfig := &kzg.KzgConfig{
G1Path: "../e2e/resources/kzg/g1.point",
G2PowerOf2Path: "../e2e/resources/kzg/g2.point.powerOf2",
CacheDir: "../e2e/resources/kzg/SRSTables",
G1Path: "../resources/g1.point",
G2PowerOf2Path: "../resources/g2.point.powerOf2",
CacheDir: "../resources/SRSTables",
SRSOrder: 3000,
SRSNumberToLoad: 3000,
NumWorker: uint64(runtime.GOMAXPROCS(0)),
Expand Down
10 changes: 5 additions & 5 deletions verify/verify_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,10 @@ func TestCommitmentVerification(t *testing.T) {

var data = []byte("inter-subjective and not objective!")

x, err := hex.DecodeString("2fc55f968a2d29d22aebf55b382528d1d9401577c166483e162355b19d8bc446")
x, err := hex.DecodeString("1021d699eac68ce312196d480266e8b82fd5fe5c4311e53313837b64db6df178")
assert.NoError(t, err)

y, err := hex.DecodeString("149e2241c21c391e069b9f317710c7f57f31ee88245a5e61f0d294b11acf9aff")
y, err := hex.DecodeString("02efa5a7813233ae13f32bae9b8f48252fa45c1b06a5d70bed471a9bea8d98ae")
assert.NoError(t, err)

c := &common.G1Commitment{
Expand All @@ -28,9 +28,9 @@ func TestCommitmentVerification(t *testing.T) {
}

kzgConfig := &kzg.KzgConfig{
G1Path: "../e2e/resources/kzg/g1.point",
G2PowerOf2Path: "../e2e/resources/kzg/g2.point.powerOf2",
CacheDir: "../e2e/resources/kzg/SRSTables",
G1Path: "../resources/g1.point",
G2PowerOf2Path: "../resources/g2.point.powerOf2",
CacheDir: "../resources/SRSTables",
SRSOrder: 3000,
SRSNumberToLoad: 3000,
NumWorker: uint64(runtime.GOMAXPROCS(0)),
Expand Down

0 comments on commit 8c1857d

Please sign in to comment.