Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use local SRS points, remove operator-setup submodule #49

Merged
merged 1 commit into from
Jun 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,16 @@ EIGENDA_PROXY_ADDR=127.0.0.1
EIGENDA_PROXY_PORT=3100

# Directory path to SRS tables
EIGENDA_PROXY_TARGET_CACHE_PATH=
EIGENDA_PROXY_TARGET_CACHE_PATH=resources/SRSTables

# Directory path to g1.point file
EIGENDA_PROXY_TARGET_KZG_G1_PATH=
EIGENDA_PROXY_TARGET_KZG_G1_PATH=resources/g1.point

# Directory path to g2.point.powerOf2 file
EIGENDA_PROXY_TARGET_G2_TAU_PATH=
EIGENDA_PROXY_TARGET_G2_TAU_PATH=resources/g2.point.powerOf2

# RPC endpoint of the EigenDA disperser
EIGENDA_PROXY_RPC=
EIGENDA_PROXY_EIGENDA_DISPERSER_RPC=

# Wait time between retries of EigenDA blob status queries (default: 5s)
EIGENDA_PROXY_STATUS_QUERY_INTERVAL=5s
Expand Down
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ go.work
.env

## kzg cache
e2e/resources/kzg/SRSTables/
resources/SRSTables/

## Vscode
/.vscode
Expand Down
3 changes: 0 additions & 3 deletions .gitmodules

This file was deleted.

5 changes: 0 additions & 5 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -54,11 +54,6 @@ gosec:
submodules:
git submodule update --init --recursive

srs:
if ! test -f /operator-setup/resources/g1.point; then \
cd operator-setup && ./srs_setup.sh; \
fi

op-devnet-allocs:
@echo "Generating devnet allocs..."
@./scripts/op-devnet-allocs.sh
Expand Down
25 changes: 11 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ In order to disperse to the EigenDA network in production, or at high throughput

Additional CLI args are provided for targeting an EigenDA network backend:

* `--eigenda-rpc`: RPC host of disperser service. (e.g, on holesky this is `disperser-holesky.eigenda.xyz:443`, full network list [here](https://docs.eigenlayer.xyz/eigenda/networks/))
* `--eigenda-disperser-rpc`: RPC host of disperser service. (e.g, on holesky this is `disperser-holesky.eigenda.xyz:443`, full network list [here](https://docs.eigenlayer.xyz/eigenda/networks/))
* `--eigenda-status-query-timeout`: (default: 30m) Duration for which a client will wait for a blob to finalize after being sent for dispersal.
* `--eigenda-status-query-retry-interval`: (default: 5s) How often a client will attempt a retry when awaiting network blob finalization.
* `--eigenda-disable-tls`: (default: false) Whether to disable TLS for grpc communication with disperser.
Expand All @@ -28,33 +28,35 @@ Additional CLI args are provided for targeting an EigenDA network backend:
* `--eigenda-cache-path`: Directory path to dump cached SRS tables
* `--eigenda-max-blob-length`: The maximum blob length that this EigenDA sidecar proxy should expect to be written or read from EigenDA. This configuration setting is used to determine how many SRS points should be loaded into memory for generating/verifying KZG commitments returned by the EigenDA disperser. Valid byte units are either base-2 or base-10 byte amounts (not bits), e.g. `30 MiB`, `4Kb`, `30MB`. The maximum blob size is a little more than `1GB`.


### Certificate verification

For additional security, there is a cert verification feature which verifies the blob metadata read from the disperser to ensure that:

1. The respective batch hash can be computed locally and matches the one persisted on-chain in the `ServiceManager` contract
2. The blob inclusion proof can be merkalized to generate the proper batch root
3. All quorum params are adequately defined and expressed when compared to their on-chain counterparts

To target this feature, the following CLI args should be provided:

* `--eigenda-svc-manager-addr`: The deployed EigenDA service manager address. The list can be found [here](https://github.com/Layr-Labs/eigenlayer-middleware/?tab=readme-ov-file#current-mainnet-deployment).
* `--eigenda-eth-rpc` : JSON RPC node endpoint for the Ethereum network used for finalizing DA blobs. See available list [here](https://docs.eigenlayer.xyz/eigenda/networks/).


### In-Memory Storage

An ephemeral memory store backend can be used for faster feedback testing when performing rollup integrations. The following cli args can be used to target the feature:

* `--memstore.enabled`: Boolean feature flag
* `--memstore.expiration`: Duration for which a blob will exist

## Metrics
## Metrics

To the see list of available metrics, run `./bin/eigenda-proxy doc metrics`

## Running Locally

1. Compile binary: `make eigenda-proxy`
2. Generate a new private key and save it as `EIGENDA_PROXY_SIGNER_PRIVATE_KEY_HEX` env var without the `0x` prefix.
3. Run binary; e.g: `./bin/eigenda-proxy --addr 127.0.0.1 --port 5050 --eigenda-rpc 127.0.0.1:443 --eigenda-status-query-timeout 45m --eigenda-g1-path e2e/resources/kzg/g1.point --eigenda-g2-tau-path e2e/resources/kzg/g2.point.powerOf2 --eigenda-max-blob-length='90Kib'`
3. Run binary; e.g: `./bin/eigenda-proxy --addr 127.0.0.1 --port 5050 --eigenda-disperser-rpc disperser-holesky.eigenda.xyz:443 --eigenda-status-query-timeout 45m --eigenda-max-blob-length='90Kib'`

**Env File**
An env file can be provided to the binary for runtime process ingestion; e.g:
Expand Down Expand Up @@ -84,23 +86,18 @@ The `raw commitment` for EigenDA is encoding certificate and kzg fields.
## Testing

### Unit

Unit tests can be ran via invoking `make test`.

### Holesky
A holesky integration test can be ran using `make holesky-test` to assert proper dispersal/retrieval against a public network. Please **note** that EigenDA Holesky network which is subject to rate-limiting and slow confirmation times *(i.e, >10 minutes per blob confirmation)*. Please advise EigenDA's [inabox](https://github.com/Layr-Labs/eigenda/tree/master/inabox#readme) if you'd like to spin-up a local DA network for faster iteration testing.

A holesky integration test can be ran using `make holesky-test` to assert proper dispersal/retrieval against a public network. Please **note** that EigenDA Holesky network which is subject to rate-limiting and slow confirmation times *(i.e, >10 minutes per blob confirmation)*. Please advise EigenDA's [inabox](https://github.com/Layr-Labs/eigenda/tree/master/inabox#readme) if you'd like to spin-up a local DA network for faster iteration testing.

### Optimism
An E2E test exists which spins up a local OP sequencer instance using the [op-e2e](https://github.com/ethereum-optimism/optimism/tree/develop/op-e2e) framework for asserting correct interaction behaviors with batch submission and state derivation. These tests can be ran via `make optimism-test`.

**NOTE:**

## Downloading Mainnet SRS

KZG commitment verification requires constructing the SRS string from the proper trusted setup values (g1, g2, g2.power_of_tau). These values can be downloaded locally using the [operator-setup](https://github.com/Layr-Labs/eigenda-operator-setup) submodule via the following commands.
An E2E test exists which spins up a local OP sequencer instance using the [op-e2e](https://github.com/ethereum-optimism/optimism/tree/develop/op-e2e) framework for asserting correct interaction behaviors with batch submission and state derivation. These tests can be ran via `make optimism-test`.

1. `make submodules`
2. `make srs`
**NOTE:**

## Hardware Requirements

Expand Down
Binary file removed e2e/resources/kzg/g1.point
Binary file not shown.
Binary file removed e2e/resources/kzg/g2.point
Binary file not shown.
Binary file removed e2e/resources/kzg/g2.point.powerOf2
Binary file not shown.
8 changes: 4 additions & 4 deletions e2e/setup.go
Original file line number Diff line number Diff line change
Expand Up @@ -67,11 +67,11 @@ func CreateTestSuite(t *testing.T, useMemory bool) (TestSuite, func()) {
},
EthRPC: ethRPC,
SvcManagerAddr: "0xD4A7E1Bd8015057293f0D0A557088c286942e84b", // incompatible with non holeskly networks
CacheDir: "../operator-setup/resources/SRSTables",
G1Path: "../operator-setup/resources/g1_abbr.point",
G2Path: "../test/resources/kzg/g2.point", // do we need this?
CacheDir: "../resources/SRSTables",
G1Path: "../resources/g1.point",
G2Path: "",
MaxBlobLength: "90kib",
G2PowerOfTauPath: "../operator-setup/resources/kzg/g2_abbr.point.powerOf2",
G2PowerOfTauPath: "../resources/g2.point.powerOf2",
PutBlobEncodingVersion: 0x00,
}

Expand Down
11 changes: 7 additions & 4 deletions eigenda/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ import (
)

const (
RPCFlagName = "eigenda-rpc"
EigenDADisperserRPCFlagName = "eigenda-disperser-rpc"
EthRPCFlagName = "eigenda-eth-rpc"
SvcManagerAddrFlagName = "eigenda-svc-manager-addr"
StatusQueryRetryIntervalFlagName = "eigenda-status-query-retry-interval"
Expand Down Expand Up @@ -118,7 +118,7 @@ func ReadConfig(ctx *cli.Context) Config {
cfg := Config{
ClientConfig: clients.EigenDAClientConfig{
/* Required Flags */
RPC: ctx.String(RPCFlagName),
RPC: ctx.String(EigenDADisperserRPCFlagName),
StatusQueryRetryInterval: ctx.Duration(StatusQueryRetryIntervalFlagName),
StatusQueryTimeout: ctx.Duration(StatusQueryTimeoutFlagName),
DisableTLS: ctx.Bool(DisableTlsFlagName),
Expand Down Expand Up @@ -152,9 +152,9 @@ func CLIFlags(envPrefix string) []cli.Flag {
}
return []cli.Flag{
&cli.StringFlag{
Name: RPCFlagName,
Name: EigenDADisperserRPCFlagName,
Usage: "RPC endpoint of the EigenDA disperser.",
EnvVars: prefixEnvVars("RPC"),
EnvVars: prefixEnvVars("EIGENDA_DISPERSER_RPC"),
},
&cli.DurationFlag{
Name: StatusQueryTimeoutFlagName,
Expand Down Expand Up @@ -213,16 +213,19 @@ func CLIFlags(envPrefix string) []cli.Flag {
Name: G1PathFlagName,
Usage: "Directory path to g1.point file",
EnvVars: prefixEnvVars("TARGET_KZG_G1_PATH"),
Value: "resources/g1.point",
},
&cli.StringFlag{
Name: G2TauFlagName,
Usage: "Directory path to g2.point.powerOf2 file",
EnvVars: prefixEnvVars("TARGET_G2_TAU_PATH"),
Value: "resources/g2.point.powerOf2",
},
&cli.StringFlag{
Name: CachePathFlagName,
Usage: "Directory path to SRS tables",
EnvVars: prefixEnvVars("TARGET_CACHE_PATH"),
Value: "resources/SRSTables/",
},
&cli.StringFlag{
Name: EthRPCFlagName,
Expand Down
1 change: 0 additions & 1 deletion operator-setup
Submodule operator-setup deleted from 2a5a91
Binary file added resources/g1.point
Binary file not shown.
Binary file added resources/g2.point.powerOf2
Binary file not shown.
12 changes: 6 additions & 6 deletions store/memory_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,9 @@ func TestGetSet(t *testing.T) {
defer cancel()

kzgConfig := &kzg.KzgConfig{
G1Path: "../e2e/resources/kzg/g1.point",
G2PowerOf2Path: "../e2e/resources/kzg/g2.point.powerOf2",
CacheDir: "../e2e/resources/kzg/SRSTables",
G1Path: "../resources/g1.point",
G2PowerOf2Path: "../resources/g2.point.powerOf2",
CacheDir: "../resources/SRSTables",
SRSOrder: 3000,
SRSNumberToLoad: 3000,
NumWorker: uint64(runtime.GOMAXPROCS(0)),
Expand Down Expand Up @@ -67,9 +67,9 @@ func TestExpiration(t *testing.T) {
defer cancel()

kzgConfig := &kzg.KzgConfig{
G1Path: "../e2e/resources/kzg/g1.point",
G2PowerOf2Path: "../e2e/resources/kzg/g2.point.powerOf2",
CacheDir: "../e2e/resources/kzg/SRSTables",
G1Path: "../resources/g1.point",
G2PowerOf2Path: "../resources/g2.point.powerOf2",
CacheDir: "../resources/SRSTables",
SRSOrder: 3000,
SRSNumberToLoad: 3000,
NumWorker: uint64(runtime.GOMAXPROCS(0)),
Expand Down
10 changes: 5 additions & 5 deletions verify/verify_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,10 @@ func TestCommitmentVerification(t *testing.T) {

var data = []byte("inter-subjective and not objective!")

x, err := hex.DecodeString("2fc55f968a2d29d22aebf55b382528d1d9401577c166483e162355b19d8bc446")
x, err := hex.DecodeString("1021d699eac68ce312196d480266e8b82fd5fe5c4311e53313837b64db6df178")
assert.NoError(t, err)

y, err := hex.DecodeString("149e2241c21c391e069b9f317710c7f57f31ee88245a5e61f0d294b11acf9aff")
y, err := hex.DecodeString("02efa5a7813233ae13f32bae9b8f48252fa45c1b06a5d70bed471a9bea8d98ae")
assert.NoError(t, err)

c := &common.G1Commitment{
Expand All @@ -28,9 +28,9 @@ func TestCommitmentVerification(t *testing.T) {
}

kzgConfig := &kzg.KzgConfig{
G1Path: "../e2e/resources/kzg/g1.point",
G2PowerOf2Path: "../e2e/resources/kzg/g2.point.powerOf2",
CacheDir: "../e2e/resources/kzg/SRSTables",
G1Path: "../resources/g1.point",
G2PowerOf2Path: "../resources/g2.point.powerOf2",
CacheDir: "../resources/SRSTables",
SRSOrder: 3000,
SRSNumberToLoad: 3000,
NumWorker: uint64(runtime.GOMAXPROCS(0)),
Expand Down
Loading