reference-configuration.adoc

Configuration Reference

Table of Contents

• Example
• Top level config
• TLS server config
• Monitoring
• Health Check endpoint
• Proxy config
• Access Log config
• Request Log config
• Tokens config
• Cache config
• Signed Response
• Cluster
  • Main Cluster Configuration
  • JSON RPC Upstream
  • Additional Options
  • Dshackle Upstream

Example

Example configuration with most options configured

host: 0.0.0.0
port: 2449

tls:
  enabled: true
  server:
    certificate: "/path/127.0.0.1.crt"
    key: "/path/127.0.0.1.p8.key"
  client:
    require: true
    ca: "/path/ca.dshackle.test.crt"

monitoring:
  enabled: true
  jvm: false
  extended: false
  prometheus:
    enabled: true
    bind: 127.0.0.1
    port: 8081
    path: /metrics

health:
  port: 8082
  host: 127.0.0.1
  path: /health
  blockchains:
   - blockchain: ethereum
     min-availability: 1

cache:
  redis:
    enabled: true
    host: redis-master
    port: 6379
    db: 0
    password: I1y0dGKy01by

signed-response:
  enabled: true
  algorithm: SECP256K1
  private-key: /path/key.pem

proxy:
  host: 0.0.0.0
  port: 8080
  websocket: true
  preserve-batch-order: false
  tls:
    enabled: true
    server:
      certificate: "/path/127.0.0.1.crt"
      key: "/path/127.0.0.1.p8.key"
    client:
      require: true
      ca: "/path/ca.dshackle.test.crt"
  routes:
    - id: eth
      blockchain: ethereum
    - id: kovan
      blockchain: kovan

tokens:
  - id: dai
    blockchain: ethereum
    name: DAI
    type: ERC-20
    address: 0x6B175474E89094C44Da98b954EedeAC495271d0F
  - id: tether
    blockchain: ethereum
    name: Tether
    type: ERC-20
    address: 0xdac17f958d2ee523a2206206994597c13d831ec7

access-log:
  enabled: true
  filename: /var/log/dshackle/access_log.jsonl

request-log:
  enabled: true
  filename: /var/log/dshackle/request_log.jsonl

cluster:
  defaults:
    - blockchains:
        - ethereum
      min-peers: 10
    - blockchains:
        - kovan
      min-peers: 3
  include:
    - "upstreams-extra.yaml"
  upstreams:
    - id: local
      blockchain: ethereum
      labels:
        fullnode: true
      methods:
        enabled:
          - name: "parity_trace"
        disabled:
          - name: "admin_shutdown"
      connection:
        ethereum:
          rpc:
            url: "http://localhost:8545"
          ws:
            url: "ws://localhost:8546"
            origin: "http://localhost"
            basic-auth:
              username: 9c199ad8f281f20154fc258fe41a6814
              password: 258fe4149c199ad8f2811a68f20154fc
    - id: infura
      blockchain: ethereum
      disable-validation: true
      connection:
        ethereum:
          rpc:
            url: "https://mainnet.infura.io/v3/fa28c968191849c1aff541ad1d8511f2"
            basic-auth:
              username: 4fc258fe41a68149c199ad8f281f2015
              password: 1a68f20154fc258fe4149c199ad8f281
    - id: bitcoin
      blockchain: bitcoin
      # use the node to fetch balances
      balance: true
      connection:
        bitcoin:
          rpc:
            url: "http://localhost:8332"
            basic-auth:
              username: bitcoin
              password: e984af45bb888428207c290
          # use Esplora index to fetch balances and utxo for an address
          esplora:
            url: "http://localhost:3001"
          # connect via ZeroMQ to get notifications about new blocks
          zeromq:
            url: "tcp://localhost:5555"
    - id: remote
      connection:
        dshackle:
          host: "10.2.0.15"
          tls:
            ca: /path/ca.dshackle.test.crt
            certificate: /path/client1.dshackle.test.crt
            key: /path/client1.dshackle.test.key

Top level config

Option	Default Value	Description
host	127.0.0.0	Host to bind gRPC server
port	2449	Port to bind gRPC server
compress	true	Enable support for gRPC compression (i.e., gzip).
tls		Setup TLS configuration for the gRPC server. See TLS server config section
monitoring		Setup Prometheus monitoring. See Monitoring section
health		Setup Health Check endpoint See Health Check endpoint section
proxy		Setup HTTP proxy that emulates all standard JSON RPC requests. See Proxy config section
egressLog		Configure egress logging (requests to Dshackle). See [egressLog] section
ingressLog		Configure ingress logging (requests made by Dshackle to upstreams). See [ingressLog] section
tokens		Configure tokens for tracking balance. See Tokens config section
cache		Caching configuration. See Cache config section.
signed-response		Signed responses See Signed Response section.
cluster		Setup connection to remote nodes.See Cluster section

TLS server config

tls:
  enabled: true
  server:
    certificate: "/path/127.0.0.1.crt"
    key: "/path/127.0.0.1.p8.key"
  client:
    require: true
    ca: "/path/ca.dshackle.test.crt"

Option	Default Value	Description
enabled	true if any value is set	Enable/Disable TLS
server.certificate		Path to x509 certificate
server.key		Path to a private key to the certificate.The key MUST BE in PKCS 8 format
client.require		If true then the server will required certificate from a client, otherwise client authentication is optional
client.ca		Certificate to validate client authentication

Monitoring

Configure Prometheus monitoring

monitoring:
  enabled: true
  jvm: false
  extended: false
  prometheus:
    enabled: true
    bind: 127.0.0.1
    port: 8081
    path: /metrics

Option	Default Value	Description
enabled	true	Enable/Disable monitoring endpoint
jvm	false	Enable/Disable JVM metrics (threads, GC, memory, etc)
extended	false	Enable/Disable additional metrics (query selectors, etc)
prometheus.enabled	true	Enable/Disable monitoring endpoint. Reserved for future use, in case of multiple different types of endpoints.
prometheus.bind	127.0.0.1	Host to bind the server
prometheus.port	8081	Port to bind the server
prometheus.path	/metrics	HTTP path to bind the server

Health Check endpoint

health:
  port: 8082
  host: 127.0.0.1
  path: /health
  blockchains:
    - blockchain: ethereum
      min-available: 2
    - blockchain: bitcoin
      min-available: 1

Option	Default Value	Description
port	8082	HTTP port to bind the server
host	127.0.0.1	HTTP host to bind the server
path	/health	HTTP path to respond on requests
blockchains		List of blockchains that must be available to consider the server healthy
[blockchain].blockchain		Blockchain id
[blockchain].min-available	1	How many available upstreams for the blockchain is required to pass

Proxy config

proxy:
  host: 0.0.0.0
  port: 8080
  preserve-batch-order: false
  tls:
    enabled: true
    server:
      certificate: "/path/127.0.0.1.crt"
      key: "/path/127.0.0.1.p8.key"
    client:
      require: true
      ca: "/path/ca.dshackle.test.crt"
  routes:
    - id: eth
      blockchain: ethereum
    - id: kovan
      blockchain: kovan

Table 1. Top config

Option	Default Value	Description
host	127.0.0.0	Host to bind HTTP server
port	8080	Port to bind HTT server
port	false	Should proxy preserve request-response correspondence when sending batch request via http
websocket	true	Enable WebSocket Proxy
tls		Setup TLS configuration for the Proxy server. See TLS server config section
preserve-batch-order	false	If false Dshackle may produce batch response in different order, which is correct as per JSON RPC Spec. If set to true then Dshackle preserves batch order based on request order. Note that latter is ineffective and use this option only when a client cannot reference responses by their IDs.
cors-origin		Access-Control-Allow-Origin contents. If empty the header will be omitted in response
cors-allowed-headers	Content-Type	Access-Control-Allow-Headers contents. Takes effect only if сors-origi is present in config
routes		Routing paths for Proxy. The proxy will handle requests as https://${HOST}:${PORT}/${ROUTE_ID} (or http:// if TLS is not enabled). For WebSocket it’s wss / ws, accordingly.

Table 2. Route config

Option	Default Value	Description
id		Internal alphanumeric id, and a path of binding url - https://${HOST}:${PORT}/${ROUTE_ID}.
blockchain		A blockchain that must be used to handle that route.

Access Log config

access-log:
  enabled: true
  filename: /var/log/dshackle/access_log.jsonl

Table 3. Access Log config

Option	Default	Description
enabled	false	Enable/Disable Access logging
The following only when enabled=true:
include-messages	false	Include request params and response result/error (i.e., a JSON) in the log. It’s an expensive operation, use it for debugging only. Note that for errors it provides only error message, not the error response itself.
type	file	file or socket
The following only when type=file:
filename	access_log.jsonl	Path to the log file
The following only when type=socket:
host	127.0.0.1	Target Host to submit data
port	- (required)	Target Port to submit data
encoding	length-prefix	length-prefix or new-line. Where length-prefix means prepend a 32-bit length to each log event; new-line is to append a \n after each log event
buffer	5000	Max size of the buffer to keep unsent events. For a slow connection, it drops all the new produced events after filling the buffer

Request Log config

request-log:
  enabled: true
  filename: /var/log/dshackle/request_log.jsonl

Table 4. Request Log config

Option	Default	Description
enabled	false	Enable/Disable logging
The following only when enabled=true:
include-params	false	Include request params in the log. It’s an expensive operation, use it for a debugging only.
type	file	file or socket
The following only when type=file:
filename	request_log.jsonl	Path to the log file
The following only when type=socket:
host	127.0.0.1	Target Host to submit data
port	- (required)	Target Port to submit data
encoding	length-prefix	length-prefix or new-line. Where length-prefix means prepend a 32-bit length to each log event; new-line is to append a \n after each log event
buffer	5000	Max size of the buffer to keep unsent events. For a slow connection, it drops all the new produced events after filling the buffer

Tokens config

tokens:
  - id: dai
    blockchain: ethereum
    name: DAI
    type: ERC-20
    address: 0x6B175474E89094C44Da98b954EedeAC495271d0F
  - id: tether
    blockchain: ethereum
    name: Tether
    type: ERC-20
    address: 0xdac17f958d2ee523a2206206994597c13d831ec7

Tokens config enables tracking of a balance amount in the configured tokens. After making the configuration above you can request balance (GetBalance), or subscribe to balance changes (SubscribeBalance), using enhanced protocol

Table 5. Token config

Option	Description
id	Internal id for reference (used in logging, etc)
blockchain	An ethereum-based blockchain where the contract is deployed
name	Name of the token, used for balance response as asset code (as converted to UPPERCASE)
type	Type of token.Only ERC-20 is supported at this moment
address	Address of the deployed contract

Cache config

cache:
  redis:
    enabled: true
    host: redis-master
    port: 6379
    db: 0
    password: I1y0dGKy01by

Table 6. Redis Config

Option	Default Value	Description
enabled	false	Enable/disable Redis cache
host	127.0.0.1	Redis host address
port	6379	Redis port
db	0	Redis DB to select
password		Password for connection, if required

Signed Response

signed-response:
  enabled: true
  algorithm: SECP256K1
  private-key: /path/key.pem

Table 7. Redis Config

Option	Default Value	Description
enabled	false	Enable/disable Signed Responses
algorithm	SECP256K1	SECP256K1 or NIST-P256
private-key		Path to a private key in PEM format

See more details at Signed Response in gRPC Methods.

Cluster

The cluster config is the main part, that defines all connection to nodes and other servers

cluster:
  defaults:
    - blockchains:
        - ethereum
      min-peers: 10
  upstreams:
    - id: local
      blockchain: ethereum
      connection:
        ethereum:
          rpc:
            url: "http://localhost:8545"
          ws:
            url: "ws://localhost:8546"
            origin: "http://localhost"
  include:
    - "upstreams-extra.yaml"

Main Cluster Configuration

Table 8. Top Level Config

Option	Description
defaults	Default options applied to all upstreams within the specified blockchain. It’s an optional configuration, and may be omitted for most of the situations.
upstreams	List of upstream servers. The main part of the config. There are two types of upstream: JSON RPC Upstream and Dshackle Upstream.
include	Path(s) to include configurations for upstream servers. Same as upstreams, but load it from an external file.

JSON RPC Upstream

- id: local
  blockchain: ethereum
  role: standard
  labels:
    fullnode: true
  methods:
    enabled:
      - name: "parity_trace"
        quorum: "not_empty"
    disabled:
      - name: "admin_shutdown"
  connection:
    ethereum:
      rpc:
        url: "http://localhost:8545"
      ws:
        url: "ws://localhost:8546"
        origin: "http://localhost"
        basic-auth:
          username: 9c199ad8f281f20154fc258fe41a6814
          password: 258fe4149c199ad8f2811a68f20154fc
        frame-size: 5mb
        msg-size: 15mb

Table 9. Main Config

Option	Required	Description
id	yes	Per-cluster identifier of an upstream
blockchain	yes	Blockchain which is the provided by the upstream. Cluster may have multiple upstreams for a single blockchain. Accepted types: bitcoin, bitcoin-testnet, ethereum, ethereum-classic, or sepolia-testnet
labels	no	Key-Value pairs that are assigned to the upstream. Used to select an upstream per-request. See Quorum and Selectors
		+ Additional options, see Additional Options
methods	no	Enable (enabled) or disable (disabled) additional JSON RPC methods that are provided by that particular upstream
methods.enabled.name, methods.disabled.name	yes	Name of the RPC method to enable/disable.
methods.enabled.quorum	no	Set quorum criteria to accept a response. always (default) - accept any response; not_empty - accept not null value, otherwise retry another upstream; not_lagging - accept response only from a fully synced upstream.
connection.ethereum	yes	Connection configuration for Ethereum API
connection.bitcoin	yes	Connection configuration for Bitcoin API

Additional Options

Option	Type	Default	Description
enabled	boolean	true	Enable/disable the upstream.
role	enum	primary	primary, secondary or fallback. First it makes the requests to the upstreams with role primary, then if none are available to upstreams with role secondary. Fallback role mean that the upstream is used only after other upstreams failed or didn’t return quorum
priority	number	10	Upstream priority used to resolve Fork Conflicts between different upstreams when a Proof-of-Stake blockchain is used. When two upstreams has different blocks at the same height then the value from Upstreams with higher priority value wins. Configuration has no effect in Proof-of-Work blockchains.
disable-validation	boolean	false	Disables all the validations of the upstream. I.e., it turns off validate-peers and validate-syncing checks if set to true.
validation-interval	number	30	Period in seconds to re-validate the upstream.
validate-peers	boolean	true	Disables validation of the peers connected to the upstream (as net_peerCount method). Dshackle assumes that if there are too few peers then the Upstream is just started and may produce invalid/outdated responses
min-peers	number	1	The minimum number of connected peer to consider the upstream valid if validate-peers is enabled. If it’s set to 0 it essentially disables the peer validation.
validate-syncing	boolean	true	Disables checking for the state of syncing on the upstream (as eth_syncing method). If the Upstream is in syncing state then the Dshackle doesn’t use it for call until it reaches the blockchain head.
timeout	number	60	Timeout in seconds to wait for an answer from the upstream before considering it as failed.
balance	boolean		Suitable for Bitcoin upstream. Tells if the Upstream can be used to call balance methods, which requires that the node has the indexing as turned on.

Ethereum Connection Options

Table 10. Connection Config for Ethereum Upstream

Option	Description
rpc.url	HTTP URL to connect to.This is required for a connection. URL can be configured with Environment Variable placeholders ${ENV_VAR_NAME}. Example: https://kovan.infura.io/v3/${INFURA_USER}
rpc.basic-auth + rpc.basic-auth.username, rpc.basic-auth.password	HTTP Basic Auth configuration, if required by the remote server. Values can also reference env variables, for example: rpc: url: "https://ethereum.com:8545" basic-auth: username: "${ETH_USERNAME}" password: "${ETH_PASSWORD}"
rpc.compress	Enable compression for HTTP connection (i.e., gzip). May not work with some servers. Defaults is false
ws.url	WebSocket URL to connect to. Optional, but optimizes performance if it’s available.
ws.origin	HTTP Origin if required by WebSocket remote server.
ws.basic-auth + …​	WebSocket Basic Auth configuration, if required by the remote server
ws.frame-size	WebSocket frame size limit. Ex 1kb, 1024 (same as 1kb), `2mb, etc. Default is 5Mb
ws.msg-size	Total limit for a message size consisting from multiple frames. Ex 1kb, 1024 (same as 1kb), `2mb, etc. Default is 15Mb
ws.connections	How many concurrent connection to make. If more than one, each used in a robin-round fashion. Defaults is 1
ws.compress	Enable compression for WebSocket connection (i.e., permessage-deflate). May not work with some servers. Defaults is false

Bitcoin Connection Options

Table 11. Connection Config for Bitcoin Upstream

Option	Description
rpc.url	HTTP URL to connect to. This is required for a connection. URL can be configured with Environment Variable placeholders ${ENV_VAR_NAME}. Example: http://${NODE_HOST}:${NODE_PORT}
rpc.basic-auth + rpc.basic-auth.username, rpc.basic-auth.password	HTTP Basic Auth configuration, which is required by the Bitcoind server. Values can also reference env variables, for example: rpc: url: "http://127.0.0.1:8332" basic-auth: username: "${NODE_USERNAME}" password: "${NODE_PASSWORD}"
rpc.compress	Enable compression for HTTP connection (i.e., gzip). May not work with some servers. Defaults is false
zeromq.address	Set up an additional connection via ZeroMQ protocol to subscribe to the new blocks. The node must be launched with the same address specified as -zmqpubhashblock="tcp://${HOST}:${POST}" or in bitcoin.conf zeromq: url: "tcp://127.0.0.1:5555"

Dshackle Upstream

Another option is using another Dshackle server as an upstream. It’s more effective, easier to secure connection, and allows to build a distributed network of servers.

- id: test1
  connection:
    dshackle:
      url: eu-api.mycompany.com:2449
      tls:
        ca: ca.api.mycompany.crt
        certificate: client-1.api.mycompany.crt
        key: client-1.api.mycompany.p8.key

Table 12. Main Config

Option	Required	Description
id	yes	Per-cluster identifier of an upstream
connection.dshackle	yes	Connection configuration for Dshackle gRPC

Table 13. Connection Config

Option	Description
host and port, or url	Address to connect to
tls	TLC configuration for the connection. It’s an optional, but it’s strongly recommended to use a TLS even for internal network. HTTP2 + gRPC is designed to be used with TLS, and some related software is unable to use it without TLS. See Authentication docs and TLS server config.
tls.ca	Path to x509 certificate to verify remote server
tls.certificate + tls.key	Client certificate (x509) and its private key (PKCS 8) used for authentication on the remote server.
compress	Enable compression for gRPC requests (i.e., gzip). Defaults is true