Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/actions/local-network/action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -92,3 +92,4 @@ runs:
run: yarn start
env:
DEBUG: 1
SDK_TEST_DATA: "true"
6 changes: 6 additions & 0 deletions .github/workflows/tests-packges-functional.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,12 @@ jobs:
- name: Run SDK functional tests
run: yarn workspace dash test:functional

- name: Run Evo SDK functional tests
run: yarn workspace @dashevo/evo-sdk test:functional

- name: Run WASM SDK functional tests
run: yarn workspace @dashevo/wasm-sdk test:functional

- name: Show Docker logs
if: ${{ failure() }}
uses: jwalton/gh-docker-logs@v2
74 changes: 74 additions & 0 deletions docs/local-network-functional-tests.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# Running wasm/js Evo functional tests against a local dashmate network

Context and gaps to close before we can point the wasm-sdk and js-evo-sdk functional suites at a dashmate `local` network instead of public testnet.

## What exists on dashmate local

- Platform boots with system contracts already present (e.g., DPNS contract `GWRSAVFMjXx8HpQFaNJMqBV7MBgMK4br5UESsB4S31Ec`, withdrawals contract) once `waitForNodeToBeReady` passes.
- Local preset exposes HTTPS DAPI on port `2443` with a self-signed certificate (`platform.gateway.listeners.dapiAndDrive.port` in the local config).
- LLMQs are formed automatically via the `local_seed` miner node; no external quorum endpoints.

## Current blockers

- SDK network support is limited to `mainnet`/`testnet`:
- `rs-sdk-trusted-context-provider` rejects `Network::Regtest` and always fetches quorums from `quorums.{mainnet,testnet}.networks.dash.org`.
- wasm-sdk and js-evo-sdk builders only accept `mainnet`/`testnet` and default to hardcoded testnet address lists.
- Functional tests hardcode testnet data:
- js-evo fixtures (`packages/js-evo-sdk/tests/fixtures/testnet.mjs`) contain testnet identities, contracts, tokens, proTx hash, epoch numbers, usernames, etc., and tests call `EvoSDK.testnetTrusted()`.
- wasm-sdk functional tests all call `WasmSdkBuilder.testnetTrusted()` and assert against the same testnet IDs.
- TLS: local DAPI uses a self-signed cert; the wasm/JS surface does not currently expose a way to inject a CA or bypass verification when custom addresses are used.

## What needs to change

1) **Add local/regtest network path**
- Extend `rs-sdk-trusted-context-provider` to handle `Network::Regtest` (or add a “local” mode) with a local quorum source or a way to skip trusted prefetch for proofs.
- Update wasm-sdk/js-evo-sdk builders and type unions to accept `local/regtest`, and allow constructing builders without hitting testnet quorum endpoints when custom addresses are provided.

2) **Allow local DAPI connectivity**
- Dashmate already issues self-signed certificates for the local preset; we only need to point the SDKs to the local HTTPS endpoint (e.g., `https://127.0.0.1:2443`) without custom CA plumbing.
- Prefer env-driven address overrides (e.g., `EVO_DAPI_ADDRESSES`) so CI/local runs can switch networks without code edits.

3) **Seed test data on local**
- System contracts are present, but testnet-only fixtures (identity `5DbL…`, token contract `H7FR…`, group contract `49PJ…`, usernames, epoch numbers, proTx hash) do not exist on a fresh local network.
- Add a seeding step/script (can reuse platform-test-suite helpers) that, after dashmate start, creates:
- A funded identity with keys.
- DPNS names, known documents.
- Token/group contracts and balances needed by assertions.
- Emit a generated fixture file (e.g., `fixtures/local.mjs`) consumed by both wasm-sdk and js-evo-sdk tests.
- Use the existing `SDK_TEST_DATA=true yarn start` mechanism to have dashmate seed test data automatically during network startup, then extract the IDs into the local fixtures for wasm/js-evo functional suites.
- For wasm-sdk specifically, required IDs/contracts are summarized in `packages/wasm-sdk/tests/functional/fixtures/requiredTestData.mjs`; ensure the seeding path populates equivalents on local. The `SDK_TEST_DATA=true` hook seeds sample identities/contracts used by SDK tests—align local fixtures to those outputs.

4) **Make tests local-first**
- Replace hardcoded `testnet` builders with a local/regtest option and env-driven address selection; functional suites should target the local dashmate network when run from this repo.
- Replace fixed IDs/proTx hashes/epoch numbers in assertions with the seeded fixture values or live queries against the local node.

## Trusted quorum options for local

- **Run a local “quorums.*” HTTP service (mirroring testnet)**
Would mimic `https://quorums.testnet.networks.dash.org` locally. This means building and packaging an extra service (or dashmate plugin) to scrape quorum info from the local network and expose `/quorums`/`/previous-quorums` endpoints, plus wiring TLS/ports. This is the closest analogue to testnet behavior but adds deployment and maintenance overhead.
- **Quorum list server repo**
The public endpoints are served from <https://github.com/dashpay/quorum-list-server>. To support a local mirror, we’ll need to extend that repo with a Dockerfile and CI pipeline so dashmate (or CI) can spin up a local quorum-list instance against the local network.
- **Add a local/regtest path in the trusted context provider (or skip trusted quorums on local)**
Instead of hitting a quorums endpoint, add network-specific logic so that when `network=local/regtest` the SDK/context skips trusted quorum fetching (or uses a minimal local source) and still functions for read-only queries. This avoids needing any quorum-list service but would mean proofs/trusted flows may be limited or disabled on local.

### Decision: use sidecar quorum service
- Implement and publish a Dockerized `quorum-list-server` (see <https://github.com/dashpay/quorum-list-server>) and integrate it into dashmate’s `local` preset docker-compose (wired to Core RPC inside the network; expose HTTPS for SDKs).
- Update SDKs/providers to accept a local quorum base URL (or auto-point `network=local` to the dashmate service) so trusted/proof flows work unchanged on local.
- Keep the skip-path as a fallback only if the sidecar is unavailable; primary path is proof-parity via the local quorum service.

### Dashmate integration notes (in progress)
- We will ship the quorum-list server as a dashmate-managed service, backed by the new Docker image (build almost done). It must be **disabled by default** and only turned on for the `local_seed` node when `dashmate setup local` runs, so regular presets/users are unaffected.
- Add the service to the local preset docker-compose, sourcing Core RPC credentials/host from the existing `local_seed` config and exposing an HTTPS port to the host for SDKs/tests to hit. Keep other nodes unaware of it unless explicitly enabled.
- Provide a config toggle/flag (e.g., `--enable-quorum-list` or a preset-scoped setting) so CI/local flows can opt in; `dashmate status` should surface whether the service is running when the local preset is active.
- When enabled, ensure the local SDK network config points to this service by default (either via preset defaults or env var wiring) so wasm/js functional suites don’t need manual URLs.
- Config toggle landed: set `platform.quorumList.enabled=true` (profile `platform-quorum`, defaults to port `2444` bound to `127.0.0.1`; the sidecar binds internally to `0.0.0.0`). Local setup now enables it on `local_seed` and wires Core RPC credentials from the dedicated `quorum_list` RPC user. Compose service name: `quorum_list` (see `packages/dashmate/docs/services/quorum_list.md`).

## Temporary guidance

- Until the above changes are implemented, do **not** run the functional suites in `packages/js-evo-sdk` or `packages/wasm-sdk` when invoking their `test` scripts; they assume public testnet data and will fail against a local dashmate network.

## Suggested bootstrap flow

1. Start dashmate local (`yarn dashmate setup local && yarn dashmate start`) and wait for readiness.
2. Run a seeding script to create identity/contracts/documents and write `fixtures/local.mjs` (and equivalent for wasm tests).
3. Export env vars (addresses, CA/insecure flag, fixture path) and run the wasm/js-evo functional suites with the new `local` network option.
18 changes: 18 additions & 0 deletions packages/dashmate/configs/defaults/getBaseConfigFactory.js
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,13 @@ export default function getBaseConfigFactory() {
whitelist: null,
lowPriority: false,
},
quorum_list: {
password: 'rpcpassword',
whitelist: [
'quorum', 'getbestblockhash', 'getblockhash', 'getblockcount', 'getblockchaininfo',
],
lowPriority: true,
},
dapi: {
password: 'rpcpassword',
whitelist: [
Expand Down Expand Up @@ -150,6 +157,17 @@ export default function getBaseConfigFactory() {
indexes: [],
},
platform: {
quorumList: {
enabled: false,
docker: {
image: 'dashpay/quorum-list-server:latest',
},
api: {
host: '127.0.0.1',
port: 2444,
},
previousBlocksOffset: 8,
},
gateway: {
docker: {
image: 'dashpay/envoy:1.30.2-impr.1',
Expand Down
10 changes: 10 additions & 0 deletions packages/dashmate/configs/getConfigFileMigrationsFactory.js
Original file line number Diff line number Diff line change
Expand Up @@ -1388,6 +1388,16 @@ export default function getConfigFileMigrationsFactory(homeDir, defaultConfigs)
&& defaultConfig.has('platform.dapi.rsDapi.docker.image')) {
options.platform.dapi.rsDapi.docker.image = defaultConfig.get('platform.dapi.rsDapi.docker.image');
}

if (!options.platform.quorumList) {
options.platform.quorumList = lodash.cloneDeep(defaultConfig.get('platform.quorumList'));
}

if (!options.core.rpc.users.quorum_list) {
options.core.rpc.users.quorum_list = lodash.cloneDeep(
defaultConfig.get('core.rpc.users.quorum_list'),
);
}
});

return configFile;
Expand Down
23 changes: 23 additions & 0 deletions packages/dashmate/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -211,6 +211,29 @@ services:
profiles:
- platform

quorum_list:
image: ${PLATFORM_QUORUM_LIST_DOCKER_IMAGE:?err}
labels:
org.dashmate.service.title: "Quorum List"
org.dashmate.config.name: "${CONFIG_NAME:?err}"
restart: unless-stopped
logging: *default-logging
depends_on:
- core
ports:
- ${PLATFORM_QUORUM_LIST_API_HOST:?err}:${PLATFORM_QUORUM_LIST_API_PORT:?err}:${PLATFORM_QUORUM_LIST_API_PORT:?err}
expose:
- ${PLATFORM_QUORUM_LIST_API_PORT:?err}
environment:
- API_HOST=0.0.0.0
- API_PORT=${PLATFORM_QUORUM_LIST_API_PORT:?err}
- DASH_RPC_URL=http://core:${CORE_RPC_PORT:?err}
- DASH_RPC_USER=quorum_list
- DASH_RPC_PASSWORD=${CORE_RPC_USERS_QUORUM_LIST_PASSWORD:?err}
- QUORUM_PREVIOUS_BLOCKS_OFFSET=${PLATFORM_QUORUM_LIST_PREVIOUS_BLOCKS_OFFSET:?err}
profiles:
- platform-quorum

volumes:
core_data:
drive_abci_data:
Expand Down
5 changes: 3 additions & 2 deletions packages/dashmate/docs/services/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ Dashmate runs and orchestrate Dash Platform components:
- [Core](./core.md): Dash Core service
- [Platform](./platform.md): Platform services (Drive, Tenderdash, DAPI)
- [Gateway](./gateway.md): API Gateway service
- [Quorum List](./quorum_list.md): Optional quorum list sidecar (local preset)
- [Dashmate Helper](./dashmate_helper.md): Helper service for Dashmate CLI

## Port Security Considerations
Expand All @@ -74,7 +75,7 @@ Dashmate runs and orchestrate Dash Platform components:
|---------------------|--------------------------------------------------------------|---------------------|
| **Public-facing** | Core P2P (9999)<br>Tenderdash P2P (26656)<br>Gateway API (443) | 0.0.0.0 (all) |
| **Configurable** | Core ZMQ (29998 mainnet / 39998 testnet / 49998 local) | configurable (see below) |
| **Localhost-only** | Core RPC (9998)<br>Insight UI (3001)<br>Dashmate Helper (9100)<br>Drive ABCI Metrics (29090)<br>Drive Debug Tools (6669, 8083)<br>Tenderdash RPC (26657)<br>Tenderdash Metrics (26660)<br>Tenderdash Debug (6060)<br>Gateway Metrics (9090)<br>Gateway Admin (9901)<br>Rate Limiter Metrics (9102) | 127.0.0.1 (local) |
| **Localhost-only** | Core RPC (9998)<br>Insight UI (3001)<br>Dashmate Helper (9100)<br>Drive ABCI Metrics (29090)<br>Drive Debug Tools (6669, 8083)<br>Tenderdash RPC (26657)<br>Tenderdash Metrics (26660)<br>Tenderdash Debug (6060)<br>Gateway Metrics (9090)<br>Gateway Admin (9901)<br>Rate Limiter Metrics (9102)<br>Quorum List API (2444, optional) | 127.0.0.1 (local) |
| **Internal only** | Drive ABCI (26658)<br>Drive gRPC (26670)<br>DAPI JSON-RPC (3004)<br>DAPI gRPC (3005)<br>DAPI Streams (3006)<br>Rate Limiter gRPC (8081)<br>Rate Limiter StatsD (9125)<br>Rate Limiter Redis (6379) | (not exposed) |

#### Core ZMQ Exposure Configuration
Expand Down Expand Up @@ -121,4 +122,4 @@ Most services provide metrics endpoints:
These can be integrated with monitoring systems like Prometheus and Grafana.
All containers that expose Prometheus metrics also advertise the Docker label `org.dashmate.config.name`, which Dashmate sets to the active config name. When you run multiple nodes on the same host, you can use that label in Prometheus relabeling rules or dashboards to distinguish each instance.

You can find example Prometheus config for local devnet monitoring in [../prometheus/](../prometheus/README.md).
You can find example Prometheus config for local devnet monitoring in [../prometheus/](../prometheus/README.md).
32 changes: 32 additions & 0 deletions packages/dashmate/docs/services/quorum_list.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Quorum List service

The quorum list sidecar exposes the local LLMQ list over HTTP for SDKs and functional tests. It mirrors the public `quorums.*.networks.dash.org` endpoints but runs against your local Core node.

## When it runs

- **Optional**: disabled by default on all presets.
- **Local preset**: enabled automatically only on the `local_seed` node when you run `dashmate setup local`, so other local nodes stay unchanged.
- Controlled by `platform.quorumList.enabled` (profile `platform-quorum`).

## Image and ports

- Image: `dashpay/quorum-list-server:latest`
- API port: `platform.quorumList.api.port` (default `2444`)
- Host binding: `platform.quorumList.api.host` (default `127.0.0.1`)
- Container bind address is `0.0.0.0` inside the compose network.

## Core RPC access

- Uses the dedicated Core RPC user `quorum_list` (added to configs via migration).
- RPC URL points to the local Core container (`http://core:${CORE_RPC_PORT}`).

## Enabling manually

```bash
dashmate config set platform.quorumList.enabled true
dashmate start --platform # or restart the node
```

## Compose service name

`quorum_list` (profile `platform-quorum`). It is only included when the toggle is on or when a command explicitly includes all platform profiles (e.g., platform-only reset/stop).
2 changes: 1 addition & 1 deletion packages/dashmate/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -166,4 +166,4 @@
},
"topicSeparator": " "
}
}
}
30 changes: 30 additions & 0 deletions packages/dashmate/src/config/configJsonSchema.js
Original file line number Diff line number Diff line change
Expand Up @@ -495,6 +495,36 @@ export default {
platform: {
type: 'object',
properties: {
quorumList: {
type: 'object',
properties: {
enabled: {
type: 'boolean',
},
docker: {
$ref: '#/definitions/docker',
},
api: {
type: 'object',
properties: {
host: {
$ref: '#/definitions/host',
},
port: {
$ref: '#/definitions/port',
},
},
required: ['host', 'port'],
additionalProperties: false,
},
previousBlocksOffset: {
type: 'integer',
minimum: 0,
},
},
required: ['enabled', 'docker', 'api', 'previousBlocksOffset'],
additionalProperties: false,
},
gateway: {
type: 'object',
properties: {
Expand Down
4 changes: 4 additions & 0 deletions packages/dashmate/src/config/getConfigProfilesFactory.js
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,10 @@ export default function getConfigProfilesFactory() {
profiles.push('platform');
}

if (config.get('platform.quorumList.enabled')) {
profiles.push('platform-quorum');
}

return Array.from(new Set(profiles));
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,9 @@ export default function setupLocalPresetTaskFactory(
// Disable platform for the seed node
config.set('platform.enable', false);
config.set('platform.drive.tenderdash.mode', 'seed');

// Enable quorum list sidecar for SDK local testing
config.set('platform.quorumList.enabled', true);
} else {
config.set('description', `local node #${nodeIndex}`);

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,10 @@ export default function assertLocalServicesRunningFactory(assertServiceRunning)
for (const config of configGroup) {
if (config.name === 'local_seed') {
await assertServiceRunning(config, 'core', expected);

if (config.get('platform.quorumList.enabled')) {
await assertServiceRunning(config, 'quorum_list', expected);
}
} else {
for (const serviceName of Object.keys(SERVICES)) {
await assertServiceRunning(config, serviceName, expected);
Expand Down
2 changes: 1 addition & 1 deletion packages/js-evo-sdk/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
"scripts": {
"build": "rm -rf dist && tsc -p tsconfig.json && webpack --config webpack.config.cjs",
"lint": "eslint \"src/**/*.ts\" \"tests/**/*.*js\"",
"test": "yarn run test:unit && yarn run test:functional",
"test": "yarn run test:unit",
"test:unit": "mocha tests/unit/**/*.spec.mjs && karma start ./tests/karma/karma.conf.cjs --single-run",
"test:functional": "mocha tests/functional/**/*.spec.mjs --exit --timeout 90000 && karma start ./tests/karma/karma.functional.conf.cjs --single-run"
},
Expand Down
Loading