|
| 1 | +# Starship End-to-End Testing Guide |
| 2 | +Welcome to the Starship E2E Testing Guide. |
| 3 | +This guide outlines how to set up, run, and add test cases for the Starship project. |
| 4 | +The following instructions will enable you to execute tests locally, customize configurations, and integrate tests into GitHub Actions for continuous integration. |
| 5 | + |
| 6 | +## Table of Contents |
| 7 | +* [Overview](#overview) |
| 8 | +* [Setup](#setup) |
| 9 | +* [Running Tests Locally](#running-tests-locally) |
| 10 | + * [Setup the Kubernetes Cluster](#setup-the-kubernetes-cluster) |
| 11 | + * [Install the Helm Chart](#install-the-helm-chart) |
| 12 | + * [Port Forward Services](#port-forward-services) |
| 13 | + * [Run Tests](#run-tests) |
| 14 | +* [Available Configurations](#available-configurations) |
| 15 | + * [GitHub Actions Integration](#github-actions-integration) |
| 16 | + * [Adding Tests to PR Workflow](#adding-tests-to-pr-workflow) |
| 17 | + |
| 18 | +## Overview |
| 19 | +This guide is intended for developers working on the Starship project to run E2E tests locally and contribute test cases. |
| 20 | +Tests can be run multiple times to ensure idempotency. A variety of configurations in the `configs/` directory allows testing different setups. |
| 21 | + |
| 22 | +## Setup |
| 23 | +Ensure you have the following prerequisites: |
| 24 | +* Kubernetes setup (recommended: Docker Desktop with kubernetes support for local setups): [Docker Desktop](https://www.docker.com/products/docker-desktop/ |
| 25 | +* `kubectl`: [Installation Guide](https://kubernetes.io/docs/tasks/tools/) |
| 26 | +* `helm`: [Installation Guide](https://helm.sh/docs/intro/install/) |
| 27 | +* `yq`: [Installation Guide](https://mikefarah.gitbook.io/yq/v3.x) |
| 28 | +* `jq`: [Installation Guide](https://jqlang.github.io/jq/download/) |
| 29 | + |
| 30 | +## Running Tests Locally |
| 31 | +1. Setup the Kubernetes Cluster: |
| 32 | + |
| 33 | + To start a local Kubernetes cluster with Docker Desktop follow steps: |
| 34 | + * From the Docker Dashboard, select the Settings. |
| 35 | + * Select Kubernetes from the left sidebar. |
| 36 | + * Next to Enable Kubernetes, select the checkbox. |
| 37 | + * Select Apply & Restart to save the settings and then click Install to confirm. |
| 38 | + |
| 39 | +2. Install the Helm Chart: |
| 40 | + |
| 41 | + Once the cluster is ready, install the Helm chart with a specific configuration file, for example, `configs/two-chain.yaml`: |
| 42 | + ```bash |
| 43 | + make install HELM_FILE=configs/two-chain.yaml |
| 44 | + ``` |
| 45 | + |
| 46 | +3. Check pod statues: |
| 47 | + |
| 48 | + Check the status of the pods to ensure everything is running before proceeding: |
| 49 | + ```bash |
| 50 | + kubectl get pods |
| 51 | + ``` |
| 52 | + |
| 53 | +4. Port Forward Services: |
| 54 | + |
| 55 | + After confirming the pods are in the Running state, forward the necessary ports: |
| 56 | + ```bash |
| 57 | + make port-forward HELM_FILE=configs/two-chain.yaml |
| 58 | + ``` |
| 59 | + |
| 60 | +5. Run Tests: |
| 61 | + |
| 62 | + Run the test suite with the following command: |
| 63 | + ```bash |
| 64 | + make test HELM_FILE=configs/two-chain.yaml |
| 65 | + ``` |
| 66 | + These tests are designed to be idempotent, meaning you can run make test multiple times as needed. |
| 67 | + |
| 68 | +## Available Configurations |
| 69 | +In the `configs/` directory, you’ll find various configuration files that specify different test setups. |
| 70 | +Some examples include: |
| 71 | +``` |
| 72 | +one-chain.yaml |
| 73 | +multi-validator.yaml |
| 74 | +simapp.yaml |
| 75 | +two-chain.yaml |
| 76 | +``` |
| 77 | + |
| 78 | +Feel free to test with any of these configurations by adjusting the `HELM_FILE` parameter. |
| 79 | + |
| 80 | +## GitHub Actions Integration |
| 81 | +### Adding Tests to PR Workflow |
| 82 | +The workflow for PR tests is defined in `.github/workflows/pr-tests.yaml`. |
| 83 | +This workflow runs on pull requests to the `main` and `release/*` branches, as well as on a daily schedule. |
| 84 | + |
| 85 | +### Resource Limitations |
| 86 | +Since GitHub Actions runners have limited resources, be cautious when adding resource-intensive configurations to the PR tests. |
| 87 | +Only the following configurations are currently enabled in the PR tests: |
| 88 | +``` |
| 89 | +one-chain-custom-scripts.yaml |
| 90 | +one-chain.yaml |
| 91 | +multi-validator.yaml |
| 92 | +simapp.yaml |
| 93 | +multi-validator-starship-faucet.yaml |
| 94 | +``` |
| 95 | + |
| 96 | +To add a new test configuration, simply include it in the `matrix.config-file` section of the `pr-test` job in `.github/workflows/pr-tests.yaml`. |
| 97 | +Ensure the configuration is efficient enough for GitHub Actions runners. |
0 commit comments