docs: running qualification test (#1147)

docs/qualification/running-qualification.md

@@ -0,0 +1,61 @@
+# How to run qualification
+
+There are three ways to run the test and a couple of environment variables that a user should be aware of.
+
+??? tip "Suggested way of running"
+    Since this is a long test and bazel tends to be pretty heavy on the system it is's a good idea to run the test in a devenv.
+    To do that you should follow the IDX guide on how to create a devenv. Once the devenv is deployed you can:
+    ```bash
+    eval `ssh-agent -s`
+    ssh-add ~/.ssh/ssh_key_that_has_access_to_k8s_repo
+    ssh devenv
+    cd /to/root/of/ic/repo
+    ./ci/container/container-run.sh
+    ...
+    ```
+
+## Running using `ict`
+`ict` is a go-lang tool that was developed to help developers run tests with less friction in having to write long test names. To use `ict` one can spin up a new shell and:
+```bash
+cd /to/root/of/ic/repo
+./ci/container/container-run.sh
+ict test guest_os_qualification -- --test_timeout=7200 --keep_going
+```
+
+## Running using `bazel test`
+Spin up a new shell and:
+```bash
+cd /to/root/of/ic/repo
+./ci/container/container-run.sh
+bazel test //rs/tests/dre:guest_os_qualification --config=systest --cache_test_results=no --test_env=IC_DASHBOARDS_DIR=/path/to/k8s_repo/bases/apps/ic-dashboards --sandbox_add_mount_pair=/path/to/k8s_repo/bases/apps/ic-dashboards --test_timeout=7200 --keep_going
+```
+
+## Environment variables
+- `OLD_VERSION`: specifies the starting version for a testnet. If it's not specified it will default to the version specified in `tests/mainnet_revision.json`.
+
+## How does it work
+Qualification test consits of multiple steps. If we have versions `A` and `B` where `A` is already deployed to the network and `B` is a version that is being qualified the steps would look like the following:
+
+1. Ensure that version `A` is on all subnets
+2. Ensure that version `A` is on all unassigned nodes
+3. Upgrading phase:
+
+    1. Deploy version `B` to application subnets
+    2. Deploy version `B` to system subnets
+    3. Deploy version `B` to unassigned nodes
+
+4. Testing phase:
+
+    1. Run performance tests
+    2. Run xnet tests
+
+5. Downgrade phase:
+
+    1. Deploy version `A` to application subnets
+    2. Deploy version `A` to system subnets
+    3. Deploy version `A` to unassigned nodes
+
+6. Testing phase:
+
+    1. Run performance tests
+    2. Run xnet tests

mkdocs.yml

@@ -20,6 +20,7 @@ nav:
       - Create a neuron: create-a-neuron.md
       - Make a release: make-release.md
       - Submit NNS proposal: nns-proposals.md
+      - Run qualification: qualification/running-qualification.md
       - Decentralization: decentralization.md
       - Release controller: release.md
       - Subnet decentralization: subnet-decentralization-whatif.md