From fb23918d86290a83eb9e6eb36481f0c74cc4f352 Mon Sep 17 00:00:00 2001
From: Nikola Milosavljevic <73236646+NikolaMilosa@users.noreply.github.com>
Date: Tue, 10 Dec 2024 13:10:27 +0100
Subject: [PATCH] docs: running qualification test (#1147)

---
 docs/qualification/running-qualification.md | 61 +++++++++++++++++++++
 mkdocs.yml                                  |  1 +
 2 files changed, 62 insertions(+)
 create mode 100644 docs/qualification/running-qualification.md

diff --git a/docs/qualification/running-qualification.md b/docs/qualification/running-qualification.md
new file mode 100644
index 000000000..6435379f0
--- /dev/null
+++ b/docs/qualification/running-qualification.md
@@ -0,0 +1,61 @@
+# How to run qualification
+
+There are three ways to run the test and a couple of environment variables that a user should be aware of.
+
+??? tip "Suggested way of running"
+    Since this is a long test and bazel tends to be pretty heavy on the system it is's a good idea to run the test in a devenv.
+    To do that you should follow the IDX guide on how to create a devenv. Once the devenv is deployed you can:
+    ```bash
+    eval `ssh-agent -s`
+    ssh-add ~/.ssh/ssh_key_that_has_access_to_k8s_repo
+    ssh devenv
+    cd /to/root/of/ic/repo
+    ./ci/container/container-run.sh
+    ...
+    ```
+
+## Running using `ict`
+`ict` is a go-lang tool that was developed to help developers run tests with less friction in having to write long test names. To use `ict` one can spin up a new shell and:
+```bash
+cd /to/root/of/ic/repo
+./ci/container/container-run.sh
+ict test guest_os_qualification -- --test_timeout=7200 --keep_going
+```
+
+## Running using `bazel test`
+Spin up a new shell and:
+```bash
+cd /to/root/of/ic/repo
+./ci/container/container-run.sh
+bazel test //rs/tests/dre:guest_os_qualification --config=systest --cache_test_results=no --test_env=IC_DASHBOARDS_DIR=/path/to/k8s_repo/bases/apps/ic-dashboards --sandbox_add_mount_pair=/path/to/k8s_repo/bases/apps/ic-dashboards --test_timeout=7200 --keep_going
+```
+
+## Environment variables
+- `OLD_VERSION`: specifies the starting version for a testnet. If it's not specified it will default to the version specified in `tests/mainnet_revision.json`.
+
+## How does it work
+Qualification test consits of multiple steps. If we have versions `A` and `B` where `A` is already deployed to the network and `B` is a version that is being qualified the steps would look like the following:
+
+1. Ensure that version `A` is on all subnets
+2. Ensure that version `A` is on all unassigned nodes
+3. Upgrading phase:
+
+    1. Deploy version `B` to application subnets
+    2. Deploy version `B` to system subnets
+    3. Deploy version `B` to unassigned nodes
+
+4. Testing phase:
+
+    1. Run performance tests
+    2. Run xnet tests
+
+5. Downgrade phase:
+
+    1. Deploy version `A` to application subnets
+    2. Deploy version `A` to system subnets
+    3. Deploy version `A` to unassigned nodes
+
+6. Testing phase:
+
+    1. Run performance tests
+    2. Run xnet tests
diff --git a/mkdocs.yml b/mkdocs.yml
index b6b963bee..89635d383 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -20,6 +20,7 @@ nav:
       - Create a neuron: create-a-neuron.md
       - Make a release: make-release.md
       - Submit NNS proposal: nns-proposals.md
+      - Run qualification: qualification/running-qualification.md
       - Decentralization: decentralization.md
       - Release controller: release.md
       - Subnet decentralization: subnet-decentralization-whatif.md