Skip to content

Testing and QA workflow

Simona Domnisoru edited this page Jul 1, 2024 · 12 revisions

After a PR has been approved, the testing stage begins. The process for testing (after migrating to continuous delivery) is as follows.

The default procedure

1. Run Cypress tests against applicable backend versions and deploy preview

For PRs that will be merged to master (either directly or through a temporary branch), the default procedure is to add the "testing" label to the PR. This will execute the Cypress tests against applicable backend versions and deploy a build to Netlify. Applicable backend versions in this context is all backend versions the app is compatible with, except the latest dev version (the tests are already running against dev when pushing new commits).

The app will be deployed to Netlify no matter if the tests are successful or not. It will be up to the dev to decide whether to wait for successful Cypress tests before moving on to step 2 (in theory it makes sense to only deploy up on successful Cypress testing, but we opted not to add another wait period for the dev here. The failed tests will be caught eventually).

2. Move JIRA-ticket to testing

After all PRs have been dealt with in the previous step, move the JIRA-ticket to testing and assign it to QA (Geetha).

3.1 Successful QA testing

QA will assign the ticket back to the dev, add a comment (about the successful testing), and set the JIRA-ticket to “Ready to release”. Dev will merge the PRs (including backports) and finally set the JIRA-ticket to “Done”.

3.2 Unsuccessful QA testing / Needs update

If QA finds that an update is needed, they will set the JIRA-ticket to “Needs update” and assign the ticket to the dev. Dev will make the required fixes and restart from step 1 (with the difference being that the labels will have to be removed and re-added to trigger a workflow run and the JIRA-ticket should be moved to "retesting" instead of "testing")

Exceptions

Dependabot PRs: Normally we don't move these through QA, but we should run Cypress tests on them. Transfix PRs: Automatically merged without additional Cypress- or QA-testing.

In addition, some PRs that don't touch production code can be exempted from testing (see skip-e2e-tests below). We will use the devs judgement here.

Optional labels

There are some optional labels that can be used.

e2e-tests: Will trigger Cypress tests only.

skip-e2e-tests: You can choose to opt-out from Cypress testing. You will NOT normally do this, but there are some valid use-cases. For example, if up on running successful tests you push a new commit, the successful test results will be invalidated. If you only made a minor change to non-production code, it might not be useful to rerun all the tests.

preview: The default label used for backports, but also available when merging to master. Will deploy to Netlify.