Skip to content

Home

Jump to bottom
Martina Scholz edited this page Oct 20, 2024 · 10 revisions

Welcome to the Release Testing Tool Wiki

Release testing plays a crucial role in the Joomla development cycle. It ensures that new versions of the CMS are stable, almost bug-free, and ready for public use. Minor updates of the Joomla CMS are released as minor updates every six (6) Tuesdays (although the schedule may vary) - Joomla! Project Roadmap.
One week before the official release, a Release Candidate (RC) version is made available for testing and community participation is essential.

Tip

The more people who test, the better the quality of the final release.

However, manually testing every feature and step can be time-consuming and resource-intensive. That's why we've tried to develop a simple and intuitive solution to streamline the testing process. Our goal is to make testing as easy and accessible as possible.

With our project, you can easily test either a brand-new Joomla site or your own existing site. Use the restore option to quickly create a local clone of your Joomla site or simply use the remote option, allowing for fast and efficient testing in a controlled environment.

This repository takes care of all the dependencies for you, so there's no need to worry about configuring them manually. We've made the process straightforward, so you can focus on testing, not troubleshooting.

Pre Requisites

  • Docker: This application relies on Docker to create isolated environments and manage its components efficiently. Please install Docker by following the official documentation.

  • WSL2 (Windows Subsystem for Linux 2): If you're using Windows, WSL2 is essential for running Docker containers seamlessly. First you need to install your preferred Distro into WSL2. Then start the WSL Console where you find yourself in a full Linux terminal.

Open a terminal or command prompt and run the following command to check if Docker and Docker Compose are installed and running correctly:

docker version && docker compose version

Installation

git clone https://github.com/joomla-projects/release-testing.git

Windows

  • Click on the start.bat file to start working in a WSL console.

Testing with the Joomla! Pre-Release Testing Tool

Joomla E2E Test Suite Options

grafik

1. Remote

This option allows you to run end-to-end (E2E) tests on a remote Joomla site using Cypress.

  • Usage: Select this option if you want to test a website hosted remotely (e.g., https://example.com).
  • How to choose: 1 -> Enter when prompted to select this option.

2. Local

This option enables you to run E2E tests on a local Joomla site installed within this testing environment with Cypress.

  • Usage: Use this when you want to test a Joomla instance that runs in the local web server which is delivered with this test suite and runs automatically within the Docker environment as local environment.
  • How to choose: 2 -> Enter to run tests on your local site.

3. Manage

This option provides tools to manage your local Joomla sites that are running within a Dockerized local web server included with this test suite. You can create new Joomla sites, restore sites from backups, or delete existing ones directly within this environment.

  • Usage: Choose this if you need to handle site management tasks such as creating or restoring Joomla sites.
  • How to choose: 3 -> Enter to access site management features.

4. Shutdown

If you encounter any issues while running tests or managing sites, this option allows you to shutdown the current Docker container stack and start the process again. This is useful for resetting the environment or troubleshooting errors.

  • Usage: Select this if you need to restart the testing environment due to problems or errors or if you want to temporarily pause your testing session and free up resources on your computer for the time being.
  • How to choose: 4 -> Enter to shutdown the container stack.

5. Quit

This option allows you to exit the Joomla E2E Test Suite and shutting down your docker containers gracefully.

  • Usage: Choose this option when you are done with testing want to close the program.
  • How to choose: 5 -> Enter to exit the suite.
joomla-cypress-release-testing.1.mp4

Overview Page

You can reach an overview of all available tools via http://localhost:8080 after starting your testing session.

grafik

Advanced configuration options

The .env file, located in the .tools/.env folder, contains environment variables that are used to configure various aspects of the test suite.
These variables are loaded into the application at runtime, so it is possible to customise the behaviour of the application to suit your needs without modifying the code.

We have applied some general settings to this file, which should be suitable for most users. However, if you require special settings or experience any issues with the settings on your system, you can change most of them to your needs.

IMAGE_NAME_TAG=joomla-cypress-included:13.14.2v4
WEB_IMAGE_NAME_TAG=joomla-cypress-web:php-8.3
WEB_LOCAL_PORT=8080
WEB_LOCAL_PORT_SSL=4433
JC_OVERRIDE_CONFIG=yes
JC_OVERRIDE_BACKUP=yes
# Options smart, always
NPM_DEPS_INSTALL=smart
# Options pnpm, npm
PACKAGE_MANAGER_PREFS=pnpm
# Default test project
DEFAULT_TEST_PROJECT=cms
  • The IMAGE_NAME_TAG and WEB_IMAGE_NAME_TAG variables specify the Docker image names and tags for the services used in the docker-compose.yml to run the testing suite.
    • IMAGE_NAME_TAG is set to joomla-cypress-included:13.14.2v4, the current version of the customized image for this testing suite that includes Cypress and other tools useful for your testing session.
    • WEB_IMAGE_NAME_TAG is set to joomla-cypress-web:php-8.3, which refers to image running PHP version 8.3.

Important

It is not advisable to make changes to this images. The joomla-cypress-included image is built directly on your machine with the associated build script. Furthermore, additional modifications are required to implement custom Docker image to run with this testing tool.

  • The WEB_LOCAL_PORT and WEB_LOCAL_PORT_SSL variables define the local ports for accessing the services running.
    • WEB_LOCAL_PORT is set by default to '8080', the port used for HTTP traffic.
    • WEB_LOCAL_PORT_SSL is set by default to '4433', used for HTTPS traffic.

Note

If you are having problems starting up the testing tool and other services are already using the ports on your computer, you can safely enter different ports here and restart.

Tip

If ports 80 and 443 are not already assigned on your system, you can enter them here. This will save you having to type the ports in the address bar of your browser.

  • JC_OVERRIDE_CONFIG and JC_OVERRIDE_BACKUP.
    Normally, the test system automatically ensures that the correct Cypress configuration for running your tests is available in the project folder and is also assigned to the correct test page. Information such as the user name, page URL, passwords and tokens can be adjusted here. A backup file of your previous configuration is also created so that you can compare it if you have any problems. You can disable both options or each one individually by setting no if you require a customised configuration for your tests.

Note

If the initial check for the existence of cypress.config.mjs fails (i.e., the file does not exist in your testing project folder), the default one will be copied independent of the setting to ensure that the necessary configuration is in place.

  • NPM_DEPS_INSTALL variable is set to smart by default. This indicates that if you initiate your testing project on multiple occasions within a 24-hour period, the dependencies will not be updated on each occasion. Alternatively, you may choose to set it to always.

  • PACKAGE_MANAGER_PREFS is set to pnpm as the default package manager for the cypress docker container. pnpm is known for its efficient handling of node modules and disk space, which should streamline the startup process for the Docker container. If you prefer using npm or want to troubleshoot the startup you can safely modify this setting by entering npm instead.

  • DEFAULT_TEST_PROJECT variable is set to cms, this is the default test project. You will find the tests in the cms subfolder, which already contains tests.
    If you are actively working with the system and creating your own test projects/subfolders, you can easily customise the default option here.

Custom options per testing-project

You can also have custom options for each test project, which will override the default configured variables that are defined in the .env file.

TODO --> je2e.env

Testing in headless mode and generate HTML Reports

joomla-cypress-release-testing3.mp4

Removing a local testing site

joomla-cypress-release-testing5.mp4

Testing on your copied remote sites with test-data reset via API

joomla-cypress-release-testing6_compressed.mp4

Demonstration of recording click paths with cypress studio

joomla-cypress-release-testing2_compressed.mp4
Clone this wiki locally