Writing Tests

Test Concept

Some sort of test concept is needed to have a clear idea of what should be tested and how.

Depending on the requirements and the complexity of the project, the test concept should be more or less detailed.

If there are no clear directives that say otherwise, I find that the easiest and most forward way is to make a bullet point list of exactly what should be tested for each page and the expected behaviour.
If use-cases have similar testing requirements and expected behaviour, they can be grouped together.

Instead of putting a lot of effort in trying to think of all possible cases in advance, I would write down the ones that come to mind and then, while implementing add the new ones that may arise.

An example of such a list for this project can be found in the testing cheatsheet.

Initialization

Prerequisites

PHPUnit and Test Traits

The tests are written with the PHPUnit testing framework which is the de-facto standard for unit testing in PHP.

For mocking, database handling, http requests, email assertions and more, the helper traits selective/test-traits are used.

composer require --dev phpunit/phpunit selective/test-traits

Configuration

The phpunit.xml file in the root directory contains the configuration for PHPUnit. See here.

The environment configuration with test-specific values is located in config/env.test.php:

// Enable ErrorException for notices and warnings
$settings['error']['display_error_details'] = true;

// ...

Test Database

Before the tests that involve the database are run, an empty test database must be created and the name added to the env.test.php file:

// ... 

// Database for integration testing must include the word "test"
$settings['db']['database'] = 'slim_example_project_test';

The tables are created during the first setUp() call when running the test suite. Once they're created, they are truncated for each further test.

Tables are created with the schema.sql file which is generated to reflect the current database by the SqlSchemaGenerator via the specific composer command composer schema:generate or when the migration files are generated.

Setup

Before each test function, the application must be bootstrapped and the database cleared if required. This is done with the setUp() function which is called automatically by PHPUnit before each test function. The setup logic is the same for all tests, which means it can be extracted into a trait. The same goes for the tearDown() function which is called after each test function.

The AppTestTrait located in tests/Traits contains the setUp() and tearDown() functions.

The setUp() function below also sets the session to a memory session and inserts the user roles into the database by default.

File: tests/Traits/AppTestTrait.php

<?php

namespace App\Test\Traits;

use App\Test\Fixture\UserRoleFixture;
use Cake\Database\Connection;
use DI\Container;
use Odan\Session\MemorySession;
use Odan\Session\SessionInterface;
use Psr\Container\ContainerInterface;
use Slim\App;
use UnexpectedValueException;

trait AppTestTrait
{
    protected ContainerInterface $container;
    
    protected App $app;

    protected function setUp(): void
    {
        // Start slim app
        $this->app = require __DIR__ . '/../../config/bootstrap.php';

        // Set $this->container to container instance
        $container = $this->app->getContainer();
        if ($container === null) {
            throw new UnexpectedValueException('Container must be initialized');
        }
        $this->container = $container;

        // Set memory sessions
        $this->container->set(SessionInterface::class, new MemorySession());

        // If setUp() is called in a testClass that uses DatabaseTestTrait, the method setUpDatabase() exists
        if (method_exists($this, 'setUpDatabase')) {
            // Check that database name from config contains the word "test"
            // This is a double security check to prevent unwanted use of dev db for testing
            if (!str_contains($container->get('settings')['db']['database'], 'test')) {
                throw new UnexpectedValueException('Test database name MUST contain the word "test"');
            }

            // Create tables
            $this->setUpDatabase($container->get('settings')['root_dir'] . '/resources/schema/schema.sql');

            if (method_exists($this, 'insertFixtures')) {
                // Automatically insert user roles
                $this->insertFixtures([UserRoleFixture::class]);
            }
        }
    }

    protected function tearDown(): void
    {
        if (method_exists($this, 'setUpDatabase')) {
            $connection = $this->container->get(Connection::class);
            $connection->rollback();
            $connection->getDriver()->disconnect();
            if ($this->container instanceof Container) {
                $this->container->set(Connection::class, null);
                $this->container->set(\PDO::class, null);
            }
        }
    }
}

In each test class, the AppTestTrait is included with the use keyword at the top of the class.

// ...

class ExampleTest extends TestCase
{
    use AppTestTrait;
    // ...
}

Unit Tests

Unit tests are located in the tests/Unit directory.

To test individual units of code (e.g. functions, classes, modules) in isolation, the parts around the tested unit are replaced by test doubles (mocks, stubs, etc.) with predefined return values. They are fake objects that need to be configured to contain specific methods and return values.
Mocks, unlike stubs, can be programmed to expect specific method calls and parameters to verify that the tested unit interacts with the mock as expected.

The MockTestTrait.php provides the mock() function which returns a mock object of the given class and automatically adds it to the container.
Now instead of the real class, the mock object is injected and used by tested class.

The detailed documentation on how the stubs and mock can be created and configured can be found here.

Here is a list of some of the frequently used functions:

• method($name) - Sets the name of the mocked method
• willReturn($value) - Sets the return value of the mocked method
• willReturnOnConsecutiveCalls($value1, $value2, ...) - Sets the return value of the mocked method to the given values in the given order
• with($value) - Sets the expected parameter value of the mocked method
• expects($count) - Sets the expected number of times the mocked method is called. $count can be one of the following:
  • never() - The mocked method is expected to be called never
  • once() - The mocked method is expected to be called once
  • exactly($count) - The mocked method is expected to be called exactly (int) $count times

Unit Test Example

Assuming exampleFunction() is the function to test in the ExampleClass class and its logic is that it retrieves a string from the database with the PDO query function and returns this string after transforming it to uppercase. The test could look like this:

File: tests/Unit/ExampleClassTest.php

<?php

namespace App\Test\Unit;

use App\Test\Traits\AppTestTrait;
use Selective\TestTrait\Traits\MockTestTrait;
use PHPUnit\Framework\TestCase;

class ExampleClassTest extends TestCase
{
    use AppTestTrait;
    use MockTestTrait;

    public function testExample(): void
    {
        // Mock the PDO class and add it to the container
        $pdoMock = $this->mock(\PDO::class);

        // Configure the mock to return "hello world" when the query() function is called 
        // and expect the function to be called once
        $pdoMock->method('query')
            ->willReturn('hello world')
            ->expects(self::once());

        // Get the real instance of the class to test
        $exampleClass = $this->container->get(ExampleClass::class);

        // Call the function to test
        $result = $exampleClass->exampleFunction();

        // Assert that the result is the expected value
        $this->assertSame('HELLO WORLD', $result);
    }
}

Integration Tests

The folder tests/Integration contains the integration test cases.

To test the overall behavior of the application, an HTTP request to a route is made with a specific request method and request body that will traverse all the layers of the application.

Requests

Requests can be created with HttpTestTrait.php and HttpJsonTestTrait.php. They provide the following functions:

• createRequest() - Creates a request object and accepts the parameters $method, $uri and $serverParams
• createFormRequest() - Creates a request object, adds the form data to the request body and sets the Content-Type header to application/x-www-form-urlencoded
• HttpJsonTestTrait: createJsonRequest() - Creates a request object, adds the JSON data to the request body and sets the Content-Type header to application/json.
  Note: the HttpTestTrait must be included as well to use this function.

The functions above expect the $uri which is the full url to the route. To reference routes by their name, the urlFor() function from the RouteTestTrait can be used.

To make the request and get the response, $this->app->handle() is called with the request as argument. $app is the instance of the application that is bootstrapped in the AppTestTrait.

// ...

class TestActionTest extends TestCase
{
    use AppTestTrait;
    use HttpTestTrait;
    use RouteTestTrait;
    // ...
    
    public function testAction(): void
    {
        $request = $this->createRequest('GET', $this->urlFor('routeName'))
        $response = $this->app->handle($request);
        
        // ...
    }
}

Asserting the Response

After the request is made, the response can be tested with assertions.

Status code

The status code can be retrieved with $response->getStatusCode() which can be verified with assertSame and the expected code.

self::assertSame(200, $response->getStatusCode());

Response header

The response header can be accessed with $response->getHeaderLine($headerName) and then compared with an expected value.

// Assert redirection to login page
self::assertSame($this->urlFor('login-page'), $response->getHeaderLine('Location'));

Response body

To assert that the response body contains a specific string, the assertResponseContains() function of the HttpTestTrait can be used.

$this->assertResponseContains('Hello World', (string)$response->getBody());

JSON response body

To verify that the returned JSON data is an exact match to an expected array, the HttpJsonTestTrait provides the assertJsonData() function.

$this->assertJsonData(['key' => 'value'], $response);

For more advanced assertions, the JSON data from the response can be accessed as an array with $this->getJsonData($response).

The HttpJsonTestTrait also provides a function assertJsonContentType to assert the response content type header.

Data Providers

To test a use-case under different conditions, the same test logic can be run with different data. Each run comes with a different set of data and possibly a different expected result.

Data providers are static functions in a "Provider" class that return an array of arrays with the different data that should be used for the different iterations of the test function.

The @dataProvider annotation can be used to specify the function that will provide the data in the parameters of the test function.

File: tests/Provider/Example/ExampleProvider.php

<?php

namespace App\Test\Provider\Example;

class ExampleProvider
{
    public static function provideExampleData(): array
    {
        return [
            // Provides 0 in the first parameter and 1 in the second
            [0, 1],
            // The data sets can be named with string keys for a more verbose output as
            // it will contain the name of the dataset that breaks a test
            'one' => [1, 2],
            // The array values can also have string keys, but they don't affect 
            // the test function parameters
            'two' => ['input' => 2, 'expected' => 3],
        ];
    }
}

The following test function will be run 3 times. In the first iteration, the $input parameter will be 0 and $expected 1, in the second 1 and 2 and in the third 2 and 3.

File: tests/Integration/Example/ExampleTest.php

<?php

namespace App\Test\Integration\Example;

use PHPUnit\Framework\TestCase;

class ExampleTest extends TestCase
{
    /**
     * @dataProvider \App\Test\Provider\Example\ExampleProvider::provideExampleData
     */
    public function testExample(int $input, int $expected): void
    {
        $this->assertSame($expected, $input + 1);
    }
}

Fixtures

For a lot of requests, pre-existing data is required in the database.
For example, to test the modification of a resource, it has to exist prior to making the update request.
An authenticated user is also required to get past the authentication middleware.

Fixture Classes

Fixtures are classes that hold example data that can be inserted into the database. Each table has its own fixture class that implements the FixtureInterface.
The fixtures are located in the tests/Fixture directory.

Each fixture has a property $table with the table name and an array $records with the default data to insert as well as getters for both properties.

File: tests/Fixture/ExampleFixture.php

<?php

namespace App\Test\Fixture;

class ExampleFixture implements FixtureInterface
{
    // Table name
    public string $table = 'example';

    // Database records
    public array $records = [
        [
            'id' => 1,
            'field_1' => 'value_1',
            'field_2' => 'value_2',
        ],
        [
            'id' => 2,
            'field_1' => 'value_1',
            'field_2' => 'value_2',
        ],
    ];
    
    public function getTable(): string
    {
        return $this->table;
    }

    public function getRecords(): array
    {
        return $this->records;
    }
}

Inserting Fixtures

Different use cases require different data. To define custom data to be inserted along with the default data of the fixture, the FixtureTestTrait provides the insertFixturesWithAttributes() function. Parameters are the instance of the fixture class and optionally an array of attributes.

The attribute array can either contain fields and values to insert for one row (e.g. ['field_name' => 'expected_value', ]) or an array of such arrays (e.g. [['field_name' => 'expected_value'], ['field_name' => 'expected_value'], ]) which will insert multiple rows.

Not all fields of the table need to be specified. The default values of the fixture will be used for the unspecified fields.

The function returns an array with the inserted data including the auto-incremented id or a 2-dimensional array if more than one row was inserted.

    // ...
    use FixtureTestTrait;
    
    public function testAction(): void
    {
        // Insert the fixture with the default values
        $rowData = $this->insertFixtureWithAttributes(new ExampleFixture());
        
        // Insert the fixture with the given attributes
        $rowData = $this->insertFixtureWithAttributes(new ExampleFixture(), ['field_1' => 'value_1', ]);
        
        // Insert 2 rows with the given attributes 
        $rowsData = $this->insertFixtureWithAttributes(
            new ExampleFixture(), [['field_1' => 'value_1'], ['field_1' => 'value_2']]
        );
        
        // ...
    }
}

The FixtureTestTrait uses the DatabaseTestTrait for the interaction with the database.

Inserting Fixtures With User Roles

User roles are inserted by default in AppTestTrait if the DatabaseTestTrait is used in a test class.

The ids of the user roles are not known in the data providers or test functions.
Instead of hard coding them, the AuthorizationTestTrait allows the user_role_id to be referenced as an Enum case and converted to its id in the test function with one of the following functions:

• getUserRoleId(UserRole $userRole) - returns the id of the given user role enum case.
• addUserRoleId(array $userAttr) - converts user role enum to the id. It accepts an array of attributes and replaces the user_role_id value if it's an enum case with the corresponding id and returns the same array.
• insertUserFixturesWithAttributes(array &$authenticatedUserAttr, ?array &$userAttr) - inserts up to two user fixtures with the given attributes replacing the user_role_id value if it's an enum case with the corresponding id.

Function insertUserFixturesWithAttributes()

This function is most useful when testing authorization cases that require an authenticated user and another user, e.g. that is linked to the ressource (owner).
It might be the authenticated user itself or another one. If the authenticated user attributes are the same as the other user, only one user is inserted.

For cleaner code, the function accepts the user attribute parameters as &references which means that the original variable from the calling function is modified without them having to be returned.
The function insertUserFixturesWithAttributes() replaces the user attributes from the parameters with the inserted user row data.

    // ...
    use DatabaseTestTrait;    
    use FixtureTestTrait;
    use AuthorizationTestTrait;
    
    /**
    * @dataProvider \App\Test\Provider\Example\ExampleProvider::provideExampleData
    * 
    * @param array $userLinkedToResourceRow e.g. ['user_role_id' => UserRole::ADMIN]
    * @param array $authenticatedUserRow e.g. ['user_role_id' => UserRole::ADVISOR]
    * @return void
     */
    public function testAction(array $userLinkedToResourceRow, array $authenticatedUserRow): void
    {
        
        // Insert authenticated user and user linked to resource with given attributes containing the user role
        $this->insertUserFixturesWithAttributes($userLinkedToResourceRow, $authenticatedUserRow);
       
       // $userLinkedToResourceRow and $authenticatedUserRow now contain the inserted user data
       // including the auto-incremented id
       $authenticatedUserId = $authenticatedUserRow['id'];
       
        // ...
    }
}

Database Assertion

To verify that the data in the database is changed or inserted as expected after a request, the DatabaseTestTrait from the test-traits package provides practical functions to assert the database content.

• assertTableRow() - Asserts that a row in the database contains the expected values
• assertTableRowEquals() - Asserts that a row in the database contains the expected values without type checking
• assertTableRowValue() - Asserts that a specific field of a row has the expected value
• assertTableRowCount() - Asserts that a table has a specific number of rows
• getTableRowCount() - Returns the number of rows in a table
• getTableRowById() - Returns the row with the given id from the given table or throws an exception if it does not exist
• findTableRowById() - Returns the row with the given id from the given table or an empty array if it does not exist

Assert table row

To assert that a row in the database contains the expected values, the function assertTableRow() or assertTableRowEquals() can be used. The first parameter is the array of expected fields and values, the second one is the table name and the third the id of the row to check.

With the code below, the function asserts that the row with id 1 in the table example has the value value_1 in the field_1 and 42 in the field_2 without considering the other fields.

// Only passes if the values in the database have the same type as the expected values
$this->assertTableRow(['field_1' => 'value_1', 'field_2' => 42], 'example', 1);
// Type of the value is not considered
$this->assertTableRowEquals(['field_1' => 'value_1', 'field_2' => '42'], 'example', 1);

Advanced Functions

The DatabaseExtensionTestTrait provides an additional set of functions to further retrieve and assert data from the database.

It contains the following functions:

• findTableRowsByColumn() - Returns an array of rows from a table where the specified column has the given value
• findTableRowsWhere() - Returns an array of rows from the given table with a custom where clause
• findLastInsertedTableRow() - Returns the last inserted row from the given table
• assertTableRowsByColumn() - Asserts that the rows have the expected values where the given column has a certain value