"In the world of Python, testing is not just a phase. It's a commitment to excellence in craftsmanship."
- Why do we need testing?
- Types of Testing
- Introduction to
unittest - Introduction to
pytest - Mocking and Patching
- Advanced Techniques
- Coverage Analysis
- Applying Testing
- Homework
Testing your code is essential for verifying its correctness, ensuring reliability, and maintaining code quality.
The general idea of testing, is ensuring that your application works as expected. Due to testing you will be able to detect bugs early during the development process and fix them immediately.
As well it can represent sort of documentation that helps developers to understand how the application should work and show the behavior of an application.
There are several types of testing which are used widely in IT society.
| Type of Testing | Description |
|---|---|
| Unit Testing | Testing individual units or components of a program in isolation to verify that each part functions correctly. |
| Integration Testing | Testing the integration or interfaces between components to ensure they work together as expected. |
| Functional Testing | Testing the application against its functional requirements to ensure it behaves as expected from an end-user perspective. |
The best approach would be to cover all of these scenarios using testing tools, but during this lesson I want to focus more on unit testing and tell about available options in Python.
The most common way to test Python application is to use built in unittest library.
Suppose we have the following logic encapsulated within our app:
def add(a, b):
return a + b
def substract(a, b):
return a - b
def divide(a, b):
return a / bAs a developer, several questions should come to mind regarding potential issues:
- What if
addorsubtractare passed non-numeric types, like a string and a number? - What happens when attempting to
divideby zero? - Will the application crash, or will it handle these situations gracefully?
Instead of making assumptions, we can write tests to guarantee that our application behaves as intended under various circumstances.
Unit tests are designed to test individual components, or "units", of your application in isolation.
This means you should be testing the smallest part of an application, like a function or a method, to ensure it does exactly what it’s supposed to do.
import unittest
# Our app
def add(a, b):
return a + b
def subtract(a, b):
return a - b
def divide(a, b):
if b == 0:
raise ValueError("Cannot divide by zero")
return a / b
# Test cases
class TestArithmeticFunctions(unittest.TestCase):
def test_add(self):
"""
- Testing substraction with correct types and assure it works as exptected.
- Testing substraction with incorrect types.
"""
self.assertEqual(add(1, 2), 3)
self.assertEqual(add(-1, 1), 0)
self.assertRaises(TypeError, add, '1', 2)
def test_subtract(self):
"""
- Testing substraction with correct types and assure it works as exptected.
- Testing substraction with incorrect types.
"""
self.assertEqual(subtract(10, 5), 5)
self.assertEqual(subtract(-1, -1), 0)
self.assertRaises(TypeError, subtract, '10', 5)
def test_divide(self):
"""
- Testing division with correct types and assure it works as exptected.
- Testing division by zero.
- Testing division with incorrect types.
"""
self.assertEqual(divide(10, 2), 5)
self.assertRaises(ValueError, divide, 10, 0)
self.assertRaises(TypeError, divide, '10', 2)
# This ensures that the tests will be run if the script is executed
if __name__ == '__main__':
unittest.main()The output shows that tests were successful. Try to modify written the tests and see what happens in case the logic we are trying to test, they should fail explaining what happens wrong, so that the developer can analyse and find out what happens wrong in application.
...
----------------------------------------------------------------------
Ran 3 tests in 0.000s
OK
Testing the OOP applications isn't really different from we have seen already.
Let's create a small BankAccount system and try to write tests for it:
class BankAccount:
def __init__(self):
self.balance = 0
def deposit(self, amount):
if amount <= 0:
raise ValueError("Deposit amount must be positive")
self.balance += amount
def withdraw(self, amount):
if amount > self.balance:
raise ValueError("Insufficient funds")
self.balance -= amount
def get_balance(self):
return self.balanceHere we would need to define the setUp method which creates an instance of BankAccount and it will be easily accessable within TestBankAccountmethod
import unittest
class TestBankAccount(unittest.TestCase):
def setUp(self):
"""
The setUp method is run before each test. It's a good place to set up
a clean environment for each test.
"""
self.account = BankAccount()
def test_deposit(self):
"""
Test that depositing money into the account works correctly.
"""
self.account.deposit(100)
self.assertEqual(self.account.get_balance(), 100)
def test_withdraw(self):
"""
Test that withdrawing money from the account works correctly.
"""
self.account.deposit(200)
self.account.withdraw(50)
self.assertEqual(self.account.get_balance(), 150)
def test_withdraw_insufficient_funds(self):
"""
Test that a ``ValueError`` is raised when trying to withdraw more money
than the balance of the account.
"""
self.account.deposit(50)
with self.assertRaises(ValueError):
self.account.withdraw(100)
def test_deposit_negative_amount(self):
"""
Test that a ``ValueError`` is raised when trying to deposit a negative amount.
"""
with self.assertRaises(ValueError):
self.account.deposit(-20)
def test_initial_balance(self):
"""
Test that the initial balance of the account is zero.
"""
self.assertEqual(self.account.get_balance(), 0)
# This ensures that the tests will be run if the script is executed
if __name__ == '__main__':
unittest.main().....
----------------------------------------------------------------------
Ran 5 tests in 0.000s
As everything in programming MUST have a correct structure, as we have seen it already
Same applies to testing with unittest, each test case class MUST have the following:
- Setup: Prepare the necessary environment or state before the actual tests run. This is often done in a method named
setUp(). - Test Cases: Individual functions that start with the word
test_to inform the test runner what to execute. - Assertions: Statements that check if the output of your code matches the expected result.
- Teardown (Optional): Clean-up steps that need to be taken after the test cases run, often implemented in a method named
tearDown().
Generally, testing should not be crossed with code which will be used in production, a great approach would be to create a tests/ directory and include all tests and files there.
my_project/
│
├── my_project/
│ ├── __init__.py
│ └── bank_account.py
│
└── tests/
├── __init__.py
└── test_bank_account.py
To run the tests, navigate to the directory containing your test script and execute the following command:
python test_bank_account.pyAlternatively, if you want to run all tests across different test files, use:
python -m unittest discoverThere is no really much difference between pytest and unittest tools for testing in terms of serving their purposes. They are both configurable, they both test your application and work pretty much similar.
But, pytest simplifies the writing of small tests yet scales to support complex functional testing. This is why pytest is my favored tool and most of production code is tested within this library.
To begin using pytest, you first need to install it via pip:
pip install pytestOnce installed, writing a test is as straightforward as defining a function prefixed with test_, and then using plain assert statements:
# test_example.py
def add(a, b):
return a + b
def test_add():
assert add(2, 3) == 5
assert add('space', 'ship') == 'spaceship'pytestpytest will automatically discover and run any files named test_*.py or *_test.py in the current directory and its subdirectories.
You can create the same tests for your application, and using pytest in testing your app.
There are some useful commands which I use in production day to day, they can narrow down the tests or provide more information:
| Command | Description |
|---|---|
pytest test_example.py |
Executes all the tests defined in test_example.py. |
pytest tests/ |
Runs all the tests found in the tests directory. |
pytest -v |
Provides a verbose output, detailing all the tests run along with their individual outcomes. |
pytest test_example.py::test_add |
Executes only the test_add function within the test_example.py file. |
pytest -k "expression" |
Runs tests that match the given substring expression, a powerful way to run specific tests within a larger suite. |
pytest --ignore=tests/ |
Ignores the specified directory or file during the test run. |
pytest --maxfail=num |
Exits the test session after num failures, useful for quickly identifying and addressing errors. |
pytest --tb=style |
Modifies the traceback output format for easier debugging. Valid styles include long, short, line, native, and no. |
pytest -x |
Stops the test run on the first failure encountered, which can be useful during development. |
pytest --lf |
Reruns only the tests that failed at the last run, or skips tests that passed. |
Don't hesitate to use anything from this command during development, they can save you a decent ampunt of time, helping with debugging of your application.
Test-Driven Development (TDD) is a modern software development practice where tests are written before the code that will make the tests pass. It follows a simple iterative cycle known as "Red-Green-Refactor":
- Red: Write a test that defines a function or improvements of a function, which should fail because the function isn't implemented yet.
- Green: Implement the function in the simplest way possible to make the test pass.
- Refactor: Clean up the code, while ensuring that tests still pass.
TDD encourages developers to think through their design before writing the code.
- Documentation: The tests serve as live documentation for the application.
- Design: Helps in building a better design as it requires writing testable code.
- Confidence: Each change is made with confidence that the existing features are not broken.
The disadvantage is that sometimes it is really hard to follow this approach, either it is time-consuming, the project is enormously big and it is hard for developer to put it alltogether.
I won't lie, if I have a deadline for the task, sometimes I might neglect using TDD, though no one should do this :)
Let's create an application using TDD approach step by step to see how it should look like:
Suppose we're building a simple calculator application. Our first feature is an addition function. We start by writing a test for this functionality before the function itself exists.
Create a test file test_calculator.py:
# test_calculator.py
def test_addition():
assert add(2, 3) == 5Running this test (pytest test_calculator.py) will result in a failure because the add function does not exist yet. This is the Red phase of TDD.
Now, we write the minimal amount of code needed to pass the test. Create a file calculator.py and implement the add function:
# calculator.py
def add(a, b):
return a + bAnd modify test_calculator.py to import the add function:
# test_calculator.py
from calculator import add
def test_addition():
assert add(2, 3) == 5Running the tests again with pytest, we see that the test now passes. This is the Green phase of TDD.
With the test passing, we can now refactor our code with confidence. This might involve the following:
- Renaming functions for clarity.
- Optimizing the algorithm.
- Restructuring the code for better readability.
After refactoring, run the tests again to ensure nothing has broken. This continuous cycle enhances the code quality over time.
.
----------------------------------------------------------------------
Ran 1 tests in 0.000s
OK
TDD requires strong discipline and may initially slow down development, especially for teams new to this approach. However, the long-term benefits are much more significant, consider adopting this practice into your applications.
Mocking objects simulate the behavior of real objects within your system.
Mock objects can be programmed with predefined responses, making them highly flexible for testing a wide range of scenarios.
- Test components in isolation from the rest of the system.
- Simulate various states of external systems or resources that are difficult or time-consuming to replicate in a test environment.
- Avoid side effects that can interfere with test outcomes.
- Control the test environment by specifying expected inputs and outputs.
Patching (often used in conjunction with mocking) involves temporarily replacing the actual implementation of a class, method, or function with a mock during test execution.
- Functions and methods, to control their outputs or side effects.
- System-level operations, such as file I/O , to prevent tests from altering the system's state.
- Libraries and frameworks, to test your code's interaction with them without requiring the actual implementation to be invoked.
pytest can integrate the unittest.mock module from the Python standard library, enabling the use of mocks and patches in your tests.
With this integration you can simulate complex behaviors and the assertion of interactions with mock objects.
Imagine we have a simple application that sends email notifications to users about important events. For simplicity, let's focus on the function that triggers sending the email.
# notifications.py
def send_email_notification(email_address, message):
if not email_address or not message:
raise ValueError("Email address and message are required")
print(f"Sending email to {email_address}: {message}")
return True# test_notifications.py
import pytest
from unittest.mock import patch
from notifications import send_email_notification
@patch('notifications.print') # Mocking the print function to simulate email sending
def test_send_email_notification(mock_print):
email_address = "user@example.com"
message = "Your application has been approved."
result = send_email_notification(email_address, message)
# Assert the mock (print function here) was called with the expected arguments
mock_print.assert_called_with(f"Sending email to {email_address}: {message}")
assert result == True
def test_send_email_notification_missing_arguments():
# Testing missing arguments to ensure our error handling works
with pytest.raises(ValueError):
send_email_notification("", "")- Mocking: In this example, we mock the
printfunction to simulate the action of sending an email. The@patchdecorator fromunittest.mockis used to replaceprintwith a mock object only for the duration of the test. - Assertion: We assert that the mock object was called with the expected arguments, simulating the check that the email notification was triggered with the correct content.
Instead of sending emails every time we run the test, we "simulated" the logic of email sending and have tested that everything works as expected.
Parameterized testing allows you to run the same test function with different inputs, reducing code duplication and making it easier to cover a wide range of scenarios.
pytest offers a simple way to parameterize tests using the @pytest.mark.parametrize decorator.
import pytest
@pytest.mark.parametrize("test_input,expected", [
(5, 25),
(9, 81),
(2, 4)
])
def test_square(test_input, expected):
assert test_input ** 2 == expectedThis test will run three times, each with the different test_input and expected values provided. (So there will be 3 tests with different values inside one, instead of creating tones of duplicate code)
Fixtures in pytest are functions run by pytest before (and sometimes after) the actual test functions to which they're applied. Fixtures are a powerful feature for setting up and tearing down test environments or contexts.
import pytest
@pytest.fixture
def sample_data():
return [1, 2, 3, 4, 5]
# After creating a fixture we can use an object to test different conditions
def test_sum(sample_data):
assert sum(sample_data) == 15
def test_length(sample_data):
assert len(sample_data) == 5The sample_data fixture is automatically injected into the test_sum and test_length function by pytest, demonstrating fixture management for reusable test data setup.
Testing how your application handles errors is as crucial as testing its success paths. pytest simplifies the process of asserting exceptions.
import pytest
def raise_custom_error():
raise ValueError("An error occurred")
def test_raise_custom_error():
with pytest.raises(ValueError) as e:
raise_custom_error()
assert str(e.value) == "An error occurred"This test checks that raise_custom_error raises a ValueError with the expected message.
All tools can be easily integrated in order to enhance the quality of your tests and after having such knowledge you as developer will be able to write well-structured easy to use tests.
Code coverage is a measure used to describe the degree to which the source code of a program is executed when a particular test suite runs.
A program with high code coverage has had more of its source code tested, which can lead to fewer bugs.
coverage.py is a tool for measuring code coverage of Python programs. It monitors your program, noting which parts of the code have been executed.
pip install coverage- Run your tests under
coveragerun:
coverage run -m pytest- Then, generate a report:
coverage reportOr for an HTML version:
coverage htmlThis report will show which lines of code were not executed by your tests, helping identify areas needing additional testing, which can be added.
Let's create a new application, but now, we will use everything we have learnt during this section
For our Task Manager application, we have the following requirements:
- Ability to add tasks with a unique identifier.
- Ability to mark tasks as complete.
- Send a notification when a task is marked as complete.
Following the TDD approach, we start by writing tests for our yet-to-be-implemented features. We focus on testing our application's core functionality and how it handles edge cases.
import pytest
from task_manager import TaskManager
@pytest.fixture
def task_manager():
return TaskManager() # Instance of our class
def test_add_task(task_manager):
task_id = task_manager.add_task("Learn pytest")
assert task_manager.get_task(task_id) == "Learn pytest"
def test_mark_task_as_complete(task_manager):
task_id = task_manager.add_task("Learn mocking")
task_manager.mark_task_as_complete(task_id)
assert task_id not in task_manager.tasks
assert task_id in task_manager.completed_tasks
def test_mark_nonexistent_task_as_complete(task_manager):
with pytest.raises(ValueError):
task_manager.mark_task_as_complete(999)After defining our tests, we proceed to implement the application logic to make these tests pass.
class TaskManager:
def __init__(self):
self.tasks = {}
self.completed_tasks = {}
def add_task(self, description):
task_id = len(self.tasks) + 1
self.tasks[task_id] = description
return task_id
def mark_task_as_complete(self, task_id):
if task_id not in self.tasks:
raise ValueError("Task ID does not exist")
self.completed_tasks[task_id] = self.tasks.pop(task_id)
def get_task(self, task_id):
return self.tasks.get(task_id, "Task does not exist")Use pytest to run your tests and ensure they all pass:
pytest test_task_manager.pyUse parameterized tests to run the same test logic with different inputs effortlessly.
@pytest.mark.parametrize("description", ["Task 1", "Task 2", "Task 3"])
def test_add_multiple_tasks(task_manager, description):
task_id = task_manager.add_task(description)
assert task_manager.get_task(task_id) == descriptionFinally, assess your test suite's effectiveness using coverage analysis tools to ensure every line of your application logic is tested.
pytest --cov=task_manager test_task_manager.pyCongratulations, we have learnt how to write Unit Tests!
Cover your existing projects with unit tests, aim to get 99% of coverage. Add mocking and patching to the application we have written in Section 8.