diff --git a/.github/workflows/pipeline.yml b/.github/workflows/pipeline.yml index 9f3ca9d..a34ca8b 100644 --- a/.github/workflows/pipeline.yml +++ b/.github/workflows/pipeline.yml @@ -27,4 +27,4 @@ jobs: run: ./scripts/run_clang_tidy.sh - name: Test - run: ./scripts/test.sh \ No newline at end of file + run: ./scripts/test.sh diff --git a/.gitignore b/.gitignore index 55ce6c0..9a8aa16 100644 --- a/.gitignore +++ b/.gitignore @@ -56,5 +56,6 @@ dkms.conf __pycache__ build pics/ -*_cat.bmp -*_monkey.bmp +images/*_cat.bmp +images/*_monkey.bmp +output_queue_mode/ diff --git a/Analysis_of_queue_benchmark_results.md b/Analysis_of_queue_benchmark_results.md new file mode 100644 index 0000000..96c9ebc --- /dev/null +++ b/Analysis_of_queue_benchmark_results.md @@ -0,0 +1,92 @@ +# Анализ производительности режима очередей (`queue mode`) + +## Введение + +В рамках данного анализа исследуется эффективность реализации параллельной обработки изображений в архитектуре очередей (`--mode=queue`). В этой модели обработка изображений разбита между тремя типами потоков, которые соединяются очередьми с ограничениями по размеру добавляемого изображения, образуя тем самым конвейер обработки: + - `readers` - отвечают за чтение исходных данных; + - `workers` - выполняют применение фильтра `bl`; + - `writers` - записывают результаты обработки. + +**Конвейер:** +``` + [Reader(s)] → [Input Queue] → [Worker(s)] → [Output Queue] → [Writer(s)] +``` +`Reader` читает изображение и помещает его во входную очередь (`Input Queue`) → `Worker` извлекает изображения из `Input Queue`, выполняет свертку, помещает результат в выходную очередь (`Output Queue`) → `Writer` извлекает результат из `Output Queue` и записывает в файл. + +Тестирование проводилось на наборе из [9 изображений](input_queue_mode/), полученных применением различных фильтров к [`cat.bmp`](images/cat.bmp). Таким образом, все изображения имеют одинаковый размер - `5.9 MiB`, что позволяет минимизировать влияние объёма данных на результаты тестирования. Однако их содержимое различно, поэтому каждое изображение необходимо отдельно загружать в память и записывать результат обратно, в отличие от случая с полностью идентичными изображениями, когда можно было бы загрузить файл один раз и использовать его повторно. + +Также стоит отметить, что расположение мьютексов при работе с условными переменными в конце функциях [`queue_push`](src/queue_mode/queue.c#L111) и [`queue_pop`](src/queue_mode/queue.c#L145) выбрано не случайно. Именно такая организация блокировок позволяет достичь минимального времени выполнения программы. Ключевым моментом является то, что в функции `queue_pop` [`pop_mutex`](src/queue_mode/queue.c#L149) освобождается только после освобождения [`push_mutex`](src/queue_mode/queue.c#L147) при отправке сигнала о появлении свободного места в очереди. Это гарантирует, что поток `worker` сможет раньше начать претендовать на доступ к новым данным из очереди. Если бы `pop_mutex` освобождался раньше, до попытки захвата [`queue_push`](src/queue_mode/queue.c#L145), `worker` оказывался бы в состоянии ожидания освобождения `push_mutex` (которое чаще всего долгое). + +Эксперимент охватывал следующие параметры: +1) Конфигурации распределения потоков между ролями: (`readers`, `workers`, `writers`); +2) Лимиты памяти : + - `6 MiB` - одновременно в очереди может находиться только одно изображение; + - `30 MiB` - до пяти изображений могут быть в очереди; + - `55 MiB` - очередь позволяет хранить все изображения. +3) Число потоков для применения фильтра `bl`: `--thread=2`, `--thread=3`, `--thread=4`, `--thread=5`. + +**Количество прогонов для каждого теста:** 40 (обеспечивает статистическую достоверность результатов). + +**Код бенчмарка**: [`tests/benchmarks/comparison_of_queue_mode.py`](tests/benchmarks/comparison_of_queue_mode.py) + +## Конфигурация системы: + + - **Процессор:** Процессор: Intel Core i7-11370H (4 физических ядра, 8 логических потоков) с фиксированной частотой 3.3 GHz (`sudo cpupower frequency-set --min 3300MHz --max 3300MHz`); + - **Кэш L1:** 80 КБ на ядро (48 КБ – кэш данных, 32 КБ – кэш инструкций); + - **Кэш L2:** 1.25 МБ на ядро; + - **Общий кэш L3:** 12 МБ. + + - **ОЗУ:** 16ГБ + + - **ОС:** Linux Kubuntu 24.04 + + - **Настройки производительности:** Использовался режим `performance` (`sudo cpupower frequency-set -g performance`) для минимизации влияния динамического изменения частоты процессора. + + Для обеспечения чистоты эксперимента система перезагружалась после каждого тестирования, а сторонние процессы были минимизированы. + +## Результаты тестирования + +В ходе тестирования измерялось: + - Общее время выполнения программы; + - Время работы каждого этапа - чтение, обработка и запись изображения. + +Графики и соответствующие им результаты находятся в папках [`tests/plots/queue_mode_`](tests/plots/), где N обозначает число потоков для применения фильтра. + +## Анализ результатов + +### Влияние количества потоков (`--thread`) и `mem_lim` +- **`--thread=2`**: + - Наилучшая конфигурация: **1 Reader, 3 Workers, 2 Writers** (`mem_lim=6`, Среднее время - **1.78 сек.**). + - Увеличение `mem_limit` до 55 MiB ухудшает время. + - Увеличение общего числа создаваемых потоков (`readers` + `workers` + `writers` + `threads`) сверх 8 не приводит к улучшению производительности, поскольку процессор поддерживает максимум 8 логических потоков. При превышении этого числа возникает конкуренция за вычислительные ресурсы, так как планировщик операционной системы вынужден выполнять частые переключения контекста между потоками. Это увеличивает накладные расходы и ухудшает производительность вместо её повышения. Поэтому в дальнейших тестах суммарное количество потоков намеренно ограничивалось значением 8. + +- **`--thread=3`**: + - Оптимальная производительность: **1 Reader, 3 Workers, 1 Writer** (`mem_lim=6`, Среднее время - **1.735 сек.**). + - При увеличении `mem_lim` время также увеличивается. + +- **`--thread=4`**: + - Лучший результат: **1 Reader, 2 Workers, 1 Writer** (`mem_lim=6`, Среднее время - **1.726 сек.**). + - При увеличении `mem_lim` время также увеличивается. + +- **`--thread=5`**: + - Минимальное среднее время: **1.74 сек.** (конфигурация `1-1-1`, `mem_lim=6`). + +**Объяснение:** + +**1.** Результаты показывают, что обработка изображения является узким местом всего pipeline'а. По этой причине увеличение числа потоков, ответственных за применение фильтра, приводит к снижению времени выполнения этого этапа, что в свою очередь положительно влияет на общее время выполнения программы. + +**2.** Во всех рассмотренных случаях наилучшие результаты достигаются при минимальном лимите памяти (**6 MiB**). Увеличение лимита до **30** и **55 MiB** не улучшает, а в большинстве случаев даже его ухудшает. Это объясняется особенностями работы синхронизации между потоками `readers` и `workers` в контексте входной очереди (`Input queue`). При малом лимите памяти (**6 MiB**), очередь позволяет хранить только одно изображение, поэтому читатель вынужден ожидать освобождения места (пока `worker` заберёт это изображение). Это создаёт небольшую задержку для `reader'а`, но позволяет `worker` немедленно приступить к обработке, что особенно важно, поскольку этап обработки является самым медленным звеном pipeline. Напротив, при увеличении лимита памяти, `reader` может загрузить сразу несколько изображений в очередь без ожидания, что повышает вероятность конкуренции за мьютексы (`push_mutex` и `pop_mutex`) между `readers` и `workers`. + В результате: + - Рабочие тратят больше времени на ожидание блокировок; + - Начало обработки первого изображения откладывается; + - Общее время выполнения программы возрастает. + +Таким образом, ограничение памяти положительно влияет на производительность, так как способствует более раннему началу самого длительного этапа - обработки изображения, минимизируя простои и конкуренцию за общие ресурсы. + +## Вывод + +Минимальное время выполнения программы зафиксировано при конфигурации `--thread=4`, `1 Reader`, `2 Workers`, `1 Writer`, `mem_lim=6` - среднее время: **1.726 сек.** + +Этот результат демонстрирует, что для достижения максимальной производительности необходимо: + - Ограничивать размер очереди, чтобы он соответствовал размеру одного изображения. + - Уменьшить время обработки изображений - найти баланс между ускорением выполнения сверстки (`--thread=4`) и количеством одновременно работающих `worker'ов`. diff --git a/CMakeLists.txt b/CMakeLists.txt index 5d14689..3254885 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -11,12 +11,8 @@ set(CMAKE_C_FLAGS_RELEASE "${CMAKE_C_FLAGS_RELEASE} -O2") add_subdirectory(src) -list(APPEND CMAKE_MODULE_PATH "${CMAKE_SOURCE_DIR}/cmake/Modules") - find_package(CMocka CONFIG) if (CMocka_FOUND) - include(AddCMockaTest) - include(AddMockedTest) - add_subdirectory(tests) enable_testing() + add_subdirectory(tests) endif(CMocka_FOUND) diff --git a/README.md b/README.md index a1d8b02..afcac55 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,7 @@ ## Description -Image-convolution is an image processing application that applies various convolution filters to images. It supports both sequential and parallel execution modes with different workload distribution strategies. The application is designed for benchmarking and comparing different parallelization approaches for image processing algorithms. +Image-convolution is an image processing tool that applies convolution filters in sequential, parallel (with different workload distribution strategies), and queue-based pipeline modes. In queue mode, readers, workers, and writers operate concurrently in a producer-consumer model with memory-limited queues. The app is designed for benchmarking different parallelization strategies and thread scalability. ## Usage Basic Command: @@ -16,14 +16,21 @@ Basic Command: ./build/src/image-convolution --mode= [--thread=] ``` ### Options -| Parameter | Description | Required | -|--------------------|---------------------------------------------------------------------|----------| -| `` | Path to input image or `--default-image` (predefined default image) | Yes | -| `` | Filter to apply (see [Available Filters](#available-filters)) | Yes | -| `--mode=` | Execution mode: `seq`, `pixel`, `row`, `column`, `block` | Yes | -| `--thread=` | Number of threads (for parallel modes) | No* | +| Parameter | Description | +|--------------------|-----------------------------------------------------------------------------| +| `` | Path to input image or `--default-image` (predefined default image) | +| `` | Filter to apply (see [Available Filters](#available-filters)) | +| `--mode=` | Execution mode: `seq`, `pixel`, `row`, `column`, `block` or `queue` | +| `--thread=` | Number of threads to use for parallel convolution (ignored if `--mode=seq`) | -*Required for all modes except seq +#### Queue options +| Parameter | Description | +|--------------------|---------------------------------------------------------------------| +| `--num=` | Number of images to process | +| `--readers=` | Number of reader threads | +| `--workers=` | Number of worker threads | +| `--writers=` | Number of writer threads | +| `--mem_lim=` | Memory limit for queues in MiB (e.g. 10) | ### Available Filters | Name | Description | Kernel Size | @@ -47,6 +54,10 @@ Basic Command: ```bash ./build/src/image-convolution images/cat.bmp gbl --mode=block --thread=4 ``` +3) Queue-Based pipeline processing: +```bash +./build/src/image-convolution images mbl --mode=queue --thread=2 --num=25 --readers=2 --workers=3 --writers=2 --mem_lim=15 +``` ## Build To build the project: @@ -68,6 +79,10 @@ The project includes three types of tests: ```bash ./scripts/perf.sh ``` +4) Queue mode performance analysis - evaluate execution time distribution across stages (reader, worker, writer) for various thread configurations under a memory limit: +```bash +./scripts/queue_benchmark.sh +``` ## Prerequisites for Benchmarks (`Performance benchmarks` and `Cache performance analysis`) Before running benchmarks, you need to set up a Python virtual environment and install dependencies: diff --git a/cmake/Modules/AddCMockaTest.cmake b/cmake/Modules/AddCMockaTest.cmake deleted file mode 100644 index 4c4a778..0000000 --- a/cmake/Modules/AddCMockaTest.cmake +++ /dev/null @@ -1,143 +0,0 @@ -# -# Copyright (c) 2007 Daniel Gollub -# Copyright (c) 2007-2018 Andreas Schneider -# Copyright (c) 2018 Anderson Toshiyuki Sasaki -# -# Redistribution and use is allowed according to the terms of the BSD license. -# -# Redistribution and use in source and binary forms, with or without -# modification, are permitted provided that the following conditions -# are met: -# -# 1. Redistributions of source code must retain the copyright -# notice, this list of conditions and the following disclaimer. -# 2. Redistributions in binary form must reproduce the copyright -# notice, this list of conditions and the following disclaimer in the -# documentation and/or other materials provided with the distribution. -# 3. The name of the author may not be used to endorse or promote products -# derived from this software without specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR -# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES -# OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. -# IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, -# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT -# NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, -# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY -# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT -# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -#.rst: -# AddCMockaTest -# ------------- -# -# This file provides a function to add a test -# -# Functions provided -# ------------------ -# -# :: -# -# add_cmocka_test(target_name -# SOURCES src1 src2 ... srcN -# [COMPILE_OPTIONS opt1 opt2 ... optN] -# [LINK_LIBRARIES lib1 lib2 ... libN] -# [LINK_OPTIONS lopt1 lop2 .. loptN] -# ) -# -# ``target_name``: -# Required, expects the name of the test which will be used to define a target -# -# ``SOURCES``: -# Required, expects one or more source files names -# -# ``COMPILE_OPTIONS``: -# Optional, expects one or more options to be passed to the compiler -# -# ``LINK_LIBRARIES``: -# Optional, expects one or more libraries to be linked with the test -# executable. -# -# ``LINK_OPTIONS``: -# Optional, expects one or more options to be passed to the linker -# -# -# Example: -# -# .. code-block:: cmake -# -# add_cmocka_test(my_test -# SOURCES my_test.c other_source.c -# COMPILE_OPTIONS -g -Wall -# LINK_LIBRARIES mylib -# LINK_OPTIONS -Wl,--enable-syscall-fixup -# ) -# -# Where ``my_test`` is the name of the test, ``my_test.c`` and -# ``other_source.c`` are sources for the binary, ``-g -Wall`` are compiler -# options to be used, ``mylib`` is a target of a library to be linked, and -# ``-Wl,--enable-syscall-fixup`` is an option passed to the linker. -# - -enable_testing() -include(CTest) - -if (CMAKE_CROSSCOMPILING) - if (WIN32) - find_program(WINE_EXECUTABLE - NAMES wine) - set(TARGET_SYSTEM_EMULATOR ${WINE_EXECUTABLE} CACHE INTERNAL "") - endif() -endif() - -function(ADD_CMOCKA_TEST _TARGET_NAME) - - set(one_value_arguments - ) - - set(multi_value_arguments - SOURCES - COMPILE_OPTIONS - COMPILE_DEFINITION - LINK_LIBRARIES - LINK_OPTIONS - ) - - cmake_parse_arguments(_add_cmocka_test - "" - "${one_value_arguments}" - "${multi_value_arguments}" - ${ARGN} - ) - - if (NOT DEFINED _add_cmocka_test_SOURCES) - message(FATAL_ERROR "No sources provided for target ${_TARGET_NAME}") - endif() - - add_executable(${_TARGET_NAME} ${_add_cmocka_test_SOURCES}) - - if (DEFINED _add_cmocka_test_COMPILE_OPTIONS) - target_compile_options(${_TARGET_NAME} - PRIVATE ${_add_cmocka_test_COMPILE_OPTIONS} - ) - endif() - - if (DEFINED _add_cmocka_test_LINK_LIBRARIES) - target_link_libraries(${_TARGET_NAME} - PRIVATE ${_add_cmocka_test_LINK_LIBRARIES} - ) - endif() - - if (DEFINED _add_cmocka_test_LINK_OPTIONS) - set_target_properties(${_TARGET_NAME} - PROPERTIES LINK_FLAGS - ${_add_cmocka_test_LINK_OPTIONS} - ) - endif() - - add_test(${_TARGET_NAME} - ${TARGET_SYSTEM_EMULATOR} ${_TARGET_NAME} - ) - -endfunction (ADD_CMOCKA_TEST) \ No newline at end of file diff --git a/cmake/Modules/AddMockedTest.cmake b/cmake/Modules/AddMockedTest.cmake deleted file mode 100644 index 7046d67..0000000 --- a/cmake/Modules/AddMockedTest.cmake +++ /dev/null @@ -1,60 +0,0 @@ -# MIT License -# -# Copyright (c) 2018 Kamil Lorenc -# -# Permission is hereby granted, free of charge, to any person obtaining a copy -# of this software and associated documentation files (the "Software"), to deal -# in the Software without restriction, including without limitation the rights -# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -# copies of the Software, and to permit persons to whom the Software is -# furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included in all -# copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -# SOFTWARE. - -## Add unit test with mocking support -# \param name unit test name (excluding extension and 'test_' prefix) -# \param SOURCES optional list of source files to include in test executable -# (beside test_${name}.c) -# \param MOCKS optional list of functions to be mocked in executable -# \param COMPILE_OPTIONS optional list of options for the compiler -# \param LINK_LIBRARIES optional list of libraries to link (used as -# -l${LINK_LIBRARIES}) -# \param LINK_OPTIONS optional list of options to be passed to linker -function(add_mocked_test name) - # parse arguments passed to the function - set(options ) - set(oneValueArgs ) - set(multiValueArgs SOURCES MOCKS COMPILE_OPTIONS LINK_LIBRARIES LINK_OPTIONS) - cmake_parse_arguments(ADD_MOCKED_TEST "${options}" "${oneValueArgs}" - "${multiValueArgs}" ${ARGN} ) - - # create link flags for mocks - set(link_flags "") - foreach (mock ${ADD_MOCKED_TEST_MOCKS}) - set(link_flags "${link_flags} -Wl,--wrap=${mock}") - endforeach(mock) - - if (CMocka_VERSION VERSION_GREATER_EQUAL "1.1.6") - list(APPEND CMOCKA_LIBRARIES cmocka::cmocka) - endif () - # define test - add_cmocka_test(test_${name} - SOURCES test_${name}.c ${ADD_MOCKED_TEST_SOURCES} - COMPILE_OPTIONS ${DEFAULT_C_COMPILE_FLAGS} - ${ADD_MOCKED_TEST_COMPILE_OPTIONS} - LINK_LIBRARIES ${CMOCKA_LIBRARIES} - ${ADD_MOCKED_TEST_LINK_LIBRARIES} - LINK_OPTIONS ${link_flags} ${ADD_MOCKED_TEST_LINK_OPTIONS}) - - # allow using includes from src/ directory - target_include_directories(test_${name} PRIVATE ${CMAKE_SOURCE_DIR}/src) -endfunction(add_mocked_test) \ No newline at end of file diff --git a/input_queue_mode/bl+gbl_cat.bmp b/input_queue_mode/bl+gbl_cat.bmp new file mode 100644 index 0000000..5498725 Binary files /dev/null and b/input_queue_mode/bl+gbl_cat.bmp differ diff --git a/input_queue_mode/bl_cat.bmp b/input_queue_mode/bl_cat.bmp new file mode 100644 index 0000000..12a1306 Binary files /dev/null and b/input_queue_mode/bl_cat.bmp differ diff --git a/input_queue_mode/ed_cat.bmp b/input_queue_mode/ed_cat.bmp new file mode 100644 index 0000000..fd85375 Binary files /dev/null and b/input_queue_mode/ed_cat.bmp differ diff --git a/input_queue_mode/em_cat.bmp b/input_queue_mode/em_cat.bmp new file mode 100644 index 0000000..167bad3 Binary files /dev/null and b/input_queue_mode/em_cat.bmp differ diff --git a/input_queue_mode/fbl+mbl_cat.bmp b/input_queue_mode/fbl+mbl_cat.bmp new file mode 100644 index 0000000..fff5622 Binary files /dev/null and b/input_queue_mode/fbl+mbl_cat.bmp differ diff --git a/input_queue_mode/fbl_cat.bmp b/input_queue_mode/fbl_cat.bmp new file mode 100644 index 0000000..30c7303 Binary files /dev/null and b/input_queue_mode/fbl_cat.bmp differ diff --git a/input_queue_mode/gbl_cat.bmp b/input_queue_mode/gbl_cat.bmp new file mode 100644 index 0000000..2fc0edb Binary files /dev/null and b/input_queue_mode/gbl_cat.bmp differ diff --git a/input_queue_mode/id_cat.bmp b/input_queue_mode/id_cat.bmp new file mode 100644 index 0000000..3e9b03a Binary files /dev/null and b/input_queue_mode/id_cat.bmp differ diff --git a/input_queue_mode/mbl_cat.bmp b/input_queue_mode/mbl_cat.bmp new file mode 100644 index 0000000..778cdd3 Binary files /dev/null and b/input_queue_mode/mbl_cat.bmp differ diff --git a/scripts/queue_benchmark.sh b/scripts/queue_benchmark.sh new file mode 100755 index 0000000..f3f4f14 --- /dev/null +++ b/scripts/queue_benchmark.sh @@ -0,0 +1,15 @@ +#!/bin/sh -e + +BASEDIR=$(realpath "$(dirname "$0")") +ROOTDIR=$(realpath "$BASEDIR/..") + +if [ -z "$1" ]; then + echo "Usage: $0 " + echo "Note that images must be in the 'input_queue_mode'" + exit 1 +fi + +NUM_OF_IMGS="$1" +MEM_LIM="$2" + +python3 "$ROOTDIR/tests/benchmarks/comparison_of_queue_mode.py" "$NUM_OF_IMGS" "$MEM_LIM" diff --git a/src/convolution/filter_application.c b/src/convolution/filter_application.c index 0b808cf..dc33d3d 100644 --- a/src/convolution/filter_application.c +++ b/src/convolution/filter_application.c @@ -44,25 +44,25 @@ void *process_dynamic(void *arg) { break; } - int block_x = block_index % data->num_cols; - int block_y = block_index / data->num_cols; + size_t block_x = block_index % data->num_cols; + size_t block_y = block_index / data->num_cols; - int start_x = block_x * data->block_width; - int start_y = block_y * data->block_height; + size_t start_x = block_x * data->block_width; + size_t start_y = block_y * data->block_height; - int end_x = min(start_x + data->block_width, data->width); - int end_y = min(start_y + data->block_height, data->height); + size_t end_x = min(start_x + data->block_width, (size_t)data->width); + size_t end_y = min(start_y + data->block_height, (size_t)data->height); - for (int y = start_y; y < end_y; y++) { - for (int x = start_x; x < end_x; x++) { + for (size_t y = start_y; y < end_y; y++) { + for (size_t x = start_x; x < end_x; x++) { double red = 0.0, green = 0.0, blue = 0.0; for (int filterY = 0; filterY < data->filter.size; filterY++) { for (int filterX = 0; filterX < data->filter.size; filterX++) { - int imageX = + size_t imageX = (x - data->filter.size / 2 + filterX + data->width) % data->width; - int imageY = + size_t imageY = (y - data->filter.size / 2 + filterY + data->height) % data->height; diff --git a/src/main.c b/src/main.c index 31954e4..975697b 100644 --- a/src/main.c +++ b/src/main.c @@ -1,195 +1,58 @@ #include "convolution/parallel_dispatch.h" #include "filters/filter.h" -#include "utils/utils.h" +#include "queue_mode/queue_dispatch.h" +#include "queue_mode/threads.h" +#include "utils/args.h" -#define STB_IMAGE_IMPLEMENTATION -#include "stb_image.h" -#define STB_IMAGE_WRITE_IMPLEMENTATION -#include "stb_image_write.h" +#include +#include -#define MODE_PREFIX_LEN 7 -#define THREAD_PREFIX_LEN 9 -#define PATH_PREFIX_LENGTH 7 // Length "images/" -#define UNDERSCORE_COUNT 2 // Number of underscores -#define NULL_TERMINATOR_LENGTH 1 // Terminating null character '\0' +#define PATH_PREFIX_LEN 7 // Length "images/" +#define UNDERSCORE_COUNT 2 // Number of underscores +#define NULL_TERMINATOR_LEN 1 // Terminating null character '\0' +#define DIR_ACCESS_RIGHTS 0755 /** - * A structure to store the parsed command-line arguments. - * - * @param image_path Path to the input image file or "images/cat.bmp" if - * --default-image is specified. - * @param filter_name Name of the filter to apply. - * @param mode Execution mode ("seq", "row", "column", "block", or "pixel"). - * @param threads_num Number of threads to use in parallel modes (ignored for "seq"). + * Loads the input image, applies the specified filter using the selected execution + * mode, and saves the resulting image. */ -typedef struct { - const char *image_path; - const char *filter_name; - const char *mode; - int threads_num; -} ProgramArgs; - -/** - * Handles runtime errors by printing an error message if the specified condition is - * true. - * - * @param condition A boolean value indicating whether an error has occurred. - * @param message String describing the error. - * @param ... Additional arguments for formatting the error message. - * - * @return `true` if the condition is true (indicating an error occurred), `false` - * otherwise. - */ -static inline bool handle_error(bool condition, const char *message, ...) { - if (condition) { - va_list args; - va_start(args, message); - error(message, args); - va_end(args); - return true; - } - return false; -} - -/** - * Parses the command-line arguments provided to the program. It validates the input - * and populates the `ProgramArgs` structure with the parsed values. - * - * @param argc The number of command-line arguments. - * @param argv An array of strings containing the command-line arguments. - * @param args A pointer to a `ProgramArgs` structure where the parsed arguments will - * be stored. - * - * @return `true` if parsing is successful, `false` if there is an error. - */ -static bool parse_args(int argc, char *argv[], ProgramArgs *args) { - if (argc < 4) { - error("Usage:\n" - " %s --mode= " - "--thread=\n" - " %s --mode=seq\n\n" - "Options:\n" - " Path to the input image file.\n" - " --default-image Use a predefined default image.\n" - " Name of the filter to apply ().\n" - " --mode= Execution mode: 'seq' for sequential or " - "'row', 'column', 'block', 'pixel' for parallel.\n" - " --thread= Number of threads to use in parallel mode " - "(ignored if --mode=seq).\n\n" - "Available Filters:\n", - argv[0], argv[0]); - for (int i = 0; i < NUM_OF_FILTERS; i++) { - error(" %-22s %s\n", filters_info[i].name, filters_info[i].description); - } - return false; - } - - args->image_path = - strcmp(argv[1], "--default-image") == 0 ? "images/cat.bmp" : argv[1]; - args->filter_name = argv[2]; - - if (strncmp(argv[3], "--mode=", MODE_PREFIX_LEN) != 0) { - handle_error(true, "Missing --mode argument\n"); - return false; - } - args->mode = argv[3] + MODE_PREFIX_LEN; - - if (strcmp(args->mode, "seq") != 0) { - if (strncmp(argv[4], "--thread=", THREAD_PREFIX_LEN) != 0) { - handle_error(true, "Missing --thread argument\n"); - return false; - } - args->threads_num = atoi(argv[4] + THREAD_PREFIX_LEN); - } - - return true; -} - -/** - * The entry point of the program. It organizes the entire process of loading an - * image, applying a convolution filter, and saving the result. - */ -int main(int argc, char *argv[]) { - ProgramArgs args = {NULL, NULL, NULL, 1}; - if (!parse_args(argc, argv, &args)) { - return -1; - } - +static int default_mode(program_args args, struct filter image_filter) { int width, height, channels; unsigned char *image = NULL; struct image_rgb channel_image = {NULL, NULL, NULL}; struct image_rgb result_channel_image = {NULL, NULL, NULL}; - struct filter image_filter = {0, 0, 0, NULL}; unsigned char *result_image = NULL; char *output_file_path = NULL; // Load image - image = stbi_load(args.image_path, &width, &height, &channels, 3); - if (handle_error(!image, "Could not open or find the image!\n")) { + image = stbi_load(args.img_path, &width, &height, &channels, 3); + if (!image) { + error("Could not open or find the image!\n"); goto cleanup_and_err; } // Initialize RGB channels channel_image = initialize_image_rgb(width, height); - if (handle_error(channel_image.red == NULL || channel_image.green == NULL || - channel_image.blue == NULL, - "Memory allocation error for channel_image.\n")) { + if (channel_image.red == NULL || channel_image.green == NULL || + channel_image.blue == NULL) { + error("Memory allocation error for channel_image.\n"); goto cleanup_and_err; } split_image_into_rgb_channels(image, channel_image, width, height); - // Create filter - if (strcmp(args.filter_name, "id") == 0) { - image_filter = create_filter(ID_SIZE, ID_FACTOR, ID_BIAS, id); - } else if (strcmp(args.filter_name, "fbl") == 0) { - image_filter = create_filter(FAST_BLUR_SIZE, FAST_BLUR_FACTOR, - FAST_BLUR_BIAS, fast_blur); - } else if (strcmp(args.filter_name, "bl") == 0) { - image_filter = create_filter(BLUR_SIZE, BLUR_FACTOR, BLUR_BIAS, blur); - } else if (strcmp(args.filter_name, "gbl") == 0) { - image_filter = create_filter(GAUS_BLUR_SIZE, GAUS_BLUR_FACTOR, - GAUS_BLUR_BIAS, gaus_blur); - } else if (strcmp(args.filter_name, "mbl") == 0) { - image_filter = create_filter(MOTION_BLUR_SIZE, MOTION_BLUR_FACTOR, - MOTION_BLUR_BIAS, motion_blur); - } else if (strcmp(args.filter_name, "ed") == 0) { - image_filter = create_filter(EDGE_DETECTION_SIZE, EDGE_DETECTION_FACTOR, - EDGE_DETECTION_BIAS, edge_detection); - } else if (strcmp(args.filter_name, "em") == 0) { - image_filter = - create_filter(EMBOSS_SIZE, EMBOSS_FACTOR, EMBOSS_BIAS, emboss); - } else if (strcmp(args.filter_name, "bl+gbl") == 0) { - image_filter = compose_filters_from_params( - BLUR_SIZE, BLUR_FACTOR, BLUR_BIAS, blur, GAUS_BLUR_SIZE, - GAUS_BLUR_FACTOR, GAUS_BLUR_BIAS, gaus_blur); - } else if (strcmp(args.filter_name, "fbl+mbl") == 0) { - image_filter = compose_filters_from_params( - FAST_BLUR_SIZE, FAST_BLUR_FACTOR, FAST_BLUR_BIAS, fast_blur, - MOTION_BLUR_SIZE, MOTION_BLUR_FACTOR, MOTION_BLUR_BIAS, motion_blur); - } else { - if (handle_error(1, "Unknown filter name: %s\n", args.filter_name)) { - goto cleanup_and_err; - } - } - - if (handle_error(image_filter.kernel == NULL, - "Memory allocation error for filter.\n")) { - goto cleanup_and_err; - } - // Initialize result channels result_channel_image = initialize_image_rgb(width, height); - if (handle_error(result_channel_image.red == NULL || - result_channel_image.green == NULL || - result_channel_image.blue == NULL, - "Memory allocation error for result_channel_image.\n")) { + if (result_channel_image.red == NULL || result_channel_image.green == NULL || + result_channel_image.blue == NULL) { + error("Memory allocation error for result_channel_image.\n"); goto cleanup_and_err; } // Apply convolution double start_time = get_time_in_seconds(); - if (handle_error(start_time == -1, "Error in clock_gettime().\n")) { + if (start_time == -1) { + error("Error in clock_gettime().\n"); goto cleanup_and_err; } @@ -210,36 +73,37 @@ int main(int argc, char *argv[]) { sequential_application(&channel_image, &result_channel_image, width, height, image_filter); } else { - if (handle_error(1, "Unknown mode name: %s\n", args.mode)) { - goto cleanup_and_err; - } + error("Unknown mode name: %s\n", args.mode); + goto cleanup_and_err; } double end_time = get_time_in_seconds(); - if (handle_error(end_time == -1, "Error in clock_gettime().\n")) { + if (end_time == -1) { + error("Error in clock_gettime().\n"); goto cleanup_and_err; } - if (handle_error(return_value != 0, "Failed to create thread.\n")) { + if (return_value != 0) { + error("Failed to create thread.\n"); goto cleanup_and_err; } // Assemble and save result - result_image = malloc((size_t)width * (size_t)height * (size_t)channels); - if (handle_error(result_image == NULL, - "Memory allocation error for result_image.\n")) { + result_image = malloc((size_t)width * (size_t)height * 3); + if (!result_image) { + error("Memory allocation error for result_image.\n"); goto cleanup_and_err; } assemble_image_from_rgb_channels(result_image, result_channel_image, width, height); - const char *file_name = extract_filename(args.image_path); - output_file_path = malloc(PATH_PREFIX_LENGTH + UNDERSCORE_COUNT + - strlen(file_name) + strlen(args.mode) + - strlen(args.filter_name) + NULL_TERMINATOR_LENGTH); - if (handle_error(output_file_path == NULL, - "Memory allocation error for output_file_path.\n")) { + const char *file_name = extract_filename(args.img_path); + output_file_path = + malloc(PATH_PREFIX_LEN + UNDERSCORE_COUNT + strlen(file_name) + + strlen(args.mode) + strlen(args.filter_name) + NULL_TERMINATOR_LEN); + if (!output_file_path) { + error("Memory allocation error for output_file_path.\n"); goto cleanup_and_err; } @@ -254,7 +118,6 @@ int main(int argc, char *argv[]) { stbi_image_free(image); free_image_rgb(&channel_image); free_image_rgb(&result_channel_image); - free_filter(&image_filter); free(result_image); free(output_file_path); @@ -266,11 +129,148 @@ int main(int argc, char *argv[]) { } free_image_rgb(&channel_image); free_image_rgb(&result_channel_image); - if (image_filter.kernel != NULL) { - free_filter(&image_filter); - } free(result_image); free(output_file_path); return -1; } + +/** + * Sets up directories, queues, and threads for reader-worker-writer pipeline. + * Processes multiple images concurrently using shared queues. + */ +static int queue_mode(program_args args, struct filter image_filter) { + if (mkdir(QUEUE_DIR_NAME, DIR_ACCESS_RIGHTS) == -1) { + if (errno != EEXIST) { + error("Error creating directory.\n"); + return 1; + } + } + + img_queue input_queue, output_queue; + if (queue_init(&input_queue, args.memory_lim, true) != 0) { + error("Memory allocation error for input_queue.\n"); + } + + if (queue_init(&output_queue, args.memory_lim, false) != 0) { + error("Memory allocation error for output_queue.\n"); + queue_destroy(&input_queue); + } + + char **file_paths = get_file_paths(args.img_path, args.img_count); + if (!file_paths) { + goto cleanup_and_err; + } + + qthreads_info info = { + .readers = NULL, + .workers = NULL, + .writers = NULL, + .pargs = &args, + .file_paths = file_paths, + .img_filter = &image_filter, + .input_q = &input_queue, + .output_q = &output_queue, + }; + + double start_time = get_time_in_seconds(); + if (start_time == -1) { + error("Error in clock_gettime().\n"); + goto cleanup_and_err; + } + + if (start_threads(&info) != 0) { + goto cleanup_and_err; + } + + double end_time = get_time_in_seconds(); + if (end_time == -1) { + error("Error in clock_gettime().\n"); + goto cleanup_and_err; + } + printf("The convolution for all images took %.6f. All images are in '%s' " + "directory.\n", + (end_time - start_time), QUEUE_DIR_NAME); + + free_file_paths(file_paths, args.img_count); + queue_destroy(&input_queue); + queue_destroy(&output_queue); + + return 0; + +cleanup_and_err: + if (file_paths) { + free_file_paths(file_paths, args.img_count); + } + queue_destroy(&input_queue); + queue_destroy(&output_queue); + + return -1; +} + +/** + * Parses command-line arguments, loads the requested filter, and runs the + * convolution either in default mode or queue mode based on user input. + */ +int main(int argc, char *argv[]) { + program_args args = {NULL, NULL, NULL, 1, 0, 0, 0, 0, 0}; + if (!parse_args(argc, argv, &args)) { + return -1; + } + + // Create filter + struct filter image_filter = {0, 0, 0, NULL}; + + if (strcmp(args.filter_name, "id") == 0) { + image_filter = create_filter(ID_SIZE, ID_FACTOR, ID_BIAS, id); + } else if (strcmp(args.filter_name, "fbl") == 0) { + image_filter = create_filter(FAST_BLUR_SIZE, FAST_BLUR_FACTOR, + FAST_BLUR_BIAS, fast_blur); + } else if (strcmp(args.filter_name, "bl") == 0) { + image_filter = create_filter(BLUR_SIZE, BLUR_FACTOR, BLUR_BIAS, blur); + } else if (strcmp(args.filter_name, "gbl") == 0) { + image_filter = create_filter(GAUS_BLUR_SIZE, GAUS_BLUR_FACTOR, + GAUS_BLUR_BIAS, gaus_blur); + } else if (strcmp(args.filter_name, "mbl") == 0) { + image_filter = create_filter(MOTION_BLUR_SIZE, MOTION_BLUR_FACTOR, + MOTION_BLUR_BIAS, motion_blur); + } else if (strcmp(args.filter_name, "ed") == 0) { + image_filter = create_filter(EDGE_DETECTION_SIZE, EDGE_DETECTION_FACTOR, + EDGE_DETECTION_BIAS, edge_detection); + } else if (strcmp(args.filter_name, "em") == 0) { + image_filter = + create_filter(EMBOSS_SIZE, EMBOSS_FACTOR, EMBOSS_BIAS, emboss); + } else if (strcmp(args.filter_name, "bl+gbl") == 0) { + image_filter = compose_filters_from_params( + BLUR_SIZE, BLUR_FACTOR, BLUR_BIAS, blur, GAUS_BLUR_SIZE, + GAUS_BLUR_FACTOR, GAUS_BLUR_BIAS, gaus_blur); + } else if (strcmp(args.filter_name, "fbl+mbl") == 0) { + image_filter = compose_filters_from_params( + FAST_BLUR_SIZE, FAST_BLUR_FACTOR, FAST_BLUR_BIAS, fast_blur, + MOTION_BLUR_SIZE, MOTION_BLUR_FACTOR, MOTION_BLUR_BIAS, motion_blur); + } else { + error("Unknown filter name: %s\n", args.filter_name); + if (image_filter.kernel != NULL) { + free_filter(&image_filter); + } + } + + if (image_filter.kernel == NULL) { + error("Memory allocation error for filter.\n"); + return -1; + } + + if (strcmp(args.mode, "queue") == 0) { + if (queue_mode(args, image_filter) != 0) { + return -1; + } + } else { + if (default_mode(args, image_filter) != 0) { + return -1; + } + } + + free_filter(&image_filter); + + return 0; +} diff --git a/src/queue_mode/queue.c b/src/queue_mode/queue.c new file mode 100644 index 0000000..ebe5a7e --- /dev/null +++ b/src/queue_mode/queue.c @@ -0,0 +1,152 @@ +#define STB_IMAGE_IMPLEMENTATION +#define STB_IMAGE_WRITE_IMPLEMENTATION + +#include "queue.h" + +int queue_init(struct img_queue *img_q, size_t max_mem, bool input_queue) { + img_q->input_queue = input_queue; + img_q->head = NULL; + img_q->tail = NULL; + atomic_store(&img_q->current_mem_usage, 0); + img_q->max_mem_usage = max_mem; + + pthread_mutex_init(&img_q->push_mutex, NULL); + pthread_mutex_init(&img_q->pop_mutex, NULL); + + if (pthread_cond_init(&img_q->cond_not_full, NULL) != 0) { + pthread_mutex_destroy(&img_q->push_mutex); + pthread_mutex_destroy(&img_q->pop_mutex); + return -1; + } + + if (pthread_cond_init(&img_q->cond_not_empty, NULL) != 0) { + pthread_mutex_destroy(&img_q->push_mutex); + pthread_mutex_destroy(&img_q->pop_mutex); + pthread_cond_destroy(&img_q->cond_not_full); + return -1; + } + + return 0; +} + +void queue_destroy(struct img_queue *img_q) { + img_info_node_t *current = img_q->head; + while (current) { + img_info_node_t *next = current->next; + + free_image_rgb(¤t->image); + free(current); + + current = next; + } + + pthread_mutex_destroy(&img_q->push_mutex); + pthread_mutex_destroy(&img_q->pop_mutex); + pthread_cond_destroy(&img_q->cond_not_full); + pthread_cond_destroy(&img_q->cond_not_empty); +} + +int queue_push(img_queue *img_q, struct image_rgb img, int width, int height, + char *filename) { + pthread_mutex_lock(&img_q->push_mutex); + + size_t img_weight = (size_t)width * (size_t)height * 3; + + // Waiting for space to become available + while (atomic_load(&img_q->current_mem_usage) + img_weight > + img_q->max_mem_usage) { + pthread_cond_wait(&img_q->cond_not_full, &img_q->push_mutex); + } + + img_info_node_t *node = malloc(sizeof(img_info_node_t)); + if (!node) { + return -1; + } + + node->image = img; + node->filename = filename; + node->next = NULL; + + // Load and split image + if (img_q->input_queue && filename) { + int channels; + unsigned char *image_data = + stbi_load(filename, &width, &height, &channels, 3); + if (!image_data) { + error("Failed to load image: %s\n", filename); + free(node); + return -1; + } + + // Initialize RGB channels + struct image_rgb channel_image = initialize_image_rgb(width, height); + if (channel_image.red == NULL || channel_image.green == NULL || + channel_image.blue == NULL) { + error("Failed to initialize image rgb for %s.\n", filename); + free(node); + stbi_image_free(image_data); + return -1; + } + + split_image_into_rgb_channels(image_data, channel_image, width, height); + + node->image = channel_image; + stbi_image_free(image_data); + } + + node->width = width; + node->height = height; + + atomic_fetch_add(&img_q->current_mem_usage, img_weight); + + // Push to the queue + if (img_q->tail) { + img_q->tail->next = node; + img_q->tail = node; + } else { + img_q->tail = node; + img_q->head = node; + } + + pthread_mutex_unlock(&img_q->push_mutex); + + pthread_mutex_lock(&img_q->pop_mutex); + pthread_cond_signal(&img_q->cond_not_empty); + pthread_mutex_unlock(&img_q->pop_mutex); + + return 0; +} + +img_info_node_t *queue_pop(img_queue *img_q) { + pthread_mutex_lock(&img_q->pop_mutex); + + // Waiting for an item in the queue + while (!img_q->head) { + pthread_cond_wait(&img_q->cond_not_empty, &img_q->pop_mutex); + } + + img_info_node_t *out_node = img_q->head; + + // Pop from the queue + img_q->head = out_node->next; + if (!img_q->head) { + img_q->tail = NULL; + } + + // Checking for a termination signal + if (!out_node->filename) { + pthread_mutex_unlock(&img_q->pop_mutex); + return out_node; + } + + atomic_fetch_sub(&img_q->current_mem_usage, + ((size_t)out_node->width * (size_t)out_node->height * 3)); + + pthread_mutex_lock(&img_q->push_mutex); + pthread_cond_signal(&img_q->cond_not_full); + pthread_mutex_unlock(&img_q->push_mutex); + + pthread_mutex_unlock(&img_q->pop_mutex); + + return out_node; +} diff --git a/src/queue_mode/queue.h b/src/queue_mode/queue.h new file mode 100644 index 0000000..33bb8ac --- /dev/null +++ b/src/queue_mode/queue.h @@ -0,0 +1,98 @@ +#pragma once + +#include "../utils/utils.h" +#include +#include +#include +#include + +#include "stb_image.h" +#include "stb_image_write.h" + +/** + * Stores image data, metadata and pointer to next node in the linked list. + * + * @param image RGB channels of the image (`struct image_rgb`). + * @param filename Name of the source file (or NULL for termination signal). + * @param width Width of the image. + * @param height Height of the image. + * @param next Pointer to the next node in the queue. + */ +typedef struct queue_img_info { + struct image_rgb image; + char *filename; + int width; + int height; + struct queue_img_info *next; +} img_info_node_t; + +/** + * Manages a linked list of images with atomic memory tracking and synchronization + * primitives for concurrent access from multiple threads. + * + * @param input_queue Boolean indicating whether this is an input queue. + * @param head Head of the queue (oldest item). + * @param tail Tail of the queue (newest item). + * @param current_mem_usage Current memory usage of all items in bytes. + * @param max_mem_usage Maximum allowed memory usage in bytes. + * @param push_mutex Mutex for protecting push operations. + * @param pop_mutex Mutex for protecting pop operations. + * @param cond_not_full Condition variable for signaling when queue is not full. + * @param cond_not_empty Condition variable for signaling when queue is not empty. + */ +typedef struct img_queue { + bool input_queue; + + img_info_node_t *head; + img_info_node_t *tail; + atomic_size_t current_mem_usage; + size_t max_mem_usage; + + pthread_mutex_t push_mutex; + pthread_mutex_t pop_mutex; + pthread_cond_t cond_not_full; + pthread_cond_t cond_not_empty; +} img_queue; + +/** + * Sets up internal synchronization primitives and prepares queue for use. + * + * @param img_q Pointer to the queue structure to initialize. + * @param max_mem Maximum memory usage allowed for the queue in bytes. + * @param input_queue Boolean indicating whether this is an input queue. + * @return `0` on success, `-1` on error during mutex/condition initialization. + */ +int queue_init(struct img_queue *img_q, size_t max_mem, bool input_queue); + +/** + * Frees all queued images and destroys synchronization primitives. + * + * @param img_q Pointer to the queue to destroy. + */ +void queue_destroy(struct img_queue *img_q); + +/** + * Creates and pushes an `img_info_node_t` to the queue, downloading and splitting + * image data if it is an input queue. Blocks the thread if the queue's memory usage + * exceeds the limit until space becomes available. + * + * @param img_q Pointer to the queue. + * @param img RGB image data to enqueue (`struct image_rgb`). + * @param width Width of the image. + * @param height Height of the image. + * @param filename Path to image file or NULL for termination signal. + * + * @return `0` on success, `-1` on memory allocation failure or image load + * error. + */ +int queue_push(struct img_queue *img_q, struct image_rgb img, int width, int height, + char *filename); + +/** + * Removes `img_info_node_t` from the queue. Blocks the thread if the queue is empty + * until a new element arrives. + * + * @param img_q Pointer to the queue. + * @return Pointer to image info node, or NULL on error. + */ +img_info_node_t *queue_pop(struct img_queue *img_q); diff --git a/src/queue_mode/queue_dispatch.c b/src/queue_mode/queue_dispatch.c new file mode 100644 index 0000000..462839a --- /dev/null +++ b/src/queue_mode/queue_dispatch.c @@ -0,0 +1,101 @@ +#include "queue_dispatch.h" + +/** + * Allocates separate arrays of `pthread_t` for each group: readers, workers, and + * writers, based on the number specified in the program arguments. + */ +int allocate_threads(qthreads_info *info) { + info->readers = malloc(info->pargs->readers_num * sizeof(pthread_t)); + if (!info->readers) { + return -1; + } + + info->workers = malloc(info->pargs->workers_num * sizeof(pthread_t)); + if (!info->workers) { + free(info->readers); + return -1; + } + + info->writers = malloc(info->pargs->writers_num * sizeof(pthread_t)); + if (!info->writers) { + free(info->readers); + free(info->workers); + return -1; + } + + return 0; +} + +/** + * Deallocates memory previously allocated by `allocate_threads`. + */ +void free_threads(qthreads_info *info) { + free(info->readers); + free(info->workers); + free(info->writers); +} + +/** + * Starts `num` threads that execute the function pointed to by `start_routine`. + * If thread creation fails, already created threads are cancelled. + */ +int create_thread_group(pthread_t *threads, uint8_t num, + void *(*start_routine)(void *), void *arg) { + for (uint8_t i = 0; i < num; ++i) { + if (pthread_create(&threads[i], NULL, start_routine, arg) != 0) { + // Cancel successfully created threads + for (uint8_t j = 0; j < i; ++j) { + pthread_cancel(threads[j]); + } + + return -1; + } + } + + return 0; +} + +/** + * Joins each thread in the provided array using `pthread_join()`. + */ +void join_thread_group(pthread_t *threads, uint8_t num) { + for (uint8_t i = 0; i < num; ++i) { + pthread_join(threads[i], NULL); + } +} + +int start_threads(qthreads_info *info) { + if (allocate_threads(info) != 0) { + error("Failed to allocate thread memory\n"); + return -1; + } + + // Create thread groups + + if (create_thread_group(info->readers, info->pargs->readers_num, reader_thread, + info) != 0) { + error("Failed to create thread.\n"); + return -1; + } + + if (create_thread_group(info->workers, info->pargs->workers_num, worker_thread, + info) != 0) { + error("Failed to create thread.\n"); + return -1; + } + + if (create_thread_group(info->writers, info->pargs->writers_num, writer_thread, + info) != 0) { + error("Failed to create thread.\n"); + return -1; + } + + // Join threads + + join_thread_group(info->readers, info->pargs->readers_num); + join_thread_group(info->workers, info->pargs->workers_num); + join_thread_group(info->writers, info->pargs->writers_num); + free_threads(info); + + return 0; +} diff --git a/src/queue_mode/queue_dispatch.h b/src/queue_mode/queue_dispatch.h new file mode 100644 index 0000000..ea013ab --- /dev/null +++ b/src/queue_mode/queue_dispatch.h @@ -0,0 +1,17 @@ +#pragma once + +#include "threads.h" +#include + +/** + * Manages the full lifecycle of reader, worker, and writer threads in "queue" mode: + * - Allocates thread storage + * - Creates threads for each group + * - Waits for all threads to finish + * - Frees allocated memory + * + * @param info Pointer to a `qthreads_info` structure containing thread + * configuration. + * @return `0` on success, `-1` on error during thread creation or allocation. + */ +int start_threads(qthreads_info *info); diff --git a/src/queue_mode/threads.c b/src/queue_mode/threads.c new file mode 100644 index 0000000..e5cf47d --- /dev/null +++ b/src/queue_mode/threads.c @@ -0,0 +1,206 @@ +#include "threads.h" + +atomic_size_t read_images = 0; // Counter for read images +atomic_size_t finished_reader_threads = 0; // Counter for finished reader threads +atomic_size_t finished_worker_threads = 0; // Counter for finished worker threads +atomic_bool input_termination_sent = + false; // Flag indicating whether input termination signal has been sent +atomic_bool output_termination_sent = + false; // Flag indicating whether output termination signal has been sent + +void *reader_thread(void *arg) { + qthreads_info *info = (qthreads_info *)arg; + + struct image_rgb dummy_img = {NULL, NULL, NULL}; + int width, height, channels; + double start_time, end_time; + size_t index; + char *path; + + while ((index = atomic_fetch_add(&read_images, 1)) < info->pargs->img_count) { + path = info->file_paths[index]; + + start_time = get_time_in_seconds(); + if (start_time == -1) { + error("Error in clock_gettime().\n"); + break; + } + + if (!stbi_info(path, &width, &height, &channels)) { + error("READER: Failed to read image info from '%s'\n", path); + continue; + } + + // Checks that the image size is less than the size limit passed by the user + double weight_mib = (double)(width * height * 3) / BYTES_IN_MEBIBYTE; + double memory_lim_mib = (double)info->pargs->memory_lim / BYTES_IN_MEBIBYTE; + if (info->pargs->memory_lim < ((size_t)width * (size_t)height * 3)) { + error("'%s' (%.1f MiB) is larger than the maximum specified size - %.1f " + "MiB.\n", + path, weight_mib, memory_lim_mib); + continue; + } + + if (queue_push(info->input_q, dummy_img, width, height, path) != 0) { + error("READER: Failed to push '%s' into input queue.\n", path); + continue; + } + + end_time = get_time_in_seconds(); + if (end_time == -1) { + error("Error in clock_gettime().\n"); + break; + } + printf("READER: '%s' -> input queue in %.6f.\n", path, + end_time - start_time); + } + atomic_fetch_add(&finished_reader_threads, 1); + + // Sending termination signals by the last thread + if (atomic_load(&finished_reader_threads) == info->pargs->readers_num && + !atomic_exchange(&input_termination_sent, true)) { + for (uint8_t i = 0; i < info->pargs->workers_num; i++) { + queue_push(info->input_q, dummy_img, 0, 0, NULL); + } + } + + printf("Reader end his work.\n"); + return NULL; +} + +void *worker_thread(void *arg) { + qthreads_info *info = (qthreads_info *)arg; + + double start_time, end_time; + img_info_node_t *out_node; + + while (1) { + start_time = get_time_in_seconds(); + if (start_time == -1) { + error("Error in clock_gettime().\n"); + break; + } + + out_node = queue_pop(info->input_q); + + // Checking for a termination signal + if (!out_node->filename) { + free(out_node); + break; + } + + struct image_rgb result_channel_image = + initialize_image_rgb(out_node->width, out_node->height); + if (!result_channel_image.red || !result_channel_image.green || + !result_channel_image.blue) { + error("WORKER: Memory allocation error for result_channel_image.\n"); + free_image_rgb(&out_node->image); + free(out_node); + continue; + } + + parallel_row(&out_node->image, &result_channel_image, out_node->width, + out_node->height, *info->img_filter, info->pargs->threads_num); + + if (queue_push(info->output_q, result_channel_image, out_node->width, + out_node->height, out_node->filename) != 0) { + error("WORKER: Failed to push processed image to output queue.\n"); + free_image_rgb(&out_node->image); + free_image_rgb(&result_channel_image); + free(out_node); + break; + } + + end_time = get_time_in_seconds(); + if (end_time == -1) { + error("Error in clock_gettime().\n"); + free_image_rgb(&out_node->image); + free(out_node); + break; + } + printf("WORKER: '%s' -> output queue in %.6f.\n", out_node->filename, + end_time - start_time); + + free_image_rgb(&out_node->image); + free(out_node); + } + atomic_fetch_add(&finished_worker_threads, 1); + + // Sending termination signals by the last thread + struct image_rgb dummy_img = {NULL, NULL, NULL}; + if (atomic_load(&finished_worker_threads) == info->pargs->workers_num && + !atomic_exchange(&output_termination_sent, true)) { + for (uint8_t i = 0; i < info->pargs->writers_num; i++) { + queue_push(info->output_q, dummy_img, 0, 0, NULL); + } + } + + printf("Worker end his work.\n"); + return NULL; +} + +void *writer_thread(void *arg) { + struct qthreads_info *info = (struct qthreads_info *)arg; + + double start_time, end_time; + img_info_node_t *out_node; + + while (1) { + start_time = get_time_in_seconds(); + if (start_time == -1) { + error("Error in clock_gettime().\n"); + break; + } + + out_node = queue_pop(info->output_q); + + // Checking for a termination signal + if (out_node->filename == NULL) { + free(out_node); + break; + } + + unsigned char *result_image = + malloc((size_t)out_node->width * (size_t)out_node->height * 3); + if (!result_image) { + error("WRITER: Memory allocation failed for output image\n"); + free_image_rgb(&out_node->image); + free(out_node); + break; + } + + assemble_image_from_rgb_channels(result_image, out_node->image, + out_node->width, out_node->height); + + char out_path[MAX_PATH_LEN]; + snprintf(out_path, sizeof(out_path), "%s/%s", QUEUE_DIR_NAME, + extract_filename(out_node->filename)); + + if (!stbi_write_bmp(out_path, out_node->width, out_node->height, 3, + result_image)) { + error("WRITER: Failed to save image '%s'\n", out_path); + free(result_image); + free_image_rgb(&out_node->image); + free(out_node); + continue; + } + + end_time = get_time_in_seconds(); + if (end_time == -1) { + error("Error in clock_gettime().\n"); + free(result_image); + free_image_rgb(&out_node->image); + free(out_node); + break; + } + printf("WRITER: '%s' -> saved in %.6f.\n", out_node->filename, + end_time - start_time); + + free(result_image); + free_image_rgb(&out_node->image); + free(out_node); + } + + printf("Writer end his work.\n"); + return NULL; +} diff --git a/src/queue_mode/threads.h b/src/queue_mode/threads.h new file mode 100644 index 0000000..362684b --- /dev/null +++ b/src/queue_mode/threads.h @@ -0,0 +1,60 @@ +#pragma once + +#include "../convolution/parallel_dispatch.h" +#include "../utils/args.h" +#include "queue.h" +#include +#include +#include + +/** + * Contains all necessary data shared between threads during queue-based processing. + * + * @param readers Array of pthread IDs for reader threads. + * @param workers Array of pthread IDs for worker threads. + * @param writers Array of pthread IDs for writer threads. + * @param pargs Parsed command-line arguments. + * @param file_paths Array of file paths to be processed. + * @param img_filter Pointer to the filter to be applied. + * @param input_q Input queue containing images read by readers and processed by + * workers. + * @param output_q Output queue containing filtered images ready to be saved by + * writers. + */ +typedef struct qthreads_info { + pthread_t *readers; + pthread_t *workers; + pthread_t *writers; + program_args *pargs; + char **file_paths; + struct filter *img_filter; + img_queue *input_q; + img_queue *output_q; +} qthreads_info; + +/** + * Reads `.bmp` files from the provided list and pushes them into the input queue. + * + * @param arg Pointer to a `qthreads_info` structure. + * @return Always returns `NULL`. + */ +void *reader_thread(void *arg); + +/** + * Dequeues images from the input queue, applies the selected filter, and enqueues + * the result to the output queue. + * + * @param arg Pointer to a `qthreads_info` structure. + * @return Always returns `NULL`. + */ +void *worker_thread(void *arg); + +/** + * @brief Thread function for saving processed images to disk. + * + * Dequeues filtered images from the output queue and saves them as `.bmp` files. + * + * @param arg Pointer to a `qthreads_info` structure. + * @return Always returns `NULL`. + */ +void *writer_thread(void *arg); diff --git a/src/utils/args.c b/src/utils/args.c new file mode 100644 index 0000000..c5d4cc8 --- /dev/null +++ b/src/utils/args.c @@ -0,0 +1,181 @@ +#include "args.h" +#include + +#define MODE_PREFIX_LEN 7 // lenght of `--mode=` +#define THREAD_PREFIX_LEN 9 // lenght of `--thread=` +#define NUM_OF_IMAGES_PREFIX_LEN 6 // lenght of '--num=' +#define QUEUE_ARGS_PREFIX_LEN \ + 10 // lenght of '--readers=', '--workers=', '--writers=' or '--mem_lim=' +#define NUM_OF_ARGS_FOR_QUEUE_MOD 5 +#define INITIAL_INDEX_FOR_QUEUE_MOD 5 +#define CHECK_NUMBER(num, str) \ + if ((num) <= 0) { \ + error("Invalid number of %s, required number > 0.\n", str); \ + return false; \ + } + +bool parse_args(int argc, char *argv[], program_args *args) { + char *queue_options = + "Queue options:\n" + " --num= Number of images to process.\n" + " --readers= Number of reader threads.\n" + " --workers= Number of worker threads.\n" + " --writers= Number of writer threads.\n" + " --mem_lim= Memory limit for queues in MiB (e.g., 10).\n\n"; + + if (argc < 4) { + error( + "Usage:\n" + " %s --mode=seq\n" + " %s " + "--mode= --thread=\n" + " %s --mode=queue " + "--thread= \\\n" + " --num= --readers= --workers= " + "--writers= --mem_lim=\n\n" + + "Options:\n" + " Path to the input image file.\n" + " --default-image Use a predefined default image.\n" + " Name of the filter to apply ().\n" + " --mode= Execution mode:\n" + " 'seq' - sequential processing,\n" + " 'row' - parallel by rows,\n" + " 'column' - parallel by columns,\n" + " 'block' - parallel by blocks,\n" + " 'pixel' - parallel by pixels,\n" + " 'queue' - queue-based parallel processing.\n" + " --thread= Number of threads to use for parallel " + "convolution.\n" + " (Ignored if --mode=seq)\n\n", + argv[0], argv[0], argv[0]); + error("%s", queue_options); + error("Available Filters:\n"); + for (int i = 0; i < NUM_OF_FILTERS; i++) { + error(" %-22s %s\n", filters_info[i].name, filters_info[i].description); + } + return false; + } + + args->img_path = + strcmp(argv[1], "--default-image") == 0 ? "images/cat.bmp" : argv[1]; + args->filter_name = argv[2]; + + if (strncmp(argv[3], "--mode=", MODE_PREFIX_LEN) != 0) { + error("Missing --mode argument\n"); + return false; + } + args->mode = argv[3] + MODE_PREFIX_LEN; + + int res_int = 0; + + if (strcmp(args->mode, "seq") != 0) { + if (strncmp(argv[4], "--thread=", THREAD_PREFIX_LEN) != 0) { + error("Missing --thread argument\n"); + return false; + } + res_int = atoi(argv[4] + THREAD_PREFIX_LEN); + CHECK_NUMBER(res_int, "threads") + args->threads_num = res_int; + } + + if (strcmp(args->mode, "queue") == 0) { + if (argc < INITIAL_INDEX_FOR_QUEUE_MOD + NUM_OF_ARGS_FOR_QUEUE_MOD) { + error("Missing queue mode parameters.\n\n"); + error("%s", queue_options); + return false; + } + + for (uint8_t i = INITIAL_INDEX_FOR_QUEUE_MOD; + i < INITIAL_INDEX_FOR_QUEUE_MOD + NUM_OF_ARGS_FOR_QUEUE_MOD; i++) { + if (strncmp(argv[i], "--num=", NUM_OF_IMAGES_PREFIX_LEN) == 0) { + res_int = atoi(argv[i] + NUM_OF_IMAGES_PREFIX_LEN); + CHECK_NUMBER(res_int, "images") + args->img_count = res_int; + + } else if (strncmp(argv[i], "--readers=", QUEUE_ARGS_PREFIX_LEN) == 0) { + res_int = atoi(argv[i] + QUEUE_ARGS_PREFIX_LEN); + CHECK_NUMBER(res_int, "reader threads") + args->readers_num = res_int; + + } else if (strncmp(argv[i], "--workers=", QUEUE_ARGS_PREFIX_LEN) == 0) { + res_int = atoi(argv[i] + QUEUE_ARGS_PREFIX_LEN); + CHECK_NUMBER(res_int, "worker threads") + args->workers_num = res_int; + + } else if (strncmp(argv[i], "--writers=", QUEUE_ARGS_PREFIX_LEN) == 0) { + res_int = atoi(argv[i] + QUEUE_ARGS_PREFIX_LEN); + CHECK_NUMBER(res_int, "writer threads") + args->writers_num = res_int; + + } else if (strncmp(argv[i], "--mem_lim=", QUEUE_ARGS_PREFIX_LEN) == 0) { + double res_double = atof(argv[i] + QUEUE_ARGS_PREFIX_LEN); + CHECK_NUMBER(res_double, "memory limit") + args->memory_lim = (size_t)ceil(res_double * BYTES_IN_MEBIBYTE); + + } else { + error("Invalid argument '%s' for queue mode.\n", argv[i]); + + return false; + } + } + } + + return true; +} + +const char *extract_filename(const char *path) { + const char *last_slash = strrchr(path, '/'); + if (last_slash == NULL) { + return path; + } + + return last_slash + 1; +} + +char **get_file_paths(const char *dir_path, size_t file_count) { + DIR *dir = opendir(dir_path); + if (!dir) { + error("Failed to open directory '%s'.\n", dir_path); + return NULL; + } + + char **file_paths = (char **)malloc(file_count * sizeof(char *)); + if (!file_paths) { + error("Memory allocation error for file_paths.\n"); + closedir(dir); + return NULL; + } + + struct dirent *entry; + size_t index = 0; + + while ((entry = readdir(dir)) && index < file_count) { + if (strstr(entry->d_name, ".bmp")) { + file_paths[index] = malloc(MAX_PATH_LEN); + if (!file_paths[index]) { + for (size_t i = 0; i < index; i++) { + free(file_paths[i]); + } + free((void *)file_paths); + closedir(dir); + return NULL; + } + + snprintf(file_paths[index], MAX_PATH_LEN, "%s/%s", dir_path, + entry->d_name); + + index++; + } + } + + closedir(dir); + return file_paths; +} + +void free_file_paths(char **file_paths, size_t file_count) { + for (size_t i = 0; i < file_count; i++) { + free(file_paths[i]); + } + free((void *)file_paths); +} diff --git a/src/utils/args.h b/src/utils/args.h new file mode 100644 index 0000000..3539672 --- /dev/null +++ b/src/utils/args.h @@ -0,0 +1,83 @@ +#pragma once + +#include "../filters/filter.h" +#include "utils.h" +#include +#include +#include +#include +#include + +#define MAX_PATH_LEN 264 +#define QUEUE_DIR_NAME "output_queue_mode" + +/** + * A structure to store the parsed command-line arguments. + * + * @param image_path Path to the input image file or "images/cat.bmp" if + * --default-image is specified. + * @param filter_name Name of the filter to apply. + * @param mode Execution mode ("seq", "row", "column", "block", "pixel" or "queue"). + * @param threads_num Number of threads to use for parallel convolution (ignored for + * "seq"). + * @param img_count Number of images to process in "queue" mode. + * @param readers_num Number of reader threads in "queue" mode. + * @param workers_num Number of worker threads in "queue" mode. + * @param writers_num Number of writer threads in "queue" mode. + * @param memory_lim Memory limit for queues in bytes (converted from MiB) in "queue" + * mode. + */ +typedef struct { + const char *img_path; + const char *filter_name; + const char *mode; + int threads_num; + + uint8_t img_count; + uint8_t readers_num; + uint8_t workers_num; + uint8_t writers_num; + size_t memory_lim; +} program_args; + +/** + * Parses the command-line arguments provided to the program. It validates the input + * and populates the `ProgramArgs` structure with the parsed values. + * + * @param argc The number of command-line arguments. + * @param argv An array of strings containing the command-line arguments. + * @param args A pointer to a `ProgramArgs` structure where the parsed arguments will + * be stored. + * + * @return `true` if parsing is successful, `false` if there is an error. + */ +bool parse_args(int argc, char *argv[], program_args *args); + +/** + * Extracts the filename from a given file path. + * + * @param path The full file path. + * + * @return A pointer to the start of the filename in the path. If no / is found, the + * entire path is returned. + */ +const char *extract_filename(const char *path); + +/** + * Scans the specified directory and collects paths of `.bmp` files up to the + * specified count. + * + * @param dir_path Path to the directory to scan. + * @param file_count Maximum number of file paths to return. + * + * @return An allocated array of file paths, or `NULL` on failure. + */ +char **get_file_paths(const char *dir_path, size_t file_count); + +/** + * Deallocates each individual file path and the array itself. + * + * @param file_paths The array of file paths to free. + * @param file_count Number of elements in the array. + */ +void free_file_paths(char **file_paths, size_t file_count); diff --git a/src/utils/utils.c b/src/utils/utils.c index 875c400..0e818ea 100644 --- a/src/utils/utils.c +++ b/src/utils/utils.c @@ -48,15 +48,6 @@ void assemble_image_from_rgb_channels(unsigned char *image, } } -const char *extract_filename(const char *path) { - const char *last_slash = strrchr(path, '/'); - if (last_slash == NULL) { - return path; - } - - return last_slash + 1; -} - double get_time_in_seconds(void) { struct timespec time; if (clock_gettime(CLOCK_MONOTONIC, &time) != 0) { diff --git a/src/utils/utils.h b/src/utils/utils.h index d9cc621..3a1870b 100644 --- a/src/utils/utils.h +++ b/src/utils/utils.h @@ -1,6 +1,7 @@ #pragma once #include +#include #include #include #include @@ -8,6 +9,7 @@ #define min(a, b) ((a) < (b) ? (a) : (b)) #define max(a, b) ((a) > (b) ? (a) : (b)) #define error(...) (fprintf(stderr, __VA_ARGS__)) +#define BYTES_IN_MEBIBYTE (1024.0 * 1024.0) /** * Represents an image split into its red, green, and blue channels. @@ -65,19 +67,9 @@ void assemble_image_from_rgb_channels(unsigned char *image, struct image_rgb channel_image, int width, int height); -/** - * Extracts the filename from a given file path. - * - * @param path The full file path. - * - * @return A pointer to the start of the filename in the path. If no / is found, the - * entire path is returned. - */ -const char *extract_filename(const char *path); - /** * Retrieves the current time in seconds using the monotonic clock. * - * @return The current time in seconds as a double. Returns -1 if an error occurs. + * @return The current time in seconds as a double. Returns `-1` if an error occurs. */ double get_time_in_seconds(void); diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 4a8153d..045358e 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -1,9 +1,19 @@ -file(GLOB_RECURSE C_SOURCES ${CMAKE_SOURCE_DIR}/src/*.c) -list(REMOVE_ITEM C_SOURCES ${CMAKE_SOURCE_DIR}/src/main.c) +file(GLOB_RECURSE SRC_SOURCES ${CMAKE_SOURCE_DIR}/src/*.c) +list(REMOVE_ITEM SRC_SOURCES + ${CMAKE_SOURCE_DIR}/src/main.c + ${CMAKE_SOURCE_DIR}/src/queue_mode/queue.c + ${CMAKE_SOURCE_DIR}/src/queue_mode/threads.c + ${CMAKE_SOURCE_DIR}/src/queue_mode/queue_dispatch.c +) -add_mocked_test(main - SOURCES ${C_SOURCES} - LINK_LIBRARIES m - ) +add_executable(unit_tests unit_tests.c utils_for_tests.c ${SRC_SOURCES}) +add_executable(sequential_tests sequential_tests.c utils_for_tests.c ${SRC_SOURCES}) +add_executable(parallel_tests parallel_tests.c utils_for_tests.c ${SRC_SOURCES}) -target_include_directories(test_main PRIVATE ${CMAKE_SOURCE_DIR}/stb_image) +target_link_libraries(unit_tests PRIVATE m cmocka) +target_link_libraries(sequential_tests PRIVATE m cmocka) +target_link_libraries(parallel_tests PRIVATE m cmocka) + +add_test(NAME unit_tests COMMAND unit_tests) +add_test(NAME sequential_tests COMMAND sequential_tests) +add_test(NAME parallel_tests COMMAND parallel_tests) diff --git a/tests/benchmarks/comparison_of_composition.py b/tests/benchmarks/comparison_of_composition.py index 81899cd..69125ff 100644 --- a/tests/benchmarks/comparison_of_composition.py +++ b/tests/benchmarks/comparison_of_composition.py @@ -106,7 +106,9 @@ def main() -> None: data_file.write(f"Result for {filter1} + {filter2}\n") - mean_time, confidence_interval = analyze_execution_data(seq_time) + mean_time, confidence_interval = analyze_execution_data( + seq_time, filter_data=True + ) results[num]["means"].append(mean_time) results[num]["conf_inter"].append(confidence_interval) diff --git a/tests/benchmarks/comparison_of_queue_mode.py b/tests/benchmarks/comparison_of_queue_mode.py new file mode 100644 index 0000000..d9b63d4 --- /dev/null +++ b/tests/benchmarks/comparison_of_queue_mode.py @@ -0,0 +1,257 @@ +import os +import numpy as np +import matplotlib.pyplot as plt +import sys +import re +from utils import ( + analyze_execution_data, + print_warning, + round_results, +) + +# Configuration constants +PROGRAM_PATH = "./build/src/image-convolution" +IMAGE_DIR = "input_queue_mode" +NUM_IMAGES = int(sys.argv[1]) +MEM_LIM = int(sys.argv[2]) +NUM_RUNS = 40 +THREAD_NUM = 2 +OUTPUT_DIR = f"tests/plots/queue_mode_{THREAD_NUM}" + +os.makedirs(OUTPUT_DIR, exist_ok=True) + +# Configurations for testing (readers, workers, writers) +CONFIGURATIONS = [ + (1, 1, 1), + (1, 2, 1), + (1, 3, 1), + (1, 3, 2), + (2, 3, 1), + (2, 3, 2), + (1, 4, 1), + (3, 3, 1), + (2, 4, 2), +] + + +def run_program(readers: int, workers: int, writers: int, mem_limit: int): + """ + Runs convolution program in queue mode with specified configuration and memory limit. + + Args: + readers (int): Number of reader threads. + workers (int): Number of worker threads. + writers (int): Number of writer threads. + mem_limit (int): Memory limit in MiB for the program execution. + + Returns: + tuple: A tuple containing four NumPy arrays: + - Total execution times (one per run) + - Reader execution times (one per image) + - Worker execution times (one per image) + - Writer execution times (one per image) + """ + command = ( + f"{PROGRAM_PATH} {IMAGE_DIR} bl --mode=queue --thread={THREAD_NUM} " + f"--readers={readers} --workers={workers} --writers={writers} " + f"--num={NUM_IMAGES} --mem_lim={mem_limit}" + ) + + all_reader_times = [] + all_worker_times = [] + all_writer_times = [] + total_exec_times = [] + + for _ in range(NUM_RUNS): + output = os.popen(command).read() + + reader_times = re.findall(r"READER:.*?in (\d+\.\d+)", output) + worker_times = re.findall(r"WORKER:.*?in (\d+\.\d+)", output) + writer_times = re.findall(r"WRITER:.*?in (\d+\.\d+)", output) + + if not reader_times or not worker_times or not writer_times: + print(f"Failed to parse output") + continue + + if ( + len(reader_times) != NUM_IMAGES + or len(worker_times) != NUM_IMAGES + or len(writer_times) != NUM_IMAGES + ): + print( + f"Incorrect number of records: R:{len(reader_times)}, W:{len(worker_times)}, Wr:{len(writer_times)}" + ) + continue + + all_reader_times.extend(float(t) for t in reader_times) + all_worker_times.extend(float(t) for t in worker_times) + all_writer_times.extend(float(t) for t in writer_times) + + match = re.search(r"The convolution for all images took (\d+\.\d+)", output) + if match: + time = float(match.group(1)) + total_exec_times.append(time) + else: + print_warning(f"Failed to parse output: {output}") + + return ( + np.array(total_exec_times), + np.array(all_reader_times), + np.array(all_worker_times), + np.array(all_writer_times), + ) + + +def main(): + """ + The main function organizes the process of comparative analysis of execution time in the queue mode, + including conducting tests, analyzing results, plotting graphs and saving results. + """ + data_file = open(f"{OUTPUT_DIR}/benchmark_results_{MEM_LIM}MiB.txt", "w+") + + exec_results = { + config: {"means": [], "conf_inter": []} for config in CONFIGURATIONS + } + reader_results = { + config: {"means": [], "conf_inter": []} for config in CONFIGURATIONS + } + worker_results = { + config: {"means": [], "conf_inter": []} for config in CONFIGURATIONS + } + writer_results = { + config: {"means": [], "conf_inter": []} for config in CONFIGURATIONS + } + + data_file.write(f"Memory Limit: {MEM_LIM} MiB\n") + + for config in CONFIGURATIONS: + data_file.write( + f"\tConfig: Readers={config[0]} Workers={config[1]} Writers={config[2]}\n" + ) + + readers, workers, writers = config + print( + f"Testing configuration: readers={readers}, workers={workers}, writers={writers}, mem_limit={MEM_LIM}" + ) + + exec_times, reader_times, worker_times, writer_times = run_program( + readers, workers, writers, MEM_LIM + ) + + # Analyze the times for each stage + def process_stage(res_array, times, filter_data: bool): + mean_time, confidence_interval = analyze_execution_data( + times, filter_data + ) + res_array[config]["means"].append(mean_time) + res_array[config]["conf_inter"].append(confidence_interval) + return round_results(mean_time, confidence_interval) + + r_mean, r_error = process_stage(reader_results, reader_times, False) + w_mean, w_error = process_stage(worker_results, worker_times, False) + wr_mean, wr_error = process_stage(writer_results, writer_times, False) + ex_mean, ex_error = process_stage(exec_results, exec_times, True) + + # Save benchmark results to a text file + data_file.write( + f"\t\t\tReader result: {r_mean} ± {r_error} s\n" + f"\t\t\tWorker result: {w_mean} ± {w_error} s\n" + f"\t\t\tWriter result: {wr_mean} ± {wr_error} s\n" + f"\t\tExecution result: {ex_mean} ± {ex_error} s\n\n" + ) + data_file.write("\n") + + # Graph 1: Total execution time for all configurations + + means = [exec_results[c]["means"][0] for c in CONFIGURATIONS] + errors = [exec_results[c]["conf_inter"][0] for c in CONFIGURATIONS] + + x_pos = np.arange(len(CONFIGURATIONS)) + plt.bar( + x=x_pos, + height=means, + yerr=errors, + align="center", + alpha=0.7, + capsize=10, + color=( + [ + "#efa94a", + "#47a76a", + "#db5856", + "#9966cc", + "#93c6cf", + "#6c3930", + "#aec636", + "#3189e0", + ] + ), + ) + plt.xticks(x_pos, CONFIGURATIONS) + plt.ylabel("Execution Time (s)") + plt.title(f"Queue Mode Performance (Mem Limit: {MEM_LIM} MiB)") + + plot_path = os.path.join( + OUTPUT_DIR, f"queue_performance_comparison_{MEM_LIM}MiB.png" + ) + plt.tight_layout() + plt.savefig(plot_path) + plt.close() + + # Graph 2: Time by stages (reader, worker, writer) for all configurations + + bar_width = 0.15 + stage_colors = { + "Reader": "#93c6cf", + "Worker": "#aec636", + "Writer": "#ae68cf", + } + + x_pos = np.arange(len(CONFIGURATIONS)) + _, ax = plt.subplots(figsize=(12, 6)) + + for i, stage in enumerate(["Reader", "Worker", "Writer"]): + if stage == "Reader": + means = [reader_results[c]["means"][0] for c in CONFIGURATIONS] + errors = [reader_results[c]["conf_inter"][0] for c in CONFIGURATIONS] + elif stage == "Worker": + means = [worker_results[c]["means"][0] for c in CONFIGURATIONS] + errors = [worker_results[c]["conf_inter"][0] for c in CONFIGURATIONS] + else: + means = [writer_results[c]["means"][0] for c in CONFIGURATIONS] + errors = [writer_results[c]["conf_inter"][0] for c in CONFIGURATIONS] + + ax.bar( + x=(x_pos + i * bar_width), + height=means, + yerr=errors, + width=bar_width, + label=stage, + capsize=5, + color=stage_colors[stage], + alpha=0.7, + ) + + plt.xticks(x_pos + bar_width) + ax.set_xticklabels( + [c for c in CONFIGURATIONS], + # ha="right", + ) + plt.ylabel("Execution Time (s)") + plt.title(f"Stage-wise Performance by Configuration ({MEM_LIM} MiB)") + plt.legend(title="Stage") + + plot_path_stages = os.path.join( + OUTPUT_DIR, f"queue_performance_by_stage_{MEM_LIM}MiB.png" + ) + plt.tight_layout() + plt.savefig(plot_path_stages) + plt.close() + + data_file.close() + + print(f"Measurements and charts saved to '{OUTPUT_DIR}' directory.") + + +if __name__ == "__main__": + main() diff --git a/tests/benchmarks/comparison_of_times.py b/tests/benchmarks/comparison_of_times.py index 24d152e..b25a529 100644 --- a/tests/benchmarks/comparison_of_times.py +++ b/tests/benchmarks/comparison_of_times.py @@ -100,7 +100,9 @@ def main() -> None: print(f"Running measurements: {filter} filter - {mode} mode") times = run_program(filter, mode) - mean_time, confidence_interval = analyze_execution_data(times) + mean_time, confidence_interval = analyze_execution_data( + times, filter_data=True + ) results[filter]["means"].append(mean_time) results[filter]["conf_inter"].append(confidence_interval) diff --git a/tests/benchmarks/comparison_using_perf.py b/tests/benchmarks/comparison_using_perf.py index f3b899b..a466fac 100644 --- a/tests/benchmarks/comparison_using_perf.py +++ b/tests/benchmarks/comparison_using_perf.py @@ -91,7 +91,7 @@ def main() -> None: ["cache-references", "cache-misses", "L1-dcache-load_misses"], [c_ref, c_mis, l1_mis], ): - mean_val, conf_inter = analyze_execution_data(data) + mean_val, conf_inter = analyze_execution_data(data, filter_data=True) rounded_mean, rounded_error = int(mean_val), int(conf_inter) results[metric]["means"].append(rounded_mean) results[metric]["conf_inter"].append(rounded_error) diff --git a/tests/benchmarks/utils.py b/tests/benchmarks/utils.py index 551137e..610d9ae 100644 --- a/tests/benchmarks/utils.py +++ b/tests/benchmarks/utils.py @@ -34,13 +34,14 @@ def print_warning(message: str): print(f"{RED_ANSI}{message}{RESET_ANSI}\n") -def analyze_execution_data(data): +def analyze_execution_data(data, filter_data: bool): """ Analyzes execution time data to compute the mean execution time and its confidence interval while performing outlier removal and normality testing. Args: data (array): A list or array of execution times. + filter_data (bool): If True, applies outlier filtering based on standard deviation. Returns: A tuple (mean_time, confidence_interval): @@ -50,17 +51,20 @@ def analyze_execution_data(data): mean_time = np.mean(data) std_time = np.std(data, ddof=1) - # Removing outliers (greater than 3 standard deviations) - filtered_data = data[ - (data > mean_time - 3 * std_time) & (data < mean_time + 3 * std_time) - ] - if len(data) - len(filtered_data) > 1: - print_warning("Too many emissions.") - - normal_test = stats.normaltest(filtered_data) - shapiro_test = stats.shapiro(filtered_data) - if normal_test.pvalue < 0.05 and shapiro_test.pvalue < 0.05: - print_warning("Data does not pass normality tests.") + if filter_data: + # Removing outliers (greater than 3 standard deviations) + filtered_data = data[ + (data > mean_time - 3 * std_time) & (data < mean_time + 3 * std_time) + ] + if len(data) - len(filtered_data) > 1: + print_warning("Too many emissions.") + + normal_test = stats.normaltest(filtered_data) + shapiro_test = stats.shapiro(filtered_data) + if normal_test.pvalue < 0.05 and shapiro_test.pvalue < 0.05: + print_warning("Data does not pass normality tests.") + else: + filtered_data = data mean_time = np.mean(filtered_data) confidence_interval = stats.t.ppf(0.975, df=len(filtered_data) - 1) * stats.sem( diff --git a/tests/parallel_tests.c b/tests/parallel_tests.c index 7c23ad5..62018bb 100644 --- a/tests/parallel_tests.c +++ b/tests/parallel_tests.c @@ -1,7 +1,7 @@ #include "../src/convolution/filter_application.h" #include "../src/convolution/parallel_dispatch.h" -#include "utils_for_tests.c" +#include "utils_for_tests.h" /** * A helper function that runs a parallel test for a given parallel implementation @@ -191,3 +191,22 @@ void test_parallel_block_with_random_image(void **state) { run_test_with_filter(true, parallel_block, &channel_image, width, height, 3); } + +int main(void) { + // Initialize random number generator with current time + srand((unsigned int)time(NULL)); + + const struct CMUnitTest parallel_tests[] = { + cmocka_unit_test(test_parallel_pixel_with_default_image), + cmocka_unit_test(test_parallel_pixel_with_random_image), + cmocka_unit_test(test_parallel_row_with_default_image), + cmocka_unit_test(test_parallel_row_with_random_image), + cmocka_unit_test(test_parallel_column_with_default_image), + cmocka_unit_test(test_parallel_column_with_random_image), + cmocka_unit_test(test_parallel_block_with_default_image), + cmocka_unit_test(test_parallel_block_with_random_image), + }; + + return cmocka_run_group_tests_name("Parallel Application Tests", parallel_tests, + NULL, NULL); +} diff --git a/tests/plots/queue_mode_2/benchmark_results_30MiB.txt b/tests/plots/queue_mode_2/benchmark_results_30MiB.txt new file mode 100644 index 0000000..a397ae1 --- /dev/null +++ b/tests/plots/queue_mode_2/benchmark_results_30MiB.txt @@ -0,0 +1,56 @@ +Memory Limit: 30 MiB + Config: Readers=1 Workers=1 Writers=1 + Reader result: 0.12 ± 0.01 s + Worker result: 0.34 ± 0.003 s + Writer result: 0.344 ± 0.004 s + Execution result: 3.101 ± 0.004 s + + Config: Readers=1 Workers=2 Writers=1 + Reader result: 0.067 ± 0.006 s + Worker result: 0.393 ± 0.007 s + Writer result: 0.22 ± 0.02 s + Execution result: 1.94 ± 0.01 s + + Config: Readers=1 Workers=3 Writers=1 + Reader result: 0.068 ± 0.004 s + Worker result: 0.56 ± 0.01 s + Writer result: 0.23 ± 0.02 s + Execution result: 2.0 ± 0.2 s + + Config: Readers=1 Workers=3 Writers=2 + Reader result: 0.069 ± 0.005 s + Worker result: 0.56 ± 0.01 s + Writer result: 0.42 ± 0.03 s + Execution result: 1.815 ± 0.009 s + + Config: Readers=2 Workers=3 Writers=1 + Reader result: 0.123 ± 0.007 s + Worker result: 0.56 ± 0.01 s + Writer result: 0.21 ± 0.02 s + Execution result: 1.85 ± 0.01 s + + Config: Readers=2 Workers=3 Writers=2 + Reader result: 0.122 ± 0.008 s + Worker result: 0.57 ± 0.01 s + Writer result: 0.4 ± 0.02 s + Execution result: 1.84 ± 0.01 s + + Config: Readers=1 Workers=4 Writers=1 + Reader result: 0.064 ± 0.002 s + Worker result: 0.71 ± 0.02 s + Writer result: 0.21 ± 0.02 s + Execution result: 1.85 ± 0.006 s + + Config: Readers=3 Workers=3 Writers=1 + Reader result: 0.16 ± 0.01 s + Worker result: 0.58 ± 0.01 s + Writer result: 0.21 ± 0.02 s + Execution result: 1.883 ± 0.009 s + + Config: Readers=2 Workers=4 Writers=2 + Reader result: 0.125 ± 0.008 s + Worker result: 0.73 ± 0.02 s + Writer result: 0.4 ± 0.03 s + Execution result: 1.889 ± 0.008 s + + diff --git a/tests/plots/queue_mode_2/benchmark_results_55MiB.txt b/tests/plots/queue_mode_2/benchmark_results_55MiB.txt new file mode 100644 index 0000000..6d7e1db --- /dev/null +++ b/tests/plots/queue_mode_2/benchmark_results_55MiB.txt @@ -0,0 +1,56 @@ +Memory Limit: 55 MiB + Config: Readers=1 Workers=1 Writers=1 + Reader result: 0.0425 ± 0.0001 s + Worker result: 0.34 ± 0.003 s + Writer result: 0.349 ± 0.008 s + Execution result: 3.101 ± 0.004 s + + Config: Readers=1 Workers=2 Writers=1 + Reader result: 0.0425 ± 0.0001 s + Worker result: 0.388 ± 0.007 s + Writer result: 0.22 ± 0.02 s + Execution result: 1.94 ± 0.05 s + + Config: Readers=1 Workers=3 Writers=1 + Reader result: 0.057 ± 0.001 s + Worker result: 0.55 ± 0.01 s + Writer result: 0.21 ± 0.02 s + Execution result: 1.83 ± 0.01 s + + Config: Readers=1 Workers=3 Writers=2 + Reader result: 0.06 ± 0.001 s + Worker result: 0.56 ± 0.01 s + Writer result: 0.39 ± 0.02 s + Execution result: 1.82 ± 0.01 s + + Config: Readers=2 Workers=3 Writers=1 + Reader result: 0.109 ± 0.005 s + Worker result: 0.56 ± 0.01 s + Writer result: 0.21 ± 0.02 s + Execution result: 1.86 ± 0.01 s + + Config: Readers=2 Workers=3 Writers=2 + Reader result: 0.106 ± 0.005 s + Worker result: 0.56 ± 0.01 s + Writer result: 0.4 ± 0.02 s + Execution result: 1.84 ± 0.01 s + + Config: Readers=1 Workers=4 Writers=1 + Reader result: 0.064 ± 0.002 s + Worker result: 0.71 ± 0.02 s + Writer result: 0.21 ± 0.02 s + Execution result: 1.848 ± 0.008 s + + Config: Readers=3 Workers=3 Writers=1 + Reader result: 0.142 ± 0.007 s + Worker result: 0.57 ± 0.02 s + Writer result: 0.25 ± 0.03 s + Execution result: 2.0 ± 0.2 s + + Config: Readers=2 Workers=4 Writers=2 + Reader result: 0.129 ± 0.008 s + Worker result: 0.73 ± 0.02 s + Writer result: 0.4 ± 0.03 s + Execution result: 1.891 ± 0.006 s + + diff --git a/tests/plots/queue_mode_2/benchmark_results_6MiB.txt b/tests/plots/queue_mode_2/benchmark_results_6MiB.txt new file mode 100644 index 0000000..548aaa3 --- /dev/null +++ b/tests/plots/queue_mode_2/benchmark_results_6MiB.txt @@ -0,0 +1,56 @@ +Memory Limit: 6 MiB + Config: Readers=1 Workers=1 Writers=1 + Reader result: 0.27 ± 0.01 s + Worker result: 0.336 ± 0.002 s + Writer result: 0.34 ± 0.003 s + Execution result: 3.057 ± 0.002 s + + Config: Readers=1 Workers=2 Writers=1 + Reader result: 0.14 ± 0.01 s + Worker result: 0.385 ± 0.004 s + Writer result: 0.21 ± 0.01 s + Execution result: 1.89 ± 0.01 s + + Config: Readers=1 Workers=3 Writers=1 + Reader result: 0.14 ± 0.02 s + Worker result: 0.65 ± 0.05 s + Writer result: 0.25 ± 0.03 s + Execution result: 2.0 ± 0.2 s + + Config: Readers=1 Workers=3 Writers=2 + Reader result: 0.14 ± 0.01 s + Worker result: 0.55 ± 0.01 s + Writer result: 0.38 ± 0.02 s + Execution result: 1.78 ± 0.01 s + + Config: Readers=2 Workers=3 Writers=1 + Reader result: 0.26 ± 0.02 s + Worker result: 0.55 ± 0.01 s + Writer result: 0.2 ± 0.02 s + Execution result: 1.788 ± 0.008 s + + Config: Readers=2 Workers=3 Writers=2 + Reader result: 0.26 ± 0.02 s + Worker result: 0.55 ± 0.01 s + Writer result: 0.39 ± 0.02 s + Execution result: 1.78 ± 0.01 s + + Config: Readers=1 Workers=4 Writers=1 + Reader result: 0.11 ± 0.01 s + Worker result: 0.7 ± 0.01 s + Writer result: 0.2 ± 0.02 s + Execution result: 1.812 ± 0.007 s + + Config: Readers=3 Workers=3 Writers=1 + Reader result: 0.37 ± 0.03 s + Worker result: 0.63 ± 0.04 s + Writer result: 0.24 ± 0.03 s + Execution result: 1.9 ± 0.2 s + + Config: Readers=2 Workers=4 Writers=2 + Reader result: 0.22 ± 0.02 s + Worker result: 0.71 ± 0.01 s + Writer result: 0.38 ± 0.03 s + Execution result: 1.808 ± 0.007 s + + diff --git a/tests/plots/queue_mode_2/queue_performance_by_stage_30MiB.png b/tests/plots/queue_mode_2/queue_performance_by_stage_30MiB.png new file mode 100644 index 0000000..063f950 Binary files /dev/null and b/tests/plots/queue_mode_2/queue_performance_by_stage_30MiB.png differ diff --git a/tests/plots/queue_mode_2/queue_performance_by_stage_55MiB.png b/tests/plots/queue_mode_2/queue_performance_by_stage_55MiB.png new file mode 100644 index 0000000..191dd18 Binary files /dev/null and b/tests/plots/queue_mode_2/queue_performance_by_stage_55MiB.png differ diff --git a/tests/plots/queue_mode_2/queue_performance_by_stage_6MiB.png b/tests/plots/queue_mode_2/queue_performance_by_stage_6MiB.png new file mode 100644 index 0000000..111c74d Binary files /dev/null and b/tests/plots/queue_mode_2/queue_performance_by_stage_6MiB.png differ diff --git a/tests/plots/queue_mode_2/queue_performance_comparison_30MiB.png b/tests/plots/queue_mode_2/queue_performance_comparison_30MiB.png new file mode 100644 index 0000000..a2c1ad0 Binary files /dev/null and b/tests/plots/queue_mode_2/queue_performance_comparison_30MiB.png differ diff --git a/tests/plots/queue_mode_2/queue_performance_comparison_55MiB.png b/tests/plots/queue_mode_2/queue_performance_comparison_55MiB.png new file mode 100644 index 0000000..bd497ab Binary files /dev/null and b/tests/plots/queue_mode_2/queue_performance_comparison_55MiB.png differ diff --git a/tests/plots/queue_mode_2/queue_performance_comparison_6MiB.png b/tests/plots/queue_mode_2/queue_performance_comparison_6MiB.png new file mode 100644 index 0000000..b4ae47e Binary files /dev/null and b/tests/plots/queue_mode_2/queue_performance_comparison_6MiB.png differ diff --git a/tests/plots/queue_mode_3/benchmark_results_30MiB.txt b/tests/plots/queue_mode_3/benchmark_results_30MiB.txt new file mode 100644 index 0000000..94ae3e5 --- /dev/null +++ b/tests/plots/queue_mode_3/benchmark_results_30MiB.txt @@ -0,0 +1,26 @@ +Memory Limit: 30 MiB + Config: Readers=1 Workers=1 Writers=1 + Reader result: 0.093 ± 0.008 s + Worker result: 0.236 ± 0.003 s + Writer result: 0.241 ± 0.004 s + Execution result: 2.165 ± 0.003 s + + Config: Readers=1 Workers=2 Writers=1 + Reader result: 0.067 ± 0.002 s + Worker result: 0.372 ± 0.008 s + Writer result: 0.2 ± 0.01 s + Execution result: 1.79 ± 0.01 s + + Config: Readers=1 Workers=2 Writers=2 + Reader result: 0.067 ± 0.002 s + Worker result: 0.374 ± 0.008 s + Writer result: 0.38 ± 0.01 s + Execution result: 1.785 ± 0.006 s + + Config: Readers=1 Workers=3 Writers=1 + Reader result: 0.073 ± 0.003 s + Worker result: 0.551 ± 0.009 s + Writer result: 0.2 ± 0.02 s + Execution result: 1.77 ± 0.01 s + + diff --git a/tests/plots/queue_mode_3/benchmark_results_55MiB.txt b/tests/plots/queue_mode_3/benchmark_results_55MiB.txt new file mode 100644 index 0000000..a02ad10 --- /dev/null +++ b/tests/plots/queue_mode_3/benchmark_results_55MiB.txt @@ -0,0 +1,26 @@ +Memory Limit: 55 MiB + Config: Readers=1 Workers=1 Writers=1 + Reader result: 0.0425 ± 0.0001 s + Worker result: 0.234 ± 0.003 s + Writer result: 0.238 ± 0.004 s + Execution result: 2.146 ± 0.001 s + + Config: Readers=1 Workers=2 Writers=1 + Reader result: 0.061 ± 0.001 s + Worker result: 0.375 ± 0.008 s + Writer result: 0.24 ± 0.03 s + Execution result: 1.9 ± 0.2 s + + Config: Readers=1 Workers=2 Writers=2 + Reader result: 0.061 ± 0.001 s + Worker result: 0.374 ± 0.008 s + Writer result: 0.38 ± 0.01 s + Execution result: 1.781 ± 0.007 s + + Config: Readers=1 Workers=3 Writers=1 + Reader result: 0.068 ± 0.002 s + Worker result: 0.555 ± 0.009 s + Writer result: 0.2 ± 0.02 s + Execution result: 1.772 ± 0.009 s + + diff --git a/tests/plots/queue_mode_3/benchmark_results_6MiB.txt b/tests/plots/queue_mode_3/benchmark_results_6MiB.txt new file mode 100644 index 0000000..015ca35 --- /dev/null +++ b/tests/plots/queue_mode_3/benchmark_results_6MiB.txt @@ -0,0 +1,26 @@ +Memory Limit: 6 MiB + Config: Readers=1 Workers=1 Writers=1 + Reader result: 0.21 ± 0.02 s + Worker result: 0.27 ± 0.02 s + Writer result: 0.28 ± 0.02 s + Execution result: 2.4 ± 0.2 s + + Config: Readers=1 Workers=2 Writers=1 + Reader result: 0.14 ± 0.01 s + Worker result: 0.368 ± 0.006 s + Writer result: 0.2 ± 0.01 s + Execution result: 1.759 ± 0.009 s + + Config: Readers=1 Workers=2 Writers=2 + Reader result: 0.14 ± 0.01 s + Worker result: 0.364 ± 0.006 s + Writer result: 0.37 ± 0.008 s + Execution result: 1.749 ± 0.006 s + + Config: Readers=1 Workers=3 Writers=1 + Reader result: 0.13 ± 0.01 s + Worker result: 0.538 ± 0.009 s + Writer result: 0.19 ± 0.02 s + Execution result: 1.735 ± 0.008 s + + diff --git a/tests/plots/queue_mode_3/queue_performance_by_stage_30MiB.png b/tests/plots/queue_mode_3/queue_performance_by_stage_30MiB.png new file mode 100644 index 0000000..c40e91e Binary files /dev/null and b/tests/plots/queue_mode_3/queue_performance_by_stage_30MiB.png differ diff --git a/tests/plots/queue_mode_3/queue_performance_by_stage_55MiB.png b/tests/plots/queue_mode_3/queue_performance_by_stage_55MiB.png new file mode 100644 index 0000000..588fa1d Binary files /dev/null and b/tests/plots/queue_mode_3/queue_performance_by_stage_55MiB.png differ diff --git a/tests/plots/queue_mode_3/queue_performance_by_stage_6MiB.png b/tests/plots/queue_mode_3/queue_performance_by_stage_6MiB.png new file mode 100644 index 0000000..64076c8 Binary files /dev/null and b/tests/plots/queue_mode_3/queue_performance_by_stage_6MiB.png differ diff --git a/tests/plots/queue_mode_3/queue_performance_comparison_30MiB.png b/tests/plots/queue_mode_3/queue_performance_comparison_30MiB.png new file mode 100644 index 0000000..8e0990a Binary files /dev/null and b/tests/plots/queue_mode_3/queue_performance_comparison_30MiB.png differ diff --git a/tests/plots/queue_mode_3/queue_performance_comparison_55MiB.png b/tests/plots/queue_mode_3/queue_performance_comparison_55MiB.png new file mode 100644 index 0000000..244ee2a Binary files /dev/null and b/tests/plots/queue_mode_3/queue_performance_comparison_55MiB.png differ diff --git a/tests/plots/queue_mode_3/queue_performance_comparison_6MiB.png b/tests/plots/queue_mode_3/queue_performance_comparison_6MiB.png new file mode 100644 index 0000000..361000e Binary files /dev/null and b/tests/plots/queue_mode_3/queue_performance_comparison_6MiB.png differ diff --git a/tests/plots/queue_mode_4/benchmark_results_30MiB.txt b/tests/plots/queue_mode_4/benchmark_results_30MiB.txt new file mode 100644 index 0000000..c901675 --- /dev/null +++ b/tests/plots/queue_mode_4/benchmark_results_30MiB.txt @@ -0,0 +1,26 @@ +Memory Limit: 30 MiB + Config: Readers=1 Workers=1 Writers=1 + Reader result: 0.082 ± 0.006 s + Worker result: 0.199 ± 0.004 s + Writer result: 0.203 ± 0.006 s + Execution result: 1.832 ± 0.003 s + + Config: Readers=1 Workers=2 Writers=1 + Reader result: 0.066 ± 0.002 s + Worker result: 0.367 ± 0.007 s + Writer result: 0.2 ± 0.01 s + Execution result: 1.759 ± 0.006 s + + Config: Readers=1 Workers=1 Writers=2 + Reader result: 0.082 ± 0.006 s + Worker result: 0.201 ± 0.004 s + Writer result: 0.42 ± 0.02 s + Execution result: 1.91 ± 0.08 s + + Config: Readers=2 Workers=1 Writers=1 + Reader result: 0.15 ± 0.01 s + Worker result: 0.203 ± 0.005 s + Writer result: 0.22 ± 0.01 s + Execution result: 2.0 ± 0.1 s + + diff --git a/tests/plots/queue_mode_4/benchmark_results_55MiB.txt b/tests/plots/queue_mode_4/benchmark_results_55MiB.txt new file mode 100644 index 0000000..174d824 --- /dev/null +++ b/tests/plots/queue_mode_4/benchmark_results_55MiB.txt @@ -0,0 +1,26 @@ +Memory Limit: 55 MiB + Config: Readers=1 Workers=1 Writers=1 + Reader result: 0.0433 ± 0.0005 s + Worker result: 0.2 ± 0.004 s + Writer result: 0.204 ± 0.006 s + Execution result: 1.837 ± 0.004 s + + Config: Readers=1 Workers=2 Writers=1 + Reader result: 0.061 ± 0.001 s + Worker result: 0.367 ± 0.007 s + Writer result: 0.24 ± 0.03 s + Execution result: 2.0 ± 0.3 s + + Config: Readers=1 Workers=1 Writers=2 + Reader result: 0.0427 ± 0.0003 s + Worker result: 0.199 ± 0.004 s + Writer result: 0.388 ± 0.007 s + Execution result: 1.833 ± 0.003 s + + Config: Readers=2 Workers=1 Writers=1 + Reader result: 0.097 ± 0.004 s + Worker result: 0.207 ± 0.006 s + Writer result: 0.212 ± 0.007 s + Execution result: 1.9 ± 0.01 s + + diff --git a/tests/plots/queue_mode_4/benchmark_results_6MiB.txt b/tests/plots/queue_mode_4/benchmark_results_6MiB.txt new file mode 100644 index 0000000..ee6e713 --- /dev/null +++ b/tests/plots/queue_mode_4/benchmark_results_6MiB.txt @@ -0,0 +1,26 @@ +Memory Limit: 6 MiB + Config: Readers=1 Workers=1 Writers=1 + Reader result: 0.16 ± 0.007 s + Worker result: 0.195 ± 0.001 s + Writer result: 0.2 ± 0.004 s + Execution result: 1.796 ± 0.004 s + + Config: Readers=1 Workers=2 Writers=1 + Reader result: 0.14 ± 0.01 s + Worker result: 0.36 ± 0.005 s + Writer result: 0.19 ± 0.01 s + Execution result: 1.726 ± 0.006 s + + Config: Readers=1 Workers=1 Writers=2 + Reader result: 0.159 ± 0.007 s + Worker result: 0.195 ± 0.001 s + Writer result: 0.378 ± 0.005 s + Execution result: 1.793 ± 0.005 s + + Config: Readers=2 Workers=1 Writers=1 + Reader result: 0.3 ± 0.01 s + Worker result: 0.197 ± 0.002 s + Writer result: 0.202 ± 0.004 s + Execution result: 1.805 ± 0.007 s + + diff --git a/tests/plots/queue_mode_4/queue_performance_by_stage_30MiB.png b/tests/plots/queue_mode_4/queue_performance_by_stage_30MiB.png new file mode 100644 index 0000000..0b46e95 Binary files /dev/null and b/tests/plots/queue_mode_4/queue_performance_by_stage_30MiB.png differ diff --git a/tests/plots/queue_mode_4/queue_performance_by_stage_55MiB.png b/tests/plots/queue_mode_4/queue_performance_by_stage_55MiB.png new file mode 100644 index 0000000..57b5bc2 Binary files /dev/null and b/tests/plots/queue_mode_4/queue_performance_by_stage_55MiB.png differ diff --git a/tests/plots/queue_mode_4/queue_performance_by_stage_6MiB.png b/tests/plots/queue_mode_4/queue_performance_by_stage_6MiB.png new file mode 100644 index 0000000..921d8e9 Binary files /dev/null and b/tests/plots/queue_mode_4/queue_performance_by_stage_6MiB.png differ diff --git a/tests/plots/queue_mode_4/queue_performance_comparison_30MiB.png b/tests/plots/queue_mode_4/queue_performance_comparison_30MiB.png new file mode 100644 index 0000000..8f369ca Binary files /dev/null and b/tests/plots/queue_mode_4/queue_performance_comparison_30MiB.png differ diff --git a/tests/plots/queue_mode_4/queue_performance_comparison_55MiB.png b/tests/plots/queue_mode_4/queue_performance_comparison_55MiB.png new file mode 100644 index 0000000..459cfcd Binary files /dev/null and b/tests/plots/queue_mode_4/queue_performance_comparison_55MiB.png differ diff --git a/tests/plots/queue_mode_4/queue_performance_comparison_6MiB.png b/tests/plots/queue_mode_4/queue_performance_comparison_6MiB.png new file mode 100644 index 0000000..0e50215 Binary files /dev/null and b/tests/plots/queue_mode_4/queue_performance_comparison_6MiB.png differ diff --git a/tests/plots/queue_mode_5/benchmark_results_30MiB.txt b/tests/plots/queue_mode_5/benchmark_results_30MiB.txt new file mode 100644 index 0000000..679d07f --- /dev/null +++ b/tests/plots/queue_mode_5/benchmark_results_30MiB.txt @@ -0,0 +1,8 @@ +Memory Limit: 30 MiB + Config: Readers=1 Workers=1 Writers=1 + Reader result: 0.082 ± 0.006 s + Worker result: 0.195 ± 0.004 s + Writer result: 0.199 ± 0.006 s + Execution result: 1.796 ± 0.004 s + + diff --git a/tests/plots/queue_mode_5/benchmark_results_55MiB.txt b/tests/plots/queue_mode_5/benchmark_results_55MiB.txt new file mode 100644 index 0000000..8a20f96 --- /dev/null +++ b/tests/plots/queue_mode_5/benchmark_results_55MiB.txt @@ -0,0 +1,8 @@ +Memory Limit: 55 MiB + Config: Readers=1 Workers=1 Writers=1 + Reader result: 0.048 ± 0.001 s + Worker result: 0.2 ± 0.004 s + Writer result: 0.204 ± 0.006 s + Execution result: 1.83 ± 0.01 s + + diff --git a/tests/plots/queue_mode_5/benchmark_results_6MiB.txt b/tests/plots/queue_mode_5/benchmark_results_6MiB.txt new file mode 100644 index 0000000..72842f4 --- /dev/null +++ b/tests/plots/queue_mode_5/benchmark_results_6MiB.txt @@ -0,0 +1,8 @@ +Memory Limit: 6 MiB + Config: Readers=1 Workers=1 Writers=1 + Reader result: 0.157 ± 0.006 s + Worker result: 0.19 ± 0.002 s + Writer result: 0.194 ± 0.004 s + Execution result: 1.74 ± 0.01 s + + diff --git a/tests/plots/queue_mode_5/queue_performance_by_stage_30MiB.png b/tests/plots/queue_mode_5/queue_performance_by_stage_30MiB.png new file mode 100644 index 0000000..b26293a Binary files /dev/null and b/tests/plots/queue_mode_5/queue_performance_by_stage_30MiB.png differ diff --git a/tests/plots/queue_mode_5/queue_performance_by_stage_55MiB.png b/tests/plots/queue_mode_5/queue_performance_by_stage_55MiB.png new file mode 100644 index 0000000..66d3c12 Binary files /dev/null and b/tests/plots/queue_mode_5/queue_performance_by_stage_55MiB.png differ diff --git a/tests/plots/queue_mode_5/queue_performance_by_stage_6MiB.png b/tests/plots/queue_mode_5/queue_performance_by_stage_6MiB.png new file mode 100644 index 0000000..46a9a37 Binary files /dev/null and b/tests/plots/queue_mode_5/queue_performance_by_stage_6MiB.png differ diff --git a/tests/plots/queue_mode_5/queue_performance_comparison_30MiB.png b/tests/plots/queue_mode_5/queue_performance_comparison_30MiB.png new file mode 100644 index 0000000..3bca272 Binary files /dev/null and b/tests/plots/queue_mode_5/queue_performance_comparison_30MiB.png differ diff --git a/tests/plots/queue_mode_5/queue_performance_comparison_55MiB.png b/tests/plots/queue_mode_5/queue_performance_comparison_55MiB.png new file mode 100644 index 0000000..42b9d3c Binary files /dev/null and b/tests/plots/queue_mode_5/queue_performance_comparison_55MiB.png differ diff --git a/tests/plots/queue_mode_5/queue_performance_comparison_6MiB.png b/tests/plots/queue_mode_5/queue_performance_comparison_6MiB.png new file mode 100644 index 0000000..ca2687f Binary files /dev/null and b/tests/plots/queue_mode_5/queue_performance_comparison_6MiB.png differ diff --git a/tests/sequential_tests.c b/tests/sequential_tests.c index d401878..3a06aa7 100644 --- a/tests/sequential_tests.c +++ b/tests/sequential_tests.c @@ -1,7 +1,7 @@ #include "../src/convolution/filter_application.h" #include "../src/convolution/parallel_dispatch.h" -#include "utils_for_tests.c" +#include "utils_for_tests.h" #define MAX_RANDOM_FILTER_SIZE 9 @@ -304,3 +304,20 @@ void test_filter_zero_padding_with_random_image(void **state) { free_filter(&filter); free_filter(&padded_filter); } + +int main(void) { + // Initialize random number generator with current time + srand((unsigned int)time(NULL)); + + const struct CMUnitTest sequential_tests[] = { + cmocka_unit_test(test_filter_compose_with_default_image), + cmocka_unit_test(test_filter_compose_with_random_image), + cmocka_unit_test(test_filter_inverse_with_default_image), + cmocka_unit_test(test_filter_inverse_with_random_image), + cmocka_unit_test(test_filter_zero_padding_with_default_image), + cmocka_unit_test(test_filter_zero_padding_with_random_image), + }; + + return cmocka_run_group_tests_name("Sequential Application Tests", + sequential_tests, NULL, NULL); +} diff --git a/tests/test_main.c b/tests/test_main.c deleted file mode 100644 index f4af0ea..0000000 --- a/tests/test_main.c +++ /dev/null @@ -1,60 +0,0 @@ -#include -#include -#include - -#include -#include - -#define STB_IMAGE_IMPLEMENTATION -#include "stb_image.h" -#define STB_IMAGE_WRITE_IMPLEMENTATION -#include "stb_image_write.h" - -#include "parallel_tests.c" -#include "sequential_tests.c" -#include "unit_tests.c" - -/** - * The main function initializes the random number generator using the current time - * and runs all test groups sequentially. - */ -int main(void) { - // Initialize random number generator with current time - srand((unsigned int)time(NULL)); - - const struct CMUnitTest core_tests[] = { - cmocka_unit_test(test_create_filter), - cmocka_unit_test(test_split_assemble_channels), - cmocka_unit_test(test_identity_filter), - }; - - const struct CMUnitTest sequential_tests[] = { - cmocka_unit_test(test_filter_compose_with_default_image), - cmocka_unit_test(test_filter_compose_with_random_image), - cmocka_unit_test(test_filter_inverse_with_default_image), - cmocka_unit_test(test_filter_inverse_with_random_image), - cmocka_unit_test(test_filter_zero_padding_with_default_image), - cmocka_unit_test(test_filter_zero_padding_with_random_image), - }; - - const struct CMUnitTest parallel_tests[] = { - cmocka_unit_test(test_parallel_pixel_with_default_image), - cmocka_unit_test(test_parallel_pixel_with_random_image), - cmocka_unit_test(test_parallel_row_with_default_image), - cmocka_unit_test(test_parallel_row_with_random_image), - cmocka_unit_test(test_parallel_column_with_default_image), - cmocka_unit_test(test_parallel_column_with_random_image), - cmocka_unit_test(test_parallel_block_with_default_image), - cmocka_unit_test(test_parallel_block_with_random_image), - }; - - int status = 0; - status |= cmocka_run_group_tests_name("Core Functionality Tests", core_tests, - NULL, NULL); - status |= cmocka_run_group_tests_name("Sequential Application Tests", - sequential_tests, NULL, NULL); - status |= cmocka_run_group_tests_name("Parallel Application Tests", - parallel_tests, NULL, NULL); - - return status; -} diff --git a/tests/unit_tests.c b/tests/unit_tests.c index e388a46..cc340d7 100644 --- a/tests/unit_tests.c +++ b/tests/unit_tests.c @@ -1,6 +1,6 @@ #include "../src/convolution/filter_application.h" -#include "utils_for_tests.c" +#include "utils_for_tests.h" #define IMAGE_WIDTH 2 #define IMAGE_HEIGHT 2 @@ -98,3 +98,17 @@ void test_identity_filter(void **state) { free_image_rgb(&channel_image); free_image_rgb(&result_channel_image); } + +int main(void) { + // Initialize random number generator with current time + srand((unsigned int)time(NULL)); + + const struct CMUnitTest core_tests[] = { + cmocka_unit_test(test_create_filter), + cmocka_unit_test(test_split_assemble_channels), + cmocka_unit_test(test_identity_filter), + }; + + return cmocka_run_group_tests_name("Core Functionality Tests", core_tests, NULL, + NULL); +} diff --git a/tests/utils_for_tests.c b/tests/utils_for_tests.c index 3edcf40..2cfe514 100644 --- a/tests/utils_for_tests.c +++ b/tests/utils_for_tests.c @@ -1,21 +1,11 @@ -#pragma once - #include "../src/filters/filter.h" #include "../src/utils/utils.h" -#define UPPER_SIZE_LIMIT 2500 // To ensure that the filter is not used for too long. -#define MIN_FACTOR 0.0005 -#define MAX_FACTOR 1.0 - -/** - * Initializes an image_rgb structure with the specified dimensions and verifies that - * memory allocation for its red, green, and blue channels is successful. - * - * @param width Image width. - * @param height Image height. - * - * @return An initialized `structure image_rgb`. - */ +#define STB_IMAGE_IMPLEMENTATION +#define STB_IMAGE_WRITE_IMPLEMENTATION + +#include "utils_for_tests.h" + struct image_rgb initialize_and_check_image_rgb(int width, int height) { struct image_rgb image = initialize_image_rgb(width, height); assert_non_null(image.red); @@ -24,15 +14,6 @@ struct image_rgb initialize_and_check_image_rgb(int width, int height) { return image; } -/** - * Creates a random test image with the specified dimensions. Each pixel's red, - * green, and blue components are assigned random values between 0 and 255. - * - * @param width Image width. - * @param height Image height. - * - * @return A `structure image_rgb` representing a randomly generated test image. - */ struct image_rgb create_test_image(int width, int height) { struct image_rgb image = initialize_and_check_image_rgb(width, height); for (size_t i = 0; i < (size_t)width * (size_t)height; i++) { @@ -43,17 +24,6 @@ struct image_rgb create_test_image(int width, int height) { return image; } -/** - * Create a random filter. - * Core elements >= 0 and bias = 0 to avoid truncating values less than zero and - * greater than 255. - * - * @param size The size of the filter kernel. - * @param kernel A 2D array representing the kernel matrix. The function populates - * this array with random values. - * - * @return A `struct filter` randomly generated filter. - */ struct filter generate_random_filter(int size, double kernel[size][size]) { struct filter random_filter; random_filter.size = size; @@ -78,16 +48,6 @@ struct filter generate_random_filter(int size, double kernel[size][size]) { return random_filter; } -/** - * Compares two images channel by channel for exact equality. - * - * @param expected Pointer to the expected image. - * @param actual Pointer to the actual image. - * @param width Image width. - * @param height Image height. - * - * @return true if channels are identical, false otherwise. - */ bool compare_channels(struct image_rgb *expected, struct image_rgb *actual, int width, int height) { for (size_t i = 0; i < (size_t)width * (size_t)height; i++) { @@ -100,16 +60,6 @@ bool compare_channels(struct image_rgb *expected, struct image_rgb *actual, return true; } -/** - * Compares two images channel by channel with a tolerance of 1. - * - * @param expected Pointer to the expected image. - * @param actual Pointer to the actual image. - * @param width Image width. - * @param height Image height. - * - * @return true if channels are similar, false otherwise. - */ bool compare_channels_with_epsilon(struct image_rgb *expected, struct image_rgb *actual, int width, int height) { for (size_t i = 0; i < (size_t)width * (size_t)height; i++) { @@ -122,12 +72,6 @@ bool compare_channels_with_epsilon(struct image_rgb *expected, return true; } -/** - * Print filter details for debugging. - * - * @param filter Pointer to the filter. - * @param name Name of the filter. - */ void print_filter(const struct filter *filter, const char *name) { printf("%s:\n", name); printf("{\n"); @@ -151,15 +95,6 @@ void print_filter(const struct filter *filter, const char *name) { printf("\n"); } -/** - * Applies zero padding to a convolution filter by embedding the original - * filter's kernel into the center of a larger kernel filled with zeros. - * - * @param padded_filter Pointer to the filter that will receive the zero-padded - * kernel. - * @param original_filter Pointer to the original filter whose kernel will be - * embedded. - */ void apply_zero_padding(struct filter *padded_filter, struct filter *original_filter) { // Filling the extended filter kernel with zeros @@ -179,14 +114,6 @@ void apply_zero_padding(struct filter *padded_filter, } } -/** - * Save an image for debugging failed tests. - * - * @param image Image to save. - * @param width Image width. - * @param height Image height. - * @param test_name Name of the test (used for filename). - */ void save_image(struct image_rgb image, int width, int height, char *test_name) { unsigned char *result_image = malloc((size_t)width * (size_t)height * (size_t)3); assert_non_null(result_image); diff --git a/tests/utils_for_tests.h b/tests/utils_for_tests.h new file mode 100644 index 0000000..7f96c23 --- /dev/null +++ b/tests/utils_for_tests.h @@ -0,0 +1,107 @@ +#pragma once + +#include +#include +#include + +#include +#include +#include + +#include "../stb_image/stb_image.h" +#include "../stb_image/stb_image_write.h" + +#define UPPER_SIZE_LIMIT 2500 // To ensure that the filter is not used for too long. +#define MIN_FACTOR 0.0005 +#define MAX_FACTOR 1.0 + +/** + * Initializes an image_rgb structure with the specified dimensions and verifies that + * memory allocation for its red, green, and blue channels is successful. + * + * @param width Image width. + * @param height Image height. + * + * @return An initialized `structure image_rgb`. + */ +struct image_rgb initialize_and_check_image_rgb(int width, int height); + +/** + * Creates a random test image with the specified dimensions. Each pixel's red, + * green, and blue components are assigned random values between 0 and 255. + * + * @param width Image width. + * @param height Image height. + * + * @return A `structure image_rgb` representing a randomly generated test image. + */ +struct image_rgb create_test_image(int width, int height); + +/** + * Create a random filter. + * Core elements >= 0 and bias = 0 to avoid truncating values less than zero and + * greater than 255. + * + * @param size The size of the filter kernel. + * @param kernel A 2D array representing the kernel matrix. The function populates + * this array with random values. + * + * @return A `struct filter` randomly generated filter. + */ +struct filter generate_random_filter(int size, double kernel[size][size]); + +/** + * Compares two images channel by channel for exact equality. + * + * @param expected Pointer to the expected image. + * @param actual Pointer to the actual image. + * @param width Image width. + * @param height Image height. + * + * @return true if channels are identical, false otherwise. + */ +bool compare_channels(struct image_rgb *expected, struct image_rgb *actual, + int width, int height); + +/** + * Compares two images channel by channel with a tolerance of 1. + * + * @param expected Pointer to the expected image. + * @param actual Pointer to the actual image. + * @param width Image width. + * @param height Image height. + * + * @return true if channels are similar, false otherwise. + */ +bool compare_channels_with_epsilon(struct image_rgb *expected, + struct image_rgb *actual, int width, int height); + +/** + * Print filter details for debugging. + * + * @param filter Pointer to the filter. + * @param name Name of the filter. + */ +void print_filter(const struct filter *filter, const char *name); + +/** + * Applies zero padding to a convolution filter by embedding the original + * filter's kernel into the center of a larger kernel filled with zeros. + * + * @param padded_filter Pointer to the filter that will receive the zero-padded + * kernel. + * @param original_filter Pointer to the original filter whose kernel will be + * embedded. + */ +void apply_zero_padding(struct filter *padded_filter, + struct filter *original_filter); + +/** + * Save an image for debugging failed tests. + * + * @param image Image to save. + * @param width Image width. + * @param height Image height. + * @param test_name Name of the test (used for filename). + */ +void save_image(struct image_rgb image, int width, int height, char *test_name);