@@ -4,22 +4,69 @@ This document outlines the benchmarking process for comparing the performance of
4
4
5
5
> About tests performance can be found at: [ ![ CodSpeed Badge] ( https://img.shields.io/endpoint?url=https://codspeed.io/badge.json )] ( https://codspeed.io/cpp-linter/cpp-linter-hooks )
6
6
7
- ## Running the Benchmark
7
+ ## Run benchmark separately
8
8
9
9
``` bash
10
- python3 testing/benchmark_hooks.py
10
+ rm -rf examples || true
11
+ git clone --depth 1 --quiet https://github.com/gouravthakur39/beginners-C-program-examples.git examples
12
+
13
+ pre-commit clean
14
+ hyperfine --warmup 1 -r 10 ' pre-commit run --config testing/test-cpp-linter-hooks.yaml --all-files'
15
+
16
+ rm -rf examples || true
17
+ git clone --depth 1 --quiet https://github.com/gouravthakur39/beginners-C-program-examples.git examples
18
+ pre-commit clean
19
+
20
+ hyperfine --warmup 1 -r 10 ' pre-commit run --config testing/test-mirrors-clang-format.yaml --all-files'
21
+ ```
22
+
23
+ Results:
24
+
25
+ ``` bash
26
+ # Updated on 2025-09-19
27
+
28
+ Cleaned /home/sxp/.cache/pre-commit.
29
+ Benchmark 1: pre-commit run --config testing/test-cpp-linter-hooks.yaml --all-files
30
+ Time (mean ± σ): 150.2 ms ± 1.8 ms [User: 121.7 ms, System: 29.2 ms]
31
+ Range (min … max): 148.3 ms … 153.9 ms 10 runs
32
+
33
+ Cleaned /home/sxp/.cache/pre-commit.
34
+ Benchmark 1: pre-commit run --config testing/test-mirrors-clang-format.yaml --all-files
35
+ Time (mean ± σ): 122.6 ms ± 1.9 ms [User: 98.0 ms, System: 24.7 ms]
36
+ Range (min … max): 120.3 ms … 125.5 ms 10 runs
11
37
```
12
38
13
- ## Results
39
+ ### Run benchmark comparison
40
+
41
+ Compare the results of both commands.
14
42
15
43
``` bash
16
- # Updated on 2025-09-02
17
- Benchmark Results:
44
+ rm -rf examples || true
45
+ git clone --depth 1 --quiet https://github.com/gouravthakur39/beginners-C-program-examples.git examples
46
+
47
+ hyperfine -i --warmup 1 -r 20 ' pre-commit run --config ../testing/test-cpp-linter-hooks.yaml --all-files' ' pre-commit run --config ../testing/test-mirrors-clang-format.yaml --all-files'
48
+ ```
18
49
19
- Hook | Avg (s) | Std (s) | Min (s) | Max (s) | Runs
20
- ---------------------+------------------+------------------+------------------+------------------+-----------------
21
- cpp-linter-hooks | 12.473 | 1.738 | 11.334 | 15.514 | 5
22
- mirrors-clang-format | 4.960 | 0.229 | 4.645 | 5.284 | 5
50
+ Results:
51
+
52
+ ``` bash
53
+ # Updated on 2025-09-19
54
+ Benchmark 1: pre-commit run --config ../testing/test-cpp-linter-hooks.yaml --all-files
55
+ Time (mean ± σ): 84.1 ms ± 3.2 ms [User: 73.5 ms, System: 10.2 ms]
56
+ Range (min … max): 79.7 ms … 95.2 ms 20 runs
23
57
24
- Results saved to testing/benchmark_results.txt
58
+ Warning: Ignoring non-zero exit code.
59
+
60
+ Benchmark 2: pre-commit run --config ../testing/test-mirrors-clang-format.yaml --all-files
61
+ Time (mean ± σ): 85.0 ms ± 3.0 ms [User: 71.8 ms, System: 13.3 ms]
62
+ Range (min … max): 81.0 ms … 91.0 ms 20 runs
63
+
64
+ Warning: Ignoring non-zero exit code.
65
+
66
+ Summary
67
+ ' pre-commit run --config ../testing/test-cpp-linter-hooks.yaml --all-files' ran
68
+ 1.01 ± 0.05 times faster than ' pre-commit run --config ../testing/test-mirrors-clang-format.yaml --all-files'
25
69
```
70
+
71
+ > [ !NOTE]
72
+ > The results may vary based on the system and environment where the benchmarks are run.
0 commit comments