Skip to content

Commit 3aafa8a

Browse files
authored
chore: update benchmark (#122)
* chore: update benchmark * Update README for clarity on pre-commit hook * Install hyperfine before running benchmarks * chore: update benchmark * fix: revert example code * fix: revert example code * chore: pre-commit autoupdate
1 parent a076581 commit 3aafa8a

10 files changed

+69
-199
lines changed

.github/workflows/benchmark.yml

Lines changed: 0 additions & 25 deletions
This file was deleted.

.pre-commit-config.yaml

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,8 +10,9 @@ repos:
1010
- id: check-toml
1111
- id: requirements-txt-fixer
1212
- repo: https://github.com/astral-sh/ruff-pre-commit
13-
rev: v0.12.11
13+
rev: v0.13.1
1414
hooks:
15-
- id: ruff
16-
args: [--fix]
15+
# Run the linter.
16+
- id: ruff-check
17+
# Run the formatter.
1718
- id: ruff-format

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66
[![Test](https://github.com/cpp-linter/cpp-linter-hooks/actions/workflows/test.yml/badge.svg)](https://github.com/cpp-linter/cpp-linter-hooks/actions/workflows/test.yml)
77
[![CodeQL](https://github.com/cpp-linter/cpp-linter-hooks/actions/workflows/codeql.yml/badge.svg)](https://github.com/cpp-linter/cpp-linter-hooks/actions/workflows/codeql.yml)
88

9-
A powerful [pre-commit](https://pre-commit.com/) hook for auto-formatting and linting C/C++ code with `clang-format` and `clang-tidy`.
9+
A pre-commit hook that automatically formats and lints your C/C++ code using `clang-format` and `clang-tidy`.
1010

1111
## Table of Contents
1212

testing/README.md

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,4 @@ pre-commit try-repo ./.. clang-tidy --verbose --all-files
99

1010
## Benchmark
1111

12-
```bash
13-
python3 testing/benchmark_hooks.py
14-
```
12+
See [benchmark](./benchmark.md).

testing/benchmark.md

Lines changed: 57 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -4,22 +4,69 @@ This document outlines the benchmarking process for comparing the performance of
44

55
> About tests performance can be found at: [![CodSpeed Badge](https://img.shields.io/endpoint?url=https://codspeed.io/badge.json)](https://codspeed.io/cpp-linter/cpp-linter-hooks)
66
7-
## Running the Benchmark
7+
## Run benchmark separately
88

99
```bash
10-
python3 testing/benchmark_hooks.py
10+
rm -rf examples || true
11+
git clone --depth 1 --quiet https://github.com/gouravthakur39/beginners-C-program-examples.git examples
12+
13+
pre-commit clean
14+
hyperfine --warmup 1 -r 10 'pre-commit run --config testing/test-cpp-linter-hooks.yaml --all-files'
15+
16+
rm -rf examples || true
17+
git clone --depth 1 --quiet https://github.com/gouravthakur39/beginners-C-program-examples.git examples
18+
pre-commit clean
19+
20+
hyperfine --warmup 1 -r 10 'pre-commit run --config testing/test-mirrors-clang-format.yaml --all-files'
21+
```
22+
23+
Results:
24+
25+
```bash
26+
# Updated on 2025-09-19
27+
28+
Cleaned /home/sxp/.cache/pre-commit.
29+
Benchmark 1: pre-commit run --config testing/test-cpp-linter-hooks.yaml --all-files
30+
Time (mean ± σ): 150.2 ms ± 1.8 ms [User: 121.7 ms, System: 29.2 ms]
31+
Range (min … max): 148.3 ms … 153.9 ms 10 runs
32+
33+
Cleaned /home/sxp/.cache/pre-commit.
34+
Benchmark 1: pre-commit run --config testing/test-mirrors-clang-format.yaml --all-files
35+
Time (mean ± σ): 122.6 ms ± 1.9 ms [User: 98.0 ms, System: 24.7 ms]
36+
Range (min … max): 120.3 ms … 125.5 ms 10 runs
1137
```
1238

13-
## Results
39+
### Run benchmark comparison
40+
41+
Compare the results of both commands.
1442

1543
```bash
16-
# Updated on 2025-09-02
17-
Benchmark Results:
44+
rm -rf examples || true
45+
git clone --depth 1 --quiet https://github.com/gouravthakur39/beginners-C-program-examples.git examples
46+
47+
hyperfine -i --warmup 1 -r 20 'pre-commit run --config ../testing/test-cpp-linter-hooks.yaml --all-files' 'pre-commit run --config ../testing/test-mirrors-clang-format.yaml --all-files'
48+
```
1849

19-
Hook | Avg (s) | Std (s) | Min (s) | Max (s) | Runs
20-
---------------------+------------------+------------------+------------------+------------------+-----------------
21-
cpp-linter-hooks | 12.473 | 1.738 | 11.334 | 15.514 | 5
22-
mirrors-clang-format | 4.960 | 0.229 | 4.645 | 5.284 | 5
50+
Results:
51+
52+
```bash
53+
# Updated on 2025-09-19
54+
Benchmark 1: pre-commit run --config ../testing/test-cpp-linter-hooks.yaml --all-files
55+
Time (mean ± σ): 84.1 ms ± 3.2 ms [User: 73.5 ms, System: 10.2 ms]
56+
Range (min … max): 79.7 ms … 95.2 ms 20 runs
2357

24-
Results saved to testing/benchmark_results.txt
58+
Warning: Ignoring non-zero exit code.
59+
60+
Benchmark 2: pre-commit run --config ../testing/test-mirrors-clang-format.yaml --all-files
61+
Time (mean ± σ): 85.0 ms ± 3.0 ms [User: 71.8 ms, System: 13.3 ms]
62+
Range (min … max): 81.0 ms … 91.0 ms 20 runs
63+
64+
Warning: Ignoring non-zero exit code.
65+
66+
Summary
67+
'pre-commit run --config ../testing/test-cpp-linter-hooks.yaml --all-files' ran
68+
1.01 ± 0.05 times faster than 'pre-commit run --config ../testing/test-mirrors-clang-format.yaml --all-files'
2569
```
70+
71+
> [!NOTE]
72+
> The results may vary based on the system and environment where the benchmarks are run.

testing/benchmark_hook_1.yaml

Lines changed: 0 additions & 6 deletions
This file was deleted.

testing/benchmark_hooks.py

Lines changed: 0 additions & 145 deletions
This file was deleted.

testing/benchmark_results.txt

Lines changed: 0 additions & 6 deletions
This file was deleted.

testing/test-cpp-linter-hooks.yaml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
repos:
2+
- repo: https://github.com/cpp-linter/cpp-linter-hooks
3+
rev: v1.1.3
4+
hooks:
5+
- id: clang-format # v 21.1.0
6+
args: [--style=file]
File renamed without changes.

0 commit comments

Comments
 (0)