Performance regression checks: workflow

[lcauser-oqc]

Performance regression checks when merging to main.

#229 introduces a number of benchmarks that can be used to measure the performance of QAT. In this PR, we add a workflow that compares the performance to main.

• Workflow that runs the performance benchmarks on main and then on the PR branch. Each benchmark will execute multiple times (this is flexible and we can configure this to fit our needs) and the best of each are compared. If the benchmark on the PR is slower by some predefined threshold (50% at this first pass), then the test will fail.
• The workflow also creates a report for each test that fails significantly (>50% slower) or gives a warning for less-significant regressions (>20%). This report is created using jinja2 and is given as both a workflow summary and a comment in the PR.
• The report will also return performance improvements (with a threshold of <0.9x the execution time of main).

Note: there are temporary tests to demonstrate the report (see the comment below) that will be removed before merging to main. This is why the workflow currently fails. Had to delete old bot comments - this was a problem specific to this branch being automatically rebased onto main before having a change to update a workflow.

[github-actions]

Performance Regression Tests

Performance changes detected in the following benchmarks: none

Pusher: @lcauser-oqc, Action: pull_request, Workflow: Performance regression tests

[bgsach]

Probably needs updating because you moved the benchmarks folder out of tests?

Apart from that, looks good.

[keriksson-rosenqvist]

LGTM. Great to have this on PR flows. 👏