Skip to content

Conversation

@NicolaasWeideman
Copy link
Collaborator

@NicolaasWeideman NicolaasWeideman commented Sep 23, 2025

  • I have reviewed the OFRAK contributor guide and attest that this pull request is in accordance with it.
  • I have made or updated a changelog entry for the changes in this pull request.

One sentence summary of this PR (This should go in the CHANGELOG!)
Adds benchmarking to the Github Actions workflow.

Link to Related Issue(s)
N/A

Please describe the changes in your request.

  • Adds benchmark test cases to the OFRAK components
  • Updates the Makefiles with the option to execute benchmark test cases.
  • Updates the Github Action workflow files to perform benchmarking and compare the results before and after a pull request.

Anyone you think should look at this, specifically?
@whyitfor

Comment on lines +50 to +51
"pytest-benchmark==3.2.3",
"py",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are test dependencies, so they should go in requirements-dev.txt.

Copy link
Member

@rbs-jacob rbs-jacob left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some tiny changes in addition to Wyatt's suggestion. Otherwise, this is a great improvement to the pipeline! Hopefully there are no performance regressions, but at least we'll be able to see them if they're there.

Comment on lines +103 to +110
--volume /ofrak_pyghidra_benchmarks:/ofrak_pyghidra/.benchmarks \
--entrypoint bash \
redballoonsecurity/ofrak/ghidra:latest \
-c "python -m ofrak_ghidra.server start \
&& ofrak license --community --i-agree \
&& make test"
&& make test \
&& make -C /ofrak_pyghidra benchmark"
cat /ofrak_pyghidra_benchmarks/benchmark.json > $GITHUB_STEP_SUMMARY
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
--volume /ofrak_pyghidra_benchmarks:/ofrak_pyghidra/.benchmarks \
--entrypoint bash \
redballoonsecurity/ofrak/ghidra:latest \
-c "python -m ofrak_ghidra.server start \
&& ofrak license --community --i-agree \
&& make test"
&& make test \
&& make -C /ofrak_pyghidra benchmark"
cat /ofrak_pyghidra_benchmarks/benchmark.json > $GITHUB_STEP_SUMMARY
--volume "$(pwd)/benchmarks":/ofrak_pyghidra/.benchmarks \
--entrypoint bash \
redballoonsecurity/ofrak/ghidra:latest \
-c "python -m ofrak_ghidra.server start \
&& ofrak license --community --i-agree \
&& make test \
&& make -C /ofrak_pyghidra benchmark"
cat benchmarks/benchmark.json >> $GITHUB_STEP_SUMMARY

GitHub won't allow me to make a proper suggestion on this code snippet, but perhaps something like the above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants