This repository provides tools and benchmark programs for the purpose of benchmarking MLKit executables across versions (and configurations) and against MLton-generated executables.
Here is an overview of the tools in this repository:
-
mlkit-bench
. This tool is what actually coordinates compilation and execution of benchmark programs. The output of the command is a json-file, which can then be processed bymlkit-bench-press
ormlkit-charting
. -
mlkit-bench-press
. This tool allows for inspection and querying of the json-file generated bymlkit-bench
. -
mlkit-charting
. This tool is useful for charting the MLKit performance on benchmarks over time. The tool makes use ofsmltojs
to create a web-site that allows for investing the performance characteristics of various versions of MLKit. The web-site is hosted at Github.
Notice: A useful setup is to have the MLKit sources checked-out at a sister-repository:
~/gits/mlkit-bench/
~/gits/mlkit/
Currently, there are two sets of benchmarks:
-
Sequential benchmarks. These benchmarks feature a variety of Standard ML programs, ranging over small micro-benchmarks to more involved benchmarks.
-
Parallel benchmarks. These benchmarks make use of the parallel threading possibilities of MLKit, which features a fork-join model for parallism combined with region-based memory management.