""" _ _
_ __ _ _ ___ ___ _ __ ___ _ __ (_) | ___
| '_ \| | | |/ __/ _ \| '_ ` _ \| '_ \| | |/ _ \
| |_) | |_| | (_| (_) | | | | | | |_) | | | __/
| .__/ \__, |\___\___/|_| |_| |_| .__/|_|_|\___|
|_| |___/ |_|
"""
A CLI tool for compiling python source code using Cython or Nuitka.
pip install pycompile
Syntax | Description |
---|---|
--input-path PATH | by default it excludes any test and __init__.py files |
--clean-source | Deletes the sources files. |
--keep-builds | Keeps the temp build files. |
--clean-executables | Deletes the shared objects (.so ) files. |
--engine | Can be cython or nuitka . |
--exclude-glob-paths | Glob file patterns for excluding specific files. |
--verbose | Increase log messages. |
pycompile -i your_python_files --clean-source --engine nuitka
By default, the Cython is being used as the default compiler.
Tip
For compiling the examples
use the following command
pycompile -i input_path --engine cython
which by default, deletes any temp build files and keeps the source files.
pycompile -i input_path --engine nuitka
After the compilation the input
dir will have the following structure.
examples
├── fib.py.py
├── fib.cpython-310-darwin.so
├── test_fib.py
Syntax | Description |
---|---|
--input-path PATH | by default it excludes any test and __init__.py files |
--engine | Can be cython , nuitka , all or none . |
--type | Can be memory , cpy , or both |
--verbose | Increase log messages. |
--profile_func_pattern TEXT | function name pattern for profiling defaults to benchmark |
For running a benchmark on the input-path
use the following command:
pycompile benchmark -i src/examples -vvv
which by default will start a memory
and a cpu
benchmark, starting with
python
and then with cython
and nuitka
Important
The python package must have a test_module.py
because both benchmark types are invoked
with pytest
runs
Note
For memory profiling the script will decorate all the functions in benchmark.py
with the profile
decorator from memory-profiler
. This is not optimal memory profiling,
because we don't actually profile
the function itself, instead we profile the caller
but it's necessary
if we want to profile
also the compiled code.
Use the profile_func_pattern
to specify the function to be profiled in different module for example
if main
is the entrypoint under main.py
use --profile_func_pattern main
.
Hence, the following structure are required for the benchmark
subcommand.
module
├── sample_funcs.py # implementation
├── main.py # entrypoint with a `main` function, during compilation will be excluded
├── test_sample_funcs.py # test cases
Memory benchmark using:3.10.9 (main, Feb 2 2023, 12:59:36) [Clang 14.0.0 (clang-1400.0.29.202)
Line # Mem usage Increment Occurrences Line Contents
=============================================================
7 49.4 MiB 49.4 MiB 1 @profile
8 def samples_benchmark():
9 127.7 MiB 78.4 MiB 1 sum_of_squares()
10 166.0 MiB 38.3 MiB 1 harmonic_mean()
11 166.0 MiB 0.0 MiB 1 fibonacci(30)
12 204.2 MiB 38.2 MiB 1 sum_numbers()
13 57.7 MiB -146.5 MiB 1 sum_strings()
46.03s call test_examples.py::test_examples
Note
For cpu profiling the same approached is being used, but instead of decorating the calling functions
it decorates
the test cases with the benchmark
from pytest-benchmark
.
CPU benchmark using:3.10.9 (main, Feb 2 2023, 12:59:36) [Clang 14.0.0 (clang-1400.0.29.202)]
------------------------------------------- benchmark: 1 tests ------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------
test_examples 3.9257 4.0640 3.9731 0.0605 3.9387 0.0917 1;0 0.2517 5 1
---------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
=================================================================================================================
29.40s call test_examples.py::test_examples
Syntax | Description |
---|---|
--input-path PATH | by default it excludes any test and __init__.py files |
--exclude-glob-paths | Glob file patterns for excluding specific files. |
--verbose | Increase log messages. |
pycompile dry_run -i ./src
For local development run the following command
make setup-local-dev
All available make
commands
make help