onnx2code

Generate plain C++ code for inference of ONNX models without dependencies

This project was made as an alternative to a final exam for the assignment "Computer Organization II". You can read the writeup in docs/TP Final onnx2code.pdf (in Spanish).

Model support

The following models have been tested and work as expected.

Model	Size
mnist	26 KB
Super_Resolution	240 KB
squeezenet1.1	9 MB
emotion_ferplus	34 MB
inception-v2	44 MB
resnet50-caffe2-v1	98 MB
VGG 16 and VGG 16-bn	527 MB
VGG 19 and VGG 19-bn	548 MB
VGG 19-caffe2	561 MB

• Minimum ONNX opset version: 7
• Quantized models are not supported

Operator support

Only float data type is supported.

Operator	Attribute support
Add, Div, Mul, Sub	✅ with broadcasting
Concat	✅ with multiple inputs ✅ axis
Conv	✅ bias ✅ stride ✅ padding (and auto_pad) ❌ dilations ❌ depthwise (group != 1)
Sum	✅ with multiple inputs ❌ with broadcasting
Relu, Tanh, Sigmoid, Clip	✅
Gemm	✅ with bias ❌ transpose A ✅ tranpose B ❌ alpha != 1 ❌ beta != 1
Identity	✅
MaxPool, AveragePool	✅ stride ✅ padding (and auto_pad) ❌ dilations ❌ storage_order != 0 ❌ count_include_pad != 0
Softmax	✅ stride ✅ axis
Transpose	✅ perm

Setting up with Docker

We provide a ready to use Docker image:

docker run --rm -it -v $pwd/mnist.onnx:/app/input.onnx:ro -v $pwd/output:/app/output:rw mlomb/onnx2code:latest --variations=im2col,loop-tiling --checks=3

The command above will generate C++ code for the mnist.onnx model in the output folder.

Setting up locally

Prerequisites

• gcc (required if checking models)
• Python 3.10
• pipenv

Clone and install dependencies with pipenv install.

Run

To generate code from an ONNX model, run the following command inside a pipenv shell:

python -m onnx2code --variation=im2col,loop-tiling mnist.onnx output_folder --checks=3