MIVisionX Model Compiler Samples

In this sample, we will learn how to run inference efficiently using OpenVX and OpenVX Extensions. The sample will go over each step required to convert a pre-trained neural net model into an OpenVX Graph and run this graph efficiently on any target hardware. In this sample, we will also learn about AMD MIVisionX which delivers open source implementation of OpenVX and OpenVX Extensions along with MIVisionX Neural Net Model Compiler & Optimizer.

MIVisionX Model Compiler & Optimizer
Prerequisites
Usage
Supported Pre-Trained Model Formats
Sample-1: Classification Using Pre-Trained ONNX Model
Sample-2: Detection Using Pre-Trained Caffe Model
Sample-3: Classification Using Pre-Trained NNEF Model
Sample-4: Classification Using Pre-Trained Caffe Model

Neural Net Model Compiler & Optimizer converts pre-trained neural network models to MIVisionX runtime code for optimized inference.

Pre-trained models in ONNX, NNEF, & Caffe formats are supported by the model compiler & optimizer. The model compiler first converts the pre-trained models to AMD Neural Net Intermediate Representation (NNIR), once the model has been translated into AMD NNIR (AMD's internal open format), the Optimizer goes through the NNIR and applies various optimizations which would allow the model to be deployed on to target hardware most efficiently. Finally, AMD NNIR is converted into OpenVX C code, which could be compiled and deployed on any targeted hardware.

Prerequisites

Ubuntu 20.04/22.04 or CentOS 7/8
ROCm supported hardware
- AMD Radeon GPU or AMD APU required
Latest ROCm
Build & Install MIVisionX

Docker for Samples

MIVisionX provides developers with docker images for Ubuntu 20.04 / 22.04. Using docker images developers can quickly prototype and build applications without having to be locked into a single system setup or lose valuable time figuring out the dependencies of the underlying software.

Docker files to build MIVisionX containers are available

MIVisionX Docker

Ubuntu 20.04
Ubuntu 22.04

Docker Workflow on Ubuntu `20.04`/`22.04`

Prerequisites

Ubuntu 20.04/22.04
ROCm supported hardware
ROCm
Docker

Docker with display option for the samples

Start docker with display

% sudo docker pull mivisionx/ubuntu-20.04:latest
% xhost +local:root
% sudo docker run -it --device=/dev/kfd --device=/dev/dri --cap-add=SYS_RAWIO --device=/dev/mem --group-add video --network host --env DISPLAY=unix$DISPLAY --privileged --volume $XAUTH:/root/.Xauthority --volume /tmp/.X11-unix/:/tmp/.X11-unix mivisionx/ubuntu-20.04:latest

Test display with MIVisionX sample

% export PATH=$PATH:/opt/rocm/bin
% export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/rocm/lib
% runvx /opt/rocm/share/mivisionx/samples/gdf/canny.gdf

Usage

Convert Pre-Trained Models into OpenVX

Use MIVisionX Neural Net Model Compiler & Optimizer to generate OpenVX code from your pre-trained neural net model. The model compiler generates annmodule.cpp & annmodule.h during the OpenVX code generation. The whole process of inference from a pre-trained neural net model will be shown in 3 different samples below.

Download or train your own Caffe Model/ONNX Model/NNEF Model.

Use MIVisionX Model Compiler to generate OpenVX C Code from the pre-trained models.

Note: MIVisionX installs all the model compiler scripts in /opt/rocm/libexec/mivisionx/model_compiler/python/ folder

Convert the pre-trained models into AMD NNIR model:

* Caffe Models

````
% python3 /opt/rocm/libexec/mivisionx/model_compiler/python/caffe_to_nnir.py <net.caffeModel> <nnirOutputFolder> --input-dims <n,c,h,w> [--verbose <0|1>]
````

* ONNX Models

````
% python3 /opt/rocm/libexec/mivisionx/model_compiler/python/onnx_to_nnir.py <onnxModel> <nnirOutputFolder> [--input_dims n,c,h,w (optional)]
````

* NNEF Models

````
% python3 /opt/rocm/libexec/mivisionx/model_compiler/python/nnef_to_nnir.py <nnefInputFolder> <outputFolder>
````

Convert an AMD NNIR model into OpenVX C code:

````
% python3 /opt/rocm/libexec/mivisionx/model_compiler/python/nnir_to_openvx.py <nnirModelFolder> <nnirModelOutputFolder>
````

Build - Inference Application

Classification	Detection	Segmentation

Once the OpenVX code is generated(annmodule.cpp & annmodule.h), follow the instructions below to build the project.

Copy the files (annmodule.cpp & annmodule.h) generated by the model compiler into this samples module_files folder.

% cp PATH/To/annmodule.h   PATH/To/MIVisionX/samples/model_compiler_samples/module_files/
% cp PATH/To/annmodule.cpp PATH/To/MIVisionX/samples/model_compiler_samples/module_files/

Build this project

% mkdir build
% cd build
% cmake PATH/To/MIVisionX/samples/model_compiler_samples/
% make

Run

Classification	Detection

./classifier	--mode				<1/2/3 - 1:classification 2:detection 3:segmentation>	[required]
		--video/--capture/--image	<video file>/<0>/<image file>				[required]
		--model_weights			<model_weights.bin>					[required]
		--label				<label text>						[required]
		--model_input_dims		<c,h,w - channel,height,width>				[required]
		--model_output_dims		<c,h,w - channel,height,width>				[required]

		--model_name			<model name>					[optional - default:NN_ModelName]
		--add				<Ax,Ay,Az - input preprocessing factor>		[optional - default:0,0,0]
		--multiply			<Mx,My,Mz - input preprocessing factor>		[optional - default:1,1,1]


[usage help]	--help/--h

label < path to labels file >

Use Classification labels or Detection labels or Segmentation Labels files in the data folder depending on the type of model you are converting to OpenVX

video < path to video file >

Run inference on pre-recorded video with this option.

image < path to image file >

Run inference on an image with this option.

capture <0>

Run inference on the live camera feed with this option.

Note: --video/--capture/--image options are not supported concurrently

Supported Pre-Trained Model Formats

Caffe
NNEF
ONNX

Sample 1 - Classification Using Pre-Trained ONNX Model

Run SqueezeNet on Video/Image

Step 1: Clone MIVisionX Inference Tutorial Project
```
 % cd && mkdir sample-1 && cd sample-1
 % git clone https://github.com/kiritigowda/MIVisionX-Inference-Tutorial.git
```
Note:
- MIVisionX needs to be pre-installed
- MIVisionX Model Compiler & Optimizer scripts are at /opt/rocm/libexec/mivisionx/model_compiler/python/
- ONNX model conversion requires ONNX install using pip install onnx
Step 2: Download pre-trained SqueezeNet ONNX model from ONNX Model Zoo - SqueezeNet Model
```
 % wget https://s3.amazonaws.com/download.onnx/models/opset_8/squeezenet.tar.gz
 % tar -xvf squeezenet.tar.gz
```
Note: pre-trained model - squeezenet/model.onnx
Step 3: Use MIVisionX Model Compiler to generate OpenVX files from the pre-trained ONNX model
- Convert .onnx to NNIR
```
 % python3 /opt/rocm/libexec/mivisionx/model_compiler/python/onnx_to_nnir.py squeezenet/model.onnx squeezenet-nnir
```
- Convert NNIR to OpenVX
```
 % python3 /opt/rocm/libexec/mivisionx/model_compiler/python/nnir_to_openvx.py squeezenet-nnir/ squeezenet-openvx
```
Note:
- annmodule.cpp & annmodule.h generated in squeezenet-openvx folder
- weights.bin generated in squeezenet-openvx folder is used for the classifier --model_weights option

Step 4: Copy the annmodule.cpp & annmodule.h files into module_files folder. CMake and build this project

Copy OpenVX generated code

 % cp ~/sample-1/squeezenet-openvx/annmodule.h ~/sample-1/MIVisionX-Inference-Tutorial/module_files/
 % cp ~/sample-1/squeezenet-openvx/annmodule.cpp ~/sample-1/MIVisionX-Inference-Tutorial/module_files/

CMake and build

 % mkdir ~/sample-1/build
 % cd ~/sample-1/build/
 % cmake ../MIVisionX-Inference-Tutorial/
 % make

Step 5: Use the command below to run the classifier

View classifier usage

 % ./classifier --help

Run SqueezeNet Classifier

 % ./classifier --mode 1 --video ../MIVisionX-Inference-Tutorial/data/images/img_05.JPG --model_weights ../squeezenet-openvx/weights.bin --label ../MIVisionX-Inference-Tutorial/data/sample_classification_labels.txt --model_input_dims 3,224,224 --model_output_dims 1000,1,1 --model_name SqueezeNet_ONNX

Sample 2 - Detection Using Pre-Trained Caffe Model

Run Tiny YoloV2 on an Image/Video

Step 1: Clone MIVisionX Inference Tutorial Project
```
 % cd && mkdir sample-2 && cd sample-2
 % git clone https://github.com/kiritigowda/MIVisionX-Inference-Tutorial.git
```
Note:
- MIVisionX needs to be pre-installed
- MIVisionX Model Compiler & Optimizer scripts are at /opt/rocm/libexec/mivisionx/model_compiler/python/

Step 2: Download pre-trained Tiny YoloV2 caffe model - yoloV2Tiny20.caffemodel

 % wget https://github.com/kiritigowda/YoloV2NCS/raw/master/models/caffemodels/yoloV2Tiny20.caffemodel

Step 3: Use MIVisionX Model Compiler to generate OpenVX files from the pre-trained caffe model
- Convert .caffemodel to NNIR
```
 % python3 /opt/rocm/libexec/mivisionx/model_compiler/python/caffe_to_nnir.py yoloV2Tiny20.caffemodel yoloV2-nnir --input-dims 1,3,416,416
```
- Convert NNIR to OpenVX
```
 % python3 /opt/rocm/libexec/mivisionx/model_compiler/python/nnir_to_openvx.py yoloV2-nnir yoloV2-openvx
```
Note:
- annmodule.cpp & annmodule.h generated in yoloV2-openvx folder
- weights.bin generated in yoloV2-openvx folder is used for the classifier --model_weights option

Step 4: Copy the annmodule.cpp & annmodule.h files into module_files folder. CMake and build this project

Copy OpenVX generated code

 % cp ~/sample-2/yoloV2-openvx/annmodule.h ~/sample-2/MIVisionX-Inference-Tutorial/module_files/
 % cp ~/sample-2/yoloV2-openvx/annmodule.cpp ~/sample-2/MIVisionX-Inference-Tutorial/module_files/

CMake and build

 % mkdir ~/sample-2/build
 % cd ~/sample-2/build/
 % cmake ../MIVisionX-Inference-Tutorial/
 % make

Step 5: Use the command below to run the classifier

View classifier usage

 % ./classifier --help

Run YoloV2 Classifier

 % ./classifier --mode 2 --video ../../data/videos/AMD_driving_virtual_20.mp4 --model_weights ../yoloV2-openvx/weights.bin --label ../MIVisionX-Inference-Tutorial/data/sample_detection_labels.txt --model_input_dims 3,416,416 --model_output_dims 125,12,12 --model_name YoloV2_Caffe --multiply 0.003922,0.003922,0.003922

Note:

Tiny YoloV2 input needs to be preprocessed
Use the --multiply option to preprocess the input by a factor 1/255

Sample 3 - Classification Using Pre-Trained NNEF Model

Run VGG 16 on a Video

Step 1: Clone MIVisionX Inference Tutorial Project
```
 % cd && mkdir sample-3 && cd sample-3
 % git clone https://github.com/kiritigowda/MIVisionX-Inference-Tutorial.git
```
Note:
- MIVisionX needs to be pre-installed
- MIVisionX Model Compiler & Optimizer scripts are at /opt/rocm/libexec/mivisionx/model_compiler/python/
- NNEF model conversion requires NNEF python parser installed

Step 2: Download pre-trained VGG 16 NNEF model

 % mkdir ~/sample-3/vgg16
 % cd ~/sample-3/vgg16
 % wget https://sfo2.digitaloceanspaces.com/nnef-public/vgg16.onnx.nnef.tgz
 % tar -xvf vgg16.onnx.nnef.tgz
 % cd ~/sample-3/

Step 3: Use MIVisionX Model Compiler to generate OpenVX files from the pre-trained caffe model
- Convert .nnef to NNIR
```
 % python3 /opt/rocm/libexec/mivisionx/model_compiler/python/nnef_to_nnir.py vgg16/ vgg16-nnir
```
- Convert NNIR to OpenVX
```
 % python3 /opt/rocm/libexec/mivisionx/model_compiler/python/nnir_to_openvx.py vgg16-nnir/ vgg16-openvx
```
Note:
- annmodule.cpp & annmodule.h generated in vgg16-openvx folder
- weights.bin generated in vgg16-openvx folder is used for the classifier --model_weights option

Step 4: Copy the annmodule.cpp & annmodule.h files into module_files folder. CMake and build this project

Copy OpenVX generated code

 % cp ~/sample-3/vgg16-openvx/annmodule.h ~/sample-3/MIVisionX-Inference-Tutorial/module_files/
 % cp ~/sample-3/vgg16-openvx/annmodule.cpp ~/sample-3/MIVisionX-Inference-Tutorial/module_files/

CMake and build

 % mkdir ~/sample-3/build
 % cd ~/sample-3/build/
 % cmake ../MIVisionX-Inference-Tutorial/
 % make

Step 5: Use the command below to run the classifier

View classifier usage

 % ./classifier --help

Run VGG-16 Classifier

 % ./classifier --mode 1 --video ../MIVisionX-Inference-Tutorial/data/images/img_01.JPG --model_weights ../vgg16-openvx/weights.bin --label ../MIVisionX-Inference-Tutorial/data/sample_classification_labels.txt --model_input_dims 3,224,224 --model_output_dims 1000,1,1 --model_name VGG16_NNEF

Sample 4 - Classification Using Pre-Trained Caffe Model

Run VGG 16 on Live Video

Step 1: Clone MIVisionX Inference Tutorial Project
```
 % cd && mkdir sample-4 && cd sample-4
 % git clone https://github.com/kiritigowda/MIVisionX-Inference-Tutorial.git
```
Note:
- MIVisionX needs to be pre-installed
- MIVisionX Model Compiler & Optimizer scripts are at /opt/rocm/libexec/mivisionx/model_compiler/python/

Step 2: Download pre-trained VGG 16 caffe model - VGG_ILSVRC_16_layers.caffemodel

 % wget http://www.robots.ox.ac.uk/~vgg/software/very_deep/caffe/VGG_ILSVRC_16_layers.caffemodel

Step 3: Use MIVisionX Model Compiler to generate OpenVX files from the pre-trained caffe model
- Convert .caffemodel to NNIR
```
 % python3 /opt/rocm/libexec/mivisionx/model_compiler/python/caffe_to_nnir.py VGG_ILSVRC_16_layers.caffemodel vgg16-nnir --input-dims 1,3,224,224
```
- Convert NNIR to OpenVX
```
 % python3 /opt/rocm/libexec/mivisionx/model_compiler/python/nnir_to_openvx.py vgg16-nnir vgg16-openvx
```
Note:
- annmodule.cpp & annmodule.h generated in vgg16-openvx folder
- weights.bin generated in vgg16-openvx folder is used for the classifier --model_weights option

Step 4: Copy the annmodule.cpp & annmodule.h files into module_files folder. CMake and build this project

Copy OpenVX generated code

 % cp ~/sample-4/vgg16-openvx/annmodule.h ~/sample-4/MIVisionX-Inference-Tutorial/module_files/
 % cp ~/sample-4/vgg16-openvx/annmodule.cpp ~/sample-4/MIVisionX-Inference-Tutorial/module_files/

CMake and build

 % mkdir ~/sample-4/build
 % cd ~/sample-4/build/
 % cmake ../MIVisionX-Inference-Tutorial/
 % make

Step 5: Use the command below to run the classifier

View classifier usage

 % ./classifier --help

Run VGG-16 Classifier

 % ./classifier --mode 1 --capture 0 --model_weights ../vgg16-openvx/weights.bin --label ../MIVisionX-Inference-Tutorial/data/sample_classification_labels.txt --model_input_dims 3,224,224 --model_output_dims 1000,1,1 --model_name VGG16_Caffe

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

README.md

README.md

MIVisionX Model Compiler Samples

Prerequisites

Docker for Samples

MIVisionX Docker

Docker Workflow on Ubuntu `20.04`/`22.04`

Prerequisites

Docker with display option for the samples

Usage

Convert Pre-Trained Models into OpenVX

Build - Inference Application

Run

label < path to labels file >

video < path to video file >

image < path to image file >

capture <0>

Supported Pre-Trained Model Formats

Sample 1 - Classification Using Pre-Trained ONNX Model

Run SqueezeNet on Video/Image

Sample 2 - Detection Using Pre-Trained Caffe Model

Run Tiny YoloV2 on an Image/Video

Sample 3 - Classification Using Pre-Trained NNEF Model

Run VGG 16 on a Video

Sample 4 - Classification Using Pre-Trained Caffe Model

Run VGG 16 on Live Video

Files

README.md

Latest commit

History

README.md

File metadata and controls

MIVisionX Model Compiler Samples

Prerequisites

Docker for Samples

MIVisionX Docker

Docker Workflow on Ubuntu 20.04/22.04

Prerequisites

Docker with display option for the samples

Usage

Convert Pre-Trained Models into OpenVX

Build - Inference Application

Run

label < path to labels file >

video < path to video file >

image < path to image file >

capture <0>

Supported Pre-Trained Model Formats

Sample 1 - Classification Using Pre-Trained ONNX Model

Run SqueezeNet on Video/Image

Sample 2 - Detection Using Pre-Trained Caffe Model

Run Tiny YoloV2 on an Image/Video

Sample 3 - Classification Using Pre-Trained NNEF Model

Run VGG 16 on a Video

Sample 4 - Classification Using Pre-Trained Caffe Model

Run VGG 16 on Live Video

Docker Workflow on Ubuntu `20.04`/`22.04`