View on GitHub

MIVisionX-Inference-Tutorial

MIVisionX toolkit is a set of comprehensive computer vision and machine intelligence libraries, utilities, and applications bundled into a single toolkit.

MIVisionX Inference Tutorial

In this tutorial, we will learn how to run inference efficiently using OpenVX and OpenVX Extensions. The tutorial will go over each step required to convert a pre-trained neural net model into an OpenVX Graph and run this graph efficiently on any target hardware. In this tutorial, we will also learn about AMD MIVisionX which delivers open source implementation of OpenVX and OpenVX Extensions along with MIVisionX Neural Net Model Compiler & Optimizer.

Neural Net Model Compiler & Optimizer converts pre-trained neural network models to MIVisionX runtime code for optimized inference.

Pre-trained models in ONNX, NNEF, & Caffe formats are supported by the model compiler & optimizer. The model compiler first converts the pre-trained models to AMD Neural Net Intermediate Representation (NNIR), once the model has been translated into AMD NNIR (AMD’s internal open format), the Optimizer goes through the NNIR and applies various optimizations which would allow the model to be deployed on to target hardware most efficiently. Finally, AMD NNIR is converted into OpenVX C code, which could be compiled and deployed on any targeted hardware.

Prerequisites

Docker for tutorial

MIVisionX provides developers with docker images for Ubuntu 18.04, Ubuntu 20.04, CentOS 7, & CentOS 8. Using docker images developers can quickly prototype and build applications without having to be locked into a single system setup or lose valuable time figuring out the dependencies of the underlying software.

Docker with display option for the tutorial

Usage

Convert Pre-Trained Models into OpenVX

Use MIVisionX Neural Net Model Compiler & Optimizer to generate OpenVX code from your pre-trained neural net model. The model compiler generates annmodule.cpp & annmodule.h during the OpenVX code generation. The whole process of inference from a pre-trained neural net model will be shown in 3 different samples below.

  1. Download or train your own Caffe Model/ONNX Model/NNEF Model.

  1. Use MIVisionX Model Compiler to generate OpenVX C Code from the pre-trained models.

    Note: MIVisionX installs all the model compiler scripts in /opt/rocm/mivisionx/model_compiler/python/ folder

* Caffe Models

````
% python3 /opt/rocm/libexec/mivisionx/model_compiler/python/caffe_to_nnir.py <net.caffeModel> <nnirOutputFolder> --input-dims <n,c,h,w> [--verbose <0|1>]
````

* ONNX Models

````
% python3 /opt/rocm/libexec/mivisionx/model_compiler/python/onnx_to_nnir.py <onnxModel> <nnirOutputFolder> [--input_dims n,c,h,w (optional)]
````

* NNEF Models

````
% python3 /opt/rocm/libexec/mivisionx/model_compiler/python/nnef_to_nnir.py <nnefInputFolder> <outputFolder>
````


````
% python3 /opt/rocm/libexec/mivisionx/model_compiler/python/nnir_to_openvx.py <nnirModelFolder> <nnirModelOutputFolder>
````

Build - Inference Application

Classification | Detection | Segmentation :————————-:|:————————-:|:————————-: | |

Once the OpenVX code is generated(annmodule.cpp & annmodule.h), follow the instructions below to build the project.

Run

Classification Detection 
./classifier	--mode				<1/2/3 - 1:classification 2:detection 3:segmentation>	[required]
		--video/--capture/--image	<video file>/<0>/<image file>				[required]
		--model_weights			<model_weights.bin>					[required]
		--label				<label text>						[required]
		--model_input_dims		<c,h,w - channel,height,width>				[required]
		--model_output_dims		<c,h,w - channel,height,width>				[required]

		--model_name			<model name>					[optional - default:NN_ModelName]
		--add				<Ax,Ay,Az - input preprocessing factor>		[optional - default:0,0,0]
		--multiply			<Mx,My,Mz - input preprocessing factor>		[optional - default:1,1,1]


[usage help]	--help/--h

label < path to labels file >

Use Classification labels or Detection labels or Segmentation Labels files in the data folder depending on the type of model you are converting to OpenVX

video < path to video file >

Run inference on pre-recorded video with this option.

image < path to image file >

Run inference on an image with this option.

capture <0>

Run inference on the live camera feed with this option.

Note: –video/–capture/–image options are not supported concurrently

Supported Pre-Trained Model Formats

Sample 1 - Classification Using Pre-Trained ONNX Model

Run SqueezeNet on Video/Image

Sample 2 - Detection Using Pre-Trained Caffe Model

Run Tiny YoloV2 on an Image/Video

Sample 3 - Classification Using Pre-Trained NNEF Model

Run VGG 16 on a Video

Sample 4 - Classification Using Pre-Trained Caffe Model

Run VGG 16 on Live Video