Face Recognition: ONNX to TensorRT conversion for Arcface model problem?

Are you also fascinated to get an inference from a face recognition model on jetson nano?

I fail to run TensorRT inference on Jetson Nano, due to PReLU activation function not supported for TensorRT 5.1. But, the PReLU channel-wise operator is available for TensorRT 6. In this blogpost, I will explain the steps required in the model conversion of ONNX to TensorRT and the reason why my steps failed to run TensorRT inference on Jetson Nano. 

Steps included to run TensorRT inference on Jetson Nano :

1. The first step is to import the model, which includes loading it from a saved file on disk and converting it to a TensorRT network from its native framework or format.

Our example loads the model in ONNX format i.e. arcface model of face recognition.

2. Next, an optimized TensorRT engine is built based on the input model, target GPU platform, and other configuration parameters specified.
3. The last step is to provide input data to the TensorRT engine to perform inference. The sample uses input data bundled with model from the ONNX model zoo to perform inference.

Sample code:

Now let’s convert the downloaded ONNX model into TensorRT arcface_trt.engine.

TensorRT module is pre-installed on Jetson Nano. The current release of TensorRT version is 5.1 by NVIDIA JetPack SDK.

1. Firstly, ensure that ONNX is installed on Jetson Nano by running the following command.

import ONNX

If this command gives an error, then ONNX is not installed on Jetson Nano.

Follow the steps to install ONNX on Jetson Nano:


sudo apt-get install cmake==3.2
sudo apt-get install protobuf-compiler
sudo apt-get install libprotoc-dev
pip install –no-binary ONNX ‘ONNX==1.5.0’


Now, ONNX is ready to run on Jetson Nano satisfying all the dependencies.

2. Now, download the ONNX model using the following command:

wget https://s3.amazonaws.com/ONNX-model-zoo/arcface/resnet100/resnet100.ONNX

3. Simply run the following script as a next step:

We are using Python API for the conversion.
import os
import tensorrt as trtbatch_size = 1
TRT_LOGGER = trt.Logger()
def build_engine_ONNX(model_file):
with trt.Builder(TRT_LOGGER) as builder, builder.create_network() as network, trt.ONNXParser(network, TRT_LOGGER) as parser:
builder.max_workspace_size = 1 << 30
builder.max_batch_size = batch_size


# Load the ONNX model and parse it in order to populate the TensorRT network.

with open(model_file, 'rb') as model:
parser.parse(model.read())
return builder.build_cuda_engine(network)


# downloaded the arcface mdoel


ONNX_file_path = './resnet100.ONNX'
engine = build_engine_ONNX(ONNX_file_path)
engine_file_path = './arcface_trt.engine'
with open(engine_file_path, "wb") as f:
f.write(engine.serialize())

After running the script, we get some error “Segmentation fault core dumped”. After doing a lot of research we have found that there is no issue with the script. There are some other reasons why we are facing this problem. The reasons and explanations are discussed in the following paragraphs.

What are the reasons for which model conversion failed?

Jetson Nano is a ARM architecture-based device where TensorRT 5.1 is already pre-installed. The image which is written on SD card of NVIDIA Jetpack SDK does not includes TensorRT 6. It is possible to convert other models to TensorRT and run inference on top of it but it’s not possible with arcface. The arcface model cannot be converted because it contains a PRELU activation function which only supports TensorRT 6.

Model cannot be converted because we are unable to upgrade the TensorRT version from 5.1 to 6. So, unless and until NVIDIA provides us a Jetpack SDK OS image with the latest version of TensorRT 6 specifically the arcface model cannot be converted.

Why can’t we upgrade from TensorRT 5.1 to TensorRT 6?

The installation file of TensorRT 6 is only supportable for AMD64 architecture which can’t be run on Jetson Nano because it is an ARM-architecture device. That’s why, the arcface ONNX model conversion is failed.

Future Work and Conclusion

As soon as, NVIDIA Jetpack SDK releases OS image with TensorRT 6 the arcface ONNX model will get converted to TensorRT and we can run inference on top of it. I am all ears to know your thoughts/ideas to make it happen if NVDIA is taking its time to update jetpack SDK. We at DataToBiz always strive for latest tools & technologies to get ahead from our competitors. Contact for further details

About Author: Sushavan is a student of B.Tech in Computer Engg. at Lovely Professional University. He worked as an intern at DataToBiz for 6 months.

Vaishali