Are you also fascinated to get an inference from a face recognition model on jetson nano?
I fail to run TensorRT inference on Jetson Nano, due to PReLU activation function not supported for TensorRT 5.1. But, the PReLU channel-wise operator is available for TensorRT 6. In this blogpost, I will explain the steps required in the model conversion of ONNX to TensorRT and the reason why my steps failed to run TensorRT inference on Jetson Nano.
Our example loads the model in ONNX format i.e. arcface model of face recognition.
Now let’s convert the downloaded ONNX model into TensorRT arcface_trt.engine.
TensorRT module is pre-installed on Jetson Nano. The current release of TensorRT version is 5.1 by NVIDIA JetPack SDK.
If this command gives an error, then ONNX is not installed on Jetson Nano.
Follow the steps to install ONNX on Jetson Nano:
sudo apt-get install cmake==3.2
sudo apt-get install protobuf-compiler
sudo apt-get install libprotoc-dev
pip install –no-binary ONNX ‘ONNX==1.5.0’
Now, ONNX is ready to run on Jetson Nano satisfying all the dependencies.
We are using Python API for the conversion.
import tensorrt as trtbatch_size = 1
TRT_LOGGER = trt.Logger()
with trt.Builder(TRT_LOGGER) as builder, builder.create_network() as network, trt.ONNXParser(network, TRT_LOGGER) as parser:
builder.max_workspace_size = 1 << 30
builder.max_batch_size = batch_size
# Load the ONNX model and parse it in order to populate the TensorRT network.
with open(model_file, 'rb') as model:
# downloaded the arcface mdoel
ONNX_file_path = './resnet100.ONNX'
engine = build_engine_ONNX(ONNX_file_path)
engine_file_path = './arcface_trt.engine'
with open(engine_file_path, "wb") as f:
After running the script, we get some error “Segmentation fault core dumped”. After doing a lot of research we have found that there is no issue with the script. There are some other reasons why we are facing this problem. The reasons and explanations are discussed in the following paragraphs.
Jetson Nano is a ARM architecture-based device where TensorRT 5.1 is already pre-installed. The image which is written on SD card of NVIDIA Jetpack SDK does not includes TensorRT 6. It is possible to convert other models to TensorRT and run inference on top of it but it’s not possible with arcface. The arcface model cannot be converted because it contains a PRELU activation function which only supports TensorRT 6.
Model cannot be converted because we are unable to upgrade the TensorRT version from 5.1 to 6. So, unless and until NVIDIA provides us a Jetpack SDK OS image with the latest version of TensorRT 6 specifically the arcface model cannot be converted.
The installation file of TensorRT 6 is only supportable for AMD64 architecture which can’t be run on Jetson Nano because it is an ARM-architecture device. That’s why, the arcface ONNX model conversion is failed.
As soon as, NVIDIA Jetpack SDK releases OS image with TensorRT 6 the arcface ONNX model will get converted to TensorRT and we can run inference on top of it. I am all ears to know your thoughts/ideas to make it happen if NVDIA is taking its time to update jetpack SDK. We at DataToBiz always strive for latest tools & technologies to get ahead from our competitors. Contact for further details