YOLOv5

This document describes how to run YOLOv5 on NPU.

info

Refer to Model Zoo Download for the example.

YOLOv5 Example Directory Structure:

$ tree ./
./
├── CMakeLists.txt
├── convert_model
│   ├── config_yml.py
│   ├── convert_model_env.sh
│   ├── python
│   │   ├── onnx_extract.py
│   │   └── yolov5s-sim.onnx
│   └── yolov5s_rt.onnx
├── figures
│   ├── dog.jpg
│   ├── onnx.jpg
│   ├── onnx-sim.jpg
│   ├── output.jpg
│   └── output_yolov5_uint8.png
├── main.cpp
├── model
│   ├── dog.jpg
│   ├── yolov5s_rt_uint8_a733.nb
│   └── yolov5s_rt_uint8_t527.nb
├── model_config.h
├── README.md
├── yolov5_post.cpp
└── yolov5_pre.cpp

Model Conversion

No need to export to ONNX model. Just use the yolov5s_rt.onnx provided in the repository.

Create Symlink for Conversion Script

X86 Linux PC

cd convert_model/
./convert_model_env.sh

Model Import/Quantization/Conversion

You need to enter the container development environment first. Refer to the Create Container section in Model Zoo Download.

info

Different platforms use corresponding Docker images:

• A733: ubuntu-npu:v2.0.10.1
• T527: ubuntu-npu:v1.8.11

X86 Linux PC

docker exec -it model-zoo /bin/bash

After entering the container, navigate to the corresponding directory and run the script.

X86 Linux PC

cd /workspace/examples/yolov5/convert_model/

X86 Linux PC

./pegasus_import.sh yolov5s_rt
./pegasus_quantize.sh yolov5s_rt uint8 12

A733

X86 Linux PC

./pegasus_export_ovx_nbg.sh yolov5s_rt uint8 a733

T527

X86 Linux PC

./pegasus_export_ovx_nbg.sh yolov5s_rt uint8 t527

The exported model files are stored in the ../model directory.

Compile Example

Now you can compile the example. First exit the container, then execute the following command to compile the example.

First, you need to configure third-party libraries and cross-compilation toolchain.

info

You can skip this step if you have already configured third-party libraries and cross-compilation toolchain in other examples.

X86 Linux PC

cd ../../../3rdparty/opencv/
unzip opencv-4.9.0-aarch64-linux-sunxi-glibc.zip
cd ../../0-toolchains/

You need to manually download via this link first, then place it in 0-toolchains/ before executing the following command:

X86 Linux PC

tar -xvf gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu.tar.xz

X86 Linux PC

cd ../examples/yolov5/

A733

X86 Linux PC

../build_linux.sh -t a733 -s debian11

T527

X86 Linux PC

../build_linux.sh -t t527 -s debian11

Model Deployment

After compilation, the example will be installed in the install directory. You can use scp to transfer it to the board.

Configure NPU Driver

info

You can skip this step if you have already configured NPU driver in other examples.

Transfer the driver library to the board's lib directory via scp.

• A733 corresponds to the common/lib_linux_aarch64/A733 directory
• T527 corresponds to the common/lib_linux_aarch64/T527 directory

Then execute the following command to export to environment variables.

Radxa SBC

echo 'export LD_LIBRARY_PATH=$HOME/lib:$LD_LIBRARY_PATH' >> ~/.bashrc

Run Example

After configuring the driver, you can run the example.

tip

For T527 platform, you need to first enable NPU by referring to the A5E's "Enable NPU on Board" documentation, then use the following command to grant the current user permission to use /dev/vipcore.

Radxa SBC

sudo chmod 777 /dev/vipcore

A733

Radxa SBC

cd yolov5_demo_linux_a733/

Radxa SBC

chmod +x ./yolov5_demo_a733
./yolov5_demo_a733 -nb model/yolov5s_rt_uint8_a733.nb -i model/dog.jpg

The running result is as follows:

$ ./yolov5_demo_a733 -nb model/yolov5s_rt_uint8_a733.nb -i model/dog.jpg
model_file=model/yolov5s_rt_uint8_a733.nb, input=model/dog.jpg, loop_count=1, malloc_mbyte=10
VIPLite driver software version 2.0.3.2-AW-2024-08-30
input  0 dim 3 640 640 1, data_format=2, quant_format=0, name=input/output[0], none-quant
output 0 dim 80 80 3 1, data_format=0, name=uid_20000_sub_uid_1_out_0, none-quant
output 1 dim 40 40 3 1, data_format=0, name=uid_20001_sub_uid_1_out_0, none-quant
output 2 dim 20 20 3 1, data_format=0, name=uid_20002_sub_uid_1_out_0, none-quant
output 3 dim 80 80 80 1, data_format=0, name=uid_20003_sub_uid_1_out_0, none-quant
output 4 dim 40 40 80 1, data_format=0, name=uid_20005_sub_uid_1_out_0, none-quant
output 5 dim 20 20 80 1, data_format=0, name=uid_20006_sub_uid_1_out_0, none-quant
nbg name=model/yolov5s_rt_uint8_a733.nb, size: 7246236.
create network 0: 10429 us.
prepare network: 1587 us.
buffer ptr: 0x190c0300, buffer size: 1228800
network: 0, loop count: 1
run time for this network 0: 31862 us.
detection num: 3
16:  93%, [ 137,  229,  303,  539], dog
 7:  70%, [ 471,   79,  687,  171], truck
 1:  47%, [ 151,  120,  560,  429], bicycle
draw objects time : 44 ms
destroy npu finished.
~NpuUint.

This performance data only calculates the time consumption of model inference. Unless otherwise specified, it does not include the time consumption of pre-processing and post-processing.

SoC	NPU	Model	Input Resolution	Network Creation Time	Network Preparation Time	Single Frame Inference Time	Post-processing Time	Total Time	Frame Rate
Allwinner A733	Vivante VIP9000	yolov5s	640×640	11.6 ms	2.8 ms	20.1 ms	32.0 ms	66.5 ms	49.8 FPS

T527

Radxa SBC

cd yolov5_demo_linux_t527/

Radxa SBC

chmod +x ./yolov5_demo_t527
./yolov5_demo_t527 -nb model/yolov5s_rt_uint8_t527.nb -i model/dog.jpg

The running result is as follows:

$ ./yolov5_demo_t527 -nb model/yolov5s_rt_uint8_t527.nb -i model/dog.jpg
model_file=model/yolov5s_rt_uint8_t527.nb, input=model/dog.jpg, loop_count=1, malloc_mbyte=10
VIPLite driver software version 1.13.0.0-AW-2023-10-19
input  0 dim 3 640 640 1, data_format=2, quant_format=0, name=input[0], none-quant
output 0 dim 80 80 3 1, data_format=0, name=uid_20000_sub_uid_1_out_0, none-quant
output 1 dim 40 40 3 1, data_format=0, name=uid_20001_sub_uid_1_out_0, none-quant
output 2 dim 20 20 3 1, data_format=0, name=uid_20002_sub_uid_1_out_0, none-quant
output 3 dim 80 80 80 1, data_format=0, name=uid_20003_sub_uid_1_out_0, none-quant
output 4 dim 40 40 80 1, data_format=0, name=uid_20004_sub_uid_1_out_0, none-quant
output 5 dim 20 20 80 1, data_format=0, name=uid_20005_sub_uid_1_out_0, none-quant
nbg name=model/yolov5s_rt_uint8_t527.nb, size: 7246236.
create network 0: 24280 us.
prepare network: 10836 us.
buffer ptr: 0xbe8ff600, buffer size: 1228800
network: 0, loop count: 1
run time for this network 0: 49985 us.
detection num: 3
16:  92%, [ 137,  229,  303,  539], dog
 7:  69%, [ 471,   79,  687,  171], truck
 1:  44%, [ 151,  120,  560,  429], bicycle
draw objects time : 81 ms
destroy npu finished.
~NpuUint.

This performance data only calculates the time consumption of model inference. Unless otherwise specified, it does not include the time consumption of pre-processing and post-processing.

SoC	NPU	Model	Input Resolution	Network Creation Time	Network Preparation Time	Single Frame Inference Time	Post-processing Time	Total Time	Frame Rate
Allwinner T527	Vivante VIP9000	yolov5s	640×640	24.3 ms	10.8 ms	50.0 ms	81.0 ms	166.1 ms	20.0 FPS