YOLO11 Pose
本文档讲述如何在 NPU 上运行 YOLO11 Pose。
信息
参考 Model Zoo 下载获取示例。
YOLO11 Pose 示例目录结构:
$ tree ./
./
├── CMakeLists.txt
├── convert_model
│ ├── config_yml.py
│ ├── convert_model_env.sh
│ ├── python
│ │ ├── onnx_extract.py
│ │ └── yolo11s-pose_640.txt
│ └── yolo11s-pose_9.txt
├── figures
│ ├── diff_img.png
│ └── out_yolo11_pose_pcq.png
├── main.cpp
├── model
│ └── COCO_train2014_000000500390.jpg
├── model_config.h
├── README.md
├── yolo11_pose_9_post.cpp
└── yolo11_pose_9_pre.cpp
模型转换
配置虚拟环境
X86 Linux PC
python -m venv .venv && source .venv/bin/activate
pip install ultralytics
导出 onnx 模型
X86 Linux PC
cd convert_model/python/
yolo export model=yolo11s-pose.pt format=onnx simplify=True dynamic=False opset=11 nms=False batch=1 device=cpu
裁剪模型
X86 Linux PC
python onnx_extract.py
mv ./yolo11s-pose_9.onnx ../
cd ..
创建转换脚本的软链接
X86 Linux PC
./convert_model_env.sh
模型导入/量化/转换
需要先进入容器开发环境。可以参考 Model Zoo 下载中创建容器这一部分。
信息
不同平台请使用对应的 Docker 镜像:
- A733:ubuntu-npu:v2.0.10.1
- T527:ubuntu-npu:v1.8.11
X86 Linux PC
docker exec -it model-zoo /bin/bash
进入容器对应目录之后运行脚本。
X86 Linux PC
cd /workspace/examples/yolo11_pose/convert_model/
X86 Linux PC
./pegasus_import.sh yolo11s-pose_9
./pegasus_quantize.sh yolo11s-pose_9 uint8 12
- A733
- T527
X86 Linux PC
./pegasus_export_ovx_nbg.sh yolo11s-pose_9 uint8 a733
X86 Linux PC
./pegasus_export_ovx_nbg.sh yolo11s-pose_9 uint8 t527
导出的模型文件存放在../model目录。
编译示例
接下来可以编译示例,先 exit 退出容器,然后执行下面的命令编译示例。
首先需要配置第三方库和交叉编译工具链。
信息
如果你已经在其他示例中配置过第三方库和交叉编译工具链则可以跳过这一步。
X86 Linux PC
cd ../../../3rdparty/opencv/
unzip opencv-4.9.0-aarch64-linux-sunxi-glibc.zip
cd ../../0-toolchains/
需要先手动点击链接下载之后放到 0-toolchains/ 再执行下面的命令:
X86 Linux PC
tar -xvf gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu.tar.xz
X86 Linux PC
cd ../examples/yolo11_pose/
- A733
- T527
X86 Linux PC
../build_linux.sh -t a733 -s debian11
X86 Linux PC
../build_linux.sh -t t527 -s debian11
模型部署
编译示例完成之后,示例会安装到 install 目录,可以使用 scp 传输到板端。
配置 NPU 驱动
信息
如果你已经在其他示例中配置过 NPU 驱动则可以跳过这一步。
将驱动库 scp 传输到板端的 lib 目录。
- A733 对应 common/lib_linux_aarch64/A733 目录
- T527 对应 common/lib_linux_aarch64/T527 目录
然后执行下面的命令导出到环境变量。
Radxa SBC
echo 'export LD_LIBRARY_PATH=$HOME/lib:$LD_LIBRARY_PATH' >> ~/.bashrc
运行示例
配置好驱动之后就可以运行示例了。
提示
对于 T527 平台,你还需要参考 A5E 的板端启用 NPU文档先启用 NPU ,然后使用下面的命令增加当前用户使用 /dev/vipcore 的权限。
Radxa SBC
sudo chmod 777 /dev/vipcore
- A733
- T527
Radxa SBC
cd yolo11_pose_demo_linux_a733/
Radxa SBC
chmod +x ./yolo11_pose_demo_a733
./yolo11_pose_demo_a733 -nb model/yolo11s-pose_9_uint8_a733.nb -i model/COCO_train2014_000000500390.jpg
运行结果如下:
$ ./yolo11_pose_demo_a733 -nb model/yolo11s-pose_9_uint8_a733.nb -i model/COCO_train2014_000000500390.jpg
model_file=model/yolo11s-pose_9_uint8_a733.nb, input=model/COCO_train2014_000000500390.jpg, loop_count=1, malloc_mbyte=10
VIPLite driver software version 2.0.3.2-AW-2024-08-30
input 0 dim 3 640 640 1, data_format=2, quant_format=0, name=input/output[0], none-quant
output 0 dim 80 80 64 1, data_format=0, name=uid_17_out_0b_uid_1_out_0, none-quant
output 1 dim 80 80 1 1, data_format=0, name=uid_16_out_0b_uid_1_out_0, none-quant
output 2 dim 80 80 51 1, data_format=0, name=uid_15_out_0b_uid_1_out_0, none-quant
output 3 dim 40 40 64 1, data_format=0, name=uid_14_out_0b_uid_1_out_0, none-quant
output 4 dim 40 40 1 1, data_format=0, name=uid_13_out_0b_uid_1_out_0, none-quant
output 5 dim 40 40 51 1, data_format=0, name=uid_12_out_0b_uid_1_out_0, none-quant
output 6 dim 20 20 64 1, data_format=0, name=uid_11_out_0b_uid_1_out_0, none-quant
output 7 dim 20 20 1 1, data_format=0, name=uid_10_out_0b_uid_1_out_0, none-quant
output 8 dim 20 20 51 1, data_format=0, name=uid_9_out_0ub_uid_1_out_0, none-quant
nbg name=model/yolo11s-pose_9_uint8_a733.nb, size: 7284048.
create network 0: 16110 us.
prepare network: 3977 us.
buffer ptr: 0x202f5380, buffer size: 1228800
network: 0, loop count: 1
run time for this network 0: 32374 us.
output 0, ptr 0x20421480, size 409600.
output 1, ptr 0x205b1500, size 6400.
output 2, ptr 0x205b7980, size 326400.
output 3, ptr 0x206f6640, size 102400.
output 4, ptr 0x2075a6c0, size 1600.
output 5, ptr 0x2075c040, size 81600.
output 6, ptr 0x207abbc0, size 25600.
output 7, ptr 0x207c4c80, size 400.
output 8, ptr 0x207c5340, size 20400.
post process time : 4 ms
detection num: 3
0: 94%, [ 370, 0, 589, 346], person
405.75 26.20 = 0.96988
419.11 23.03 = 0.96501
405.65 21.63 = 0.29929
441.04 31.18 = 0.99146
421.11 22.33 = 0.04379
455.76 67.51 = 0.99977
430.35 62.14 = 0.99950
466.39 121.18 = 0.99797
405.08 109.99 = 0.98330
447.50 96.32 = 0.98985
382.14 70.42 = 0.94582
466.06 166.69 = 0.99986
455.44 165.19 = 0.99974
411.43 242.60 = 0.99939
497.02 230.87 = 0.99880
408.66 307.99 = 0.98213
562.98 301.11 = 0.97806
0: 88%, [ 86, 27, 292, 389], person
146.77 66.48 = 0.99659
157.56 60.56 = 0.99517
138.54 62.15 = 0.93738
177.10 58.15 = 0.97191
136.75 58.00 = 0.22714
182.16 88.95 = 0.99876
146.17 100.58 = 0.99755
210.99 144.46 = 0.99757
161.48 152.14 = 0.98186
171.08 179.70 = 0.99453
131.07 188.60 = 0.97326
222.98 197.17 = 0.99975
178.42 204.61 = 0.99950
250.05 264.09 = 0.99831
151.41 290.25 = 0.99650
287.21 296.16 = 0.97016
127.74 355.67 = 0.95755
0: 92%, [ 228, 39, 399, 407], person
275.86 94.61 = 0.99351
286.44 88.28 = 0.98999
267.42 87.73 = 0.88035
308.03 73.30 = 0.97833
265.48 74.83 = 0.23741
339.54 98.91 = 0.99963
280.47 109.74 = 0.99938
372.16 125.90 = 0.99505
272.82 170.12 = 0.98157
380.93 163.22 = 0.98073
243.21 204.51 = 0.94730
339.07 225.60 = 0.99986
302.82 223.45 = 0.99980
294.02 310.02 = 0.99952
314.44 286.69 = 0.99926
270.90 355.43 = 0.99344
374.00 318.30 = 0.99277
destroy npu finished.
~NpuUint.
此性能数据仅计算模型推理的时间消耗。如无特别说明,不包含预处理和后处理的时间消耗。
| SoC | NPU | 模型 | 输入分辨率 | 网络创建耗时 | 网络准备耗时 | 单帧推理耗时 | 后处理耗时 | 总耗时 | 帧率 |
|---|---|---|---|---|---|---|---|---|---|
| 全志 A733 | Vivante VIP9000 | yolo11s-pose | 640×640 | 16.1 ms | 4.0 ms | 32.4 ms | 4.0 ms | 56.5 ms | 30.9 FPS |
Radxa SBC
cd yolo11_pose_demo_linux_t527/
Radxa SBC
chmod +x ./yolo11_pose_demo_t527
./yolo11_pose_demo_t527 -nb model/yolo11s-pose_9_uint8_t527.nb -i model/COCO_train2014_000000500390.jpg
运行结果如下:
$ ./yolo11_pose_demo_t527 -nb model/yolo11s-pose_9_uint8_t527.nb -i model/COCO_train2014_000000500390.jpg
model_file=model/yolo11s-pose_9_uint8_t527.nb, input=model/COCO_train2014_000000500390.jpg, loop_count=1, malloc_mbyte=10
VIPLite driver software version 1.13.0.0-AW-2023-10-19
input 0 dim 3 640 640 1, data_format=2, quant_format=0, name=input[0], none-quant
output 0 dim 80 80 64 1, data_format=0, name=uid_20000_sub_uid_1_out_0, none-quant
output 1 dim 80 80 1 1, data_format=0, name=uid_20001_sub_uid_1_out_0, none-quant
output 2 dim 80 80 51 1, data_format=0, name=uid_20002_sub_uid_1_out_0, none-quant
output 3 dim 40 40 64 1, data_format=0, name=uid_20003_sub_uid_1_out_0, none-quant
output 4 dim 40 40 1 1, data_format=0, name=uid_20004_sub_uid_1_out_0, none-quant
output 5 dim 40 40 51 1, data_format=0, name=uid_20005_sub_uid_1_out_0, none-quant
output 6 dim 20 20 64 1, data_format=0, name=uid_20006_sub_uid_1_out_0, none-quant
output 7 dim 20 20 1 1, data_format=0, name=uid_20007_sub_uid_1_out_0, none-quant
output 8 dim 20 20 51 1, data_format=0, name=uid_20008_sub_uid_1_out_0, none-quant
nbg name=model/yolo11s-pose_9_uint8_t527.nb, size: 8148288.
create network 0: 23417 us.
prepare network: 10280 us.
buffer ptr: 0x22f74380, buffer size: 1228800
network: 0, loop count: 1
run time for this network 0: 75989 us.
output 0, ptr 0x230a0440, size 409600.
output 1, ptr 0x23230500, size 6400.
output 2, ptr 0x23236980, size 326400.
output 3, ptr 0x23375600, size 102400.
output 4, ptr 0x233d9680, size 1600.
output 5, ptr 0x233db040, size 81600.
output 6, ptr 0x2342abc0, size 25600.
output 7, ptr 0x23443c40, size 400.
output 8, ptr 0x23444300, size 20400.
post process time : 11 ms
detection num: 3
0: 94%, [ 371, 0, 587, 346], person
406.01 30.36 = 0.96783
418.34 26.25 = 0.95618
406.01 26.25 = 0.45198
434.78 30.36 = 0.98485
418.34 26.25 = 0.12014
455.34 71.46 = 0.99981
426.56 67.35 = 0.99938
467.67 120.79 = 0.99790
406.01 104.35 = 0.98050
451.23 96.12 = 0.98748
389.57 75.57 = 0.93298
475.89 174.23 = 0.99984
459.45 170.12 = 0.99967
414.23 244.11 = 0.99955
484.11 235.88 = 0.99903
410.12 301.65 = 0.99145
558.10 297.54 = 0.98825
0: 87%, [ 86, 27, 292, 389], person
147.67 66.46 = 0.99650
160.00 58.25 = 0.99518
139.45 58.25 = 0.94058
180.55 58.25 = 0.96977
135.34 54.13 = 0.22788
180.55 87.02 = 0.99866
147.67 99.35 = 0.99729
213.44 144.57 = 0.99729
164.11 148.68 = 0.98050
172.33 177.45 = 0.99417
131.23 189.78 = 0.97160
221.66 198.00 = 0.99971
176.44 202.12 = 0.99946
250.43 263.77 = 0.99816
151.78 288.44 = 0.99627
283.32 292.55 = 0.96783
123.00 354.21 = 0.95618
0: 92%, [ 228, 38, 399, 408], person
275.67 96.12 = 0.99247
288.00 87.90 = 0.98897
267.45 87.90 = 0.86560
308.55 75.57 = 0.97646
267.45 75.57 = 0.22788
341.44 100.23 = 0.99960
279.78 108.46 = 0.99930
374.32 124.90 = 0.99487
275.67 170.12 = 0.97924
382.54 161.89 = 0.98050
242.78 203.00 = 0.94407
341.44 227.66 = 0.99985
304.44 223.55 = 0.99978
292.11 309.88 = 0.99949
312.66 285.21 = 0.99920
271.56 355.09 = 0.99294
374.32 318.10 = 0.99198
destroy npu finished.
~NpuUint.
此性能数据仅计算模型推理的时间消耗。如无特别说明,不包含预处理和后处理的时间消耗。
| SoC | NPU | 模型 | 输入分辨率 | 网络创建耗时 | 网络准备耗时 | 单帧推理耗时 | 后处理耗时 | 总耗时 | 帧率 |
|---|---|---|---|---|---|---|---|---|---|
| 全志 T527 | Vivante VIP9000 | yolo11s-pose | 640×640 | 23.4 ms | 10.3 ms | 76.0 ms | 11.0 ms | 120.7 ms | 13.2 FPS |