PPSeg
This document describes how to run PPSeg on NPU.
Refer to Model Zoo Download for the example.
PPSeg Example Directory Structure:
$ tree ./
./
├── CMakeLists.txt
├── convert_model
│ ├── config_yml.py
│ ├── convert_model_env.sh
│ ├── model.pdparams
│ └── pp_liteseg_cityscapes.txt
├── figures
│ └── out_ppseg.png
├── main.cpp
├── model
│ └── munster_000022_000019_leftImg8bit.png
├── model_config.h
├── ppseg_post.cpp
├── ppseg_pre.cpp
└── README.md
Model Conversion
Download Model
Click to download pp_liteseg_cityscapes.
Then move the model to the convert_model/ directory.
Create Symlink for Conversion Script
cd convert_model/
./convert_model_env.sh
Model Import/Quantization/Conversion
You need to enter the container development environment first. Refer to the Create Container section in Model Zoo Download.
Different platforms use corresponding Docker images:
- A733: ubuntu-npu:v2.0.10.1
- T527: ubuntu-npu:v1.8.11
docker exec -it model-zoo /bin/bash
After entering the container, navigate to the corresponding directory and run the script.
cd /workspace/examples/ppseg/convert_model/
./pegasus_import.sh pp_liteseg_cityscapes
./pegasus_quantize.sh pp_liteseg_cityscapes pcq 10
- A733
- T527
./pegasus_export_ovx_nbg.sh pp_liteseg_cityscapes pcq a733
./pegasus_export_ovx_nbg.sh pp_liteseg_cityscapes pcq t527
The exported model files are stored in the ../model directory.
Compile Example
Now you can compile the example. First exit the container, then execute the following command to compile the example.
First, you need to configure third-party libraries and cross-compilation toolchain.
You can skip this step if you have already configured third-party libraries and cross-compilation toolchain in other examples.
cd ../../../3rdparty/opencv/
unzip opencv-4.9.0-aarch64-linux-sunxi-glibc.zip
cd ../../0-toolchains/
You need to manually download via this link first, then place it in 0-toolchains/ before executing the following command:
tar -xvf gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu.tar.xz
cd ../examples/ppseg/
- A733
- T527
../build_linux.sh -t a733 -s debian11
../build_linux.sh -t t527 -s debian11
Model Deployment
After compilation, the example will be installed in the install directory. You can use scp to transfer it to the board.
Configure NPU Driver
You can skip this step if you have already configured NPU driver in other examples.
Transfer the driver library to the board's lib directory via scp.
- A733 corresponds to the common/lib_linux_aarch64/A733 directory
- T527 corresponds to the common/lib_linux_aarch64/T527 directory
Then execute the following command to export to environment variables.
echo 'export LD_LIBRARY_PATH=$HOME/lib:$LD_LIBRARY_PATH' >> ~/.bashrc
Run Example
After configuring the driver, you can run the example.
For T527 platform, you need to first enable NPU by referring to the A5E's "Enable NPU on Board" documentation, then use the following command to grant the current user permission to use /dev/vipcore.
sudo chmod 777 /dev/vipcore
- A733
- T527
cd ppseg_demo_linux_a733/
chmod +x ./ppseg_demo_a733
./ppseg_demo_a733 -nb model/pp_liteseg_cityscapes_pcq_a733.nb -i model/munster_000022_000019_leftImg8bit.png
The running result is as follows:
$ ./ppseg_demo_a733 -nb model/pp_liteseg_cityscapes_pcq_a733.nb -i model/munster_000022_000019_leftImg8bit.png
model_file=model/pp_liteseg_cityscapes_pcq_a733.nb, input=model/munster_000022_000019_leftImg8bit.png, loop_count=1, malloc_mbyte=10
VIPLite driver software version 2.0.3.2-AW-2024-08-30
input 0 dim 3 512 512 1, data_format=2, quant_format=0, name=input/output[0], none-quant
output 0 dim 512 512 19 1, data_format=0, name=uid_20000_sub_uid_1_out_0, none-quant
nbg name=model/pp_liteseg_cityscapes_pcq_a733.nb, size: 10615496.
create network 0: 30474 us.
prepare network: 1578 us.
buffer ptr: 0x190c0300, buffer size: 786432
network: 0, loop count: 1
run time for this network 0: 81842 us.
output 0, ptr 0xffffa2a2b040, size 4980736.
post process time : 146 ms
ppseg_postprocess finished.
destroy npu finished.
~NpuUint.
This performance data only calculates the time consumption of model inference. Unless otherwise specified, it does not include the time consumption of pre-processing and post-processing.
| SoC | NPU | Model | Input Resolution | Network Creation Time | Network Preparation Time | Single Frame Inference Time | Post-processing Time | Total Time | Frame Rate |
|---|---|---|---|---|---|---|---|---|---|
| Allwinner A733 | Vivante VIP9000 | pp_liteseg_cityscapes | 512×512 | 30.5 ms | 1.6 ms | 81.8 ms | 146.0 ms | 260 ms | 12.2 FPS |
cd ppseg_demo_linux_t527/
chmod +x ./ppseg_demo_t527
./ppseg_demo_t527 -nb model/pp_liteseg_cityscapes_pcq_t527.nb -i model/munster_000022_000019_leftImg8bit.png
The running result is as follows:
$ ./ppseg_demo_t527 -nb model/pp_liteseg_cityscapes_pcq_t527.nb -i model/munster_000022_000019_leftImg8bit.png
model_file=model/pp_liteseg_cityscapes_pcq_t527.nb, input=model/munster_000022_000019_leftImg8bit.png, loop_count=1, malloc_mbyte=10
VIPLite driver software version 1.13.0.0-AW-2023-10-19
input 0 dim 3 512 512 1, data_format=2, quant_format=0, name=input[0], none-quant
output 0 dim 512 512 19 1, data_format=0, name=uid_20000_sub_uid_1_out_0, none-quant
nbg name=model/pp_liteseg_cityscapes_pcq_t527.nb, size: 10894912.
create network 0: 39396 us.
prepare network: 6386 us.
buffer ptr: 0x2fa182c0, buffer size: 786432
network: 0, loop count: 1
run time for this network 0: 122524 us.
output 0, ptr 0xffff8f7d9040, size 4980736.
post process time : 346 ms
ppseg_postprocess finished.
destroy npu finished.
~NpuUint.
This performance data only calculates the time consumption of model inference. Unless otherwise specified, it does not include the time consumption of pre-processing and post-processing.
| SoC | NPU | Model | Input Resolution | Network Creation Time | Network Preparation Time | Single Frame Inference Time | Post-processing Time | Total Time | Frame Rate |
|---|---|---|---|---|---|---|---|---|---|
| Allwinner T527 | Vivante VIP9000 | pp_liteseg_cityscapes | 512×512 | 39.4 ms | 6.4 ms | 122.5 ms | 346.0 ms | 514.3 ms | 8.2 FPS |