NPU Model Test Tool: vpm_run

tip

This document demonstrates how to use the model testing tool vpm_run on Allwinner T527/A733 series chips.

vpm_run is an edge-side program based on the VIPLite driver, primarily used for quickly validating and testing model files.

• vpm_run can serve as a general-purpose model runtime environment capable of running any correctly deployed NBG model file.
• vpm_run is based on the VIPLite network layer API.
• vpm_run includes a default post-processing program for TOP5 results.

Download the ai-sdk Example Repository

X86 PC / Device

git clone https://github.com/ZIFENG278/ai-sdk.git

Using vpm_run on the Board

Navigate to the directory containing the vpm_run example code.

Device

cd ai-sdk/examples/vpm_run

Compile the Example

Device

make AI_SDK_PLATFORM=a733
make install AI_SDK_PLATFORM=a733 INSTALL_PREFIX=./

Parameter explanation:

AI_SDK_PLATFORM: Specify the SoC, options are a733, t527.

INSTALL_PREFIX: Specify the installation path.

Usage

vpm_run -s sample.txt -l loop_run_count -d device_index
-s sample.txt:     to include one ore more network binary graph (NBG) data file resource.
                   See sample.txt for details.
-l loop_run_count: the number of loop run network.
-d device_index:   specify this NBG runs device.
-t time_out:       specify milliseconds time out of network.
-b bypass_level:   set value 1 to bypass saving output txt/binary file and showing top5.
--show_top5 level: set value 1 to show top5 when bypass_level is 0.
--save_txt level:  set value 1 to save txt output when bypass_level is 0.
-c core_index:     specify this start core index of device.
--layer_profile_dump:      set value 1 to enable NPD function.
--preload:         set value 1 to enable preload coeff into vipsram.
--op_segment:      set which operations will be run. example: --op_segment 10,20 means run 10 ~ 20
--layer_dump:  layer dump. eg: --layer_dump -1 dump all layer, --layer_dump 19, --layer_dump 18,20
-h : help
example: ./vpm_run -s sample.txt -l 10 -d 1 specify the NBG runs 10 times on device 1.

Run Example

• Import environment variables

Device

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/rock/ai-sdk/viplite-tina/lib/aarch64-none-linux-gnu/NPU_VERSION # NPU_SW_VERSION

tip

Specify NPU_SW_VERSION, A733 select v2.0, T527 select v1.13, NPU information对照请参考 NPU 版本对照表

• Make configuration file

  This example configuration file runs two NBG models, the first is a multi-input model, saving output files, and comparing with golden, the second is a single-input model, not saving output files.

  [network]
  ./network_binary.nb
  [input]
  #Multi-input model, configure multiple input files; single-input model, configure 1 input file.
  ./input_0.dat
  ./input_1.dat
  [golden]
  #Optional
  ./int8_output_003.dat
  [output]
  #Optional
  ./output_0.dat
  
  [network]
  ./network_binary_single.nb
  [input]
  ./input1.tensor

  sample.txt file example as above, main label introduction as follows:

  • [network]：NBG file path.
  • [input]：Input data path. Input data format supports two types: Acuity Toolkits tool inference stage generated input.tensor file; IDE tool simulation stage generated input.dat file.
  • [golden]：Optional golden data path. If the golden tag exists, it will be used as golden data and vpm_run run output tensor for binary comparison. If the comparison is consistent, it will print pass, and if the comparison is inconsistent, vpm_run will exit.
  • [output]：Optional. Output data save path.

• Enter the example installation directory

Device

cd INSTALL_PREFIX/etc/npu/vpm_run
# ./vpm_run nbg_model input_picture
./vpm_run ./model/network_binary.nb ./input_data/input_image.jpg

tip

The example automatically installs a sample NBG model provided by Radxa. You can manually specify the path to your own NBG model.

(.venv) rock@radxa-cubie-a7a:~/ai-sdk/examples/vpm_run/etc/npu/vpm_run$ ./vpm_run -s sample.txt -l 1
loop_count=1, device_index=0, core_index=-1, file_name=sample.txt, time_out=0x0, bypass=1
enable_npd=0, preload=0
show_top50, save_txt=0
init vip lite, driver version=0x00020003...
VIPLite driver software version 2.0.3.2-AW-2024-08-30
vip lite init OK.

cid=0x1000003b, device_count=1
  device[0] core_count=1
config file read network count=1
init test resources, task_count: 1 ...
create/prepare networks ...
task i=0, binary name: ./network_binary.nb
nbg name=./network_binary.nb
create network 0: 12354 us.
input 0 dim 3 224 224 1, data_format=5, quant_format=1, name=input/output[0], dfp=13
output 0 dim 1000 1 0 0, data_format=1, name=uid_2_sub_uid_1_out_0, none-quant
memory pool size=1073536byte
network core count=1
prepare network 0: 1409 us.
golden file count=0
input 0 name: ./input_0.dat
read input and golden 0: 541 us.
task: 0, loop count: 1
start to run network=./network_binary.nb
run time for this network 0: 3160 us.
run network done...
profile inference time=2945us, cycle=2916338
destroy test resource task_count=1
vpm run ret=0