Skip to main content

Handwritten Digit Classification Model: LeNet

tip

This document demonstrates how to run the LeNet handwritten digit classification model on Allwinner T527/A733 series chips.

Deploying LeNet on the board requires two steps:

  • Use the ACUITY Toolkit on the PC to convert models from different frameworks into NBG format.
  • Use the awnn API on the board to perform inference with the model.

Download the ai-sdk Example Repository​

X86 PC / Device
git clone https://github.com/ZIFENG278/ai-sdk.git

Model Conversion on PC​

tip

Radxa provides a pre-converted lenet.nb model. Users can directly refer to LeNet Inference on the Board and skip the PC model conversion section.

tip

The files used in the LeNet example are already included in the ai-sdk example repository under models/lenet.

  • Enter the ACUITY Toolkit Docker container.

    For ACUITY Toolkit Docker environment setup, refer to ACUITY Toolkit Environment Setup.

    Configure environment variables:

    X86 Linux PC
    cd ai-sdk/models
    source env.sh v3 # NPU_VERSION

    For A733, choose v3; for T527, choose v2.

    tip

    Refer to the NPU Version Comparison Table for NPU version selection.

  • Navigate to the LeNet model directory:

    X86 Linux PC
    cd ai-sdk/models/lenet

  • Create a quantization calibration dataset.

    Use a set of images for quantization calibration. Save the image paths in dataset.txt.

    X86 Linux PC
    vim dataset.txt
    ./input_image/6.jpg 6
    ./input_image/1.jpg 1
    ./input_image/2.jpg 2
    ./input_image/5.jpg 5
    ./input_image/3.jpg 3
    ./input_image/4.jpg 4
    ./input_image/8.jpg 8
    ./input_image/7.jpg 7
    ./input_image/0.jpg 0
    ./input_image/9.jpg 9

  • Directory structure:

    .
    |-- channel_mean_value.txt
    |-- dataset.txt
    |-- input_image
    |   |-- 0.jpg
    |   |-- 1.jpg
    |   |-- 2.jpg
    |   |-- 3.jpg
    |   |-- 4.jpg
    |   |-- 5.jpg
    |   |-- 6.jpg
    |   |-- 7.jpg
    |   |-- 8.jpg
    |   `-- 9.jpg
    |-- lenet.caffemodel
    `-- lenet.prototxt

  • Parse the model.

    tip

    The pegasus script is located in ai-sdk/scripts and can be copied to the models directory.

    Use pegasus_import.sh to parse the model into an intermediate representation (IR). This generates lenet.json (model structure) and lenet.data (model weights).

    X86 Linux PC
    ./pegasus_import.sh lenet/

  • Modify the lenet_inputmeta.yml file.

    Update the scale value based on the formula scale = 1 / std.

    scale = 1 / 255
    scale = 0.00392157
    input_meta:
      databases:
      - path: dataset.txt
        type: TEXT
        ports:
        - lid: input_0
          category: image
          dtype: float32
          sparse: false
          tensor_name:
          layout: nchw
          shape:
          - 1
          - 1
          - 28
          - 28
          fitting: scale
          preprocess:
            reverse_channel: true
            mean:
            - 0
            scale:
            - 0.00392157
            preproc_node_params:
              add_preproc_node: false
              preproc_type: IMAGE_GRAY
              # preproc_dtype_converter:
                # quantizer: asymmetric_affine
                # qtype: uint8
                # scale: 1.0
                # zero_point: 0
              preproc_image_size:
              - 28
              - 28
              preproc_crop:
                enable_preproc_crop: false
                crop_rect:
                - 0
                - 0
                - 28
                - 28
              preproc_perm:
              - 0
              - 1
              - 2
              - 3
          redirect_to_output: false

  • Quantize the model.

    Use pegasus_quantize.sh to quantize the model into uint8 format.

    X86 Linux PC
    ./pegasus_quantize.sh lenet/ uint8 10

  • Compile the model.

    Use pegasus_export_ovx.sh to compile the model into NBG format.

    X86 Linux PC
    ./pegasus_export_ovx.sh lenet/ uint8

    The NBG model is saved in lenet/wksp/lenet_uint8_nbg_unify/network_binary.nb.

LeNet Inference on the Board​

Navigate to the LeNet example code directory.

Device
cd ai-sdk/examples/lenet

Compile the Example​

Device
make AI_SDK_PLATFORM=a733
make install AI_SDK_PLATFORM=a733 INSTALL_PREFIX=./

Parameter explanation:

  • AI_SDK_PLATFORM: Specify the SoC, options are a733 or t527.
  • INSTALL_PREFIX: Specify the installation path.

Run the Example​

Set environment variables.

Device
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/rock/ai-sdk/viplite-tina/lib/aarch64-none-linux-gnu/NPU_VERSION # NPU_SW_VERSION
tip

Specify NPU_SW_VERSION. For A733, choose v2.0; for T527, choose v1.13. Refer to the NPU Version Comparison Table for details.

Navigate to the example installation directory.

Device
cd INSTALL_PREFIX/etc/npu/lenet
# ./lenet nbg_model input_picture
./lenet ./model/lenet.nb ./input_data/lenet.dat
tip

The example automatically installs the lenet.nb model provided by Radxa. You can manually specify the path to your converted NBG model.

lenet input image

(.venv) rock@radxa-cubie-a7a:~/ai-sdk/examples/lenet/etc/npu/lenet$ ./lenet ./model/network_binary.nb ./input_data/lenet.dat
./lenet nbg input
VIPLite driver software version 2.0.3.2-AW-2024-08-30
viplite init OK.
VIPLite driver version=0x00020003...
VIP cid=0x1000003b, device_count=1
* device[0] core_count=1
awnn_init total: 4.07 ms.
  vip_create_network ./model/network_binary.nb: 1.10 ms.
input 0 dim 28 28 1 1, data_format=2, name=input/output[0], elements=134284329, scale=0.003922, zero_point=0
create input buffer 0: 784
output 0 dim 10 1, data_format=1, name=uid_8_sub_uid_1_out_0, elements=10, none-quant
create output buffer 0: 20
memory pool size=0 bytes
  load_param ./model/network_binary.nb: 0.24 ms.
  prepare network ./model/network_binary.nb: 0.13 ms.
  set network io ./model/network_binary.nb: 0.01 ms.
awnn_create total: 1.54 ms.
memcpy(0xffffacd12000, 0xaaaada2fed20, 784)  load_input_data: 0.01 ms.
  vip_flush_buffer input: 0.01 ms.
awnn_set_input_buffers total: 0.04 ms.
  vip_run_network: 0.22 ms.
  vip_flush_buffer output: 0.00 ms.
    fp16 memcpy: 0.00 ms.
  tensor to fp: 0.01 ms.
awnn_run total: 0.27 ms.
0.999512 0.000000 0.000120 0.000000 0.000000 0.000163 0.000152 0.000000 0.000000 0.000082
awnn_destroy total: 0.47 ms.
awnn_uninit total: 0.67 ms.