Skip to main content

Zipformer

This document describes how to run the Zipformer ASR example on the NPU.

info

Refer to Model Zoo Download for the example.

Zipformer uses encoder, decoder, and joiner sub-models for automatic speech recognition.

Zipformer example directory structure:

$ tree ./
./
├── CMakeLists.txt
├── convert_model_decoder
├── convert_model_encoder
├── convert_model_joiner
├── model
│   ├── decoder_int16_a733.nb
│   ├── encoder_int16_a733.nb
│   ├── joiner_int16_a733.nb
│   ├── test.wav
│   └── tokens.txt
└── README.md

Model Conversion​

Enter the container development environment first. See Create and Start Container in the Model Zoo download page.

info

Select the Docker image that matches the NPU:

  • A733: ubuntu-npu:v2.0.10.2
  • T527: ubuntu-npu:v1.8.13

Download the floating-point ONNX models from the Allwinner netdisk:

X86 Linux PC
docker exec -it model-zoo /bin/bash

Set this environment variable before converting the encoder:

X86 Linux PC
export VIV_VX_ENABLE_GRAPH_TRANSFORM="-Dump[transform.rewrite.gather_to_stride_slice=0]"

Convert encoder​

X86 Linux PC
cd /workspace/examples/zipformer/convert_model_encoder/
./convert_model_env.sh
./pegasus_import.sh encoder
./pegasus_quantize.sh encoder int16 10
X86 Linux PC
./pegasus_export_ovx_nbg.sh encoder int16 a733

Convert decoder​

X86 Linux PC
cd /workspace/examples/zipformer/convert_model_decoder/
./convert_model_env.sh
./pegasus_import.sh decoder
./pegasus_quantize.sh decoder int16 10
X86 Linux PC
./pegasus_export_ovx_nbg.sh decoder int16 a733

Convert joiner​

X86 Linux PC
cd /workspace/examples/zipformer/convert_model_joiner/
./convert_model_env.sh
./pegasus_import.sh joiner
./pegasus_quantize.sh joiner int16 10
X86 Linux PC
./pegasus_export_ovx_nbg.sh joiner int16 a733

The exported models are stored in the ../model directory.

Build the Example​

Then compile the example. Exit the container first, then run the commands below.

Configure the cross-compilation toolchain first.

info

Skip this step if you have already configured it in another example.

X86 Linux PC
cd ../../../0-toolchains/

Download the toolchain from this link, put it in 0-toolchains/, then run:

X86 Linux PC
tar -xvf gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu.tar.xz
cd ../examples/zipformer/
X86 Linux PC
../build_linux.sh -t a733 -s debian11

Model Deployment​

After compilation, the example will be installed in the install directory. You can use scp to transfer it to the board.

Configure NPU Driver​

info

You can skip this step if you have already configured NPU driver in other examples.

Transfer the driver library to the board's lib directory via scp.

  • A733 corresponds to the common/npuruntime/lib_linux_aarch64/A733 directory
  • T527 corresponds to the common/npuruntime/lib_linux_aarch64/T527 directory

Then execute the following command to export to environment variables.

Radxa SBC
echo 'export LD_LIBRARY_PATH=$HOME/lib:$LD_LIBRARY_PATH' >> ~/.bashrc

Run Example​

After configuring the driver, you can run the example.

tip

For T527 platform, you need to first enable NPU by referring to the A5E's "Enable NPU on Board" documentation, then use the following command to grant the current user permission to use /dev/vipcore.

Radxa SBC
sudo chmod 777 /dev/vipcore
Radxa SBC
cd zipformer_demo_linux_a733/
Radxa SBC
chmod +x ./zipformer_demo_a733
./zipformer_demo_a733 -nb0 model/encoder_int16_a733.nb -nb1 model/decoder_int16_a733.nb -nb2 model/joiner_int16_a733.nb -i model/test.wav

The running result is as follows:

$ ./zipformer_demo_a733 -nb0 model/encoder_int16_a733.nb -nb1 model/decoder_int16_a733.nb -nb2 model/joiner_int16_a733.nb -i model/test.wav
encoder_path=model/encoder_int16_a733.nb, decoder_path=model/decoder_int16_a733.nb, joiner_path=model/joiner_int16_a733.nb, audio_path=model/test.wav
VIPLite driver software version 2.0.3.2-AW-2024-08-30
nbg name=model/encoder_int16_a733.nb, size: 37228136.
create network 0: 25435 us.
prepare network: 3858 us.
nbg name=model/decoder_int16_a733.nb, size: 6973440.
create network 1: 3971 us.
prepare network: 244 us.
nbg name=model/joiner_int16_a733.nb, size: 5509168.
create network 2: 2820 us.
prepare network: 56 us.

Real Time Factor(RTF): 0.053 = 0.309s(infer_time) / 5.840s(audio_length)

Timestamp (s): 0.96, 1.04, 1.24, 1.32, 1.44, 1.72, 1.80, 2.04, 2.56, 3.04, 3.08, 3.48, 3.84, 4.68, 4.76, 5.00, 5.24, 5.48

Zipformer output: 这是第一种第二种叫呃嗯与 ALWAYS什么意思啊
destroy npu finished.
~NpuUint.

This performance data only calculates the time consumption of model inference. Unless otherwise specified, it does not include the time consumption of pre-processing and post-processing.

SoCNPUModelInput ResolutionNetwork Creation TimeNetwork Preparation TimeSingle Frame Inference TimePost-processing TimeTotal TimeFrame Rate
Allwinner A733Vivante VIP9000zipformer5.84 s32.2 ms4.2 ms309.0 ms345.4 msRTF 0.053

    You need to be logged into GitHub to post a comment. If you are already logged in, please ignore this message.

    Radxa-docs © 2026 by Radxa Computer (Shenzhen) Co.,Ltd. is licensed under CC BY 4.0