ResNet-3D Example

This document explains how to use the QAI AppBuilder Python API to perform inference with the ResNet-3D video classification model using the Qualcomm® Hexagon™ Processor (NPU).

Supported Devices

Device	SoC
Fogwise® AIRbox Q900	QCS9075

Install QAI AppBuilder

tip

1. Please install QAI AppBuilder according to the QAI AppBuilder Installation Guide.

2. Please configure the ADSP environment variables according to Configuring ADSP Environment Variables.

Run Example

Install Dependencies

Device

pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models transformers av

Run the Script

• Navigate to the example directory

  QCS9075

  Device

  cd ai-engine-direct-helper/samples/python

• Prepare the input video. The following video is used as an example:

  Input video

• Execute inference

  Device

  python3 resnet_3d/resnet_3d.py

  $ python3 resnet_3d/resnet_3d.py
       0.0ms [WARNING]  <W> Initializing HtpProvider
  
  /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc
       0.0ms [WARNING]  <W> This META does not have Alloc2 Support
  
       0.0ms [WARNING]  <W> This META does not have Alloc2 Support
  
       0.0ms [WARNING]  <W> This META does not have Alloc2 Support
  
       0.0ms [WARNING]  <W> This META does not have Alloc2 Support
  
     157.9ms [WARNING] Time: Read model file to memory. 35.20
  
       0.0ms [WARNING]  <W> This META does not have Alloc2 Support
  
       0.0ms [WARNING]  <W> This META does not have Alloc2 Support
  
       0.0ms [WARNING]  <W> This META does not have Alloc2 Support
  
       0.0ms [WARNING]  <W> This META does not have Alloc2 Support
  
       0.0ms [WARNING]  <W> This META does not have Alloc2 Support
  
       0.0ms [WARNING]  <W> This META does not have Alloc2 Support
  
       0.0ms [WARNING]  <W> This META does not have Alloc2 Support
  
     213.3ms [WARNING] Time: contextCreateFromBinary. 55.38
  
     213.4ms [WARNING] Time: UnmapViewOfFile. 0.00
  
     215.6ms [WARNING] Time: model_initialize resnet_3d 215.54
  
  /mnt/ssd/qualcomm/702/zzf_fork/test/ai-engine-direct-helper/.venv/lib/python3.12/site-packages/torchvision/io/_video_deprecation_warning.py:5: UserWarning: The video decoding and encoding capabilities of torchvision are deprecated from version 0.22 and will be removed in version 0.24. We recommend that you migrate to TorchCodec, where we'll consolidate the future decoding/encoding capabilities of PyTorch: https://github.com/pytorch/torchcodec
    warnings.warn(
    3691.5ms [WARNING] Time: model_inference resnet_3d 27.89
  
  Top 5 predictions:
  surfing water, kitesurfing, windsurfing, jetskiing, water sliding
  
       0.0ms [WARNING]  <W> This META does not have Alloc2 Support
  
       0.0ms [WARNING]  <W> This META does not have Alloc2 Support
  
       0.0ms [WARNING]  <W> This META does not have Alloc2 Support
  
       0.0ms [WARNING]  <W> This META does not have Alloc2 Support
  
  /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc
    3795.4ms [WARNING] Time: model_destroy resnet_3d 92.13

  Recognition Results

  Top 5 predictions:
  surfing water, kitesurfing, windsurfing, jetskiing, water sliding