OpenAI-Clip 示例

此文档讲述使用 QAI AppBuilder Python API 利用 Qualcomm® Hexagon™ Processor (NPU) 推理 OpenAI-Clip CLIP 模型。

示例支持设备

设备	SoC
Fogwise® AIRbox Q900	QCS9075

安装 QAI AppBuilder

提示

1. 请根据 QAI AppBuilder 安装方法 安装 QAI AppBuilder。

2. 请根据 创建 ADSP 环境变量 配置 ADSP 环境变量。

运行示例

安装依赖

Device

pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models transformers openai-clip

运行脚本

• 进入示例目录

  QCS9075

  Device

  cd ai-engine-direct-helper/samples/python

• 准备输入图片，这里以以下图片为输入示例

  

  image1 image2 image3

• 执行推理

  Device

  python3 openai_clip/openai_clip.py --text camping under the stars

  $ python3 openai_clip/openai_clip.py
  /mnt/ssd/qualcomm/702/zzf_fork/test/ai-engine-direct-helper/.venv/lib/python3.12/site-packages/clip/clip.py:6: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
    from pkg_resources import packaging
       0.0ms [WARNING]  <W> Initializing HtpProvider
  
  /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc
       0.0ms [WARNING]  <W> This META does not have Alloc2 Support
  
       0.0ms [WARNING]  <W> This META does not have Alloc2 Support
  
       0.0ms [WARNING]  <W> This META does not have Alloc2 Support
  
       0.0ms [WARNING]  <W> This META does not have Alloc2 Support
  
     232.1ms [WARNING] Time: Read model file to memory. 121.58
  
       0.0ms [WARNING]  <W> This META does not have Alloc2 Support
  
       0.0ms [WARNING]  <W> This META does not have Alloc2 Support
  
       0.0ms [WARNING]  <W> This META does not have Alloc2 Support
  
       0.0ms [WARNING]  <W> This META does not have Alloc2 Support
  
       0.0ms [WARNING]  <W> This META does not have Alloc2 Support
  
       0.0ms [WARNING]  <W> This META does not have Alloc2 Support
  
       0.0ms [WARNING]  <W> This META does not have Alloc2 Support
  
     361.8ms [WARNING] Time: contextCreateFromBinary. 129.68
  
     361.9ms [WARNING] Time: UnmapViewOfFile. 0.00
  
     365.4ms [WARNING] Time: model_initialize openai_clip 365.30
  
    1453.9ms [WARNING] Time: model_inference openai_clip 25.61
  
    1478.6ms [WARNING] Time: model_inference openai_clip 24.52
  
    1504.1ms [WARNING] Time: model_inference openai_clip 25.45
  
  Searching images by prompt: camping under the stars
       Image with name: image2.jpg has a similarity score=[19.906252]
       Image with name: image1.jpg has a similarity score=[16.578127]
       Image with name: image3.jpg has a similarity score=[26.828127]
  Displaying the most relevant image
       0.0ms [WARNING]  <W> This META does not have Alloc2 Support
  
       0.0ms [WARNING]  <W> This META does not have Alloc2 Support
  
  /usr/bin/xdg-open: 882: www-browser: not found
       0.0ms [WARNING]  <W> This META does not have Alloc2 Support
  
  /usr/bin/xdg-open: 882: links2: not found
       0.0ms [WARNING]  <W> This META does not have Alloc2 Support
  
  /usr/bin/xdg-open: 882: elinks: not found
  /usr/bin/xdg-open: 882: links: not found
  /usr/bin/xdg-open: 882: lynx: not found
  /usr/bin/xdg-open: 882: w3m: not found
  xdg-open: no method available for opening '/tmp/tmpiq2mgemq.PNG'
  /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc
    7951.5ms [WARNING] Time: model_destroy openai_clip 122.86

  结果显示与 camping under the stars 最匹配的图片是 image3.jpg

  

  image3