OpenAI-Clip Example
This document describes how to use the QAI AppBuilder Python API to run inference with the OpenAI-CLIP model on Qualcomm® Hexagon™ Processor (NPU).
Supported Devices
| Device | SoC |
|---|---|
| Fogwise® AIRbox Q900 | QCS9075 |
Install QAI AppBuilder
tip
-
Please install QAI AppBuilder according to QAI AppBuilder Installation Guide.
-
Please configure ADSP environment variables according to Create ADSP Environment Variables.
Run the Example
Install Dependencies
Device
pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models transformers openai-clip
Run the Script
-
Navigate to the example directory
- QCS9075
Devicecd ai-engine-direct-helper/samples/python -
Prepare input images. The following images are used as examples:
image1 image2 image3
-
Run inference
Devicepython3 openai_clip/openai_clip.py --text camping under the stars$ python3 openai_clip/openai_clip.py
/mnt/ssd/qualcomm/702/zzf_fork/test/ai-engine-direct-helper/.venv/lib/python3.12/site-packages/clip/clip.py:6: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
from pkg_resources import packaging
0.0ms [WARNING] <W> Initializing HtpProvider
/prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc
0.0ms [WARNING] <W> This META does not have Alloc2 Support
0.0ms [WARNING] <W> This META does not have Alloc2 Support
0.0ms [WARNING] <W> This META does not have Alloc2 Support
0.0ms [WARNING] <W> This META does not have Alloc2 Support
232.1ms [WARNING] Time: Read model file to memory. 121.58
0.0ms [WARNING] <W> This META does not have Alloc2 Support
0.0ms [WARNING] <W> This META does not have Alloc2 Support
0.0ms [WARNING] <W> This META does not have Alloc2 Support
0.0ms [WARNING] <W> This META does not have Alloc2 Support
0.0ms [WARNING] <W> This META does not have Alloc2 Support
0.0ms [WARNING] <W> This META does not have Alloc2 Support
0.0ms [WARNING] <W> This META does not have Alloc2 Support
361.8ms [WARNING] Time: contextCreateFromBinary. 129.68
361.9ms [WARNING] Time: UnmapViewOfFile. 0.00
365.4ms [WARNING] Time: model_initialize openai_clip 365.30
1453.9ms [WARNING] Time: model_inference openai_clip 25.61
1478.6ms [WARNING] Time: model_inference openai_clip 24.52
1504.1ms [WARNING] Time: model_inference openai_clip 25.45
Searching images by prompt: camping under the stars
Image with name: image2.jpg has a similarity score=[19.906252]
Image with name: image1.jpg has a similarity score=[16.578127]
Image with name: image3.jpg has a similarity score=[26.828127]
Displaying the most relevant image
0.0ms [WARNING] <W> This META does not have Alloc2 Support
0.0ms [WARNING] <W> This META does not have Alloc2 Support
/usr/bin/xdg-open: 882: www-browser: not found
0.0ms [WARNING] <W> This META does not have Alloc2 Support
/usr/bin/xdg-open: 882: links2: not found
0.0ms [WARNING] <W> This META does not have Alloc2 Support
/usr/bin/xdg-open: 882: elinks: not found
/usr/bin/xdg-open: 882: links: not found
/usr/bin/xdg-open: 882: lynx: not found
/usr/bin/xdg-open: 882: w3m: not found
xdg-open: no method available for opening '/tmp/tmpiq2mgemq.PNG'
/prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc
7951.5ms [WARNING] Time: model_destroy openai_clip 122.86The results show that the image most similar to
camping under the starsisimage3.jpg
image3