OpenAI-Clip Example

This document describes how to use the QAI AppBuilder Python API to run inference with the OpenAI-CLIP model on Qualcomm® Hexagon™ Processor (NPU).

Supported devices

Device	SoC
Fogwise® AIRbox Q900	QCS9075

Install QAI AppBuilder

tip

1. Install QAI AppBuilder by following the QAI AppBuilder installation guide.

2. Configure ADSP environment variables as described in Create ADSP environment variables.

Run the sample

Install dependencies

Install sample dependencies in the activated virtual environment:

Device

pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless "openai-clip==1.0.1" ftfy "setuptools<81"

Run the script

• Enter the upstream samples directory

  Device

  cd qai-appbuilder/samples

• Prepare input data (use the sample input if provided, or pass script arguments)

image1 image2 image3

• Run inference

  Device

  python3 Multimodal/Image_Classification/openai_clip/openai_clip.py --chipset 9075 --text "mountain"

On success the terminal prints a similarity score per image, for example:

Image with name: image2.jpg has a similarity score=[[18.687502]]
Image with name: image1.jpg has a similarity score=[[16.109377]]
Image with name: image3.jpg has a similarity score=[[34.031254]]

(Scores vary with the images and query text; the highest score best matches the text.)

• Example result

image3

tip

openai-clip relies on the older pkg_resources API; pin setuptools below 81. Prefer running the script directly. The first run downloads the model via Qualcomm® AI Hub:

python3 run_inference.py --list
python3 Multimodal/Image_Classification/openai_clip/openai_clip.py --chipset 9075 --text "mountain"