Nomic-Embed-Text Example
This document describes how to use the QAI AppBuilder Python API to run inference with the Nomic-Embed-Text text encoding model on Qualcomm® Hexagon™ Processor (NPU).
Supported Devices
| Device | SoC |
|---|---|
| Fogwise® AIRbox Q900 | QCS9075 |
Install QAI AppBuilder
tip
-
Please install QAI AppBuilder according to QAI AppBuilder Installation Guide.
-
Please configure ADSP environment variables according to Create ADSP Environment Variables.
Run the Example
Install Dependencies
Device
pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models transformers
Run the Script
-
Navigate to the example directory
- QCS9075
Devicecd ai-engine-direct-helper/samples/python -
Run inference
Devicepython3 nomic_embed_text/nomic_embed_text.py$ python3 nomic_embed_text/nomic_embed_text.py
0.0ms [WARNING] <W> Initializing HtpProvider
/prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc
0.0ms [WARNING] <W> This META does not have Alloc2 Support
0.0ms [WARNING] <W> This META does not have Alloc2 Support
0.0ms [WARNING] <W> This META does not have Alloc2 Support
0.0ms [WARNING] <W> This META does not have Alloc2 Support
226.8ms [WARNING] Time: Read model file to memory. 107.40
0.0ms [WARNING] <W> This META does not have Alloc2 Support
0.0ms [WARNING] <W> This META does not have Alloc2 Support
0.0ms [WARNING] <W> This META does not have Alloc2 Support
0.0ms [WARNING] <W> This META does not have Alloc2 Support
0.0ms [WARNING] <W> This META does not have Alloc2 Support
0.0ms [WARNING] <W> This META does not have Alloc2 Support
0.0ms [WARNING] <W> This META does not have Alloc2 Support
324.1ms [WARNING] Time: contextCreateFromBinary. 97.18
324.1ms [WARNING] Time: UnmapViewOfFile. 0.00
327.1ms [WARNING] Time: model_initialize nomic_embed_text 327.00
343.6ms [WARNING] Time: model_inference nomic_embed_text 10.11
Embeddings:
[ 2.41241474e-02 -4.50897263e-03 -2.15209976e-01 -4.53796424e-02
1.46770489e-03 4.48608436e-02 -2.05802941e-03 1.30081186e-02
-2.21405048e-02 -6.17675819e-02 1.77621914e-05 7.71484450e-02
5.78918494e-02 5.52978553e-02 2.69470233e-02 -4.53491248e-02
3.90319861e-02 -8.25195387e-02 -5.13305701e-02 1.16806040e-02
-2.47650165e-02 -5.70983924e-02 6.58798264e-03 -6.29806565e-03
2.16308609e-01 9.33837984e-03 -3.27453651e-02 1.02417000e-01
2.17590351e-02 3.63349938e-03 1.28631601e-02 -1.32064829e-02
1.50604257e-02 2.56042499e-02 6.51855543e-02 -3.80249061e-02
1.70040142e-03 3.19480919e-03 1.28250131e-02 1.43127451e-02
2.92968773e-03 -4.25720261e-03 8.76617525e-03 -5.45043983e-02
7.24487379e-02 -1.21612558e-02 -1.37939462e-02 -1.17721567e-02
6.06994666e-02 -1.56402607e-02 -3.29895057e-02 -1.77764904e-03
-3.09295673e-02 3.49731483e-02 3.03497333e-02 5.01403846e-02
6.08215369e-02 2.45971698e-02 1.79595966e-02 1.87683124e-02
6.84204176e-02 7.29980543e-02 2.01873798e-02 6.91528395e-02
1.96990985e-02 -3.41796912e-02 -7.67231046e-04 8.52050856e-02
5.60913123e-02 -1.63269062e-02 7.55615309e-02 -2.16979999e-02
4.34875526e-02 1.93786640e-02 6.58798264e-03 1.62200946e-02
2.09045429e-02 -3.43017615e-02 -4.82482947e-02 -7.37915114e-02
5.18493690e-02 -9.25903395e-02 1.05407722e-01 -4.20837440e-02
1.24588022e-02 -4.14123572e-02 -1.32904062e-02 7.17163133e-03
-4.33959998e-02 3.15551795e-02 6.09130897e-02 2.07519550e-02
-2.68402118e-02 4.03747596e-02 -4.18090858e-02 1.57775898e-02
-4.78744553e-03 -6.70623826e-03 -4.66308631e-02 -3.60717811e-02
-9.71222017e-03 4.33349656e-03 -2.19726581e-02 2.30560321e-02
7.20214918e-02 4.43115272e-02 -4.73022498e-02 -7.61718825e-02
-4.11987342e-02 -4.75769080e-02 -3.17077674e-02 3.72924842e-02
-7.02667283e-03 2.29034442e-02 -1.76544208e-02 5.59234666e-03
3.74450721e-02 -8.04443434e-02 3.09295673e-02 7.23877028e-02
-8.33129957e-02 2.06451435e-02 8.11576901e-04 4.22363319e-02
1.62200946e-02 4.13513221e-02 -8.30688551e-02 -3.15246619e-02
2.56729149e-03 -3.37829627e-02 -2.52990741e-02 -3.06549091e-02
-1.01547251e-02 -4.36706580e-02 4.28466834e-02 9.04083345e-03
-2.00805683e-02 -3.41491736e-02 4.34875526e-02 1.84631366e-02
3.17993201e-02 2.14386005e-02 1.50070200e-02 -5.69458045e-02
-4.06494178e-02 -2.87017841e-02 6.92749098e-02 -3.66516151e-02
8.38756619e-04 -4.78744553e-03 -1.58081073e-02 2.99072284e-02
2.27508564e-02 3.61633338e-02 1.66473407e-02 -3.07617206e-02
-5.39855994e-02 -9.80377290e-03 5.27343787e-02 -1.64947528e-02
2.93540978e-03 -4.39453162e-02 -8.68530348e-02 1.49841318e-02
7.54928635e-03 -9.79003981e-02 4.23812913e-03 3.79333533e-02
4.14428748e-02 2.45056171e-02 8.48388765e-03 -6.26831129e-02
-3.14636268e-02 9.82284662e-04 -2.68707294e-02 -4.19921912e-02
-1.14517221e-02 -6.64062575e-02 3.82385291e-02 7.20977830e-03
7.65991285e-02 -5.41992225e-02 2.29949970e-02 3.59191932e-02
-2.84881610e-02 -3.38439979e-02 1.17034921e-02 9.36126802e-03
-4.04663123e-02 -3.29284705e-02 -6.25610426e-02 1.85699482e-02
-8.16650465e-02 -2.84881610e-02 -3.53088416e-02 -3.37829627e-02
1.52816782e-02 6.61850022e-03 5.45043983e-02 -5.33142127e-02
-3.93066444e-02 -1.22985849e-02 -1.91650409e-02 -7.98034761e-03
7.43389188e-04 9.42993239e-02 4.79126014e-02 2.72216816e-02
5.44128455e-02 3.80859412e-02 1.03332527e-01 -6.76269606e-02
-3.86047401e-02 -4.93774451e-02 5.10788010e-03 5.16414701e-04
3.11431903e-02 -4.71801795e-02 2.41699237e-02 2.67791767e-02
1.48315439e-02 -3.46374549e-02 5.94177283e-02 -1.06201181e-02
6.31713942e-02 -2.00958271e-02 -3.12347431e-02 1.44042978e-02
-1.09252937e-01 2.44712853e-03 -5.78308143e-02 -2.55584735e-02
3.06243915e-02 2.87017841e-02 1.12235553e-04 6.24465989e-03
2.92663593e-02 4.35180701e-02 -2.05383319e-02 -6.28051832e-02
-5.83190955e-02 7.96508864e-02 5.04760779e-02 6.17218064e-03
-8.22753981e-02 3.06701679e-02 -4.98352088e-02 1.56021127e-02
1.25732431e-02 -1.12838754e-02 -3.18603553e-02 -2.19268817e-02
-1.43661508e-02 3.89404334e-02 4.02221717e-02 -3.65600623e-02
-5.61523475e-02 1.40838632e-02 -3.44238319e-02 -3.15856971e-02
4.85534705e-02 -1.01867683e-01 4.33654822e-02 -5.83190955e-02
-4.94995154e-02 -1.06124887e-02 -6.00891151e-02 1.51214609e-02
6.64672926e-02 -6.98852614e-02 -3.58772301e-03 2.17895526e-02
3.07617206e-02 3.03955097e-02 -1.13906870e-02 -7.06481980e-03
2.38037133e-03 9.26208589e-03 -4.83398475e-02 9.26971529e-03
-3.26232947e-02 -1.14822397e-02 -8.05664137e-02 -4.54406776e-02
-1.70135517e-02 4.29992713e-02 5.15136756e-02 3.87268104e-02
-2.23083515e-02 -2.96630878e-02 3.32946815e-02 1.15509043e-02
6.46972703e-03 1.03027351e-01 -5.15136756e-02 1.44805918e-02
6.10961951e-02 1.43203745e-02 -7.97119215e-02 -1.64337177e-02
-3.32641639e-02 2.47497577e-02 3.29284705e-02 -4.26025428e-02
-2.39105243e-02 1.45950327e-02 1.55792246e-02 -4.04357947e-02
7.18383864e-02 2.07824726e-02 -8.57544020e-02 6.94580153e-02
-9.28344801e-02 4.58908128e-03 -5.43823279e-02 2.66723651e-02
-1.87835712e-02 5.48095740e-02 9.75341871e-02 2.11181659e-02
-4.49218787e-02 -5.65490760e-02 -5.21545447e-02 -6.97631910e-02
3.87878455e-02 1.12686167e-02 3.94897498e-02 5.50537147e-02
2.33612079e-02 -4.33959998e-02 2.44903583e-02 1.21383676e-02
-4.91638221e-02 -2.73742694e-02 -4.21752967e-02 3.28674354e-02
-1.45111093e-02 -3.08837909e-02 -1.08566293e-02 8.66088942e-02
2.14996357e-02 -4.25720261e-03 5.08308469e-04 -4.74548377e-02
-2.05230732e-02 -5.14221229e-02 -8.78906343e-03 -5.35965012e-03
-1.34201059e-02 1.50451669e-02 -1.49536142e-02 4.24499549e-02
-2.27355976e-02 3.18298377e-02 -5.84125577e-04 -3.09143085e-02
7.52258347e-03 6.65283278e-02 2.72979755e-02 -2.58026142e-02
2.94303917e-03 3.52859526e-04 3.31420936e-02 9.76562593e-03
-4.97131385e-02 2.02178978e-03 3.74450721e-02 2.48146080e-03
3.79333533e-02 8.08715913e-03 -3.27148475e-02 -2.69317646e-02
-2.42157001e-02 4.19006385e-02 2.75268573e-02 4.73632850e-02
-1.00097664e-01 -6.15234412e-02 1.02996835e-02 -2.12249774e-02
-4.45251502e-02 -6.92138746e-02 4.54406776e-02 -2.92358417e-02
2.97927880e-03 6.73828200e-02 4.59899940e-02 2.00805683e-02
1.33285532e-02 1.31225598e-03 -7.61032151e-03 -8.88061617e-03
2.07061786e-02 1.19972241e-03 -1.21154794e-02 1.90429706e-02
5.27343787e-02 2.87017841e-02 -3.32031287e-02 1.87835712e-02
-2.16217060e-02 -2.12478661e-03 -8.14056490e-03 -1.14501961e-01
1.14440927e-02 3.71933007e-03 -2.95410175e-02 -2.03399677e-02
3.63159217e-02 -1.47781381e-02 4.47082557e-02 -1.11846933e-02
1.22222910e-02 -3.87878455e-02 -7.10296677e-03 2.98004169e-02
3.20129432e-02 -6.11572303e-02 -3.82995643e-02 1.98974628e-02
8.17871168e-02 6.12487830e-02 3.95202674e-02 3.47900428e-02
-3.25927772e-02 9.42993239e-02 5.86547889e-02 5.72814979e-02
1.37329111e-02 -9.53979567e-02 -1.60675067e-02 4.14123572e-02
2.33764667e-02 3.53698768e-02 -4.33044471e-02 -3.56750526e-02
-5.64098416e-04 -2.61383075e-02 1.24282846e-02 -3.62586998e-03
4.86450233e-02 -3.13415565e-02 -5.13610877e-02 -1.89971942e-02
1.85394306e-02 1.38549820e-01 1.28051773e-01 -5.66711463e-02
-4.99572791e-02 -5.78689622e-03 -1.12075815e-02 5.44433631e-02
5.10864295e-02 1.31454477e-02 1.12670906e-01 -8.23974684e-02
5.42907752e-02 -1.43814096e-02 -6.16455125e-03 2.03704853e-02
4.37622108e-02 4.82177772e-02 -4.18701209e-02 -1.38397226e-02
1.66168232e-02 -8.55712965e-02 1.35803232e-02 -2.05383319e-02
2.05688495e-02 2.71606464e-02 -3.46984901e-02 -3.33557166e-02
3.99475135e-02 -5.79528846e-02 -6.28051832e-02 2.44750995e-02
-3.88717675e-03 2.40898156e-03 -6.31332444e-03 2.38952655e-02
1.12915048e-02 2.56347680e-03 -5.83190955e-02 -4.41894569e-02
1.88140888e-02 5.84411658e-02 2.09198017e-02 4.80041541e-02
-9.94873140e-03 5.40771522e-02 9.94110201e-03 -6.61850022e-03
5.13305701e-02 2.78015155e-02 -3.58276404e-02 8.48388765e-03
-3.97338904e-02 2.51159687e-02 -2.02331562e-02 -2.45513935e-02
1.50146494e-02 1.08795175e-02 -1.56173715e-02 3.21655311e-02
7.26699876e-03 -1.08947763e-02 5.22460975e-02 -1.05224617e-01
-2.36206073e-02 2.08435077e-02 -5.32531776e-02 4.02221717e-02]
0.0ms [WARNING] <W> This META does not have Alloc2 Support
0.0ms [WARNING] <W> This META does not have Alloc2 Support
0.0ms [WARNING] <W> This META does not have Alloc2 Support
0.0ms [WARNING] <W> This META does not have Alloc2 Support
/prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc
468.5ms [WARNING] Time: model_destroy nomic_embed_text 120.04