Video Search-TPU

VideoSearch-TPU is a text-based video search application that uses the Image CLIP model, ported to SG2300X chip series products using the Sophon SDK for local TPU hardware-accelerated inference. This application enables quick searching of video content using text and returns the corresponding frames and positions of the search results.

Clone the repository:

git clone https://github.com/zifeng-radxa/VideoSearch-tpu.git

Download CLIP bmodels and copy the bmodels folder to VideoSearch-tpu/inference/clip_model/.

cd VideoSearch-tpu/inference/clip_model/
rm -rf ./bmodels
wget https://github.com/radxa-edge/TPU-Edge-AI/releases/download/ImageSearch/ImageSearch_bmodel.zip
unzip ImageSearch_bmodel.zip
mv ImageSearch_bmodel/bmodels .

The file structure will be as follows:

.
├── dbs
│   ├── CH
│   └── EN
├── inference
│   ├── clip
│   ├── clip_model
│   │   ├── ImageSearch_bmodel
│   │   │   └── bmodels
│   │   │       ├── CH
│   │   │       └── EN
│   │   └── saved_tokenizer
│   │       ├── bert_chinese_tokenizer-fast
│   │       └── bert_chinese_tokenizer-slow
│   └── utils
├── scene_snapshot
└── video_collection

Create a virtual environment:

It's necessary to create a virtual environment; otherwise, it may affect the normal operation of other applications. For virtual environment usage, please refer here.
```
cd VideoSearch-tpu
python3 -m virtualenv .venv
source .venv/bin/activate
```

Install dependencies:

pip3 install --upgrade pip
pip3 install -r requirements.txt
pip3 install https://github.com/radxa-edge/TPU-Edge-AI/releases/download/v0.1.0/tpu_perf-1.2.31-py3-none-manylinux2014_aarch64.whl

Import environment variables:

VideoSearch does not support sophon-opencv. If sophon-opencv is set in the environment variables, please unset it.
```
export LD_LIBRARY_PATH=/opt/sophon/libsophon-current/lib:$LD_LIBRARY_PATH
```
Start the web service:
```
streamlit run app.py EN
```
Access port 8501 of the Airbox IP address in a web browser.

Video Search-TPU

Application Display​

Application Display