Deploy AI Large Models on CasaOS
Install radxa One-Click Start Large Model Apps
Here, we'll explain how to install radxa large model Apps on CasaOS. Third-party Apps in CasaOS are installed in the form of Docker images, uploaded to the CasaOS App store refer here. Currently, radxa has uploaded four large model Apps (Stable Diffusion, whisper, chatglm2 chatdoc, imagesearch, llama3 chatbot) to docker hub. For more detailed information, please refer to App construction methods.
| App | Docker Image | Container port |
|---|---|---|
| Radxa Stable Diffusion | radxazifeng278/radxa_sd_app:0.3.0 | 8999 |
| Radxa whisper | radxazifeng278/radxa_whisper_app:0.1.0 | 7860 |
| Radxa ImageSearch | radxazifeng278/radxa_imgsearch_app:0.1.0 | 8501 |
| Radxa chatdoc | radxazifeng278/radxa_chatdoc_app:0.1.0 | 8501 |
| Radxa Llama3 chatbot | radxazifeng278/radxa_llama3_app:0.1.0 | 8003 |
- Click the "+" icon in the CasaOS App column and select "Install a customized app"
Install radxa Stable Diffusion App
Stable Diffusion Factory Pre-Installed
- Fill in the required information for installing radxa stable diffusion container
-
Information Details
Docker Image: Name of the image, fill in to automatically pull the image from docker hub
radxazifeng278/radxa_sd_app:0.3.0Title: App name, user-defined
Icon URL: App icon (optional)
Web UI port: Specify the default access port when accessing the App, users can choose from 7000-10000, ensuring it does not conflict with other CasaOS APP ports
Port: Docker container port mapping, Host port is the port specified in Web UI, container port is 8999
8999Devices: Device mounting, mount TPU into the Docker container, all mounts are recommended
- Host:
/dev - Container:
/dev
Privileges: Enable to obtain maximum permissions
Memory limit: Container memory limit, recommended to fill, otherwise the service may fail to start
CPU Shares: CPU capability, Medium recommended
radxa Stable Diffusion Docker Image radxazifeng278/radxa_sd_app:0.3.0 Title stable diffusion Web UI port 9001 Host Port 9001 Container Port 8999 Host Devices /dev Container Devices /dev CPU Shares Medium - Host:
Install radxa Whisper Speech Recognition Summary App
-
Fill in the required information for installing radxa whisper container
radxa Whisper Docker Image radxazifeng278/radxa_whisper_app:0.1.0 Title whisper Web UI port 8004 Host Port 8004 Container Port 7860 Host Devices /dev Container Devices /dev CPU Shares Medium
Install radxa ImageSearch App
-
Fill in the required information for installing radxa ImageSearch container
radxa ImageSearch Docker Image radxazifeng278/radxa_imgsearch_app:0.1.0 Title Image_Search Web UI port 8007 Host Port 8007 Container Port 8501 Host Devices /dev Container Devices /dev CPU Shares Medium
Install radxa ChatGLM2 Chatdoc Document Chat App
Before installing chatdoc, it is necessary to refer to Memory Allocation Modification Tool to modify the memory allocation of SG2300X
Recommended allocation -NPU 7168, -VPU 2048, -VPP 3072
-
Fill in the required information for installing radxa chatdoc container
radxa Chatdoc Docker Image radxazifeng278/radxa_chatdoc_app:0.1.0 Title chatdoc Web UI port 8009 Host Port 8009 Container Port 8501 Host Devices /dev Container Devices /dev CPU Shares Medium
Install radxa Llama3 Chatbot GPT App
Llama3 Factory Pre-Installed
When starting the Llama3 Chatbot, please close other TPU-related apps to avoid underlying incompatibility issues.
-
Fill in the required information for installing the radxa Llama3 Chatbot container
Radxa Llama3 Chatbot Docker Image radxazifeng278/radxa_llama3_app:0.1.0 Title llama3 chatbot Web UI port 8008 Host Port 8008 Container Port 8003 Host Devices /dev Container Devices /dev CPU Shares Medium
App Startup and Uninstallation
In CasaOS, the App is highlighted when it is open, and it appears gray when it is closed.
To start the App, simply click on the icon, or you can start it by selecting the icon in the upper right corner of the selected App.
To uninstall the App, simply click Uninstall.
