Skip to main content

Deploy AI Large Models on CasaOS

Install radxa One-Click Start Large Model Apps​

Here, we'll explain how to install radxa large model Apps on CasaOS. Third-party Apps in CasaOS are installed in the form of Docker images, uploaded to the CasaOS App store refer here. Currently, radxa has uploaded four large model Apps (Stable Diffusion, whisper, chatglm2 chatdoc, imagesearch, llama3 chatbot) to docker hub. For more detailed information, please refer to App construction methods.

AppDocker ImageContainer port
Radxa Stable Diffusionradxazifeng278/radxa_sd_app:0.3.08999
Radxa whisperradxazifeng278/radxa_whisper_app:0.1.07860
Radxa ImageSearchradxazifeng278/radxa_imgsearch_app:0.1.08501
Radxa chatdocradxazifeng278/radxa_chatdoc_app:0.1.08501
Radxa Llama3 chatbotradxazifeng278/radxa_llama3_app:0.1.08003
  • Click the "+" icon in the CasaOS App column and select "Install a customized app"

casaos_2.webp

Install radxa Stable Diffusion App​

tip

Stable Diffusion Factory Pre-Installed

  • Fill in the required information for installing radxa stable diffusion container

  • Information Details

    Docker Image: Name of the image, fill in to automatically pull the image from docker hub

    radxazifeng278/radxa_sd_app:0.3.0

    Title: App name, user-defined

    Icon URL: App icon (optional)

    Web UI port: Specify the default access port when accessing the App, users can choose from 7000-10000, ensuring it does not conflict with other CasaOS APP ports

    Port: Docker container port mapping, Host port is the port specified in Web UI, container port is 8999

    8999

    Devices: Device mounting, mount TPU into the Docker container, all mounts are recommended

    • Host: 
      /dev
    • Container: 
      /dev

    Privileges: Enable to obtain maximum permissions

    Memory limit: Container memory limit, recommended to fill, otherwise the service may fail to start

    CPU Shares: CPU capability, Medium recommended

    radxa Stable Diffusion
    Docker Imageradxazifeng278/radxa_sd_app:0.3.0
    Titlestable diffusion
    Web UI port9001
    Host Port9001
    Container Port8999
    Host Devices/dev
    Container Devices/dev
    CPU SharesMedium

Install radxa Whisper Speech Recognition Summary App​

  • Fill in the required information for installing radxa whisper container

    radxa Whisper
    Docker Imageradxazifeng278/radxa_whisper_app:0.1.0
    Titlewhisper
    Web UI port8004
    Host Port8004
    Container Port7860
    Host Devices/dev
    Container Devices/dev
    CPU SharesMedium

Install radxa ImageSearch App​

  • Fill in the required information for installing radxa ImageSearch container

    radxa ImageSearch
    Docker Imageradxazifeng278/radxa_imgsearch_app:0.1.0
    TitleImage_Search
    Web UI port8007
    Host Port8007
    Container Port8501
    Host Devices/dev
    Container Devices/dev
    CPU SharesMedium

Install radxa ChatGLM2 Chatdoc Document Chat App​

tip

Before installing chatdoc, it is necessary to refer to Memory Allocation Modification Tool to modify the memory allocation of SG2300X

Recommended allocation -NPU 7168, -VPU 2048, -VPP 3072

  • Fill in the required information for installing radxa chatdoc container

    radxa Chatdoc
    Docker Imageradxazifeng278/radxa_chatdoc_app:0.1.0
    Titlechatdoc
    Web UI port8009
    Host Port8009
    Container Port8501
    Host Devices/dev
    Container Devices/dev
    CPU SharesMedium

Install radxa Llama3 Chatbot GPT App​

tip

Llama3 Factory Pre-Installed

When starting the Llama3 Chatbot, please close other TPU-related apps to avoid underlying incompatibility issues.

  • Fill in the required information for installing the radxa Llama3 Chatbot container

    Radxa Llama3 Chatbot
    Docker Imageradxazifeng278/radxa_llama3_app:0.1.0
    Titlellama3 chatbot
    Web UI port8008
    Host Port8008
    Container Port8003
    Host Devices/dev
    Container Devices/dev
    CPU SharesMedium

App Startup and Uninstallation​

In CasaOS, the App is highlighted when it is open, and it appears gray when it is closed.

To start the App, simply click on the icon, or you can start it by selecting the icon in the upper right corner of the selected App.

To uninstall the App, simply click Uninstall.

casaos_3.webp