Host your own AI with Unraid: an open source alternative to ChatGPT
Are you tired of relying on proprietary AI tools from tech giants like Google and OpenAI? Do you want to explore the possibilities of open-source alternatives that give you full control over your data? In our new video, we'll take you through the process of setting up a Generative AI chatbot on an UnRAID server, using a few popular open-source tools: Ollama, Open WebUI, and Stable Diffusion (with Automatic1111).
For those who don’t know, Unraid is a Linux-based operating system designed for home servers. Its approachable user interface and strong community support make it an ideal platform for hosting various applications, including AI-powered ones. Keep an eye out for another one of our videos coming soon on how to set up your own server with Unraid!
Installing the Apps
Our first step once we’re logged into our Unraid interface will be to download all the relevant apps. As you go through the process, make sure to carefully read the instructions for anything you install. We’ll try to highlight the important dependencies and setup steps for the apps we’re working with today, but it's a good habit to get into and may hold valuable information for troubleshooting. Similarly, it's very important that you keep a document recording all the steps you take along with any important information like the port numbers/addresses for your various apps for you to reference.
With that out of the way, let's look at the specific apps for today:
Ollama: An open-source platform for hosting chat-bots on your own machine.
Special instructions: Requires “Nvidia Driver” Plugin available in the same App store. Note that this driver plugin prevents passing your video card to a VM which could be a problem for certain use cases like setting up a VM for gaming
Open WebUI: A self-hosted webUI that is compatible with Ollama and Stable Diffusion. While Ollama will host and manage the models, Open WebUI is where you’ll be able to access and interact with those models
Special Instructions: The default port # is 8080 which we recommend changing to avoid conflict as its an extremely common port #. During setup you’ll also need to input the “OLLAMA_BASE_URL” which is whatever URL you would use to access Ollama on your server, generally [server name]:[port number for app].
Automatic1111: A popular webUI for Stable Diffusion. Allows you to generate images from text and other images with a huge degree of customization and control.
Special Instructions: Because of the size of the Docker image for Automatic1111, it gives a warning that it may cause “no space left on device” error during installation. If you encounter this error, you’ll need to increase the vDisk size for Docker itself. You’ll need to completely stop Docker in order to change this setting.
App Setup
Once everything is installed, you’ll have just a little bit of setup to do within Open WebUI and Automatic1111 before you’re ready to start interacting with your AI chat-bot!
Open WebUI:
All of the settings we need in Open WebUI can be found in the Admin Panel which you can access from the profile icons in either the top right or bottom left of the window. Once there you’ll need to navigate to from the dashboard to the settings tab at the top, and then to the connections tab where you should see a setting called “Ollama API” with a toggle and a text box. If you don’t see the text box you’ll need to turn the toggle on and then you can paste the same “OLLAMA_BASE_URL” from earlier into the text box. Save your settings and your Open WebUI is now connected to Ollama!
Now all that’s left is to download some models for you to use. Navigate to the models tab on the left where you should see your Ollama URL already populated in the “Manage Ollama Settings” box. Below that you should see a “Pull a model from Ollama.com” box which is where we’ll do the actual downloading of models. First you'll want to check out the link to Ollama’s website where you can find a list of all available models. Then it's as simple as typing the model name into the text box and hitting the download button on the right!
We recommend starting with llama3.1, a powerful LLM (large language model) great for analyzing data and answering questions, and llava, a Multimodal Model that has the ability to analyze images which will be especially useful to us later. Once downloaded you can start chatting right away by clicking “new chat” and selecting your desired model from the dropdown in the top left.
Automatic1111:
Stable Diffusion has tons of settings that you can mess with to greatly change the images it outputs, but luckily it’s pretty simple to get started. Unlike OpenWebUI where you can use Ollama to download models directly through the App, we’ll have to manually find and download models into our Automatic1111 models folder. We recommend doing this via command line, but if you’re not comfortable messing with command line you can also download the model normally and upload the file in the Unraid UI. If you want to do it through the UI you’ll need to install the plugin “Dynamix File Manager” which adds extra functionality to the file explorer in the shares tab.
You can find models for download on Hugging Face and other platforms. You can find official models from stability AI or you can browse the huge amount of community-trained models that are often specialized for specific types of images or particular styles. Note that Stable Diffusion will cache the model you’re using into your vram so to avoid errors make sure you pick a model that is appropriately sized for your hardware (for example a 3gb model with passively take up 3gb of your video card’s vram for as long as Automatic1111 is running). While searching you’ll see a lot of files in the repositories that could be useful for a more advanced user, but for now you’ll just need the safetensors file, just go for the simplest named one if the model has multiple safetensors available.
Once you’ve chosen a model, we can go ahead and download it into the appropriate folder with a curl command. To start off we’ll need to open the Unraid terminal by clicking “>_” icon in the top right corner and navigating to our stable diffusion folder with a cd command. If you’re unsure of the folder location, you can find it through the shares tab in your ui and once you’ve found it you can right click the file path at the top to get a copyable version of the path to paste into your cd command. Your Automatic1111 folder is likely inside your app data folder and from within Automatic 1111 the path will be data/models/Stable-diffusion
Once you’re certain that you’re in the right folder you can use the following curl command. For this example we’re downloading a model called Dreamshaper 8:
curl -L (handle redirects) -o (define output) [desired output file name] “[download link url]”
Which in this case would look like:
curl -L -o dreamshaper_8.safetensors “https://huggingface.co/Lykon/DreamShaper/resolve/main/DreamShaper_8_pruned.safetensors?download=true”
If all goes well, you should see your download progress in the terminal window and once it’s complete you should be able to refresh your Unraid window and see the model file in the chosen folder! Now that we have a model, we’re ready to go into our Automatic1111 UI using the link we saved when we first installed it (again this should be the server name followed by : and the app port number). Once inside the UI you’ll see a drop-down menu labeled “Stable Diffusion checkpoint” with a refresh button next to it. You’ll need to first hit the refresh button to get Automatic1111 to recognize your new model and then select the model from the drop-down. You’re now ready to start generating images!
Combining Automatic1111 with Open WebUI
The last step for setup will be connecting Automatic1111 to Open WebUI so you can generate images directly from conversation with your chat-bots. You’ll need to navigate back to the Admin Panel settings inside of your Open WebUI and click into the Images tab. After you toggle the image feature on you’ll need to paste the url you use to access Automatic1111 into the “AUTOMATIC1111 Base URL” field. Finally, you’ll need to choose a default model by copying either the model ID or full model name from your Automatic1111 UI and pasting it into the “Set Default Model” field back in your Open WebUI settings. You’re now ready to start generating images inside any of your chats! You can also edit the size and number of steps taken to create the image but the default values are fine.
This feature isn’t yet as mature as the integration of DALL-E into ChatGPT, so you unfortunately wont be able to directly ask for images. Instead you can find a “generate image” button at the bottom of any response that your bots give you and it will create an image based on the response. As a workaround you can ask the bot to create detailed descriptions of the thing you want to make an image of, or you can even ask it to make a prompt for Stable Diffusion. For image-to-image you’ll need to use a multi-modal model like llava to analyze and describe your base image and then have it change that description to your specifications.
Thanks for checking out our Unraid AI tutorial!
We’ve covered everything you need to know to start building your own AI tools hosted on your own server. If you found this useful or interesting, definitely follow our socials for more content like this. Next, we’ll be posting about setting up Unraid itself, with more content digging deeper into these tools on the horizon!
