How to Setup Ollama gpt-oss with Open WebUI

What is Ollama?

Ollama is a local AI model runner that allows you to download, run, and interact with the LLMs (Large Language Models) entirely on your computer. It is ideal for offline use, ensures data privacy, and offers full control over your AI environment, etc. Recently, Ollama partnered with OpenAI to bring its latest state-of-the-art open weight models to the platform. Two models, i.e., 20b and 120b, which are designed for very powerful reasoning, agentic tasks, and versatile developer use cases. In this article, we will download and configure gpt-oss:20b, which is approximately 14 GB in size.

Let’s go to the official Ollama website at https://ollama.com/download to download Ollama. I am using Windows OS, so I will download Ollama for the Windows version.

Once downloaded, run the setup to install the Ollama. Then, click on the Install button.

It will take a few minutes, depending on your system configuration.

Now, go to URL https://ollama.com/search and search for the gpt-oss LLM. 

Copy and run “ollama run gpt-oss:20b” to start and interact with gpt-oss modal locally through the Ollama. Ollama will pull the gpt-oss model files to your computer, and this process may take time depending on your internet speed.

Once downloaded successfully, Ollama loads the gpt-oss model into memory and starts an interactive mode in the terminal.

Now let’s set up the Open WebUI, an extensible, feature-rich, and user-friendly self-hosted AI platform designed to operate entirely offline. It supports various LLM runners like Ollama and is compatible with OpenAPI-compatible APIs. There are multiple ways to set up the Open WebUI, which you can find at https://github.com/open-webui/open-webui. I will be using Docker for the installation. If you don’t have Docker installed, you can download Docker Desktop from the official Docker website.

Once downloaded, run the Docker Desktop setup and follow the steps as shown in the images below.

Click on the Close and restart button to reboot your Windows system.

Open Docker Desktop and accept the terms and conditions.

After accepting the terms, Docker Desktop will be ready to use.

Now, execute the below Docker command in the terminal. This one-line command sets up Open WebUI in a container that is already linked to your system, allowing it to communicate with Ollama.

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

After successful execution of the above command, you will see that your container is up and running (in my case, it is running at localhost on port 3000).

Now, access the URL http://localhost:3000. You will be prompted to create an admin account. Provide your details and click on ‘Create Admin account’.

Now, you can see that my model, i.e., gpt-oss:20b, is displayed and ready for use. 

The Open WebUI interface is a user-friendly interface and resembles the “ChatGPT” interface, but it runs entirely on your computer or machine, fully under your control. You can also pull multiple models like LLaMa, Mistral, etc., and can switch easily between them.

I hope you enjoyed this article and found the setup guide for Ollama GPT-OSS with Open WebUI helpful.