How to Setup Ollama gpt-oss with Open WebUI

What is Ollama?

Ollama is a local AI model runner that allows you to download, run, and interact with the LLMs (Large Language Models) entirely on your computer. It is ideal for offline use, ensures data privacy, and offers full control over your AI environment, etc. Recently, Ollama partnered with OpenAI to bring its latest state-of-the-art open weight models to the platform. Two models, i.e., 20b and 120b, which are designed for very powerful reasoning, agentic tasks, and versatile developer use cases. In this article, we will download and configure gpt-oss:20b, which is approximately 14 GB in size.

Let’s go to the official Ollama website at https://ollama.com/download to download Ollama. I am using Windows OS, so I will download Ollama for the Windows version.

Download Ollama

Once downloaded, run the setup to install the Ollama. Then, click on the Install button.

Ollama

It will take a few minutes, depending on your system configuration.

Installing Ollama

Now, go to URL https://ollama.com/search and search for the gpt-oss LLM.

gpt-oss 20b

Copy and run “ollama run gpt-oss:20b” to start and interact with gpt-oss modal locally through the Ollama. Ollama will pull the gpt-oss model files to your computer, and this process may take time depending on your internet speed.

Ollama run gpt-oss 20b

Once downloaded successfully, Ollama loads the gpt-oss model into memory and starts an interactive mode in the terminal.

Meaning of LLM

Now let’s set up the Open WebUI, an extensible, feature-rich, and user-friendly self-hosted AI platform designed to operate entirely offline. It supports various LLM runners like Ollama and is compatible with OpenAPI-compatible APIs. There are multiple ways to set up the Open WebUI, which you can find at https://github.com/open-webui/open-webui. I will be using Docker for the installation. If you don’t have Docker installed, you can download Docker Desktop from the official Docker website.

Learn about Docker for AI

Once downloaded, run the Docker Desktop setup and follow the steps as shown in the images below.

Docker Desktop

Click on the Close and restart button to reboot your Windows system.

Docker Desktop

Open Docker Desktop and accept the terms and conditions.

Docker Subscription Service Agreement

After accepting the terms, Docker Desktop will be ready to use.

Docker Desktop

Now, execute the below Docker command in the terminal. This one-line command sets up Open WebUI in a container that is already linked to your system, allowing it to communicate with Ollama.

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

Docker Command

After successful execution of the above command, you will see that your container is up and running (in my case, it is running at localhost on port 3000).

Docker Desktop execution of the command

Now, access the URL http://localhost:3000. You will be prompted to create an admin account. Provide your details and click on ‘Create Admin account’.

Open WebUI

Now, you can see that my model, i.e., gpt-oss:20b, is displayed and ready for use.

gpt-oss 20b

The Open WebUI interface is a user-friendly interface and resembles the “ChatGPT” interface, but it runs entirely on your computer or machine, fully under your control. You can also pull multiple models like LLaMa, Mistral, etc., and can switch easily between them.

gpt-oss 20b

I hope you enjoyed this article and found the setup guide for Ollama GPT-OSS with Open WebUI helpful.