Setup Ollama with Open WebUI With Docker in Local Windows

What is Ollama?

Ollama is an open-source platform that allows users to run large language models (LLMs) locally on their own machines, such as laptops or desktops. This local execution offers privacy and data control advantages compared to cloud-based AI solutions. Ollama supports multiple operating systems, including macOS, Linux, and Windows (in preview), and provides a command-line interface (CLI) for managing and interacting with AI models like Llama 3 and others. Users can download, run, customize, and fine-tune these models offline, which helps reduce latency and reliance on external servers while protecting sensitive data.

Key features of Ollama include

Local execution of LLMs, enhancing security and privacy.
Access to a library of pre-trained models, including popular ones for natural language processing, code generation, and more.
Compatibility with various platforms and GPU support for efficient performance.
Integration capabilities with programming languages such as Python.
Tools for managing models through commands like downloading, running, copying, and removing models.

Developers, researchers, and businesses use Ollama for tasks such as creative writing, coding assistance, language translation, customer support chatbots, and healthcare applications.

Key Features

Local Model Execution: Run LLMs such as Llama 3 entirely on your own hardware.
Pre-Trained Model Library: Access a collection of ready-to-use models for.
- Natural Language Processing (NLP)
- Code Generation
- Translation
- Creative Writing
- Chatbots & More
Command-Line Interface (CLI): Manage models with simple commands.
- Download models
- Run models
- Copy or Remove models
Programming Integration: Integrates with programming languages like Python for seamless workflow automation.

Steps to Set Up Ollama.

1. Go to Ollama Website -> Ollama Website

2. Click on the download button as shown below.

Download

3. Select Windows and click on the download button. Click here

Window

4. Once you click on download, the .exe file will be downloaded as shown below.

llama

Double-click on the .exe file below to open a pop-up, then click the install button. Once installation is completed, go to the models tab and pick one model.

Model

Goto Model Tab (Click here).

Click on any model. For now, I am using the llama model. Now copy its run command.

LLama Model

Paste the Run command into the Command Prompt and execute it.

ollama run llama3.1:8b

Run Command

Now you can see Llama 3.1:8b is installed on the local machine.

Local machine

Docker

Go to the Docker website (Docker Website), click on Download Docker Desktop.
Now, install Docker Desktop and restart your computer.
Complete the account and create a Docker account.

What is Open WebUI? (Open WebUI Website)

Open WebUI is an open-source, web-based user interface designed to make interacting with large language models (LLMs) easier and more intuitive.

It’s often used alongside Ollama or other local/remote LLM backends, providing a chat-like interface in your browser rather than relying solely on the command line.

Key Points About Open WebUI

Provides a modern, interactive chat interface for LLMs.
Eliminates the need for complex CLI commands when running models locally.
Model Management
- Select from multiple installed LLMs.
- Switch models without restarting the backend.
UI Customization: Change themes, layouts, and other interface settings.

Steps to Run OpenWebUI

Now, run Docker if it is not running.
Copy the command for ollama that is in your local directory.

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

Execute the above command in the Command Prompt.
Now click on port(s) -> 3000:8080. Once you click on it, the UI will be opened.
After successful signing in, you will be able to see the screen below. From there, you can select any model you have installed and start using it.

You can change your installed model from the left side, as shown below.

Code

You can also use Ollama models for any project based on your requirements. Also, an Agentic AI framework like Autogen gives native support with Ollama.

Conclusion

Ollama is the engine that runs the models locally, while Open WebUI is the dashboard/chat interface that makes interacting with those models more user-friendly.