LifeBytes Journal

Running an AI chatbot locally is no longer a “lab-only” project. With today’s lightweight models and tools like Ollama and Open WebUI, you can build a private, fast, and surprisingly capable assistant on a Windows PC—without sending prompts to third-party cloud services. This tutorial walks you through a practical setup that works well for IT notes, scripting help, documentation drafts, and troubleshooting ideas, all while keeping your data on your own machine.

This guide focuses on an up-to-date approach: Ollama provides an easy local model runtime, and Open WebUI gives you a clean web interface with chat history, model selection, and basic admin options. The result feels like a polished “ChatGPT-style” experience, but running on your own hardware.

Prerequisites

Before you start, confirm you have the following:

1) A Windows 10/11 system (64-bit). 2) At least 16 GB RAM recommended (8 GB can work with smaller models). 3) Enough disk space for models (5–20 GB depending on what you install). 4) Optional but helpful: an NVIDIA GPU for faster inference. CPU-only still works—just slower.

Step 1: Install Ollama on Windows

Ollama is the engine that downloads and runs the model files locally. Install Ollama from its official site and complete the installer. After installation, Ollama runs a local service and exposes an API on your machine.

To verify it’s working, open PowerShell and run a quick model test. First, pull a model and run it:

Command:

ollama run llama3.1

If the model downloads and you see a prompt where you can type, Ollama is functioning. Type something simple like “Explain DNS in one paragraph” and confirm you get a response.

Step 2: Choose a Model That Fits Your Hardware

Local AI is all about picking a model that matches your PC. As a rule, smaller models load faster and use less RAM, while larger models can be more accurate but require better hardware.

Here are practical starting points you can try with Ollama:

• llama3.1: good general assistant for many tasks.
• mistral: fast and solid for summaries and troubleshooting.
• phi3: lightweight option for lower-end machines.

To download a model without launching it immediately, you can use:

ollama pull mistral

Step 3: Install Open WebUI (Web Interface)

Ollama is powerful, but the default terminal chat is not ideal for daily use. Open WebUI adds a browser-based interface so you can manage chats, switch models, and work comfortably.

The simplest method on Windows is to run Open WebUI using Docker Desktop. Install Docker Desktop, enable WSL 2 integration if prompted, then open PowerShell and run:

docker run -d --name open-webui -p 3000:8080 -e OLLAMA_BASE_URL=http://host.docker.internal:11434 ghcr.io/open-webui/open-webui:main

This command downloads the latest Open WebUI image and connects it to Ollama running on your host. After it starts, open your browser and go to:

http://localhost:3000

Create the first admin user when prompted. Once logged in, Open WebUI should automatically detect your Ollama models. If you don’t see them, check the Ollama service is running and confirm the URL points to http://host.docker.internal:11434.

Step 4: Test a Chat and Tune the Basics

In Open WebUI, select a model (for example, llama3.1) and start a new conversation. A good first test prompt is something specific:

Write a PowerShell script that checks free disk space on C: and warns if it is below 15%.

If you get a usable script, your pipeline is working end-to-end: browser UI → Open WebUI → Ollama → local model → response back to your browser.

If answers feel slow, try a smaller model, close memory-heavy apps, and keep your prompt concise. On CPU-only systems, switching to a lighter model often makes a bigger difference than any other tweak.

Step 5: Common Problems and Fixes

Open WebUI can’t connect to Ollama: Confirm Ollama is running and listening locally. Restart the Ollama service, then restart the Open WebUI container:
docker restart open-webui

Models don’t appear in the UI: Pull a model first using Ollama (for example, ollama pull llama3.1), then refresh Open WebUI.

Performance is poor: Use a smaller model like phi3 or mistral. Also ensure Windows power mode is not set to battery saver, and keep adequate free RAM available.

Disk fills up quickly: Local models are large. Remove models you don’t use with:
ollama rm <modelname>

Why This Setup Is Worth It

A local chatbot won’t replace every cloud AI feature, but it shines in day-to-day technical work: drafting SOPs, generating scripts, summarizing logs you can’t upload, and brainstorming troubleshooting steps while keeping everything on your own PC. Once you have Ollama and Open WebUI running, adding new models is a one-command task, and the browser interface makes it feel like a real tool—not a demo.

If you want to go further later, you can explore model fine-tuning, retrieval-augmented generation (RAG) with your internal documents, or running the same stack on a small home server. For now, this Windows setup is a clean, practical starting point for private AI that you control.

Overview

This step-by-step guide shows you how to run a local AI assistant on Windows 11 using Ollama and Open WebUI. You will install Ollama, download a model, and connect a user-friendly web interface via Docker. The tutorial is beginner-friendly yet covers advanced options like GPU acceleration, authentication, and storage tuning. By the end, you will have a private, fast, and offline-capable AI setup on your own PC.

Prerequisites

Before you start, make sure you have: Windows 11 (22H2 or newer), administrator rights, and at least 8 GB RAM. For GPU acceleration, install the latest graphics driver. Ollama uses CUDA for NVIDIA GPUs and DirectML for AMD/Intel; GPU use is automatic if supported. You do not need WSL for this guide. An optional Docker Desktop installation is required for Open WebUI.

Step 1 — Install Ollama for Windows

1) Download the official installer from https://ollama.com/download and complete the setup.
2) Open PowerShell and verify the installation: ollama --version.
3) Start the Ollama service if it is not already running: ollama serve (you can keep it in the background by closing the window after confirming it is running as a service).

Step 2 — Pull and test a model

1) In PowerShell, download a model. For a good balance of speed and quality, try: ollama pull llama3.
2) Run it interactively: ollama run llama3, then ask a question like: What can you do?.
3) Exit the session with /bye when finished. Models are stored locally in %LOCALAPPDATA%\Ollama\models by default.

Step 3 — Install Docker Desktop (for Open WebUI)

Open WebUI gives you a clean web interface for prompts, chat history, and multi-model workflows. Install Docker Desktop from https://www.docker.com/products/docker-desktop/ and start it. Ensure the Docker engine is running (the whale icon should be active in the system tray).

Step 4 — Launch Open WebUI linked to Ollama

Run the following Docker command in PowerShell to start Open WebUI and connect it to your local Ollama instance exposed at http://localhost:11434:
docker run -d --name open-webui -p 3000:8080 -e OLLAMA_BASE_URL=http://host.docker.internal:11434 -v open-webui:/app/backend/data ghcr.io/open-webui/open-webui:latest
Once the container is healthy, open http://localhost:3000 in your browser. Choose a model (for example, llama3) and start chatting.

Optional — Enable authentication for Open WebUI

To protect your UI with a login, recreate the container with auth variables:
docker rm -f open-webui
docker run -d --name open-webui -p 3000:8080 -e OLLAMA_BASE_URL=http://host.docker.internal:11434 -e WEBUI_AUTH=true -e DEFAULT_USERNAME=admin -e DEFAULT_PASSWORD=changeMeNow -v open-webui:/app/backend/data ghcr.io/open-webui/open-webui:latest
Visit http://localhost:3000 and sign in with your credentials.

Optional — GPU acceleration tips

Ollama automatically uses your GPU when supported drivers are present. To nudge usage, you can set the number of GPUs: setx OLLAMA_NUM_GPU 1 then restart the Ollama service or your PC. If you have an NVIDIA GPU, ensure the latest Game Ready or Studio driver is installed. For AMD/Intel, keep your driver and Windows up to date to benefit from DirectML improvements. During the first run, the model may compile kernels; subsequent runs are faster.

Optional — Move the models folder to another drive

If you want models on a larger drive, set this environment variable and restart the service: setx OLLAMA_MODELS "D:\Ollama\Models". Move the existing folder from %LOCALAPPDATA%\Ollama\models to the new location to avoid re-downloading large files.

Troubleshooting

Open WebUI cannot connect to Ollama: Make sure Ollama is running: curl http://localhost:11434/api/tags should return a JSON list of models. If it works on the host but not in Docker, confirm the container uses host.docker.internal and port 11434 as shown in the command. Also check Windows Firewall for any blocked inbound rules on Docker or Ollama.

Models are slow or fail to load: Try a smaller model first: ollama pull phi3:mini and run ollama run phi3:mini. Close heavy apps, ensure you have enough RAM/VRAM, and avoid aggressive antivirus scanning of the models folder.

Docker errors on startup: Open Docker Desktop and verify that the engine is running. If ports are already in use, change the mapping (for example, -p 3001:8080) and refresh the browser at the new address.

Usage tips

Inside Open WebUI, create multiple chats per model for different tasks, enable markdown rendering, and configure system prompts for role-specific behavior. In PowerShell, you can also run one-off prompts without the UI: ollama run llama3 "Write a haiku about morning coffee." For reproducibility, export your Open WebUI data with the named volume and back it up regularly.

What you achieved

You now have a private, local AI assistant on Windows 11 powered by Ollama and Open WebUI. You can switch models, run fully offline, and take advantage of your GPU for faster responses. This setup is ideal for coding help, note-taking, drafting, and research without sending your data to external servers.

Overview

This step-by-step guide shows how to run local large language models with GPU acceleration on a Windows 11 machine using WSL2 (Ubuntu), Ollama, and Open WebUI. You will install WSL, enable NVIDIA GPU pass-through, run Ollama to host models like Llama 3.1, and add a friendly browser interface via Open WebUI. The result is a fast, private AI workstation with minimal overhead.

Prerequisites

You need Windows 11 with WSL2 enabled, an NVIDIA GPU with a recent driver (version 555 or newer recommended), and at least 16 GB RAM. If you use Docker Desktop you can keep it installed, but this guide keeps things simple by running both Ollama and Open WebUI directly in WSL.

1) Install WSL2 and Ubuntu

Open PowerShell as Administrator and run: wsl --install -d Ubuntu-24.04. Reboot if asked, then finish the Ubuntu setup (username and password). Inside Ubuntu, update packages with: sudo apt update && sudo apt upgrade -y. This gives you a clean base Linux environment.

2) Enable GPU for WSL

Install the latest NVIDIA Windows driver from nvidia.com that supports CUDA in WSL. After installation, reboot Windows. You do not usually need the full CUDA toolkit inside WSL; the Windows driver exposes the runtime. If a model later fails to use the GPU, install minimal CUDA userspace as a fallback: sudo apt install -y nvidia-cuda-toolkit.

3) Turn on systemd in WSL (recommended)

Enabling systemd lets you manage services cleanly. In Ubuntu, run: sudo nano /etc/wsl.conf and add: [boot] on one line and systemd=true on the next. Save and exit. In PowerShell run: wsl --shutdown, then start Ubuntu again from the Start menu. Verify with: systemctl --version (it should show a version, not an error).

4) Install Ollama

In Ubuntu, install Ollama with: curl -fsSL https://ollama.com/install.sh | sh. If systemd is enabled, start and enable the service: sudo systemctl enable --now ollama. Otherwise run it manually in the background: nohup ollama serve >/dev/null 2>&1 &. Ollama listens on port 11434 by default.

5) Pull a model and test

Download a capable model such as Llama 3.1 8B by running: ollama pull llama3.1:8b. When it completes, test a quick prompt: ollama run llama3.1:8b "Write a haiku about WSL2." To confirm GPU usage, keep a Windows terminal open with nvidia-smi and watch for activity while the prompt runs. If you prefer to force CPU temporarily, set: OLLAMA_NO_GPU=1 before the command.

6) Install Open WebUI (no Docker required)

Open WebUI provides a clean browser interface for Ollama. Install Python and pip if needed: sudo apt install -y python3-pip python3-venv. Then install Open WebUI system-wide: pipx works great, but pip also works. For simplicity run: pip install --upgrade pip && pip install open-webui. Start it with: open-webui serve --host 0.0.0.0 --port 3000. If the command is not found, try: python3 -m open_webui serve --host 0.0.0.0 --port 3000.

7) Connect Open WebUI to Ollama

By default, Open WebUI will attempt to connect to a local Ollama server at http://127.0.0.1:11434. If it does not auto-detect, open the Web UI in your Windows browser at http://localhost:3000, go to Settings, then Connections, and set the base URL to http://127.0.0.1:11434. Save the setting and try a prompt using the model you pulled earlier.

8) Make services persistent

If systemd is enabled, Ollama already runs as a service. To run Open WebUI as a user service, create a file: mkdir -p ~/.config/systemd/user and then nano ~/.config/systemd/user/open-webui.service. Paste the following lines:

[Unit] Description=Open WebUI After=network.target

[Service] ExecStart=/usr/bin/env open-webui serve --host 0.0.0.0 --port 3000 Restart=always

[Install] WantedBy=default.target

Save the file, then run: systemctl --user daemon-reload && systemctl --user enable --now open-webui. Ensure lingering is on so it starts after reboot: loginctl enable-linger $USER. Now both services will start automatically whenever WSL launches.

9) Performance and resource tips

If you have a large GPU (12 GB+), try bigger models like llama3.1:70b but expect high VRAM use. For mid-range GPUs, 7B–8B models are ideal. You can limit VRAM use by setting OLLAMA_NUM_GPU=1 for single-GPU systems. Keep enough disk space under your WSL distro; models can be multiple gigabytes each. You can remove unused models with: ollama rm MODEL_NAME.

10) Troubleshooting

If Open WebUI cannot connect, confirm Ollama is running with: curl http://127.0.0.1:11434/api/tags. If GPU is not used, update to the latest NVIDIA driver, reboot, and re-test. From the Windows side, run nvidia-smi while prompting to confirm activity. If you see CUDA errors, install minimal CUDA libraries in WSL (nvidia-cuda-toolkit) and try again. On AMD GPUs, GPU acceleration in WSL is limited; use CPU or run Ollama natively on Windows if supported for your hardware.

Security notes

By default, Open WebUI’s first account becomes admin. Set a strong password and avoid exposing port 3000 to the internet. If you later publish this service beyond localhost, place it behind a reverse proxy with TLS and enable authentication. Consider binding Open WebUI to 127.0.0.1 only and accessing it via Windows browser for local use.

Wrap-up

You have a private, GPU-accelerated AI stack on Windows 11 using WSL2, Ollama, and Open WebUI. This setup is fast, secure, and flexible, letting you try different models without cloud costs. Keep your drivers updated, manage models carefully to save disk, and enjoy local inference with a smooth web experience.