LifeBytes Journal

Overview

This step-by-step guide shows you how to run local Large Language Models (LLMs) on Ubuntu using Ollama and Open WebUI. You will install Ollama, optionally enable NVIDIA GPU acceleration, and deploy Open WebUI in Docker to get a fast, friendly chat interface. By the end, you will have a private AI assistant running on your own hardware with secure access options and practical troubleshooting tips.

Prerequisites

Use Ubuntu 22.04 or 24.04 with at least 8 GB of RAM (16 GB recommended). For GPU acceleration, an NVIDIA GPU with 8 GB or more VRAM is ideal. You need sudo access and open ports 11434 for Ollama and 3000 (or your choice) for Open WebUI. This guide covers both CPU-only and GPU setups, so you can start even without a supported GPU.

Step 1: (Optional) Install NVIDIA Drivers and CUDA

If you plan to use a GPU, first confirm your hardware with lspci | grep -i nvidia. Install the recommended driver via sudo ubuntu-drivers autoinstall, then reboot. After rebooting, verify the driver with nvidia-smi. If you will run Open WebUI with GPU access in Docker, also install the NVIDIA container runtime using sudo apt-get install -y nvidia-container-toolkit and configure Docker with sudo nvidia-ctk runtime configure followed by sudo systemctl restart docker.

Step 2: Install Ollama on Ubuntu

Install Ollama with a single command: curl -fsSL https://ollama.com/install.sh | sh. This creates a system service and exposes the local API on http://127.0.0.1:11434. Check the version with ollama -v and verify the service using systemctl status ollama. If you need remote access on your LAN, set the host binding by creating an override file. Run sudo systemctl edit ollama, add [Service] and Environment="OLLAMA_HOST=0.0.0.0:11434", then save, sudo systemctl daemon-reload, and sudo systemctl restart ollama. Only expose Ollama on trusted networks or behind a reverse proxy with authentication.

Step 3: Pull and Run Models with Ollama

Pull a small, fast model to test your setup. For general chat, use ollama pull llama3.2:3b. For coding tasks, try ollama pull qwen2.5-coder:7b or a quantized variant like :q4_0 for lower memory usage. Run an interactive session with ollama run llama3.2 and type your prompt. To generate from the shell, try echo "Explain RAID levels simply" | ollama run llama3.2. Ollama will use the GPU automatically if supported; otherwise it falls back to CPU. Tune performance with environment variables such as OLLAMA_NUM_PARALLEL=1 to reduce memory pressure and OLLAMA_KV_SIZE=512 for larger context windows when your memory allows.

Step 4: Deploy Open WebUI with Docker

Open WebUI provides a clean web interface and multi-model support. If Docker is not installed, add it with sudo apt-get update && sudo apt-get install -y docker.io and ensure it runs at startup with sudo systemctl enable --now docker. Launch Open WebUI connected to Ollama using docker run -d --name open-webui -p 3000:8080 -e OLLAMA_BASE_URL=http://localhost:11434 -v open-webui:/app/backend/data -v /var/lib/ollama:/root/.ollama --restart unless-stopped ghcr.io/open-webui/open-webui:latest. If Open WebUI runs on a different host from Ollama, set OLLAMA_BASE_URL to the Ollama server’s IP, for example http://192.168.1.50:11434. For GPU inside the container, add --gpus all and make sure the NVIDIA container toolkit is configured.

Step 5: Secure Access with a Reverse Proxy and HTTPS

If you plan to reach the interface over the internet, place Open WebUI behind a reverse proxy with TLS and authentication. A simple option is Caddy, which can obtain and renew certificates automatically. For example, you can point a domain to your server and configure Caddy to proxy yourdomain.com to localhost:3000 and enable basic auth. With Nginx, use an SSL server block, set proxy_pass http://127.0.0.1:3000, and enable rate limiting and headers like X-Frame-Options and Content-Security-Policy. Always avoid exposing the raw Ollama port unless you fully trust the network.

Step 6: Updates, Backups, and Autostart

Update Ollama by rerunning the installer or using your package manager if you installed via a repo. To update Open WebUI, pull the latest image with docker pull ghcr.io/open-webui/open-webui:latest and restart the container. Persist your data by backing up /var/lib/ollama and the Docker volume open-webui. Both Ollama and Docker containers start automatically on boot, but you can confirm with systemctl is-enabled ollama and the container’s --restart unless-stopped flag.

API Quick Test

You can call Ollama’s local API directly. After pulling a model, try curl http://localhost:11434/api/generate -d '{"model":"llama3.2","prompt":"Give me three bullet points about containers"}'. This is useful for integrating local LLMs into scripts, chatbots, or development tools without sending data to third parties.

Troubleshooting

If model loading fails with “no space left on device,” free disk space with df -h, remove unused Docker images with docker system prune -a, or delete old models in /var/lib/ollama. If nvidia-smi returns an error, reinstall the driver and ensure Secure Boot is either disabled or configured with signed modules. If port 11434 or 3000 is already in use, change the binding (for example OLLAMA_HOST=0.0.0.0:11435) or stop the conflicting process. On low-memory hosts, choose smaller or more heavily quantized models (for example :q4_0), reduce parallel requests with OLLAMA_NUM_PARALLEL=1, and close other memory-hungry services.

What You Achieved

You now have a private, production-ready local AI stack on Ubuntu. Ollama runs the model backend with optional GPU acceleration, while Open WebUI delivers a modern chat interface. With a reverse proxy and backups in place, you can confidently use local LLMs for coding assistance, content drafting, documentation, and experimentation without sending your data to the cloud.

This step-by-step guide shows you how to run a fast, private, and local AI chatbot on Ubuntu 22.04 or 24.04 using Ollama and Open WebUI. You will install the Ollama runtime, pull a modern large language model, and add a clean chat interface via Open WebUI in Docker. Optional steps cover NVIDIA GPU acceleration, API usage, and persistence. The result is a secure, offline-friendly setup suitable for helpdesk, coding assistance, or knowledge base querying without sending data to the cloud.

Why Ollama + Open WebUI

Ollama makes it simple to run and manage open-source LLMs locally (Llama 3.x, Mistral, Phi, Qwen, and more). Open WebUI adds a user-friendly, browser-based chat interface with conversation history, prompt templates, and multi-model support. Together they form a robust, low-maintenance local AI stack for Linux desktops and servers.

Prerequisites

- Ubuntu 22.04 or 24.04 with a non-root sudo user.
- At least 8 GB RAM (16 GB recommended for larger models).
- Optional NVIDIA GPU for acceleration (T4/RTX/RTX A-series, etc.).
- Internet access to download models and containers.

Step 1 — Install Ollama

1) Update packages:
sudo apt update && sudo apt install -y curl ca-certificates
2) Install Ollama:
curl -fsSL https://ollama.com/install.sh | sh
3) Enable as a service:
sudo systemctl enable --now ollama
4) Verify the API is up:
curl http://localhost:11434/api/tags
If you see JSON, Ollama is running correctly.

Step 2 — Pull and test a model

Pull a compact, capable model first to validate your setup:
ollama pull llama3.2
Run an interactive test:
ollama run llama3.2
Type a prompt, then press Ctrl+C to exit. You can later try larger models (for example, ollama pull mistral or ollama pull llama3.1), but start small to confirm everything works.

Step 3 — Install Docker Engine

1) Add Docker’s repo key and source:
sudo apt-get update && sudo apt-get install -y ca-certificates curl gnupg
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(. /etc/os-release; echo $UBUNTU_CODENAME) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
2) Install Docker and the Compose plugin:
sudo apt-get update && sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
3) Add your user to the docker group and refresh your shell:
sudo usermod -aG docker $USER
newgrp docker

Step 4 — Run Open WebUI connected to Ollama

Start Open WebUI and point it to the Ollama API on the host. The --add-host flag maps host.docker.internal to your host’s gateway so the container can reach http://localhost:11434 on the host:

docker run -d --name open-webui --restart=unless-stopped -p 3000:8080 \
--add-host=host.docker.internal:host-gateway \
-e OLLAMA_BASE_URL=http://host.docker.internal:11434 \
-v open-webui:/app/backend/data \
ghcr.io/open-webui/open-webui:main

Open your browser to http://SERVER_IP:3000 (or http://localhost:3000). Create the first admin account, choose a model (for example, llama3.2), and start chatting.

Step 5 — Enable NVIDIA GPU acceleration (optional)

1) Install the latest NVIDIA driver for your GPU using Ubuntu’s Additional Drivers or apt. Reboot if prompted.
2) Install the NVIDIA Container Toolkit so Docker can access the GPU:
distribution=$(. /etc/os-release; echo $ID$VERSION_ID)
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -fsSL https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#' | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt update && sudo apt install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
3) Recreate Open WebUI with GPU access:
docker rm -f open-webui
docker run -d --name open-webui --restart=unless-stopped --gpus all -p 3000:8080 \
--add-host=host.docker.internal:host-gateway \
-e OLLAMA_BASE_URL=http://host.docker.internal:11434 \
-v open-webui:/app/backend/data \
ghcr.io/open-webui/open-webui:main
4) Ollama will also use the GPU automatically when a compatible model is loaded. You can confirm GPU use with nvidia-smi during inference.

Step 6 — Use the Ollama HTTP API

You can script local inference via HTTP without the UI. Example generation request:
curl http://localhost:11434/api/generate -d '{"model":"llama3.2","prompt":"Write a haiku about backups."}'
Chat format with memory:
curl http://localhost:11434/api/chat -d '{"model":"llama3.2","messages":[{"role":"user","content":"Explain DNS in one sentence."}]}'

Step 7 — Persistence, autostart, and updates

- Ollama models are stored under ~/.ollama/models. Back up this directory to avoid re-downloading models.
- The Open WebUI container uses a named volume (open-webui) for its data, which persists across restarts.
- Ollama is already set to start at boot (systemctl enable ollama). The WebUI container uses --restart=unless-stopped so it will auto-start after a reboot.
- Update Ollama: curl -fsSL https://ollama.com/install.sh | sh
- Update Open WebUI: docker pull ghcr.io/open-webui/open-webui:main && docker restart open-webui

Troubleshooting

- Open WebUI cannot connect to Ollama: ensure you used --add-host=host.docker.internal:host-gateway and that curl http://localhost:11434/api/tags works on the host.
- Port already in use: change -p 3000:8080 to a different host port like -p 3333:8080.
- Out of memory or slow responses: try a smaller model (for example, llama3.2 or phi3). Close other apps or add swap. For CPU-only hosts, expect slower performance on large models.
- GPU not used: verify drivers, nvidia-smi, and that the container runs with --gpus all. Pull a GPU-optimized model variant if available.

What you can do next

- Connect knowledge bases or documents using Open WebUI’s RAG features to power local search over PDFs and wikis.
- Add multiple models and switch per chat, benchmarking speed and quality.
- Put Nginx or Caddy in front of :3000 for HTTPS and trusted network access.
- Automate prompts with shell scripts or Python by calling the local Ollama API.

You now have a private, local AI assistant on Ubuntu with a clean web interface, GPU-ready acceleration, and a stable upgrade path—all without sending your data to third-party services.

Overview

If you want a fast, private, and low-cost way to chat with large language models on your own machine, pairing Ollama with Open WebUI inside Docker is a great setup. Ollama handles model downloads and inference (CPU or NVIDIA GPU), while Open WebUI provides a clean, modern chat interface in your browser. This guide shows how to deploy both with Docker Compose on Ubuntu 22.04/24.04 and enable GPU acceleration for significant speedups.

By the end, you will have a persistent, self-hosted AI chat running at http://localhost:3000, with models managed by Ollama at http://localhost:11434. The instructions also include CPU-only notes, backup tips, and troubleshooting for common pitfalls.

Prerequisites

- Ubuntu 22.04 or 24.04 with sudo access. Windows and macOS work with Docker too, but this tutorial focuses on Ubuntu.
- For GPU acceleration: an NVIDIA GPU with a recent driver (typically 525+). CPU-only also works, just slower.
- Docker Engine and Docker Compose plugin (we will install them below).
- At least 16 GB RAM for 7B–8B models; more is better for larger models.
- Open ports: 11434 (Ollama API) and 3000 (Open WebUI).

Step 1 — Install Docker Engine and Compose

Commands:
sudo apt-get update
sudo apt-get install -y ca-certificates curl gnupg
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo $VERSION_CODENAME) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo usermod -aG docker $USER && newgrp docker

Verify Docker works: docker run --rm hello-world. Verify Compose works: docker compose version.

Step 2 — Install NVIDIA Driver (GPU users)

If you already have a recent NVIDIA driver, you can skip this step. Otherwise, install the recommended driver and reboot:

sudo ubuntu-drivers autoinstall
sudo reboot

After reboot, confirm the GPU is visible: nvidia-smi. You should see your GPU model and driver version.

Step 3 — Enable GPU inside Docker

Install the NVIDIA Container Toolkit so Docker containers can access your GPU:

distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit.gpg
curl -fsSL https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \
sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit

Configure Docker to use the NVIDIA runtime by default:

sudo mkdir -p /etc/docker
cat <<'EOF' | sudo tee /etc/docker/daemon.json
{
"default-runtime": "nvidia",
"runtimes": {
"nvidia": {
"path": "nvidia-container-runtime",
"runtimeArgs": []
}
}
}
EOF
sudo systemctl restart docker

Test GPU access in containers: docker run --rm --gpus all nvidia/cuda:12.3.2-base-ubuntu22.04 nvidia-smi. If you see the usual output, you are set.

CPU-only? Skip Step 3 and the GPU test. The rest works the same, just remove the NVIDIA-specific line from the compose file noted below.

Step 4 — Create a Docker Compose file

Create a new folder and the compose file:

mkdir -p ~/ai-stack && cd ~/ai-stack
nano docker-compose.yml

Paste the following content, then save:

version: "3.9"
services:
ollama:
image: ollama/ollama:latest
container_name: ollama
restart: unless-stopped
ports:
- "11434:11434"
environment:
- OLLAMA_KEEP_ALIVE=30m
volumes:
- ollama:/root/.ollama
runtime: nvidia

open-webui:
image: ghcr.io/open-webui/open-webui:main
container_name: open-webui
restart: unless-stopped
depends_on:
- ollama
environment:
- OLLAMA_API_BASE_URL=http://ollama:11434
ports:
- "3000:8080"
volumes:
- openwebui:/app/backend/data

volumes:
ollama:
openwebui:

Note: If you are running CPU-only, delete the line runtime: nvidia and keep everything else.

Step 5 — Launch the stack and pull a model

docker compose up -d

Wait a few seconds and confirm both containers are healthy: docker ps. Next, pull a model into Ollama. Good starters are llama3.1:8b, mistral:7b, or a small qwen2:7b.

docker exec -it ollama ollama pull llama3.1:8b

List installed models with: docker exec -it ollama ollama list.

Step 6 — Chat in Open WebUI

Open http://localhost:3000 in your browser. Open WebUI should auto-detect Ollama via the environment variable, but you can also set the API in Settings > Connections to http://ollama:11434 (inside Docker) or http://localhost:11434 (host access). Create a new chat, choose your model (for example, llama3.1:8b), and start chatting locally.

Backups, Updates, and Performance Tips

Persistence: Your models and chats are stored in the named volumes ollama and openwebui. Back them up with docker run --rm -v ollama:/data -v $(pwd):/backup alpine tar czf /backup/ollama.tgz -C / data (and similarly for openwebui).

Updates: Pull fresh images and recreate: docker compose pull && docker compose up -d. Ollama keeps your models; no need to re-download.

Performance: Prefer GPU for best speed. If RAM/VRAM is tight, choose smaller or quantized models (e.g., llama3.1:8b-q4_K_M). Set OLLAMA_KEEP_ALIVE to keep models warm between requests.

Remote access: If exposing over the internet, place Open WebUI behind a reverse proxy (Nginx, Caddy, or Traefik) and enable authentication and TLS. Never expose Ollama directly without controls.

Troubleshooting

GPU not detected: Check nvidia-smi works on the host. Then run docker run --rm --gpus all nvidia/cuda:12.3.2-base-ubuntu22.04 nvidia-smi. If that fails, revisit Step 3 and confirm /etc/docker/daemon.json is correct and Docker was restarted.

Open WebUI cannot reach Ollama: Ensure both containers are up. Verify docker logs open-webui and confirm OLLAMA_API_BASE_URL is http://ollama:11434. From the host, curl http://localhost:11434/api/tags should list installed models.

Downloads are slow: Models can be several GB. Use a wired connection or pre-fetch models off-peak. You can also copy existing models into the ollama volume if you have them from another machine.

Port conflicts: If ports 11434 or 3000 are in use, change them in the compose file (left side of the colon) and recreate the stack.

What You Achieved

You now have a self-hosted AI chat stack running locally with Docker. Ollama manages lightweight, high-quality models, and Open WebUI provides a comfortable chat experience. With GPU acceleration, responses are significantly faster, and your data never leaves your machine. Extend this setup with a reverse proxy, add more models, or integrate the Ollama API into your own apps for a powerful private AI workstation.

Overview

Running a local large language model (LLM) is easier than ever thanks to Ollama and Open WebUI. Ollama handles model downloads and inference, while Open WebUI gives you a clean, chat-style interface in your browser. In this tutorial, you'll install both on Ubuntu 24.04 (works on 22.04 too), enable NVIDIA GPU acceleration, and deploy them with Docker Compose. The result is a fast, private AI stack you control.

What You'll Need

- Ubuntu 24.04 or 22.04 (fresh or existing server/desktop).
- An NVIDIA GPU with recent drivers (Turing/RTX or newer recommended).
- Root or sudo access.
- Open ports 3000 (Open WebUI) and 11434 (Ollama) on your firewall if you access remotely.

Step 1: Install NVIDIA Drivers and Verify CUDA

First, update your system and install the recommended NVIDIA driver. On Ubuntu Desktop you can use Additional Drivers, but the CLI route is reliable:

sudo apt update && sudo apt -y upgrade
sudo ubuntu-drivers autoinstall
sudo reboot

After reboot, confirm the GPU is visible:

nvidia-smi

If you see a driver table with your GPU, you're set. If not, re-run the install or check Secure Boot status (disable or enroll the MOK as needed).

Step 2: Install Docker, Compose, and NVIDIA Container Toolkit

Install Docker from the official repository so you get the latest engine and the Compose plugin:

sudo apt-get remove -y docker docker.io containerd runc || true
sudo apt update
sudo apt install -y ca-certificates curl gnupg
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(. /etc/os-release; echo $VERSION_CODENAME) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo usermod -aG docker $USER
newgrp docker

Add NVIDIA Container Toolkit so Docker can access the GPU:

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
distribution=$(. /etc/os-release; echo $ID$VERSION_ID)
curl -fsSL https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#' | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt update
sudo apt install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

Verify Docker can see the GPU:

docker run --rm --gpus all nvidia/cuda:12.3.2-base-ubuntu22.04 nvidia-smi

Step 3: Create a Docker Compose File for Ollama and Open WebUI

Create a working directory and a docker-compose.yml:

mkdir -p ~/ollama-openwebui && cd ~/ollama-openwebui
nano docker-compose.yml

Paste the following content, then save:

version: "3.9"
services:
ollama:
image: ollama/ollama:latest
container_name: ollama
restart: unless-stopped
ports:
- "11434:11434"
volumes:
- ollama:/root/.ollama
environment:
- OLLAMA_KEEP_ALIVE=24h
gpus: all

openwebui:
image: ghcr.io/open-webui/open-webui:main
container_name: open-webui
restart: unless-stopped
ports:
- "3000:8080"
environment:
- OLLAMA_API_BASE_URL=http://ollama:11434
- ENABLE_AUTH=True
volumes:
- openwebui:/app/backend/data
depends_on:
- ollama

volumes:
ollama:
openwebui:

Bring everything up:

docker compose up -d

Open WebUI will be available at http://<your-server-ip>:3000 and Ollama's API at http://<your-server-ip>:11434.

Step 4: Pull a Model and Test Inference

Use Ollama to pull an LLM. Llama 3 8B is a good starting point if you have at least ~8–10 GB of free VRAM:

docker exec -it ollama ollama pull llama3:8b

You can test quickly from the CLI:

docker exec -it ollama ollama run llama3:8b

Or open your browser and navigate to Open WebUI (port 3000). Create an account on first visit, select the model, and start chatting. If GPU is being used, you should see activity in:

watch -n 1 nvidia-smi

Step 5: Secure and Maintain the Stack

- Firewall: Allow only needed ports (adjust to your network policy). For local-only use, block remote access to 3000/11434.
- Reverse proxy: For TLS and a friendly domain, put Nginx or Caddy in front of Open WebUI and obtain a Let's Encrypt certificate.
- Updates: Keep images fresh and restart the stack regularly:

docker compose pull
docker compose up -d

Back up volumes so you don't lose chats or downloaded models:

docker run --rm -v ollama:/data -v "$(pwd)":/backup alpine tar czf /backup/ollama-vol.tgz -C /data .
docker run --rm -v openwebui:/data -v "$(pwd)":/backup alpine tar czf /backup/openwebui-vol.tgz -C /data .

Troubleshooting

- No GPU in containers: Confirm the toolkit is active. Check docker info | grep -i nvidia. Re-run sudo nvidia-ctk runtime configure --runtime=docker and restart Docker.
- Model out-of-memory (OOM): Use a smaller model or quantized variant (e.g., llama3:8b-instruct-q4_0). Close other GPU apps. You can also reduce context in Open WebUI settings.
- Slow generation: Ensure you're not falling back to CPU (watch nvidia-smi). Update drivers and Docker images. Use recent CUDA-compatible drivers (550+ often recommended).
- Open WebUI cannot reach Ollama: Check the environment OLLAMA_API_BASE_URL=http://ollama:11434. View logs with docker logs open-webui and docker logs ollama.
- Port conflicts: Change the host ports in docker-compose.yml (e.g., map "127.0.0.1:3000:8080" to bind only locally).

Where Models Are Stored and How to Clean Up

Models live in the Ollama volume (/root/.ollama inside the container). To list installed models:

docker exec -it ollama ollama list

Remove a model you no longer need:

docker exec -it ollama ollama rm llama3:8b

If you ever want to stop and remove the stack:

docker compose down

To reclaim space including volumes (this deletes your models and chat history), run:

docker compose down -v

Wrap-Up

You now have a private, GPU-accelerated LLM environment powered by Ollama and Open WebUI on Ubuntu. With Docker Compose, updates and maintenance are straightforward, and volumes keep your data persistent. From here, try different models (Mistral, Phi-3, Llama 3 Instruct), experiment with prompt templates, and fine-tune performance for your hardware. Enjoy your local AI workstation or server—no cloud required.