LifeBytes Journal

Overview

This step-by-step guide shows you how to self-host Ollama with Open WebUI on Ubuntu 22.04/24.04 and use your NVIDIA GPU for fast, private large language model (LLM) inference. You will install the correct NVIDIA drivers, Docker, and NVIDIA Container Toolkit, then deploy Ollama and Open WebUI with Docker Compose. The tutorial also covers updating, backing up models, and troubleshooting common errors such as GPU visibility and port conflicts.

Prerequisites

Before you begin, make sure you have: (1) Ubuntu 22.04 or 24.04 with sudo access, (2) an NVIDIA GPU with at least 6 GB VRAM for medium models (smaller models can work with less), (3) a stable internet connection, and (4) at least 20 GB free disk space for images and model files.

Step 1: Install NVIDIA Driver and Verify CUDA

Use Ubuntu’s built-in tool to install a matching proprietary driver. If Secure Boot is enabled, you may need to enroll a Machine Owner Key (MOK) during installation to load the NVIDIA kernel module.

sudo apt update
sudo ubuntu-drivers install
sudo reboot

After the reboot, confirm the driver is active:

nvidia-smi

You should see a table with your GPU and driver version. If you get an error, check Secure Boot (disable it or enroll the NVIDIA module), then repeat the install.

Step 2: Install Docker Engine and NVIDIA Container Toolkit

Install Docker from the official repository and add your user to the docker group so you can run containers without sudo.

sudo apt-get install -y ca-certificates curl gnupg
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo $VERSION_CODENAME) stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo usermod -aG docker $USER
newgrp docker

Add NVIDIA Container Toolkit so containers can use your GPU:

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit.gpg
distribution=$(. /etc/os-release; echo $ID$VERSION_ID)
curl -fsSL https://nvidia.github.io/libnvidia-container/stable/$distribution/libnvidia-container.list | \
sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit.gpg] https://#g' | \
sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

Verify Docker can see your GPU:

docker run --rm --gpus all nvidia/cuda:12.5.0-base-ubuntu22.04 nvidia-smi

Step 3: Deploy Ollama and Open WebUI with Docker Compose

We will bind both services to localhost for safety. You can put a reverse proxy in front later for remote access.

mkdir -p ~/ollama-stack && cd ~/ollama-stack
nano compose.yaml

Paste the following compose file (save and exit):

services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    restart: unless-stopped
    ports:
      - "127.0.0.1:11434:11434"
    environment:
      - OLLAMA_HOST=0.0.0.0
      - OLLAMA_NUM_PARALLEL=1
    volumes:
      - ollama:/root/.ollama
    gpus: all

  open-webui:
    image: ghcr.io/open-webui/open-webui:latest
    container_name: open-webui
    restart: unless-stopped
    depends_on:
      - ollama
    environment:
      - OLLAMA_API_BASE=http://ollama:11434
    ports:
      - "127.0.0.1:3000:8080"
    volumes:
      - openwebui:/app/backend/data

volumes:
  ollama:
  openwebui:

Start the stack:

docker compose up -d

Pull a model into Ollama (example: a small, fast model):

docker exec -it ollama ollama pull llama3.2:3b

Quick API test:

curl http://127.0.0.1:11434/api/generate \
  -H "Content-Type: application/json" \
  -d '{"model":"llama3.2:3b","prompt":"Say hello in one sentence."}'

Open your browser at http://127.0.0.1:3000 and select Ollama as the provider. Choose the model you pulled and start chatting.

Step 4: Updates and Backups

To update Ollama and Open WebUI to the latest images while keeping your models and data, run:

cd ~/ollama-stack
docker compose pull
docker compose up -d

Back up volumes (models and WebUI data) with a simple tar archive:

docker stop open-webui ollama
docker run --rm -v ollama:/data -v "$PWD":/backup alpine \
  sh -c 'tar czf /backup/ollama-vol.tar.gz -C /data .'
docker run --rm -v openwebui:/data -v "$PWD":/backup alpine \
  sh -c 'tar czf /backup/openwebui-vol.tar.gz -C /data .'
docker start ollama open-webui

Troubleshooting

No CUDA-capable device detected: Ensure the NVIDIA driver is loaded (nvidia-smi works on the host). If Secure Boot is on, enroll the MOK or disable Secure Boot. Confirm the container sees the GPU with the CUDA test image. Re-run: sudo nvidia-ctk runtime configure --runtime=docker and restart Docker.

Compose error: unknown field "gpus": Your Docker Compose is outdated. Update Docker or use: docker run --gpus all ... Alternatively, in compose.yaml, remove gpus: all and start Ollama with: docker run -d --gpus all -p 127.0.0.1:11434:11434 -v ollama:/root/.ollama --name ollama ollama/ollama:latest

Port already in use: Change the host ports in compose.yaml (for example, 127.0.0.1:11435:11434 and 127.0.0.1:3001:8080) and re-run docker compose up -d.

Out-of-memory or slow responses: Choose a smaller or more quantized model (e.g., llama3.2:1b or a Q4 version if available). Limit parallel requests with OLLAMA_NUM_PARALLEL=1. Ensure you have adequate swap configured on the host for large models.

Security Tips

Keep services bound to 127.0.0.1 and place a reverse proxy with TLS in front (Caddy, Traefik, or Nginx) if you need remote access. For Open WebUI, enable authentication in its settings. Restrict firewall rules to only allow your reverse proxy and management IPs. Regularly update images and prune unused layers with docker system prune -af.

Clean Uninstall

To remove the stack and its volumes (this deletes downloaded models and chat data), run:

cd ~/ollama-stack
docker compose down -v

Conclusion

You have a fully private, GPU-accelerated local AI setup with Ollama and Open WebUI running on Ubuntu. This stack is easy to update, simple to back up, and flexible: you can try multiple models, script against the API, or place it behind a secure reverse proxy for team access. With one machine and an NVIDIA GPU, you now own your LLM workflow end to end.

Overview

If you want to publish a self‑hosted service on the internet without opening ports on your router, Cloudflare Tunnel is a modern, zero‑trust solution. In this tutorial, you will deploy Cloudflare Tunnel with Docker, route a subdomain to a local container, and add single‑sign‑on (SSO) protection. This setup works well for homelabs and small businesses, is fast to roll out, and uses Cloudflare’s free plan.

Prerequisites

1) A domain added to Cloudflare (DNS managed by Cloudflare). 2) A Linux server or VM with internet access. 3) Docker and the Docker Compose plugin installed. 4) Basic command‑line access via SSH.

Step 1 — Install Docker and Compose

On Debian/Ubuntu, you can install from the OS repo for a quick start. For production, prefer Docker’s official repositories. Quick start example:

sudo apt update && sudo apt install -y docker.io docker-compose-plugin

Verify installation:

docker --version
docker compose version

Step 2 — Prepare a working directory

Create a project folder to keep tunnel files and your Compose file organized:

mkdir -p ~/cf-tunnel/cloudflared && cd ~/cf-tunnel

Step 3 — Create the Tunnel in Cloudflare

Open Cloudflare Dashboard > Zero Trust > Networks > Tunnels > Add a tunnel. Choose “Cloudflared” and give it a friendly name (for example, homelab-tunnel). After creation, click the tunnel and add a “Public Hostname.” Set Hostname to a subdomain like app.yourdomain.com, and Type to HTTP. For the service URL, use a local address you will run (for example, http://app:3000). Saving this will also create the DNS CNAME for you automatically.

In the same page, download the credentials file (a JSON file with your tunnel ID) and, if offered, the suggested config.yml. Save the JSON to ~/cf-tunnel/cloudflared/ and note the filename; it looks like xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.json.

Step 4 — Create cloudflared config.yml

If you did not download a config from the dashboard, create one now at ~/cf-tunnel/cloudflared/config.yml with the following content (replace placeholders):

tunnel: YOUR-TUNNEL-UUID
credentials-file: /etc/cloudflared/YOUR-TUNNEL-UUID.json

ingress:
  - hostname: app.yourdomain.com
    service: http://app:3000
  - service: http_status:404

This tells cloudflared to forward traffic for your subdomain to the local container named app on port 3000, and return a 404 for everything else.

Step 5 — Create a Docker Compose file

We will run a sample application and cloudflared in the same Docker network. Create ~/cf-tunnel/compose.yml with:

services:
  app:
    image: traefik/whoami:latest
    container_name: whoami
    expose:
     - "80"
    restart: unless-stopped

  cloudflared:
    image: cloudflare/cloudflared:latest
    container_name: cloudflared
    command: tunnel --config /etc/cloudflared/config.yml run
    volumes:
     - ./cloudflared:/etc/cloudflared:ro
    depends_on:
     - app
    restart: unless-stopped

The app service is a tiny HTTP server used as a demo. Cloudflared reads your config and credentials from the mounted folder.

Step 6 — Start the stack and test

Run the following from ~/cf-tunnel:

docker compose up -d
docker logs -f cloudflared

When logs show “Connected to Cloudflare,” visit https://app.yourdomain.com. You should see a response from “whoami.” Use curl -I https://app.yourdomain.com to verify status 200 over HTTPS.

Optional — Add Zero Trust access

To protect your app with SSO, open Cloudflare Dashboard > Zero Trust > Access > Applications > Add an application > Self‑hosted. Set the application domain to app.yourdomain.com. Add a policy to “Allow” specific emails, domains, or identity providers (Google, GitHub, Azure AD). Save. Your app now prompts users to authenticate before reaching your origin.

Maintenance and updates

Update images periodically to receive security patches and performance improvements. Use:

docker compose pull && docker compose up -d

Because the tunnel runs outbound over HTTPS, you do not need to open inbound ports on your router. Keep your server’s system packages current and restrict SSH access with keys and a firewall.

Troubleshooting

502 Bad Gateway: Usually means cloudflared cannot reach your container. Confirm the service name and port in config.yml, ensure both containers share the same Docker network (Compose sets this by default), and that the app is listening on the correct port.

404 Not Found: If you have multiple hostnames, make sure the correct hostname entry exists in the ingress block and that it points to the right service. The last rule should be a 404 fallback.

DNS not resolving: In the Tunnel page, verify the “Public Hostname” exists and the DNS CNAME was created. If needed, create a CNAME for app.yourdomain.com pointing to YOUR-TUNNEL-UUID.cfargotunnel.com.

Authentication loop with Access: Clear cookies or add your domain to “Allowed Cookie Domains” in the Access app settings. Also confirm your policy “Allow” rules match your user identity.

Connectivity drops: Check server time (NTP), ensure no outbound firewall is blocking HTTPS to Cloudflare, and consider running two cloudflared replicas for high availability.

What you achieved

You deployed a secure reverse tunnel with Docker that exposes a local container to the internet under your own domain, without port forwarding or a public IP. You also learned how to enable Cloudflare Access as a zero‑trust layer for SSO and policy control. This pattern scales to multiple services by adding more ingress rules or additional public hostnames in the dashboard, making it a clean, maintainable approach to self‑hosting.