LifeBytes Journal

Overview

This guide shows you how to run a private, local AI chatbot on Ubuntu using Ollama (for running large language models on CPU or GPU) and Open WebUI (a clean web interface). You will be able to chat with models like Llama 3 locally, without sending data to the cloud. The steps work on Ubuntu 22.04 and 24.04, and are suitable for both desktops and servers.

What You Will Set Up

You will install Ollama on the host, pull a model, and run Open WebUI in Docker. Open WebUI will connect to the Ollama API on port 11434. The result is a self-hosted, secure, and fast AI assistant accessible at http://YOUR_SERVER_IP:3000.

Prerequisites

- Ubuntu 22.04 or 24.04 with sudo access
- At least 8 GB RAM (16 GB+ recommended for 7B–8B models; use smaller quantized models on low-RAM systems)
- Optional: NVIDIA or AMD GPU for faster inference
- Internet access to download packages and models

Step 1: Install Ollama

Ollama is a lightweight runtime that serves models locally over an HTTP API (default: 11434). Install it with one command:

curl -fsSL https://ollama.com/install.sh | sh

Verify the installation and service:

ollama --version
systemctl status ollama

If the service is inactive, start it:

sudo systemctl enable --now ollama

Step 2: Pull a Model (Llama 3.1 as an example)

Download a good general-purpose model. Llama 3.1 8B is a solid balance for many machines:

ollama pull llama3.1:8b

Test it in the terminal:

ollama run llama3.1:8b
>>> Write a 1-sentence productivity tip.

Tip: If you run out of memory, pull a quantized variant, for example:

ollama pull llama3.1:8b-instruct-q4_K_M

Step 3: Install Docker Engine

Open WebUI ships a reliable container image. Install Docker from the official repository:

sudo apt update
sudo apt install -y ca-certificates curl gnupg
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo $VERSION_CODENAME) stable" \
| sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo usermod -aG docker $USER

Log out and in again (or run a new shell) to use Docker without sudo.

Step 4: Run Open WebUI and Connect to Ollama

Start Open WebUI in Docker, mapping port 3000 and pointing it to the Ollama API on the host. The host gateway alias works on modern Docker versions:

docker run -d --name open-webui \
  --restart unless-stopped \
  -p 3000:8080 \
  --add-host=host.docker.internal:host-gateway \
  -e OLLAMA_BASE_URL=http://host.docker.internal:11434 \
  -v open-webui:/app/backend/data \
  ghcr.io/open-webui/open-webui:latest

Open your browser at http://YOUR_SERVER_IP:3000, create an admin account on first launch, and select the default model (e.g., llama3.1:8b). You can now chat and manage prompts, files, and settings.

If the container cannot reach the host, an alternative is host networking:

docker run -d --name open-webui \
  --restart unless-stopped \
  --network=host \
  -e OLLAMA_BASE_URL=http://127.0.0.1:11434 \
  -v open-webui:/app/backend/data \
  ghcr.io/open-webui/open-webui:latest

Step 5: Secure Access

- By default, Open WebUI uses account-based sign-in. In Settings > Admin panel, require authentication for all users.
- If you run on a public server, restrict binding to localhost and use an SSH tunnel:

# bind only to localhost:
docker run -d --name open-webui \
  --restart unless-stopped \
  -p 127.0.0.1:3000:8080 \
  --add-host=host.docker.internal:host-gateway \
  -e OLLAMA_BASE_URL=http://host.docker.internal:11434 \
  -v open-webui:/app/backend/data \
  ghcr.io/open-webui/open-webui:latest

# from your laptop:
ssh -L 3000:localhost:3000 [email protected]

Step 6: Use Your GPU (Optional)

Ollama will use your GPU if supported drivers are installed. For NVIDIA, install the proprietary driver and CUDA libraries (the standard Ubuntu “Additional Drivers” tool works). For AMD, install ROCm per Ubuntu/AMD documentation. Because Open WebUI talks to Ollama’s API, you do not need GPU support inside the Open WebUI container—only in Ollama on the host.

Step 7: Model Management Tips

- List models: ollama list
- Show model metadata: ollama show llama3.1:8b
- Remove a model: ollama rm MODEL_NAME
- Try alternatives: ollama pull mistral, ollama pull qwen2, or ollama pull phi3:mini for lower memory systems.

Troubleshooting

- Port in use: Change the mapped port, for example -p 4000:8080 and open http://YOUR_SERVER_IP:4000.
- Container cannot reach Ollama: Use --network=host or ensure --add-host=host.docker.internal:host-gateway is present.
- Out-of-memory: Pull a smaller/quantized model (e.g., q4_K_M variants) or close other apps.
- Slow responses: Prefer GPU, reduce context length in Open WebUI settings, or choose a smaller model.

Updating and Maintenance

- Update Ollama: rerun the install script, then restart the service: curl -fsSL https://ollama.com/install.sh | sh && sudo systemctl restart ollama.
- Update Open WebUI:

docker pull ghcr.io/open-webui/open-webui:latest
docker stop open-webui && docker rm open-webui
# run again with the same docker run command used earlier

Conclusion

With Ollama and Open WebUI, you can host a private, fast, and flexible chatbot on your own Ubuntu machine. This setup keeps your data local, supports multiple open models, and can leverage your GPU for speed. Whether you are a helpdesk, a developer, or a power user, this self-hosted stack gives you full control over your AI workflow.

Overview

This step-by-step guide shows how to run private, local large language models with Ollama and a modern chat interface using Open WebUI. We will cover installing Ollama on Ubuntu and Windows, enabling GPU acceleration, pulling popular models like Llama 3, and deploying Open WebUI with Docker so you can chat, run tools, and manage prompts from a browser. The result is a fast, secure, and offline-friendly AI stack that you control.

Prerequisites

You will need a 64-bit system, administrator privileges, and at least 16 GB of RAM for 7B–8B models. GPU acceleration is recommended for speed: keep your NVIDIA/AMD/Intel graphics drivers up to date. Ollama listens on port 11434 by default, and Open WebUI will run on port 3000. Ensure your firewall allows local access or your chosen LAN range.

Step 1 — Install Ollama

Ubuntu 22.04/24.04: Install Ollama with the official script, which adds the service and keeps it updated.

curl -fsSL https://ollama.com/install.sh | sh
sudo systemctl enable --now ollama
curl http://127.0.0.1:11434/api/version

You should see a version string from the last command. If not, check the service: sudo systemctl status ollama.

Windows 11/10: Install via the official MSI or Winget, then verify the local API.

winget install Ollama.Ollama
curl http://127.0.0.1:11434/api/version

On Windows, Ollama runs as a user service. If you use a third-party firewall, allow local traffic to port 11434.

Step 2 — Enable GPU Acceleration

GPU acceleration in Ollama is automatic when compatible drivers and runtimes are present. On Linux, install your vendor’s proprietary GPU driver. On Windows, use the latest Game Ready/Studio driver from the GPU vendor. After pulling a model and making a test prompt, watch the Ollama logs. If the run mentions the GPU and performance is high (tokens per second are significantly better than CPU), acceleration is working.

If you suspect CPU fallback, update drivers, make sure your GPU has enough VRAM for the chosen model size, and try a smaller variant (for example, 8B instead of 13B). On laptops with hybrid graphics, set the app/GPU preferences so Ollama can use the discrete GPU.

Step 3 — Pull a Model and Test Locally

Pull a model using the Ollama CLI. Popular, high-quality choices include Llama 3 and Mistral. The first run downloads and prepares weights; subsequent runs start instantly.

# Examples (pick one)
ollama pull llama3:8b
ollama pull llama3.1:8b
ollama pull mistral:7b

Now run a quick prompt:

ollama run llama3:8b
# At the prompt, type:
# What are three creative use cases for local AI at home?

If responses are slow or you see out-of-memory errors, switch to a smaller model or close GPU-intensive applications.

Step 4 — Deploy Open WebUI with Docker

Open WebUI adds a polished browser interface, prompt library, chat history, and extensions like RAG (retrieve and ground answers in your documents). We will connect it to your host’s Ollama instance. The following Docker Compose works on Linux and Windows. It uses host.docker.internal to reach the host-based Ollama API and maps persistent storage for Open WebUI data.

mkdir -p ~/openwebui && cd ~/openwebui
cat > docker-compose.yml <<'YAML'
services:
  openwebui:
    image: ghcr.io/open-webui/open-webui:latest
    container_name: open-webui
    ports:
      - "3000:8080"
    environment:
      - OLLAMA_BASE_URL=http://host.docker.internal:11434
    extra_hosts:
      - "host.docker.internal:host-gateway"
    volumes:
      - open-webui-data:/app/backend/data
    restart: unless-stopped
volumes:
  open-webui-data:
YAML

docker compose up -d

Open your browser and visit http://localhost:3000. Create your account when prompted, pick your default model (for example, llama3:8b), and send a test message. If the UI cannot connect, ensure the Ollama service is running and that your firewall allows local connections to port 11434.

Optional — Run Both Ollama and Open WebUI in Docker

If you prefer everything containerized, you can run Ollama and Open WebUI in the same Compose file. This is convenient on servers. GPU pass-through in Docker requires recent drivers and, on Linux, the NVIDIA Container Toolkit. When in doubt, keep Ollama native and only containerize Open WebUI, as shown above.

Security, Updates, and Backups

Do not expose ports 11434 or 3000 directly to the internet. If you need remote access, place Open WebUI behind a reverse proxy (Nginx, Caddy, or Traefik) with HTTPS and strong authentication, or publish it through a zero-trust tunnel. Inside Open WebUI, enable authentication and limit registration to trusted users. Keep Docker images current by pulling the latest tags and recreating containers. On Ubuntu, the Ollama installer provides updates via its repository; on Windows, check for updates in the app or Winget. Back up ~/.ollama (models and configs) and your open-webui-data volume to preserve chat history and settings.

Troubleshooting

If Open WebUI says “Cannot connect to Ollama,” verify the API at http://127.0.0.1:11434/api/version and confirm your Compose file includes extra_hosts with host-gateway on Linux. On Windows with Docker Desktop, host.docker.internal works out of the box. If GPU acceleration is missing, update drivers, reboot, and try a smaller model. When Docker containers fail to start, check logs with docker logs open-webui and make sure ports 3000 and 11434 are not in use by other applications.

What You Can Do Next

With Ollama and Open WebUI running, you can add multiple models, create custom system prompts, and enable RAG by uploading PDFs or notes so the model answers with context from your documents. You can also script batch prompts via the Ollama HTTP API, integrate with automation tools, or point a browser extension to your local endpoint to replace cloud calls. The stack is private, fast, and easy to maintain—ideal for personal knowledge work or secure team deployments.