Open WebUI Advanced: RAG, Pipelines, and Custom System Prompts

In Open WebUI:

1. Go to Workspace → Documents
2. Click Upload a file
3. Upload your runbooks, network diagrams (as text), configuration notes, or wiki exports

Supported formats: PDF, TXT, DOCX, Markdown, and more. Open WebUI automatically chunks and embeds the documents using a local embedding model.

In any chat:

1. Click the + (attachment) button
2. Select Add Documents from Library
3. Choose the documents you want to include

The model now has access to the document content for this conversation. Ask questions like “What is our CT naming convention?” or “Which node runs the monitoring stack?” and the model will retrieve and cite the relevant sections.

The default embedding model is adequate but nomic-embed-text is significantly better for technical documentation:

# Inside the Open WebUI container (CT 103) or any Ollama node
ollama pull nomic-embed-text

Then in Open WebUI: Admin → Settings → Documents → Embedding Model: select nomic-embed-text:latest.

Re-index your existing documents: Workspace → Documents → select all → Re-index.

Go to Workspace → Models → Create a model (or Assistants in newer versions).

System prompt:

You are a Proxmox VE expert for the MyOfficeLab homelab cluster.

Cluster details:
- 4 nodes: pvelab01 (10.0.0.70), pvelab02 (.71), pvelab03 (.72), pvelab04 (.73)
- CT naming convention: sequential IDs starting from 100
- Storage: local-lvm for containers, local for ISOs and templates
- API token: myofficelab-app@pve!app-token (PVEAdmin)
- Monitoring: Prometheus + Grafana in CT 105

When giving commands, always specify which node to run them on. Prefer pct exec for running commands inside containers. Always confirm destructive operations before executing.

Click Save.

Create another assistant:

System prompt:

You are a technical writer for the Build a Homelab blog at buildahomelab.dev.

Style guide:
- Audience: intermediate-level homelab enthusiasts who know Linux basics
- Tone: direct and practical — no padding, no "as we can see" filler
- Structure: use Step components for task sequences, Callout for warnings/tips
- Code: always in Terminal blocks with descriptive titles
- Length: detailed enough to reproduce the result, not longer
- Links: include Amazon affiliate links for recommended hardware where natural

When asked to draft a post, follow the established format with frontmatter, imports, and component usage matching existing posts.

In Open WebUI: Account → API Keys → Generate new key

Copy the key — it’s only shown once.

curl http://10.0.0.69:8080/api/openai/chat/completions -H "Authorization: Bearer YOUR_API_KEY" -H "Content-Type: application/json" -d '{
  "model": "llama3.2",
  "messages": [{"role": "user", "content": "What is 2+2?"}],
  "stream": false
}'

The response matches the OpenAI API format exactly — same JSON structure as the real API.

pip install openai

python3 << 'EOF'
from openai import OpenAI

client = OpenAI(
  base_url="http://10.0.0.69:8080/api/openai",
  api_key="YOUR_API_KEY"
)

response = client.chat.completions.create(
  model="qwen2.5-coder:14b",
  messages=[
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "Write a one-line bash command to count running LXC containers."}
  ]
)
print(response.choices[0].message.content)
EOF

This pattern lets you use local inference anywhere you’d use the OpenAI API — for free, with no rate limits, and without sending data to external servers.

SearXNG is a self-hosted meta-search engine that aggregates results from Google, DuckDuckGo, and others without tracking.

Create a container for SearXNG (CT 113, Ubuntu 24.04, 512MB RAM), then:

apt-get update && apt-get install -y python3-venv git

git clone https://github.com/searxng/searxng.git /opt/searxng
cd /opt/searxng
python3 -m venv venv
venv/bin/pip install -e .

# Generate a secret key
sed -i "s/ultrasecretkey/$(openssl rand -hex 32)/" searx/settings.yml

# Start SearXNG
venv/bin/python searx/webapp.py

In Open WebUI: Admin → Settings → Web Search:

• Enable Web Search
• Set Search Engine URL to http://10.0.0.82:8080/search?q=<query>&format=json (your SearXNG container IP)

Now in any chat, click the search icon (globe) to enable web search for that message. The model will retrieve fresh search results and incorporate them into its response.

In any chat, click the Controls panel (sliders icon):

For code generation tasks on CPU: Temperature 0.2, Top P 0.9, Context 4096 gives the best results — deterministic enough to write correct code, fast enough to be usable.