Ollama Setup Guide - Dual Disk Configuration¶

Overview¶

Ollama is a tool for running Large Language Models (LLMs) locally. This guide documents how Ollama was configured on a dual-disk system to ensure models are stored on the data disk, not the system disk.

Why This Configuration?¶

The Storage Challenge¶

LLM models are large: - Small models: 2-4GB (e.g., Phi-2, TinyLlama) - Medium models: 7-13GB (e.g., Llama 2 7B, Mistral 7B) - Large models: 30-70GB (e.g., Llama 2 70B, Mixtral)

With the system disk at 79% capacity (5.7GB free), it's important to store models on the data disk to avoid filling the system partition.

Our Solution¶

System Disk (/dev/sda - 60GB)        Data Disk (/dev/sdb - 200GB)
├── / (28GB partition, 79% full)     └── /srv/dockerdata (28GB free)
├── /usr/local/bin/ollama                 └── ollama/
└── /etc/systemd/system/                      ├── models/ ← Models here
                                              └── config/

Installation Process¶

1. Directory Setup¶

# Create Ollama directories on data disk
sudo mkdir -p /srv/dockerdata/ollama/{models,config}
sudo chown -R ollama:ollama /srv/dockerdata/ollama

Why these directories? - models/: Stores downloaded LLM models (the large files) - config/: Future configuration files

2. Custom Installation Script¶

We created /srv/dockerdata/ollama/install-ollama.sh:

#!/bin/bash
# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# Configure systemd to use our data directory
sudo mkdir -p /etc/systemd/system/ollama.service.d
sudo tee /etc/systemd/system/ollama.service.d/override.conf > /dev/null <<EOF
[Service]
Environment="OLLAMA_MODELS=/srv/dockerdata/ollama/models"
Environment="OLLAMA_HOST=0.0.0.0"
EOF

# Create convenience symlink
ln -sf /srv/dockerdata/ollama ~/ollama-data

# Reload and start service
sudo systemctl daemon-reload
sudo systemctl restart ollama

3. Key Configuration Decisions¶

Systemd Override Pattern¶

Instead of modifying the main service file, we use systemd's override mechanism:

/etc/systemd/system/ollama.service.d/override.conf

Why? - Survives Ollama updates - Clear separation of custom config - Easy to revert

Environment Variables¶

Environment="OLLAMA_MODELS=/srv/dockerdata/ollama/models"
Environment="OLLAMA_HOST=0.0.0.0"

Why these settings? - OLLAMA_MODELS: Redirects model storage to data disk - OLLAMA_HOST=0.0.0.0: Allows access from Docker containers (needed for Open WebUI)

4. Permission Challenges¶

Initial attempt failed with:

Error: mkdir /srv/dockerdata/ollama/models/blobs: permission denied

Root Cause: Ollama service runs as user ollama, not root or rkb

Solution:

sudo chown -R ollama:ollama /srv/dockerdata/ollama

Lesson: Always check which user a service runs as!

Integration with Open WebUI¶

Docker Networking Challenge¶

Open WebUI runs in Docker, Ollama runs on the host. They can't use localhost to communicate.

Solution¶

In /srv/dockerdata/open-webui/.env:

OLLAMA_BASE_URL=http://172.17.0.1:11434

Why 172.17.0.1? - This is Docker's default gateway IP - It's how containers reach the host - Port 11434 is Ollama's API port

Convenience Features¶

Symlink for Easy Access¶

~/ollama-data → /srv/dockerdata/ollama

Why? - Quick access without typing long paths - Easy to check model sizes: ls -lh ~/ollama-data/models

User Environment Variables¶

Added to ~/.bashrc:

export OLLAMA_MODELS=/srv/dockerdata/ollama/models
export OLLAMA_HOST=0.0.0.0

Why? For consistency when running Ollama CLI commands

Common Operations¶

Check Disk Usage¶

# See how much space models are using
du -sh /srv/dockerdata/ollama/models

# Check available space
df -h /srv/dockerdata

List Models¶

ollama list

Pull Models (Storage-Aware)¶

# Small model (good for testing)
ollama pull tinyllama      # ~637MB

# Medium models
ollama pull llama3.2       # ~2GB
ollama pull mistral        # ~4.1GB

# Check space before pulling large models!
df -h /srv/dockerdata
ollama pull llama2:70b     # ~40GB - ensure space available

Monitor Service¶

# Check status
sudo systemctl status ollama

# View logs
sudo journalctl -u ollama -f

# Restart if needed
sudo systemctl restart ollama

Troubleshooting¶

Service Won't Start¶

# Check logs for permission errors
sudo journalctl -u ollama -n 50

# Verify ownership
ls -la /srv/dockerdata/ollama

# Fix if needed
sudo chown -R ollama:ollama /srv/dockerdata/ollama

Can't Connect from Docker¶

# Test from host
curl http://localhost:11434/api/tags

# Test Docker gateway
curl http://172.17.0.1:11434/api/tags

# Check Ollama is listening on all interfaces
ss -tlnp | grep 11434

Models Not Saving to Data Disk¶

# Verify environment variable
sudo systemctl show ollama | grep Environment

# Should show:
# Environment=OLLAMA_MODELS=/srv/dockerdata/ollama/models

Best Practices¶

1. Check Space Before Pulling Models¶

df -h /srv/dockerdata

2. Start with Small Models¶

Test with tiny models first to ensure everything works

3. Regular Cleanup¶

# Remove unused models
ollama rm model-name

# List all models with sizes
ollama list

4. Monitor Disk Usage¶

Models can accumulate quickly. Set up alerts when disk usage exceeds 80%

Architecture Decisions Explained¶

Why Not Use Docker for Ollama?¶

Direct GPU Access: Easier on bare metal (when GPU available)
Performance: No containerization overhead
Simplicity: One less container to manage
Storage: Systemd override simpler than Docker volume mounts

Why Systemd Override Instead of Config File?¶

Ollama doesn't use config files: Environment variables only
Systemd integration: Proper service management
Persistence: Survives reboots and updates

Why This Directory Structure?¶

/srv/dockerdata/ollama/
├── models/          # Large files, need data disk
├── config/          # Future expansion
└── install-ollama.sh # Documentation as code

Keeps everything organized and discoverable.

Future Considerations¶

When System Disk Gets Upgraded¶

The symlink approach means no changes needed - Ollama will continue using the data disk.

If Moving to Container¶

Would need to: 1. Create Ollama container with volume mount 2. Update Open WebUI to use container name instead of gateway IP 3. Migrate existing models

GPU Addition¶

When GPU is added: 1. Install NVIDIA drivers 2. Ollama will auto-detect and use GPU 3. No configuration changes needed

Summary¶

This setup ensures: - ✅ Models stored on data disk (184GB available) - ✅ System disk protected from filling up - ✅ Accessible from Docker containers - ✅ Survives updates and reboots - ✅ Easy management via symlinks

The key insight: Always consider storage implications before installing tools that download large files. In a dual-disk setup, explicitly configure where large data goes.