Common Issues and Solutions¶

This guide covers frequently encountered issues and their solutions across the rbnk.uk infrastructure.

Quick Diagnostics¶

Before diving into specific issues, run these quick checks:

# Check all services status
docker ps -a

# View recent logs
docker logs --tail 50 CONTAINER_NAME

# Check disk space
df -h

# Verify network connectivity
docker network ls
ping 8.8.8.8

Service Access Issues¶

Cannot Access Service via Browser¶

Symptoms: Service running but getting connection refused or timeout

Solutions: 1. Check Traefik routing:

docker logs traefik --tail 100 | grep SERVICE_NAME

Verify DNS record:
```
dig SERVICE_NAME.rbnk.uk
```
Check Proxmox port forwarding (if external):
Ensure ports 80/443 are forwarded from Proxmox host to VM
See Proxmox Port Forwarding

Verify Traefik labels:

docker inspect CONTAINER_NAME | grep -A 10 Labels

502 Bad Gateway Error¶

Common causes: - Service not running - Wrong port configuration - Network connectivity issues

Solutions:

# Check if service is running
docker ps | grep SERVICE_NAME

# Restart service
docker compose -f SERVICE/docker-compose.yml restart

# Check service logs
docker compose -f SERVICE/docker-compose.yml logs --tail 50

Container Issues¶

Container Keeps Restarting¶

Check logs first:

docker logs CONTAINER_NAME --tail 100

Common causes: 1. Permission issues: Check file ownership 2. Port conflicts: netstat -tulpn | grep PORT 3. Missing environment variables: Review .env file 4. Database connection issues: Verify database is running

Container Won't Start¶

Solutions:

# Remove and recreate
docker compose -f SERVICE/docker-compose.yml down
docker compose -f SERVICE/docker-compose.yml up -d

# Check for image issues
docker compose -f SERVICE/docker-compose.yml pull

Database Issues¶

PostgreSQL Connection Refused¶

For Supabase services:

# Check if database is running
docker ps | grep supa-db

# Test connection
docker exec -it supa-db psql -U postgres

# Check logs
docker logs supa-db --tail 50

Database Permission Errors¶

Fix ownership (as seen with Gitea):

# Inside container
docker exec -u root CONTAINER_NAME chown -R postgres:postgres /path/to/data

# From host
docker compose -f SERVICE/docker-compose.yml down
sudo chown -R 999:999 ./SERVICE/data
docker compose -f SERVICE/docker-compose.yml up -d

Network Issues¶

Service Can't Reach Another Service¶

Check Docker networks:

# List networks
docker network ls

# Inspect network
docker network inspect NETWORK_NAME

# Test connectivity
docker exec SERVICE1 ping SERVICE2

Common fix: Ensure both services are on the same network in docker-compose.yml

DNS Resolution Issues¶

Internal DNS:

# Test from container
docker exec CONTAINER nslookup OTHER_SERVICE

External DNS:

# Flush DNS cache
sudo resolvectl flush-caches

# For Tailscale DNS issues
tailscale status
sudo systemctl restart systemd-resolved

SSL Certificate Issues¶

See detailed guide: SSL Certificate Issues

Quick fixes: 1. Check certificate resolver name (should be "cf" not "cloudflare") 2. Verify Cloudflare API credentials in Traefik 3. Check rate limits haven't been hit

Storage Issues¶

Disk Space Full¶

Quick cleanup:

# Check usage
df -h
du -sh /srv/dockerdata/*

# Clean Docker
docker system prune -a --volumes

# Remove old logs
find /srv/dockerdata -name "*.log" -mtime +30 -delete

# Clean backup directory
ls -lah /srv/dockerdata/_backup/

Permission Denied Errors¶

Fix ownership:

# Find correct user
docker exec CONTAINER id

# Fix from host
sudo chown -R UID:GID ./SERVICE/data

Performance Issues¶

Service Running Slowly¶

Check resource usage:
```
docker stats CONTAINER_NAME
htop
```
View Grafana dashboards: https://grafana.rbnk.uk

Check logs for errors:

docker logs CONTAINER --since 1h | grep -i error

High Memory Usage¶

# Check memory limits
docker inspect CONTAINER | grep -A 5 Memory

# Add limits in docker-compose.yml
services:
  SERVICE:
    mem_limit: 512m

Git/Version Control Issues¶

Cannot Push to Gitea¶

Check SSH key: ssh -T [email protected]
Verify repository permissions
Check Gitea logs: docker logs gitea

Sync Issues with GitHub¶

Check sync job:

sudo systemctl status gitea-sync
sudo journalctl -u gitea-sync -n 50

Notification Issues¶

Notifications Not Sending¶

Test with helper script:

notify "Test" "Test message" normal test

Check Apprise configuration:

curl https://apprise.rbnk.uk/json/urls/apprise

Verify ntfy authentication:

curl -u admin:PASSWORD https://ntfy.rbnk.uk/test \
  -d "Test message"

Emergency Recovery¶

Service Completely Broken¶

Restore from backup:

# Stop service
docker compose -f SERVICE/docker-compose.yml down

# Restore data
cp -r /srv/dockerdata/_backup/SERVICE/* ./SERVICE/

# Start service
docker compose -f SERVICE/docker-compose.yml up -d

Restore from R2:

docker exec rclone rclone copy \
  cloudflare:infrastructure-backups/latest/SERVICE \
  /srv/dockerdata/SERVICE

Complete System Recovery¶

See Backup & Restore Guide

Getting Help¶

Check logs first - Most issues are evident in logs
Review documentation - Service-specific docs often have troubleshooting sections
Monitor dashboards - Grafana often shows issues before they become critical
Test systematically - Isolate the problem by testing each component

Prevention¶

Regular backups - Automated daily backups to R2
Monitor alerts - Set up AlertManager notifications
Keep updated - Regular Docker image updates
Document changes - Track what was changed and when