Adding Services Guide¶

This guide covers the process of adding new services to the infrastructure, from initial setup to production deployment.

Table of Contents¶

When to Use stackwiz vs Manual Setup
Docker Compose Best Practices
Traefik Integration Patterns
Network Configuration Decisions
Volume and Data Management
Environment Variable Management
Testing New Services
Documentation Requirements
Git Workflow for Infrastructure Changes

When to Use stackwiz vs Manual Setup¶

Use stackwiz when:¶

Service follows standard patterns (web app with database)
You need Traefik integration out of the box
Creating Pocketbase-based applications
You want automatic DNS record creation
You want consistent directory structure

# Example: Adding a new web application
stackwiz mywebapp

# Or with Pocketbase backend
stackwiz --pocketbase mywebapp

# Legacy simple template (basic cases only)
./_templates/mkstack.sh mywebapp

Manual setup when:¶

Complex multi-service stacks (like Supabase)
Special networking requirements
Custom initialization scripts needed
Non-standard port configurations

Docker Compose Best Practices¶

1. Service Naming Conventions¶

Use clear, descriptive names that indicate the service purpose:

services:
  myapp-web:      # Main application
  myapp-db:       # Database
  myapp-redis:    # Cache layer
  myapp-worker:   # Background jobs

2. Version and Structure¶

Always use Compose file version 3.8+ for full feature support:

version: '3.8'

services:
  # Services defined here

networks:
  # Network configuration

volumes:
  # Named volumes

3. Health Checks¶

Implement health checks for critical services:

services:
  myapp-web:
    image: myapp:latest
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s

4. Resource Limits¶

Set appropriate resource constraints:

services:
  myapp-web:
    deploy:
      resources:
        limits:
          cpus: '2.0'
          memory: 2G
        reservations:
          cpus: '0.5'
          memory: 512M

Traefik Integration Patterns¶

Basic Web Service¶

Standard HTTP service with automatic SSL:

services:
  myapp:
    image: myapp:latest
    networks:
      - traefik_proxy
    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=traefik_proxy"
      - "traefik.http.routers.myapp.rule=Host(`myapp.rbnk.uk`)"
      - "traefik.http.routers.myapp.entrypoints=websecure"
      - "traefik.http.routers.myapp.tls=true"
      - "traefik.http.routers.myapp.tls.certresolver=cloudflare"
      - "traefik.http.services.myapp.loadbalancer.server.port=3000"

Service with Multiple Routes¶

Example from Supabase with path-based routing:

labels:
  # Main studio interface
  - "traefik.http.routers.supabase-studio.rule=Host(`supabase.rbnk.uk`)"

  # API endpoints
  - "traefik.http.routers.supabase-rest.rule=Host(`supabase.rbnk.uk`) && PathPrefix(`/rest/v1`)"
  - "traefik.http.routers.supabase-auth.rule=Host(`supabase.rbnk.uk`) && PathPrefix(`/auth/v1`)"

  # Different backend services
  - "traefik.http.services.supabase-rest.loadbalancer.server.port=3000"
  - "traefik.http.services.supabase-auth.loadbalancer.server.port=9999"

WebSocket Support¶

For services requiring WebSocket connections:

labels:
  - "traefik.http.routers.myapp-ws.rule=Host(`myapp.rbnk.uk`) && PathPrefix(`/ws`)"
  - "traefik.http.routers.myapp-ws.service=myapp-ws"
  - "traefik.http.services.myapp-ws.loadbalancer.server.port=3000"
  # Enable sticky sessions for WebSockets
  - "traefik.http.services.myapp-ws.loadbalancer.sticky=true"
  - "traefik.http.services.myapp-ws.loadbalancer.sticky.cookie.name=myapp_ws"

Network Configuration Decisions¶

1. External vs Internal Networks¶

External (traefik_proxy): For services that need public access

networks:
  traefik_proxy:
    external: true

Internal: For service-to-service communication

networks:
  internal:
    driver: bridge
    internal: true

2. Multi-Network Services¶

Services can connect to multiple networks:

services:
  myapp-api:
    networks:
      - traefik_proxy      # Public access
      - supabase_default   # Database access
      - internal           # Internal services

3. Network Aliases¶

Use aliases for service discovery:

services:
  myapp-db:
    networks:
      internal:
        aliases:
          - database
          - postgres

Volume and Data Management¶

1. Named Volumes vs Bind Mounts¶

Named Volumes (Preferred for data):

volumes:
  myapp-data:
    driver: local

services:
  myapp:
    volumes:
      - myapp-data:/var/lib/myapp

Bind Mounts (For config files):

services:
  myapp:
    volumes:
      - ./config/myapp.conf:/etc/myapp/myapp.conf:ro
      - ./data:/data

2. Backup Considerations¶

Structure volumes for easy backup:

services:
  myapp-db:
    volumes:
      # All data in one parent directory
      - ./data/postgres:/var/lib/postgresql/data
      - ./data/backups:/backups

3. Init Scripts Pattern¶

From Supabase bootstrap example:

volumes:
  - ./bootstrap-sql:/docker-entrypoint-initdb.d:ro

Files execute alphabetically:

bootstrap-sql/
├── 00-initial-schema.sql
├── 01-core-tables.sql
├── 02-auth-setup.sql
└── 03-app-tables.sql

Environment Variable Management¶

1. Environment File Structure¶

# Stack-specific .env file
touch myapp/.env
chmod 640 myapp/.env

Example .env:

# Database
POSTGRES_USER=myapp
POSTGRES_PASSWORD=secure_password_here
POSTGRES_DB=myapp

# Application
APP_SECRET_KEY=generate_with_openssl_rand
API_KEY=another_secure_key

# External Services
REDIS_URL=redis://myapp-redis:6379
DATABASE_URL=postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@myapp-db:5432/${POSTGRES_DB}

Use common environment variables:

services:
  myapp-web:
    env_file: .env
    environment:
      - DATABASE_URL=postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@myapp-db:5432/${POSTGRES_DB}

  myapp-worker:
    env_file: .env
    environment:
      - DATABASE_URL=postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@myapp-db:5432/${POSTGRES_DB}

3. Secret Generation¶

# Generate secure passwords
openssl rand -base64 32

# Generate JWT secrets
openssl rand -hex 64

# Generate API keys
uuidgen | tr -d '-'

# For LiteLLM services, use virtual keys instead of master key
# See: LiteLLM Virtual Keys documentation

4. LiteLLM Integration¶

If your service needs LLM access, use virtual keys instead of the master key:

# Create virtual key via API (see LiteLLM Virtual Keys guide)
curl -X POST "https://llm.rbnk.uk/key/generate" \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "budget_id": "service-budget-id",
    "metadata": {"service": "myapp"}
  }'

For complete virtual key implementation details, see: LiteLLM Virtual Keys

Testing New Services¶

1. Local Testing First¶

# Test without Traefik
docker compose -f myapp/docker-compose.yml up

# Check service is accessible
curl http://localhost:3000/health

# Check logs
docker compose -f myapp/docker-compose.yml logs -f

2. Network Testing¶

# Verify network connectivity
docker compose -f myapp/docker-compose.yml exec myapp-web ping myapp-db

# Test database connection
docker compose -f myapp/docker-compose.yml exec myapp-web psql $DATABASE_URL -c "SELECT 1"

3. Traefik Integration Testing¶

# Add to docker-compose.yml and restart
docker compose -f myapp/docker-compose.yml up -d

# Check Traefik picked up the service
curl http://localhost:8080/api/http/routers | jq '.[] | select(.rule | contains("myapp"))'

# Test SSL endpoint
curl -v https://myapp.rbnk.uk/health

4. Load Testing¶

# Simple load test
ab -n 1000 -c 10 https://myapp.rbnk.uk/

# Monitor resources during test
docker stats myapp-myapp-web-1

Documentation Requirements¶

1. Service README¶

Create myapp/README.md:

# MyApp Service

## Overview
Brief description of what this service does

## Architecture
- Main application: Node.js/Python/etc
- Database: PostgreSQL
- Cache: Redis (if applicable)

## Configuration
Key environment variables:
- `DATABASE_URL`: PostgreSQL connection string
- `REDIS_URL`: Redis connection string
- `API_KEY`: External API authentication

## Endpoints
- `https://myapp.rbnk.uk` - Main web interface
- `https://myapp.rbnk.uk/api` - REST API

## Maintenance
### Backup
Automated via `_scripts/backup-myapp.sh`

### Updates
```bash
docker compose -f myapp/docker-compose.yml pull
docker compose -f myapp/docker-compose.yml up -d

Troubleshooting¶

Common issues and solutions

### 2. Update Main Documentation
Add to `/srv/dockerdata/README.md`:
- Service entry in services table
- URL in endpoints section
- Any special considerations

## Git Workflow for Infrastructure Changes

### 1. Initial Repository Setup
```bash
# Configure dual remotes (one-time setup)
cd /srv/dockerdata
/srv/dockerdata/_scripts/setup-git-remotes.sh

# This creates:
# - origin → Gitea (primary)
# - gitea → Gitea (explicit)
# - github → GitHub (optional mirror)

2. Feature Branch Workflow¶

# Create feature branch
git checkout -b add-myapp-service

# Make changes
vim myapp/docker-compose.yml
vim myapp/.env.example  # Never commit real .env

# Test thoroughly
docker compose -f myapp/docker-compose.yml up -d

# Commit with clear message
git add myapp/
git commit -m "Add MyApp service with PostgreSQL and Redis

- Web interface at myapp.rbnk.uk
- Includes automated backups
- Integrated with Traefik for SSL"

# Push to Gitea (primary)
git push -u origin add-myapp-service

# Or push to all remotes
git push-all

3. Environment File Handling¶

# Create example file
cp myapp/.env myapp/.env.example
# Edit to remove sensitive values
vim myapp/.env.example

# Add to .gitignore
echo "myapp/.env" >> .gitignore

4. Pre-deployment Checklist¶

[ ] Service starts successfully
[ ] Health checks pass
[ ] Traefik routing works
[ ] SSL certificate obtained
[ ] Backups configured
[ ] Monitoring/alerts set up
[ ] Documentation complete
[ ] .env.example created
[ ] No secrets in commits

5. Deployment Process¶

# On production server
git pull origin main

# Copy environment file
cp myapp/.env.example myapp/.env
vim myapp/.env  # Add real values

# Deploy
docker compose -f myapp/docker-compose.yml up -d

# Verify
docker compose -f myapp/docker-compose.yml ps
curl https://myapp.rbnk.uk/health

Common Patterns and Examples¶

Database-Backed Web App¶

See _templates/stack-template/ for the standard pattern

Microservices Stack¶

See supabase/ for complex multi-service example

Simple Static Site¶

version: '3.8'
services:
  static-site:
    image: nginx:alpine
    volumes:
      - ./public:/usr/share/nginx/html:ro
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
    networks:
      - traefik_proxy
    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=traefik_proxy"
      - "traefik.http.routers.static.rule=Host(`static.rbnk.uk`)"
      - "traefik.http.routers.static.entrypoints=websecure"
      - "traefik.http.routers.static.tls=true"
      - "traefik.http.routers.static.tls.certresolver=cloudflare"

API with Rate Limiting¶

labels:
  # Standard routing
  - "traefik.http.routers.api.rule=Host(`api.rbnk.uk`)"

  # Rate limiting middleware
  - "traefik.http.middlewares.api-ratelimit.ratelimit.average=100"
  - "traefik.http.middlewares.api-ratelimit.ratelimit.burst=200"
  - "traefik.http.routers.api.middlewares=api-ratelimit"

Remember: Always test thoroughly in development before deploying to production!