LiteLLM Virtual Keys Implementation¶

Overview¶

Virtual keys in LiteLLM provide isolated API access with individual cost tracking, security boundaries, and budget controls. This implementation replaces the use of master keys in production services with dedicated virtual keys that offer enhanced security, monitoring, and cost management capabilities.

Why Virtual Keys?¶

Cost Isolation: Each service has its own budget limits and usage tracking
Security Boundary: Services can only access their allocated resources
Usage Monitoring: Real-time tracking per service for cost optimization
Budget Protection: Automatic spending limits prevent cost overruns
Audit Trail: Clear visibility into which service is consuming resources

Architecture¶

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   n8n Service   │    │  Open WebUI     │    │  Browser-use    │
│                 │    │                 │    │                 │
│ Virtual Key:    │    │ Virtual Key:    │    │ Virtual Key:    │
│ sk-7UauGn...    │    │ sk-ddZoN7...    │    │ sk-3EnNwo...    │
│ Budget: $20/mo  │    │ Budget: $15/mo  │    │ Budget: $10/mo  │
└─────────┬───────┘    └─────────┬───────┘    └─────────┬───────┘
          │                      │                      │
          └──────────────────────┼──────────────────────┘
                                 │
                    ┌────────────▼────────────┐
                    │     LiteLLM Proxy      │
                    │                        │
                    │ Master Key (Admin):    │
                    │ sk-Bl4ckMagic          │
                    │                        │
                    │ Database: PostgreSQL   │
                    │ Cache: Redis           │
                    └────────────┬───────────┘
                                 │
                    ┌────────────▼────────────┐
                    │    AI Providers        │
                    │ OpenAI | Anthropic     │
                    │ Gemini | Groq | etc.   │
                    └─────────────────────────┘

Virtual Keys Generated¶

Production Virtual Keys¶

Service	Virtual Key	Monthly Budget	Team/Priority	Purpose
n8n Workflows	`sk-7UauGnGKN-YdD3R3TAmE7g`	$20	operations/High	Workflow automation, daily log summaries, infrastructure monitoring
Open WebUI	`sk-ddZoN7gANtxebnt2foO04A`	$15	users/Medium	Interactive AI chat interface for end users
Browser-use	`sk-3EnNwo4pg1s4yOEaFDGHYA`	$10	automation/Low	Automated browser tasks and web scraping
Testing/Dev	`sk-hOsuTU0dMn0IudR8-7DiDg`	$5	development/Low	Development testing and experimentation

Master Key (Administrative Use Only)¶

Key Type	Key Value	Access Level	Usage
Master Key	`sk-Bl4ckMagic`	Full Admin	Dashboard access, key management, system configuration

Security Note: The master key should never be used in application configurations. It provides full administrative access to the LiteLLM system and should only be used for dashboard access and key management.

Service Configurations¶

n8n Workflows Configuration¶

File: /srv/dockerdata/n8n-compose/.env

# LiteLLM Configuration (using virtual key for n8n workflows)
LITELLM_API_KEY=sk-7UauGnGKN-YdD3R3TAmE7g

Usage Context: - Workflow automation and AI-powered infrastructure monitoring - Daily log summary generation using Gemini 2.5 Flash Lite - Cost tracking for automated AI operations - Budget limit: $20/month (High priority)

Open WebUI Configuration¶

Setup: Configured through the Open WebUI admin interface

Access Admin Settings:
Navigate to https://openwebui.rbnk.uk
Go to Admin Panel → Settings → Connections

OpenAI API Configuration:

API Base URL: http://litellm:4000/v1
API Key: sk-ddZoN7gANtxebnt2foO04A

Model Access:
All LiteLLM configured models available
Real-time cost tracking per user session
Budget limit: $15/month (Medium priority)

Benefits: - User-level cost tracking through virtual key - Isolated budget prevents runaway costs - Seamless access to multiple AI providers

Browser-use Configuration¶

File: /srv/dockerdata/browser-use/.env

# LiteLLM Configuration (uses internal Docker network with virtual key)
LITELLM_API_URL=http://litellm:4000
LITELLM_API_KEY=sk-3EnNwo4pg1s4yOEaFDGHYA

Usage Context: - Automated browser automation tasks - Web scraping and data extraction - AI-powered web interactions - Budget limit: $10/month (Low priority)

Testing/Development Configuration¶

Virtual Key: sk-hOsuTU0dMn0IudR8-7DiDg

Usage: - Development testing and experimentation - New service integration testing - Cost-controlled development environment - Budget limit: $5/month (Low priority)

Key Management and Administration¶

Creating Virtual Keys¶

Virtual keys are created through the LiteLLM admin dashboard:

Access Dashboard: https://llm.rbnk.uk/ui
Login: Use master key sk-Bl4ckMagic
Navigate: Go to "Keys" section
Create Key: Set budget, team, and permissions

CLI Method (Alternative):

# Create a new virtual key via API
curl -X POST "https://llm.rbnk.uk/key/new" \
  -H "Authorization: Bearer sk-Bl4ckMagic" \
  -H "Content-Type: application/json" \
  -d '{
    "key_alias": "service-name",
    "team_id": "operations",
    "max_budget": 20.00,
    "duration": "monthly"
  }'

Monitoring Key Usage¶

Real-time Dashboard¶

Access comprehensive usage statistics at https://llm.rbnk.uk/ui: - Current spend per key - Request volume and patterns
- Model usage distribution - Cost trends and projections

API Monitoring¶

# Check specific key information
curl -X GET "https://llm.rbnk.uk/key/info" \
  -H "Authorization: Bearer sk-7UauGnGKN-YdD3R3TAmE7g"

# Get usage statistics
curl -X GET "https://llm.rbnk.uk/spend" \
  -H "Authorization: Bearer sk-Bl4ckMagic" \
  -G -d "api_key=sk-7UauGnGKN-YdD3R3TAmE7g"

# List all keys (admin only)
curl -X GET "https://llm.rbnk.uk/key/info/all" \
  -H "Authorization: Bearer sk-Bl4ckMagic"

Budget Monitoring Commands¶

# Check current spend across all services
./scripts/check-litellm-spend.sh

# Generate cost report
curl -X GET "https://llm.rbnk.uk/spend/report" \
  -H "Authorization: Bearer sk-Bl4ckMagic"

# Check budget alerts
curl -X GET "https://llm.rbnk.uk/alerts" \
  -H "Authorization: Bearer sk-Bl4ckMagic"

Budget Management¶

Setting Budget Limits¶

# Update budget for existing key
curl -X POST "https://llm.rbnk.uk/key/update" \
  -H "Authorization: Bearer sk-Bl4ckMagic" \
  -H "Content-Type: application/json" \
  -d '{
    "key": "sk-7UauGnGKN-YdD3R3TAmE7g",
    "max_budget": 25.00,
    "duration": "monthly"
  }'

Budget Alert Configuration¶

Budget alerts are configured in /srv/dockerdata/litellm-test/litellm-config.yaml:

alerting:
  - alerting_type: "budget"
    alerting_threshold: 0.8    # Alert at 80% budget
    webhook_url: "https://n8n.rbnk.uk/webhook/litellm-budget-alert"

Benefits of Virtual Keys Implementation¶

1. Cost Tracking and Control¶

Granular Visibility: Track costs per service rather than aggregate spending
Budget Protection: Automatic spending limits prevent cost overruns
Real-time Monitoring: Immediate visibility into usage patterns and costs
Predictable Budgets: Monthly limits provide financial predictability

2. Security Isolation¶

Principle of Least Privilege: Each service only has access to its allocated resources
Key Rotation: Virtual keys can be rotated without affecting other services
Audit Trail: Clear tracking of which service made which API calls
Access Control: Fine-grained permissions per service

3. Operational Benefits¶

Simplified Management: Centralized key management through LiteLLM dashboard
Service Independence: Services can be managed independently
Easy Scaling: New services get their own virtual keys with appropriate budgets
Cost Optimization: Identify high-usage services for optimization

4. Monitoring and Alerting¶

Proactive Alerts: Notifications when services approach budget limits
Usage Analytics: Detailed metrics for capacity planning
Performance Tracking: Model performance and latency per service
Trend Analysis: Historical usage patterns for forecasting

Usage Guidelines¶

For Service Administrators¶

Monitor Usage Regularly: Check dashboard weekly for cost trends
Set Appropriate Budgets: Size budgets based on expected usage
Review Alerts: Respond promptly to budget notifications
Update Keys Safely: Use proper rotation procedures when updating keys

For Developers¶

Use Correct Keys: Always use service-specific virtual keys, never master key
Handle Rate Limits: Implement proper retry logic for rate limit responses
Log Usage: Include cost tracking in application monitoring
Test Budget Limits: Verify graceful handling when budgets are exceeded

Best Practices¶

# Test virtual key access
curl -X POST "https://llm.rbnk.uk/v1/chat/completions" \
  -H "Authorization: Bearer sk-7UauGnGKN-YdD3R3TAmE7g" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "Test message"}],
    "max_tokens": 10
  }'

# Verify budget status before high-usage operations
curl -X GET "https://llm.rbnk.uk/key/info" \
  -H "Authorization: Bearer sk-7UauGnGKN-YdD3R3TAmE7g" \
  | jq '.spend, .max_budget'

Troubleshooting¶

Common Issues and Solutions¶

1. "Invalid API Key" Error¶

Symptoms: 401 Unauthorized responses Solutions:

# Verify key is valid
curl -X GET "https://llm.rbnk.uk/key/info" \
  -H "Authorization: Bearer sk-VIRTUAL-KEY-HERE"

# Check key status in dashboard
# Navigate to https://llm.rbnk.uk/ui → Keys section

# Verify service configuration
grep -r "LITELLM_API_KEY\|OPENAI_API_KEY" /srv/dockerdata/*/

2. Budget Exceeded Errors¶

Symptoms: 429 "Budget exceeded" responses Solutions:

# Check current spend
curl -X GET "https://llm.rbnk.uk/spend" \
  -H "Authorization: Bearer sk-Bl4ckMagic" \
  -G -d "api_key=sk-VIRTUAL-KEY-HERE"

# Increase budget if justified
curl -X POST "https://llm.rbnk.uk/key/update" \
  -H "Authorization: Bearer sk-Bl4ckMagic" \
  -H "Content-Type: application/json" \
  -d '{"key": "sk-VIRTUAL-KEY-HERE", "max_budget": 30.00}'

# Wait for budget reset (monthly cycle)

3. High Usage Alerts¶

Symptoms: Frequent budget warning notifications Investigation Steps:

# Analyze usage patterns
curl -X GET "https://llm.rbnk.uk/spend/detailed" \
  -H "Authorization: Bearer sk-Bl4ckMagic" \
  -G -d "api_key=sk-VIRTUAL-KEY-HERE"

# Check for unusual spikes
curl -X GET "https://llm.rbnk.uk/usage/hourly" \
  -H "Authorization: Bearer sk-VIRTUAL-KEY-HERE"

# Review application logs for excessive API calls
docker logs n8n-compose_n8n_1 --since=24h | grep -i "litellm\|api"

4. Service Connection Issues¶

Symptoms: Services can't connect to LiteLLM Debugging:

# Test internal connectivity
docker exec n8n-compose_n8n_1 curl -I http://litellm:4000/health

# Check network configuration
docker network inspect traefik_proxy
docker network inspect supabase_internal

# Verify container status
docker ps | grep litellm
docker logs litellm --tail 50

5. Virtual Key Creation Failures¶

Symptoms: Cannot create new virtual keys Solutions:

# Verify master key access
curl -X GET "https://llm.rbnk.uk/health" \
  -H "Authorization: Bearer sk-Bl4ckMagic"

# Check database connectivity
docker exec litellm psql $DATABASE_URL -c "SELECT 1;"

# Review LiteLLM logs for errors
docker logs litellm --tail 100 | grep -i "error\|fail"

Debugging Commands¶

# Check all virtual keys status
curl -X GET "https://llm.rbnk.uk/key/info/all" \
  -H "Authorization: Bearer sk-Bl4ckMagic" | jq

# Monitor real-time usage
curl -X GET "https://llm.rbnk.uk/health/detailed" \
  -H "Authorization: Bearer sk-Bl4ckMagic"

# Test model access with virtual key
curl -X POST "https://llm.rbnk.uk/v1/models" \
  -H "Authorization: Bearer sk-VIRTUAL-KEY-HERE"

# Check service configurations
for service in n8n-compose open-webui browser-use; do
  echo "=== $service ==="
  grep -i litellm /srv/dockerdata/$service/.env 2>/dev/null || echo "No LiteLLM config found"
done

Security Notes¶

Key Protection Best Practices¶

Environment Variables: Store virtual keys in .env files with proper permissions (640)
No Hardcoding: Never commit virtual keys to version control
Rotation Strategy: Regularly rotate keys, especially for high-usage services
Access Monitoring: Regularly audit key usage for unauthorized access

Security Configuration¶

# Secure .env file permissions
chmod 640 /srv/dockerdata/*//.env
chown root:docker /srv/dockerdata/*//.env

# Audit key access
grep -r "sk-" /srv/dockerdata/docs/ || echo "No keys found in docs (good!)"

# Monitor for exposed keys
curl -s https://llm.rbnk.uk/metrics | grep -i "unauthorized\|invalid" || true

Network Security¶

Internal Communication: Services communicate via Docker networks
SSL Termination: External access only through Traefik with SSL
Access Control: Virtual keys provide service-level isolation
Audit Logging: All API calls are logged with key attribution

Key Lifecycle Management¶

Creation: Through admin dashboard with appropriate budgets
Usage: Service-specific implementation with proper error handling
Monitoring: Regular review of usage patterns and costs
Rotation: Planned rotation schedule for security
Revocation: Immediate revocation capability for compromised keys

Integration Examples¶

Adding a New Service¶

When adding a new service that needs LiteLLM access:

Create Virtual Key:

curl -X POST "https://llm.rbnk.uk/key/new" \
  -H "Authorization: Bearer sk-Bl4ckMagic" \
  -H "Content-Type: application/json" \
  -d '{
    "key_alias": "new-service",
    "team_id": "automation",
    "max_budget": 10.00,
    "duration": "monthly",
    "models": ["gpt-4o-mini", "gemini-2.5-flash"]
  }'

Configure Service:

# Add to service .env file
echo "LITELLM_API_KEY=sk-NEW-VIRTUAL-KEY" >> /srv/dockerdata/new-service/.env

# Set appropriate permissions
chmod 640 /srv/dockerdata/new-service/.env

Test Integration:

# Verify key works
curl -X POST "http://litellm:4000/v1/chat/completions" \
  -H "Authorization: Bearer sk-NEW-VIRTUAL-KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "Test"}],
    "max_tokens": 10
  }'

Model-Specific Access Control¶

Restrict virtual keys to specific models:

curl -X POST "https://llm.rbnk.uk/key/update" \
  -H "Authorization: Bearer sk-Bl4ckMagic" \
  -H "Content-Type: application/json" \
  -d '{
    "key": "sk-VIRTUAL-KEY-HERE",
    "models": ["gpt-4o-mini", "gemini-2.5-flash"],
    "max_budget": 15.00
  }'

LiteLLM Service Documentation - Complete LiteLLM setup and configuration
n8n Service Documentation - n8n workflow automation platform
Open WebUI Documentation - AI chat interface
Browser-use Documentation - Automated browser tasks
Cost Monitoring Guide - Infrastructure cost tracking
Security Best Practices - API key management guidelines

Last Updated: 2025-12-08
Virtual Keys Generated: 2025-12-08
Next Review: 2025-01-08