Paperless-AI Document Management Service Documentation¶

Table of Contents¶

Overview
Architecture
Why Paperless-AI
Configuration Details
Access Information
Common Operations
Document Processing
RAG Service Integration
Backup and Restore
Troubleshooting Guide
Integration with Infrastructure
Security Considerations
Monitoring and Logging

Overview¶

Paperless-AI is an AI-powered extension for Paperless-ngx that brings automatic document classification, smart tagging, and semantic search using OpenAI-compatible APIs and Ollama. It enables fully automated document workflows, contextual chat, and powerful customization through an intuitive web interface.

In this infrastructure, Paperless-AI is deployed as a containerized service providing: - Automated Document Processing with AI-powered classification and tagging - RAG-Based AI Chat for natural language document search and Q&A - Manual Processing via web interface for AI tagging when needed - Multi-Provider AI Support including OpenAI, Ollama, DeepSeek, and more - Semantic Document Search that understands full document context - Smart Workflows with customizable automation rules

The service is exposed via the domain paperless-ai.rbnk.uk with SSL termination handled by Traefik reverse proxy.

Architecture¶

Core Components¶

Paperless-AI Service
├── Node.js Application (Port 3000)
│   ├── Web Interface (EJS templates)
│   ├── AI Service Factory (Multi-provider support)
│   ├── Document Processing Engine
│   └── RAG Service Integration
├── Data Persistence
│   ├── Configuration Storage (/app/data)
│   ├── Document Processing Results
│   └── Chat History & RAG Index
└── External Integrations
    ├── Paperless-ngx API
    ├── AI Provider APIs (OpenAI, Ollama, etc.)
    └── RAG Service (Port 8000)

Network Architecture¶

Container: paperless-ai on traefik_proxy network
Internal Port: 3000 (configurable via PAPERLESS_AI_PORT)
External Access: Port 3003 (localhost only) + Traefik HTTPS
RAG Service: Internal communication on port 8000
Paperless-ngx: API integration via configured endpoints

Storage Architecture¶

Named Volume: paperless-ai_data mounted at /app/data
Configuration: Environment variables stored in /app/data/.env
Document Index: RAG embeddings and search index
Chat History: Conversation logs and context

Why Paperless-AI¶

In a document-heavy infrastructure, manual document management becomes time-consuming and error-prone. Paperless-AI provides:

Automation: Eliminates manual tagging and classification workflows
Intelligence: Understands document context and relationships
Efficiency: Natural language search across entire document archive
Accuracy: AI-powered classification with customizable restrictions
Scalability: Handles growing document volumes automatically
Integration: Seamless connection with existing Paperless-ngx installation

Key Advantages¶

Semantic Search: Find documents by meaning, not just keywords
Smart Classification: Automatically assigns tags, types, and correspondents
Contextual Chat: Ask questions about your documents in natural language
Flexible AI Backends: Support for multiple AI providers and local models
Privacy-Friendly: Can run entirely on local infrastructure with Ollama
Customizable: Configurable automation rules and restrictions

Configuration Details¶

Environment Variables¶

The service configuration is managed through environment variables in /app/data/.env:

Core Settings¶

# Application Settings
PAPERLESS_AI_PORT=3000                    # Internal service port
PUID=1000                                 # User ID for file permissions
PGID=1000                                 # Group ID for file permissions

# RAG Service Configuration
RAG_SERVICE_URL=http://localhost:8000     # RAG service endpoint
RAG_SERVICE_ENABLED=true                  # Enable RAG-based search

# Paperless-ngx Integration
PAPERLESS_URL=https://paperless.rbnk.uk   # Paperless-ngx instance URL
PAPERLESS_TOKEN=your_api_token_here       # API authentication token

# AI Provider Configuration
AI_PROVIDER=ollama                        # Primary AI provider
OPENAI_API_KEY=sk-...                     # OpenAI API key (if using OpenAI)
OLLAMA_BASE_URL=http://ollama:11434       # Ollama service URL

# Processing Features
ACTIVATE_TAGGING=yes                      # Enable automatic tagging
ACTIVATE_CORRESPONDENTS=yes               # Enable correspondent detection
ACTIVATE_DOCUMENT_TYPE=yes                # Enable document type classification
ACTIVATE_TITLE=yes                        # Enable title generation
ACTIVATE_CUSTOM_FIELDS=yes                # Enable custom field processing

# AI Restrictions
RESTRICT_TO_EXISTING_TAGS=no              # Limit to existing tags only
RESTRICT_TO_EXISTING_CORRESPONDENTS=no    # Limit to existing correspondents
RESTRICT_TO_EXISTING_DOCUMENT_TYPES=no    # Limit to existing document types

Supported AI Providers¶

Ollama: Local models (Mistral, Llama, Phi-3, Gemma-2)
OpenAI: GPT-3.5/GPT-4 models
DeepSeek.ai: Cost-effective alternatives
OpenRouter.ai: Access to multiple models
Perplexity.ai: Enhanced reasoning capabilities
Together.ai: High-performance inference
LiteLLM: Unified API proxy (if available in infrastructure)
Gemini: Google's AI models
Azure OpenAI: Enterprise OpenAI deployment

Docker Compose Configuration¶

services:
  paperless-ai:
    image: clusterzx/paperless-ai
    container_name: paperless-ai
    restart: unless-stopped
    cap_drop:
      - ALL                               # Security hardening
    security_opt:
      - no-new-privileges=true            # Prevent privilege escalation
    environment:
      - PUID=1000
      - PGID=1000
      - PAPERLESS_AI_PORT=${PAPERLESS_AI_PORT:-3000}
      - RAG_SERVICE_URL=http://localhost:8000
      - RAG_SERVICE_ENABLED=true
    networks:
      - traefik_proxy                     # External access via Traefik
    ports:
      - "127.0.0.1:3003:${PAPERLESS_AI_PORT:-3000}"  # Local access only
    volumes:
      - paperless-ai_data:/app/data       # Persistent storage
    labels:
      # Traefik HTTPS routing configuration
      - "traefik.enable=true"
      - "traefik.docker.network=traefik_proxy"
      - "traefik.http.routers.paperless-ai.rule=Host(`paperless-ai.rbnk.uk`)"
      - "traefik.http.routers.paperless-ai.entrypoints=websecure"
      - "traefik.http.routers.paperless-ai.tls.certresolver=cf"

Access Information¶

Web Interface Access¶

Production URL: https://paperless-ai.rbnk.uk
Local Development: http://localhost:3003
Internal Container: http://paperless-ai:3000

Authentication¶

Paperless-AI relies on Paperless-ngx for authentication and authorization: - Configure Paperless-ngx API token in environment variables - User permissions inherited from Paperless-ngx setup - No additional authentication layer required

Key Interface Sections¶

Dashboard: Overview of processing status and recent activity
Chat: RAG-powered document search and Q&A interface
Manual Processing: Web interface for manual AI tagging
Settings: Configuration of AI providers and processing rules
Playground: Testing area for AI prompts and responses
History: Log of all processing activities and chat sessions

Common Operations¶

Service Management¶

# Start Paperless-AI service
cd /srv/dockerdata/paperless-ai
docker compose up -d

# Stop service
docker compose down

# Restart service
docker compose restart

# View logs
docker compose logs -f paperless-ai

# Access container shell
docker exec -it paperless-ai sh

# Check service status
docker compose ps

Configuration Updates¶

# Edit configuration
docker exec -it paperless-ai vi /app/data/.env

# Restart to apply changes
docker compose restart paperless-ai

# Verify new configuration
docker compose logs paperless-ai | grep -i config

Health Checks¶

# Check service health
curl -s http://localhost:3003/health || echo "Service unavailable"

# Verify Traefik routing
curl -I https://paperless-ai.rbnk.uk

# Test RAG service connectivity
docker exec paperless-ai curl -s http://localhost:8000/health

Document Processing¶

Automated Processing Workflow¶

Document Detection: Monitors Paperless-ngx for new documents
Content Analysis: AI analyzes document content and structure
Classification: Assigns document type based on content
Tagging: Generates relevant tags from document content
Correspondent Detection: Identifies sender/recipient information
Title Generation: Creates descriptive document titles
Custom Fields: Populates custom metadata fields
Indexing: Updates RAG search index for semantic search

Manual Processing¶

# Access manual processing interface
https://paperless-ai.rbnk.uk/manual

# Process specific document
POST /api/documents/{id}/process

# Bulk processing
POST /api/documents/bulk-process

Processing Customization¶

Configure processing behavior via environment variables:

# Enable/disable specific features
ACTIVATE_TAGGING=yes
ACTIVATE_CORRESPONDENTS=yes
ACTIVATE_DOCUMENT_TYPE=yes
ACTIVATE_TITLE=yes

# Restrict to existing values only
RESTRICT_TO_EXISTING_TAGS=yes
RESTRICT_TO_EXISTING_CORRESPONDENTS=yes
RESTRICT_TO_EXISTING_DOCUMENT_TYPES=yes

RAG Service Integration¶

RAG (Retrieval-Augmented Generation) Features¶

Semantic Search: Find documents by meaning and context
Question Answering: Ask natural language questions about documents
Document Summarization: Generate summaries of document collections
Context Awareness: Maintains conversation context across queries
Multi-Document Analysis: Analyze relationships between documents

RAG Service Configuration¶

# RAG service settings
RAG_SERVICE_URL=http://localhost:8000
RAG_SERVICE_ENABLED=true

# Vector database configuration
VECTOR_DB_TYPE=chromadb              # Default vector database
EMBEDDING_MODEL=sentence-transformers # Embedding model for vectors

# Search configuration
MAX_SEARCH_RESULTS=10                # Maximum documents per search
SIMILARITY_THRESHOLD=0.7             # Minimum similarity score

Using RAG Chat Interface¶

Access Chat: Navigate to https://paperless-ai.rbnk.uk/chat
Ask Questions: Type natural language questions about your documents
View Sources: See which documents informed the AI's response
Refine Queries: Use follow-up questions to drill down into details
Export Results: Save important conversations and insights

Example RAG Queries¶

# Finding specific documents
"When did I sign my rental agreement?"
"What was the amount of the last electricity bill?"
"Which documents mention my health insurance?"

# Document analysis
"Summarize all invoices from last quarter"
"What are the key terms in my employment contract?"
"Compare insurance policies from different providers"

# Trend analysis
"How have my utility costs changed over time?"
"What recurring expenses appear in my documents?"
"Show me all correspondence with my bank"

Backup and Restore¶

Automated Backup (Infrastructure Script)¶

Paperless-AI data is included in the automated infrastructure backup system:

# Location in backup script
/srv/dockerdata/rclone/backup-infrastructure.sh

# Backup includes:
- Configuration files (/app/data/.env)
- Document processing results
- RAG index and embeddings
- Chat history and logs
- User preferences and settings

Manual Backup Operations¶

# Create manual backup
docker run --rm \
  -v paperless-ai_data:/source \
  -v /srv/dockerdata/_backup:/dest \
  alpine cp -a /source/. /dest/paperless-ai-backup/

# Compress backup
cd /srv/dockerdata/_backup
tar -czf paperless-ai-backup-$(date +%Y%m%d).tar.gz paperless-ai-backup/

# Verify backup
tar -tzf paperless-ai-backup-$(date +%Y%m%d).tar.gz | head -10

Restore Procedures¶

# Stop service
cd /srv/dockerdata/paperless-ai
docker compose down

# Restore from backup
cd /srv/dockerdata/_backup
tar -xzf paperless-ai-backup-YYYYMMDD.tar.gz

# Copy data back
docker run --rm \
  -v /srv/dockerdata/_backup/paperless-ai-backup:/source \
  -v paperless-ai_data:/dest \
  alpine cp -a /source/. /dest/

# Start service
cd /srv/dockerdata/paperless-ai
docker compose up -d

# Verify restoration
docker compose logs paperless-ai | grep -i "startup\|ready\|error"

Backup Verification¶

# Check backup size
ls -lh /srv/dockerdata/_backup/paperless-ai-backup*

# Verify critical files
tar -tzf backup.tar.gz | grep -E "(config|env|index|chat)"

# Test restore in temporary location
mkdir /tmp/restore-test
tar -xzf backup.tar.gz -C /tmp/restore-test
ls -la /tmp/restore-test/paperless-ai-backup/

Troubleshooting Guide¶

Common Issues and Solutions¶

Service Won't Start¶

# Check container status
docker compose ps paperless-ai

# View startup logs
docker compose logs paperless-ai

# Common causes:
# 1. Port conflict
netstat -tlnp | grep 3003

# 2. Volume permission issues
docker run --rm -v paperless-ai_data:/data alpine ls -la /data

# 3. Configuration errors
docker exec paperless-ai cat /app/data/.env

Authentication Issues¶

# Verify Paperless-ngx connection
docker exec paperless-ai curl -H "Authorization: Token YOUR_TOKEN" \
  https://paperless.rbnk.uk/api/documents/

# Check API token validity
docker compose logs paperless-ai | grep -i "auth\|token\|401\|403"

# Test connectivity
docker exec paperless-ai nslookup paperless.rbnk.uk

RAG Service Problems¶

# Check RAG service status
docker exec paperless-ai curl -s http://localhost:8000/health

# Verify RAG configuration
docker compose logs paperless-ai | grep -i "rag\|vector\|embedding"

# Restart RAG service
docker exec paperless-ai supervisorctl restart rag-service

# Check RAG logs
docker exec paperless-ai tail -f /var/log/rag-service.log

AI Provider Issues¶

# Test AI provider connectivity
# For OpenAI
docker exec paperless-ai curl -H "Authorization: Bearer $OPENAI_API_KEY" \
  https://api.openai.com/v1/models

# For Ollama
docker exec paperless-ai curl http://ollama:11434/api/tags

# Check AI service logs
docker compose logs paperless-ai | grep -i "ai\|model\|provider"

Performance Issues¶

# Check system resources
docker stats paperless-ai

# Monitor processing queue
docker exec paperless-ai cat /app/data/processing-queue.json

# Check database connections
docker exec paperless-ai netstat -an | grep ESTABLISHED

# Clear processing cache
docker exec paperless-ai rm -rf /app/data/cache/*
docker compose restart paperless-ai

Debug Mode¶

# Enable debug logging
docker exec -it paperless-ai vi /app/data/.env
# Add: DEBUG=true
# Add: LOG_LEVEL=debug

# Restart with debug mode
docker compose restart paperless-ai

# View debug logs
docker compose logs -f paperless-ai | grep -i debug

# Disable debug mode
docker exec -it paperless-ai vi /app/data/.env
# Remove debug settings
docker compose restart paperless-ai

Log Analysis¶

# View recent application logs
docker compose logs --tail=100 paperless-ai

# Search for specific errors
docker compose logs paperless-ai | grep -i error

# Monitor real-time logs
docker compose logs -f paperless-ai

# Export logs for analysis
docker compose logs paperless-ai > paperless-ai-debug.log

Integration with Infrastructure¶

Traefik Integration¶

Paperless-AI integrates with the Traefik reverse proxy for external access:

labels:
  - "traefik.enable=true"
  - "traefik.docker.network=traefik_proxy"
  - "traefik.http.routers.paperless-ai.rule=Host(`paperless-ai.rbnk.uk`)"
  - "traefik.http.routers.paperless-ai.entrypoints=websecure"
  - "traefik.http.routers.paperless-ai.tls.certresolver=cf"

Monitoring Integration¶

Prometheus Metrics¶

The service exposes metrics for monitoring:

# Check if metrics endpoint exists
curl -s http://localhost:3003/metrics

# Common metrics:
# - paperless_ai_documents_processed_total
# - paperless_ai_processing_duration_seconds
# - paperless_ai_rag_queries_total
# - paperless_ai_errors_total

Grafana Dashboard¶

Create Grafana dashboard for Paperless-AI monitoring:

# Access Grafana
https://grafana.rbnk.uk

# Import dashboard template or create custom dashboard with:
# - Processing throughput
# - Error rates
# - RAG query performance
# - AI provider response times

Notification Integration¶

# Configure notifications for processing events
curl -X POST https://apprise.rbnk.uk/notify/apprise \
  -H "Content-Type: application/json" \
  -d '{
    "title": "Paperless-AI Processing Complete",
    "body": "Successfully processed 15 new documents"
  }'

# Integration with n8n workflows
# Create webhook endpoints for:
# - Document processing completion
# - Error notifications
# - Daily processing reports

LiteLLM Integration¶

If LiteLLM is available in the infrastructure:

# Configure Paperless-AI to use LiteLLM proxy
LITELLM_BASE_URL=https://litellm.rbnk.uk
AI_PROVIDER=litellm

# Benefits:
# - Unified API for multiple providers
# - Cost tracking and usage monitoring
# - Rate limiting and quota management
# - Failover between providers

Security Considerations¶

Container Security¶

The service implements several security hardening measures:

cap_drop:
  - ALL                           # Remove all Linux capabilities
security_opt:
  - no-new-privileges=true        # Prevent privilege escalation

Network Security¶

Network Isolation: Runs on traefik_proxy network only
No Exposed Ports: Only accessible via Traefik HTTPS
Local Binding: Direct port access limited to localhost (127.0.0.1)

Data Security¶

# File permissions
PUID=1000                        # Non-root user
PGID=1000                        # Non-root group

# Environment variables protection
chmod 640 /srv/dockerdata/paperless-ai/.env

# Sensitive data in environment variables only
# - API keys
# - Authentication tokens
# - Database credentials

API Security¶

Token-based Authentication: Uses Paperless-ngx API tokens
HTTPS Only: All external communication encrypted
No Default Credentials: Requires explicit configuration

Privacy Considerations¶

# Local AI processing (privacy-friendly)
AI_PROVIDER=ollama
OLLAMA_BASE_URL=http://ollama:11434

# Data retention settings
CHAT_HISTORY_RETENTION_DAYS=90
PROCESSING_LOG_RETENTION_DAYS=30

# RAG data privacy
VECTOR_DB_ENCRYPTION=true
EMBEDDING_LOCAL_ONLY=true

Audit and Compliance¶

# Enable audit logging
AUDIT_LOGGING=true
AUDIT_LOG_PATH=/app/data/audit.log

# Log sensitive operations:
# - Document access
# - Configuration changes
# - AI provider API calls
# - User authentication events

# Review audit logs
docker exec paperless-ai tail -f /app/data/audit.log

Monitoring and Logging¶

Application Logs¶

# View application logs
docker compose logs paperless-ai

# Monitor specific log levels
docker compose logs paperless-ai | grep -E "(ERROR|WARN|INFO)"

# Export logs for analysis
docker compose logs paperless-ai --since=24h > daily-logs.txt

Service Health Monitoring¶

# Health check endpoint
curl -s http://localhost:3003/health | jq

# Response format:
{
  "status": "healthy",
  "timestamp": "2024-01-15T10:30:00Z",
  "services": {
    "rag": "healthy",
    "ai_provider": "healthy",
    "paperless_api": "healthy"
  },
  "version": "1.0.0"
}

Performance Monitoring¶

# Monitor container resources
docker stats paperless-ai

# Check processing performance
curl -s http://localhost:3003/api/stats | jq

# Monitor AI provider response times
docker compose logs paperless-ai | grep "ai_response_time"

Integration with Infrastructure Monitoring¶

Prometheus Monitoring¶

# Add to Prometheus configuration
- job_name: 'paperless-ai'
  static_configs:
    - targets: ['paperless-ai:3000']
  metrics_path: '/metrics'
  scrape_interval: 30s

Alerting Rules¶

# Add to AlertManager rules
- alert: PaperlessAIDown
  expr: up{job="paperless-ai"} == 0
  for: 5m
  labels:
    severity: warning
  annotations:
    summary: "Paperless-AI service is down"

- alert: PaperlessAIHighErrorRate
  expr: rate(paperless_ai_errors_total[5m]) > 0.1
  for: 2m
  labels:
    severity: warning
  annotations:
    summary: "High error rate in Paperless-AI"

Log Aggregation¶

# Configure log shipping to Loki (if available)
# Add to promtail configuration
- job_name: paperless-ai
  static_configs:
    - targets:
        - localhost
      labels:
        job: paperless-ai
        __path__: /var/lib/docker/containers/*paperless-ai*/*.log

Maintenance and Updates¶

# Check for updates
docker pull clusterzx/paperless-ai:latest

# Update service
cd /srv/dockerdata/paperless-ai
docker compose pull
docker compose up -d

# Verify update
docker compose logs paperless-ai | grep -i version

# Rollback if needed
docker tag clusterzx/paperless-ai:latest clusterzx/paperless-ai:backup
docker compose down
docker compose up -d

Paperless-ngx Documentation - Core document management system
LiteLLM Documentation - AI provider proxy integration
LiteLLM Virtual Keys - Secure API key management
Monitoring Documentation - Infrastructure monitoring setup
Backup Documentation - Backup and restore procedures
Traefik Documentation - Reverse proxy configuration

For additional support and updates, see the official Paperless-AI repository.