michaelschiemer/deployment/README-PRODUCTION.md

Production-Ready Deployment Infrastructure

This directory contains production-ready deployment infrastructure for the Custom PHP Framework using containerized deployment with Ansible automation.

🚀 Key Features

• Container-Based Deployments: Pre-built images, no build on production servers
• Idempotent Operations: Repeatable deployments with consistent results
• Zero-Downtime Deployments: Smart container recreation with health checks
• Rollback Support: Quick rollback to previous versions with tag management
• Security Hardened: No secrets in repo, vault-encrypted sensitive data
• Optional CDN Integration: Flag-based CDN configuration updates
• Comprehensive Health Checks: Container and HTTP health validation
• Backup Management: Configurable backup creation and retention

📁 Directory Structure

deployment/
├── deploy-production.sh         # Production deployment script
├── rollback-production.sh       # Production rollback script
├── infrastructure/              # Ansible automation
│   ├── ansible.cfg             # Production-hardened Ansible config
│   ├── inventories/            # Environment-specific inventories
│   │   └── production/
│   │       └── hosts.yml       # Production servers and configuration
│   ├── group_vars/             # Shared variables
│   │   └── all/
│   │       ├── main.yml        # Global configuration
│   │       └── vault.yml       # Encrypted secrets
│   ├── templates/              # Environment file templates
│   │   ├── production.env.template
│   │   └── staging.env.template
│   └── playbooks/              # Ansible automation playbooks
│       ├── deploy-application.yml  # Main deployment playbook
│       ├── rollback.yml           # Rollback playbook
│       └── update-cdn.yml         # Optional CDN update
├── applications/               # Docker Compose configurations
│   ├── docker-compose.yml      # Base compose file
│   ├── docker-compose.production.yml  # Production overlay
│   └── environments/           # Environment templates (for reference)
└── .gitignore                  # Excludes sensitive files

🛠 Prerequisites

Required Tools

• Ansible 2.9+ with community.docker collection
• Docker on target servers
• SSH access to production servers with key-based authentication
• Vault password file for encrypted secrets

Infrastructure Requirements

• Pre-built container images in registry
• Production server: 94.16.110.151 with deploy user
• Domain: michaelschiemer.de with SSL certificates
• SSH key: ~/.ssh/deploy_key

🔧 Configuration

1. Vault Password File

Create vault password file (not in repo):

# Create vault password file
echo "your_vault_password" > ~/.ansible_vault_pass
chmod 600 ~/.ansible_vault_pass

# Set environment variable
export ANSIBLE_VAULT_PASSWORD_FILE=~/.ansible_vault_pass

2. SSH Key Setup

Ensure your SSH key is properly configured:

# Copy your SSH key to the expected location
cp ~/.ssh/your_production_key ~/.ssh/deploy_key
chmod 600 ~/.ssh/deploy_key

# Test connection
ssh -i ~/.ssh/deploy_key deploy@94.16.110.151

3. Ansible Collections

Install required Ansible collections:

ansible-galaxy collection install community.docker

🚀 Deployment

Production Deployment

Deploy a specific version to production:

# Basic deployment
./deploy-production.sh 1.2.3

# Deployment with CDN update
./deploy-production.sh 1.2.3 --cdn-update

# Deployment without backup
./deploy-production.sh 1.2.3 --no-backup

# Custom backup retention
./deploy-production.sh 1.2.3 --retention-days 60

# Using environment variables
IMAGE_TAG=1.2.3 CDN_UPDATE=true ./deploy-production.sh

Rollback

Rollback to a previous version:

# Rollback to previous version
./rollback-production.sh 1.2.2

# Force rollback without confirmation
./rollback-production.sh 1.2.2 --force

# Using environment variables
ROLLBACK_TAG=1.2.2 ./rollback-production.sh

Manual Ansible Commands

Run specific playbooks manually:

cd infrastructure

# Deploy application
ansible-playbook -i inventories/production/hosts.yml playbooks/deploy-application.yml \
  -e IMAGE_TAG=1.2.3 -e DOMAIN_NAME=michaelschiemer.de

# Rollback application
ansible-playbook -i inventories/production/hosts.yml playbooks/rollback.yml \
  -e ROLLBACK_TAG=1.2.2

# Update CDN only
ansible-playbook -i inventories/production/hosts.yml playbooks/update-cdn.yml \
  -e CDN_UPDATE=true

📊 Environment Variables

Deployment Variables

IMAGE_TAG=1.2.3              # Container image tag (required)
DOMAIN_NAME=michaelschiemer.de # Target domain
CDN_UPDATE=true              # Enable CDN configuration update
BACKUP_ENABLED=true          # Enable pre-deployment backup
BACKUP_RETENTION_DAYS=30     # Backup retention period

Rollback Variables

ROLLBACK_TAG=1.2.2          # Target rollback version (required)
DOMAIN_NAME=michaelschiemer.de # Target domain

CI/CD Variables

ANSIBLE_VAULT_PASSWORD_FILE=~/.ansible_vault_pass # Vault password file
DB_PASSWORD=secretpassword   # Database password (encrypted in vault)
REDIS_PASSWORD=redispass    # Redis password (encrypted in vault)
MAIL_PASSWORD=mailpass      # Mail service password (encrypted in vault)

🔒 Security

Secrets Management

• All secrets stored in group_vars/all/vault.yml (encrypted)
• No secrets in repository or playbooks
• Vault password via file or environment variable
• SSH key-based authentication only

Access Control

• Deploy user with limited sudo privileges
• Container security policies enforced
• Firewall configured via base-security role
• SSH hardening enabled

Environment File Security

• Environment files rendered from templates at deploy time
• Sensitive values injected from vault
• File permissions: 0600 (owner read/write only)
• Backup of previous versions created

📈 Monitoring & Health Checks

Container Health Checks

• Individual container health monitoring
• HTTP health endpoint validation
• Configurable retry count and intervals
• Automatic failure detection

Application Monitoring

• Health endpoint: https://michaelschiemer.de/health
• Container status monitoring
• Log aggregation via Docker logging drivers
• Optional Prometheus metrics

Backup Monitoring

• Configurable backup retention
• Automatic cleanup of old backups
• Backup success/failure tracking
• Emergency recovery tag storage

🔄 CI/CD Integration

GitHub Actions Example

name: Deploy to Production

on:
  push:
    tags: ['v*']

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Setup Ansible
        run: |
          pip install ansible
          ansible-galaxy collection install community.docker
          
      - name: Deploy to Production
        run: |
          echo "${{ secrets.ANSIBLE_VAULT_PASSWORD }}" > ~/.ansible_vault_pass
          chmod 600 ~/.ansible_vault_pass
          cd deployment
          ./deploy-production.sh ${GITHUB_REF#refs/tags/v} --no-backup
        env:
          ANSIBLE_VAULT_PASSWORD_FILE: ~/.ansible_vault_pass
          ANSIBLE_HOST_KEY_CHECKING: false

Jenkins Pipeline Example

pipeline {
    agent any
    
    environment {
        ANSIBLE_VAULT_PASSWORD_FILE = credentials('ansible-vault-password')
        IMAGE_TAG = "${params.VERSION}"
    }
    
    parameters {
        string(name: 'VERSION', defaultValue: '', description: 'Version to deploy')
        booleanParam(name: 'CDN_UPDATE', defaultValue: false, description: 'Update CDN')
    }
    
    stages {
        stage('Deploy') {
            steps {
                dir('deployment') {
                    sh """
                        ./deploy-production.sh ${IMAGE_TAG} \
                        ${params.CDN_UPDATE ? '--cdn-update' : ''}
                    """
                }
            }
        }
    }
}

🚨 Troubleshooting

Common Issues

1. Vault Password Issues

# Error: vault password file not found
export ANSIBLE_VAULT_PASSWORD_FILE=~/.ansible_vault_pass

2. SSH Connection Issues

# Test SSH connection
ssh -i ~/.ssh/deploy_key deploy@94.16.110.151

# Update SSH key path in inventory if needed

3. Container Health Check Failures

# Check container logs
docker logs <container_name>

# Check health endpoint manually
curl -k https://michaelschiemer.de/health

4. Permission Issues

# Fix deployment script permissions
chmod +x deploy-production.sh rollback-production.sh

Emergency Procedures

1. Complete Rollback

# Get last successful release
cat /var/www/html/.last_successful_release

# Rollback to last known good version
./rollback-production.sh <last_good_version> --force

2. Manual Container Restart

# SSH to server and restart containers
ssh deploy@94.16.110.151
cd /var/www/html
docker-compose restart

3. Emergency Recovery

# Use emergency recovery tag if available
cat /var/www/html/.emergency_recovery_tag
./rollback-production.sh <emergency_recovery_tag> --force

📝 Best Practices

Deployment Best Practices

1. Always use specific version tags (not latest)
2. Test deployments on staging first
3. Monitor application health after deployment
4. Keep backup enabled for production
5. Use CDN updates only when static assets change

Security Best Practices

1. Rotate vault passwords regularly
2. Keep SSH keys secure and rotated
3. Use minimal privilege principles
4. Monitor access logs
5. Keep Ansible and collections updated

Operational Best Practices

1. Document all deployment changes
2. Monitor resource usage trends
3. Plan for backup storage requirements
4. Test rollback procedures regularly
5. Keep deployment logs for auditing

📞 Support

For issues with deployment infrastructure:

1. Check troubleshooting section above
2. Review Ansible logs in infrastructure/logs/
3. Verify container health and logs
4. Test connectivity and permissions
5. Contact system administrator if needed

---

Note: This infrastructure follows production-ready practices with security, reliability, and operability in mind. Always test changes in staging before applying to production.