michaelschiemer/docs/deployment/QUICKSTART.md

# Production Deployment - Quick Start Guide

**Goal**: Get the application running in production in under 30 minutes.

This is a simplified, essential-steps-only guide. For comprehensive documentation, see:
- [Complete Deployment Workflow](DEPLOYMENT_WORKFLOW.md)
- [Production Deployment Guide](PRODUCTION_DEPLOYMENT.md)
- [Ansible Deployment](ANSIBLE_DEPLOYMENT.md)

---

## Prerequisites

- Ubuntu 22.04+ server with root access
- Domain name pointing to server IP
- Port 80 and 443 open in firewall

---

## Step 1: Initial Server Setup (10 minutes)

```bash
# SSH into server
ssh root@your-server.com

# Update system
apt update && apt upgrade -y

# Install Docker
curl -fsSL https://get.docker.com -o get-docker.sh
sh get-docker.sh

# Install Docker Compose
apt install docker-compose-plugin -y

# Install certbot for SSL
apt install certbot -y

# Create application user
useradd -m -s /bin/bash appuser
usermod -aG docker appuser
```

---

## Step 2: SSL Certificate (5 minutes)

```bash
# Create webroot directory
mkdir -p /var/www/certbot

# Get SSL certificate
certbot certonly --webroot \
    -w /var/www/certbot \
    -d yourdomain.com \
    --email your-email@example.com \
    --agree-tos \
    --non-interactive

# Verify certificates
ls -la /etc/letsencrypt/live/yourdomain.com/
```

**Expected output**: `fullchain.pem` and `privkey.pem` files

---

## Step 3: Clone Application (2 minutes)

```bash
# Switch to app user
su - appuser

# Clone repository
git clone https://github.com/your-org/your-app.git /home/appuser/app
cd /home/appuser/app

# Checkout production branch
git checkout main
```

---

## Step 4: Generate Secrets (3 minutes)

```bash
# Generate Vault encryption key
php scripts/deployment/generate-vault-key.php

# Save output - YOU MUST STORE THIS SECURELY!
# Example output: vault_key_abc123def456...
```

**⚠️ CRITICAL**: Store this key in your password manager. You cannot recover it if lost.

---

## Step 5: Create Environment File (5 minutes)

```bash
# Copy example
cp .env.example .env.production

# Edit configuration
nano .env.production
```

**Minimal required configuration**:

```env
# Application
APP_ENV=production
APP_DEBUG=false
APP_URL=https://yourdomain.com

# Database
DB_HOST=database
DB_PORT=3306
DB_NAME=app_production
DB_USER=app_user
DB_PASS=GENERATE_STRONG_PASSWORD_HERE

# Vault
VAULT_ENCRYPTION_KEY=YOUR_GENERATED_KEY_FROM_STEP_4

# Logging
LOG_PATH=/var/log/app
LOG_LEVEL=INFO

# Admin Access
ADMIN_ALLOWED_IPS=YOUR.SERVER.IP,127.0.0.1
```

**Generate strong passwords**:
```bash
# Generate DB password
openssl rand -base64 32

# Generate JWT secret
openssl rand -base64 64
```

---

## Step 6: Build and Start (3 minutes)

```bash
# Build containers
docker compose -f docker-compose.production.yml build

# Start containers
docker compose -f docker-compose.production.yml up -d

# Check status
docker compose -f docker-compose.production.yml ps
```

**Expected output**: All containers should be "Up"

---

## Step 7: Initialize Database (2 minutes)

```bash
# Run migrations
docker compose -f docker-compose.production.yml exec php php console.php db:migrate

# Verify migration status
docker compose -f docker-compose.production.yml exec php php console.php db:status
```

---

## Step 8: Verify Health (1 minute)

```bash
# Check health endpoint
curl http://localhost/health/summary

# Expected output (healthy):
{
  "timestamp": "2025-01-15T10:00:00+00:00",
  "overall_status": "healthy",
  "overall_healthy": true,
  "summary": {
    "total_checks": 8,
    "healthy": 8,
    "warning": 0,
    "unhealthy": 0
  }
}
```

If unhealthy, check logs:
```bash
docker compose -f docker-compose.production.yml logs php
```

---

## Step 9: Configure Nginx Reverse Proxy

```bash
# Exit to root user
exit

# Create Nginx config
nano /etc/nginx/sites-available/app
```

**Nginx configuration**:

```nginx
server {
    listen 80;
    server_name yourdomain.com;
    return 301 https://$server_name$request_uri;
}

server {
    listen 443 ssl http2;
    server_name yourdomain.com;

    ssl_certificate /etc/letsencrypt/live/yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/yourdomain.com/privkey.pem;

    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_prefer_server_ciphers on;
    ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256';

    location / {
        proxy_pass http://localhost:8080;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

    location /health {
        proxy_pass http://localhost:8080/health;
        access_log off;
    }
}
```

**Enable and restart**:

```bash
# Enable site
ln -s /etc/nginx/sites-available/app /etc/nginx/sites-enabled/

# Test configuration
nginx -t

# Restart Nginx
systemctl restart nginx
```

---

## Step 10: Final Verification

```bash
# Test HTTPS endpoint
curl -f https://yourdomain.com/health/summary

# Test detailed health
curl -f https://yourdomain.com/health/detailed

# Test metrics (should be accessible)
curl -f https://yourdomain.com/metrics
```

**✅ Success criteria**:
- All curl commands return 200 OK
- No SSL certificate warnings
- Health endpoint shows all checks healthy

---

## Post-Deployment Tasks

### Setup Automatic Certificate Renewal

```bash
# Test renewal
certbot renew --dry-run

# Certbot automatically creates cron job, verify:
systemctl status certbot.timer
```

### Setup Log Rotation

```bash
# Create logrotate config
nano /etc/logrotate.d/app
```

```
/var/log/app/*.log {
    daily
    rotate 14
    compress
    delaycompress
    notifempty
    missingok
    create 0644 appuser appuser
    sharedscripts
    postrotate
        docker compose -f /home/appuser/app/docker-compose.production.yml exec php php console.php cache:clear > /dev/null 2>&1 || true
    endscript
}
```

### Setup Monitoring (Optional but Recommended)

```bash
# Install monitoring stack
cd /home/appuser/app
docker compose -f docker-compose.monitoring.yml up -d

# Access Grafana
# URL: http://your-server:3000
# Default credentials: admin/admin
```

### Setup Backups

```bash
# Create backup script
nano /home/appuser/backup-production.sh
```

```bash
#!/bin/bash
set -e

BACKUP_DIR="/opt/backups"
DATE=$(date +%Y%m%d_%H%M%S)

# Backup database
docker compose -f /home/appuser/app/docker-compose.production.yml \
    exec -T database mysqldump -u app_user -p${DB_PASS} app_production | \
    gzip > ${BACKUP_DIR}/db_${DATE}.sql.gz

# Backup Vault
tar -czf ${BACKUP_DIR}/vault_${DATE}.tar.gz /opt/vault

# Keep only last 7 days
find ${BACKUP_DIR} -name "*.gz" -mtime +7 -delete

echo "Backup completed: ${DATE}"
```

```bash
# Make executable
chmod +x /home/appuser/backup-production.sh

# Add to crontab (daily at 2 AM)
crontab -e
```

Add line:
```
0 2 * * * /home/appuser/backup-production.sh >> /var/log/backup.log 2>&1
```

---

## Troubleshooting

### Container won't start

```bash
# Check logs
docker compose -f docker-compose.production.yml logs php

# Common issues:
# - Wrong database credentials → Check .env.production
# - Port already in use → Check: netstat -tulpn | grep 8080
# - Permission issues → Run: chown -R appuser:appuser /home/appuser/app
```

### Health checks failing

```bash
# Check specific health check
curl http://localhost/health/category/database

# Common issues:
# - Database not migrated → Run: php console.php db:migrate
# - Cache not writable → Check: ls -la /var/cache/app
# - Queue not running → Check: docker compose ps
```

### SSL certificate issues

```bash
# Check certificate validity
openssl x509 -in /etc/letsencrypt/live/yourdomain.com/fullchain.pem -noout -dates

# Renew certificate
certbot renew --force-renewal

# Restart Nginx
systemctl restart nginx
```

### Application errors

```bash
# Check application logs
docker compose -f docker-compose.production.yml logs -f php

# Check Nginx logs
tail -f /var/log/nginx/error.log

# Check system logs
journalctl -u nginx -f
```

---

## Security Hardening (Do This After Deployment)

### 1. Firewall Configuration

```bash
# Install UFW
apt install ufw -y

# Allow SSH
ufw allow 22/tcp

# Allow HTTP/HTTPS
ufw allow 80/tcp
ufw allow 443/tcp

# Enable firewall
ufw enable
```

### 2. SSH Key-Only Authentication

```bash
# Generate SSH key on local machine
ssh-keygen -t ed25519 -C "your-email@example.com"

# Copy to server
ssh-copy-id root@your-server.com

# Disable password authentication
nano /etc/ssh/sshd_config
```

Set:
```
PasswordAuthentication no
PermitRootLogin prohibit-password
```

Restart SSH:
```bash
systemctl restart sshd
```

### 3. Fail2Ban

```bash
# Install fail2ban
apt install fail2ban -y

# Create jail for Nginx
nano /etc/fail2ban/jail.d/nginx-limit.conf
```

```ini
[nginx-limit-req]
enabled = true
filter = nginx-limit-req
logpath = /var/log/nginx/error.log
maxretry = 10
findtime = 60
bantime = 3600
```

```bash
# Restart fail2ban
systemctl restart fail2ban
```

---

## Deployment Updates (Ongoing)

For deploying updates after initial setup:

```bash
# SSH to server
ssh appuser@your-server.com

# Navigate to app
cd /home/appuser/app

# Pull latest code
git pull origin main

# Rebuild containers
docker compose -f docker-compose.production.yml build

# Run migrations (if any)
docker compose -f docker-compose.production.yml exec php php console.php db:migrate

# Restart containers
docker compose -f docker-compose.production.yml up -d

# Verify health
curl -f http://localhost/health/summary
```

**For zero-downtime deployments**, use the automated script:
```bash
./scripts/deployment/blue-green-deploy.sh
```

---

## Getting Help

If you encounter issues not covered in this quick start:

1. **Check detailed documentation**:
   - [Complete Deployment Workflow](DEPLOYMENT_WORKFLOW.md)
   - [Production Deployment Guide](PRODUCTION_DEPLOYMENT.md)
   - [Troubleshooting Guide](PRODUCTION_DEPLOYMENT.md#troubleshooting)

2. **Check application logs**:
   ```bash
   docker compose -f docker-compose.production.yml logs -f
   ```

3. **Check health endpoints**:
   ```bash
   curl http://localhost/health/detailed | jq
   ```

4. **Check metrics**:
   ```bash
   curl http://localhost/metrics
   ```

---

## Success Checklist

✅ Before considering deployment complete:

- [ ] SSL certificate installed and valid
- [ ] Application accessible via HTTPS
- [ ] All health checks passing (green)
- [ ] Database migrations applied successfully
- [ ] Logs being written to `/var/log/app`
- [ ] Automatic certificate renewal configured
- [ ] Backup script running daily
- [ ] Firewall configured (ports 22, 80, 443 only)
- [ ] SSH key-only authentication enabled
- [ ] Fail2Ban installed and monitoring
- [ ] Monitoring stack running (optional)
- [ ] Team has access to Vault encryption key
- [ ] Database backup verified and restorable

---

## Next Steps

After successful deployment:

1. **Setup Monitoring Alerts**: Configure Prometheus alerting rules
2. **Performance Tuning**: Review metrics and optimize based on actual traffic
3. **Security Audit**: Run security scan with tools like OWASP ZAP
4. **Documentation**: Document any custom configuration changes
5. **Team Training**: Ensure team knows deployment and rollback procedures
6. **Disaster Recovery**: Test backup restoration procedure

---

## Estimated Timeline

- **Initial deployment (following this guide)**: 30 minutes
- **Security hardening**: 15 minutes
- **Monitoring setup**: 20 minutes
- **Total time to production**: ~1 hour

---

**You're ready for production! 🚀**

For questions or issues, refer to the comprehensive guides linked throughout this document.