michaelschiemer/docs/deployment/DEPLOYMENT_CHECKLIST.md

Production Deployment Checklist

Print this and check off items as you complete them.

---

Pre-Deployment Checklist

Infrastructure

• Server meets requirements (Ubuntu 22.04+, 4GB RAM, 40GB disk)
• Domain name configured and pointing to server IP
• DNS propagation verified (nslookup yourdomain.com)
• Firewall rules configured (ports 22, 80, 443 open)
• SSH access to server confirmed
• Root or sudo access verified

Security

• SSH key pair generated
• SSH key added to server
• Vault encryption key generated
• Vault key stored in password manager
• Database passwords generated (32+ characters)
• JWT secrets generated (64+ characters)
• Admin allowed IPs list prepared
• SSL certificate email address ready

Code

• Application repository accessible
• Production branch exists and tested
• All tests passing locally
• Database migrations reviewed
• .env.example up to date
• Dependencies reviewed (composer.json, package.json)

---

Deployment Steps Checklist

Step 1: Server Setup

• SSH into server
• System updated (apt update && upgrade)
• Docker installed
• Docker Compose installed
• Certbot installed
• Application user created
• Application user added to docker group
• Directory structure created (/var/www/app, /var/log/app, /opt/vault)

Step 2: SSL Certificate

• Webroot directory created (/var/www/certbot)
• Certbot certificate obtained
• Certificate files verified (fullchain.pem, privkey.pem)
• Certificate expiration date checked (>30 days)
• Auto-renewal tested (certbot renew --dry-run)

Step 3: Application Code

• Repository cloned to /home/appuser/app
• Production branch checked out
• Git configured (user.name, user.email)
• File permissions set correctly (chown -R appuser:appuser)

Step 4: Environment Configuration

• .env.production created from .env.example
• APP_ENV set to "production"
• APP_DEBUG set to "false"
• APP_URL configured with domain
• Database credentials configured
• VAULT_ENCRYPTION_KEY added
• LOG_PATH configured
• ADMIN_ALLOWED_IPS configured
• All required environment variables set
• Sensitive values NOT committed to git

Step 5: Docker Containers

• docker-compose.production.yml reviewed
• Containers built (docker compose build)
• Containers started (docker compose up -d)
• All containers running (docker compose ps)
• Container logs checked for errors
• Container networking verified

Step 6: Database

• Database container healthy
• Database migrations applied (php console.php db:migrate)
• Migration status verified (php console.php db:status)
• Database backup created
• Database connection tested

Step 7: Health Checks

• Health endpoint accessible (curl http://localhost/health/summary)
• All health checks passing (overall_healthy: true)
• Database health check: healthy
• Cache health check: healthy
• Queue health check: healthy
• Filesystem health check: healthy
• SSL health check: healthy
• Detailed health endpoint tested

Step 8: Nginx Configuration

• Nginx installed
• Site configuration created (/etc/nginx/sites-available/app)
• SSL certificates paths correct in config
• Proxy settings configured
• Site enabled (symlink in sites-enabled)
• Nginx configuration tested (nginx -t)
• Nginx restarted
• HTTPS redirect working (http → https)

Step 9: Application Verification

• HTTPS endpoint accessible (https://yourdomain.com)
• SSL certificate valid (no browser warnings)
• Homepage loads correctly
• API endpoints responding
• Authentication working
• Admin panel accessible (from allowed IPs)
• File uploads working
• Background jobs processing
• Email sending configured

Step 10: Monitoring

• Metrics endpoint accessible (/metrics)
• Prometheus metrics valid format
• Health checks integrated with monitoring
• Log files being created (/var/log/app/)
• Log rotation configured
• Disk space monitored
• Memory usage monitored
• CPU usage monitored

---

Post-Deployment Checklist

Security Hardening

• UFW firewall enabled
• Only required ports open (22, 80, 443)
• SSH password authentication disabled
• Root login disabled via SSH
• Fail2Ban installed and configured
• Security headers verified (X-Frame-Options, CSP, etc.)
• OWASP security scan performed
• SSL Labs test passed (A+ rating)

Backups

• Database backup script created
• Vault backup script created
• Backup directory created (/opt/backups)
• Backup cron job configured
• Backup restoration tested
• Backup retention policy configured (7 days)
• Off-site backup configured (optional but recommended)

Monitoring & Alerts

• Grafana installed (optional)
• Prometheus configured (optional)
• Alert rules configured
• Email notifications configured
• Disk space alerts set (>90% usage)
• Memory alerts set (>90% usage)
• Health check alerts set
• SSL expiration alerts set (30 days)

Documentation

• Deployment procedure documented
• Server credentials documented (in secure location)
• Vault encryption key documented (in secure location)
• Database backup location documented
• Rollback procedure documented
• Team access granted and documented
• On-call rotation documented

Performance

• Performance baseline established
• Slow query log enabled
• Cache hit rate monitored
• Response time benchmarked
• Load testing performed
• Database indexes optimized
• Asset compression enabled (gzip)
• CDN configured (optional)

Compliance & Legal

• Privacy policy deployed
• Terms of service deployed
• Cookie consent implemented (if EU traffic)
• GDPR compliance verified (if EU traffic)
• Data retention policies documented
• Incident response plan documented

---

Rollback Checklist

Use this if deployment fails and you need to rollback:

Immediate Rollback

• Stop new containers: docker compose down
• Start old containers: docker compose -f docker-compose.old.yml up -d
• Verify health: curl http://localhost/health/summary
• Rollback database migrations: php console.php db:rollback
• Clear cache: php console.php cache:clear
• Verify application functionality
• Notify team of rollback

Post-Rollback

• Document rollback reason
• Identify root cause
• Create fix for issue
• Test fix in staging
• Plan next deployment attempt
• Update deployment procedure if needed

---

Weekly Maintenance Checklist

Perform these checks weekly:

• Review application logs for errors
• Check disk space (should be <80%)
• Review health check status
• Verify backups running successfully
• Check SSL certificate expiration (>30 days remaining)
• Review security logs (fail2ban)
• Check for system updates
• Review performance metrics
• Test backup restoration (monthly)

---

Monthly Maintenance Checklist

Perform these checks monthly:

• Apply system security updates
• Review and update dependencies (composer update, npm update)
• Rotate secrets (API keys, tokens) if required
• Review and archive old logs
• Perform security audit
• Review and update documentation
• Test disaster recovery procedure
• Review and optimize database performance
• Review monitoring alerts effectiveness
• Update deployment runbook with lessons learned

---

Quarterly Maintenance Checklist

Perform these checks quarterly:

• Rotate Vault encryption key
• Rotate database passwords
• Review and update security policies
• Conduct penetration testing
• Review and optimize infrastructure costs
• Update disaster recovery plan
• Review team access and permissions
• Conduct deployment drill with team
• Review compliance requirements
• Update technical documentation

---

Emergency Contacts

Fill this in and keep it secure:

Server Provider: _______________________
Support Phone: _________________________
Support Email: _________________________

Domain Registrar: ______________________
Support Phone: _________________________
Support Email: _________________________

SSL Provider: __________________________
Support Phone: _________________________
Support Email: _________________________

Database Backup Location: ______________
Vault Key Location: ____________________
SSH Key Location: ______________________

Team Lead: _____________________________
On-Call Phone: _________________________

DevOps Lead: ___________________________
On-Call Phone: _________________________

Security Contact: ______________________
Emergency Phone: _______________________

---

Deployment Sign-Off

Deployment Details:

Date: _____________________
Deployed By: ______________
Version/Commit: ___________
Environment: Production
Deployment Method: [ ] Manual [ ] Script [ ] Ansible

Health Check Status: [ ] All Passing
SSL Certificate: [ ] Valid
Database Migrations: [ ] Applied
Backups: [ ] Verified

Issues During Deployment:
_____________________________________________
_____________________________________________

Post-Deployment Notes:
_____________________________________________
_____________________________________________

Signed: ___________________ Date: __________

---

Continuous Improvement

After each deployment, answer these questions:

1. What went well?

   • ---

   • ---

2. What could be improved?

   • ---

   • ---

3. What was unexpected?

   • ---

   • ---

4. Action items for next deployment:

   • ---

   • ---

5. Documentation updates needed:

   • ---

   • ---

---

Remember: This checklist should be updated after each deployment to reflect lessons learned and process improvements.