michael/michaelschiemer

Fork 0

Files

Michael Schiemer 77abc65cd7

Deploy Application / deploy (push) Has been cancelled

Details

fix: DockerSecretsResolver - don't normalize absolute paths like /var/www/html/...

2025-11-24 21:28:25 +01:00

23 KiB

Raw Blame History

SSH Deployment Guide

Comprehensive guide for deploying the Custom PHP Framework using SSH-based deployment scripts.

Overview

This deployment system uses simple SSH/SCP-based scripts to deploy the framework to staging and production environments. It replaces Gitea Actions with a straightforward bash script approach.

Key Features:

✅ Simple SSH/SCP deployment (no CI/CD platform dependency)
✅ Automatic Docker image building and registry pushing
✅ Database backups before production deployments
✅ Automatic rollback on deployment failure
✅ Health checks and smoke tests
✅ Timestamped backup retention
✅ Color-coded output for easy monitoring

Prerequisites

Required Software

Local Machine:

Docker (for building images)
Docker Compose (for compose file validation)
SSH client (openssh-client)
SCP client (usually bundled with SSH)
Bash shell

Remote Servers (staging/production):

Docker and Docker Compose installed
SSH server running
Docker private registry accessible (localhost:5000 or custom)
Deployment user with Docker permissions
Directory structure: /opt/framework-staging/ or /opt/framework-production/

SSH Key Setup

Generate SSH keys for deployment (if not already done):

# Generate deployment SSH key
ssh-keygen -t rsa -b 4096 -f ~/.ssh/framework-deploy \
  -C "framework-deployment" -N ""

# Copy public key to staging server
ssh-copy-id -i ~/.ssh/framework-deploy.pub deploy@staging.michaelschiemer.de

# Copy public key to production server
ssh-copy-id -i ~/.ssh/framework-deploy.pub deploy@michaelschiemer.de

# Test connection
ssh -i ~/.ssh/framework-deploy deploy@staging.michaelschiemer.de "echo 'SSH connection successful'"

SSH Config (~/.ssh/config):

# Staging Server
Host staging.michaelschiemer.de
    User deploy
    IdentityFile ~/.ssh/framework-deploy
    Port 22

# Production Server
Host michaelschiemer.de
    User deploy
    IdentityFile ~/.ssh/framework-deploy
    Port 22

Environment Variables

Staging Deployment:

export STAGING_HOST=staging.michaelschiemer.de
export STAGING_USER=deploy
export STAGING_SSH_PORT=22

Production Deployment:

export PRODUCTION_HOST=michaelschiemer.de
export PRODUCTION_USER=deploy
export PRODUCTION_SSH_PORT=22

Optional Configuration:

# Docker Registry (default: localhost:5000)
export REGISTRY=your-registry.com

# Image Configuration
export IMAGE_NAME=framework
export IMAGE_TAG=latest  # or staging

# Production Options
export SKIP_BACKUP=false  # Skip database backup (not recommended)
export FORCE_REBUILD=false  # Force Docker rebuild

Persistent Configuration (.bashrc or .zshrc):

# Add to ~/.bashrc or ~/.zshrc
export STAGING_HOST=staging.michaelschiemer.de
export STAGING_USER=deploy
export PRODUCTION_HOST=michaelschiemer.de
export PRODUCTION_USER=deploy

Deployment Scripts

1. Staging Deployment

Script: deployment/scripts/deploy-staging.sh

Purpose: Deploy to staging environment for testing

Usage:

# Basic deployment
./deployment/scripts/deploy-staging.sh

# With custom configuration
STAGING_HOST=custom.staging.com ./deployment/scripts/deploy-staging.sh

What It Does:

Builds Docker image with ENV=staging
Pushes image to private registry
Creates timestamped backup of current deployment
Copies deployment files via SCP
Stops existing containers
Starts new containers
Waits 30 seconds for services to initialize
Performs health checks
Automatic rollback on failure

Backup Retention: Keeps last 5 backups, deletes older

Deployment Path: /opt/framework-staging/current/

Expected Output:

==================================================
🚀 Starting Staging Deployment
==================================================
Registry: localhost:5000
Image: framework:staging
Remote: deploy@staging.michaelschiemer.de:22
Path: /opt/framework-staging

[1/7] Building Docker image...
[2/7] Pushing image to registry...
[3/7] Preparing deployment files...
[4/7] Creating remote directory and backup...
Backing up current deployment...
Backup created: backup_20250124_153022
[5/7] Copying deployment files to server...
[6/7] Executing deployment on server...
==================================================
Starting Staging Deployment on Server
==================================================
[1/5] Pulling latest Docker images...
[2/5] Stopping existing containers...
[3/5] Starting new containers...
[4/5] Waiting for services to be healthy...
[5/5] Verifying deployment...
==================================================
✅ Staging Deployment Complete
==================================================
[7/7] Performing health checks...
Waiting 30 seconds for services to initialize...
Checking container status...
✅ Health check complete!

==================================================
✅ Staging Deployment Successful
==================================================
URL: https://staging.michaelschiemer.de
Deployed at: Thu Jan 24 15:30:45 CET 2025

2. Production Deployment

Script: deployment/scripts/deploy-production.sh

Purpose: Deploy to production environment

⚠️ WARNING: Production deployments include:

Automatic database backup (mandatory unless skipped)
60-second service initialization wait
Smoke tests for main page and API health
Automatic rollback on any failure

Usage:

# Standard production deployment
./deployment/scripts/deploy-production.sh

# Skip database backup (NOT RECOMMENDED)
SKIP_BACKUP=true ./deployment/scripts/deploy-production.sh

# Force Docker rebuild
FORCE_REBUILD=true ./deployment/scripts/deploy-production.sh

What It Does:

Builds Docker image with ENV=production
Pushes image to private registry
Creates database backup (aborts if backup fails)
Creates timestamped backup of current deployment
Copies deployment files via SCP
Stops existing containers gracefully
Starts new containers
Waits 60 seconds for services to initialize
Runs database migrations with --force
Performs comprehensive health checks:
- Container status
- PHP-FPM process check
- Redis connection test
Runs smoke tests:
- Main page accessibility (https://michaelschiemer.de/)
- API health endpoint (https://michaelschiemer.de/api/health)
Automatic rollback on any failure

Backup Retention: Keeps last 10 backups, deletes older

Deployment Path: /opt/framework-production/current/

Database Backup Location: /var/www/html/storage/backups/backup_YYYYMMDD_HHMMSS.sql

Expected Output:

==================================================
🚀 Starting Production Deployment
==================================================
Registry: localhost:5000
Image: framework:latest
Remote: deploy@michaelschiemer.de:22
Path: /opt/framework-production
Skip Backup: false

[1/8] Building Docker image...
[2/8] Pushing image to registry...
[3/8] Preparing deployment files...
[4/8] Creating remote directory and backup...
[5/8] Copying deployment files to server...
[6/8] Executing deployment on server...
==================================================
Starting Production Deployment on Server
==================================================
[0/6] Creating database backup...
✅ Database backup created: backup_20250124_153045.sql
[1/6] Pulling latest Docker images...
[2/6] Stopping existing containers (graceful shutdown)...
[3/6] Starting new containers...
[4/6] Waiting for services to be healthy...
[5/6] Running database migrations...
[6/6] Verifying deployment...
==================================================
✅ Production Deployment Complete
==================================================
[7/8] Performing health checks...
Waiting 60 seconds for services to initialize...
Checking container status...
✅ All health checks passed!
[8/8] Running smoke tests...
✅ Main page accessible
✅ API health check passed
✅ Smoke tests completed successfully

==================================================
✅ Production Deployment Successful
==================================================
URL: https://michaelschiemer.de
Deployed at: Thu Jan 24 15:32:15 CET 2025

3. Rollback Script

Script: deployment/scripts/rollback.sh

Purpose: Restore previous deployment from backup

Usage:

# Rollback staging to latest backup
./deployment/scripts/rollback.sh staging

# Rollback production to latest backup
./deployment/scripts/rollback.sh production

# Rollback to specific backup
./deployment/scripts/rollback.sh production backup_20250124_143022

What It Does:

Lists available backups
Confirms rollback operation (requires "yes")
Stops current deployment
Archives failed deployment as failed_YYYYMMDD_HHMMSS
Restores specified backup
Starts restored deployment
Performs health checks

Arguments:

environment: staging or production (required)
backup_name: Specific backup to restore (optional, defaults to latest)

Example Session:

$ ./deployment/scripts/rollback.sh production

==================================================
🔄 Starting Rollback: production
==================================================
Remote: deploy@michaelschiemer.de:22
Path: /opt/framework-production
Target Backup: Latest available

⚠️  WARNING: This will rollback the production deployment
   Current deployment will be stopped and replaced with backup

Are you sure you want to continue? (yes/no): yes

[1/5] Listing available backups...
Available backups:
backup_20250124_153045
backup_20250124_120000
backup_20250123_183015

[2/5] Determining backup to restore...
Using latest backup: backup_20250124_153045
✅ Backup backup_20250124_153045 verified

[3/5] Stopping current deployment...
✅ Current deployment stopped

[4/5] Restoring backup...
Archiving failed deployment as failed_20250124_154512...
Restoring backup backup_20250124_153045...
✅ Backup restored

[5/5] Starting restored deployment...
Starting containers...
Waiting for services to start...
✅ Restored deployment is running

==================================================
✅ Rollback Complete
==================================================
Environment: production
Restored: backup_20250124_153045
Completed at: Thu Jan 24 15:45:30 CET 2025

Failed deployment archived as: failed_20250124_154512

Deployment Workflows

Staging Deployment Workflow

Step-by-Step Process:

Prepare Changes:

# Make code changes locally
git add .
git commit -m "feat: new feature"
git push origin staging

Deploy to Staging:

# Set environment variables (if not in ~/.bashrc)
export STAGING_HOST=staging.michaelschiemer.de
export STAGING_USER=deploy

# Run deployment
./deployment/scripts/deploy-staging.sh

Verify Deployment:

# Check application
curl -k https://staging.michaelschiemer.de/health

# Monitor logs
ssh deploy@staging.michaelschiemer.de \
  "cd /opt/framework-staging/current && docker-compose logs -f"

# Check container status
ssh deploy@staging.michaelschiemer.de \
  "cd /opt/framework-staging/current && docker-compose ps"

Test Application:
- Perform manual testing
- Run automated tests
- Verify feature functionality
- Check performance

If Issues Found:

# Rollback staging
./deployment/scripts/rollback.sh staging

# Or continue testing for non-critical issues

Production Deployment Workflow

Step-by-Step Process:

Pre-Deployment Checklist:
- Code reviewed and approved
- Successfully deployed and tested in staging
- Database migrations tested
- Backup plan confirmed
- Rollback plan confirmed
- Team notified of deployment window

Prepare Production Branch:

# Merge staging to main
git checkout main
git merge staging
git push origin main

Verify Environment Variables:

# Required variables
echo $PRODUCTION_HOST  # Should be: michaelschiemer.de
echo $PRODUCTION_USER  # Should be: deploy

# If not set
export PRODUCTION_HOST=michaelschiemer.de
export PRODUCTION_USER=deploy

Deploy to Production:

# IMPORTANT: Do NOT skip database backup
./deployment/scripts/deploy-production.sh

# Monitor output carefully for any errors

Post-Deployment Verification:

# 1. Check main application
curl -k https://michaelschiemer.de/

# 2. Check API health
curl -k https://michaelschiemer.de/api/health

# 3. Monitor logs for errors
ssh deploy@michaelschiemer.de \
  "cd /opt/framework-production/current && docker-compose logs -f --tail=100"

# 4. Check container status
ssh deploy@michaelschiemer.de \
  "cd /opt/framework-production/current && docker-compose ps"

# 5. Verify database migrations applied
ssh deploy@michaelschiemer.de \
  "cd /opt/framework-production/current && \
   docker-compose exec production-app php console.php db:status"

Smoke Testing:
- Test critical user paths
- Verify authentication
- Test key API endpoints
- Check database connectivity
- Verify external integrations

If Deployment Fails:

# Automatic rollback should have occurred
# If manual rollback needed:
./deployment/scripts/rollback.sh production

# Monitor rollback
ssh deploy@michaelschiemer.de \
  "cd /opt/framework-production/current && docker-compose logs -f"

Post-Deployment:
- Monitor application metrics
- Watch error logs for 30 minutes
- Notify team of successful deployment
- Document any issues encountered

Troubleshooting

SSH Connection Issues

Problem: Permission denied (publickey)

Solutions:

# Verify SSH key exists
ls -la ~/.ssh/framework-deploy*

# Test SSH connection
ssh -i ~/.ssh/framework-deploy deploy@staging.michaelschiemer.de "echo 'SSH works'"

# Check SSH config
cat ~/.ssh/config

# Re-copy public key
ssh-copy-id -i ~/.ssh/framework-deploy.pub deploy@staging.michaelschiemer.de

# Check server-side authorized_keys
ssh deploy@staging.michaelschiemer.de "cat ~/.ssh/authorized_keys"

Docker Build Failures

Problem: Docker build fails during deployment

Solutions:

# Check Docker is running
docker info

# Test build locally
docker build \
  --file docker/php/Dockerfile \
  --tag localhost:5000/framework:test \
  --build-arg ENV=staging \
  .

# Check Dockerfile syntax
docker build --file docker/php/Dockerfile --no-cache .

# Clear Docker cache
docker system prune -a

Registry Push Failures

Problem: docker push fails

Solutions:

# Check registry is accessible
curl http://localhost:5000/v2/

# Verify image exists locally
docker images | grep framework

# Test manual push
docker push localhost:5000/framework:staging

# Check registry logs
docker logs registry  # If running registry as container

Deployment Script Fails

Problem: Deployment script exits with error

Solutions:

# Run with bash debug mode
bash -x ./deployment/scripts/deploy-staging.sh

# Check remote directory exists
ssh deploy@staging.michaelschiemer.de "ls -la /opt/framework-staging"

# Verify Docker Compose files
ssh deploy@staging.michaelschiemer.de \
  "cd /opt/framework-staging/current && docker-compose config"

# Check deployment logs on server
ssh deploy@staging.michaelschiemer.de \
  "cd /opt/framework-staging/current && docker-compose logs"

Health Check Failures

Problem: Health checks fail but containers are running

Solutions:

# Check container logs
ssh deploy@staging.michaelschiemer.de \
  "cd /opt/framework-staging/current && docker-compose logs --tail=50"

# Check PHP-FPM status
ssh deploy@staging.michaelschiemer.de \
  "cd /opt/framework-staging/current && \
   docker-compose exec staging-app pgrep php-fpm"

# Test health endpoint manually
ssh deploy@staging.michaelschiemer.de \
  "curl -k http://localhost/health"

# Check Nginx configuration
ssh deploy@staging.michaelschiemer.de \
  "cd /opt/framework-staging/current && \
   docker-compose exec staging-nginx nginx -t"

Rollback Issues

Problem: Rollback script fails

Solutions:

# List available backups
ssh deploy@production \
  "cd /opt/framework-production && ls -dt backup_*"

# Manually restore backup
ssh deploy@production "
  cd /opt/framework-production
  docker-compose -f current/docker-compose.base.yml \
                 -f current/docker-compose.prod.yml down
  rm -rf current
  cp -r backup_20250124_153045 current
  cd current
  docker-compose -f docker-compose.base.yml \
                 -f docker-compose.prod.yml up -d
"

# Check failed deployment archive
ssh deploy@production "ls -dt /opt/framework-production/failed_*"

Database Migration Failures

Problem: Migrations fail during deployment

Solutions:

# Check migration status
ssh deploy@production \
  "cd /opt/framework-production/current && \
   docker-compose exec production-app php console.php db:status"

# Manually run migrations
ssh deploy@production \
  "cd /opt/framework-production/current && \
   docker-compose exec production-app php console.php db:migrate --force"

# Rollback migrations
ssh deploy@production \
  "cd /opt/framework-production/current && \
   docker-compose exec production-app php console.php db:rollback"

# Check database connectivity
ssh deploy@production \
  "cd /opt/framework-production/current && \
   docker-compose exec production-app php console.php db:check"

Security Best Practices

SSH Key Management

✅ Do:

Use 4096-bit RSA keys minimum
Generate separate keys for staging and production
Store private keys securely (never commit to git)
Rotate keys quarterly
Use SSH config for key management

❌ Don't:

Use password-only authentication
Share keys between environments
Commit private keys to version control
Use personal SSH keys for deployments

Environment Variables

✅ Do:

Use environment variables for secrets
Document required variables
Use different credentials per environment
Validate variables before deployment

❌ Don't:

Hard-code credentials in scripts
Commit .env files with secrets
Use production credentials in staging

Deployment User Permissions

Recommended Setup:

# On remote server
# Create deployment user
sudo useradd -m -s /bin/bash deploy

# Add to docker group
sudo usermod -aG docker deploy

# Set directory ownership
sudo chown -R deploy:deploy /opt/framework-staging
sudo chown -R deploy:deploy /opt/framework-production

# Restrict sudo (if needed)
# Add to /etc/sudoers.d/deploy
deploy ALL=(ALL) NOPASSWD: /usr/bin/docker, /usr/bin/docker-compose

Backup Management

✅ Do:

Automate database backups
Keep multiple backup versions
Test backup restoration regularly
Monitor backup disk space

❌ Don't:

Skip backups in production
Keep unlimited backups (disk space)
Store backups only on deployment server

Monitoring and Maintenance

Health Monitoring

Automated Checks:

# Cron job for health monitoring
# Add to crontab -e on deployment server
*/5 * * * * curl -f -k https://michaelschiemer.de/health || echo "Health check failed" | mail -s "Production Health Alert" admin@michaelschiemer.de

Manual Checks:

# Check all services
ssh deploy@production \
  "cd /opt/framework-production/current && docker-compose ps"

# Check resource usage
ssh deploy@production "docker stats --no-stream"

# Check disk space
ssh deploy@production "df -h /opt/framework-production"

Log Management

View Logs:

# Follow logs
ssh deploy@production \
  "cd /opt/framework-production/current && docker-compose logs -f"

# View specific service logs
ssh deploy@production \
  "cd /opt/framework-production/current && \
   docker-compose logs -f production-app"

# Last 100 lines
ssh deploy@production \
  "cd /opt/framework-production/current && \
   docker-compose logs --tail=100"

Backup Cleanup

Manual Cleanup:

# List backups by size
ssh deploy@production "du -sh /opt/framework-production/backup_* | sort -h"

# Remove specific old backup
ssh deploy@production "rm -rf /opt/framework-production/backup_20240101_000000"

# Keep only last 5 backups
ssh deploy@staging "
  cd /opt/framework-staging
  ls -dt backup_* | tail -n +6 | xargs rm -rf
"

Appendix

Directory Structure

Local Project:

michaelschiemer/
├── deployment/
│   ├── scripts/
│   │   ├── deploy-staging.sh       # Staging deployment
│   │   ├── deploy-production.sh    # Production deployment
│   │   └── rollback.sh             # Rollback script
│   ├── docs/
│   │   └── DEPLOYMENT_GUIDE.md     # This file
│   └── legacy/
│       └── gitea-workflows/        # Archived Gitea workflows
├── docker-compose.base.yml
├── docker-compose.staging.yml
├── docker-compose.prod.yml
└── docker/
    └── php/
        └── Dockerfile

Remote Server:

/opt/framework-staging/    or    /opt/framework-production/
├── current/                          # Active deployment
│   ├── docker-compose.base.yml
│   ├── docker-compose.staging.yml
│   ├── docker/
│   └── deploy.sh
├── backup_20250124_153045/          # Timestamped backups
├── backup_20250124_120000/
├── backup_20250123_183015/
└── failed_20250124_154512/          # Failed deployment (if rollback occurred)

Environment Variable Reference

Variable	Required	Default	Description
`STAGING_HOST`	Yes*	staging.michaelschiemer.de	Staging server hostname/IP
`STAGING_USER`	No	deploy	Staging SSH user
`STAGING_SSH_PORT`	No	22	Staging SSH port
`PRODUCTION_HOST`	Yes*	michaelschiemer.de	Production server hostname/IP
`PRODUCTION_USER`	No	deploy	Production SSH user
`PRODUCTION_SSH_PORT`	No	22	Production SSH port
`REGISTRY`	No	localhost:5000	Docker registry URL
`IMAGE_NAME`	No	framework	Docker image name
`IMAGE_TAG`	No	staging/latest	Docker image tag
`SKIP_BACKUP`	No	false	Skip database backup (production)
`FORCE_REBUILD`	No	false	Force Docker image rebuild

*Required for respective deployment type

Common Commands Reference

Local Commands:

# Deploy staging
./deployment/scripts/deploy-staging.sh

# Deploy production
./deployment/scripts/deploy-production.sh

# Rollback staging
./deployment/scripts/rollback.sh staging

# Rollback production
./deployment/scripts/rollback.sh production

# Test SSH connection
ssh deploy@staging.michaelschiemer.de "echo 'SSH works'"

Remote Commands (via SSH):

# View logs
docker-compose logs -f

# Check status
docker-compose ps

# Restart services
docker-compose restart

# Stop services
docker-compose down

# Start services
docker-compose up -d

# Execute command in container
docker-compose exec production-app php console.php db:status

# View container logs
docker-compose logs production-app --tail=50

Last Updated: 2025-01-24 Framework Version: 2.x Deployment Method: SSH-based deployment scripts

23 KiB Raw Blame History

SSH Deployment Guide

Overview

Prerequisites

Required Software

SSH Key Setup

Environment Variables

Deployment Scripts

1. Staging Deployment

2. Production Deployment

3. Rollback Script

Deployment Workflows

Staging Deployment Workflow

Production Deployment Workflow

Troubleshooting

SSH Connection Issues

Docker Build Failures

Registry Push Failures

Deployment Script Fails

Health Check Failures

Rollback Issues

Database Migration Failures

Security Best Practices

SSH Key Management

Environment Variables

Deployment User Permissions

Backup Management

Monitoring and Maintenance

Health Monitoring

Log Management

Backup Cleanup

Appendix

Directory Structure

Environment Variable Reference

Common Commands Reference

23 KiB

Raw Blame History