fix(Discovery): Add comprehensive debug logging for router initialization

- Add initializer count logging in DiscoveryServiceBootstrapper - Add route structure analysis in RouterSetup - Add request parameter logging in HttpRouter - Update PHP production config for better OPcache handling - Fix various config and error handling improvements
2025-10-27 22:23:18 +01:00
parent e326e3d6c6
commit 70e45fb56e
56 changed files with 1519 additions and 355 deletions
--- a/deployment/infrastructure/DEPLOYMENT_ANALYSIS.md
+++ b/deployment/infrastructure/DEPLOYMENT_ANALYSIS.md
@@ -0,0 +1,554 @@
+# Production Deployment Analysis & Fix Strategy
+
+**Date**: 2025-10-27
+**Status**: CRITICAL - Production website returning HTTP 500 errors
+**Root Cause**: Database connection configuration error (DB_PORT mismatch)
+
+---
+
+## 1. Complete Deployment Flow Analysis
+
+### Deployment Architecture
+
+The project uses a **release-based deployment pattern** with shared configuration:
+
+```
+/home/deploy/michaelschiemer/
+├── releases/
+│   ├── 1761566515/          # Current release (timestamped)
+│   ├── 1761565432/          # Previous releases
+│   └── ...
+├── shared/
+│   └── .env.production      # Shared configuration file
+└── current -> releases/1761566515/  # Symlink to active release
+```
+
+**Key Characteristics**:
+- **Releases Directory**: Each deployment creates a new timestamped release
+- **Shared Directory**: Configuration files persist across deployments
+- **Current Symlink**: Points to the active release
+- **Symlink Chain**: `current/.env.production` → `shared/.env.production` → Used by application
+
+### .env File Sources (3 Different Files Identified)
+
+#### 1. Root Directory: `/home/michael/dev/michaelschiemer/.env.production`
+- **Size**: 2.9K
+- **Checksum**: 9f33068713432c1dc4008724dc6923b0
+- **DB_PORT**: 5432 (CORRECT for PostgreSQL)
+- **DB_USERNAME**: mdb_user (with underscore)
+- **DB_PASSWORD**: Qo2KNgGqeYksEhKr57pgugakxlothn8J
+- **Purpose**: Framework default configuration
+- **Status**: CORRECT database configuration
+
+#### 2. Deployment Directory: `/home/michael/dev/michaelschiemer/deployment/applications/environments/.env.production`
+- **Size**: 4.3K
+- **Checksum**: b516bf86beed813df03a30f655687b72
+- **DB_PORT**: 5432 (CORRECT for PostgreSQL)
+- **DB_USERNAME**: mdb_user (with underscore)
+- **DB_PASSWORD**: Qo2KNgGqeYksEhKr57pgugakxlothn8J
+- **Purpose**: Application-specific production configuration
+- **Status**: CORRECT and MORE COMPLETE (includes Redis, Queue, Mail, Monitoring configs)
+
+#### 3. Production Server: `/home/deploy/michaelschiemer/shared/.env.production`
+- **Size**: 3.0K (modified Oct 26 20:56)
+- **Line 15**: `DB_PORT=3306` (WRONG - MySQL port instead of PostgreSQL)
+- **Line 67**: `DB_PORT=` (duplicate empty entry)
+- **DB_USERNAME**: mdb-user (with hyphen - likely wrong)
+- **DB_PASSWORD**: StartSimple2024! (different from local configs)
+- **Status**: CORRUPTED - Wrong database configuration causing HTTP 500 errors
+
+### Deployment Playbook Flow
+
+**File**: `/home/michael/dev/michaelschiemer/deployment/infrastructure/playbooks/deploy-rsync-based.yml`
+
+**Critical Configuration**:
+```yaml
+local_project_path: "{{ playbook_dir }}/../../.."  # 3 dirs up = /home/michael/dev/michaelschiemer
+shared_files:
+  - .env.production  # Marked as SHARED file
+rsync_excludes:
+  - .env
+  - .env.local
+  - .env.development
+```
+
+**Deployment Steps**:
+1. **Rsync files** from `{{ local_project_path }}` (framework root) to release directory
+   - Excludes: `.env`, `.env.local`, `.env.development`
+   - Includes: `.env.production` from root directory
+2. **Create release directory**: `/home/deploy/michaelschiemer/releases/{{ timestamp }}`
+3. **Copy files** to release directory
+4. **Create symlinks**:
+   - `release/.env.production` → `../../shared/.env.production`
+   - `release/.env` → `../../shared/.env.production`
+5. **Update current** symlink → latest release
+6. **Restart containers** via docker-compose
+
+**CRITICAL ISSUE IDENTIFIED**:
+The playbook does NOT have a task to initially copy `.env.production` to `shared/.env.production`. It only creates symlinks assuming the file already exists. This means:
+- Initial setup requires MANUAL copy of `.env.production` to `shared/`
+- Updates to `.env.production` require MANUAL sync to production server
+- The rsync'd `.env.production` in release directory is IGNORED (symlink overrides it)
+
+---
+
+## 2. Production Server .env Status
+
+### Current State (BROKEN)
+
+```bash
+# /home/deploy/michaelschiemer/shared/.env.production
+
+Line 15: DB_PORT=3306          # WRONG - MySQL port (should be 5432 for PostgreSQL)
+Line 67: DB_PORT=               # Duplicate empty entry
+
+DB_USERNAME=mdb-user            # Wrong format (should be mdb_user with underscore)
+DB_PASSWORD=StartSimple2024!    # Wrong password (doesn't match local configs)
+```
+
+### Container Status
+
+```
+CONTAINER     STATUS                    ISSUE
+php           Up 27 minutes (healthy)   -
+db            Up 40 minutes (healthy)   PostgreSQL running on port 5432
+redis         Up 40 minutes (healthy)   -
+web           Up 40 minutes (UNHEALTHY) Nginx cannot connect to PHP due to DB error
+queue-worker  Restarting (1) 4s ago     PHP crashing due to DB connection error
+```
+
+### Error Pattern
+
+- **HTTP 500** on all requests (/, /impressum, etc.)
+- **Root Cause**: PHP application cannot connect to database because:
+  1. `DB_PORT=3306` (MySQL) instead of `5432` (PostgreSQL)
+  2. Wrong username format (`mdb-user` vs `mdb_user`)
+  3. Wrong password
+- **Impact**: All PHP processes fail to initialize → Nginx returns 500
+
+---
+
+## 3. Deployment Command Documentation
+
+### WORKING Commands (Current Playbook)
+
+#### Deploy via Ansible Playbook
+```bash
+cd /home/michael/dev/michaelschiemer/deployment/infrastructure
+
+# Full production deployment
+ansible-playbook \
+  -i inventories/production/hosts.yml \
+  playbooks/deploy-rsync-based.yml \
+  --vault-password-file .vault_pass
+
+# With specific variables
+ansible-playbook \
+  -i inventories/production/hosts.yml \
+  playbooks/deploy-rsync-based.yml \
+  --vault-password-file .vault_pass \
+  -e "deployment_branch=main"
+```
+
+#### Check Production Status
+```bash
+# Check containers
+ansible web_servers \
+  -i inventories/production/hosts.yml \
+  -m shell -a "docker ps -a" \
+  --vault-password-file .vault_pass
+
+# Check .env configuration
+ansible web_servers \
+  -i inventories/production/hosts.yml \
+  -m shell -a "cat /home/deploy/michaelschiemer/shared/.env.production" \
+  --vault-password-file .vault_pass
+
+# Check application logs
+ansible web_servers \
+  -i inventories/production/hosts.yml \
+  -m shell -a "docker logs web --tail 50" \
+  --vault-password-file .vault_pass
+```
+
+### COMMANDS TO CREATE (User Requirements)
+
+#### 1. Simple Manual Deploy Script
+```bash
+#!/bin/bash
+# File: /home/michael/dev/michaelschiemer/deployment/infrastructure/scripts/deploy.sh
+
+set -e
+
+cd "$(dirname "$0")/.."
+
+echo "🚀 Deploying to production..."
+
+ansible-playbook \
+  -i inventories/production/hosts.yml \
+  playbooks/deploy-rsync-based.yml \
+  --vault-password-file .vault_pass
+
+echo "✅ Deployment complete!"
+echo "🔍 Check status: docker ps"
+```
+
+#### 2. .env Update Script
+```bash
+#!/bin/bash
+# File: /home/michael/dev/michaelschiemer/deployment/infrastructure/scripts/update-env.sh
+
+set -e
+
+cd "$(dirname "$0")/../.."
+
+SOURCE_ENV="deployment/applications/environments/.env.production"
+REMOTE_PATH="/home/deploy/michaelschiemer/shared/.env.production"
+
+if [[ ! -f "$SOURCE_ENV" ]]; then
+    echo "❌ Source .env.production not found at: $SOURCE_ENV"
+    exit 1
+fi
+
+echo "📤 Uploading .env.production to production server..."
+
+ansible web_servers \
+  -i deployment/infrastructure/inventories/production/hosts.yml \
+  -m copy \
+  -a "src=$SOURCE_ENV dest=$REMOTE_PATH mode=0644" \
+  --vault-password-file deployment/infrastructure/.vault_pass
+
+echo "🔄 Restarting containers..."
+
+ansible web_servers \
+  -i deployment/infrastructure/inventories/production/hosts.yml \
+  -m shell \
+  -a "cd /home/deploy/michaelschiemer/current && docker-compose restart php web queue-worker" \
+  --vault-password-file deployment/infrastructure/.vault_pass
+
+echo "✅ .env.production updated and containers restarted!"
+```
+
+#### 3. Quick Production Sync
+```bash
+#!/bin/bash
+# File: /home/michael/dev/michaelschiemer/deployment/infrastructure/scripts/quick-sync.sh
+
+set -e
+
+cd "$(dirname "$0")/../.."
+
+# Sync code changes (no .env update)
+rsync -avz \
+  --exclude '.env' \
+  --exclude '.env.local' \
+  --exclude 'node_modules/' \
+  --exclude '.git/' \
+  ./ deploy@94.16.110.151:/home/deploy/michaelschiemer/current/
+
+# Restart containers
+ansible web_servers \
+  -i deployment/infrastructure/inventories/production/hosts.yml \
+  -m shell \
+  -a "cd /home/deploy/michaelschiemer/current && docker-compose restart php web" \
+  --vault-password-file deployment/infrastructure/.vault_pass
+
+echo "✅ Quick sync complete!"
+```
+
+### SCRIPTS TO REMOVE (Unused/Deprecated)
+
+1. **`/home/michael/dev/michaelschiemer/deploy.sh`** (if exists in root)
+   - Reason: Conflicting with playbook-based deployment
+
+2. **`/home/michael/dev/michaelschiemer/.env.local`** (if exists)
+   - Reason: Not used in production, causes confusion
+
+3. **Duplicate .env files** in root:
+   - Keep: `.env.production` (source of truth for framework defaults)
+   - Remove: `.env.backup.*`, `.env.old`, etc.
+
+---
+
+## 4. Fix Strategy (Step-by-Step)
+
+### IMMEDIATE FIX (Restore Production)
+
+#### Step 1: Update Production .env.production File
+
+```bash
+cd /home/michael/dev/michaelschiemer
+
+# Copy correct .env.production to production server
+ansible web_servers \
+  -i deployment/infrastructure/inventories/production/hosts.yml \
+  -m copy \
+  -a "src=deployment/applications/environments/.env.production dest=/home/deploy/michaelschiemer/shared/.env.production mode=0644" \
+  --vault-password-file deployment/infrastructure/.vault_pass
+```
+
+**Why this file?**
+- Most complete configuration (4.3K vs 2.9K)
+- Includes Redis, Queue, Mail, Monitoring configs
+- Correct DB_PORT=5432
+- Correct DB credentials
+
+#### Step 2: Verify .env.production on Server
+
+```bash
+ansible web_servers \
+  -i deployment/infrastructure/inventories/production/hosts.yml \
+  -m shell \
+  -a "grep -E '(DB_PORT|DB_USERNAME|DB_PASSWORD)' /home/deploy/michaelschiemer/shared/.env.production" \
+  --vault-password-file deployment/infrastructure/.vault_pass
+```
+
+**Expected Output**:
+```
+DB_PORT=5432
+DB_USERNAME=mdb_user
+DB_PASSWORD=Qo2KNgGqeYksEhKr57pgugakxlothn8J
+```
+
+#### Step 3: Restart Containers
+
+```bash
+ansible web_servers \
+  -i deployment/infrastructure/inventories/production/hosts.yml \
+  -m shell \
+  -a "cd /home/deploy/michaelschiemer/current && docker-compose restart php web queue-worker" \
+  --vault-password-file deployment/infrastructure/.vault_pass
+```
+
+#### Step 4: Verify Website Functionality
+
+```bash
+# Check HTTP status
+curl -I https://michaelschiemer.de
+
+# Expected: HTTP/2 200 OK (instead of 500)
+
+# Check container health
+ansible web_servers \
+  -i deployment/infrastructure/inventories/production/hosts.yml \
+  -m shell \
+  -a "docker ps | grep -E '(web|php|queue-worker)'" \
+  --vault-password-file deployment/infrastructure/.vault_pass
+```
+
+**Expected**: All containers should be "Up" and "healthy"
+
+### LONG-TERM FIX (Prevent Future Issues)
+
+#### 1. Update Playbook to Sync .env.production
+
+Add task to `deploy-rsync-based.yml`:
+
+```yaml
+# After "Synchronize project files" task, add:
+
+- name: Sync .env.production to shared directory
+  copy:
+    src: "{{ local_project_path }}/deployment/applications/environments/.env.production"
+    dest: "{{ project_path }}/shared/.env.production"
+    mode: '0644'
+  when: sync_env_to_shared | default(true)
+  tags:
+    - deploy
+    - config
+```
+
+#### 2. Create Helper Scripts
+
+Create the 3 scripts documented in section 3:
+- `scripts/deploy.sh` - Simple wrapper for playbook
+- `scripts/update-env.sh` - Update .env.production only
+- `scripts/quick-sync.sh` - Quick code sync without full deployment
+
+#### 3. Establish Source of Truth
+
+**Decision**: Use `deployment/applications/environments/.env.production` as source of truth
+- Most complete configuration
+- Application-specific settings
+- Includes all production services
+
+**Action**: Document in README.md:
+```markdown
+## Production Configuration
+
+**Source of Truth**: `deployment/applications/environments/.env.production`
+
+To update production .env:
+1. Edit `deployment/applications/environments/.env.production`
+2. Run `./deployment/infrastructure/scripts/update-env.sh`
+3. Containers will auto-restart with new config
+```
+
+#### 4. Add .env Validation
+
+Create pre-deployment validation script:
+
+```bash
+#!/bin/bash
+# scripts/validate-env.sh
+
+ENV_FILE="deployment/applications/environments/.env.production"
+
+echo "🔍 Validating .env.production..."
+
+# Check required variables
+REQUIRED_VARS=(
+    "DB_DRIVER"
+    "DB_HOST"
+    "DB_PORT"
+    "DB_DATABASE"
+    "DB_USERNAME"
+    "DB_PASSWORD"
+)
+
+for var in "${REQUIRED_VARS[@]}"; do
+    if ! grep -q "^${var}=" "$ENV_FILE"; then
+        echo "❌ Missing required variable: $var"
+        exit 1
+    fi
+done
+
+# Check PostgreSQL port
+if ! grep -q "^DB_PORT=5432" "$ENV_FILE"; then
+    echo "⚠️  Warning: DB_PORT should be 5432 for PostgreSQL"
+fi
+
+echo "✅ .env.production validation passed"
+```
+
+---
+
+## 5. Cleanup Recommendations
+
+### Files to Remove
+
+#### In Framework Root (`/home/michael/dev/michaelschiemer/`)
+```bash
+# List files to remove
+find . -maxdepth 1 -name ".env.backup*" -o -name ".env.old*" -o -name ".env.local"
+
+# Remove after confirmation
+rm -f .env.backup* .env.old* .env.local
+```
+
+#### In Deployment Directory
+```bash
+# Check for duplicate/old deployment scripts
+find deployment/ -name "deploy-old.yml" -o -name "*.backup"
+```
+
+#### On Production Server
+```bash
+# Clean up old releases (keep last 5)
+ansible web_servers \
+  -i deployment/infrastructure/inventories/production/hosts.yml \
+  -m shell \
+  -a "cd /home/deploy/michaelschiemer/releases && ls -t | tail -n +6 | xargs rm -rf" \
+  --vault-password-file deployment/infrastructure/.vault_pass
+
+# Remove duplicate .env files in current release
+ansible web_servers \
+  -i deployment/infrastructure/inventories/production/hosts.yml \
+  -m shell \
+  -a "cd /home/deploy/michaelschiemer/current && rm -f .env.backup* .env.old*" \
+  --vault-password-file deployment/infrastructure/.vault_pass
+```
+
+### Configuration to Keep
+
+**Essential Files**:
+- `/.env.production` - Framework defaults (keep for reference)
+- `/deployment/applications/environments/.env.production` - Source of truth
+- `/deployment/infrastructure/playbooks/deploy-rsync-based.yml` - Main playbook
+- `/deployment/infrastructure/inventories/production/hosts.yml` - Inventory
+
+**Symlinks (Do Not Remove)**:
+- `/home/deploy/michaelschiemer/current/.env.production` → `shared/.env.production`
+- `/home/deploy/michaelschiemer/current/.env` → `shared/.env.production`
+
+---
+
+## 6. Post-Fix Verification Checklist
+
+```bash
+# 1. Website accessible
+curl -I https://michaelschiemer.de
+# Expected: HTTP/2 200 OK
+
+# 2. All containers healthy
+ansible web_servers \
+  -i deployment/infrastructure/inventories/production/hosts.yml \
+  -m shell -a "docker ps" \
+  --vault-password-file deployment/infrastructure/.vault_pass
+# Expected: All "Up" and "(healthy)"
+
+# 3. Database connection working
+ansible web_servers \
+  -i deployment/infrastructure/inventories/production/hosts.yml \
+  -m shell -a "docker exec php php -r \"new PDO('pgsql:host=db;port=5432;dbname=michaelschiemer', 'mdb_user', 'Qo2KNgGqeYksEhKr57pgugakxlothn8J');\"" \
+  --vault-password-file deployment/infrastructure/.vault_pass
+# Expected: No errors
+
+# 4. Application logs clean
+ansible web_servers \
+  -i deployment/infrastructure/inventories/production/hosts.yml \
+  -m shell -a "docker logs web --tail 20" \
+  --vault-password-file deployment/infrastructure/.vault_pass
+# Expected: HTTP 200 responses, no 500 errors
+
+# 5. Queue worker stable
+ansible web_servers \
+  -i deployment/infrastructure/inventories/production/hosts.yml \
+  -m shell -a "docker ps | grep queue-worker" \
+  --vault-password-file deployment/infrastructure/.vault_pass
+# Expected: "Up" status (not "Restarting")
+```
+
+---
+
+## 7. Future Deployment Best Practices
+
+1. **Always validate .env before deployment**
+   - Run `scripts/validate-env.sh` pre-deployment
+   - Check DB_PORT=5432 for PostgreSQL
+   - Verify credentials match database server
+
+2. **Use playbook for all deployments**
+   - Consistent process
+   - Automated rollback capability
+   - Proper symlink management
+
+3. **Monitor container health post-deployment**
+   - Check `docker ps` output
+   - Verify all containers "(healthy)"
+   - Check application logs for errors
+
+4. **Keep .env.production in sync**
+   - Single source of truth: `deployment/applications/environments/.env.production`
+   - Use `update-env.sh` script for updates
+   - Never manually edit on production server
+
+5. **Regular backups**
+   - Backup `shared/.env.production` before changes
+   - Keep last 5 releases for quick rollback
+   - Document any manual production changes
+
+---
+
+## Summary
+
+**Current Status**: Production broken due to DB_PORT configuration error
+**Root Cause**: Manual edits to `shared/.env.production` with wrong PostgreSQL port
+**Fix Time**: ~5 minutes (copy correct .env + restart containers)
+**Prevention**: Automated .env sync in playbook + validation scripts
+
+**Next Steps**:
+1. Execute Step 1-4 of Fix Strategy (IMMEDIATE)
+2. Verify website returns HTTP 200
+3. Implement long-term fixes (playbook updates, scripts)
+4. Document deployment process in README.md
--- a/deployment/infrastructure/DEPLOYMENT_FIX_SUMMARY.md
+++ b/deployment/infrastructure/DEPLOYMENT_FIX_SUMMARY.md
@@ -0,0 +1,286 @@
+# Production Deployment Fix Summary
+
+**Date**: 2025-10-27
+**Status**: PARTIALLY FIXED - DB configuration corrected, but additional issues remain
+
+---
+
+## What Was Fixed
+
+### 1. Database Configuration Corrected ✅
+
+**Problem**: Wrong DB_PORT in production `.env.production`
+- Line 15: `DB_PORT=3306` (MySQL port)
+- Line 67: `DB_PORT=` (duplicate empty entry)
+- Wrong username: `mdb-user` (should be `mdb_user`)
+- Wrong password
+
+**Solution Applied**:
+```bash
+# Copied correct .env.production from source of truth
+ansible web_servers -m copy \
+  -a "src=deployment/applications/environments/.env.production \
+      dest=/home/deploy/michaelschiemer/shared/.env.production" \
+  --vault-password-file deployment/infrastructure/.vault_pass
+```
+
+**Verification**:
+```bash
+DB_PORT=5432                              # ✅ Correct
+DB_USERNAME=mdb_user                      # ✅ Correct
+DB_PASSWORD=Qo2KNgGqeYksEhKr57pgugakxlothn8J  # ✅ Correct
+```
+
+### 2. Containers Restarted ✅
+
+```bash
+docker compose restart php web queue-worker
+```
+
+**Current Status**:
+- **php**: Up 6 minutes (healthy) ✅
+- **db**: Up 53 minutes (healthy) ✅
+- **redis**: Up 53 minutes (healthy) ✅
+- **web**: Up 6 minutes (UNHEALTHY) ⚠️
+- **queue-worker**: Restarting (1) ❌
+
+---
+
+## Remaining Issues
+
+### Issue 1: Web Container Unhealthy ⚠️
+
+**Symptom**: Website still returns HTTP 500
+
+**Possible Causes**:
+1. **PHP-FPM not responding** - Web container can't connect to PHP
+2. **Application error** - PHP code failing during bootstrap
+3. **Missing files** - Application files not properly deployed
+4. **Permissions** - Web server can't access application files
+
+**Next Steps to Diagnose**:
+```bash
+# Check if PHP-FPM is accessible from web container
+docker exec web curl http://php:9000
+
+# Check Nginx configuration
+docker exec web nginx -t
+
+# Check web container health check
+docker inspect web --format='{{json .State.Health}}' | jq
+
+# Check if application files exist
+docker exec web ls -la /var/www/html/public/index.php
+```
+
+### Issue 2: Queue Worker Crashing ❌
+
+**Symptom**: Continuous restart loop
+
+**Possible Causes**:
+1. **Same DB connection issue** (should be fixed now)
+2. **Missing queue configuration**
+3. **Redis connection issue**
+4. **Application code error in queue worker**
+
+**Next Steps to Diagnose**:
+```bash
+# Check queue-worker logs
+docker logs queue-worker --tail 100
+
+# Try running queue worker manually
+docker exec php php artisan queue:work --tries=1 --once
+```
+
+---
+
+## Scripts Created ✅
+
+### 1. Simple Deployment Script
+**Location**: `/home/michael/dev/michaelschiemer/deployment/infrastructure/scripts/deploy.sh`
+```bash
+./deployment/infrastructure/scripts/deploy.sh
+```
+
+### 2. .env Update Script
+**Location**: `/home/michael/dev/michaelschiemer/deployment/infrastructure/scripts/update-env.sh`
+```bash
+./deployment/infrastructure/scripts/update-env.sh
+```
+
+### 3. Quick Sync Script
+**Location**: `/home/michael/dev/michaelschiemer/deployment/infrastructure/scripts/quick-sync.sh`
+```bash
+./deployment/infrastructure/scripts/quick-sync.sh
+```
+
+**Note**: All scripts updated to use `docker compose` (v2) instead of `docker-compose` (v1)
+
+---
+
+## Documentation Created ✅
+
+### Comprehensive Deployment Analysis
+**Location**: `/home/michael/dev/michaelschiemer/deployment/infrastructure/DEPLOYMENT_ANALYSIS.md`
+
+**Contents**:
+1. Complete deployment flow analysis
+2. .env file sources and conflicts
+3. Deployment command documentation
+4. Step-by-step fix strategy
+5. Cleanup recommendations
+6. Post-fix verification checklist
+
+---
+
+## Recommended Next Actions
+
+### Immediate (To Fix HTTP 500)
+
+1. **Check Application Bootstrap**:
+```bash
+# Test if PHP application can start
+ansible web_servers -m shell \
+  -a "docker exec php php /var/www/html/public/index.php" \
+  --vault-password-file deployment/infrastructure/.vault_pass
+```
+
+2. **Check Nginx-PHP Connection**:
+```bash
+# Test PHP-FPM socket
+ansible web_servers -m shell \
+  -a "docker exec web curl -v http://php:9000" \
+  --vault-password-file deployment/infrastructure/.vault_pass
+```
+
+3. **Check Application Logs**:
+```bash
+# Look for PHP errors
+ansible web_servers -m shell \
+  -a "docker exec php ls -la /var/www/html/storage/logs/" \
+  --vault-password-file deployment/infrastructure/.vault_pass
+```
+
+4. **Verify File Permissions**:
+```bash
+# Check if web server can read files
+ansible web_servers -m shell \
+  -a "docker exec web ls -la /var/www/html/public/" \
+  --vault-password-file deployment/infrastructure/.vault_pass
+```
+
+### Short-Term (Within 24h)
+
+1. **Fix Web Container Health** - Resolve HTTP 500 errors
+2. **Fix Queue Worker** - Stop crash loop
+3. **Full Deployment Test** - Run complete deployment playbook
+4. **Verify All Services** - Ensure all containers healthy
+
+### Long-Term (This Week)
+
+1. **Update Playbook** - Add .env.production sync task
+2. **Add Validation** - Pre-deployment .env validation script
+3. **Document Process** - Update README with deployment guide
+4. **Setup Monitoring** - Add health check alerts
+5. **Cleanup Old Files** - Remove duplicate .env files
+
+---
+
+## Key Learnings
+
+### 1. Deployment Flow Issues
+
+**Problem**: Playbook doesn't sync `.env.production` to `shared/`
+**Impact**: Manual updates required for configuration changes
+**Solution**: Add sync task to playbook
+
+### 2. Multiple .env Sources
+
+**Problem**: 3 different `.env.production` files with conflicting content
+**Resolution**: Use `deployment/applications/environments/.env.production` as source of truth
+
+### 3. Docker Compose Version
+
+**Problem**: Production uses Docker Compose v2 (`docker compose`)
+**Impact**: Scripts using v1 syntax (`docker-compose`) fail
+**Solution**: All scripts updated to v2 syntax
+
+### 4. Symlink Chain Complexity
+
+**Structure**:
+```
+current/.env → shared/.env.production
+current/.env.production → shared/.env.production
+```
+
+**Risk**: If `shared/.env.production` is wrong, ALL releases break
+**Mitigation**: Validate before deploy, backup before changes
+
+---
+
+## Quick Reference
+
+### Check Production Status
+```bash
+cd /home/michael/dev/michaelschiemer/deployment/infrastructure
+
+# Container status
+ansible web_servers -i inventories/production/hosts.yml \
+  -m shell -a "docker ps" --vault-password-file .vault_pass
+
+# .env configuration
+ansible web_servers -i inventories/production/hosts.yml \
+  -m shell -a "cat /home/deploy/michaelschiemer/shared/.env.production" \
+  --vault-password-file .vault_pass
+
+# Application logs
+ansible web_servers -i inventories/production/hosts.yml \
+  -m shell -a "docker logs web --tail 50" --vault-password-file .vault_pass
+```
+
+### Deploy to Production
+```bash
+# Full deployment
+./deployment/infrastructure/scripts/deploy.sh
+
+# Update .env only
+./deployment/infrastructure/scripts/update-env.sh
+
+# Quick code sync
+./deployment/infrastructure/scripts/quick-sync.sh
+```
+
+### Emergency Rollback
+```bash
+# List releases
+ansible web_servers -i inventories/production/hosts.yml \
+  -m shell -a "ls -la /home/deploy/michaelschiemer/releases/" \
+  --vault-password-file .vault_pass
+
+# Switch to previous release
+ansible web_servers -i inventories/production/hosts.yml \
+  -m shell -a "ln -sfn /home/deploy/michaelschiemer/releases/PREVIOUS_TIMESTAMP \
+               /home/deploy/michaelschiemer/current" \
+  --vault-password-file .vault_pass
+
+# Restart containers
+ansible web_servers -i inventories/production/hosts.yml \
+  -m shell -a "cd /home/deploy/michaelschiemer/current && docker compose restart" \
+  --vault-password-file .vault_pass
+```
+
+---
+
+## Support Contacts
+
+**Documentation**:
+- Deployment Analysis: `deployment/infrastructure/DEPLOYMENT_ANALYSIS.md`
+- This Summary: `deployment/infrastructure/DEPLOYMENT_FIX_SUMMARY.md`
+
+**Scripts**:
+- All scripts in: `deployment/infrastructure/scripts/`
+- Make executable: `chmod +x deployment/infrastructure/scripts/*.sh`
+
+**Configuration**:
+- Source of Truth: `deployment/applications/environments/.env.production`
+- Production File: `/home/deploy/michaelschiemer/shared/.env.production`
--- a/deployment/infrastructure/deploy.sh
+++ b/deployment/infrastructure/deploy.sh
@@ -0,0 +1,17 @@
+#!/bin/bash
+# Quick deployment script with force flag
+# Usage: ./deploy.sh
+
+cd "$(dirname "$0")"
+
+echo "🚀 Starting deployment to production..."
+echo ""
+
+ansible-playbook \
+    -i inventories/production/hosts.yml \
+    playbooks/deploy-rsync-based.yml \
+    --vault-password-file .vault_pass \
+    --extra-vars 'force_deploy=true'
+
+echo ""
+echo "✅ Deployment completed!"
--- a/deployment/infrastructure/logs.sh
+++ b/deployment/infrastructure/logs.sh
@@ -0,0 +1,14 @@
+#!/bin/bash
+# Quick script to show PHP logs from production server
+# Usage: ./logs.sh [lines]
+# Default: 50 lines
+
+LINES="${1:-50}"
+
+echo "📋 Showing last $LINES lines of PHP logs from production..."
+echo ""
+
+ssh -i ~/.ssh/production deploy@michaelschiemer.de "docker logs php --tail $LINES"
+
+echo ""
+echo "✅ Done!"
--- a/deployment/infrastructure/nginx-logs.sh
+++ b/deployment/infrastructure/nginx-logs.sh
@@ -0,0 +1,14 @@
+#!/bin/bash
+# Show Nginx error logs from production server
+# Usage: ./nginx-logs.sh [lines]
+# Default: 50 lines
+
+LINES="${1:-50}"
+
+echo "📋 Showing last $LINES lines of Nginx error logs from production..."
+echo ""
+
+ssh -i ~/.ssh/production deploy@michaelschiemer.de "docker exec web tail -n $LINES /var/log/nginx/error.log"
+
+echo ""
+echo "✅ Done!"
--- a/deployment/infrastructure/restart.sh
+++ b/deployment/infrastructure/restart.sh
@@ -0,0 +1,21 @@
+#!/bin/bash
+# Restart specific Docker container on production server
+# Usage: ./restart.sh [container_name]
+# Example: ./restart.sh php
+# Without argument: restarts all containers
+
+CONTAINER="${1:-all}"
+
+echo "🔄 Restarting container(s) on production server..."
+echo ""
+
+if [ "$CONTAINER" = "all" ]; then
+    echo "Restarting ALL containers..."
+    ssh -i ~/.ssh/production deploy@michaelschiemer.de "cd /home/deploy/michaelschiemer/current && docker compose -f docker-compose.yml -f docker-compose.production.yml restart"
+else
+    echo "Restarting container: $CONTAINER"
+    ssh -i ~/.ssh/production deploy@michaelschiemer.de "docker restart $CONTAINER"
+fi
+
+echo ""
+echo "✅ Done!"
--- a/deployment/infrastructure/scripts/deploy.sh
+++ b/deployment/infrastructure/scripts/deploy.sh
@@ -0,0 +1,32 @@
+#!/bin/bash
+# Simple Production Deployment Script
+# Usage: ./deploy.sh
+
+set -e
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+INFRA_DIR="$(dirname "$SCRIPT_DIR")"
+
+cd "$INFRA_DIR"
+
+echo "🚀 Deploying to production..."
+echo "📍 Infrastructure directory: $INFRA_DIR"
+echo ""
+
+# Check if vault password file exists
+if [[ ! -f ".vault_pass" ]]; then
+    echo "❌ Vault password file not found: .vault_pass"
+    echo "   Create this file with your Ansible Vault password"
+    exit 1
+fi
+
+# Run deployment playbook
+ansible-playbook \
+  -i inventories/production/hosts.yml \
+  playbooks/deploy-rsync-based.yml \
+  --vault-password-file .vault_pass
+
+echo ""
+echo "✅ Deployment complete!"
+echo "🔍 Check status:"
+echo "   ansible web_servers -i inventories/production/hosts.yml -m shell -a 'docker ps' --vault-password-file .vault_pass"
--- a/deployment/infrastructure/scripts/quick-sync.sh
+++ b/deployment/infrastructure/scripts/quick-sync.sh
@@ -0,0 +1,46 @@
+#!/bin/bash
+# Quick Production Code Sync
+# Usage: ./quick-sync.sh
+# Note: Does NOT update .env.production (use update-env.sh for that)
+
+set -e
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PROJECT_ROOT="$(dirname "$(dirname "$SCRIPT_DIR")")"
+INFRA_DIR="$PROJECT_ROOT/deployment/infrastructure"
+
+cd "$PROJECT_ROOT"
+
+echo "🔄 Quick sync to production (code only)..."
+echo "📍 Project root: $PROJECT_ROOT"
+echo ""
+
+# Sync code changes (excludes .env files and development artifacts)
+rsync -avz \
+  --exclude '.env' \
+  --exclude '.env.local' \
+  --exclude '.env.development' \
+  --exclude '.env.production' \
+  --exclude 'node_modules/' \
+  --exclude '.git/' \
+  --exclude 'vendor/' \
+  --exclude 'tests/' \
+  --exclude '.idea/' \
+  --exclude '.vscode/' \
+  --exclude '*.log' \
+  ./ deploy@94.16.110.151:/home/deploy/michaelschiemer/current/
+
+echo ""
+echo "🔄 Restarting PHP and web containers..."
+
+ansible web_servers \
+  -i "$INFRA_DIR/inventories/production/hosts.yml" \
+  -m shell \
+  -a "cd /home/deploy/michaelschiemer/current && docker compose restart php web" \
+  --vault-password-file "$INFRA_DIR/.vault_pass"
+
+echo ""
+echo "✅ Quick sync complete!"
+echo ""
+echo "⚠️  Note: This does NOT update .env.production"
+echo "   To update configuration, use: ./update-env.sh"
--- a/deployment/infrastructure/scripts/update-env.sh
+++ b/deployment/infrastructure/scripts/update-env.sh
@@ -0,0 +1,74 @@
+#!/bin/bash
+# Update Production .env.production File
+# Usage: ./update-env.sh
+
+set -e
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PROJECT_ROOT="$(dirname "$(dirname "$SCRIPT_DIR")")"
+INFRA_DIR="$PROJECT_ROOT/deployment/infrastructure"
+
+SOURCE_ENV="$PROJECT_ROOT/deployment/applications/environments/.env.production"
+REMOTE_PATH="/home/deploy/michaelschiemer/shared/.env.production"
+
+cd "$PROJECT_ROOT"
+
+echo "🔍 Validating .env.production..."
+
+if [[ ! -f "$SOURCE_ENV" ]]; then
+    echo "❌ Source .env.production not found at: $SOURCE_ENV"
+    exit 1
+fi
+
+# Validate required variables
+REQUIRED_VARS=("DB_DRIVER" "DB_HOST" "DB_PORT" "DB_DATABASE" "DB_USERNAME" "DB_PASSWORD")
+VALIDATION_FAILED=0
+
+for var in "${REQUIRED_VARS[@]}"; do
+    if ! grep -q "^${var}=" "$SOURCE_ENV"; then
+        echo "❌ Missing required variable: $var"
+        VALIDATION_FAILED=1
+    fi
+done
+
+# Check PostgreSQL port
+if ! grep -q "^DB_PORT=5432" "$SOURCE_ENV"; then
+    echo "⚠️  Warning: DB_PORT should be 5432 for PostgreSQL"
+    read -p "Continue anyway? (y/N): " -n 1 -r
+    echo
+    if [[ ! $REPLY =~ ^[Yy]$ ]]; then
+        exit 1
+    fi
+fi
+
+if [[ $VALIDATION_FAILED -eq 1 ]]; then
+    echo "❌ Validation failed"
+    exit 1
+fi
+
+echo "✅ Validation passed"
+echo ""
+
+echo "📤 Uploading .env.production to production server..."
+
+ansible web_servers \
+  -i "$INFRA_DIR/inventories/production/hosts.yml" \
+  -m copy \
+  -a "src=$SOURCE_ENV dest=$REMOTE_PATH mode=0644" \
+  --vault-password-file "$INFRA_DIR/.vault_pass"
+
+echo ""
+echo "🔄 Restarting containers..."
+
+ansible web_servers \
+  -i "$INFRA_DIR/inventories/production/hosts.yml" \
+  -m shell \
+  -a "cd /home/deploy/michaelschiemer/current && docker compose restart php web queue-worker" \
+  --vault-password-file "$INFRA_DIR/.vault_pass"
+
+echo ""
+echo "✅ .env.production updated and containers restarted!"
+echo ""
+echo "🔍 Verify:"
+echo "   curl -I https://michaelschiemer.de"
+echo "   (Should return HTTP/2 200 OK)"
--- a/deployment/infrastructure/status.sh
+++ b/deployment/infrastructure/status.sh
@@ -0,0 +1,11 @@
+#!/bin/bash
+# Show Docker container status on production server
+# Usage: ./status.sh
+
+echo "🐳 Docker Container Status on Production Server"
+echo ""
+
+ssh -i ~/.ssh/production deploy@michaelschiemer.de "docker ps -a --format 'table {{.Names}}\t{{.Status}}\t{{.Ports}}'"
+
+echo ""
+echo "✅ Done!"