michaelschiemer/docs/deployment/database-migration-strategy.md

# Production Database Migration Strategy

Sichere und zuverlässige Database Migration Strategies für Production Deployment des Custom PHP Frameworks.

## Migration System Overview

Das Framework nutzt ein **Safe Rollback Architecture** System:

```
Migration Interface (Forward-Only)
    ↓
    └─→ SafelyReversible Interface (Optional - nur bei safe rollback)
            ↓
            └─→ MigrationRunner
                    ↓
                    ├─→ Apply (up)
                    └─→ Rollback (down) - nur wenn SafelyReversible
```

**Core Principle**: Migrations sind **forward-only by default**. Rollback nur wenn SICHER (no data loss).

## Safe vs Unsafe Migrations

### ✅ Safe for Rollback (implement SafelyReversible)

Diese Migrations können sicher rückgängig gemacht werden:

- **Creating new tables** (can be dropped without data loss)
- **Adding nullable columns** (can be removed)
- **Creating/dropping indexes** (no data affected)
- **Renaming columns** (data preserved)
- **Adding/removing foreign keys** (constraints only)
- **Adding CHECK constraints** (can be removed)
- **Creating empty tables** (no data to lose)

### ❌ Unsafe for Rollback (only Migration interface)

Diese Migrations können NICHT sicher zurückgerollt werden:

- **Dropping columns with data** (data is LOST)
- **Transforming data formats** (original format lost)
- **Changing column types** (data loss risk)
- **Merging/splitting tables** (data restructured)
- **Deleting data** (information cannot be restored)
- **Complex data migrations** (multiple steps, state changes)

## Migration Implementation

### 1. Safe Migration Example

```php
<?php

declare(strict_types=1);

namespace App\Domain\User\Migrations;

use App\Framework\Database\Migration\{Migration, SafelyReversible};
use App\Framework\Database\ConnectionInterface;
use App\Framework\Database\Schema\{Blueprint, Schema};
use App\Framework\Database\ValueObjects\{TableName, ColumnName, IndexName};
use App\Framework\Database\Migration\ValueObjects\MigrationVersion;

/**
 * Create user_profiles table
 *
 * SAFE: New table can be dropped without data loss
 */
final readonly class CreateUserProfilesTable implements Migration, SafelyReversible
{
    public function up(ConnectionInterface $connection): void
    {
        $schema = new Schema($connection);

        $schema->create(TableName::fromString('user_profiles'), function (Blueprint $table) {
            // Primary Key
            $table->string(ColumnName::fromString('ulid'), 26)->primary();

            // Foreign Key to users
            $table->string(ColumnName::fromString('user_id'), 26);

            // Profile Data (nullable - safe to drop)
            $table->string(ColumnName::fromString('bio'))->nullable();
            $table->string(ColumnName::fromString('avatar_url'))->nullable();
            $table->string(ColumnName::fromString('website'))->nullable();

            // Timestamps
            $table->timestamps();

            // Foreign Key Constraint
            $table->foreign(ColumnName::fromString('user_id'))
                ->references(ColumnName::fromString('ulid'))
                ->on(TableName::fromString('users'))
                ->onDelete(ForeignKeyAction::CASCADE);

            // Index
            $table->index(
                ColumnName::fromString('user_id'),
                IndexName::fromString('idx_user_profiles_user_id')
            );
        });

        $schema->execute();
    }

    /**
     * Rollback is SAFE because:
     * - Table is new (no existing data to lose)
     * - If table has data, it's from testing/staging only
     * - Production: Use fix-forward migration instead
     */
    public function down(ConnectionInterface $connection): void
    {
        $schema = new Schema($connection);
        $schema->dropIfExists(TableName::fromString('user_profiles'));
        $schema->execute();
    }

    public function getVersion(): MigrationVersion
    {
        return MigrationVersion::fromString('2024_01_15_143000');
    }

    public function getDescription(): string
    {
        return 'Create user_profiles table';
    }
}
```

### 2. Unsafe Migration Example

```php
<?php

declare(strict_types=1);

namespace App\Domain\User\Migrations;

use App\Framework\Database\Migration\Migration;
use App\Framework\Database\ConnectionInterface;
use App\Framework\Database\Schema\{Blueprint, Schema};
use App\Framework\Database\Migration\ValueObjects\MigrationVersion;

/**
 * Remove deprecated legacy_id column
 *
 * UNSAFE: Dropping column with data - NOT reversible
 */
final readonly class RemoveLegacyIdColumn implements Migration
{
    public function up(ConnectionInterface $connection): void
    {
        $schema = new Schema($connection);

        $schema->table('users', function (Blueprint $table) {
            // Data is LOST after this operation!
            $table->dropColumn('legacy_id');
        });

        $schema->execute();
    }

    // NO down() method - data cannot be recovered
    // Use fix-forward migration if needed

    public function getVersion(): MigrationVersion
    {
        return MigrationVersion::fromString('2024_01_15_150000');
    }

    public function getDescription(): string
    {
        return 'Remove deprecated legacy_id column from users table';
    }
}
```

### 3. Fix-Forward Migration Example

Statt unsicheren Rollback: Neue Forward-Migration erstellen.

```php
<?php

declare(strict_types=1);

/**
 * Restore accidentally removed deprecated_field
 *
 * Fix-Forward Strategy: Create new migration to undo changes
 */
final readonly class RestoreDeprecatedField implements Migration, SafelyReversible
{
    public function up(ConnectionInterface $connection): void
    {
        $schema = new Schema($connection);

        $schema->table('users', function (Blueprint $table) {
            // Restore column (data cannot be recovered, but column structure can)
            $table->string('deprecated_field')->nullable();
        });

        $schema->execute();
    }

    /**
     * SAFE: Column is empty after restoration, can be dropped
     */
    public function down(ConnectionInterface $connection): void
    {
        $schema = new Schema($connection);

        $schema->table('users', function (Blueprint $table) {
            $table->dropColumn('deprecated_field');
        });

        $schema->execute();
    }

    public function getVersion(): MigrationVersion
    {
        return MigrationVersion::fromString('2024_01_15_160000');
    }

    public function getDescription(): string
    {
        return 'Restore deprecated_field column to users table';
    }
}
```

## Migration Commands

### Console Commands

```bash
# Create new migration
php console.php make:migration CreateUsersTable [Domain]

# Run all pending migrations
php console.php db:migrate

# Check migration status
php console.php db:status

# Rollback last migration (only if SafelyReversible)
php console.php db:rollback [steps]

# Test migration (dry-run)
php console.php db:migrate --dry-run

# Force migration (skip confirmation)
php console.php db:migrate --force
```

### Production Migration Workflow

```bash
# 1. Backup database before migration
./scripts/backup-database.sh

# 2. Check migration status
docker exec php php console.php db:status

# 3. Test migration (dry-run)
docker exec php php console.php db:migrate --dry-run

# 4. Apply migrations
docker exec php php console.php db:migrate

# 5. Verify migration success
docker exec php php console.php db:status

# 6. Run application health check
curl -f https://your-domain.com/health || exit 1
```

## Migration Rollback Safety Check

Der `MigrationRunner` prüft automatisch, ob eine Migration sicher rollbar ist:

```bash
$ php console.php db:rollback 1

🔄 Rolling back migrations...
⚠️  Safety Check: Only migrations implementing SafelyReversible will be rolled back.

❌ Rollback failed: Migration 2024_01_15_150000 does not support safe rollback

ℹ️  This migration cannot be safely rolled back.
   Reason: Data loss would occur during rollback.

💡 Recommendation:
   Create a new forward migration to undo the changes instead:
   php console.php make:migration FixYourChanges

📖 See docs/deployment/database-migration-strategy.md for guidelines.
```

## Production Migration Best Practices

### 1. Pre-Migration Checklist

- [ ] **Test in Staging**: Run migration in staging environment first
- [ ] **Backup Database**: Create full database backup before migration
- [ ] **Review SQL**: Inspect generated SQL for correctness
- [ ] **Check Dependencies**: Verify migration dependencies are applied
- [ ] **Monitor Resources**: Ensure sufficient disk space and memory
- [ ] **Schedule Maintenance**: Plan migration during low-traffic window
- [ ] **Prepare Rollback**: Have rollback plan ready (fix-forward migration)
- [ ] **Team Notification**: Inform team of deployment window
- [ ] **Health Checks**: Verify health check endpoints before migration

### 2. Migration Execution Strategy

**Option A: Zero-Downtime Migration** (Recommended)

```bash
# 1. Deploy new code (migration not yet applied)
./scripts/deploy-production.sh --skip-migrations

# 2. Verify application works with old schema
curl -f https://your-domain.com/health

# 3. Apply backward-compatible migration
docker exec php php console.php db:migrate

# 4. Verify application works with new schema
curl -f https://your-domain.com/health

# 5. Complete deployment
```

**Option B: Maintenance Window Migration**

```bash
# 1. Enable maintenance mode
./scripts/maintenance-mode.sh enable

# 2. Backup database
./scripts/backup-database.sh

# 3. Apply migrations
docker exec php php console.php db:migrate

# 4. Verify health
curl -f https://your-domain.com/health

# 5. Disable maintenance mode
./scripts/maintenance-mode.sh disable
```

### 3. Backward-Compatible Migrations

**Guidelines for Zero-Downtime**:

✅ **Safe Operations**:
- Adding nullable columns
- Creating new tables
- Adding indexes (CONCURRENTLY in PostgreSQL)
- Renaming columns (in two-step process)

❌ **Unsafe Operations**:
- Dropping columns (old code will fail)
- Renaming columns (one-step process)
- Changing column types (old code may fail)
- Adding NOT NULL columns (without default)

**Two-Step Column Rename Example**:

```php
// Step 1: Add new column (nullable)
final readonly class AddNewEmailColumn implements Migration, SafelyReversible
{
    public function up(ConnectionInterface $connection): void
    {
        $schema = new Schema($connection);
        $schema->table('users', function (Blueprint $table) {
            $table->string('email_address')->nullable();
        });
        $schema->execute();

        // Copy data from old column
        $connection->execute("UPDATE users SET email_address = email WHERE email_address IS NULL");
    }
}

// Deploy new code (uses email_address, falls back to email)

// Step 2: Drop old column (after code deployed)
final readonly class DropOldEmailColumn implements Migration
{
    public function up(ConnectionInterface $connection): void
    {
        $schema = new Schema($connection);
        $schema->table('users', function (Blueprint $table) {
            $table->dropColumn('email');
        });
        $schema->execute();
    }
}
```

### 4. Performance Considerations

**Long-Running Migrations**:
```php
// ❌ Blocking operation (can timeout)
$table->addIndex(['email']); // Locks table during index creation

// ✅ Non-blocking operation (PostgreSQL)
$connection->execute("CREATE INDEX CONCURRENTLY idx_users_email ON users (email)");
```

**Large Table Migrations**:
```php
// ❌ Single UPDATE for millions of rows
$connection->execute("UPDATE users SET status = 'active' WHERE status IS NULL");

// ✅ Batch processing
$batchSize = 10000;
$offset = 0;

do {
    $affected = $connection->execute(
        "UPDATE users SET status = 'active'
         WHERE status IS NULL
         AND ulid IN (
             SELECT ulid FROM users WHERE status IS NULL LIMIT ? OFFSET ?
         )",
        [$batchSize, $offset]
    );
    $offset += $batchSize;

    // Small delay to reduce load
    usleep(100000); // 100ms
} while ($affected > 0);
```

### 5. Data Migration Patterns

**Option 1: In-Migration Data Transform**:
```php
final readonly class MigrateUserRoles implements Migration
{
    public function up(ConnectionInterface $connection): void
    {
        // Schema change
        $schema = new Schema($connection);
        $schema->table('users', function (Blueprint $table) {
            $table->string('role')->default('user');
        });
        $schema->execute();

        // Data migration
        $connection->execute("
            UPDATE users
            SET role = CASE
                WHEN is_admin THEN 'admin'
                WHEN is_moderator THEN 'moderator'
                ELSE 'user'
            END
        ");

        // Drop old columns
        $schema->table('users', function (Blueprint $table) {
            $table->dropColumn('is_admin', 'is_moderator');
        });
        $schema->execute();
    }
}
```

**Option 2: Separate Data Migration Script**:
```php
// Migration: Schema only
final readonly class AddRoleColumn implements Migration, SafelyReversible
{
    public function up(ConnectionInterface $connection): void
    {
        $schema = new Schema($connection);
        $schema->table('users', function (Blueprint $table) {
            $table->string('role')->nullable();
        });
        $schema->execute();
    }
}

// Separate script: Data migration
// scripts/data-migrations/migrate-user-roles.php
$repository = $container->get(UserRepository::class);
$users = $repository->findAll();

foreach ($users as $user) {
    $role = $user->isAdmin() ? 'admin'
          : ($user->isModerator() ? 'moderator' : 'user');
    $user->setRole($role);
    $repository->save($user);
}
```

## Migration Testing

### 1. Local Testing

```bash
# Reset database and re-run all migrations
php console.php db:fresh

# Run specific migration
php console.php db:migrate --step=1

# Test rollback (if SafelyReversible)
php console.php db:rollback --step=1

# Re-apply migration
php console.php db:migrate --step=1
```

### 2. Staging Environment Testing

```bash
# Copy production data to staging (anonymized)
./scripts/anonymize-production-data.sh

# Apply migrations in staging
ssh staging "cd /app && php console.php db:migrate"

# Run integration tests
ssh staging "cd /app && ./vendor/bin/pest --testsuite=Integration"

# Verify application health
curl -f https://staging.your-domain.com/health
```

### 3. Migration Test Checklist

- [ ] **Migration applies successfully** (no errors)
- [ ] **Schema matches expectations** (inspect database)
- [ ] **Data integrity preserved** (no data loss)
- [ ] **Application health checks pass** (all endpoints work)
- [ ] **Performance acceptable** (migration completes in reasonable time)
- [ ] **Rollback works** (if SafelyReversible)
- [ ] **Idempotency verified** (can run migration multiple times)
- [ ] **Foreign key constraints intact** (referential integrity)
- [ ] **Indexes created properly** (query performance maintained)

## Monitoring & Validation

### Post-Migration Health Checks

```bash
# 1. Database connection test
docker exec php php console.php db:test-connection

# 2. Table integrity check
docker exec php php console.php db:check-integrity

# 3. Application health check
curl -f https://your-domain.com/health

# 4. Check for errors in logs
docker-compose logs php | grep -i error

# 5. Verify data consistency
docker exec php php console.php db:verify-data
```

### Migration Metrics

**Track These Metrics**:
- Migration execution time
- Database size before/after
- Number of affected rows
- Query performance (slow query log)
- Error rate (application logs)
- Health check failures

**Example Monitoring**:
```php
// Log migration metrics
$startTime = microtime(true);
$sizeBeforeMB = $this->getDatabaseSize();

$migration->up($connection);

$executionTimeMs = (microtime(true) - $startTime) * 1000;
$sizeAfterMB = $this->getDatabaseSize();

$this->logger->info('Migration completed', [
    'migration' => $migration->getVersion()->value,
    'execution_time_ms' => $executionTimeMs,
    'size_before_mb' => $sizeBeforeMB,
    'size_after_mb' => $sizeAfterMB,
    'size_delta_mb' => $sizeAfterMB - $sizeBeforeMB,
]);
```

## Rollback Strategy

### When to Rollback vs Fix-Forward

**Rollback (if SafelyReversible)**:
- Migration applied in last 5 minutes
- No production traffic affected yet
- Schema change only (no data migration)
- Quick fix available

**Fix-Forward (Recommended for Production)**:
- Migration applied > 5 minutes ago
- Production traffic affected
- Data migrations applied
- Complex schema changes
- Uncertain about data loss

### Rollback Execution

```bash
# Check if migration supports rollback
docker exec php php console.php db:status

# Rollback last migration (if safe)
docker exec php php console.php db:rollback --step=1

# Verify rollback success
docker exec php php console.php db:status

# Check application health
curl -f https://your-domain.com/health
```

### Fix-Forward Execution

```bash
# 1. Create fix-forward migration
docker exec php php console.php make:migration FixBrokenMigration

# 2. Implement fix in new migration
# Edit: src/.../Migrations/2024_XX_XX_XXXXXX_FixBrokenMigration.php

# 3. Test in staging
ssh staging "cd /app && php console.php db:migrate"

# 4. Apply in production
docker exec php php console.php db:migrate

# 5. Verify fix
curl -f https://your-domain.com/health
```

## Disaster Recovery

### Database Backup Before Migration

```bash
#!/bin/bash
# scripts/backup-database.sh

BACKUP_DIR="/backups/database"
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
DB_NAME="michaelschiemer_prod"

# Create backup directory
mkdir -p "$BACKUP_DIR"

# Backup database
docker exec db pg_dump -U postgres "$DB_NAME" | \
    gzip > "$BACKUP_DIR/backup_${TIMESTAMP}.sql.gz"

# Verify backup
if [ $? -eq 0 ]; then
    echo "✅ Backup successful: $BACKUP_DIR/backup_${TIMESTAMP}.sql.gz"
else
    echo "❌ Backup failed"
    exit 1
fi

# Keep only last 30 days of backups
find "$BACKUP_DIR" -name "backup_*.sql.gz" -mtime +30 -delete
```

### Database Restore

```bash
#!/bin/bash
# scripts/restore-database.sh

BACKUP_FILE="$1"

if [ -z "$BACKUP_FILE" ]; then
    echo "Usage: $0 /path/to/backup.sql.gz"
    exit 1
fi

# Confirm restore
echo "⚠️  This will REPLACE the current database with backup: $BACKUP_FILE"
read -p "Are you sure? (yes/no): " confirm

if [ "$confirm" != "yes" ]; then
    echo "Restore cancelled"
    exit 0
fi

# Drop and recreate database
docker exec db psql -U postgres -c "DROP DATABASE IF EXISTS michaelschiemer_prod"
docker exec db psql -U postgres -c "CREATE DATABASE michaelschiemer_prod"

# Restore backup
gunzip -c "$BACKUP_FILE" | docker exec -i db psql -U postgres michaelschiemer_prod

echo "✅ Database restored from backup"
```

## Troubleshooting

### Problem: Migration Times Out

**Cause**: Long-running operation on large table

**Solution**:
```php
// Increase timeout for migration
$connection->execute("SET statement_timeout = '300s'");

// Or split into smaller batches
$batchSize = 10000;
// ... batch processing code
```

### Problem: Migration Fails Midway

**Cause**: Error during schema change or data migration

**Solution**:
```bash
# Check migration status
docker exec php php console.php db:status

# If migration is partially applied:
# 1. Manual cleanup (if needed)
docker exec db psql -U postgres michaelschiemer_prod

# 2. Mark migration as failed
docker exec php php console.php db:reset [version]

# 3. Fix migration code

# 4. Re-run migration
docker exec php php console.php db:migrate
```

### Problem: Foreign Key Constraint Violation

**Cause**: Data inconsistency or missing referenced rows

**Solution**:
```sql
-- Find orphaned rows
SELECT * FROM child_table
WHERE parent_id NOT IN (SELECT id FROM parent_table);

-- Fix data before migration
DELETE FROM child_table
WHERE parent_id NOT IN (SELECT id FROM parent_table);

-- Then run migration
```

## See Also

- **Production Prerequisites**: `docs/deployment/production-prerequisites.md`
- **Database Patterns**: `docs/claude/database-patterns.md`
- **Deployment Guide**: `docs/deployment/deployment-guide.md` (TODO)
- **Rollback Guide**: `docs/deployment/rollback-guide.md` (TODO)