fix: Gitea Traefik routing and connection pool optimization
Some checks failed
🚀 Build & Deploy Image / Determine Build Necessity (push) Failing after 10m14s
🚀 Build & Deploy Image / Build Runtime Base Image (push) Has been skipped
🚀 Build & Deploy Image / Build Docker Image (push) Has been skipped
🚀 Build & Deploy Image / Run Tests & Quality Checks (push) Has been skipped
🚀 Build & Deploy Image / Auto-deploy to Staging (push) Has been skipped
🚀 Build & Deploy Image / Auto-deploy to Production (push) Has been skipped
Security Vulnerability Scan / Check for Dependency Changes (push) Failing after 11m25s
Security Vulnerability Scan / Composer Security Audit (push) Has been cancelled
Some checks failed
🚀 Build & Deploy Image / Determine Build Necessity (push) Failing after 10m14s
🚀 Build & Deploy Image / Build Runtime Base Image (push) Has been skipped
🚀 Build & Deploy Image / Build Docker Image (push) Has been skipped
🚀 Build & Deploy Image / Run Tests & Quality Checks (push) Has been skipped
🚀 Build & Deploy Image / Auto-deploy to Staging (push) Has been skipped
🚀 Build & Deploy Image / Auto-deploy to Production (push) Has been skipped
Security Vulnerability Scan / Check for Dependency Changes (push) Failing after 11m25s
Security Vulnerability Scan / Composer Security Audit (push) Has been cancelled
- Remove middleware reference from Gitea Traefik labels (caused routing issues) - Optimize Gitea connection pool settings (MAX_IDLE_CONNS=30, authentication_timeout=180s) - Add explicit service reference in Traefik labels - Fix intermittent 504 timeouts by improving PostgreSQL connection handling Fixes Gitea unreachability via git.michaelschiemer.de
This commit is contained in:
286
deployment/docs/guides/migrate-to-separate-databases.md
Normal file
286
deployment/docs/guides/migrate-to-separate-databases.md
Normal file
@@ -0,0 +1,286 @@
|
||||
# Migration Guide: Separate Database Stacks
|
||||
|
||||
## Overview
|
||||
|
||||
This guide describes how to migrate from a shared PostgreSQL stack to separate database stacks for Production and Staging.
|
||||
|
||||
## Current State (Before Migration)
|
||||
|
||||
- Single PostgreSQL stack: `deployment/stacks/postgresql/`
|
||||
- Both Production and Staging connect to the same database instance
|
||||
- Database name: `michaelschiemer` (Production) and `michaelschiemer_staging` (Staging) in same instance
|
||||
|
||||
## Target State (After Migration)
|
||||
|
||||
- **Production Database**: `deployment/stacks/postgresql-production/`
|
||||
- Container: `postgres-production`
|
||||
- Network: `postgres-production-internal`
|
||||
- Database: `michaelschiemer`
|
||||
|
||||
- **Staging Database**: `deployment/stacks/postgresql-staging/`
|
||||
- Container: `postgres-staging`
|
||||
- Network: `postgres-staging-internal`
|
||||
- Database: `michaelschiemer_staging`
|
||||
|
||||
## Migration Steps
|
||||
|
||||
### Phase 1: Backup Existing Database
|
||||
|
||||
**CRITICAL**: Create backups before migration!
|
||||
|
||||
```bash
|
||||
# On production server
|
||||
cd ~/deployment/stacks/postgresql
|
||||
|
||||
# Create backup of production database
|
||||
docker exec postgres-backup /scripts/backup.sh
|
||||
|
||||
# Create backup of staging database (if exists in same instance)
|
||||
docker exec postgres psql -U postgres -d michaelschiemer_staging -c "SELECT 1;" && \
|
||||
docker exec postgres-backup /scripts/backup.sh || echo "Staging database not found"
|
||||
|
||||
# List backups
|
||||
ls -lh backups/
|
||||
```
|
||||
|
||||
### Phase 2: Deploy New Database Stacks
|
||||
|
||||
#### 2.1 Deploy PostgreSQL Production Stack
|
||||
|
||||
```bash
|
||||
# On production server
|
||||
cd ~/deployment/stacks/postgresql-production
|
||||
|
||||
# Create .env file
|
||||
cat > .env <<EOF
|
||||
POSTGRES_DB=michaelschiemer
|
||||
POSTGRES_USER=postgres
|
||||
POSTGRES_PASSWORD=<same-as-production-password>
|
||||
BACKUP_RETENTION_DAYS=7
|
||||
BACKUP_SCHEDULE=0 2 * * *
|
||||
EOF
|
||||
|
||||
# Start PostgreSQL Production
|
||||
docker compose up -d
|
||||
|
||||
# Verify
|
||||
docker compose ps
|
||||
docker exec postgres-production pg_isready -U postgres -d michaelschiemer
|
||||
```
|
||||
|
||||
#### 2.2 Deploy PostgreSQL Staging Stack
|
||||
|
||||
```bash
|
||||
# On production server
|
||||
cd ~/deployment/stacks/postgresql-staging
|
||||
|
||||
# Create .env file
|
||||
cat > .env <<EOF
|
||||
POSTGRES_DB=michaelschiemer_staging
|
||||
POSTGRES_USER=postgres
|
||||
POSTGRES_PASSWORD=<staging-password-can-differ>
|
||||
BACKUP_RETENTION_DAYS=3
|
||||
BACKUP_SCHEDULE=0 3 * * *
|
||||
EOF
|
||||
|
||||
# Start PostgreSQL Staging
|
||||
docker compose up -d
|
||||
|
||||
# Verify
|
||||
docker compose ps
|
||||
docker exec postgres-staging pg_isready -U postgres -d michaelschiemer_staging
|
||||
```
|
||||
|
||||
### Phase 3: Restore Data to New Stacks
|
||||
|
||||
#### 3.1 Restore Production Database
|
||||
|
||||
```bash
|
||||
# Find latest production backup
|
||||
LATEST_PROD_BACKUP=$(ls -t ~/deployment/stacks/postgresql/backups/postgres_michaelschiemer_*.sql.gz | head -1)
|
||||
|
||||
# Copy backup to new stack
|
||||
cp "$LATEST_PROD_BACKUP" ~/deployment/stacks/postgresql-production/backups/
|
||||
|
||||
# Restore to new production database
|
||||
cd ~/deployment/stacks/postgresql-production
|
||||
docker exec -it postgres-production-backup /scripts/restore.sh "/backups/$(basename $LATEST_PROD_BACKUP)"
|
||||
```
|
||||
|
||||
#### 3.2 Restore Staging Database (if exists)
|
||||
|
||||
```bash
|
||||
# Find latest staging backup (if exists)
|
||||
LATEST_STAGING_BACKUP=$(ls -t ~/deployment/stacks/postgresql/backups/postgres_michaelschiemer_staging_*.sql.gz 2>/dev/null | head -1)
|
||||
|
||||
if [ -n "$LATEST_STAGING_BACKUP" ]; then
|
||||
# Copy backup to new stack
|
||||
cp "$LATEST_STAGING_BACKUP" ~/deployment/stacks/postgresql-staging/backups/
|
||||
|
||||
# Restore to new staging database
|
||||
cd ~/deployment/stacks/postgresql-staging
|
||||
docker exec -it postgres-staging-backup /scripts/restore.sh "/backups/$(basename $LATEST_STAGING_BACKUP)"
|
||||
else
|
||||
echo "No staging backup found - creating empty staging database"
|
||||
fi
|
||||
```
|
||||
|
||||
### Phase 4: Update Application Stacks
|
||||
|
||||
#### 4.1 Update Production Application Stack
|
||||
|
||||
The Production Application Stack should already be configured to use `postgres-production` via:
|
||||
- Environment variable: `DB_HOST=postgres-production`
|
||||
- Network: Connected to `postgres-production-internal`
|
||||
|
||||
**Verify connection**:
|
||||
```bash
|
||||
cd ~/deployment/stacks/production
|
||||
docker compose -f docker-compose.base.yml -f docker-compose.production.yml exec app \
|
||||
php -r "echo 'DB Connection: '; \$pdo = new PDO('pgsql:host=postgres-production;dbname=michaelschiemer', 'postgres', getenv('DB_PASSWORD')); echo 'OK\n';"
|
||||
```
|
||||
|
||||
#### 4.2 Update Staging Application Stack
|
||||
|
||||
The Staging Application Stack should already be configured to use `postgres-staging` via:
|
||||
- Environment variable: `DB_HOST=postgres-staging`
|
||||
- Network: Connected to `postgres-staging-internal`
|
||||
|
||||
**Verify connection**:
|
||||
```bash
|
||||
cd ~/deployment/stacks/staging
|
||||
docker compose -f docker-compose.base.yml -f docker-compose.staging.yml exec staging-app \
|
||||
php -r "echo 'DB Connection: '; \$pdo = new PDO('pgsql:host=postgres-staging;dbname=michaelschiemer_staging', 'postgres', getenv('DB_PASSWORD')); echo 'OK\n';"
|
||||
```
|
||||
|
||||
### Phase 5: Verify Migration
|
||||
|
||||
#### 5.1 Production Verification
|
||||
|
||||
```bash
|
||||
# Check production database
|
||||
docker exec postgres-production psql -U postgres -d michaelschiemer -c "\dt"
|
||||
|
||||
# Check production application
|
||||
curl -k https://michaelschiemer.de/health
|
||||
|
||||
# Check production logs
|
||||
cd ~/deployment/stacks/production
|
||||
docker compose logs app | tail -20
|
||||
```
|
||||
|
||||
#### 5.2 Staging Verification
|
||||
|
||||
```bash
|
||||
# Check staging database
|
||||
docker exec postgres-staging psql -U postgres -d michaelschiemer_staging -c "\dt"
|
||||
|
||||
# Check staging application
|
||||
curl -k https://staging.michaelschiemer.de/health
|
||||
|
||||
# Check staging logs
|
||||
cd ~/deployment/stacks/staging
|
||||
docker compose logs staging-app | tail -20
|
||||
```
|
||||
|
||||
### Phase 6: Cleanup (Optional)
|
||||
|
||||
**WARNING**: Only remove old stack after successful migration and verification!
|
||||
|
||||
```bash
|
||||
# Stop old PostgreSQL stack
|
||||
cd ~/deployment/stacks/postgresql
|
||||
docker compose down
|
||||
|
||||
# Keep backups for safety - do NOT delete yet
|
||||
# Consider keeping for at least 30 days
|
||||
|
||||
# Remove old stack (only after 30+ days of successful operation)
|
||||
# rm -rf ~/deployment/stacks/postgresql
|
||||
```
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
If migration fails:
|
||||
|
||||
### Rollback to Shared Database
|
||||
|
||||
```bash
|
||||
# 1. Stop new database stacks
|
||||
cd ~/deployment/stacks/postgresql-production
|
||||
docker compose down
|
||||
|
||||
cd ~/deployment/stacks/postgresql-staging
|
||||
docker compose down
|
||||
|
||||
# 2. Restart old shared stack
|
||||
cd ~/deployment/stacks/postgresql
|
||||
docker compose up -d
|
||||
|
||||
# 3. Update application stacks to use old database
|
||||
# Revert DB_HOST changes in docker-compose files
|
||||
# Restart application stacks
|
||||
```
|
||||
|
||||
## Network Configuration
|
||||
|
||||
### Production Application Stack
|
||||
|
||||
Must be connected to:
|
||||
- `traefik-public` (for external access)
|
||||
- `postgres-production-internal` (for database access)
|
||||
- `app-internal` (for Redis, if shared)
|
||||
|
||||
### Staging Application Stack
|
||||
|
||||
Must be connected to:
|
||||
- `traefik-public` (for external access)
|
||||
- `postgres-staging-internal` (for database access)
|
||||
- `staging-internal` (for staging services)
|
||||
|
||||
## Backup Strategy
|
||||
|
||||
### Production Backups
|
||||
|
||||
- **Location**: `~/deployment/stacks/postgresql-production/backups/`
|
||||
- **Retention**: 7 days
|
||||
- **Schedule**: Daily at 2:00 AM
|
||||
|
||||
### Staging Backups
|
||||
|
||||
- **Location**: `~/deployment/stacks/postgresql-staging/backups/`
|
||||
- **Retention**: 3 days
|
||||
- **Schedule**: Daily at 3:00 AM
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Database Connection Refused
|
||||
|
||||
```bash
|
||||
# Check if database container is running
|
||||
docker ps | grep postgres-production
|
||||
docker ps | grep postgres-staging
|
||||
|
||||
# Check network connectivity
|
||||
docker network inspect postgres-production-internal
|
||||
docker network inspect postgres-staging-internal
|
||||
|
||||
# Test connection from application container
|
||||
docker exec <app-container> nc -zv postgres-production 5432
|
||||
docker exec <app-container> nc -zv postgres-staging 5432
|
||||
```
|
||||
|
||||
### Migration Data Loss
|
||||
|
||||
If data is missing after migration:
|
||||
|
||||
1. **Stop application** to prevent further writes
|
||||
2. **Restore from backup** (see Phase 3)
|
||||
3. **Verify data** before restarting application
|
||||
|
||||
## Additional Resources
|
||||
|
||||
- **PostgreSQL Production Stack**: `deployment/stacks/postgresql-production/README.md`
|
||||
- **PostgreSQL Staging Stack**: `deployment/stacks/postgresql-staging/README.md`
|
||||
- **Backup Playbook**: `deployment/ansible/playbooks/backup.yml`
|
||||
|
||||
Reference in New Issue
Block a user