fix: Gitea Traefik routing and connection pool optimization
Some checks failed
🚀 Build & Deploy Image / Determine Build Necessity (push) Failing after 10m14s
🚀 Build & Deploy Image / Build Runtime Base Image (push) Has been skipped
🚀 Build & Deploy Image / Build Docker Image (push) Has been skipped
🚀 Build & Deploy Image / Run Tests & Quality Checks (push) Has been skipped
🚀 Build & Deploy Image / Auto-deploy to Staging (push) Has been skipped
🚀 Build & Deploy Image / Auto-deploy to Production (push) Has been skipped
Security Vulnerability Scan / Check for Dependency Changes (push) Failing after 11m25s
Security Vulnerability Scan / Composer Security Audit (push) Has been cancelled

- Remove middleware reference from Gitea Traefik labels (caused routing issues)
- Optimize Gitea connection pool settings (MAX_IDLE_CONNS=30, authentication_timeout=180s)
- Add explicit service reference in Traefik labels
- Fix intermittent 504 timeouts by improving PostgreSQL connection handling

Fixes Gitea unreachability via git.michaelschiemer.de
This commit is contained in:
2025-11-09 14:46:15 +01:00
parent 85c369e846
commit 36ef2a1e2c
1366 changed files with 104925 additions and 28719 deletions

View File

@@ -0,0 +1,286 @@
# Migration Guide: Separate Database Stacks
## Overview
This guide describes how to migrate from a shared PostgreSQL stack to separate database stacks for Production and Staging.
## Current State (Before Migration)
- Single PostgreSQL stack: `deployment/stacks/postgresql/`
- Both Production and Staging connect to the same database instance
- Database name: `michaelschiemer` (Production) and `michaelschiemer_staging` (Staging) in same instance
## Target State (After Migration)
- **Production Database**: `deployment/stacks/postgresql-production/`
- Container: `postgres-production`
- Network: `postgres-production-internal`
- Database: `michaelschiemer`
- **Staging Database**: `deployment/stacks/postgresql-staging/`
- Container: `postgres-staging`
- Network: `postgres-staging-internal`
- Database: `michaelschiemer_staging`
## Migration Steps
### Phase 1: Backup Existing Database
**CRITICAL**: Create backups before migration!
```bash
# On production server
cd ~/deployment/stacks/postgresql
# Create backup of production database
docker exec postgres-backup /scripts/backup.sh
# Create backup of staging database (if exists in same instance)
docker exec postgres psql -U postgres -d michaelschiemer_staging -c "SELECT 1;" && \
docker exec postgres-backup /scripts/backup.sh || echo "Staging database not found"
# List backups
ls -lh backups/
```
### Phase 2: Deploy New Database Stacks
#### 2.1 Deploy PostgreSQL Production Stack
```bash
# On production server
cd ~/deployment/stacks/postgresql-production
# Create .env file
cat > .env <<EOF
POSTGRES_DB=michaelschiemer
POSTGRES_USER=postgres
POSTGRES_PASSWORD=<same-as-production-password>
BACKUP_RETENTION_DAYS=7
BACKUP_SCHEDULE=0 2 * * *
EOF
# Start PostgreSQL Production
docker compose up -d
# Verify
docker compose ps
docker exec postgres-production pg_isready -U postgres -d michaelschiemer
```
#### 2.2 Deploy PostgreSQL Staging Stack
```bash
# On production server
cd ~/deployment/stacks/postgresql-staging
# Create .env file
cat > .env <<EOF
POSTGRES_DB=michaelschiemer_staging
POSTGRES_USER=postgres
POSTGRES_PASSWORD=<staging-password-can-differ>
BACKUP_RETENTION_DAYS=3
BACKUP_SCHEDULE=0 3 * * *
EOF
# Start PostgreSQL Staging
docker compose up -d
# Verify
docker compose ps
docker exec postgres-staging pg_isready -U postgres -d michaelschiemer_staging
```
### Phase 3: Restore Data to New Stacks
#### 3.1 Restore Production Database
```bash
# Find latest production backup
LATEST_PROD_BACKUP=$(ls -t ~/deployment/stacks/postgresql/backups/postgres_michaelschiemer_*.sql.gz | head -1)
# Copy backup to new stack
cp "$LATEST_PROD_BACKUP" ~/deployment/stacks/postgresql-production/backups/
# Restore to new production database
cd ~/deployment/stacks/postgresql-production
docker exec -it postgres-production-backup /scripts/restore.sh "/backups/$(basename $LATEST_PROD_BACKUP)"
```
#### 3.2 Restore Staging Database (if exists)
```bash
# Find latest staging backup (if exists)
LATEST_STAGING_BACKUP=$(ls -t ~/deployment/stacks/postgresql/backups/postgres_michaelschiemer_staging_*.sql.gz 2>/dev/null | head -1)
if [ -n "$LATEST_STAGING_BACKUP" ]; then
# Copy backup to new stack
cp "$LATEST_STAGING_BACKUP" ~/deployment/stacks/postgresql-staging/backups/
# Restore to new staging database
cd ~/deployment/stacks/postgresql-staging
docker exec -it postgres-staging-backup /scripts/restore.sh "/backups/$(basename $LATEST_STAGING_BACKUP)"
else
echo "No staging backup found - creating empty staging database"
fi
```
### Phase 4: Update Application Stacks
#### 4.1 Update Production Application Stack
The Production Application Stack should already be configured to use `postgres-production` via:
- Environment variable: `DB_HOST=postgres-production`
- Network: Connected to `postgres-production-internal`
**Verify connection**:
```bash
cd ~/deployment/stacks/production
docker compose -f docker-compose.base.yml -f docker-compose.production.yml exec app \
php -r "echo 'DB Connection: '; \$pdo = new PDO('pgsql:host=postgres-production;dbname=michaelschiemer', 'postgres', getenv('DB_PASSWORD')); echo 'OK\n';"
```
#### 4.2 Update Staging Application Stack
The Staging Application Stack should already be configured to use `postgres-staging` via:
- Environment variable: `DB_HOST=postgres-staging`
- Network: Connected to `postgres-staging-internal`
**Verify connection**:
```bash
cd ~/deployment/stacks/staging
docker compose -f docker-compose.base.yml -f docker-compose.staging.yml exec staging-app \
php -r "echo 'DB Connection: '; \$pdo = new PDO('pgsql:host=postgres-staging;dbname=michaelschiemer_staging', 'postgres', getenv('DB_PASSWORD')); echo 'OK\n';"
```
### Phase 5: Verify Migration
#### 5.1 Production Verification
```bash
# Check production database
docker exec postgres-production psql -U postgres -d michaelschiemer -c "\dt"
# Check production application
curl -k https://michaelschiemer.de/health
# Check production logs
cd ~/deployment/stacks/production
docker compose logs app | tail -20
```
#### 5.2 Staging Verification
```bash
# Check staging database
docker exec postgres-staging psql -U postgres -d michaelschiemer_staging -c "\dt"
# Check staging application
curl -k https://staging.michaelschiemer.de/health
# Check staging logs
cd ~/deployment/stacks/staging
docker compose logs staging-app | tail -20
```
### Phase 6: Cleanup (Optional)
**WARNING**: Only remove old stack after successful migration and verification!
```bash
# Stop old PostgreSQL stack
cd ~/deployment/stacks/postgresql
docker compose down
# Keep backups for safety - do NOT delete yet
# Consider keeping for at least 30 days
# Remove old stack (only after 30+ days of successful operation)
# rm -rf ~/deployment/stacks/postgresql
```
## Rollback Plan
If migration fails:
### Rollback to Shared Database
```bash
# 1. Stop new database stacks
cd ~/deployment/stacks/postgresql-production
docker compose down
cd ~/deployment/stacks/postgresql-staging
docker compose down
# 2. Restart old shared stack
cd ~/deployment/stacks/postgresql
docker compose up -d
# 3. Update application stacks to use old database
# Revert DB_HOST changes in docker-compose files
# Restart application stacks
```
## Network Configuration
### Production Application Stack
Must be connected to:
- `traefik-public` (for external access)
- `postgres-production-internal` (for database access)
- `app-internal` (for Redis, if shared)
### Staging Application Stack
Must be connected to:
- `traefik-public` (for external access)
- `postgres-staging-internal` (for database access)
- `staging-internal` (for staging services)
## Backup Strategy
### Production Backups
- **Location**: `~/deployment/stacks/postgresql-production/backups/`
- **Retention**: 7 days
- **Schedule**: Daily at 2:00 AM
### Staging Backups
- **Location**: `~/deployment/stacks/postgresql-staging/backups/`
- **Retention**: 3 days
- **Schedule**: Daily at 3:00 AM
## Troubleshooting
### Database Connection Refused
```bash
# Check if database container is running
docker ps | grep postgres-production
docker ps | grep postgres-staging
# Check network connectivity
docker network inspect postgres-production-internal
docker network inspect postgres-staging-internal
# Test connection from application container
docker exec <app-container> nc -zv postgres-production 5432
docker exec <app-container> nc -zv postgres-staging 5432
```
### Migration Data Loss
If data is missing after migration:
1. **Stop application** to prevent further writes
2. **Restore from backup** (see Phase 3)
3. **Verify data** before restarting application
## Additional Resources
- **PostgreSQL Production Stack**: `deployment/stacks/postgresql-production/README.md`
- **PostgreSQL Staging Stack**: `deployment/stacks/postgresql-staging/README.md`
- **Backup Playbook**: `deployment/ansible/playbooks/backup.yml`