feat: CI/CD pipeline setup complete - Ansible playbooks updated, secrets configured, workflow ready
This commit is contained in:
643
.deployment-archive-20251030-111806/README.md
Normal file
643
.deployment-archive-20251030-111806/README.md
Normal file
@@ -0,0 +1,643 @@
|
||||
# Automated Deployment System
|
||||
|
||||
Ansible-basierte Deployment-Automatisierung für das Framework.
|
||||
|
||||
## Überblick
|
||||
|
||||
Dieses System ermöglicht automatisierte Deployments direkt auf dem Produktionsserver, wodurch die problematischen SSH-Transfers von großen Docker Images elimin werden.
|
||||
|
||||
## Vorteile
|
||||
|
||||
- **Kein Image-Transfer**: Build erfolgt direkt auf dem Produktionsserver
|
||||
- **Zuverlässig**: Keine "Broken pipe" SSH-Fehler mehr
|
||||
- **Schnell**: Direkter Build nutzt Server-Ressourcen optimal
|
||||
- **Wiederholbar**: Idempotente Ansible-Playbooks
|
||||
- **Versioniert**: Alle Deployment-Konfigurationen in Git
|
||||
|
||||
## Architektur
|
||||
|
||||
### Primary: Gitea Actions (Automated CI/CD)
|
||||
|
||||
```
|
||||
Lokale Entwicklung → Git Push → Gitea
|
||||
↓
|
||||
Gitea Actions Runner (on Production)
|
||||
↓
|
||||
Build & Test & Deploy
|
||||
↓
|
||||
Docker Swarm Rolling Update
|
||||
↓
|
||||
Health Check & Auto-Rollback
|
||||
```
|
||||
|
||||
### Fallback: Manual Ansible Deployment
|
||||
|
||||
```
|
||||
Lokale Entwicklung → Manual Trigger → Ansible Playbook
|
||||
↓
|
||||
Docker Build (Server)
|
||||
↓
|
||||
Docker Swarm Update
|
||||
↓
|
||||
Health Check
|
||||
```
|
||||
|
||||
## Komponenten
|
||||
|
||||
### 1. Gitea Actions Workflow (Primary)
|
||||
|
||||
**Location**: `.gitea/workflows/deploy.yml`
|
||||
|
||||
**Trigger**: Push to `main` branch
|
||||
|
||||
**Stages**:
|
||||
1. **Checkout**: Repository auf Runner auschecken
|
||||
2. **Build**: Docker Image mit Produktions-Optimierungen bauen
|
||||
3. **Push to Registry**: Image zu lokalem Registry pushen
|
||||
4. **Deploy**: Rolling Update via Docker Swarm
|
||||
5. **Health Check**: Automatische Verfügbarkeitsprüfung (3 Versuche)
|
||||
6. **Auto-Rollback**: Bei Health Check Failure automatischer Rollback
|
||||
|
||||
**Secrets** (in Gitea konfiguriert):
|
||||
- `DOCKER_REGISTRY`: localhost:5000
|
||||
- `STACK_NAME`: framework
|
||||
- `HEALTH_CHECK_URL`: https://michaelschiemer.de/health
|
||||
|
||||
### 2. Gitea Runner Setup (Production Server)
|
||||
|
||||
**Location**: `deployment/ansible/playbooks/setup-gitea-runner.yml`
|
||||
|
||||
**Installation**:
|
||||
```bash
|
||||
cd deployment/ansible
|
||||
ansible-playbook -i inventory/production.yml playbooks/setup-gitea-runner.yml
|
||||
```
|
||||
|
||||
**Features**:
|
||||
- Systemd Service für automatischen Start
|
||||
- Docker-in-Docker Support
|
||||
- Isolation via User `gitea-runner`
|
||||
- Logs: `journalctl -u gitea-runner -f`
|
||||
|
||||
### 3. Emergency Deployment Scripts
|
||||
|
||||
**Fallback-Szenarien** wenn Gitea Actions nicht verfügbar:
|
||||
|
||||
#### `scripts/deployment-diagnostics.sh`
|
||||
- Umfassende System-Diagnose
|
||||
- SSH, Docker Swarm, Services, Images, Networks Status
|
||||
- Health Checks und Resource Usage
|
||||
- Quick Mode: `--quick`, Verbose: `--verbose`
|
||||
|
||||
#### `scripts/service-recovery.sh`
|
||||
- Service Status Check
|
||||
- Service Restart
|
||||
- Full Recovery Procedure (5 Steps)
|
||||
- Cache Clearing
|
||||
|
||||
#### `scripts/manual-deploy-fallback.sh`
|
||||
- Manuelles Deployment ohne Gitea Actions
|
||||
- Lokaler Image Build
|
||||
- Push zu Registry
|
||||
- Ansible Deployment
|
||||
- Health Checks
|
||||
|
||||
#### `scripts/emergency-rollback.sh`
|
||||
- Schneller Rollback zu vorheriger Version
|
||||
- Listet verfügbare Image Tags
|
||||
- Direkter Rollback ohne Health Checks
|
||||
- Manuelle Verifikation erforderlich
|
||||
|
||||
### 4. Script Framework (Shared Libraries)
|
||||
|
||||
**Libraries**:
|
||||
- `scripts/lib/common.sh` - Logging, Error Handling, Utilities
|
||||
- `scripts/lib/ansible.sh` - Ansible Integration
|
||||
|
||||
**Features**:
|
||||
- Farbcodierte Logging-Funktionen (Info, Success, Warning, Error, Debug)
|
||||
- Automatische Pre-Deployment Checks
|
||||
- User-Confirmation Prompts
|
||||
- Post-Deployment Health Checks
|
||||
- Performance Metrics (Deployment Duration)
|
||||
- Retry-Logic mit exponential Backoff
|
||||
- Cleanup Handlers mit trap
|
||||
|
||||
### Ansible Konfiguration
|
||||
|
||||
- `ansible/ansible.cfg` - Ansible-Grundkonfiguration
|
||||
- `ansible/inventory/production.yml` - Produktionsserver-Inventar
|
||||
- `ansible/playbooks/deploy.yml` - Haupt-Deployment-Playbook
|
||||
|
||||
### Deployment Workflow
|
||||
|
||||
1. **Code Push**: Code-Änderungen nach Git pushen
|
||||
2. **SSH auf Server**: Auf Produktionsserver verbinden
|
||||
3. **Ansible ausführen**: Deployment-Playbook starten
|
||||
4. **Automatischer Build**: Docker Image wird auf Server gebaut
|
||||
5. **Service Update**: Docker Swarm Services werden aktualisiert
|
||||
6. **Health Check**: Automatische Verfügbarkeitsprüfung
|
||||
|
||||
## Verwendung
|
||||
|
||||
### Primary: Automated Deployment via Gitea Actions (Empfohlen)
|
||||
|
||||
Der Standard-Workflow ist vollautomatisch über Git-Push:
|
||||
|
||||
```bash
|
||||
# 1. Lokale Entwicklung abschließen
|
||||
git add .
|
||||
git commit -m "feat: new feature implementation"
|
||||
|
||||
# 2. Push to main branch triggert automatisches Deployment
|
||||
git push origin main
|
||||
|
||||
# 3. Gitea Actions führt automatisch aus:
|
||||
# - Docker Image Build (auf Production Server)
|
||||
# - Push zu lokalem Registry (localhost:5000)
|
||||
# - Docker Swarm Rolling Update
|
||||
# - Health Check (3 Versuche)
|
||||
# - Auto-Rollback bei Failure
|
||||
|
||||
# 4. Deployment Status monitoren
|
||||
# Gitea UI: https://git.michaelschiemer.de/<user>/<repo>/actions
|
||||
# Oder via SSH auf Server:
|
||||
ssh -i ~/.ssh/production deploy@94.16.110.151
|
||||
journalctl -u gitea-runner -f
|
||||
```
|
||||
|
||||
**Deployment-Zeit**: ~3-4 Minuten von Push bis Live
|
||||
|
||||
### Deployment Monitoring
|
||||
|
||||
```bash
|
||||
# Gitea Actions Logs (via Gitea UI)
|
||||
https://git.michaelschiemer.de/<user>/<repo>/actions
|
||||
|
||||
# Gitea Runner Logs (auf Production Server)
|
||||
ssh -i ~/.ssh/production deploy@94.16.110.151
|
||||
journalctl -u gitea-runner -f
|
||||
|
||||
# Service Status prüfen
|
||||
ssh -i ~/.ssh/production deploy@94.16.110.151
|
||||
docker stack services framework
|
||||
docker service logs framework_web --tail 50
|
||||
```
|
||||
|
||||
### Emergency/Fallback: Diagnostic & Recovery Scripts
|
||||
|
||||
Bei Problemen stehen Emergency Scripts zur Verfügung:
|
||||
|
||||
#### System-Diagnose
|
||||
```bash
|
||||
# Umfassende System-Diagnose
|
||||
./scripts/deployment-diagnostics.sh
|
||||
|
||||
# Quick Check (nur kritische Checks)
|
||||
./scripts/deployment-diagnostics.sh --quick
|
||||
|
||||
# Verbose Mode (mit Logs)
|
||||
./scripts/deployment-diagnostics.sh --verbose
|
||||
```
|
||||
|
||||
**Diagnostics umfasst**:
|
||||
- Local Environment (Git, Docker, Ansible, SSH)
|
||||
- SSH Connectivity zu Production
|
||||
- Docker Swarm Status (Manager/Worker Nodes)
|
||||
- Framework Services Status (Web, Queue-Worker)
|
||||
- Docker Images & Registry
|
||||
- Gitea Runner Service Status
|
||||
- Resource Usage (Disk, Memory, Docker)
|
||||
- Application Health Endpoints
|
||||
|
||||
#### Service Recovery
|
||||
|
||||
```bash
|
||||
# Service Status prüfen
|
||||
./scripts/service-recovery.sh status
|
||||
|
||||
# Services neu starten
|
||||
./scripts/service-recovery.sh restart
|
||||
|
||||
# Full Recovery Procedure (5 Steps)
|
||||
./scripts/service-recovery.sh recover
|
||||
|
||||
# Caches löschen
|
||||
./scripts/service-recovery.sh clear-cache
|
||||
```
|
||||
|
||||
**5-Step Recovery Procedure**:
|
||||
1. Check current status
|
||||
2. Verify Docker Swarm health (reinit if needed)
|
||||
3. Verify networks and volumes
|
||||
4. Force restart services
|
||||
5. Run health checks
|
||||
|
||||
#### Manual Deployment Fallback
|
||||
|
||||
Wenn Gitea Actions nicht verfügbar:
|
||||
|
||||
```bash
|
||||
# Manual Deployment (aktueller Branch)
|
||||
./scripts/manual-deploy-fallback.sh
|
||||
|
||||
# Manual Deployment (spezifischer Branch)
|
||||
./scripts/manual-deploy-fallback.sh feature/new-deployment
|
||||
|
||||
# Workflow:
|
||||
# 1. Prerequisites Check (Git clean, Docker, Ansible, SSH)
|
||||
# 2. Docker Image Build (lokal)
|
||||
# 3. Push zu Registry
|
||||
# 4. Ansible Deployment
|
||||
# 5. Health Checks
|
||||
```
|
||||
|
||||
#### Emergency Rollback
|
||||
|
||||
Schneller Rollback zu vorheriger Version:
|
||||
|
||||
```bash
|
||||
# Interactive Mode - wähle Version aus Liste
|
||||
./scripts/emergency-rollback.sh
|
||||
|
||||
# Liste verfügbare Versionen
|
||||
./scripts/emergency-rollback.sh list
|
||||
|
||||
# Direkt zu spezifischer Version
|
||||
./scripts/emergency-rollback.sh abc1234-1234567890
|
||||
|
||||
# Workflow:
|
||||
# 1. Zeigt aktuelle Version
|
||||
# 2. Zeigt verfügbare Image Tags
|
||||
# 3. Confirmation: Type 'ROLLBACK' to confirm
|
||||
# 4. Ansible Emergency Rollback
|
||||
# 5. Manuelle Verifikation erforderlich
|
||||
```
|
||||
|
||||
**⚠️ Wichtig**: Emergency Rollback macht KEINEN automatischen Health Check - manuelle Verifikation erforderlich!
|
||||
|
||||
### Tertiary Fallback: Direkt mit Ansible
|
||||
|
||||
Als letztes Mittel direkte Ansible-Ausführung:
|
||||
|
||||
```bash
|
||||
cd /home/michael/dev/michaelschiemer/deployment/ansible
|
||||
ansible-playbook -i inventory/production.yml playbooks/deploy.yml
|
||||
```
|
||||
|
||||
## Konfiguration
|
||||
|
||||
### Produktionsserver
|
||||
|
||||
Server-Details in `ansible/inventory/production.yml`:
|
||||
- **Host**: 94.16.110.151
|
||||
- **User**: deploy
|
||||
- **SSH-Key**: ~/.ssh/production
|
||||
|
||||
### Gitea Actions Secrets (Primary Deployment)
|
||||
|
||||
Konfiguriert in Gitea Repository Settings → Actions → Secrets:
|
||||
|
||||
- **DOCKER_REGISTRY**: `localhost:5000` (lokaler Registry auf Production Server)
|
||||
- **STACK_NAME**: `framework` (Docker Swarm Stack Name)
|
||||
- **HEALTH_CHECK_URL**: `https://michaelschiemer.de/health` (Health Check Endpoint)
|
||||
|
||||
**Secrets hinzufügen**:
|
||||
1. Gitea UI → Repository Settings → Actions → Secrets
|
||||
2. Add Secret für jede Variable
|
||||
3. Gitea Runner muss Zugriff auf Registry haben (localhost:5000)
|
||||
|
||||
### Gitea Runner Setup (Production Server)
|
||||
|
||||
**Systemd Service**:
|
||||
```bash
|
||||
# Status prüfen
|
||||
sudo systemctl status gitea-runner
|
||||
|
||||
# Logs verfolgen
|
||||
journalctl -u gitea-runner -f
|
||||
|
||||
# Service starten/stoppen
|
||||
sudo systemctl start gitea-runner
|
||||
sudo systemctl stop gitea-runner
|
||||
```
|
||||
|
||||
**Runner-Konfiguration**:
|
||||
- **Location**: Läuft auf Production Server (94.16.110.151)
|
||||
- **User**: `gitea-runner` (isolierter Service-User)
|
||||
- **Docker Access**: Docker-in-Docker Support aktiviert
|
||||
- **Logs**: `journalctl -u gitea-runner -f`
|
||||
|
||||
**Setup via Ansible**:
|
||||
```bash
|
||||
cd deployment/ansible
|
||||
ansible-playbook -i inventory/production.yml playbooks/setup-gitea-runner.yml
|
||||
```
|
||||
|
||||
### Docker Registry
|
||||
|
||||
**Primary Registry** (Production Server lokal):
|
||||
- **URL**: `localhost:5000` (für Runner auf Production Server)
|
||||
- **External**: `git.michaelschiemer.de:5000` (für externe Zugriffe)
|
||||
- **Image Name**: `framework`
|
||||
- **Tags**:
|
||||
- `latest` - Aktuelle Version
|
||||
- `{commit-sha}-{timestamp}` - Versionierte Images für Rollbacks
|
||||
|
||||
**Registry Access**:
|
||||
- Runner nutzt `localhost:5000` (lokaler Zugriff)
|
||||
- Manuelle Deployments nutzen `git.michaelschiemer.de:5000` (external)
|
||||
- Authentifizierung via Docker Login (falls erforderlich)
|
||||
|
||||
### Docker Swarm Stack
|
||||
|
||||
**Stack-Konfiguration**: `docker-compose.prod.yml`
|
||||
|
||||
**Services**:
|
||||
- **framework_web**: Web-Service (3 Replicas für High Availability)
|
||||
- **framework_queue-worker**: Queue-Worker (2 Replicas)
|
||||
|
||||
**Rolling Update Config**:
|
||||
```yaml
|
||||
deploy:
|
||||
replicas: 3
|
||||
update_config:
|
||||
parallelism: 1 # Ein Container pro Schritt
|
||||
delay: 10s # 10 Sekunden Pause zwischen Updates
|
||||
order: start-first # Neuer Container startet vor Stoppen des alten
|
||||
rollback_config:
|
||||
parallelism: 1
|
||||
delay: 5s
|
||||
```
|
||||
|
||||
**Stack Management**:
|
||||
```bash
|
||||
# Stack Status
|
||||
docker stack services framework
|
||||
|
||||
# Service Logs
|
||||
docker service logs framework_web --tail 50
|
||||
|
||||
# Stack Update (manuell)
|
||||
docker stack deploy -c docker-compose.prod.yml framework
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Troubleshooting-Workflow
|
||||
|
||||
Bei Problemen mit dem Deployment-System folge diesem strukturierten Workflow:
|
||||
|
||||
**Level 1: Quick Diagnostics** (Erste Anlaufstelle)
|
||||
```bash
|
||||
# Umfassende System-Diagnose
|
||||
./scripts/deployment-diagnostics.sh
|
||||
|
||||
# Quick Check (nur kritische Checks)
|
||||
./scripts/deployment-diagnostics.sh --quick
|
||||
|
||||
# Verbose Mode (mit detaillierten Logs)
|
||||
./scripts/deployment-diagnostics.sh --verbose
|
||||
```
|
||||
|
||||
**Level 2: Service Recovery** (Bei Service-Ausfällen)
|
||||
```bash
|
||||
# Service Status prüfen
|
||||
./scripts/service-recovery.sh status
|
||||
|
||||
# Services neu starten
|
||||
./scripts/service-recovery.sh restart
|
||||
|
||||
# Full Recovery Procedure (5 automatisierte Steps)
|
||||
./scripts/service-recovery.sh recover
|
||||
|
||||
# Caches löschen (bei Cache-Problemen)
|
||||
./scripts/service-recovery.sh clear-cache
|
||||
```
|
||||
|
||||
**Level 3: Manual Deployment Fallback** (Bei Gitea Actions Problemen)
|
||||
```bash
|
||||
# Manuelles Deployment (aktueller Branch)
|
||||
./scripts/manual-deploy-fallback.sh
|
||||
|
||||
# Manuelles Deployment (spezifischer Branch)
|
||||
./scripts/manual-deploy-fallback.sh feature/new-feature
|
||||
```
|
||||
|
||||
**Level 4: Emergency Rollback** (Bei kritischen Production-Problemen)
|
||||
```bash
|
||||
# Interactive Mode - Version aus Liste wählen
|
||||
./scripts/emergency-rollback.sh
|
||||
|
||||
# Verfügbare Versionen anzeigen
|
||||
./scripts/emergency-rollback.sh list
|
||||
|
||||
# Direkt zu spezifischer Version rollback
|
||||
./scripts/emergency-rollback.sh abc1234-1234567890
|
||||
```
|
||||
|
||||
### Häufige Probleme
|
||||
|
||||
#### Gitea Actions Workflow schlägt fehl
|
||||
|
||||
**Diagnose**:
|
||||
```bash
|
||||
# Gitea Runner Status prüfen (auf Production Server)
|
||||
ssh -i ~/.ssh/production deploy@94.16.110.151
|
||||
journalctl -u gitea-runner -f
|
||||
```
|
||||
|
||||
**Lösungen**:
|
||||
- Runner nicht aktiv: `sudo systemctl start gitea-runner`
|
||||
- Secrets fehlen: Gitea UI → Repository Settings → Actions → Secrets prüfen
|
||||
- Docker Registry nicht erreichbar: `docker login localhost:5000`
|
||||
|
||||
#### Services sind nicht erreichbar
|
||||
|
||||
**Diagnose**:
|
||||
```bash
|
||||
# Quick Health Check
|
||||
./scripts/deployment-diagnostics.sh --quick
|
||||
```
|
||||
|
||||
**Lösungen**:
|
||||
```bash
|
||||
# Services automatisch recovern
|
||||
./scripts/service-recovery.sh recover
|
||||
```
|
||||
|
||||
#### Deployment hängt oder ist langsam
|
||||
|
||||
**Diagnose**:
|
||||
```bash
|
||||
# Umfassende Diagnose mit Resource-Checks
|
||||
./scripts/deployment-diagnostics.sh --verbose
|
||||
```
|
||||
|
||||
**Lösungen**:
|
||||
- Disk Space voll: Alte Docker Images aufräumen (`docker system prune -a`)
|
||||
- Memory Issues: Services neu starten (`./scripts/service-recovery.sh restart`)
|
||||
- Netzwerk-Probleme: Docker Swarm Overlay Network prüfen
|
||||
|
||||
#### Health Checks schlagen fehl
|
||||
|
||||
**Diagnose**:
|
||||
```bash
|
||||
# Application Health direkt testen
|
||||
curl -k https://michaelschiemer.de/health
|
||||
curl -k https://michaelschiemer.de/health/database
|
||||
curl -k https://michaelschiemer.de/health/redis
|
||||
```
|
||||
|
||||
**Lösungen**:
|
||||
```bash
|
||||
# Service Logs prüfen
|
||||
ssh -i ~/.ssh/production deploy@94.16.110.151
|
||||
docker service logs framework_web --tail 100
|
||||
|
||||
# Caches löschen falls Health Check Cache-Issues zeigt
|
||||
./scripts/service-recovery.sh clear-cache
|
||||
```
|
||||
|
||||
#### Rollback nach fehlgeschlagenem Deployment
|
||||
|
||||
**Schneller Emergency Rollback**:
|
||||
```bash
|
||||
# 1. Verfügbare Versionen anzeigen
|
||||
./scripts/emergency-rollback.sh list
|
||||
|
||||
# 2. Zu letzter funktionierender Version rollback
|
||||
./scripts/emergency-rollback.sh <previous-tag>
|
||||
|
||||
# 3. Manuelle Verifikation
|
||||
curl -k https://michaelschiemer.de/health
|
||||
```
|
||||
|
||||
**⚠️ Wichtig**: Emergency Rollback macht KEINEN automatischen Health Check - manuelle Verifikation erforderlich!
|
||||
|
||||
## Nächste Schritte
|
||||
|
||||
### Git-Integration ✅ Completed
|
||||
|
||||
Gitea Actions CI/CD ist vollständig implementiert und operational:
|
||||
- ✅ Automatic Trigger bei Push zu main-Branch
|
||||
- ✅ Gitea Webhook Integration
|
||||
- ✅ Automated Build, Test & Deploy Pipeline
|
||||
- ✅ Health Checks mit Auto-Rollback
|
||||
|
||||
**Aktuelle Features**:
|
||||
- Zero-downtime Rolling Updates
|
||||
- Automatic Rollback bei Deployment-Failures
|
||||
- Versioned Image Tagging für manuelle Rollbacks
|
||||
- Comprehensive Emergency Recovery Scripts
|
||||
|
||||
### Monitoring (Geplante Verbesserungen)
|
||||
|
||||
**Short-Term** (1-2 Monate):
|
||||
- Deployment-Benachrichtigungen via Email/Slack
|
||||
- Prometheus/Grafana Integration für Metrics
|
||||
- Application Performance Monitoring (APM)
|
||||
- Automated Health Check Dashboards
|
||||
|
||||
**Mid-Term** (3-6 Monate):
|
||||
- Log Aggregation mit ELK/Loki Stack
|
||||
- Distributed Tracing für Microservices
|
||||
- Alerting Rules für kritische Metriken
|
||||
- Capacity Planning & Resource Forecasting
|
||||
|
||||
**Long-Term** (6-12 Monate):
|
||||
- Cost Optimization Dashboards
|
||||
- Predictive Failure Detection
|
||||
- Automated Performance Tuning
|
||||
- Multi-Region Deployment Support
|
||||
|
||||
## Sicherheit
|
||||
|
||||
### Production Security Measures
|
||||
|
||||
- **SSH-Key-basierte Authentifizierung**: Zugriff nur mit autorisiertem Private Key (~/.ssh/production)
|
||||
- **Keine Passwörter in Konfiguration**: Alle Credentials via Gitea Actions Secrets oder Docker Secrets
|
||||
- **Docker Secrets für sensitive Daten**: Database-Credentials, API-Keys, Encryption-Keys
|
||||
- **Gitea Runner Isolation**: Dedicated Service-User `gitea-runner` mit minimalen Permissions
|
||||
- **Registry Access Control**: Localhost-only Registry für zusätzliche Security
|
||||
- **HTTPS-only Communication**: Alle Deployments über verschlüsselte Verbindungen
|
||||
|
||||
### Deployment Authorization
|
||||
|
||||
- **Gitea Repository Access**: Push-Rechte erforderlich für automatisches Deployment
|
||||
- **Emergency Script Access**: SSH-Key + authorized_keys auf Production Server
|
||||
- **Manual Rollback**: Manuelle Intervention via authorized SSH-Key
|
||||
|
||||
## Performance
|
||||
|
||||
### Deployment Performance Metrics
|
||||
|
||||
- **Build-Zeit**: ~2-3 Minuten (je nach Docker Layer Caching)
|
||||
- **Registry Push**: ~30-60 Sekunden (Image Size: ~500MB)
|
||||
- **Deployment-Zeit**: ~60-90 Sekunden (Rolling Update mit 3 Replicas)
|
||||
- **Health Check Duration**: ~10-15 Sekunden (3 Retry-Attempts)
|
||||
- **Gesamt**: ~3-4 Minuten von Push bis Live (bei erfolgreichem Deployment)
|
||||
|
||||
### Rollback Performance
|
||||
|
||||
- **Automated Rollback**: ~30 Sekunden (bei Health Check Failure)
|
||||
- **Manual Emergency Rollback**: ~60 Sekunden (via emergency-rollback.sh)
|
||||
- **Service Recovery**: ~90 Sekunden (via service-recovery.sh recover)
|
||||
|
||||
### Optimizations in Place
|
||||
|
||||
- **Docker Layer Caching**: Wiederverwendung unveränderter Layer
|
||||
- **Multi-Stage Builds**: Kleinere Production Images
|
||||
- **Parallel Replica Updates**: Minimale Downtime durch start-first Strategy
|
||||
- **Local Registry**: Kein externes Network Bottleneck
|
||||
|
||||
## Support
|
||||
|
||||
### Erste Anlaufstellen bei Problemen
|
||||
|
||||
**1. Emergency Scripts nutzen** (Empfohlen):
|
||||
```bash
|
||||
# Quick Diagnostics - System-Gesundheit prüfen
|
||||
./scripts/deployment-diagnostics.sh --quick
|
||||
|
||||
# Service Recovery - Automatische Wiederherstellung
|
||||
./scripts/service-recovery.sh recover
|
||||
|
||||
# Manual Deployment - Fallback wenn Gitea Actions ausfällt
|
||||
./scripts/manual-deploy-fallback.sh
|
||||
|
||||
# Emergency Rollback - Schneller Rollback zu vorheriger Version
|
||||
./scripts/emergency-rollback.sh list
|
||||
```
|
||||
|
||||
**2. Gitea Actions Logs prüfen**:
|
||||
- Gitea UI → Repository → Actions Tab
|
||||
- Oder via SSH: `journalctl -u gitea-runner -f`
|
||||
|
||||
**3. Service Logs direkt prüfen**:
|
||||
```bash
|
||||
ssh -i ~/.ssh/production deploy@94.16.110.151
|
||||
docker service logs framework_web --tail 100
|
||||
docker service logs framework_queue-worker --tail 100
|
||||
```
|
||||
|
||||
**4. Docker Stack Status**:
|
||||
```bash
|
||||
ssh -i ~/.ssh/production deploy@94.16.110.151
|
||||
docker stack services framework
|
||||
docker stack ps framework --no-trunc
|
||||
```
|
||||
|
||||
### Eskalationspfad
|
||||
|
||||
1. **Level 1**: Automatische Diagnostics → `./scripts/deployment-diagnostics.sh`
|
||||
2. **Level 2**: Service Recovery → `./scripts/service-recovery.sh recover`
|
||||
3. **Level 3**: Manual Deployment → `./scripts/manual-deploy-fallback.sh`
|
||||
4. **Level 4**: Emergency Rollback → `./scripts/emergency-rollback.sh`
|
||||
5. **Level 5**: Direct Ansible → `cd deployment/ansible && ansible-playbook -i inventory/production.yml playbooks/deploy.yml`
|
||||
|
||||
### Kontakte
|
||||
|
||||
- **Production Server**: deploy@94.16.110.151 (SSH-Key erforderlich)
|
||||
- **Documentation**: `/home/michael/dev/michaelschiemer/deployment/README.md`
|
||||
- **Emergency Scripts**: `/home/michael/dev/michaelschiemer/deployment/scripts/`
|
||||
Reference in New Issue
Block a user