michaelschiemer/deployment/README.md

# Pragmatic Production Deployment Setup

## Architecture Overview

This deployment setup uses separate Docker Compose stacks for better maintainability and clear separation of concerns.

### Docker Compose Structure

The project uses a **Base + Override Pattern** to prevent configuration drift between environments:

- **`docker-compose.base.yml`** - Shared base configuration (services, networks, volumes)
- **`docker-compose.local.yml`** - Local development overrides (ports, host mounts, debug flags)
- **`docker-compose.staging.yml`** - Staging environment overrides (Traefik labels, staging volumes)
- **`docker-compose.production.yml`** - Production environment overrides (security, logging, resources)

**Usage:**
```bash
# Local development
docker compose -f docker-compose.base.yml -f docker-compose.local.yml up

# Staging
docker compose -f docker-compose.base.yml -f docker-compose.staging.yml up

# Production
docker compose -f docker-compose.base.yml -f docker-compose.production.yml up
```

**Benefits:**
- ✅ Single source of truth for shared configuration
- ✅ Environment-specific differences clearly visible
- ✅ Reduced configuration drift between environments
- ✅ Easier maintenance and updates

### Infrastructure Components

```
Production Server (94.16.110.151)
├── Stack 1: Traefik (Reverse Proxy & SSL)
├── Stack 2: Gitea (Git Server + MySQL + Redis)
├── Stack 3: Docker Registry (Private Registry)
├── Stack 4: Application (PHP + Nginx + Redis + Queue Workers)
├── Stack 5: PostgreSQL (Database)
└── Stack 6: Monitoring (Portainer + Grafana + Prometheus)

Development Machine
└── Gitea Actions Runner (local, Docker-in-Docker)
```

## Deployment Flow

```
Developer → git push
         ↓
    Gitea (Production)
         ↓
    Gitea Actions (Dev Machine)
         ↓
    Build Docker Image
         ↓
    Push to Private Registry
         ↓
    SSH/Ansible → Production Server
         ↓
    docker compose pull
         ↓
    docker compose up -d
```

## Directory Structure

### Local Repository Structure

```
deployment/
├── ansible/                   # Ansible config, playbooks, inventory, templates
├── gitea-runner/              # Self-hosted Gitea Actions runner stack
├── stacks/                    # Docker Compose stacks
│   ├── application/          # Main PHP application
│   ├── gitea/                # Git server
│   ├── minio/                # Object storage
│   ├── monitoring/           # Portainer, Grafana, Prometheus
│   ├── postgresql/           # PostgreSQL database
│   ├── registry/             # Private Docker registry
│   ├── staging/              # Optional staging stack
│   └── traefik/              # Reverse proxy with SSL certificates
├── docs/                      # 📚 Dokumentation (siehe docs/README.md)
│   ├── guides/               # Anleitungen & Guides
│   ├── reference/            # Referenz-Dokumentation
│   ├── status/               # Status & Tracking
│   ├── tests/                # Test-Dokumentation
│   └── history/              # Logs & Historie
└── README.md (dieses Dokument)
```

### Server Directory Structure

Auf dem Production-Server existieren zwei Hauptverzeichnisse:

```
/home/deploy/
├── deployment/          # Infrastructure-as-Code (24M)
│   ├── stacks/         # Docker Compose Stacks
│   └── backups/        # Backup-Dateien
└── michaelschiemer/    # Application Code (491M)
    ├── current/        # Symlink → Aktuelle deployed Version
    └── .archive/       # Alte Versionen (Rollback)
```

**📖 Detaillierte Erklärung:** Siehe [docs/server-directory-structure.md](docs/server-directory-structure.md)

## Getting Started

### 🧪 Pipeline-Tests vorbereiten

**Vor dem ersten Deployment:**

1. **Prerequisites prüfen:**
   ```bash
   ./deployment/scripts/test-pipeline-prerequisites.sh
   ```

2. **Test-Anleitung lesen:**
   - [Pipeline Test Checklist](docs/guides/pipeline-test-checklist.md) ⭐ - Schritt-für-Schritt Anleitung
   - [Pipeline Testing Guide](docs/guides/pipeline-testing-guide.md) - Übersicht und Troubleshooting

3. **Backup-Test durchführen:**
   ```bash
   ./deployment/scripts/test-backup.sh
   ```

### 🚀 Quick Start: Code deployen

**Empfohlener Workflow (Staging → Production):**

1. **Push auf `staging` Branch** (Standard für Entwicklung)
```bash
git add .
git commit -m "feat: Add new feature"
git push origin staging  # → Automatisches Deployment zu Staging
```

2. **Testen auf Staging**
- Staging URL: `https://staging.michaelschiemer.de`
- Tests durchführen und verifizieren

3. **Merge nach `main`** (nur nach erfolgreichem Test)
```bash
git checkout main
git merge staging
git push origin main  # → Automatisches Deployment zu Production
```

**⚠️ Wichtig:** Niemals direkt auf `main` pushen - immer erst auf `staging` testen!

**Pipeline-Status:** `https://git.michaelschiemer.de/michael/michaelschiemer/actions`

**📖 Vollständige Anleitung:** Siehe [docs/guides/quick-start.md](docs/guides/quick-start.md) oder [docs/guides/code-change-workflow.md](docs/guides/code-change-workflow.md)

---

### Initial Setup (nur bei erstem Setup)

**Prerequisites:**

**Production Server:**
- Docker & Docker Compose installed
- Firewall configured (ports 80, 443, 2222)
- User `deploy` with Docker permissions
- SSH access configured

**Development Machine:**
- Docker & Docker Compose installed
- Ansible installed
- SSH key configured for production server

**Deployment via Ansible:**
```bash
cd deployment/ansible
ansible-playbook -i inventory/production.yml playbooks/setup-infrastructure.yml
```

Dieses Playbook deployed alle Stacks:
- Traefik (Reverse Proxy & SSL)
- PostgreSQL (Database)
- Docker Registry (Private Registry)
- Gitea (Git Server)
- Monitoring (Portainer, Grafana, Prometheus)
- **Production Stack** (PHP Application + Nginx + Redis + Queue Workers)

**Gitea Initial Setup (nach Infrastructure Deployment):**
```bash
# Automatische Initial Setup via Ansible
cd deployment/ansible

# 1. Gitea Initial Configuration (Admin-User erstellen)
ansible-playbook -i inventory/production.yml \
  playbooks/setup-gitea-initial-config.yml \
  --vault-password-file secrets/.vault_pass

# 2. Repository in Gitea erstellen und Git-Remote konfigurieren
ansible-playbook -i inventory/production.yml \
  playbooks/setup-gitea-repository.yml \
  --vault-password-file secrets/.vault_pass \
  -e "repo_name=michaelschiemer" \
  -e "repo_owner=michael" \
  -e "repo_private=false"
```

**📖 Vollständige Setup-Anleitung:** Siehe [SETUP-GUIDE.md](SETUP-GUIDE.md)

## Stack Documentation

Each stack has its own README with detailed configuration:

- [Traefik](stacks/traefik/README.md) - Reverse proxy setup
- [Gitea](stacks/gitea/README.md) - Git server configuration
- [Registry](stacks/registry/README.md) - Private registry setup
- [Production](stacks/production/README.md) - Production application deployment
- [PostgreSQL](stacks/postgresql/README.md) - Database configuration
- [Monitoring](stacks/monitoring/README.md) - Monitoring stack

## Deployment Commands

### Automatisches Deployment (Empfohlen)

**Standard-Workflow: Staging → Production**

1. **Push auf `staging`** (Standard für Entwicklung)
```bash
git add .
git commit -m "feat: Add new feature"
git push origin staging  # → Deployt zu Staging
```

2. **Testen auf Staging**, dann **Merge nach `main`**
```bash
git checkout main
git merge staging
git push origin main  # → Deployt zu Production
```

**📖 Vollständige Command-Referenz:** Siehe [docs/guides/deployment-commands.md](docs/guides/deployment-commands.md)

### Update Specific Stack
```bash
cd stacks/<stack-name>
docker compose pull
docker compose up -d
```

## CI/CD Pipeline

The CI/CD pipeline is defined in `.gitea/workflows/build-image.yml` and runs automatically on push to `staging` or `main` branch.

### Recommended Workflow: Staging → Production

**1. Push to `staging` (Standard for Development)**
```bash
# Make changes locally
# ... edit files ...

# Commit and push to staging
git add .
git commit -m "feat: Add new feature"
git push origin staging  # → Deploys to Staging
```

**What happens automatically on `staging`:**
- ✅ Tests run (~2-5 min)
- ✅ Docker image is built (~3-5 min)
- ✅ Image is pushed to registry (~1-2 min)
- ✅ Deployment to Staging via SSH/SCP (~2-4 min)
- ✅ Staging stack is updated

**2. Test on Staging**
- Staging URL: `https://staging.michaelschiemer.de`
- Verify functionality and run tests

**3. Merge to `main` (Only after successful testing)**
```bash
git checkout main
git merge staging
git push origin main  # → Deploys to Production
```

**What happens automatically on `main`:**
- ✅ Tests run (~2-5 min)
- ✅ Docker image is built (~3-5 min)
- ✅ Image is pushed to registry (~1-2 min)
- ✅ Deployment to Production via SSH/SCP (~2-4 min)
- ✅ Production stack is updated

**Total time per deployment:** ~8-15 minutes

**Status check:**
- Pipeline status: `https://git.michaelschiemer.de/michael/michaelschiemer/actions`
- Staging status: `ssh deploy@94.16.110.151 "cd ~/deployment/stacks/staging && docker compose ps"`
- Production status: `ssh deploy@94.16.110.151 "cd ~/deployment/stacks/production && docker compose ps"`

**⚠️ Important:** Never push directly to `main` - always test on `staging` first!

**📖 Vollständige Dokumentation:**

- **[docs/guides/quick-start.md](docs/guides/quick-start.md)** ⭐ - Schnellstart-Guide für Deployment
- **[docs/guides/code-change-workflow.md](docs/guides/code-change-workflow.md)** - Kompletter Guide für Codeänderungen
- **[docs/reference/application-stack.md](docs/reference/application-stack.md)** - Detaillierter Deployment-Ablauf
- **[docs/status/ci-cd-status.md](docs/status/ci-cd-status.md)** - CI/CD Pipeline Status & Checkliste
- **[docs/status/deployment-summary.md](docs/status/deployment-summary.md)** - Projekt-Status Übersicht

### Pipeline Details

The CI/CD pipeline runs on push to `staging` or `main` branch:

**On `staging` branch:**
1. **Build Stage**: Build Docker image
2. **Push Stage**: Push to private registry
3. **Deploy Stage**: Deploy to Staging via SSH/SCP

**On `main` branch:**
1. **Build Stage**: Build Docker image
2. **Push Stage**: Push to private registry
3. **Deploy Stage**: Deploy to Production via SSH/SCP

## Monitoring

Access monitoring tools:

- **Portainer**: https://portainer.yourdomain.com
- **Grafana**: https://grafana.yourdomain.com
- **Prometheus**: https://prometheus.yourdomain.com

## Backup & Recovery

### Current State

Infrastructure backups are handled per stack. The PostgreSQL stack ships helper scripts under `stacks/postgresql/scripts/` (see `backup.sh` and `restore.sh`). Registry and Gitea data snapshots are currently managed manually on the host.

### Roadmap

An Ansible-level backup/restore playbook is still planned. Track progress in `DEPLOYMENT-TODO.md` and update this section once the playbook is available.

## Security

- All external services behind Traefik with HTTPS
- Private registry with BasicAuth
- Secrets managed via Ansible Vault
- Regular security updates via Watchtower

## Troubleshooting

### Check Stack Health
```bash
cd stacks/<stack-name>
docker compose ps
docker compose logs -f
```

### Check Service Connectivity
```bash
curl -I https://app.yourdomain.com
docker network inspect traefik-public
```

### View Logs
```bash
# Application logs (Production)
docker compose -f docker-compose.base.yml -f docker-compose.production.yml logs -f php

# Traefik logs
docker compose -f stacks/traefik/docker-compose.yml logs -f
```

## 📚 Dokumentation Index

**Vollständige Dokumentations-Übersicht:** Siehe [docs/README.md](docs/README.md)

**Wichtigste Dokumente:**
- **[docs/guides/quick-start.md](docs/guides/quick-start.md)** ⭐ - Schnellstart
- **[docs/guides/code-change-workflow.md](docs/guides/code-change-workflow.md)** - Code deployen
- **[docs/reference/application-stack.md](docs/reference/application-stack.md)** - Deployment-Details
- **[docs/status/ci-cd-status.md](docs/status/ci-cd-status.md)** - CI/CD Status
- **[docs/status/deployment-summary.md](docs/status/deployment-summary.md)** - Projekt-Übersicht

## Support

Für spezifische Fragen helfen die folgenden Dokumente weiter:
- [docs/reference/workflow-troubleshooting.md](docs/reference/workflow-troubleshooting.md) – Fehleranalyse für Laufzeiten & Pipelines
- [docs/status/ci-cd-status.md](docs/status/ci-cd-status.md) – Pipeline-Status & Checklisten
- [docs/status/deployment-summary.md](docs/status/deployment-summary.md) – Aktueller Projektüberblick
- [docs/reference/application-stack.md](docs/reference/application-stack.md) – Detaillierte Deployment-Schritte

## License

This deployment configuration is part of the Custom PHP Framework project.