feat: update deployment configuration and encrypted env loader
- Update Ansible playbooks and roles for application deployment - Add new Gitea/Traefik troubleshooting playbooks - Update Docker Compose configurations (base, local, staging, production) - Enhance EncryptedEnvLoader with improved error handling - Add deployment scripts (autossh setup, migration, secret testing) - Update CI/CD workflows and documentation - Add Semaphore stack configuration
This commit is contained in:
191
ENV_SETUP.md
191
ENV_SETUP.md
@@ -1,12 +1,15 @@
|
||||
# Environment Configuration Guide
|
||||
|
||||
## 📁 .env File Structure (Simplified)
|
||||
## 📁 .env File Structure (Base + Override Pattern)
|
||||
|
||||
Nach der Konsolidierung vom 27.10.2024 gibt es nur noch **2 .env Files** im Root:
|
||||
Die neue Struktur verwendet ein **Base + Override Pattern** (analog zu docker-compose):
|
||||
|
||||
```
|
||||
├── .env # Development (aktiv, gitignored)
|
||||
└── .env.example # Template für neue Entwickler
|
||||
├── .env.example # Template für neue Entwickler (vollständige Dokumentation)
|
||||
├── .env.base # Gemeinsame Variablen für alle Environments (versioniert)
|
||||
├── .env.local # Lokale Development-Overrides (gitignored)
|
||||
├── .env.staging # Staging-spezifische Overrides (optional, gitignored)
|
||||
└── .env.production # Production (generiert durch Ansible, nicht im Repo)
|
||||
```
|
||||
|
||||
## 🏗️ Development Setup
|
||||
@@ -14,21 +17,32 @@ Nach der Konsolidierung vom 27.10.2024 gibt es nur noch **2 .env Files** im Root
|
||||
### Initial Setup
|
||||
|
||||
```bash
|
||||
# 1. Copy example file
|
||||
cp .env.example .env
|
||||
# 1. .env.base ist bereits im Repository (gemeinsame Variablen)
|
||||
# 2. Erstelle .env.local für lokale Overrides
|
||||
cp .env.example .env.local
|
||||
|
||||
# 2. Anpassungen für lokale Entwicklung
|
||||
# - DB Credentials
|
||||
# - API Keys
|
||||
# - Feature Flags
|
||||
# 3. Passe .env.local an deine lokale Entwicklung an
|
||||
# - DB Credentials (lokal)
|
||||
# - API Keys (lokal)
|
||||
# - Debug-Flags
|
||||
```
|
||||
|
||||
### Active File: `.env`
|
||||
### Framework lädt automatisch: `.env.base` → `.env.local` (Overrides)
|
||||
|
||||
- ✅ Wird von Docker Compose verwendet
|
||||
- ✅ Wird von PHP Application gelesen
|
||||
- ❌ NICHT committen (gitignored)
|
||||
- ✅ Jeder Entwickler hat eigene Version
|
||||
**Priorität:**
|
||||
1. System Environment Variables (Docker ENV)
|
||||
2. `.env.base` (gemeinsame Basis)
|
||||
3. `.env.local` (lokale Overrides)
|
||||
4. `.env.secrets` (verschlüsselte Secrets, optional)
|
||||
|
||||
**Wichtig:** `env_file` in Docker Compose ist nicht nötig!
|
||||
- Framework lädt automatisch `.env.base` → `.env.local` via `EncryptedEnvLoader`
|
||||
- Docker Compose `env_file` ist optional und wird nur für Container-interne Variablen benötigt
|
||||
- PHP-Anwendung lädt ENV-Variablen direkt aus den Dateien
|
||||
|
||||
**Backward Compatibility:**
|
||||
- Falls `.env.base` oder `.env.local` nicht existieren, wird `.env` geladen (Fallback)
|
||||
- Migration: Bestehende `.env` Files funktionieren weiterhin
|
||||
|
||||
## 🚀 Production Deployment
|
||||
|
||||
@@ -63,54 +77,111 @@ ansible-playbook -i inventories/production/hosts.yml \
|
||||
3. File nach `/home/deploy/michaelschiemer/shared/.env.production` deployen
|
||||
4. Docker Compose mounted diesen File in Container
|
||||
|
||||
## 🔒 Security Best Practices
|
||||
## 🔒 Security & Secret Management
|
||||
|
||||
### Docker Secrets (Production & Staging)
|
||||
|
||||
**Production und Staging verwenden Docker Secrets:**
|
||||
|
||||
1. **Ansible Vault → Docker Secrets Dateien**
|
||||
- Ansible Playbook erstellt Secret-Dateien in `secrets/` Verzeichnis
|
||||
- Dateien haben sichere Permissions (0600)
|
||||
|
||||
2. **Docker Compose Secrets**
|
||||
- Secrets werden in `docker-compose.base.yml` definiert
|
||||
- Environment-Variablen nutzen `*_FILE` Pattern (z.B. `DB_PASSWORD_FILE=/run/secrets/db_user_password`)
|
||||
- Framework lädt automatisch via `DockerSecretsResolver`
|
||||
|
||||
3. **Framework Support**
|
||||
- `DockerSecretsResolver` unterstützt automatisch `*_FILE` Pattern
|
||||
- Kein manuelles Secret-Loading mehr nötig (wird automatisch vom Framework behandelt)
|
||||
|
||||
### Development
|
||||
|
||||
```bash
|
||||
# .env niemals committen
|
||||
# .env.local niemals committen
|
||||
git status
|
||||
# Should show: .env (untracked) ✅
|
||||
# Should show: .env.local (untracked) ✅
|
||||
|
||||
# .env.base ist versioniert (keine Secrets!)
|
||||
# Falls versehentlich staged:
|
||||
git reset HEAD .env
|
||||
git reset HEAD .env.local
|
||||
```
|
||||
|
||||
### Production
|
||||
|
||||
- ✅ Secrets in Ansible Vault
|
||||
- ✅ Ansible erstellt Docker Secrets Dateien (`secrets/*.txt`)
|
||||
- ✅ Docker Compose Secrets aktiviert
|
||||
- ✅ Framework lädt automatisch via `*_FILE` Pattern
|
||||
- ✅ .env.production auf Server wird NICHT ins Repository committed
|
||||
- ✅ Template `.env.production.j2` enthält nur Platzhalter
|
||||
- ✅ Echte Werte werden zur Deploy-Zeit eingesetzt
|
||||
- ✅ Template `application.env.j2` verwendet `*_FILE` Pattern
|
||||
|
||||
## 📝 Adding New Environment Variables
|
||||
|
||||
### Development
|
||||
|
||||
```bash
|
||||
# 1. Add to .env.example with placeholder
|
||||
echo "NEW_API_KEY=your_api_key_here" >> .env.example
|
||||
# 1. Add to .env.base if shared across environments
|
||||
echo "NEW_API_KEY=" >> .env.base
|
||||
|
||||
# 2. Add actual value to your local .env
|
||||
echo "NEW_API_KEY=abc123..." >> .env
|
||||
# 2. Add to .env.local for local development
|
||||
echo "NEW_API_KEY=abc123..." >> .env.local
|
||||
|
||||
# 3. Update .env.example for documentation
|
||||
echo "NEW_API_KEY=your_api_key_here" >> .env.example
|
||||
```
|
||||
|
||||
### Production
|
||||
**Hinweis:** Wenn die Variable nur für lokale Entwicklung ist, nur in `.env.local` hinzufügen.
|
||||
|
||||
### Production (mit Docker Secrets)
|
||||
|
||||
```bash
|
||||
# 1. Add to Ansible Template (use *_FILE pattern for secrets)
|
||||
# File: deployment/ansible/templates/application.env.j2
|
||||
echo "# Use Docker Secrets via *_FILE pattern" >> application.env.j2
|
||||
echo "NEW_API_KEY_FILE=/run/secrets/new_api_key" >> application.env.j2
|
||||
|
||||
# 2. Add to docker-compose.base.yml secrets section
|
||||
# File: docker-compose.base.yml
|
||||
# secrets:
|
||||
# new_api_key:
|
||||
# file: ./secrets/new_api_key.txt
|
||||
|
||||
# 3. Add secret to Ansible Vault
|
||||
ansible-vault edit deployment/ansible/secrets/production.vault.yml
|
||||
# Add: vault_new_api_key: "production_value"
|
||||
|
||||
# 4. Update setup-production-secrets.yml to create secret file
|
||||
# File: deployment/ansible/playbooks/setup-production-secrets.yml
|
||||
# Add to loop:
|
||||
# - name: new_api_key
|
||||
# value: "{{ vault_new_api_key }}"
|
||||
|
||||
# 5. Deploy
|
||||
cd deployment/ansible
|
||||
ansible-playbook -i inventory/production.yml \
|
||||
playbooks/setup-production-secrets.yml \
|
||||
--vault-password-file .vault_pass
|
||||
```
|
||||
|
||||
### Production (ohne Docker Secrets, fallback)
|
||||
|
||||
Falls Docker Secrets nicht verwendet werden sollen:
|
||||
|
||||
```bash
|
||||
# 1. Add to Ansible Template
|
||||
# File: deployment/infrastructure/templates/.env.production.j2
|
||||
echo "NEW_API_KEY={{ vault_new_api_key }}" >> .env.production.j2
|
||||
# File: deployment/ansible/templates/application.env.j2
|
||||
echo "NEW_API_KEY={{ vault_new_api_key }}" >> application.env.j2
|
||||
|
||||
# 2. Add secret to Ansible Vault
|
||||
ansible-vault edit deployment/infrastructure/group_vars/production/vault.yml
|
||||
ansible-vault edit deployment/ansible/secrets/production.vault.yml
|
||||
# Add: vault_new_api_key: "production_value"
|
||||
|
||||
# 3. Deploy
|
||||
cd deployment/infrastructure
|
||||
ansible-playbook -i inventories/production/hosts.yml \
|
||||
playbooks/deploy-rsync-based.yml \
|
||||
--vault-password-file .vault_pass
|
||||
cd deployment/ansible
|
||||
ansible-playbook -i inventory/production.yml \
|
||||
playbooks/deploy-update.yml
|
||||
```
|
||||
|
||||
## 🗑️ Removed Files (Consolidation 27.10.2024)
|
||||
@@ -129,35 +200,67 @@ Diese Files wurden gelöscht, da sie redundant/nicht verwendet wurden:
|
||||
## ✅ Current State
|
||||
|
||||
### Local Development
|
||||
- ✅ Einziges aktives File: `.env`
|
||||
- ✅ Template: `.env.example`
|
||||
- ✅ Klar und eindeutig
|
||||
- ✅ Base File: `.env.base` (versioniert, gemeinsame Variablen)
|
||||
- ✅ Override File: `.env.local` (gitignored, lokale Anpassungen)
|
||||
- ✅ Template: `.env.example` (Dokumentation)
|
||||
- ✅ Framework lädt automatisch: `.env.base` → `.env.local` (Overrides)
|
||||
|
||||
### Production
|
||||
- ✅ Single Source: `/home/deploy/michaelschiemer/shared/.env.production` (auf Server)
|
||||
- ✅ Verwaltet durch: Ansible Template `.env.production.j2`
|
||||
- ✅ Secrets in: Ansible Vault
|
||||
- ✅ Verwaltet durch: Ansible Template `application.env.j2`
|
||||
- ✅ Secrets: Docker Secrets (`secrets/*.txt` Dateien)
|
||||
- ✅ Framework lädt automatisch via `*_FILE` Pattern (`DockerSecretsResolver`)
|
||||
- ✅ Keine Duplikate
|
||||
|
||||
### Staging
|
||||
- ✅ Docker Compose Environment Variables
|
||||
- ✅ Docker Secrets aktiviert (wie Production)
|
||||
- ✅ Optional: `.env.staging` für Staging-spezifische Overrides
|
||||
|
||||
## 🔍 Verification
|
||||
|
||||
```bash
|
||||
# Check local .env files
|
||||
ls -la .env*
|
||||
# Should show: .env, .env.example
|
||||
# Should show: .env.base (versioniert), .env.local (gitignored), .env.example
|
||||
|
||||
# Check Ansible template exists
|
||||
ls -la deployment/infrastructure/templates/.env.production.j2
|
||||
ls -la deployment/ansible/templates/application.env.j2
|
||||
# Should exist
|
||||
|
||||
# Check Docker Secrets files exist (on server)
|
||||
ls -la {{ app_stack_path }}/secrets/
|
||||
# Should show: db_user_password.txt, redis_password.txt, app_key.txt, etc.
|
||||
|
||||
# Check NO old files remain
|
||||
find . -name ".env.production" -o -name ".env.*.example" | grep -v .env.example
|
||||
find . -name ".env.production" -o -name ".env.*.example" | grep -v .env.example | grep -v .env.base
|
||||
# Should be empty
|
||||
```
|
||||
|
||||
## 📞 Support
|
||||
|
||||
Bei Fragen zum .env Setup:
|
||||
- Development: Siehe `.env.example`
|
||||
- Production: Siehe `deployment/infrastructure/templates/.env.production.j2`
|
||||
- Secrets: Kontaktiere DevOps Team für Ansible Vault Zugriff
|
||||
- Development: Siehe `.env.base` (gemeinsame Variablen) und `.env.example` (Dokumentation)
|
||||
- Production: Siehe `deployment/ansible/templates/application.env.j2`
|
||||
- Secrets: Docker Secrets aktiviert, verwaltet durch Ansible Vault
|
||||
- Migration: Framework unterstützt Fallback auf `.env` (alte Struktur)
|
||||
|
||||
## 🔄 Migration von alter Struktur
|
||||
|
||||
**Von `.env` zu `.env.base` + `.env.local`:**
|
||||
|
||||
```bash
|
||||
# 1. Erstelle .env.base (gemeinsame Variablen extrahieren)
|
||||
# (wird automatisch vom Framework erkannt)
|
||||
|
||||
# 2. Erstelle .env.local (nur lokale Overrides)
|
||||
cp .env .env.local
|
||||
|
||||
# 3. Entferne gemeinsame Variablen aus .env.local
|
||||
# (nur lokale Anpassungen behalten)
|
||||
|
||||
# 4. Alte .env kann später entfernt werden
|
||||
# (nach erfolgreicher Migration)
|
||||
```
|
||||
|
||||
**Hinweis:** Framework lädt automatisch `.env.base` + `.env.local`. Falls diese nicht existieren, wird `.env` als Fallback geladen (Backward Compatibility).
|
||||
|
||||
Reference in New Issue
Block a user