michaelschiemer/deployment/legacy/ARCHITECTURE_ANALYSIS.md

# Legacy Deployment Architecture Analysis

**Created**: 2025-01-24
**Status**: Archived - System being redesigned

## Executive Summary

This document analyzes the existing deployment architecture that led to the decision to rebuild from scratch.

## Discovered Issues

### 1. Docker Swarm vs Docker Compose Confusion

**Problem**: System designed for Docker Swarm but running with Docker Compose
- Stack files reference Swarm features (secrets, configs)
- Docker Swarm not initialized on target server
- Local development uses Docker Compose
- Production deployment unclear which to use

**Impact**: Container startup failures, service discovery issues

### 2. Distributed Stack Files

**Current Structure**:
```
deployment/stacks/
├── traefik/              # Reverse proxy
├── postgresql-production/
├── postgresql-staging/
├── gitea/                # Git server
├── redis/
├── minio/
├── monitoring/
├── registry/
└── semaphore/
```

**Problems**:
- No clear dependency graph between stacks
- Unclear startup order
- Volume mounts across stacks
- Network configuration scattered

### 3. Ansible Deployment Confusion

**Ansible Usage**:
- Server provisioning (install-docker.yml)
- Application deployment (sync-application-code.yml)
- Container recreation (recreate-containers-with-env.yml)
- Stack synchronization (sync-stacks.yml)

**Problem**: Ansible used for BOTH provisioning AND deployment
- Should only provision servers
- Deployment should be via CI/CD
- Creates unclear responsibilities

### 4. Environment-Specific Issues

**Environments Identified**:
- `local` - Developer machines (Docker Compose)
- `staging` - Hetzner server (unclear Docker Compose vs Swarm)
- `production` - Hetzner server (unclear Docker Compose vs Swarm)

**Problems**:
- No unified docker-compose files per environment
- Environment variables scattered (.env, secrets, Ansible vars)
- SSL certificates managed differently per environment

### 5. Specific Container Failures

**postgres-production-backup**:
- Container doesn't exist (was in restart loop)
- Volume mounts not accessible: `/scripts/backup-entrypoint.sh`
- Exit code 255 (file not found)
- Restart policy causing loop

**Root Causes**:
- Relative volume paths in docker-compose.yml
- Container running from different working directory
- Stack not properly initialized

### 6. Network Architecture Unclear

**Networks Found**:
- `traefik-public` (external)
- `app-internal` (external, for PostgreSQL)
- `backend`, `cache`, `postgres-production-internal`

**Problems**:
- Which stacks share which networks?
- How do services discover each other?
- Traefik routing configuration scattered

## Architecture Diagram (Current State)

```
┌─────────────────────────────────────────────────────────────┐
│ Server (Docker Compose? Docker Swarm? Unclear)             │
│                                                             │
│  ┌──────────────┐    ┌──────────────┐    ┌──────────────┐ │
│  │   Traefik    │───▶│     App      │───▶│  PostgreSQL  │ │
│  │   Stack      │    │   Stack      │    │    Stack     │ │
│  └──────────────┘    └──────────────┘    └──────────────┘ │
│         │                    │                    │        │
│         │                    │                    │        │
│  ┌──────▼──────┐    ┌───────▼────┐    ┌──────────▼─────┐ │
│  │   Gitea     │    │   Redis    │    │     MinIO      │ │
│  │   Stack     │    │   Stack    │    │    Stack       │ │
│  └─────────────┘    └────────────┘    └────────────────┘ │
│                                                             │
│  Networks: traefik-public, app-internal, backend, cache    │
│  Volumes: Relative paths, absolute paths, mixed           │
│  Secrets: Docker secrets (Swarm), .env files, Ansible vars│
└─────────────────────────────────────────────────────────────┘

    ▲
    │ Deployment via?
    │ - docker-compose up?
    │ - docker stack deploy?
    │ - Ansible playbooks?
    │ UNCLEAR
    │
┌───┴────────────────────────────────────────────────┐
│ Developer Machine / CI/CD (Gitea)                  │
│ - Ansible playbooks in deployment/ansible/         │
│ - Stack files in deployment/stacks/                │
│ - Application code in src/                         │
└─────────────────────────────────────────────────────┘
```

## Decision Rationale: Rebuild vs Repair

### Why Rebuild?

1. **Architectural Clarity**: Current system mixes concepts (Swarm/Compose, provisioning/deployment)
2. **Environment Separation**: Clean separation of local/staging/prod configurations
3. **CI/CD Integration**: Design for Gitea Actions from start
4. **Maintainability**: Single source of truth per environment
5. **Debugging Difficulty**: Current issues are symptoms of architectural problems

### What to Keep?

- ✅ Traefik configuration (reverse proxy setup is solid)
- ✅ PostgreSQL backup scripts (logic is good, just needs proper mounting)
- ✅ SSL certificate generation (Let's Encrypt integration works)
- ✅ Ansible server provisioning playbooks (keep for initial setup)

### What to Redesign?

- ❌ Stack organization (too fragmented)
- ❌ Deployment method (unclear Ansible vs CI/CD)
- ❌ Environment configuration (scattered variables)
- ❌ Volume mount strategy (relative paths causing issues)
- ❌ Network architecture (unclear dependencies)

## Lessons Learned

1. **Consistency is Key**: Choose Docker Compose OR Docker Swarm, not both
2. **Environment Files**: One docker-compose.{env}.yml per environment
3. **Ansible Scope**: Only for server provisioning, NOT deployment
4. **CI/CD First**: Gitea Actions should handle deployment
5. **Volume Paths**: Always use absolute paths or named volumes
6. **Network Clarity**: Explicit network definitions, clear service discovery

## Next Steps

See `deployment/NEW_ARCHITECTURE.md` for the redesigned system.

## Archive Contents

This `deployment/legacy/` directory contains:
- Original stack files (archived)
- Ansible playbooks (reference only)
- This analysis document

**DO NOT USE THESE FILES FOR NEW DEPLOYMENTS**