feat: CI/CD pipeline setup complete - Ansible playbooks updated, secrets configured, workflow ready

2025-10-31 01:39:24 +01:00
parent 55c04e4fd0
commit e26eb2aa12
601 changed files with 44184 additions and 32477 deletions
--- a/deployment/stacks/gitea/.env.example
+++ b/deployment/stacks/gitea/.env.example
@@ -0,0 +1,20 @@
+# Gitea Configuration
+# Copy this file to .env and adjust values
+
+# Timezone
+TZ=Europe/Berlin
+
+# Gitea Domain
+GITEA_DOMAIN=git.michaelschiemer.de
+
+# MySQL Configuration
+MYSQL_ROOT_PASSWORD=<generate-strong-password>
+MYSQL_DATABASE=gitea
+MYSQL_USER=gitea
+MYSQL_PASSWORD=<generate-strong-password>
+
+# Redis Configuration
+REDIS_PASSWORD=<generate-strong-password>
+
+# Gitea Settings
+DISABLE_REGISTRATION=true  # Set to false to allow user registration
--- a/deployment/stacks/gitea/README.md
+++ b/deployment/stacks/gitea/README.md
@@ -0,0 +1,452 @@
+# Gitea Stack - Self-Hosted Git Server
+
+## Overview
+
+Gitea acts as the central Git server with integrated CI/CD capabilities through Gitea Actions, handling:
+- Git repository hosting
+- User and organization management
+- Pull requests and code reviews
+- Issue tracking
+- Gitea Actions for CI/CD (runner runs on development machine)
+- API for automation
+
+## Services
+
+- **git.michaelschiemer.de** - Gitea Web Interface
+- **git.michaelschiemer.de:2222** - SSH for Git operations
+- **MySQL 8.0** - Database backend
+- **Redis 7** - Cache, session, and queue storage
+
+## Prerequisites
+
+1. **Traefik Stack Running**
+   ```bash
+   cd ../traefik
+   docker compose up -d
+   ```
+
+2. **DNS Configuration**
+   Point `git.michaelschiemer.de` to your server IP (94.16.110.151)
+
+3. **SSH Port Availability**
+   Ensure port 2222 is open in your firewall for Git SSH operations
+
+## Configuration
+
+### 1. Create Environment File
+
+```bash
+cp .env.example .env
+```
+
+### 2. Generate Strong Passwords
+
+```bash
+# MySQL root password
+openssl rand -base64 32
+
+# MySQL gitea password
+openssl rand -base64 32
+
+# Redis password
+openssl rand -base64 32
+```
+
+Update `.env` with generated passwords:
+```env
+MYSQL_ROOT_PASSWORD=<generated-password-1>
+MYSQL_PASSWORD=<generated-password-2>
+REDIS_PASSWORD=<generated-password-3>
+```
+
+### 3. Adjust Configuration (Optional)
+
+Edit `.env` for:
+- Domain customization
+- User registration settings
+- Database configuration
+
+## Deployment
+
+### Initial Setup
+
+```bash
+# Deploy stack
+docker compose up -d
+
+# Check logs
+docker compose logs -f
+
+# Wait for MySQL initialization (30-60 seconds)
+docker compose logs mysql | grep "ready for connections"
+
+# Verify services are healthy
+docker compose ps
+```
+
+### First Time Configuration
+
+1. **Access Gitea**: https://git.michaelschiemer.de
+
+2. **Initial Setup Wizard**:
+   - Database settings are pre-configured via environment variables
+   - Set up admin account:
+     - Username: `admin` (or your preference)
+     - Email: `kontakt@michaelschiemer.de`
+     - Password: Strong password
+   - Server and third-party settings: Use defaults
+   - Click "Install Gitea"
+
+3. **Verify SSH Access**:
+   ```bash
+   # Test SSH connection (replace 'git' with your username after setup)
+   ssh -T -p 2222 git@git.michaelschiemer.de
+   ```
+
+## Usage
+
+### Creating a Repository
+
+1. Log in to https://git.michaelschiemer.de
+2. Click "+" → "New Repository"
+3. Fill in repository details
+4. Clone via HTTPS or SSH:
+   ```bash
+   # HTTPS
+   git clone https://git.michaelschiemer.de/username/repo.git
+
+   # SSH
+   git clone ssh://git@git.michaelschiemer.de:2222/username/repo.git
+   ```
+
+### Gitea Actions
+
+Gitea Actions (GitHub Actions compatible) are enabled by default. To use them:
+
+1. **Create `.gitea/workflows/` directory** in your repository
+2. **Add workflow YAML files** (e.g., `deploy.yml`)
+3. **Register a Runner** (see Runner setup section below)
+
+**Note**: The Gitea Actions Runner should run on your **development machine**, not on the production server. See Stack 9 documentation for runner setup.
+
+### User Management
+
+**Disable Registration** (Default):
+- Set `DISABLE_REGISTRATION=true` in `.env` (already default)
+- Create users via Admin Panel
+
+**Enable Registration**:
+- Set `DISABLE_REGISTRATION=false` in `.env`
+- Restart: `docker compose restart gitea`
+
+### Organizations and Teams
+
+1. Navigate to Organizations
+2. Create organization
+3. Add repositories to organization
+4. Manage teams and permissions
+
+## API Access
+
+Gitea provides a comprehensive API:
+
+```bash
+# Generate API token
+# Settings → Applications → Generate New Token
+
+# Example: List repositories
+curl -H "Authorization: token YOUR_TOKEN" \
+     https://git.michaelschiemer.de/api/v1/user/repos
+```
+
+**API Documentation**: https://git.michaelschiemer.de/api/swagger
+
+## Backup & Recovery
+
+### Manual Backup
+
+```bash
+# Backup script (run on production server)
+#!/bin/bash
+BACKUP_DIR="/backups/gitea"
+DATE=$(date +%Y%m%d_%H%M%S)
+
+# Create backup directory
+mkdir -p $BACKUP_DIR
+
+# Backup Gitea data
+docker run --rm \
+  -v gitea-data:/data \
+  -v $BACKUP_DIR:/backup \
+  alpine tar czf /backup/gitea-data-$DATE.tar.gz -C /data .
+
+# Backup MySQL database
+docker exec gitea-mysql mysqldump \
+  -u root -p$MYSQL_ROOT_PASSWORD \
+  --all-databases \
+  --single-transaction \
+  --quick \
+  --lock-tables=false \
+  > $BACKUP_DIR/gitea-mysql-$DATE.sql
+
+# Backup Redis data
+docker run --rm \
+  -v gitea-redis-data:/data \
+  -v $BACKUP_DIR:/backup \
+  alpine tar czf /backup/gitea-redis-$DATE.tar.gz -C /data .
+
+echo "Backup completed: $BACKUP_DIR/*-$DATE.*"
+```
+
+### Restore from Backup
+
+```bash
+# Stop services
+docker compose down
+
+# Restore Gitea data
+docker run --rm \
+  -v gitea-data:/data \
+  -v /backups/gitea:/backup \
+  alpine tar xzf /backup/gitea-data-YYYYMMDD_HHMMSS.tar.gz -C /data
+
+# Restore MySQL
+cat /backups/gitea/gitea-mysql-YYYYMMDD_HHMMSS.sql | \
+  docker exec -i gitea-mysql mysql -u root -p$MYSQL_ROOT_PASSWORD
+
+# Restore Redis
+docker run --rm \
+  -v gitea-redis-data:/data \
+  -v /backups/gitea:/backup \
+  alpine tar xzf /backup/gitea-redis-YYYYMMDD_HHMMSS.tar.gz -C /data
+
+# Start services
+docker compose up -d
+```
+
+### Automated Backups
+
+Add to crontab on production server:
+
+```bash
+# Daily backup at 2 AM
+0 2 * * * /path/to/backup-gitea.sh
+
+# Keep only last 7 days
+0 3 * * * find /backups/gitea -type f -mtime +7 -delete
+```
+
+## Monitoring
+
+### Health Checks
+
+```bash
+# Check service health
+docker compose ps
+
+# Gitea health endpoint
+curl -f https://git.michaelschiemer.de/api/healthz
+
+# MySQL health
+docker exec gitea-mysql mysqladmin ping -h localhost -u root -p$MYSQL_ROOT_PASSWORD
+
+# Redis health
+docker exec gitea-redis redis-cli -a $REDIS_PASSWORD ping
+```
+
+### Logs
+
+```bash
+# All services
+docker compose logs -f
+
+# Gitea only
+docker compose logs -f gitea
+
+# MySQL only
+docker compose logs -f mysql
+
+# Redis only
+docker compose logs -f redis
+
+# MySQL slow queries
+docker exec gitea-mysql tail -f /var/log/mysql/slow-queries.log
+```
+
+### Resource Usage
+
+```bash
+# Container stats
+docker stats gitea gitea-mysql gitea-redis
+
+# Disk usage
+docker system df -v | grep gitea
+```
+
+## Troubleshooting
+
+### Gitea Not Starting
+
+```bash
+# Check logs
+docker compose logs gitea
+
+# Common issues:
+# 1. MySQL not ready - wait 30-60 seconds
+# 2. Database connection failed - check MYSQL_PASSWORD in .env
+# 3. Redis connection failed - check REDIS_PASSWORD
+```
+
+### SSH Not Working
+
+```bash
+# Verify port 2222 is open
+sudo ufw status | grep 2222
+
+# Open if needed
+sudo ufw allow 2222/tcp
+
+# Test SSH connection
+ssh -T -p 2222 git@git.michaelschiemer.de
+
+# Check Gitea SSH settings
+# Admin Panel → Configuration → Server and Other Services → SSH Server Domain
+```
+
+### Database Connection Issues
+
+```bash
+# Verify MySQL is running and healthy
+docker compose ps mysql
+
+# Test database connection
+docker exec gitea-mysql mysql -u gitea -p$MYSQL_PASSWORD -e "SELECT 1;"
+
+# Check MySQL logs
+docker compose logs mysql | grep -i error
+```
+
+### Redis Connection Issues
+
+```bash
+# Verify Redis is running
+docker compose ps redis
+
+# Test Redis connection
+docker exec gitea-redis redis-cli -a $REDIS_PASSWORD ping
+
+# Check Redis logs
+docker compose logs redis
+```
+
+### Performance Issues
+
+```bash
+# Check MySQL slow queries
+docker exec gitea-mysql tail -100 /var/log/mysql/slow-queries.log
+
+# Analyze MySQL performance
+docker exec gitea-mysql mysql -u root -p$MYSQL_ROOT_PASSWORD \
+  -e "SHOW PROCESSLIST;"
+
+# Check Redis memory usage
+docker exec gitea-redis redis-cli -a $REDIS_PASSWORD INFO memory
+```
+
+### Reset Admin Password
+
+```bash
+# Connect to Gitea container
+docker exec -it gitea bash
+
+# Change admin password
+gitea admin user change-password --username admin --password new-password
+```
+
+## Security
+
+### Security Best Practices
+
+1. **Disable User Registration**: Set `DISABLE_REGISTRATION=true`
+2. **Strong Passwords**: Use generated passwords for all services
+3. **Regular Updates**: Keep Gitea, MySQL, and Redis updated
+4. **SSH Keys**: Prefer SSH keys over HTTPS for Git operations
+5. **2FA**: Enable two-factor authentication for admin accounts
+6. **API Token Security**: Rotate tokens regularly
+7. **Firewall**: Only expose ports 80, 443, and 2222
+
+### Update Stack
+
+```bash
+# Pull latest images
+docker compose pull
+
+# Recreate containers
+docker compose up -d
+
+# Verify
+docker compose ps
+```
+
+### Security Headers
+
+Security headers are applied via Traefik's `default-chain@file` middleware:
+- HSTS
+- Content-Type Nosniff
+- XSS Protection
+- Frame Deny
+- CSP
+
+## Integration with Other Stacks
+
+### Docker Registry (Stack 3)
+
+Gitea Actions can push built images to the private Docker Registry:
+
+```yaml
+# .gitea/workflows/deploy.yml
+- name: Push to Registry
+  run: |
+    docker login registry.michaelschiemer.de -u ${{ secrets.REGISTRY_USER }} -p ${{ secrets.REGISTRY_PASS }}
+    docker push registry.michaelschiemer.de/myapp:latest
+```
+
+### Application Stack (Stack 4)
+
+Deploy applications via Gitea Actions + Ansible:
+
+```yaml
+- name: Deploy to Production
+  run: |
+    ansible-playbook -i inventory/production deploy.yml
+```
+
+## Performance Tuning
+
+### MySQL Optimization
+
+Adjust `mysql/conf.d/gitea.cnf`:
+- `innodb_buffer_pool_size`: Increase for more RAM
+- `max_connections`: Increase for more concurrent users
+- `slow_query_log`: Monitor slow queries
+
+### Redis Optimization
+
+```bash
+# Add to docker-compose.yml redis command:
+# --maxmemory 512mb --maxmemory-policy allkeys-lru
+```
+
+### Gitea Configuration
+
+Edit via Admin Panel → Configuration or `app.ini`:
+- Enable caching for static assets
+- Adjust session timeout
+- Configure queue workers for Actions
+
+## Additional Resources
+
+- **Gitea Documentation**: https://docs.gitea.io/
+- **Gitea Actions**: https://docs.gitea.io/en-us/usage/actions/overview/
+- **API Documentation**: https://git.michaelschiemer.de/api/swagger
+- **MySQL Tuning**: https://dev.mysql.com/doc/refman/8.0/en/optimization.html
--- a/deployment/stacks/gitea/docker-compose.yml
+++ b/deployment/stacks/gitea/docker-compose.yml
@@ -7,7 +7,6 @@ services:
    restart: unless-stopped
    depends_on:
      - postgres
-      - redis
    networks:
      - traefik-public
      - gitea-internal
@@ -15,18 +14,16 @@ services:
      - TZ=Europe/Berlin
      - USER_UID=1000
      - USER_GID=1000
+      - POSTGRES_PASSWORD=gitea_password
      - GITEA__database__DB_TYPE=postgres
      - GITEA__database__HOST=postgres:5432
      - GITEA__database__NAME=${POSTGRES_DB:-gitea}
      - GITEA__database__USER=${POSTGRES_USER:-gitea}
      - GITEA__database__PASSWD=${POSTGRES_PASSWORD:-gitea_password}
-      - GITEA__cache__ENABLED=true
-      - GITEA__cache__ADAPTER=redis
-      - GITEA__cache__HOST=redis://:${REDIS_PASSWORD:-gitea_redis}@redis:6379/0
-      - GITEA__session__PROVIDER=redis
-      - GITEA__session__PROVIDER_CONFIG=redis://:${REDIS_PASSWORD:-gitea_redis}@redis:6379/1
-      - GITEA__queue__TYPE=redis
-      - GITEA__queue__CONN_STR=redis://:${REDIS_PASSWORD:-gitea_redis}@redis:6379/2
+      - GITEA__cache__ENABLED=false
+      - GITEA__cache__ADAPTER=memory
+      - GITEA__session__PROVIDER=file
+      - GITEA__queue__TYPE=channel
      - GITEA__server__DOMAIN=${GITEA_DOMAIN:-git.michaelschiemer.de}
      - GITEA__server__ROOT_URL=https://${GITEA_DOMAIN:-git.michaelschiemer.de}/
      - GITEA__server__SSH_DOMAIN=${GITEA_DOMAIN:-git.michaelschiemer.de}
@@ -43,10 +40,12 @@ services:
      - "traefik.enable=true"

      # HTTP Router
-      - "traefik.http.routers.gitea.rule=Host(`${GITEA_DOMAIN:-git.michaelschiemer.de}`)"
+      - "traefik.http.routers.gitea.rule=Host(`git.michaelschiemer.de`)"
      - "traefik.http.routers.gitea.entrypoints=websecure"
      - "traefik.http.routers.gitea.tls=true"
      - "traefik.http.routers.gitea.tls.certresolver=letsencrypt"
+      # Priority to ensure this router is matched before catchall (catchall has no explicit priority, so default is 0)
+      - "traefik.http.routers.gitea.priority=100"

      # Service
      - "traefik.http.services.gitea.loadbalancer.server.port=3000"
@@ -68,9 +67,9 @@ services:
      - gitea-internal
    environment:
      - TZ=Europe/Berlin
-      - POSTGRES_DB=${POSTGRES_DB:-gitea}
-      - POSTGRES_USER=${POSTGRES_USER:-gitea}
-      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-gitea_password}
+      - POSTGRES_DB=gitea
+      - POSTGRES_USER=gitea
+      - POSTGRES_PASSWORD=gitea_password
    volumes:
      - postgres-data:/var/lib/postgresql/data
    healthcheck:
@@ -80,31 +79,32 @@ services:
      retries: 3
      start_period: 30s

-  redis:
-    image: redis:7-alpine
-    container_name: gitea-redis
-    restart: unless-stopped
-    networks:
-      - gitea-internal
-    environment:
-      - TZ=Europe/Berlin
-    volumes:
-      - redis-data:/data
-    command: redis-server --appendonly yes --requirepass ${REDIS_PASSWORD:-gitea_redis}
-    healthcheck:
-      test: ["CMD", "redis-cli", "-a", "${REDIS_PASSWORD}", "ping"]
-      interval: 30s
-      timeout: 10s
-      retries: 3
-      start_period: 10s
+  # redis (disabled for now; Gitea configured to not use redis)
+  # redis:
+  #   image: redis:7
+  #   container_name: gitea-redis
+  #   restart: unless-stopped
+  #   networks:
+  #     - gitea-internal
+  #   environment:
+  #     - TZ=Europe/Berlin
+  #   volumes:
+  #     - redis-data:/data
+  #   command: redis-server --appendonly yes
+  #   healthcheck:
+  #     test: ["CMD", "redis-cli", "ping"]
+  #     interval: 30s
+  #     timeout: 10s
+  #     retries: 3
+  #     start_period: 10s

 volumes:
  gitea-data:
    name: gitea-data
  postgres-data:
    name: gitea-postgres-data
-  redis-data:
-    name: gitea-redis-data
+  # redis-data:
+  #   name: gitea-redis-data

 networks:
  traefik-public:
--- a/deployment/stacks/gitea/mysql/conf.d/gitea.cnf
+++ b/deployment/stacks/gitea/mysql/conf.d/gitea.cnf
@@ -0,0 +1,33 @@
+[mysqld]
+# Gitea-optimized MySQL configuration
+
+# Character set
+character-set-server = utf8mb4
+collation-server = utf8mb4_unicode_ci
+
+# InnoDB settings
+innodb_buffer_pool_size = 256M
+innodb_log_file_size = 64M
+innodb_flush_log_at_trx_commit = 2
+innodb_flush_method = O_DIRECT
+
+# Connection settings
+max_connections = 200
+max_allowed_packet = 64M
+
+# Query cache (disabled in MySQL 8.0+)
+# Performance schema
+performance_schema = ON
+
+# Logging
+slow_query_log = 1
+slow_query_log_file = /var/log/mysql/slow-queries.log
+long_query_time = 2
+
+# Binary logging for backups
+log_bin = /var/log/mysql/mysql-bin.log
+binlog_expire_logs_seconds = 604800  # 7 days
+max_binlog_size = 100M
+
+[client]
+default-character-set = utf8mb4