feat: CI/CD pipeline setup complete - Ansible playbooks updated, secrets configured, workflow ready

2025-10-31 01:39:24 +01:00
parent 55c04e4fd0
commit e26eb2aa12
601 changed files with 44184 additions and 32477 deletions
--- a/deployment/stacks/registry/README.md
+++ b/deployment/stacks/registry/README.md
@@ -0,0 +1,610 @@
+# Docker Registry Stack - Private Container Registry
+
+## Overview
+
+Private Docker Registry mit BasicAuth für sichere Container Image Storage.
+
+**Features**:
+- Private Docker Registry v2.8
+- BasicAuth Authentifizierung
+- SSL via Traefik
+- Automatic garbage collection
+- Image deletion support
+- Persistent storage
+
+## Services
+
+- **registry.michaelschiemer.de** - Docker Registry (BasicAuth protected)
+
+## Prerequisites
+
+1. **Traefik Stack Running**
+   ```bash
+   cd ../traefik
+   docker compose up -d
+   ```
+
+2. **DNS Configuration**
+   Point `registry.michaelschiemer.de` to your server IP (94.16.110.151)
+
+3. **htpasswd Utility**
+   ```bash
+   # Install if not available
+   sudo apt-get install apache2-utils
+   ```
+
+## Configuration
+
+### 1. Create Environment File
+
+```bash
+cp .env.example .env
+```
+
+### 2. Generate Registry HTTP Secret
+
+```bash
+openssl rand -hex 32
+```
+
+Update `.env`:
+```env
+REGISTRY_HTTP_SECRET=<generated-secret>
+```
+
+### 3. Create Registry Users
+
+```bash
+# Create htpasswd file with first user
+htpasswd -Bc auth/htpasswd admin
+
+# Add additional users
+htpasswd -B auth/htpasswd developer
+
+# Verify users
+cat auth/htpasswd
+```
+
+**Important**: Use `-B` (bcrypt) for best security. `-c` creates new file (only for first user).
+
+## Deployment
+
+### Initial Setup
+
+```bash
+# Ensure Traefik is running
+docker network inspect traefik-public
+
+# Create auth directory and users
+mkdir -p auth
+htpasswd -Bc auth/htpasswd admin
+
+# Start registry
+docker compose up -d
+
+# Check logs
+docker compose logs -f
+
+# Verify health
+docker compose ps
+```
+
+### Verify Deployment
+
+```bash
+# Test registry endpoint
+curl https://registry.michaelschiemer.de/v2/
+
+# Expected: Authentication required (401)
+
+# Test with credentials
+curl -u admin:yourpassword https://registry.michaelschiemer.de/v2/_catalog
+
+# Expected: {"repositories":[]}
+```
+
+## Usage
+
+### Docker Login
+
+```bash
+# Login to registry
+docker login registry.michaelschiemer.de
+
+# Enter username and password when prompted
+```
+
+### Push Images
+
+```bash
+# Tag local image for registry
+docker tag myapp:latest registry.michaelschiemer.de/myapp:latest
+
+# Push to registry
+docker push registry.michaelschiemer.de/myapp:latest
+```
+
+### Pull Images
+
+```bash
+# Pull from registry
+docker pull registry.michaelschiemer.de/myapp:latest
+```
+
+### List Images
+
+```bash
+# List all repositories
+curl -u admin:password https://registry.michaelschiemer.de/v2/_catalog
+
+# List tags for repository
+curl -u admin:password https://registry.michaelschiemer.de/v2/myapp/tags/list
+```
+
+### Delete Images
+
+```bash
+# Get image digest
+curl -I -H "Accept: application/vnd.docker.distribution.manifest.v2+json" \
+  -u admin:password \
+  https://registry.michaelschiemer.de/v2/myapp/manifests/latest
+
+# Delete by digest
+curl -X DELETE -u admin:password \
+  https://registry.michaelschiemer.de/v2/myapp/manifests/sha256:...
+
+# Run garbage collection
+docker exec registry bin/registry garbage-collect /etc/docker/registry/config.yml
+```
+
+## Integration with Other Stacks
+
+### Gitea Actions (Stack 2)
+
+Push built images from Gitea Actions:
+
+```yaml
+# .gitea/workflows/build.yml
+name: Build and Push
+
+on:
+  push:
+    branches: [main]
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+
+      - name: Login to Registry
+        run: |
+          echo "${{ secrets.REGISTRY_PASSWORD }}" | \
+          docker login registry.michaelschiemer.de \
+            -u "${{ secrets.REGISTRY_USER }}" \
+            --password-stdin
+
+      - name: Build and Push
+        run: |
+          docker build -t registry.michaelschiemer.de/myapp:${{ github.sha }} .
+          docker push registry.michaelschiemer.de/myapp:${{ github.sha }}
+```
+
+### Application Stack (Stack 4)
+
+Pull images in application deployment:
+
+```yaml
+# In application docker-compose.yml
+services:
+  app:
+    image: registry.michaelschiemer.de/myapp:latest
+    # ... rest of configuration
+```
+
+**Note**: Ensure Docker daemon has registry credentials configured.
+
+## User Management
+
+### Add User
+
+```bash
+# Add new user
+htpasswd -B auth/htpasswd newuser
+
+# Restart registry to apply
+docker compose restart
+```
+
+### Remove User
+
+```bash
+# Edit htpasswd file and remove user line
+nano auth/htpasswd
+
+# Restart registry
+docker compose restart
+```
+
+### Change Password
+
+```bash
+# Update password (removes old entry)
+htpasswd -B auth/htpasswd username
+
+# Restart registry
+docker compose restart
+```
+
+## Backup & Recovery
+
+### Manual Backup
+
+```bash
+#!/bin/bash
+# backup-registry.sh
+
+BACKUP_DIR="/backups/registry"
+DATE=$(date +%Y%m%d_%H%M%S)
+
+mkdir -p $BACKUP_DIR
+
+# Backup registry data
+docker run --rm \
+  -v registry-data:/data \
+  -v $BACKUP_DIR:/backup \
+  alpine tar czf /backup/registry-data-$DATE.tar.gz -C /data .
+
+# Backup auth configuration
+tar czf $BACKUP_DIR/registry-auth-$DATE.tar.gz auth/
+
+echo "Backup completed: $BACKUP_DIR/*-$DATE.tar.gz"
+```
+
+### Restore from Backup
+
+```bash
+# Stop registry
+docker compose down
+
+# Restore registry data
+docker run --rm \
+  -v registry-data:/data \
+  -v /backups/registry:/backup \
+  alpine tar xzf /backup/registry-data-YYYYMMDD_HHMMSS.tar.gz -C /data
+
+# Restore auth
+tar xzf /backups/registry/registry-auth-YYYYMMDD_HHMMSS.tar.gz
+
+# Start registry
+docker compose up -d
+```
+
+### Automated Backups
+
+Add to crontab:
+
+```bash
+# Daily backup at 3 AM
+0 3 * * * /path/to/backup-registry.sh
+
+# Keep only last 14 days
+0 4 * * * find /backups/registry -type f -mtime +14 -delete
+```
+
+## Monitoring
+
+### Health Checks
+
+```bash
+# Check registry health
+docker compose ps
+
+# Registry health endpoint
+curl -f https://registry.michaelschiemer.de/v2/
+
+# Check storage usage
+docker exec registry du -sh /var/lib/registry
+```
+
+### Logs
+
+```bash
+# View logs
+docker compose logs -f
+
+# Check for errors
+docker compose logs registry | grep -i error
+
+# Monitor access logs
+docker compose logs -f registry | grep "GET /v2"
+```
+
+### Storage Statistics
+
+```bash
+# Check volume size
+docker volume inspect registry-data
+
+# Check disk usage
+docker system df -v | grep registry
+
+# List images in registry
+curl -u admin:password https://registry.michaelschiemer.de/v2/_catalog | jq
+```
+
+## Garbage Collection
+
+### Manual Garbage Collection
+
+```bash
+# Run garbage collection
+docker exec registry bin/registry garbage-collect \
+  /etc/docker/registry/config.yml
+
+# With dry-run
+docker exec registry bin/registry garbage-collect \
+  --dry-run \
+  /etc/docker/registry/config.yml
+```
+
+### Scheduled Garbage Collection
+
+Add to crontab (on production server):
+
+```bash
+# Weekly garbage collection (Sunday 2 AM)
+0 2 * * 0 docker exec registry bin/registry garbage-collect /etc/docker/registry/config.yml
+```
+
+**Note**: Automatic upload purging is enabled (168h old uploads cleaned every 24h).
+
+## Troubleshooting
+
+### Authentication Failed
+
+```bash
+# Check htpasswd file exists
+ls -la auth/htpasswd
+
+# Verify htpasswd format (should be bcrypt)
+cat auth/htpasswd
+# Format: username:$2y$...
+
+# Test authentication
+curl -u username:password https://registry.michaelschiemer.de/v2/
+
+# Check registry logs
+docker compose logs registry | grep auth
+```
+
+### Cannot Push Images
+
+```bash
+# Verify Docker login
+cat ~/.docker/config.json | grep registry.michaelschiemer.de
+
+# Re-login
+docker logout registry.michaelschiemer.de
+docker login registry.michaelschiemer.de
+
+# Check storage space
+df -h /var/lib/docker
+
+# Check registry logs
+docker compose logs -f registry
+```
+
+### SSL Certificate Issues
+
+```bash
+# Verify Traefik certificate
+docker exec traefik cat /acme.json | grep registry.michaelschiemer.de
+
+# Force certificate renewal (via Traefik)
+# Remove acme.json and restart Traefik
+
+# Test SSL
+openssl s_client -connect registry.michaelschiemer.de:443 -servername registry.michaelschiemer.de < /dev/null
+```
+
+### Registry Not Accessible
+
+```bash
+# Check service is running
+docker compose ps
+
+# Check Traefik routing
+docker exec traefik cat /etc/traefik/traefik.yml
+
+# Check network
+docker network inspect traefik-public | grep registry
+
+# Test from server
+curl -k https://localhost:5000/v2/
+```
+
+### Storage Issues
+
+```bash
+# Check volume mount
+docker exec registry df -h /var/lib/registry
+
+# Check for corrupted layers
+docker exec registry find /var/lib/registry -type f -name "data" | wc -l
+
+# Run garbage collection
+docker exec registry bin/registry garbage-collect /etc/docker/registry/config.yml
+
+# Check for orphaned data
+docker volume prune
+```
+
+## Security
+
+### Security Best Practices
+
+1. **Strong Passwords**: Use bcrypt (htpasswd -B) with strong passwords
+2. **SSL Only**: Always use HTTPS (enforced via Traefik)
+3. **User Management**: Regularly review and rotate credentials
+4. **Access Logging**: Monitor access logs for suspicious activity
+5. **Firewall**: Only expose port 443 (handled by Traefik)
+6. **Backup Encryption**: Encrypt backups containing sensitive data
+7. **Minimal Permissions**: Limit registry access to necessary users
+
+### Update Stack
+
+```bash
+# Pull latest images
+docker compose pull
+
+# Recreate containers
+docker compose up -d
+
+# Verify
+docker compose ps
+```
+
+### Security Headers
+
+Security headers are applied via Traefik's `default-chain@file` middleware:
+- HSTS
+- Content-Type Nosniff
+- XSS Protection
+- Frame Deny
+
+## Docker Daemon Configuration
+
+### Configure Docker to Trust Registry
+
+On machines that will push/pull from registry:
+
+```bash
+# Edit daemon.json
+sudo nano /etc/docker/daemon.json
+```
+
+Add:
+```json
+{
+  "insecure-registries": [],
+  "registry-mirrors": [],
+  "log-driver": "json-file",
+  "log-opts": {
+    "max-size": "10m",
+    "max-file": "3"
+  }
+}
+```
+
+**Note**: No need to add registry to `insecure-registries` since we use SSL.
+
+```bash
+# Restart Docker
+sudo systemctl restart docker
+
+# Verify
+docker info | grep Registry
+```
+
+### Configure Credentials
+
+```bash
+# Login once per machine
+docker login registry.michaelschiemer.de
+
+# Credentials stored in ~/.docker/config.json
+```
+
+## Performance Tuning
+
+### Registry Configuration
+
+For high-traffic registries, edit `docker-compose.yml`:
+
+```yaml
+environment:
+  # Increase concurrent operations
+  - REGISTRY_STORAGE_MAXCONCURRENCY=50
+
+  # Cache settings
+  - REGISTRY_STORAGE_CACHE_BLOBDESCRIPTOR=inmemory
+
+  # Rate limiting
+  - REGISTRY_HTTP_RATELIMIT_REQUESTS_PER_SECOND=100
+```
+
+### Storage Optimization
+
+```bash
+# Enable compression for layers (reduces storage)
+# Already enabled in v2.8
+
+# Monitor storage growth
+du -sh /var/lib/docker/volumes/registry-data/
+
+# Schedule regular garbage collection
+# See "Scheduled Garbage Collection" section
+```
+
+## Migration from Docker Hub
+
+### Pull and Re-push Images
+
+```bash
+#!/bin/bash
+# migrate-images.sh
+
+IMAGES=(
+    "nginx:latest"
+    "node:18-alpine"
+    "postgres:14"
+)
+
+for image in "${IMAGES[@]}"; do
+    echo "Migrating $image..."
+
+    # Pull from Docker Hub
+    docker pull $image
+
+    # Tag for private registry
+    docker tag $image registry.michaelschiemer.de/$image
+
+    # Push to private registry
+    docker push registry.michaelschiemer.de/$image
+
+    echo "✅ Migrated $image"
+done
+```
+
+## API Reference
+
+### Registry API v2
+
+```bash
+# List catalog
+GET /v2/_catalog
+
+# List tags
+GET /v2/<name>/tags/list
+
+# Get manifest
+GET /v2/<name>/manifests/<reference>
+
+# Delete manifest
+DELETE /v2/<name>/manifests/<digest>
+
+# Check blob exists
+HEAD /v2/<name>/blobs/<digest>
+```
+
+**Authentication**: All endpoints require BasicAuth.
+
+**Documentation**: https://docs.docker.com/registry/spec/api/
+
+## Additional Resources
+
+- **Docker Registry Documentation**: https://docs.docker.com/registry/
+- **Registry Configuration**: https://docs.docker.com/registry/configuration/
+- **Storage Drivers**: https://docs.docker.com/registry/storage-drivers/
+- **Token Authentication**: https://docs.docker.com/registry/spec/auth/token/