chore: Update deployment configuration and documentation

- Update Gitea configuration (remove DEFAULT_ACTIONS_URL) - Fix deployment documentation - Update Ansible playbooks - Clean up deprecated files - Add new deployment scripts and templates
2025-10-31 21:11:11 +01:00
parent cf4748f8db
commit 16d586ecdf
92 changed files with 4601 additions and 10524 deletions
--- a/deployment/stacks/minio/.env.example
+++ b/deployment/stacks/minio/.env.example
@@ -0,0 +1,17 @@
+# MinIO Object Storage Stack Configuration
+# Copy this file to .env and adjust values
+
+# Timezone
+TZ=Europe/Berlin
+
+# MinIO Root Credentials
+# Generate secure password with: openssl rand -base64 32
+MINIO_ROOT_USER=minioadmin
+MINIO_ROOT_PASSWORD=<generate-with-openssl-rand-base64-32>
+
+# Domain Configuration
+# API endpoint (S3-compatible)
+MINIO_API_DOMAIN=minio-api.michaelschiemer.de
+
+# Console endpoint (Web UI)
+MINIO_CONSOLE_DOMAIN=minio.michaelschiemer.de
--- a/deployment/stacks/minio/README.md
+++ b/deployment/stacks/minio/README.md
@@ -0,0 +1,657 @@
+# MinIO Object Storage Stack - S3-compatible Object Storage
+
+## Overview
+
+MinIO ist ein hochperformanter, S3-kompatibler Object Storage Service für private Cloud- und Edge-Computing-Umgebungen.
+
+**Features**:
+- S3-kompatible API (Port 9000)
+- Web-basierte Console für Management (Port 9001)
+- SSL via Traefik
+- Persistent storage
+- Health checks und Monitoring
+- Multi-Tenant Bucket Management
+
+## Services
+
+- **minio-api.michaelschiemer.de** - S3-kompatible API Endpoint
+- **minio.michaelschiemer.de** - Web Console (Management UI)
+
+## Prerequisites
+
+1. **Traefik Stack Running**
+   ```bash
+   cd ../traefik
+   docker compose up -d
+   ```
+
+2. **DNS Configuration**
+   Point these domains to your server IP (94.16.110.151):
+   - `minio-api.michaelschiemer.de`
+   - `minio.michaelschiemer.de`
+
+## Configuration
+
+### 1. Create Environment File
+
+```bash
+cp .env.example .env
+```
+
+### 2. Generate MinIO Root Password
+
+```bash
+openssl rand -base64 32
+```
+
+Update `.env`:
+```env
+MINIO_ROOT_PASSWORD=<generated-password>
+```
+
+**Important**: Change default `MINIO_ROOT_USER` in production!
+
+### 3. Adjust Domains (Optional)
+
+Edit `.env` to customize domains:
+```env
+MINIO_API_DOMAIN=storage-api.example.com
+MINIO_CONSOLE_DOMAIN=storage.example.com
+```
+
+## Deployment
+
+### Initial Setup
+
+```bash
+# Ensure Traefik is running
+docker network inspect traefik-public
+
+# Start MinIO
+docker compose up -d
+
+# Check logs
+docker compose logs -f
+
+# Verify health
+docker compose ps
+```
+
+### Verify Deployment
+
+```bash
+# Test API endpoint
+curl -I https://minio-api.michaelschiemer.de/minio/health/live
+
+# Expected: HTTP/2 200
+
+# Access Console
+open https://minio.michaelschiemer.de
+```
+
+## Usage
+
+### Web Console Access
+
+1. Navigate to: https://minio.michaelschiemer.de
+2. Login with:
+   - **Access Key**: Value of `MINIO_ROOT_USER`
+   - **Secret Key**: Value of `MINIO_ROOT_PASSWORD`
+
+### Create Bucket via Console
+
+1. Login to Console
+2. Click "Create Bucket"
+3. Enter bucket name (e.g., `my-bucket`)
+4. Configure bucket settings:
+   - Versioning
+   - Object Locking
+   - Quota
+   - Retention policies
+
+### S3 API Access
+
+#### Using AWS CLI
+
+```bash
+# Install AWS CLI (if not installed)
+pip install awscli
+
+# Configure MinIO endpoint
+aws configure set default.s3.signature_version s3v4
+aws configure set default.s3.addressing_style virtual
+
+# Set credentials
+export AWS_ACCESS_KEY_ID=minioadmin
+export AWS_SECRET_ACCESS_KEY=<MINIO_ROOT_PASSWORD>
+
+# Test connection
+aws --endpoint-url https://minio-api.michaelschiemer.de s3 ls
+
+# Create bucket
+aws --endpoint-url https://minio-api.michaelschiemer.de s3 mb s3://my-bucket
+
+# Upload file
+aws --endpoint-url https://minio-api.michaelschiemer.de s3 cp file.txt s3://my-bucket/
+
+# Download file
+aws --endpoint-url https://minio-api.michaelschiemer.de s3 cp s3://my-bucket/file.txt ./
+
+# List objects
+aws --endpoint-url https://minio-api.michaelschiemer.de s3 ls s3://my-bucket/
+
+# Delete object
+aws --endpoint-url https://minio-api.michaelschiemer.de s3 rm s3://my-bucket/file.txt
+
+# Delete bucket
+aws --endpoint-url https://minio-api.michaelschiemer.de s3 rb s3://my-bucket
+```
+
+#### Using MinIO Client (mc)
+
+```bash
+# Install MinIO Client
+wget https://dl.min.io/client/mc/release/linux-amd64/mc
+chmod +x mc
+sudo mv mc /usr/local/bin/
+
+# Configure alias
+mc alias set minio https://minio-api.michaelschiemer.de minioadmin <MINIO_ROOT_PASSWORD>
+
+# Test connection
+mc admin info minio
+
+# List buckets
+mc ls minio
+
+# Create bucket
+mc mb minio/my-bucket
+
+# Upload file
+mc cp file.txt minio/my-bucket/
+
+# Download file
+mc cp minio/my-bucket/file.txt ./
+
+# List objects
+mc ls minio/my-bucket/
+
+# Remove object
+mc rm minio/my-bucket/file.txt
+
+# Remove bucket
+mc rb minio/my-bucket
+```
+
+#### Using cURL
+
+```bash
+# List buckets
+curl -X GET https://minio-api.michaelschiemer.de \
+  -u "minioadmin:<MINIO_ROOT_PASSWORD>"
+
+# Upload file (using presigned URL from Console or SDK)
+```
+
+### Programmatic Access
+
+#### PHP (Using AWS SDK)
+
+```php
+use Aws\S3\S3Client;
+use Aws\Exception\AwsException;
+
+$s3Client = new S3Client([
+    'version' => 'latest',
+    'region'  => 'us-east-1',
+    'endpoint' => 'https://minio-api.michaelschiemer.de',
+    'use_path_style_endpoint' => true,
+    'credentials' => [
+        'key'    => 'minioadmin',
+        'secret' => '<MINIO_ROOT_PASSWORD>',
+    ],
+]);
+
+// Create bucket
+$s3Client->createBucket(['Bucket' => 'my-bucket']);
+
+// Upload file
+$s3Client->putObject([
+    'Bucket' => 'my-bucket',
+    'Key'    => 'file.txt',
+    'Body'   => fopen('/path/to/file.txt', 'r'),
+]);
+
+// Download file
+$result = $s3Client->getObject([
+    'Bucket' => 'my-bucket',
+    'Key'    => 'file.txt',
+]);
+echo $result['Body'];
+```
+
+#### JavaScript/Node.js
+
+```javascript
+const AWS = require('aws-sdk');
+const s3 = new AWS.S3({
+  endpoint: 'https://minio-api.michaelschiemer.de',
+  accessKeyId: 'minioadmin',
+  secretAccessKey: '<MINIO_ROOT_PASSWORD>',
+  s3ForcePathStyle: true,
+  signatureVersion: 'v4',
+});
+
+// Create bucket
+s3.createBucket({ Bucket: 'my-bucket' }, (err, data) => {
+  if (err) console.error(err);
+  else console.log('Bucket created');
+});
+
+// Upload file
+const params = {
+  Bucket: 'my-bucket',
+  Key: 'file.txt',
+  Body: require('fs').createReadStream('/path/to/file.txt'),
+};
+s3.upload(params, (err, data) => {
+  if (err) console.error(err);
+  else console.log('File uploaded:', data.Location);
+});
+```
+
+#### Python (boto3)
+
+```python
+import boto3
+from botocore.client import Config
+
+s3_client = boto3.client(
+    's3',
+    endpoint_url='https://minio-api.michaelschiemer.de',
+    aws_access_key_id='minioadmin',
+    aws_secret_access_key='<MINIO_ROOT_PASSWORD>',
+    config=Config(signature_version='s3v4'),
+    region_name='us-east-1'
+)
+
+# Create bucket
+s3_client.create_bucket(Bucket='my-bucket')
+
+# Upload file
+s3_client.upload_file('/path/to/file.txt', 'my-bucket', 'file.txt')
+
+# Download file
+s3_client.download_file('my-bucket', 'file.txt', '/path/to/downloaded.txt')
+
+# List objects
+response = s3_client.list_objects_v2(Bucket='my-bucket')
+for obj in response.get('Contents', []):
+    print(obj['Key'])
+```
+
+## User Management
+
+### Create Access Keys via Console
+
+1. Login to Console: https://minio.michaelschiemer.de
+2. Navigate to "Access Keys" → "Create Access Key"
+3. Assign policies (read-only, read-write, admin)
+4. Save Access Key and Secret Key (only shown once!)
+
+### Create Access Keys via mc CLI
+
+```bash
+# Create new user
+mc admin user add minio myuser mypassword
+
+# Create access key for user
+mc admin user svcacct add minio myuser --name my-access-key
+
+# Output shows Access Key and Secret Key
+```
+
+### Policy Management
+
+```bash
+# List policies
+mc admin policy list minio
+
+# Create custom policy
+mc admin policy add minio readwrite-policy /path/to/policy.json
+
+# Assign policy to user
+mc admin policy set minio readwrite-policy user=myuser
+
+# Remove policy from user
+mc admin policy remove minio readwrite-policy user=myuser
+```
+
+**Example Policy** (`policy.json`):
+```json
+{
+  "Version": "2012-10-17",
+  "Statement": [
+    {
+      "Effect": "Allow",
+      "Action": [
+        "s3:GetObject",
+        "s3:PutObject",
+        "s3:DeleteObject"
+      ],
+      "Resource": ["arn:aws:s3:::my-bucket/*"]
+    },
+    {
+      "Effect": "Allow",
+      "Action": ["s3:ListBucket"],
+      "Resource": ["arn:aws:s3:::my-bucket"]
+    }
+  ]
+}
+```
+
+## Integration with Application Stack
+
+### Environment Variables
+
+Add to `application/.env`:
+
+```env
+# MinIO Object Storage
+MINIO_ENDPOINT=https://minio-api.michaelschiemer.de
+MINIO_ACCESS_KEY=<access-key>
+MINIO_SECRET_KEY=<secret-key>
+MINIO_BUCKET=<bucket-name>
+MINIO_USE_SSL=true
+MINIO_REGION=us-east-1
+```
+
+### PHP Integration
+
+```php
+// Use AWS SDK or MinIO PHP SDK
+use Aws\S3\S3Client;
+
+$s3Client = new S3Client([
+    'version' => 'latest',
+    'region'  => $_ENV['MINIO_REGION'],
+    'endpoint' => $_ENV['MINIO_ENDPOINT'],
+    'use_path_style_endpoint' => true,
+    'credentials' => [
+        'key'    => $_ENV['MINIO_ACCESS_KEY'],
+        'secret' => $_ENV['MINIO_SECRET_KEY'],
+    ],
+]);
+```
+
+## Backup & Recovery
+
+### Manual Backup
+
+```bash
+#!/bin/bash
+# backup-minio.sh
+
+BACKUP_DIR="/backups/minio"
+DATE=$(date +%Y%m%d_%H%M%S)
+
+mkdir -p $BACKUP_DIR
+
+# Backup MinIO data
+docker run --rm \
+  -v minio-data:/data \
+  -v $BACKUP_DIR:/backup \
+  alpine tar czf /backup/minio-data-$DATE.tar.gz -C /data .
+
+echo "Backup completed: $BACKUP_DIR/minio-data-$DATE.tar.gz"
+```
+
+### Restore from Backup
+
+```bash
+# Stop MinIO
+docker compose down
+
+# Restore data
+docker run --rm \
+  -v minio-data:/data \
+  -v /backups/minio:/backup \
+  alpine tar xzf /backup/minio-data-YYYYMMDD_HHMMSS.tar.gz -C /data
+
+# Start MinIO
+docker compose up -d
+```
+
+### Automated Backups
+
+Add to crontab:
+
+```bash
+# Daily backup at 3 AM
+0 3 * * * /path/to/backup-minio.sh
+
+# Keep only last 30 days
+0 4 * * * find /backups/minio -type f -mtime +30 -delete
+```
+
+### Bucket-Level Backup (Recommended)
+
+Use MinIO's built-in replication or external tools:
+
+```bash
+# Sync bucket to external storage
+mc mirror minio/my-bucket /backup/my-bucket/
+
+# Or sync to another MinIO instance
+mc mirror minio/my-bucket remote-minio/my-bucket/
+```
+
+## Monitoring
+
+### Health Checks
+
+```bash
+# Check MinIO health
+docker compose ps
+
+# API health endpoint
+curl -f https://minio-api.michaelschiemer.de/minio/health/live
+
+# Check storage usage
+docker exec minio du -sh /data
+```
+
+### Logs
+
+```bash
+# View logs
+docker compose logs -f
+
+# Check for errors
+docker compose logs minio | grep -i error
+
+# Monitor API access
+docker compose logs -f minio | grep "GET /"
+```
+
+### Storage Statistics
+
+```bash
+# Check volume size
+docker volume inspect minio-data
+
+# Check disk usage
+docker system df -v | grep minio
+
+# List buckets via mc
+mc ls minio
+```
+
+### Metrics (via mc)
+
+```bash
+# Server info
+mc admin info minio
+
+# Service status
+mc admin service status minio
+
+# Trace operations
+mc admin trace minio
+```
+
+## Performance Tuning
+
+### MinIO Configuration
+
+For high-traffic scenarios, edit `docker-compose.yml`:
+
+```yaml
+environment:
+  # Increase concurrent operations
+  - MINIO_CACHE_DRIVES=/mnt/cache1,/mnt/cache2
+  - MINIO_CACHE_QUOTA=80
+  - MINIO_CACHE_AFTER=0
+  - MINIO_CACHE_WATERMARK_LOW=70
+  - MINIO_CACHE_WATERMARK_HIGH=90
+```
+
+### Storage Optimization
+
+```bash
+# Monitor storage growth
+du -sh /var/lib/docker/volumes/minio-data/
+
+# Enable compression (handled automatically by MinIO)
+
+# Set bucket quotas via Console or mc
+mc admin quota set minio/my-bucket --hard 100GB
+```
+
+## Troubleshooting
+
+### Cannot Access Console
+
+```bash
+# Check service is running
+docker compose ps
+
+# Check Traefik routing
+docker exec traefik cat /etc/traefik/traefik.yml
+
+# Check network
+docker network inspect traefik-public | grep minio
+
+# Test from server
+curl -k https://localhost:9001
+```
+
+### Authentication Failed
+
+```bash
+# Verify environment variables
+docker exec minio env | grep MINIO_ROOT
+
+# Check logs
+docker compose logs minio | grep -i auth
+
+# Reset root password (stop container, remove volume, restart)
+```
+
+### SSL Certificate Issues
+
+```bash
+# Verify Traefik certificate
+docker exec traefik cat /acme.json | grep minio
+
+# Test SSL
+openssl s_client -connect minio-api.michaelschiemer.de:443 \
+  -servername minio-api.michaelschiemer.de < /dev/null
+```
+
+### Storage Issues
+
+```bash
+# Check volume mount
+docker exec minio df -h /data
+
+# Check for corrupted data
+docker exec minio find /data -type f -name "*.json" | head
+
+# Check disk space
+df -h /var/lib/docker/volumes/minio-data/
+```
+
+### API Connection Errors
+
+```bash
+# Verify endpoint URL
+curl -I https://minio-api.michaelschiemer.de/minio/health/live
+
+# Test with credentials
+curl -u minioadmin:<password> https://minio-api.michaelschiemer.de
+
+# Check CORS settings (if needed for web apps)
+mc admin config set minio api cors_allow_origin "https://yourdomain.com"
+```
+
+## Security
+
+### Security Best Practices
+
+1. **Strong Credentials**: Use strong passwords for root user and access keys
+2. **Change Default Root User**: Don't use `minioadmin` in production
+3. **SSL Only**: Always use HTTPS (enforced via Traefik)
+4. **Access Key Rotation**: Regularly rotate access keys
+5. **Policy-Based Access**: Use IAM policies to limit permissions
+6. **Bucket Policies**: Configure bucket-level policies
+7. **Audit Logging**: Enable audit logging for compliance
+8. **Encryption**: Enable encryption at rest and in transit
+
+### Enable Audit Logging
+
+```bash
+# Configure audit logging
+mc admin config set minio audit_webhook \
+  endpoint=https://log-service.example.com/webhook \
+  auth_token=<token>
+```
+
+### Enable Encryption
+
+```bash
+# Set encryption keys
+mc admin config set minio encryption \
+  sse-s3 enable \
+  kms endpoint=https://kms.example.com \
+  kms-key-id=<key-id>
+```
+
+### Update Stack
+
+```bash
+# Pull latest images
+docker compose pull
+
+# Recreate containers
+docker compose up -d
+
+# Verify
+docker compose ps
+```
+
+### Security Headers
+
+Security headers are applied via Traefik's `default-chain@file` middleware:
+- HSTS
+- Content-Type Nosniff
+- XSS Protection
+- Frame Deny
+
+## Additional Resources
+
+- **MinIO Documentation**: https://min.io/docs/
+- **S3 API Compatibility**: https://min.io/docs/minio/linux/reference/minio-mc/mc.html
+- **MinIO Client (mc)**: https://min.io/docs/minio/linux/reference/minio-mc.html
+- **MinIO Erasure Coding**: https://min.io/docs/minio/linux/operations/concepts/erasure-coding.html
+- **MinIO Security**: https://min.io/docs/minio/linux/operations/security.html
--- a/deployment/stacks/minio/docker-compose.yml
+++ b/deployment/stacks/minio/docker-compose.yml
@@ -0,0 +1,50 @@
+services:
+  minio:
+    image: minio/minio:latest
+    container_name: minio
+    restart: unless-stopped
+    networks:
+      - traefik-public
+    environment:
+      - TZ=Europe/Berlin
+      - MINIO_ROOT_USER=${MINIO_ROOT_USER:-minioadmin}
+      - MINIO_ROOT_PASSWORD=${MINIO_ROOT_PASSWORD}
+    command: server /data --console-address ":9001"
+    volumes:
+      - minio-data:/data
+      - /etc/timezone:/etc/timezone:ro
+      - /etc/localtime:/etc/localtime:ro
+    healthcheck:
+      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      start_period: 10s
+    labels:
+      - "traefik.enable=true"
+      
+      # API Router (S3-compatible endpoint)
+      - "traefik.http.routers.minio-api.rule=Host(`${MINIO_API_DOMAIN:-minio-api.michaelschiemer.de}`)"
+      - "traefik.http.routers.minio-api.entrypoints=websecure"
+      - "traefik.http.routers.minio-api.tls=true"
+      - "traefik.http.routers.minio-api.tls.certresolver=letsencrypt"
+      - "traefik.http.routers.minio-api.service=minio-api"
+      - "traefik.http.routers.minio-api.middlewares=default-chain@file"
+      - "traefik.http.services.minio-api.loadbalancer.server.port=9000"
+      
+      # Console Router (Web UI)
+      - "traefik.http.routers.minio-console.rule=Host(`${MINIO_CONSOLE_DOMAIN:-minio.michaelschiemer.de}`)"
+      - "traefik.http.routers.minio-console.entrypoints=websecure"
+      - "traefik.http.routers.minio-console.tls=true"
+      - "traefik.http.routers.minio-console.tls.certresolver=letsencrypt"
+      - "traefik.http.routers.minio-console.service=minio-console"
+      - "traefik.http.routers.minio-console.middlewares=default-chain@file"
+      - "traefik.http.services.minio-console.loadbalancer.server.port=9001"
+
+volumes:
+  minio-data:
+    name: minio-data
+
+networks:
+  traefik-public:
+    external: true