michaelschiemer/deployment/stacks/minio/README.md

# MinIO Object Storage Stack - S3-compatible Object Storage

## Overview

MinIO ist ein hochperformanter, S3-kompatibler Object Storage Service für private Cloud- und Edge-Computing-Umgebungen.

**Features**:
- S3-kompatible API (Port 9000)
- Web-basierte Console für Management (Port 9001)
- SSL via Traefik
- Persistent storage
- Health checks und Monitoring
- Multi-Tenant Bucket Management

## Services

- **minio-api.michaelschiemer.de** - S3-kompatible API Endpoint
- **minio.michaelschiemer.de** - Web Console (Management UI)

## Prerequisites

1. **Traefik Stack Running**
   ```bash
   cd ../traefik
   docker compose up -d
   ```

2. **DNS Configuration**
   Point these domains to your server IP (94.16.110.151):
   - `minio-api.michaelschiemer.de`
   - `minio.michaelschiemer.de`

## Configuration

### 1. Create Environment File

```bash
cp .env.example .env
```

### 2. Generate MinIO Root Password

```bash
openssl rand -base64 32
```

Update `.env`:
```env
MINIO_ROOT_PASSWORD=<generated-password>
```

**Important**: Change default `MINIO_ROOT_USER` in production!

### 3. Adjust Domains (Optional)

Edit `.env` to customize domains:
```env
MINIO_API_DOMAIN=storage-api.example.com
MINIO_CONSOLE_DOMAIN=storage.example.com
```

## Deployment

### Initial Setup

```bash
# Ensure Traefik is running
docker network inspect traefik-public

# Start MinIO
docker compose up -d

# Check logs
docker compose logs -f

# Verify health
docker compose ps
```

### Verify Deployment

```bash
# Test API endpoint
curl -I https://minio-api.michaelschiemer.de/minio/health/live

# Expected: HTTP/2 200

# Access Console
open https://minio.michaelschiemer.de
```

## Usage

### Web Console Access

1. Navigate to: https://minio.michaelschiemer.de
2. Login with:
   - **Access Key**: Value of `MINIO_ROOT_USER`
   - **Secret Key**: Value of `MINIO_ROOT_PASSWORD`

### Create Bucket via Console

1. Login to Console
2. Click "Create Bucket"
3. Enter bucket name (e.g., `my-bucket`)
4. Configure bucket settings:
   - Versioning
   - Object Locking
   - Quota
   - Retention policies

### S3 API Access

#### Using AWS CLI

```bash
# Install AWS CLI (if not installed)
pip install awscli

# Configure MinIO endpoint
aws configure set default.s3.signature_version s3v4
aws configure set default.s3.addressing_style virtual

# Set credentials
export AWS_ACCESS_KEY_ID=minioadmin
export AWS_SECRET_ACCESS_KEY=<MINIO_ROOT_PASSWORD>

# Test connection
aws --endpoint-url https://minio-api.michaelschiemer.de s3 ls

# Create bucket
aws --endpoint-url https://minio-api.michaelschiemer.de s3 mb s3://my-bucket

# Upload file
aws --endpoint-url https://minio-api.michaelschiemer.de s3 cp file.txt s3://my-bucket/

# Download file
aws --endpoint-url https://minio-api.michaelschiemer.de s3 cp s3://my-bucket/file.txt ./

# List objects
aws --endpoint-url https://minio-api.michaelschiemer.de s3 ls s3://my-bucket/

# Delete object
aws --endpoint-url https://minio-api.michaelschiemer.de s3 rm s3://my-bucket/file.txt

# Delete bucket
aws --endpoint-url https://minio-api.michaelschiemer.de s3 rb s3://my-bucket
```

#### Using MinIO Client (mc)

```bash
# Install MinIO Client
wget https://dl.min.io/client/mc/release/linux-amd64/mc
chmod +x mc
sudo mv mc /usr/local/bin/

# Configure alias
mc alias set minio https://minio-api.michaelschiemer.de minioadmin <MINIO_ROOT_PASSWORD>

# Test connection
mc admin info minio

# List buckets
mc ls minio

# Create bucket
mc mb minio/my-bucket

# Upload file
mc cp file.txt minio/my-bucket/

# Download file
mc cp minio/my-bucket/file.txt ./

# List objects
mc ls minio/my-bucket/

# Remove object
mc rm minio/my-bucket/file.txt

# Remove bucket
mc rb minio/my-bucket
```

#### Using cURL

```bash
# List buckets
curl -X GET https://minio-api.michaelschiemer.de \
  -u "minioadmin:<MINIO_ROOT_PASSWORD>"

# Upload file (using presigned URL from Console or SDK)
```

### Programmatic Access

#### PHP (Using AWS SDK)

```php
use Aws\S3\S3Client;
use Aws\Exception\AwsException;

$s3Client = new S3Client([
    'version' => 'latest',
    'region'  => 'us-east-1',
    'endpoint' => 'https://minio-api.michaelschiemer.de',
    'use_path_style_endpoint' => true,
    'credentials' => [
        'key'    => 'minioadmin',
        'secret' => '<MINIO_ROOT_PASSWORD>',
    ],
]);

// Create bucket
$s3Client->createBucket(['Bucket' => 'my-bucket']);

// Upload file
$s3Client->putObject([
    'Bucket' => 'my-bucket',
    'Key'    => 'file.txt',
    'Body'   => fopen('/path/to/file.txt', 'r'),
]);

// Download file
$result = $s3Client->getObject([
    'Bucket' => 'my-bucket',
    'Key'    => 'file.txt',
]);
echo $result['Body'];
```

#### JavaScript/Node.js

```javascript
const AWS = require('aws-sdk');
const s3 = new AWS.S3({
  endpoint: 'https://minio-api.michaelschiemer.de',
  accessKeyId: 'minioadmin',
  secretAccessKey: '<MINIO_ROOT_PASSWORD>',
  s3ForcePathStyle: true,
  signatureVersion: 'v4',
});

// Create bucket
s3.createBucket({ Bucket: 'my-bucket' }, (err, data) => {
  if (err) console.error(err);
  else console.log('Bucket created');
});

// Upload file
const params = {
  Bucket: 'my-bucket',
  Key: 'file.txt',
  Body: require('fs').createReadStream('/path/to/file.txt'),
};
s3.upload(params, (err, data) => {
  if (err) console.error(err);
  else console.log('File uploaded:', data.Location);
});
```

#### Python (boto3)

```python
import boto3
from botocore.client import Config

s3_client = boto3.client(
    's3',
    endpoint_url='https://minio-api.michaelschiemer.de',
    aws_access_key_id='minioadmin',
    aws_secret_access_key='<MINIO_ROOT_PASSWORD>',
    config=Config(signature_version='s3v4'),
    region_name='us-east-1'
)

# Create bucket
s3_client.create_bucket(Bucket='my-bucket')

# Upload file
s3_client.upload_file('/path/to/file.txt', 'my-bucket', 'file.txt')

# Download file
s3_client.download_file('my-bucket', 'file.txt', '/path/to/downloaded.txt')

# List objects
response = s3_client.list_objects_v2(Bucket='my-bucket')
for obj in response.get('Contents', []):
    print(obj['Key'])
```

## User Management

### Create Access Keys via Console

1. Login to Console: https://minio.michaelschiemer.de
2. Navigate to "Access Keys" → "Create Access Key"
3. Assign policies (read-only, read-write, admin)
4. Save Access Key and Secret Key (only shown once!)

### Create Access Keys via mc CLI

```bash
# Create new user
mc admin user add minio myuser mypassword

# Create access key for user
mc admin user svcacct add minio myuser --name my-access-key

# Output shows Access Key and Secret Key
```

### Policy Management

```bash
# List policies
mc admin policy list minio

# Create custom policy
mc admin policy add minio readwrite-policy /path/to/policy.json

# Assign policy to user
mc admin policy set minio readwrite-policy user=myuser

# Remove policy from user
mc admin policy remove minio readwrite-policy user=myuser
```

**Example Policy** (`policy.json`):
```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject"
      ],
      "Resource": ["arn:aws:s3:::my-bucket/*"]
    },
    {
      "Effect": "Allow",
      "Action": ["s3:ListBucket"],
      "Resource": ["arn:aws:s3:::my-bucket"]
    }
  ]
}
```

## Integration with Application Stack

### Environment Variables

Add to `application/.env`:

```env
# MinIO Object Storage
MINIO_ENDPOINT=https://minio-api.michaelschiemer.de
MINIO_ACCESS_KEY=<access-key>
MINIO_SECRET_KEY=<secret-key>
MINIO_BUCKET=<bucket-name>
MINIO_USE_SSL=true
MINIO_REGION=us-east-1
```

### PHP Integration

```php
// Use AWS SDK or MinIO PHP SDK
use Aws\S3\S3Client;

$s3Client = new S3Client([
    'version' => 'latest',
    'region'  => $_ENV['MINIO_REGION'],
    'endpoint' => $_ENV['MINIO_ENDPOINT'],
    'use_path_style_endpoint' => true,
    'credentials' => [
        'key'    => $_ENV['MINIO_ACCESS_KEY'],
        'secret' => $_ENV['MINIO_SECRET_KEY'],
    ],
]);
```

## Backup & Recovery

### Manual Backup

```bash
#!/bin/bash
# backup-minio.sh

BACKUP_DIR="/backups/minio"
DATE=$(date +%Y%m%d_%H%M%S)

mkdir -p $BACKUP_DIR

# Backup MinIO data
docker run --rm \
  -v minio-data:/data \
  -v $BACKUP_DIR:/backup \
  alpine tar czf /backup/minio-data-$DATE.tar.gz -C /data .

echo "Backup completed: $BACKUP_DIR/minio-data-$DATE.tar.gz"
```

### Restore from Backup

```bash
# Stop MinIO
docker compose down

# Restore data
docker run --rm \
  -v minio-data:/data \
  -v /backups/minio:/backup \
  alpine tar xzf /backup/minio-data-YYYYMMDD_HHMMSS.tar.gz -C /data

# Start MinIO
docker compose up -d
```

### Automated Backups

Add to crontab:

```bash
# Daily backup at 3 AM
0 3 * * * /path/to/backup-minio.sh

# Keep only last 30 days
0 4 * * * find /backups/minio -type f -mtime +30 -delete
```

### Bucket-Level Backup (Recommended)

Use MinIO's built-in replication or external tools:

```bash
# Sync bucket to external storage
mc mirror minio/my-bucket /backup/my-bucket/

# Or sync to another MinIO instance
mc mirror minio/my-bucket remote-minio/my-bucket/
```

## Monitoring

### Health Checks

```bash
# Check MinIO health
docker compose ps

# API health endpoint
curl -f https://minio-api.michaelschiemer.de/minio/health/live

# Check storage usage
docker exec minio du -sh /data
```

### Logs

```bash
# View logs
docker compose logs -f

# Check for errors
docker compose logs minio | grep -i error

# Monitor API access
docker compose logs -f minio | grep "GET /"
```

### Storage Statistics

```bash
# Check volume size
docker volume inspect minio-data

# Check disk usage
docker system df -v | grep minio

# List buckets via mc
mc ls minio
```

### Metrics (via mc)

```bash
# Server info
mc admin info minio

# Service status
mc admin service status minio

# Trace operations
mc admin trace minio
```

## Performance Tuning

### MinIO Configuration

For high-traffic scenarios, edit `docker-compose.yml`:

```yaml
environment:
  # Increase concurrent operations
  - MINIO_CACHE_DRIVES=/mnt/cache1,/mnt/cache2
  - MINIO_CACHE_QUOTA=80
  - MINIO_CACHE_AFTER=0
  - MINIO_CACHE_WATERMARK_LOW=70
  - MINIO_CACHE_WATERMARK_HIGH=90
```

### Storage Optimization

```bash
# Monitor storage growth
du -sh /var/lib/docker/volumes/minio-data/

# Enable compression (handled automatically by MinIO)

# Set bucket quotas via Console or mc
mc admin quota set minio/my-bucket --hard 100GB
```

## Troubleshooting

### Cannot Access Console

```bash
# Check service is running
docker compose ps

# Check Traefik routing
docker exec traefik cat /etc/traefik/traefik.yml

# Check network
docker network inspect traefik-public | grep minio

# Test from server
curl -k https://localhost:9001
```

### Authentication Failed

```bash
# Verify environment variables
docker exec minio env | grep MINIO_ROOT

# Check logs
docker compose logs minio | grep -i auth

# Reset root password (stop container, remove volume, restart)
```

### SSL Certificate Issues

```bash
# Verify Traefik certificate
docker exec traefik cat /acme.json | grep minio

# Test SSL
openssl s_client -connect minio-api.michaelschiemer.de:443 \
  -servername minio-api.michaelschiemer.de < /dev/null
```

### Storage Issues

```bash
# Check volume mount
docker exec minio df -h /data

# Check for corrupted data
docker exec minio find /data -type f -name "*.json" | head

# Check disk space
df -h /var/lib/docker/volumes/minio-data/
```

### API Connection Errors

```bash
# Verify endpoint URL
curl -I https://minio-api.michaelschiemer.de/minio/health/live

# Test with credentials
curl -u minioadmin:<password> https://minio-api.michaelschiemer.de

# Check CORS settings (if needed for web apps)
mc admin config set minio api cors_allow_origin "https://yourdomain.com"
```

## Security

### Security Best Practices

1. **Strong Credentials**: Use strong passwords for root user and access keys
2. **Change Default Root User**: Don't use `minioadmin` in production
3. **SSL Only**: Always use HTTPS (enforced via Traefik)
4. **Access Key Rotation**: Regularly rotate access keys
5. **Policy-Based Access**: Use IAM policies to limit permissions
6. **Bucket Policies**: Configure bucket-level policies
7. **Audit Logging**: Enable audit logging for compliance
8. **Encryption**: Enable encryption at rest and in transit

### Enable Audit Logging

```bash
# Configure audit logging
mc admin config set minio audit_webhook \
  endpoint=https://log-service.example.com/webhook \
  auth_token=<token>
```

### Enable Encryption

```bash
# Set encryption keys
mc admin config set minio encryption \
  sse-s3 enable \
  kms endpoint=https://kms.example.com \
  kms-key-id=<key-id>
```

### Update Stack

```bash
# Pull latest images
docker compose pull

# Recreate containers
docker compose up -d

# Verify
docker compose ps
```

### Security Headers

Security headers are applied via Traefik's `default-chain@file` middleware:
- HSTS
- Content-Type Nosniff
- XSS Protection
- Frame Deny

## Additional Resources

- **MinIO Documentation**: https://min.io/docs/
- **S3 API Compatibility**: https://min.io/docs/minio/linux/reference/minio-mc/mc.html
- **MinIO Client (mc)**: https://min.io/docs/minio/linux/reference/minio-mc.html
- **MinIO Erasure Coding**: https://min.io/docs/minio/linux/operations/concepts/erasure-coding.html
- **MinIO Security**: https://min.io/docs/minio/linux/operations/security.html