michaelschiemer/deployment/stacks/minio/README.md

MinIO Object Storage Stack - S3-compatible Object Storage

Overview

MinIO ist ein hochperformanter, S3-kompatibler Object Storage Service für private Cloud- und Edge-Computing-Umgebungen.

Features:

• S3-kompatible API (Port 9000)
• Web-basierte Console für Management (Port 9001)
• SSL via Traefik
• Persistent storage
• Health checks und Monitoring
• Multi-Tenant Bucket Management

Services

• minio-api.michaelschiemer.de - S3-kompatible API Endpoint
• minio.michaelschiemer.de - Web Console (Management UI)

Prerequisites

1. Traefik Stack Running

   cd ../traefik
   docker compose up -d
   

2. DNS Configuration Point these domains to your server IP (94.16.110.151):

   • minio-api.michaelschiemer.de
   • minio.michaelschiemer.de

Configuration

1. Create Environment File

cp .env.example .env

2. Generate MinIO Root Password

openssl rand -base64 32

Update .env:

MINIO_ROOT_PASSWORD=<generated-password>

Important: Change default MINIO_ROOT_USER in production!

3. Adjust Domains (Optional)

Edit .env to customize domains:

MINIO_API_DOMAIN=storage-api.example.com
MINIO_CONSOLE_DOMAIN=storage.example.com

Deployment

Initial Setup

# Ensure Traefik is running
docker network inspect traefik-public

# Start MinIO
docker compose up -d

# Check logs
docker compose logs -f

# Verify health
docker compose ps

Verify Deployment

# Test API endpoint
curl -I https://minio-api.michaelschiemer.de/minio/health/live

# Expected: HTTP/2 200

# Access Console
open https://minio.michaelschiemer.de

Usage

Web Console Access

1. Navigate to: https://minio.michaelschiemer.de
2. Login with:
   • Access Key: Value of MINIO_ROOT_USER
   • Secret Key: Value of MINIO_ROOT_PASSWORD

Create Bucket via Console

1. Login to Console
2. Click "Create Bucket"
3. Enter bucket name (e.g., my-bucket)
4. Configure bucket settings:
   • Versioning
   • Object Locking
   • Quota
   • Retention policies

S3 API Access

Using AWS CLI

# Install AWS CLI (if not installed)
pip install awscli

# Configure MinIO endpoint
aws configure set default.s3.signature_version s3v4
aws configure set default.s3.addressing_style virtual

# Set credentials
export AWS_ACCESS_KEY_ID=minioadmin
export AWS_SECRET_ACCESS_KEY=<MINIO_ROOT_PASSWORD>

# Test connection
aws --endpoint-url https://minio-api.michaelschiemer.de s3 ls

# Create bucket
aws --endpoint-url https://minio-api.michaelschiemer.de s3 mb s3://my-bucket

# Upload file
aws --endpoint-url https://minio-api.michaelschiemer.de s3 cp file.txt s3://my-bucket/

# Download file
aws --endpoint-url https://minio-api.michaelschiemer.de s3 cp s3://my-bucket/file.txt ./

# List objects
aws --endpoint-url https://minio-api.michaelschiemer.de s3 ls s3://my-bucket/

# Delete object
aws --endpoint-url https://minio-api.michaelschiemer.de s3 rm s3://my-bucket/file.txt

# Delete bucket
aws --endpoint-url https://minio-api.michaelschiemer.de s3 rb s3://my-bucket

Using MinIO Client (mc)

# Install MinIO Client
wget https://dl.min.io/client/mc/release/linux-amd64/mc
chmod +x mc
sudo mv mc /usr/local/bin/

# Configure alias
mc alias set minio https://minio-api.michaelschiemer.de minioadmin <MINIO_ROOT_PASSWORD>

# Test connection
mc admin info minio

# List buckets
mc ls minio

# Create bucket
mc mb minio/my-bucket

# Upload file
mc cp file.txt minio/my-bucket/

# Download file
mc cp minio/my-bucket/file.txt ./

# List objects
mc ls minio/my-bucket/

# Remove object
mc rm minio/my-bucket/file.txt

# Remove bucket
mc rb minio/my-bucket

Using cURL

# List buckets
curl -X GET https://minio-api.michaelschiemer.de \
  -u "minioadmin:<MINIO_ROOT_PASSWORD>"

# Upload file (using presigned URL from Console or SDK)

Programmatic Access

PHP (Using AWS SDK)

use Aws\S3\S3Client;
use Aws\Exception\AwsException;

$s3Client = new S3Client([
    'version' => 'latest',
    'region'  => 'us-east-1',
    'endpoint' => 'https://minio-api.michaelschiemer.de',
    'use_path_style_endpoint' => true,
    'credentials' => [
        'key'    => 'minioadmin',
        'secret' => '<MINIO_ROOT_PASSWORD>',
    ],
]);

// Create bucket
$s3Client->createBucket(['Bucket' => 'my-bucket']);

// Upload file
$s3Client->putObject([
    'Bucket' => 'my-bucket',
    'Key'    => 'file.txt',
    'Body'   => fopen('/path/to/file.txt', 'r'),
]);

// Download file
$result = $s3Client->getObject([
    'Bucket' => 'my-bucket',
    'Key'    => 'file.txt',
]);
echo $result['Body'];

JavaScript/Node.js

const AWS = require('aws-sdk');
const s3 = new AWS.S3({
  endpoint: 'https://minio-api.michaelschiemer.de',
  accessKeyId: 'minioadmin',
  secretAccessKey: '<MINIO_ROOT_PASSWORD>',
  s3ForcePathStyle: true,
  signatureVersion: 'v4',
});

// Create bucket
s3.createBucket({ Bucket: 'my-bucket' }, (err, data) => {
  if (err) console.error(err);
  else console.log('Bucket created');
});

// Upload file
const params = {
  Bucket: 'my-bucket',
  Key: 'file.txt',
  Body: require('fs').createReadStream('/path/to/file.txt'),
};
s3.upload(params, (err, data) => {
  if (err) console.error(err);
  else console.log('File uploaded:', data.Location);
});

Python (boto3)

import boto3
from botocore.client import Config

s3_client = boto3.client(
    's3',
    endpoint_url='https://minio-api.michaelschiemer.de',
    aws_access_key_id='minioadmin',
    aws_secret_access_key='<MINIO_ROOT_PASSWORD>',
    config=Config(signature_version='s3v4'),
    region_name='us-east-1'
)

# Create bucket
s3_client.create_bucket(Bucket='my-bucket')

# Upload file
s3_client.upload_file('/path/to/file.txt', 'my-bucket', 'file.txt')

# Download file
s3_client.download_file('my-bucket', 'file.txt', '/path/to/downloaded.txt')

# List objects
response = s3_client.list_objects_v2(Bucket='my-bucket')
for obj in response.get('Contents', []):
    print(obj['Key'])

User Management

Create Access Keys via Console

1. Login to Console: https://minio.michaelschiemer.de
2. Navigate to "Access Keys" → "Create Access Key"
3. Assign policies (read-only, read-write, admin)
4. Save Access Key and Secret Key (only shown once!)

Create Access Keys via mc CLI

# Create new user
mc admin user add minio myuser mypassword

# Create access key for user
mc admin user svcacct add minio myuser --name my-access-key

# Output shows Access Key and Secret Key

Policy Management

# List policies
mc admin policy list minio

# Create custom policy
mc admin policy add minio readwrite-policy /path/to/policy.json

# Assign policy to user
mc admin policy set minio readwrite-policy user=myuser

# Remove policy from user
mc admin policy remove minio readwrite-policy user=myuser

Example Policy (policy.json):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject"
      ],
      "Resource": ["arn:aws:s3:::my-bucket/*"]
    },
    {
      "Effect": "Allow",
      "Action": ["s3:ListBucket"],
      "Resource": ["arn:aws:s3:::my-bucket"]
    }
  ]
}

Integration with Application Stack

Environment Variables

Add to application/.env:

# MinIO Object Storage
MINIO_ENDPOINT=https://minio-api.michaelschiemer.de
MINIO_ACCESS_KEY=<access-key>
MINIO_SECRET_KEY=<secret-key>
MINIO_BUCKET=<bucket-name>
MINIO_USE_SSL=true
MINIO_REGION=us-east-1

PHP Integration

// Use AWS SDK or MinIO PHP SDK
use Aws\S3\S3Client;

$s3Client = new S3Client([
    'version' => 'latest',
    'region'  => $_ENV['MINIO_REGION'],
    'endpoint' => $_ENV['MINIO_ENDPOINT'],
    'use_path_style_endpoint' => true,
    'credentials' => [
        'key'    => $_ENV['MINIO_ACCESS_KEY'],
        'secret' => $_ENV['MINIO_SECRET_KEY'],
    ],
]);

Backup & Recovery

Manual Backup

#!/bin/bash
# backup-minio.sh

BACKUP_DIR="/backups/minio"
DATE=$(date +%Y%m%d_%H%M%S)

mkdir -p $BACKUP_DIR

# Backup MinIO data
docker run --rm \
  -v minio-data:/data \
  -v $BACKUP_DIR:/backup \
  alpine tar czf /backup/minio-data-$DATE.tar.gz -C /data .

echo "Backup completed: $BACKUP_DIR/minio-data-$DATE.tar.gz"

Restore from Backup

# Stop MinIO
docker compose down

# Restore data
docker run --rm \
  -v minio-data:/data \
  -v /backups/minio:/backup \
  alpine tar xzf /backup/minio-data-YYYYMMDD_HHMMSS.tar.gz -C /data

# Start MinIO
docker compose up -d

Automated Backups

Add to crontab:

# Daily backup at 3 AM
0 3 * * * /path/to/backup-minio.sh

# Keep only last 30 days
0 4 * * * find /backups/minio -type f -mtime +30 -delete

Bucket-Level Backup (Recommended)

Use MinIO's built-in replication or external tools:

# Sync bucket to external storage
mc mirror minio/my-bucket /backup/my-bucket/

# Or sync to another MinIO instance
mc mirror minio/my-bucket remote-minio/my-bucket/

Monitoring

Health Checks

# Check MinIO health
docker compose ps

# API health endpoint
curl -f https://minio-api.michaelschiemer.de/minio/health/live

# Check storage usage
docker exec minio du -sh /data

Logs

# View logs
docker compose logs -f

# Check for errors
docker compose logs minio | grep -i error

# Monitor API access
docker compose logs -f minio | grep "GET /"

Storage Statistics

# Check volume size
docker volume inspect minio-data

# Check disk usage
docker system df -v | grep minio

# List buckets via mc
mc ls minio

Metrics (via mc)

# Server info
mc admin info minio

# Service status
mc admin service status minio

# Trace operations
mc admin trace minio

Performance Tuning

MinIO Configuration

For high-traffic scenarios, edit docker-compose.yml:

environment:
  # Increase concurrent operations
  - MINIO_CACHE_DRIVES=/mnt/cache1,/mnt/cache2
  - MINIO_CACHE_QUOTA=80
  - MINIO_CACHE_AFTER=0
  - MINIO_CACHE_WATERMARK_LOW=70
  - MINIO_CACHE_WATERMARK_HIGH=90

Storage Optimization

# Monitor storage growth
du -sh /var/lib/docker/volumes/minio-data/

# Enable compression (handled automatically by MinIO)

# Set bucket quotas via Console or mc
mc admin quota set minio/my-bucket --hard 100GB

Troubleshooting

Cannot Access Console

# Check service is running
docker compose ps

# Check Traefik routing
docker exec traefik cat /etc/traefik/traefik.yml

# Check network
docker network inspect traefik-public | grep minio

# Test from server
curl -k https://localhost:9001

Authentication Failed

# Verify environment variables
docker exec minio env | grep MINIO_ROOT

# Check logs
docker compose logs minio | grep -i auth

# Reset root password (stop container, remove volume, restart)

SSL Certificate Issues

# Verify Traefik certificate
docker exec traefik cat /acme.json | grep minio

# Test SSL
openssl s_client -connect minio-api.michaelschiemer.de:443 \
  -servername minio-api.michaelschiemer.de < /dev/null

Storage Issues

# Check volume mount
docker exec minio df -h /data

# Check for corrupted data
docker exec minio find /data -type f -name "*.json" | head

# Check disk space
df -h /var/lib/docker/volumes/minio-data/

API Connection Errors

# Verify endpoint URL
curl -I https://minio-api.michaelschiemer.de/minio/health/live

# Test with credentials
curl -u minioadmin:<password> https://minio-api.michaelschiemer.de

# Check CORS settings (if needed for web apps)
mc admin config set minio api cors_allow_origin "https://yourdomain.com"

Security

Security Best Practices

1. Strong Credentials: Use strong passwords for root user and access keys
2. Change Default Root User: Don't use minioadmin in production
3. SSL Only: Always use HTTPS (enforced via Traefik)
4. Access Key Rotation: Regularly rotate access keys
5. Policy-Based Access: Use IAM policies to limit permissions
6. Bucket Policies: Configure bucket-level policies
7. Audit Logging: Enable audit logging for compliance
8. Encryption: Enable encryption at rest and in transit

Enable Audit Logging

# Configure audit logging
mc admin config set minio audit_webhook \
  endpoint=https://log-service.example.com/webhook \
  auth_token=<token>

Enable Encryption

# Set encryption keys
mc admin config set minio encryption \
  sse-s3 enable \
  kms endpoint=https://kms.example.com \
  kms-key-id=<key-id>

Update Stack

# Pull latest images
docker compose pull

# Recreate containers
docker compose up -d

# Verify
docker compose ps

Security Headers

Security headers are applied via Traefik's default-chain@file middleware:

• HSTS
• Content-Type Nosniff
• XSS Protection
• Frame Deny

Additional Resources

• MinIO Documentation: https://min.io/docs/
• S3 API Compatibility: https://min.io/docs/minio/linux/reference/minio-mc/mc.html
• MinIO Client (mc): https://min.io/docs/minio/linux/reference/minio-mc.html
• MinIO Erasure Coding: https://min.io/docs/minio/linux/operations/concepts/erasure-coding.html
• MinIO Security: https://min.io/docs/minio/linux/operations/security.html