# MinIO Object Storage Stack - S3-compatible Object Storage ## Overview MinIO ist ein hochperformanter, S3-kompatibler Object Storage Service für private Cloud- und Edge-Computing-Umgebungen. **Features**: - S3-kompatible API (Port 9000) - Web-basierte Console für Management (Port 9001) - SSL via Traefik - Persistent storage - Health checks und Monitoring - Multi-Tenant Bucket Management ## Services - **minio-api.michaelschiemer.de** - S3-kompatible API Endpoint - **minio.michaelschiemer.de** - Web Console (Management UI) ## Prerequisites 1. **Traefik Stack Running** ```bash cd ../traefik docker compose up -d ``` 2. **DNS Configuration** Point these domains to your server IP (94.16.110.151): - `minio-api.michaelschiemer.de` - `minio.michaelschiemer.de` ## Configuration ### 1. Create Environment File ```bash cp .env.example .env ``` ### 2. Generate MinIO Root Password ```bash openssl rand -base64 32 ``` Update `.env`: ```env MINIO_ROOT_PASSWORD= ``` **Important**: Change default `MINIO_ROOT_USER` in production! ### 3. Adjust Domains (Optional) Edit `.env` to customize domains: ```env MINIO_API_DOMAIN=storage-api.example.com MINIO_CONSOLE_DOMAIN=storage.example.com ``` ## Deployment ### Initial Setup ```bash # Ensure Traefik is running docker network inspect traefik-public # Start MinIO docker compose up -d # Check logs docker compose logs -f # Verify health docker compose ps ``` ### Verify Deployment ```bash # Test API endpoint curl -I https://minio-api.michaelschiemer.de/minio/health/live # Expected: HTTP/2 200 # Access Console open https://minio.michaelschiemer.de ``` ## Usage ### Web Console Access 1. Navigate to: https://minio.michaelschiemer.de 2. Login with: - **Access Key**: Value of `MINIO_ROOT_USER` - **Secret Key**: Value of `MINIO_ROOT_PASSWORD` ### Create Bucket via Console 1. Login to Console 2. Click "Create Bucket" 3. Enter bucket name (e.g., `my-bucket`) 4. Configure bucket settings: - Versioning - Object Locking - Quota - Retention policies ### S3 API Access #### Using AWS CLI ```bash # Install AWS CLI (if not installed) pip install awscli # Configure MinIO endpoint aws configure set default.s3.signature_version s3v4 aws configure set default.s3.addressing_style virtual # Set credentials export AWS_ACCESS_KEY_ID=minioadmin export AWS_SECRET_ACCESS_KEY= # Test connection aws --endpoint-url https://minio-api.michaelschiemer.de s3 ls # Create bucket aws --endpoint-url https://minio-api.michaelschiemer.de s3 mb s3://my-bucket # Upload file aws --endpoint-url https://minio-api.michaelschiemer.de s3 cp file.txt s3://my-bucket/ # Download file aws --endpoint-url https://minio-api.michaelschiemer.de s3 cp s3://my-bucket/file.txt ./ # List objects aws --endpoint-url https://minio-api.michaelschiemer.de s3 ls s3://my-bucket/ # Delete object aws --endpoint-url https://minio-api.michaelschiemer.de s3 rm s3://my-bucket/file.txt # Delete bucket aws --endpoint-url https://minio-api.michaelschiemer.de s3 rb s3://my-bucket ``` #### Using MinIO Client (mc) ```bash # Install MinIO Client wget https://dl.min.io/client/mc/release/linux-amd64/mc chmod +x mc sudo mv mc /usr/local/bin/ # Configure alias mc alias set minio https://minio-api.michaelschiemer.de minioadmin # Test connection mc admin info minio # List buckets mc ls minio # Create bucket mc mb minio/my-bucket # Upload file mc cp file.txt minio/my-bucket/ # Download file mc cp minio/my-bucket/file.txt ./ # List objects mc ls minio/my-bucket/ # Remove object mc rm minio/my-bucket/file.txt # Remove bucket mc rb minio/my-bucket ``` #### Using cURL ```bash # List buckets curl -X GET https://minio-api.michaelschiemer.de \ -u "minioadmin:" # Upload file (using presigned URL from Console or SDK) ``` ### Programmatic Access #### PHP (Using AWS SDK) ```php use Aws\S3\S3Client; use Aws\Exception\AwsException; $s3Client = new S3Client([ 'version' => 'latest', 'region' => 'us-east-1', 'endpoint' => 'https://minio-api.michaelschiemer.de', 'use_path_style_endpoint' => true, 'credentials' => [ 'key' => 'minioadmin', 'secret' => '', ], ]); // Create bucket $s3Client->createBucket(['Bucket' => 'my-bucket']); // Upload file $s3Client->putObject([ 'Bucket' => 'my-bucket', 'Key' => 'file.txt', 'Body' => fopen('/path/to/file.txt', 'r'), ]); // Download file $result = $s3Client->getObject([ 'Bucket' => 'my-bucket', 'Key' => 'file.txt', ]); echo $result['Body']; ``` #### JavaScript/Node.js ```javascript const AWS = require('aws-sdk'); const s3 = new AWS.S3({ endpoint: 'https://minio-api.michaelschiemer.de', accessKeyId: 'minioadmin', secretAccessKey: '', s3ForcePathStyle: true, signatureVersion: 'v4', }); // Create bucket s3.createBucket({ Bucket: 'my-bucket' }, (err, data) => { if (err) console.error(err); else console.log('Bucket created'); }); // Upload file const params = { Bucket: 'my-bucket', Key: 'file.txt', Body: require('fs').createReadStream('/path/to/file.txt'), }; s3.upload(params, (err, data) => { if (err) console.error(err); else console.log('File uploaded:', data.Location); }); ``` #### Python (boto3) ```python import boto3 from botocore.client import Config s3_client = boto3.client( 's3', endpoint_url='https://minio-api.michaelschiemer.de', aws_access_key_id='minioadmin', aws_secret_access_key='', config=Config(signature_version='s3v4'), region_name='us-east-1' ) # Create bucket s3_client.create_bucket(Bucket='my-bucket') # Upload file s3_client.upload_file('/path/to/file.txt', 'my-bucket', 'file.txt') # Download file s3_client.download_file('my-bucket', 'file.txt', '/path/to/downloaded.txt') # List objects response = s3_client.list_objects_v2(Bucket='my-bucket') for obj in response.get('Contents', []): print(obj['Key']) ``` ## User Management ### Create Access Keys via Console 1. Login to Console: https://minio.michaelschiemer.de 2. Navigate to "Access Keys" → "Create Access Key" 3. Assign policies (read-only, read-write, admin) 4. Save Access Key and Secret Key (only shown once!) ### Create Access Keys via mc CLI ```bash # Create new user mc admin user add minio myuser mypassword # Create access key for user mc admin user svcacct add minio myuser --name my-access-key # Output shows Access Key and Secret Key ``` ### Policy Management ```bash # List policies mc admin policy list minio # Create custom policy mc admin policy add minio readwrite-policy /path/to/policy.json # Assign policy to user mc admin policy set minio readwrite-policy user=myuser # Remove policy from user mc admin policy remove minio readwrite-policy user=myuser ``` **Example Policy** (`policy.json`): ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:GetObject", "s3:PutObject", "s3:DeleteObject" ], "Resource": ["arn:aws:s3:::my-bucket/*"] }, { "Effect": "Allow", "Action": ["s3:ListBucket"], "Resource": ["arn:aws:s3:::my-bucket"] } ] } ``` ## Integration with Application Stack ### Environment Variables Add to `application/.env`: ```env # MinIO Object Storage MINIO_ENDPOINT=https://minio-api.michaelschiemer.de MINIO_ACCESS_KEY= MINIO_SECRET_KEY= MINIO_BUCKET= MINIO_USE_SSL=true MINIO_REGION=us-east-1 ``` ### PHP Integration ```php // Use AWS SDK or MinIO PHP SDK use Aws\S3\S3Client; $s3Client = new S3Client([ 'version' => 'latest', 'region' => $_ENV['MINIO_REGION'], 'endpoint' => $_ENV['MINIO_ENDPOINT'], 'use_path_style_endpoint' => true, 'credentials' => [ 'key' => $_ENV['MINIO_ACCESS_KEY'], 'secret' => $_ENV['MINIO_SECRET_KEY'], ], ]); ``` ## Backup & Recovery ### Manual Backup ```bash #!/bin/bash # backup-minio.sh BACKUP_DIR="/backups/minio" DATE=$(date +%Y%m%d_%H%M%S) mkdir -p $BACKUP_DIR # Backup MinIO data docker run --rm \ -v minio-data:/data \ -v $BACKUP_DIR:/backup \ alpine tar czf /backup/minio-data-$DATE.tar.gz -C /data . echo "Backup completed: $BACKUP_DIR/minio-data-$DATE.tar.gz" ``` ### Restore from Backup ```bash # Stop MinIO docker compose down # Restore data docker run --rm \ -v minio-data:/data \ -v /backups/minio:/backup \ alpine tar xzf /backup/minio-data-YYYYMMDD_HHMMSS.tar.gz -C /data # Start MinIO docker compose up -d ``` ### Automated Backups Add to crontab: ```bash # Daily backup at 3 AM 0 3 * * * /path/to/backup-minio.sh # Keep only last 30 days 0 4 * * * find /backups/minio -type f -mtime +30 -delete ``` ### Bucket-Level Backup (Recommended) Use MinIO's built-in replication or external tools: ```bash # Sync bucket to external storage mc mirror minio/my-bucket /backup/my-bucket/ # Or sync to another MinIO instance mc mirror minio/my-bucket remote-minio/my-bucket/ ``` ## Monitoring ### Health Checks ```bash # Check MinIO health docker compose ps # API health endpoint curl -f https://minio-api.michaelschiemer.de/minio/health/live # Check storage usage docker exec minio du -sh /data ``` ### Logs ```bash # View logs docker compose logs -f # Check for errors docker compose logs minio | grep -i error # Monitor API access docker compose logs -f minio | grep "GET /" ``` ### Storage Statistics ```bash # Check volume size docker volume inspect minio-data # Check disk usage docker system df -v | grep minio # List buckets via mc mc ls minio ``` ### Metrics (via mc) ```bash # Server info mc admin info minio # Service status mc admin service status minio # Trace operations mc admin trace minio ``` ## Performance Tuning ### MinIO Configuration For high-traffic scenarios, edit `docker-compose.yml`: ```yaml environment: # Increase concurrent operations - MINIO_CACHE_DRIVES=/mnt/cache1,/mnt/cache2 - MINIO_CACHE_QUOTA=80 - MINIO_CACHE_AFTER=0 - MINIO_CACHE_WATERMARK_LOW=70 - MINIO_CACHE_WATERMARK_HIGH=90 ``` ### Storage Optimization ```bash # Monitor storage growth du -sh /var/lib/docker/volumes/minio-data/ # Enable compression (handled automatically by MinIO) # Set bucket quotas via Console or mc mc admin quota set minio/my-bucket --hard 100GB ``` ## Troubleshooting ### Cannot Access Console ```bash # Check service is running docker compose ps # Check Traefik routing docker exec traefik cat /etc/traefik/traefik.yml # Check network docker network inspect traefik-public | grep minio # Test from server curl -k https://localhost:9001 ``` ### Authentication Failed ```bash # Verify environment variables docker exec minio env | grep MINIO_ROOT # Check logs docker compose logs minio | grep -i auth # Reset root password (stop container, remove volume, restart) ``` ### SSL Certificate Issues ```bash # Verify Traefik certificate docker exec traefik cat /acme.json | grep minio # Test SSL openssl s_client -connect minio-api.michaelschiemer.de:443 \ -servername minio-api.michaelschiemer.de < /dev/null ``` ### Storage Issues ```bash # Check volume mount docker exec minio df -h /data # Check for corrupted data docker exec minio find /data -type f -name "*.json" | head # Check disk space df -h /var/lib/docker/volumes/minio-data/ ``` ### API Connection Errors ```bash # Verify endpoint URL curl -I https://minio-api.michaelschiemer.de/minio/health/live # Test with credentials curl -u minioadmin: https://minio-api.michaelschiemer.de # Check CORS settings (if needed for web apps) mc admin config set minio api cors_allow_origin "https://yourdomain.com" ``` ## Security ### Security Best Practices 1. **Strong Credentials**: Use strong passwords for root user and access keys 2. **Change Default Root User**: Don't use `minioadmin` in production 3. **SSL Only**: Always use HTTPS (enforced via Traefik) 4. **Access Key Rotation**: Regularly rotate access keys 5. **Policy-Based Access**: Use IAM policies to limit permissions 6. **Bucket Policies**: Configure bucket-level policies 7. **Audit Logging**: Enable audit logging for compliance 8. **Encryption**: Enable encryption at rest and in transit ### Enable Audit Logging ```bash # Configure audit logging mc admin config set minio audit_webhook \ endpoint=https://log-service.example.com/webhook \ auth_token= ``` ### Enable Encryption ```bash # Set encryption keys mc admin config set minio encryption \ sse-s3 enable \ kms endpoint=https://kms.example.com \ kms-key-id= ``` ### Update Stack ```bash # Pull latest images docker compose pull # Recreate containers docker compose up -d # Verify docker compose ps ``` ### Security Headers Security headers are applied via Traefik's `default-chain@file` middleware: - HSTS - Content-Type Nosniff - XSS Protection - Frame Deny ## Additional Resources - **MinIO Documentation**: https://min.io/docs/ - **S3 API Compatibility**: https://min.io/docs/minio/linux/reference/minio-mc/mc.html - **MinIO Client (mc)**: https://min.io/docs/minio/linux/reference/minio-mc.html - **MinIO Erasure Coding**: https://min.io/docs/minio/linux/operations/concepts/erasure-coding.html - **MinIO Security**: https://min.io/docs/minio/linux/operations/security.html