feat(Production): Complete production deployment infrastructure

- Add comprehensive health check system with multiple endpoints
- Add Prometheus metrics endpoint
- Add production logging configurations (5 strategies)
- Add complete deployment documentation suite:
  * QUICKSTART.md - 30-minute deployment guide
  * DEPLOYMENT_CHECKLIST.md - Printable verification checklist
  * DEPLOYMENT_WORKFLOW.md - Complete deployment lifecycle
  * PRODUCTION_DEPLOYMENT.md - Comprehensive technical reference
  * production-logging.md - Logging configuration guide
  * ANSIBLE_DEPLOYMENT.md - Infrastructure as Code automation
  * README.md - Navigation hub
  * DEPLOYMENT_SUMMARY.md - Executive summary
- Add deployment scripts and automation
- Add DEPLOYMENT_PLAN.md - Concrete plan for immediate deployment
- Update README with production-ready features

All production infrastructure is now complete and ready for deployment.
This commit is contained in:
2025-10-25 19:18:37 +02:00
parent caa85db796
commit fc3d7e6357
83016 changed files with 378904 additions and 20919 deletions

View File

@@ -0,0 +1,837 @@
# Logging Best Practices
Comprehensive guide for effective logging in production with the Custom PHP Framework.
## Overview
This document provides best practices for implementing, configuring, and maintaining the logging infrastructure in production environments.
## Choosing the Right Handler
### RotatingFileHandler vs Standard FileHandler
**Use RotatingFileHandler when**:
- Logs grow continuously and need automatic management
- Disk space constraints require automatic cleanup
- Historical log retention with automatic archival is needed
- Production environments requiring hands-off log management
**Use Standard FileHandler when**:
- External log rotation tools (logrotate) are already in place
- Logs are shipped to external systems immediately
- Development/testing environments with manual cleanup
- Very low-volume logging scenarios
### Factory Method Selection
```php
use App\Framework\Logging\Handlers\RotatingFileHandler;
use App\Framework\Core\ValueObjects\Byte;
// Production: INFO and above, daily rotation, 30 days retention
$handler = RotatingFileHandler::production('/var/www/html/storage/logs/app.log');
// Daily rotation: New log file every day at midnight
$handler = RotatingFileHandler::daily(
logFile: '/var/www/html/storage/logs/app.log',
maxFiles: 30, // Keep 30 days of logs
compress: true // Compress rotated files (saves ~70% disk space)
);
// Weekly rotation: New log file every Monday
$handler = RotatingFileHandler::weekly(
logFile: '/var/www/html/storage/logs/weekly.log',
maxFiles: 12, // Keep 3 months of weekly logs
compress: true
);
// Size-based rotation: Rotate when file reaches size limit
$handler = RotatingFileHandler::withSizeRotation(
logFile: '/var/www/html/storage/logs/api.log',
maxFileSize: Byte::fromMegabytes(100), // Rotate at 100MB
maxFiles: 10, // Keep 10 rotated files
compress: true
);
```
## Rotation Strategy Selection
### Daily Rotation
**Best for**:
- Application logs with moderate volume (1-100 MB/day)
- Compliance requirements (daily log archival)
- Debugging workflows that align with daily releases/deployments
- General production application logging
**Configuration**:
```php
$handler = RotatingFileHandler::daily(
logFile: '/var/www/html/storage/logs/app.log',
maxFiles: 30, // 30 days retention
compress: true, // Save disk space
minLevel: LogLevel::INFO // Production: INFO and above
);
```
**Retention Guidelines**:
- Development: 7 days
- Staging: 14 days
- Production: 30-90 days (compliance dependent)
### Weekly Rotation
**Best for**:
- Low-volume logs (<10 MB/day)
- Background job logs
- Scheduled task logs
- Non-critical system logs
**Configuration**:
```php
$handler = RotatingFileHandler::weekly(
logFile: '/var/www/html/storage/logs/background.log',
maxFiles: 12, // 3 months retention
compress: true
);
```
### Size-Based Rotation
**Best for**:
- High-volume API logs
- Variable traffic patterns (burst handling)
- Logs where size control is more important than time boundaries
- Debug logs during development
**Configuration**:
```php
// High-traffic API logs
$handler = RotatingFileHandler::withSizeRotation(
logFile: '/var/www/html/storage/logs/api.log',
maxFileSize: Byte::fromMegabytes(100),
maxFiles: 20, // 2GB total (20 × 100MB)
compress: true
);
// Development: Smaller files for easier viewing
$handler = RotatingFileHandler::withSizeRotation(
logFile: '/var/www/html/storage/logs/debug.log',
maxFileSize: Byte::fromMegabytes(10),
maxFiles: 5,
compress: false // No compression in dev for easier access
);
```
**Size Recommendations**:
- Development: 10-50 MB
- Staging: 50-100 MB
- Production: 100-500 MB (balance between rotation frequency and file count)
## Log Level Guidelines
### Production Log Levels
```php
use App\Framework\Logging\LogLevel;
// Production: INFO and above (default)
$handler = RotatingFileHandler::production($logFile);
// Custom production configuration
$handler = RotatingFileHandler::daily($logFile)
->withMinLevel(LogLevel::INFO);
```
**Level Selection**:
- **ERROR**: Critical failures requiring immediate attention
- **WARNING**: Potentially harmful situations that recovered
- **INFO**: Important business events (user registration, order completion)
- **DEBUG**: Detailed development information (disabled in production)
**Production Recommendations**:
- Main application log: INFO and above
- Security log: WARNING and above
- API access log: INFO (all requests)
- Background jobs: INFO (job start/completion)
### Development Log Levels
```php
// Development: All levels including DEBUG
$handler = RotatingFileHandler::daily(
logFile: '/var/www/html/storage/logs/dev.log',
maxFiles: 7,
compress: false
)->withMinLevel(LogLevel::DEBUG);
```
### Separate Log Files by Severity
```php
// Error-only log for critical monitoring
$errorHandler = RotatingFileHandler::daily(
logFile: '/var/www/html/storage/logs/errors.log',
maxFiles: 90 // Keep errors longer
)->withMinLevel(LogLevel::ERROR);
// General application log
$appHandler = RotatingFileHandler::production(
'/var/www/html/storage/logs/app.log'
);
// Attach both handlers to logger
$logger->addHandler($errorHandler);
$logger->addHandler($appHandler);
```
## Production Configuration Examples
### Standard Web Application
```php
// Application logs: Daily rotation, 30 days retention
$appHandler = RotatingFileHandler::production(
'/var/www/html/storage/logs/app.log'
);
// Error logs: Keep longer for pattern analysis
$errorHandler = RotatingFileHandler::daily(
logFile: '/var/www/html/storage/logs/errors.log',
maxFiles: 90,
compress: true
)->withMinLevel(LogLevel::ERROR);
// Security logs: Weekly rotation, compressed
$securityHandler = RotatingFileHandler::weekly(
logFile: '/var/www/html/storage/logs/security.log',
maxFiles: 52, // 1 year
compress: true
);
```
### High-Traffic API
```php
// API access logs: Size-based rotation for burst traffic
$apiHandler = RotatingFileHandler::withSizeRotation(
logFile: '/var/www/html/storage/logs/api.log',
maxFileSize: Byte::fromMegabytes(200),
maxFiles: 20, // 4GB total
compress: true
)->withMinLevel(LogLevel::INFO);
// API errors: Separate file for monitoring
$apiErrorHandler = RotatingFileHandler::daily(
logFile: '/var/www/html/storage/logs/api-errors.log',
maxFiles: 60
)->withMinLevel(LogLevel::ERROR);
```
### Background Job Processing
```php
// Job logs: Daily rotation
$jobHandler = RotatingFileHandler::daily(
logFile: '/var/www/html/storage/logs/jobs.log',
maxFiles: 14, // 2 weeks sufficient for job logs
compress: true
)->withMinLevel(LogLevel::INFO);
// Failed job logs: Keep longer for debugging
$failedJobHandler = RotatingFileHandler::weekly(
logFile: '/var/www/html/storage/logs/failed-jobs.log',
maxFiles: 12, // 3 months
compress: true
)->withMinLevel(LogLevel::ERROR);
```
## Compression Best Practices
### When to Enable Compression
**Enable compression when**:
- Disk space is limited
- Log retention period is >7 days
- Logs are rarely accessed after rotation
- Production environments
**Disable compression when**:
- Development/debugging (frequent log access)
- Logs are shipped to external systems immediately
- Performance is critical (compression adds overhead)
- Very low log volume (<10 MB/day)
### Compression Trade-offs
```php
// Production: Compression enabled (recommended)
$handler = RotatingFileHandler::production($logFile);
// Development: Compression disabled for easier access
$handler = RotatingFileHandler::daily(
logFile: $logFile,
maxFiles: 7,
compress: false // No compression in dev
);
```
**Benefits**:
- Space savings: ~70% reduction on average
- Reduced backup costs
- Longer retention within disk limits
**Costs**:
- CPU overhead during rotation (negligible)
- Requires decompression to read old logs
- Slightly slower rotation process
## Health Monitoring
### LogHealthCheckCommand Usage
```bash
# Manual health check
docker exec php php console.php logs:health-check
# Detailed output with file information
docker exec php php console.php logs:health-check --detailed
# Auto-fix permission issues
docker exec php php console.php logs:health-check --fix-permissions
```
### Scheduled Health Checks
```php
use App\Framework\Scheduler\Services\SchedulerService;
use App\Framework\Scheduler\Schedules\CronSchedule;
// Daily health check at 2 AM
$scheduler->schedule(
'log-health-check',
CronSchedule::fromExpression('0 2 * * *'),
function() {
$command = new LogHealthCheckCommand();
$input = new ConsoleInput(['logs:health-check'], new ConsoleOutput(), null);
$exitCode = $command->execute($input);
if ($exitCode !== ExitCode::SUCCESS) {
// Alert operations team
$this->alerting->sendAlert(
level: AlertLevel::WARNING,
message: 'Log health check failed'
);
}
return ['exit_code' => $exitCode];
}
);
```
### Health Check Metrics
Monitor these metrics from health checks:
- **Directory existence**: All required log directories present
- **Write permissions**: Can write to log directories
- **Disk space**: >100MB free (warning), >10% free (healthy)
- **Large files**: Individual files >50MB (rotation issue)
- **Total log size**: Track growth trends
### Automated Alerts
```php
// Alert on health check failures
final readonly class LogHealthMonitor
{
public function checkAndAlert(): void
{
$health = $this->runHealthCheck();
if (!$health['directories_ok']) {
$this->alerting->sendAlert(
level: AlertLevel::CRITICAL,
message: 'Log directories missing or inaccessible',
context: $health['directory_issues']
);
}
if ($health['disk_space_low']) {
$this->alerting->sendAlert(
level: AlertLevel::WARNING,
message: "Low disk space: {$health['free_space']}",
context: $health['disk_info']
);
}
if ($health['large_files_count'] > 0) {
$this->alerting->sendAlert(
level: AlertLevel::INFO,
message: "{$health['large_files_count']} log files exceed 50MB",
context: $health['large_files']
);
}
}
}
```
## Docker/Container Deployment
### Directory Structure
```
storage/
└── logs/
├── app/ # Application logs
├── debug/ # Debug logs (dev only)
├── security/ # Security events
└── jobs/ # Background job logs
```
### Docker Volume Configuration
```yaml
# docker-compose.yml
services:
php:
volumes:
# Host mount for logs (accessible from host for monitoring)
- ./storage/logs:/var/www/html/storage/logs
```
**Benefits of host mounts**:
- Logs persist across container restarts
- Direct access from host for monitoring tools
- Backup integration without entering container
- Log rotation tools on host can access files
### Permissions Setup
```bash
# Fix permissions for Docker container
chmod -R 777 storage/logs
# Or use Alpine container for one-time fix
docker run --rm -v $(pwd)/storage:/mnt alpine chmod -R 777 /mnt/logs
```
### Production Container Configuration
```dockerfile
# Dockerfile
FROM php:8.3-fpm
# Create log directories
RUN mkdir -p /var/www/html/storage/logs/app && \
mkdir -p /var/www/html/storage/logs/security && \
mkdir -p /var/www/html/storage/logs/jobs && \
chown -R www-data:www-data /var/www/html/storage && \
chmod -R 755 /var/www/html/storage/logs
```
## Performance Considerations
### Log Volume Management
**High-volume scenarios**:
```php
// Use size-based rotation for predictable disk usage
$handler = RotatingFileHandler::withSizeRotation(
logFile: '/var/www/html/storage/logs/high-volume.log',
maxFileSize: Byte::fromMegabytes(100),
maxFiles: 20,
compress: true
);
// Separate handlers for different log types
$errorHandler = RotatingFileHandler::daily($errorLog)
->withMinLevel(LogLevel::ERROR); // Errors only
$debugHandler = RotatingFileHandler::withSizeRotation($debugLog, Byte::fromMegabytes(50))
->withMinLevel(LogLevel::DEBUG); // Debug info
```
**Performance Benchmarks** (from integration tests):
- 1000 log entries: <5 seconds
- Rotation overhead: <100ms per rotation
- Compression overhead: ~200ms for 100MB file
### Memory Management
```php
// For very high-volume logging, consider async logging
use App\Framework\Queue\Queue;
final readonly class AsyncLoggingHandler implements LogHandler
{
public function handle(LogRecord $record): void
{
// Queue log writes for background processing
$job = new WriteLogJob($record);
$this->queue->push(JobPayload::immediate($job));
}
}
```
### Disk I/O Optimization
**Best practices**:
- Write logs synchronously for errors (data loss prevention)
- Buffer INFO/DEBUG logs for batch writes (performance)
- Use separate handlers for different log types (parallel writes)
- Monitor disk I/O metrics during rotation
## Security Considerations
### Log File Permissions
```bash
# Recommended permissions
chmod 755 storage/logs # Directory: Read + execute for all, write for owner
chmod 644 storage/logs/*.log # Files: Read for all, write for owner
```
**Security guidelines**:
- Never 777 permissions in production (security risk)
- Use 755 for directories (allows reading but prevents unauthorized writes)
- Use 644 for files (readable by monitoring tools, writable only by application)
- Separate user for web server and log access
### Sensitive Data Logging
**DO NOT log**:
- Passwords (plaintext or hashed)
- API keys and secrets
- Credit card numbers
- Personal identification numbers (SSN, etc.)
- Session tokens
**Safe logging patterns**:
```php
// ❌ Unsafe: Logs sensitive data
$logger->info('User login', [
'password' => $password, // NEVER log passwords
'api_key' => $apiKey // NEVER log keys
]);
// ✅ Safe: Logs only necessary information
$logger->info('User login', [
'user_id' => $userId,
'ip_address' => $ipAddress,
'user_agent' => $userAgent
]);
// ✅ Safe: Mask sensitive data
$logger->info('Payment processed', [
'card_last_four' => substr($cardNumber, -4), // Only last 4 digits
'amount' => $amount,
'transaction_id' => $transactionId
]);
```
### Security Event Logging
```php
// Use framework's OWASP security logger for security events
use App\Framework\Security\OWASPSecurityLogger;
use App\Framework\Security\OWASPEventIdentifier;
$this->owaspLogger->logSecurityEvent(
new SecurityEventType(OWASPEventIdentifier::AUTHN_LOGIN_FAILURE),
request: $request,
context: [
'username' => $username,
'ip_address' => $request->server->getRemoteAddr(),
'failure_reason' => 'Invalid credentials'
]
);
```
## Troubleshooting
### Common Issues
#### Issue: Logs not rotating
**Symptoms**:
- Log file grows continuously
- No `.1`, `.2` rotation files created
**Diagnosis**:
```bash
# Check file permissions
ls -la storage/logs/
# Check disk space
df -h
# Verify handler configuration
docker exec php php console.php logs:health-check --detailed
```
**Solutions**:
```php
// Verify rotation configuration
$handler = RotatingFileHandler::daily($logFile, maxFiles: 30);
// Check if logs are actually being written
if (!file_exists($logFile)) {
// Permission issue or incorrect path
}
// Manually trigger rotation for testing
touch($logFile, strtotime('yesterday'));
$handler->handle($logRecord); // Should rotate now
```
#### Issue: Permission denied errors
**Symptoms**:
- Cannot write to log files
- Health check reports permission errors
**Solutions**:
```bash
# Fix permissions (development)
chmod -R 777 storage/logs
# Fix permissions (production - more secure)
chown -R www-data:www-data storage/logs
chmod -R 755 storage/logs
chmod -R 644 storage/logs/*.log
# Use health check auto-fix
docker exec php php console.php logs:health-check --fix-permissions
```
#### Issue: Disk space full
**Symptoms**:
- Health check reports low disk space
- Log rotation fails
**Solutions**:
```bash
# Check disk usage
du -sh storage/logs/*
# Find largest log files
find storage/logs -type f -exec ls -lh {} \; | sort -k5 -rh | head -10
# Manually clean old logs
find storage/logs -name "*.log.*" -mtime +30 -delete
# Increase compression for existing logs
find storage/logs -name "*.log.[0-9]*" ! -name "*.gz" -exec gzip {} \;
```
**Prevention**:
```php
// Reduce retention period
$handler = RotatingFileHandler::daily($logFile, maxFiles: 14); // 14 days instead of 30
// Reduce file size for size-based rotation
$handler = RotatingFileHandler::withSizeRotation(
$logFile,
maxFileSize: Byte::fromMegabytes(50), // 50MB instead of 100MB
maxFiles: 10
);
// Enable compression if not already
$handler = RotatingFileHandler::daily($logFile, compress: true);
```
#### Issue: Performance degradation
**Symptoms**:
- Slow request processing
- High disk I/O during logging
**Solutions**:
```php
// Separate handlers by log type to parallelize writes
$errorHandler = RotatingFileHandler::daily('/var/www/html/storage/logs/errors.log')
->withMinLevel(LogLevel::ERROR);
$infoHandler = RotatingFileHandler::daily('/var/www/html/storage/logs/info.log')
->withMinLevel(LogLevel::INFO)
->withMaxLevel(LogLevel::WARNING);
// Consider async logging for high-volume scenarios
use App\Framework\Logging\Handlers\QueuedLogHandler;
$asyncHandler = new QueuedLogHandler($queue, $handler);
```
## Integration with Framework Systems
### Scheduler Integration
```php
// Automated log rotation cleanup
$scheduler->schedule(
'log-cleanup',
CronSchedule::fromExpression('0 3 * * *'), // Daily at 3 AM
function() {
// Remove logs older than retention policy
$this->logCleanupService->cleanupExpiredLogs();
return ['cleaned' => true];
}
);
// Health monitoring
$scheduler->schedule(
'log-health-check',
IntervalSchedule::every(Duration::fromHours(6)),
function() {
$command = new LogHealthCheckCommand();
$input = new ConsoleInput(['logs:health-check'], new ConsoleOutput(), null);
return ['exit_code' => $command->execute($input)];
}
);
```
### Queue Integration
```php
// Background log processing for high-volume scenarios
final class ProcessLogBatchJob
{
public function handle(): void
{
$logBatch = $this->logBuffer->flush();
foreach ($logBatch as $logRecord) {
$this->handler->handle($logRecord);
}
}
}
```
### Event System Integration
```php
// Log important domain events
use App\Framework\Event\EventHandler;
#[EventHandler]
final readonly class UserRegistrationLogger
{
public function handle(UserRegisteredEvent $event): void
{
$this->logger->info('User registered', [
'user_id' => $event->userId,
'email' => $event->email->getMasked(),
'registration_method' => $event->method
]);
}
}
```
## Monitoring and Alerting
### Metrics to Track
1. **Log Volume Metrics**:
- Logs per hour/day
- Growth rate trends
- Rotation frequency
2. **Performance Metrics**:
- Write latency (should be <10ms)
- Rotation duration (should be <1s)
- Disk I/O utilization
3. **Health Metrics**:
- Disk space free percentage
- Number of files exceeding size thresholds
- Permission check failures
4. **Error Metrics**:
- Error log volume (sudden spikes indicate issues)
- Failed rotation attempts
- Write permission errors
### Alert Configuration
```php
// Example monitoring integration
final readonly class LogMonitoringService
{
public function collectMetrics(): array
{
return [
'total_log_size' => $this->getTotalLogSize(),
'log_file_count' => $this->getLogFileCount(),
'largest_file_size' => $this->getLargestFileSize(),
'disk_space_free' => $this->getDiskSpaceFree(),
'rotation_count_24h' => $this->getRotationCount(Duration::fromHours(24))
];
}
public function checkThresholds(array $metrics): array
{
$alerts = [];
if ($metrics['disk_space_free'] < 1_000_000_000) { // <1GB
$alerts[] = [
'level' => 'critical',
'message' => 'Log disk space critically low'
];
}
if ($metrics['largest_file_size'] > 500_000_000) { // >500MB
$alerts[] = [
'level' => 'warning',
'message' => 'Large log file detected (rotation issue?)'
];
}
return $alerts;
}
}
```
## Summary Checklist
### Production Deployment
- [ ] Configure appropriate rotation strategy (daily/weekly/size-based)
- [ ] Set retention period based on compliance requirements
- [ ] Enable compression for rotated files
- [ ] Set production log level (INFO and above)
- [ ] Configure separate log files by severity (errors, security, etc.)
- [ ] Set up Docker volume mounts for log persistence
- [ ] Fix file permissions (755 directories, 644 files)
- [ ] Schedule health checks (daily recommended)
- [ ] Configure monitoring and alerting
- [ ] Test log rotation under load
- [ ] Document log access procedures for operations team
### Development Setup
- [ ] Use DEBUG log level for development
- [ ] Disable compression for easier access
- [ ] Use shorter retention (7 days sufficient)
- [ ] Smaller file size limits for size-based rotation
- [ ] Regular cleanup to prevent disk space issues
- [ ] Test rotation locally before deploying
### Ongoing Maintenance
- [ ] Monitor disk space weekly
- [ ] Review health check results daily
- [ ] Analyze log volume trends monthly
- [ ] Clean up old logs exceeding retention period
- [ ] Review and update alerting thresholds quarterly
- [ ] Test disaster recovery procedures (log restoration)
- [ ] Update documentation as logging patterns evolve
## Additional Resources
- **LogHealthCheckCommand**: `docs/logging/log-health-check-command.md`
- **RotatingFileHandler Implementation**: `src/Framework/Logging/Handlers/RotatingFileHandler.php`
- **Integration Tests**: `tests/Integration/Framework/Logging/LoggingInfrastructureTest.php`
- **OWASP Security Logging**: `docs/claude/security-patterns.md`