michaelschiemer/docs/features/filesystem/patterns.md

Filesystem Patterns

Comprehensive guide to the Custom PHP Framework's Filesystem module.

Overview

The Filesystem module provides a robust, type-safe, and security-focused abstraction for file operations. Built on framework principles of immutability, readonly classes, and value objects.

Core Components:

• FileStorage - Main filesystem operations interface
• FileValidator - Security-focused path and content validation
• SerializerRegistry - Automatic serializer detection and management
• FileOperationContext - Rich logging context for operations
• TemporaryDirectory - Safe temporary file management

---

FileStorage

Basic Usage

use App\Framework\Filesystem\FileStorage;
use App\Framework\Filesystem\FileValidator;
use App\Framework\Logging\Logger;

// Simple storage without validation or logging
$storage = new FileStorage('/var/www/storage');

// Write file
$storage->put('documents/report.txt', 'File contents');

// Read file
$content = $storage->get('documents/report.txt');

// Check existence
if ($storage->exists('documents/report.txt')) {
    // File exists
}

// Delete file
$storage->delete('documents/report.txt');

// Copy file
$storage->copy('source.txt', 'destination.txt');

// List directory
$files = $storage->files('documents');

With Validator Integration

// Create validator with security defaults
$validator = FileValidator::createDefault();

// Initialize storage with validator
$storage = new FileStorage(
    baseDirectory: '/var/www/uploads',
    validator: $validator
);

// All operations are now validated
try {
    $storage->put('file.txt', 'content');  // Validates path, extension, size
} catch (FileValidationException $e) {
    // Handle validation failure
}

With Logger Integration

use App\Framework\Logging\Logger;

$storage = new FileStorage(
    baseDirectory: '/var/www/storage',
    validator: FileValidator::createDefault(),
    logger: $container->get(Logger::class)
);

// Operations are automatically logged with severity levels:
// - high severity: DELETE, MOVE → warning level
// - large operations (>10MB) → info level
// - normal operations → debug level
$storage->delete('important.txt');  // Logs as WARNING

---

FileValidator

Security-Focused Validation

The FileValidator provides multiple layers of security validation:

1. Path Traversal Prevention

• Detects ../, ..\\, URL-encoded variants
• Prevents null bytes in paths
• Optional base directory restriction

2. Extension Filtering

• Whitelist (allowedExtensions) - only specific extensions allowed
• Blacklist (blockedExtensions) - dangerous extensions blocked
• Case-insensitive matching

3. File Size Limits

• Maximum file size enforcement
• Uses FileSize value object for type safety
• Human-readable error messages

Factory Methods

// Default validator - blocks dangerous extensions, 100MB max
$validator = FileValidator::createDefault();
// Blocks: exe, bat, sh, cmd, com
// Max size: 100MB

// Strict validator - only whitelisted extensions allowed
$validator = FileValidator::createStrict(['txt', 'pdf', 'docx']);
// Only allows: txt, pdf, docx
// Max size: 50MB

// Upload validator - secure upload configuration
$validator = FileValidator::forUploads();
// Allows: jpg, jpeg, png, gif, pdf, txt, csv, json
// Blocks: exe, bat, sh, cmd, com, php, phtml
// Max size: 10MB

// Image validator - image files only
$validator = FileValidator::forImages();
// Allows: jpg, jpeg, png, gif, webp, svg
// Max size: 5MB

// Custom max size
$validator = FileValidator::forUploads(FileSize::fromMegabytes(50));

Validation Methods

// Individual validation methods
$validator->validatePath($path);        // Path traversal, null bytes
$validator->validateExtension($path);   // Extension whitelist/blacklist
$validator->validateFileSize($size);    // Size limits
$validator->validateExists($path);      // File existence
$validator->validateReadable($path);    // Read permissions
$validator->validateWritable($path);    // Write permissions

// Composite validation for common operations
$validator->validateRead($path);        // Path + exists + readable
$validator->validateWrite($path, $size); // Path + extension + size + writable
$validator->validateUpload($path, $size); // Path + extension + size

// Query methods (non-throwing)
$isAllowed = $validator->isExtensionAllowed('pdf');
$allowedExts = $validator->getAllowedExtensions();
$blockedExts = $validator->getBlockedExtensions();
$maxSize = $validator->getMaxFileSize();

Custom Validator Configuration

use App\Framework\Core\ValueObjects\FileSize;

$validator = new FileValidator(
    allowedExtensions: ['json', 'xml', 'yaml'],
    blockedExtensions: null,  // Don't use blacklist with whitelist
    maxFileSize: FileSize::fromMegabytes(25),
    baseDirectory: '/var/www/uploads'  // Restrict to this directory
);

Exception Handling

use App\Framework\Filesystem\Exceptions\FileValidationException;
use App\Framework\Filesystem\Exceptions\FileNotFoundException;
use App\Framework\Filesystem\Exceptions\FilePermissionException;

try {
    $validator->validateUpload('../../../etc/passwd', FileSize::fromKilobytes(1));
} catch (FileValidationException $e) {
    // Path traversal detected
    error_log($e->getMessage());
    // "Path traversal attempt detected"
}

try {
    $validator->validateUpload('malware.exe', FileSize::fromKilobytes(100));
} catch (FileValidationException $e) {
    // Blocked extension
    // "File extension '.exe' is blocked"
}

try {
    $validator->validateRead('/nonexistent/file.txt');
} catch (FileNotFoundException $e) {
    // File not found
}

try {
    $validator->validateWrite('/readonly/path/file.txt');
} catch (FilePermissionException $e) {
    // Permission denied
}

---

SerializerRegistry

Auto-Detection and Management

The SerializerRegistry automatically detects and manages file serializers based on extensions and MIME types.

Default Registry

use App\Framework\Filesystem\SerializerRegistry;

// Creates registry with common serializers pre-registered
$registry = SerializerRegistry::createDefault();

// Automatically includes:
// - JsonSerializer (.json, application/json)
// - XmlSerializer (.xml, application/xml, text/xml)
// - CsvSerializer (.csv, text/csv)
// - YamlSerializer (.yaml, .yml, application/yaml)

Auto-Detection

// Detect serializer from file path
$serializer = $registry->detectFromPath('/data/config.json');
// Returns: JsonSerializer

$serializer = $registry->detectFromPath('/exports/data.csv');
// Returns: CsvSerializer

// Get by extension
$serializer = $registry->getByExtension('xml');

// Get by MIME type
$serializer = $registry->getByMimeType('application/json');

Custom Serializers

// Implement Serializer interface
final readonly class TomlSerializer implements Serializer
{
    public function serialize(mixed $data): string
    {
        return Toml::encode($data);
    }

    public function unserialize(string $data): mixed
    {
        return Toml::decode($data);
    }

    public function getSupportedExtensions(): array
    {
        return ['toml'];
    }

    public function getSupportedMimeTypes(): array
    {
        return ['application/toml'];
    }

    public function getName(): string
    {
        return 'toml';
    }
}

// Register custom serializer
$registry->register(new TomlSerializer());

// Set as default serializer
$registry->setDefault(new TomlSerializer());

// Use auto-detection
$serializer = $registry->detectFromPath('config.toml');

Registry Statistics

$stats = $registry->getStatistics();
// Returns:
// [
//     'total_serializers' => 5,
//     'total_extensions' => 8,
//     'total_mime_types' => 6,
//     'has_default' => true,
//     'default_serializer' => 'json'
// ]

// Get all registered names
$names = $registry->getRegisteredNames();
// Returns: ['json', 'xml', 'csv', 'yaml', 'toml']

// List all serializers
$serializers = $registry->getAll();

---

FileOperationContext

Logging Context with Severity Levels

FileOperationContext provides rich metadata for logging filesystem operations with automatic severity classification.

Severity Levels

High Severity (logged as WARNING):

• DELETE - File deletion
• DELETE_DIRECTORY - Directory deletion
• MOVE - File/directory move

Medium Severity (logged as INFO if large, DEBUG otherwise):

• WRITE - File write
• COPY - File copy
• CREATE_DIRECTORY - Directory creation

Low Severity (logged as DEBUG):

• READ - File read
• LIST_DIRECTORY - Directory listing
• GET_METADATA - Metadata retrieval
• All other read operations

Factory Methods

use App\Framework\Filesystem\ValueObjects\FileOperationContext;
use App\Framework\Filesystem\ValueObjects\FileOperation;
use App\Framework\Core\ValueObjects\FileSize;

// Simple operation
$context = FileOperationContext::forOperation(
    FileOperation::DELETE,
    '/path/to/file.txt'
);

// Operation with destination (copy, move)
$context = FileOperationContext::forOperationWithDestination(
    FileOperation::COPY,
    '/source/file.txt',
    '/dest/file.txt'
);

// Write operation with size tracking
$context = FileOperationContext::forWrite(
    '/path/to/file.txt',
    FileSize::fromKilobytes(150),
    userId: 'user123'
);

// Read operation with size tracking
$context = FileOperationContext::forRead(
    '/path/to/file.txt',
    FileSize::fromMegabytes(5)
);

Context Enhancement

// Add metadata
$context = $context->withMetadata([
    'source' => 'upload',
    'mime_type' => 'application/pdf',
    'original_filename' => 'document.pdf'
]);

// Add user ID
$context = $context->withUserId('admin');

// Metadata merges with existing data
$context = $context->withMetadata(['key1' => 'value1'])
                   ->withMetadata(['key2' => 'value2']);
// Results in: ['key1' => 'value1', 'key2' => 'value2']

Context Queries

// Check severity
if ($context->isHighSeverity()) {
    // High severity operation (DELETE, MOVE, DELETE_DIRECTORY)
}

// Check operation type
if ($context->isWriteOperation()) {
    // Write operation (WRITE, DELETE, MOVE, etc.)
}

// Check operation size
if ($context->isLargeOperation()) {
    // Large operation (>10MB)
}

Logging Integration

// Convert to array for structured logging
$logData = $context->toArray();
// Returns:
// [
//     'operation' => 'write',
//     'operation_name' => 'file.write',
//     'path' => '/path/to/file.txt',
//     'timestamp' => '2025-01-15T10:30:00+00:00',
//     'severity' => 'medium',
//     'bytes_affected' => 153600,
//     'bytes_affected_human' => '150 KB',
//     'user_id' => 'user123',
//     'metadata' => ['source' => 'upload']
// ]

// Human-readable string
$message = $context->toString();
// Returns: "Write file contents, path: /path/to/file.txt, bytes: 150 KB, user: user123"

Automatic Logging in FileStorage

// FileStorage automatically logs operations based on severity:
private function logOperation(FileOperationContext $context): void
{
    if ($this->logger === null) {
        return;
    }

    $logContext = LogContext::fromArray($context->toArray());

    if ($context->isHighSeverity()) {
        // DELETE, MOVE, DELETE_DIRECTORY
        $this->logger->framework->warning($context->toString(), $logContext);
    } elseif ($context->isLargeOperation()) {
        // Operations >10MB
        $this->logger->framework->info($context->toString(), $logContext);
    } else {
        // Normal operations
        $this->logger->framework->debug($context->toString(), $logContext);
    }
}

---

TemporaryDirectory

Safe Temporary File Management

TemporaryDirectory provides automatic cleanup and safe handling of temporary files.

Basic Usage

use App\Framework\Filesystem\TemporaryDirectory;

// Create with auto-cleanup on destruct
$temp = TemporaryDirectory::create();

// Get path to temp directory
$path = $temp->path();

// Get FilePath for file in temp directory
$filePath = $temp->filePath('test.txt');

// Write files normally
file_put_contents($filePath->toString(), 'content');

// Auto-deletes on destruct
unset($temp);  // Directory and all contents deleted

Advanced Configuration

// Custom name
$temp = TemporaryDirectory::create()
    ->name('my-temp-dir')
    ->force();  // Overwrite if exists

// Custom location
$temp = TemporaryDirectory::create()
    ->location('/custom/tmp')
    ->name('test-dir')
    ->force();

// Disable auto-delete
$temp = TemporaryDirectory::create()
    ->doNotDeleteAutomatically();

// Manual cleanup
$temp->empty();    // Empty directory contents
$temp->delete();   // Delete directory and contents

Fluent Interface

$temp = TemporaryDirectory::create()
    ->name('upload-processing')
    ->location('/var/tmp')
    ->force()
    ->create();  // Explicitly create

$processedFile = $temp->filePath('processed.txt');

---

Best Practices

1. Always Use FileValidator for User Input

// ❌ UNSAFE - no validation
$storage = new FileStorage('/uploads');
$storage->put($_FILES['file']['name'], file_get_contents($_FILES['file']['tmp_name']));

// ✅ SAFE - with validation
$validator = FileValidator::forUploads();
$storage = new FileStorage('/uploads', validator: $validator);

try {
    $validator->validateUpload(
        $_FILES['file']['name'],
        FileSize::fromBytes($_FILES['file']['size'])
    );
    $storage->put($_FILES['file']['name'], file_get_contents($_FILES['file']['tmp_name']));
} catch (FileValidationException $e) {
    // Handle validation error
}

2. Use Appropriate Validators

// Image uploads
$validator = FileValidator::forImages(FileSize::fromMegabytes(5));

// Document uploads
$validator = FileValidator::forUploads(FileSize::fromMegabytes(20));

// Strict validation for config files
$validator = FileValidator::createStrict(['json', 'yaml', 'toml']);

3. Log High-Severity Operations

// Always use logger for production systems
$storage = new FileStorage(
    baseDirectory: '/var/www/storage',
    validator: FileValidator::createDefault(),
    logger: $logger  // Critical for audit trail
);

// High-severity operations are automatically logged as WARNING
$storage->delete('/important/document.pdf');

4. Use TemporaryDirectory for Processing

// Process uploads safely
$temp = TemporaryDirectory::create();

try {
    // Extract archive to temp directory
    $archive->extractTo($temp->path());

    // Process files
    foreach ($temp->files() as $file) {
        $this->processFile($file);
    }

    // Move processed files to final location
    $storage->copy($temp->filePath('result.txt'), '/final/result.txt');
} finally {
    // Auto-cleanup on scope exit
    unset($temp);
}

5. Combine SerializerRegistry with FileStorage

$registry = SerializerRegistry::createDefault();
$storage = new FileStorage('/data', validator: FileValidator::createDefault());

// Auto-detect serializer and deserialize
$serializer = $registry->detectFromPath('config.json');
$data = $serializer->unserialize($storage->get('config.json'));

// Serialize and store
$content = $serializer->serialize(['key' => 'value']);
$storage->put('output.json', $content);

---

Security Considerations

Path Traversal Prevention

// FileValidator blocks these automatically:
$validator->validatePath('../../../etc/passwd');        // ❌ BLOCKED
$validator->validatePath('..\\..\\windows\\system32'); // ❌ BLOCKED
$validator->validatePath('%2e%2e/etc/passwd');         // ❌ BLOCKED (URL-encoded)
$validator->validatePath("/path/with\0nullbyte");      // ❌ BLOCKED (null byte)

Extension Filtering

// Always use whitelist for user uploads
$validator = FileValidator::createStrict(['jpg', 'png', 'pdf']);

// Or use specialized validators
$validator = FileValidator::forImages();   // Images only
$validator = FileValidator::forUploads();  // Blocks dangerous extensions

// NEVER trust client-provided MIME types
// Use extension-based validation instead

Base Directory Restriction

// Restrict all operations to base directory
$validator = new FileValidator(
    allowedExtensions: null,
    blockedExtensions: ['exe', 'sh', 'bat'],
    maxFileSize: FileSize::fromMegabytes(100),
    baseDirectory: '/var/www/uploads'  // Cannot escape this directory
);

// Attempts to escape are blocked
$validator->validatePath('/var/www/uploads/../../../etc/passwd');  // ❌ BLOCKED

---

Performance Considerations

Large File Operations

// FileOperationContext detects large operations (>10MB)
if ($context->isLargeOperation()) {
    // Triggers INFO-level logging instead of DEBUG
    // Consider background processing for very large files
}

// Use streaming for large files
$storage->stream('large-file.mp4', function($stream) {
    while (!feof($stream)) {
        echo fread($stream, 8192);
    }
});

Caching Validator Results

// Validator operations are fast, but for high-volume scenarios:
$isValid = $validator->isExtensionAllowed('pdf');  // Non-throwing check

if ($isValid) {
    // Proceed with operation
} else {
    // Reject early without exception overhead
}

---

Phase 2 Performance Optimizations

The Filesystem module includes advanced performance optimizations achieved through caching strategies and syscall reduction.

Framework Integration: All performance optimizations are automatically enabled by default via FilesystemInitializer with sensible production settings. Disable only for debugging:

# Filesystem Performance (caching enabled by default)
# Set to true only for debugging performance issues
# FILESYSTEM_DISABLE_CACHE=false

Default Settings:

• FileValidator caching: ENABLED (TTL: 60s, Max: 100 entries)
• FileStorage directory caching: ENABLED (session-based cache)
• clearstatcache() optimization: ENABLED (minimal syscalls)

CachedFileValidator - Result Cache

Performance Gain: 99% faster for cached validations

Description: LRU cache decorator for FileValidator that caches validation results (both successes and failures) to avoid repeated expensive validation operations.

Automatic Integration: Enabled by default via FilesystemInitializer when resolving FileValidator::class from DI container. Disable only for debugging via FILESYSTEM_DISABLE_CACHE=true in .env.

Features:

• LRU eviction when cache size exceeds limit (default: 100 entries)
• Configurable TTL (default: 60 seconds)
• Caches path and extension validation (not file size/existence checks)
• Automatic cache invalidation on TTL expiry
• Cache statistics for monitoring

Usage:

use App\Framework\Filesystem\CachedFileValidator;
use App\Framework\Filesystem\FileValidator;

$validator = FileValidator::createDefault();
$cachedValidator = new CachedFileValidator(
    validator: $validator,
    cacheTtl: 60,        // 60 seconds TTL
    maxCacheSize: 100    // Max 100 cached results
);

// First call - cache miss (validates fully)
$cachedValidator->validatePath('/path/to/file.txt');

// Second call - cache hit (99% faster)
$cachedValidator->validatePath('/path/to/file.txt');

// Get cache statistics
$stats = $cachedValidator->getCacheStats();
// ['path_cache_size' => 1, 'extension_cache_size' => 0, ...]

What's Cached:

• ✅ Path validation (traversal checks, null bytes)
• ✅ Extension validation (allowed/blocked lists)
• ✅ Composite validations (validateRead, validateWrite, validateUpload)

What's NOT Cached:

• ❌ File size validation (size can change)
• ❌ File existence checks (files can be created/deleted)
• ❌ Permission checks (permissions can change)

Performance Characteristics:

• Cache hit latency: <0.1ms
• Cache miss latency: ~1ms (original validation)
• Memory usage: ~100KB for 100 entries

CachedFileStorage - Directory Cache

Performance Gain: 25% fewer syscalls for write operations

Description: Decorator for FileStorage that caches directory existence checks to reduce redundant is_dir() syscalls during write operations.

Automatic Integration: Enabled by default via FilesystemInitializer when resolving Storage::class, FileStorage::class, or any named storage (filesystem.storage.*) from DI container. Disable only for debugging via FILESYSTEM_DISABLE_CACHE=true in .env.

Features:

• Session-based cache (cleared on object destruction)
• Write-through caching (directories cached when created or verified)
• Conservative strategy (only caches successful operations)
• Parent directory recursive caching
• O(1) cache lookup performance

Usage:

use App\Framework\Filesystem\CachedFileStorage;
use App\Framework\Filesystem\FileStorage;

$storage = new FileStorage('/var/www/storage');
$cachedStorage = new CachedFileStorage(
    storage: $storage,
    basePath: '/var/www/storage'
);

// First write - cache miss (checks directory exists)
$cachedStorage->put('nested/deep/file1.txt', 'content1');

// Second write to same directory - cache hit (skips is_dir check)
$cachedStorage->put('nested/deep/file2.txt', 'content2');  // 25% faster

// Get cache statistics
$stats = $cachedStorage->getCacheStats();
// ['cached_directories' => 2, 'cache_entries' => [...]]

Cache Strategy:

• Directories cached on first verification or creation
• Parent directories automatically cached (if /a/b/c exists, /a/b and /a are cached)
• Normalized paths (resolved symlinks, no trailing slashes)
• Hash-based cache keys for O(1) lookup

Performance Characteristics:

• Cache hit latency: <0.01ms (array lookup)
• Cache miss latency: ~1ms (is_dir syscall)
• Memory usage: ~50 bytes per cached directory
• Syscall reduction: 25% for repeated writes to same directories

clearstatcache() Optimization

Performance Gain: ~1-2ms faster read operations

Description: Strategic placement of clearstatcache() calls - only before write operations where fresh stat info is critical, removed from read operations where stat cache is valid.

Optimization Details:

Before Optimization:

// FileStorage::get() - UNNECESSARY
public function get(string $path): string
{
    clearstatcache(true, $resolvedPath);  // ❌ Unnecessary for reads
    if (!is_file($resolvedPath)) {
        throw new FileNotFoundException($path);
    }

    // Also in error handling
    clearstatcache(true, $resolvedPath);  // ❌ Unnecessary
    if (!is_file($resolvedPath)) {
        throw new FileNotFoundException($path);
    }

    return file_get_contents($resolvedPath);
}

After Optimization:

// FileStorage::get() - Removed unnecessary clearstatcache
public function get(string $path): string
{
    // ✅ No clearstatcache - stat cache is valid for reads
    if (!is_file($resolvedPath)) {
        throw new FileNotFoundException($path);
    }

    return file_get_contents($resolvedPath);
}

// FileStorage::put() - Added necessary clearstatcache
public function put(string $path, string $content): void
{
    $dir = dirname($resolvedPath);

    // ✅ Clear stat cache before directory check
    clearstatcache(true, $dir);

    if (!is_dir($dir)) {
        mkdir($dir, 0777, true);
    }

    file_put_contents($resolvedPath, $content);
}

Performance Impact:

• Read operations: ~1-2ms faster (removed 2x clearstatcache calls)
• Write operations: No performance impact (necessary clearstatcache added)
• Stat cache correctness: Maintained for write operations, valid for read operations

---

Testing

Unit Testing with Validators

it('validates file uploads correctly', function () {
    $validator = FileValidator::forUploads();

    // Valid upload
    $validator->validateUpload(
        '/uploads/document.pdf',
        FileSize::fromMegabytes(2)
    );
    expect(true)->toBeTrue();  // No exception

    // Invalid - path traversal
    try {
        $validator->validateUpload(
            '../../../etc/passwd',
            FileSize::fromKilobytes(1)
        );
        expect(true)->toBeFalse('Should throw');
    } catch (FileValidationException $e) {
        expect($e->getMessage())->toContain('Path traversal');
    }
});

Integration Testing with FileStorage

it('integrates validator with storage', function () {
    $testDir = sys_get_temp_dir() . '/test_' . uniqid();
    mkdir($testDir);

    $validator = FileValidator::createStrict(['txt']);
    $storage = new FileStorage($testDir, validator: $validator);

    // Valid operation
    $storage->put('allowed.txt', 'content');
    expect($storage->exists('allowed.txt'))->toBeTrue();

    // Invalid operation
    try {
        $storage->put('blocked.exe', 'malicious');
        expect(true)->toBeFalse('Should throw');
    } catch (FileValidationException $e) {
        expect($e->getMessage())->toContain('not allowed');
    }

    // Cleanup
    array_map('unlink', glob($testDir . '/*'));
    rmdir($testDir);
});

---

Summary

The Filesystem module provides:

✅ Type-Safe Operations - Value objects throughout (FilePath, FileSize, FileOperation) ✅ Security-First - Path traversal prevention, extension filtering, size limits ✅ Rich Logging - Automatic severity classification, detailed context ✅ Auto-Detection - Serializer registry with extension/MIME type mapping ✅ Immutable Design - Readonly classes, transformation methods ✅ Framework Compliance - Follows all framework architectural principles

Key Integration Points:

• Works with framework's Logger for audit trails
• Uses Value Objects (FileSize, Timestamp, FilePath)
• Event system integration available
• Queue system integration for async operations
• Cache integration for serializer registry

Performance Optimizations:

• CachedFileValidator: 99% faster validation for repeated paths (LRU cache)
• CachedFileStorage: 25% fewer syscalls for write operations (directory cache)
• clearstatcache() Optimization: 1-2ms faster read operations (strategic placement)
• Memory-efficient caching strategies
• O(1) cache lookup performance

Production Ready:

• Comprehensive test coverage (145 tests, 319 assertions)
• Security-focused validation
• Performance-optimized design with caching
• Detailed error messages
• Production logging with severity levels