# LiveComponents Upload Guide Comprehensive guide for implementing chunked, resumable file uploads with progress tracking and virus scanning. ## Table of Contents - [Overview](#overview) - [Quick Start](#quick-start) - [Chunked Upload Architecture](#chunked-upload-architecture) - [Backend Implementation](#backend-implementation) - [Client-Side Implementation](#client-side-implementation) - [Progress Tracking](#progress-tracking) - [Quarantine & Virus Scanning](#quarantine--virus-scanning) - [Resume Capability](#resume-capability) - [Security](#security) - [Testing](#testing) - [Production Deployment](#production-deployment) --- ## Overview LiveComponents provides a **production-ready chunked upload system** with: ✅ **Chunked Uploads** - Split large files into manageable chunks (512KB default) ✅ **Resume Capability** - Resume interrupted uploads from last chunk ✅ **Integrity Verification** - SHA-256 hashing on client and server ✅ **Progress Tracking** - Real-time progress via Server-Sent Events (SSE) ✅ **Quarantine System** - Automatic virus scanning integration ✅ **Parallel Uploads** - Concurrent chunk uploads (3 parallel by default) ✅ **Retry Logic** - Automatic retry with exponential backoff ✅ **Memory Efficient** - Stream-based chunk assembly ### Use Cases - **Large File Uploads**: Videos, images, documents (>100MB) - **Unreliable Networks**: Mobile connections, slow networks - **Background Uploads**: Long-running uploads with resume capability - **Secure Uploads**: Virus scanning and integrity verification --- ## Quick Start ### 1. HTML Template ```html

Upload Files

Uploaded Files

``` ### 2. JavaScript Integration ```javascript import { ChunkedUploader } from '@js/modules/livecomponent'; const fileInput = document.getElementById('file-upload-input'); const uploadBtn = document.getElementById('upload-btn'); const progressBar = document.getElementById('progress-fill'); const progressText = document.getElementById('progress-text'); // Enable button when file selected fileInput.addEventListener('change', (e) => { uploadBtn.disabled = e.target.files.length === 0; }); // Initialize uploader const uploader = new ChunkedUploader('file-uploader:main', { chunkSize: 512 * 1024, // 512KB chunks maxConcurrentChunks: 3, // 3 parallel uploads maxRetries: 3, // Retry failed chunks 3 times enableSSE: true, // Real-time progress via SSE // Progress callback onProgress: (progress) => { progressBar.style.width = `${progress}%`; progressText.textContent = `${progress.toFixed(1)}%`; }, // Complete callback onComplete: (result) => { alert(`Upload complete! File ID: ${result.fileId}`); fileInput.value = ''; uploadBtn.disabled = true; }, // Error callback onError: (error) => { alert(`Upload failed: ${error.message}`); } }); // Upload on button click uploadBtn.addEventListener('click', async () => { const file = fileInput.files[0]; if (!file) return; try { const result = await uploader.uploadFile(file); console.log('Upload successful:', result); } catch (error) { console.error('Upload failed:', error); } }); ``` ### 3. Component Backend ```php fileRepository->findByUser($this->state->userId); return $this->state->withFiles($files); } public function render(): string { return 'components/file-uploader'; } } ``` **That's it!** The framework handles chunking, hashing, upload, assembly, and progress tracking automatically. --- ## Chunked Upload Architecture ### Upload Lifecycle ``` 1. Initialize Session ├─ Generate unique session ID ├─ Calculate total chunks ├─ Store session metadata └─ Return session ID to client 2. Upload Chunks (parallel) ├─ Hash chunk on client (SHA-256) ├─ Upload chunk with hash ├─ Verify hash on server ├─ Store chunk temporarily └─ Update session progress 3. Complete Upload ├─ Assemble all chunks ├─ Verify final file hash ├─ Move to quarantine ├─ Trigger virus scan └─ Return file ID 4. Post-Processing ├─ Virus scan (async) ├─ Move from quarantine ├─ Generate thumbnails └─ Update component state ``` ### API Endpoints | Endpoint | Method | Purpose | |----------|--------|---------| | `/live-component/upload/init` | POST | Initialize upload session | | `/live-component/upload/chunk` | POST | Upload single chunk | | `/live-component/upload/complete` | POST | Finalize upload | | `/live-component/upload/abort` | POST | Cancel upload | | `/live-component/upload/status/{sessionId}` | GET | Get upload status | --- ## Backend Implementation ### Upload Session Management The framework automatically manages upload sessions via `UploadSessionStore`: ```php use App\Framework\LiveComponents\Upload\UploadSession; use App\Framework\LiveComponents\Upload\UploadSessionId; use App\Framework\Core\ValueObjects\Duration; // Framework creates sessions automatically // You can query session status: $sessionId = UploadSessionId::fromString($sessionIdString); $session = $this->uploadSessionStore->get($sessionId); if ($session === null) { // Session expired or doesn't exist } // Session contains: $session->id; // UploadSessionId $session->fileName; // Original filename $session->totalSize; // Total file size $session->totalChunks; // Number of chunks $session->uploadedChunks; // Array of uploaded chunk indices $session->componentId; // Associated component ID $session->userId; // User ID (if authenticated) $session->createdAt; // Session creation time $session->expiresAt; // Session expiration ``` ### Chunk Integrity Verification ```php use App\Framework\LiveComponents\Upload\IntegrityValidator; use App\Framework\LiveComponents\Upload\ChunkHash; // Framework automatically verifies chunk integrity // But you can also use IntegrityValidator manually: $validator = $container->get(IntegrityValidator::class); // Verify chunk hash $expectedHash = ChunkHash::fromHex($clientProvidedHash); $isValid = $validator->verifyChunkHash($chunkPath, $expectedHash); if (!$isValid) { throw new ChunkIntegrityException('Chunk hash mismatch'); } // Verify final file hash $fileHash = $validator->hashFile($assembledFilePath); $isValid = $validator->verifyFileHash( $assembledFilePath, ChunkHash::fromHex($expectedFileHash) ); ``` ### Custom Post-Processing ```php use App\Framework\LiveComponents\Upload\ChunkedUploadManager; use App\Framework\Event\EventHandler; use App\Framework\LiveComponents\Upload\Events\UploadCompletedEvent; #[EventHandler] final readonly class ProcessUploadedFileHandler { public function handle(UploadCompletedEvent $event): void { // File is in quarantine, waiting for virus scan // You can process metadata here $file = new File( id: $event->fileId, originalName: $event->originalFileName, size: $event->fileSize, uploadedBy: $event->userId, componentId: $event->componentId ); $this->fileRepository->save($file); // Generate thumbnails (if image) if ($this->isImage($event->mimeType)) { $this->thumbnailGenerator->generate($event->filePath); } // Extract metadata (if video) if ($this->isVideo($event->mimeType)) { $this->videoMetadataExtractor->extract($event->filePath); } } } ``` ### Storage Configuration ```php // Environment configuration UPLOAD_TEMP_DIR=/var/www/storage/uploads/temp UPLOAD_QUARANTINE_DIR=/var/www/storage/uploads/quarantine UPLOAD_FINAL_DIR=/var/www/storage/uploads/files UPLOAD_MAX_FILE_SIZE=2147483648 # 2GB UPLOAD_CHUNK_SIZE=524288 # 512KB UPLOAD_SESSION_TTL=3600 # 1 hour ``` --- ## Client-Side Implementation ### Basic Upload ```javascript import { ChunkedUploader } from '@js/modules/livecomponent'; const uploader = new ChunkedUploader('file-uploader:main'); // Upload a file const result = await uploader.uploadFile(file); console.log('File uploaded:', result.fileId); ``` ### Advanced Configuration ```javascript const uploader = new ChunkedUploader('file-uploader:main', { // Chunk configuration chunkSize: 512 * 1024, // 512KB chunks (default) maxConcurrentChunks: 3, // 3 parallel uploads (default) // Retry configuration maxRetries: 3, // Retry failed chunks (default: 3) retryDelay: 1000, // Initial retry delay (default: 1s) retryBackoffMultiplier: 2, // Exponential backoff (default: 2) // Progress tracking enableSSE: true, // Real-time progress via SSE (default: true) progressUpdateInterval: 100, // Progress callback interval (default: 100ms) // Callbacks onProgress: (progress) => { console.log(`Upload: ${progress.toFixed(2)}%`); }, onChunkUpload: (chunkIndex, totalChunks) => { console.log(`Chunk ${chunkIndex + 1}/${totalChunks} uploaded`); }, onComplete: (result) => { console.log('Upload complete:', result); }, onError: (error) => { console.error('Upload failed:', error); } }); ``` ### Multiple File Upload ```javascript const files = Array.from(fileInput.files); // Sequential uploads for (const file of files) { try { const result = await uploader.uploadFile(file); console.log(`${file.name} uploaded successfully`); } catch (error) { console.error(`${file.name} failed:`, error); } } // Parallel uploads (careful with concurrency) const uploads = files.map(file => uploader.uploadFile(file)); const results = await Promise.allSettled(uploads); results.forEach((result, index) => { if (result.status === 'fulfilled') { console.log(`${files[index].name} uploaded`); } else { console.error(`${files[index].name} failed:`, result.reason); } }); ``` ### Custom Progress UI ```javascript const uploader = new ChunkedUploader('file-uploader:main', { onProgress: (progress) => { // Update progress bar progressBar.style.width = `${progress}%`; progressText.textContent = `${progress.toFixed(1)}%`; // Update status text if (progress < 25) { statusText.textContent = 'Starting upload...'; } else if (progress < 75) { statusText.textContent = 'Uploading...'; } else if (progress < 100) { statusText.textContent = 'Almost done...'; } }, onChunkUpload: (chunkIndex, totalChunks) => { chunksText.textContent = `${chunkIndex + 1} / ${totalChunks} chunks`; }, onComplete: (result) => { progressBar.classList.add('complete'); statusText.textContent = 'Upload complete!'; // Show success message setTimeout(() => { resetUploadUI(); }, 2000); } }); ``` --- ## Progress Tracking ### Real-Time Progress via SSE The framework automatically broadcasts upload progress via Server-Sent Events: ```javascript // SSE automatically enabled by default const uploader = new ChunkedUploader('file-uploader:main', { enableSSE: true, // Real-time progress onProgress: (progress) => { // Called automatically as chunks upload console.log(`Progress: ${progress}%`); } }); // Progress events are automatically synchronized between: // - Chunk upload completion // - Server-side progress updates // - SSE broadcasts ``` ### Manual Progress Polling ```javascript // Disable SSE and poll manually const uploader = new ChunkedUploader('file-uploader:main', { enableSSE: false // Disable automatic SSE }); // Poll upload status const sessionId = 'upload-session-id'; async function pollUploadStatus() { const response = await fetch(`/live-component/upload/status/${sessionId}`); const status = await response.json(); console.log('Upload progress:', { uploaded: status.uploaded_chunks, total: status.total_chunks, progress: (status.uploaded_chunks / status.total_chunks) * 100 }); if (status.uploaded_chunks < status.total_chunks) { setTimeout(pollUploadStatus, 1000); // Poll every second } } ``` ### Progress Events ```javascript // Listen to granular progress events uploader.on('upload:started', (data) => { console.log('Upload started:', data.fileName); }); uploader.on('chunk:uploaded', (data) => { console.log(`Chunk ${data.chunkIndex} uploaded`); }); uploader.on('upload:progress', (data) => { console.log(`Progress: ${data.progress}%`); }); uploader.on('upload:complete', (data) => { console.log('Upload complete:', data.fileId); }); uploader.on('upload:error', (error) => { console.error('Upload error:', error); }); ``` --- ## Quarantine & Virus Scanning ### How Quarantine Works ``` 1. Upload Complete ↓ 2. File Assembled ↓ 3. Move to Quarantine (automatic) ├─ File stored in quarantine directory ├─ QuarantineStatus: PENDING └─ Virus scan triggered (async) ↓ 4. Virus Scan ├─ ClamAV / VirusTotal integration ├─ QuarantineStatus: SCANNING └─ Scan result returned ↓ 5. Post-Scan ├─ Clean → Move to final storage, QuarantineStatus: RELEASED └─ Infected → Delete file, QuarantineStatus: QUARANTINED ``` ### Virus Scanner Integration ```php use App\Framework\LiveComponents\Upload\QuarantineService; use App\Framework\LiveComponents\Upload\ScanResult; use App\Framework\LiveComponents\Upload\ScanStatus; // Custom virus scanner implementation final readonly class ClamAVScanner implements VirusScanner { public function scan(string $filePath): ScanResult { // Execute ClamAV scan $output = shell_exec("clamscan --no-summary {$filePath}"); $isClean = str_contains($output, 'OK'); if ($isClean) { return ScanResult::clean(); } // Extract virus signature preg_match('/FOUND: (.+)/', $output, $matches); $signature = $matches[1] ?? 'Unknown'; return ScanResult::infected($signature); } } // Register scanner $container->singleton(VirusScanner::class, new ClamAVScanner()); ``` ### VirusTotal Integration ```php final readonly class VirusTotalScanner implements VirusScanner { public function __construct( private string $apiKey, private HttpClient $httpClient ) {} public function scan(string $filePath): ScanResult { // Upload file to VirusTotal $response = $this->httpClient->post('https://www.virustotal.com/api/v3/files', [ 'file' => new CURLFile($filePath) ], [ 'x-apikey' => $this->apiKey ]); $data = json_decode($response->body, true); // Check scan results $positives = $data['data']['attributes']['last_analysis_stats']['malicious'] ?? 0; if ($positives > 0) { return ScanResult::infected( signature: "{$positives} engines detected malware" ); } return ScanResult::clean(); } } ``` ### Quarantine Status Tracking ```php use App\Framework\LiveComponents\Upload\QuarantineStatus; // Check quarantine status $status = $this->quarantineService->getStatus($fileId); match ($status) { QuarantineStatus::PENDING => 'Waiting for scan...', QuarantineStatus::SCANNING => 'Scanning for viruses...', QuarantineStatus::QUARANTINED => 'File infected and quarantined', QuarantineStatus::RELEASED => 'File clean and available', }; // Listen to scan completion $this->eventDispatcher->listen(FileScanCompletedEvent::class, function($event) { if ($event->scanResult->isClean()) { // File is safe - notify user $this->notificationService->send( $event->userId, "Your file '{$event->fileName}' has been uploaded successfully" ); } else { // File infected - alert security team $this->securityLogger->alert('Infected file uploaded', [ 'file_id' => $event->fileId, 'user_id' => $event->userId, 'signature' => $event->scanResult->signature ]); } }); ``` --- ## Resume Capability ### How Resume Works 1. **Upload Interrupted** (network failure, browser close, etc.) 2. **User Returns** and selects same file 3. **Framework Detects** existing session by file hash 4. **Resume Upload** from last successfully uploaded chunk ### Resume Implementation ```javascript const uploader = new ChunkedUploader('file-uploader:main', { enableResume: true // Enable resume (default: true) }); // Framework automatically: // 1. Hashes file to generate consistent session ID // 2. Checks server for existing session // 3. Resumes from last uploaded chunk // 4. Continues upload seamlessly // Manual resume control uploader.on('upload:resuming', (data) => { console.log(`Resuming from chunk ${data.lastChunkIndex}`); console.log(`${data.uploadedChunks}/${data.totalChunks} chunks already uploaded`); // Show resume message to user showNotification(`Resuming upload from ${data.progress}%`); }); ``` ### Session Expiration ```php // Configure session TTL UPLOAD_SESSION_TTL=3600 # 1 hour // Sessions expire after TTL // After expiration, upload must restart from beginning ``` ### Resume Best Practices ```javascript // 1. Always use same file (don't modify between uploads) const file = fileInput.files[0]; const result = await uploader.uploadFile(file); // 2. Don't change filename (affects session ID) // ❌ Bad const renamedFile = new File([file], 'new-name.jpg', { type: file.type }); // ✅ Good const originalFile = fileInput.files[0]; // 3. Handle resume gracefully uploader.on('upload:resuming', (data) => { // Inform user alert(`Resuming upload from ${data.progress.toFixed(1)}%`); }); uploader.on('upload:cannot-resume', (reason) => { // Session expired or file changed alert(`Cannot resume: ${reason}. Starting new upload...`); }); ``` --- ## Security ### Upload Security Checklist - [x] **File Size Limits** - Enforce max file size (2GB default) - [x] **File Type Validation** - MIME type and extension checking - [x] **Integrity Verification** - SHA-256 hashing client and server - [x] **Virus Scanning** - Automatic quarantine and scan - [x] **Rate Limiting** - Prevent upload abuse - [x] **Authentication** - Only authenticated users can upload - [x] **CSRF Protection** - Token validation on all uploads - [x] **Idempotency** - Prevent duplicate uploads ### File Type Validation ```php // Configure allowed file types $allowedMimeTypes = [ 'image/jpeg', 'image/png', 'image/gif', 'image/webp', 'video/mp4', 'application/pdf', 'application/msword', 'application/vnd.openxmlformats-officedocument.wordprocessingml.document' ]; $allowedExtensions = [ 'jpg', 'jpeg', 'png', 'gif', 'webp', 'mp4', 'pdf', 'doc', 'docx' ]; // Validation happens automatically in ChunkedUploadController // But you can also validate manually: if (!in_array($file->getMimeType(), $allowedMimeTypes)) { throw new InvalidFileTypeException('File type not allowed'); } if (!in_array($file->getExtension(), $allowedExtensions)) { throw new InvalidFileTypeException('File extension not allowed'); } ``` ### Size Limits ```javascript // Client-side validation (before upload) const MAX_FILE_SIZE = 2 * 1024 * 1024 * 1024; // 2GB if (file.size > MAX_FILE_SIZE) { alert('File too large. Maximum size is 2GB.'); return; } // Server-side enforcement (automatic) // Configured via UPLOAD_MAX_FILE_SIZE environment variable ``` ### Rate Limiting ```php use App\Framework\LiveComponents\Attributes\Action; #[Action( rateLimit: 10, // Max 10 upload requests rateLimitWindow: 3600 // Per hour )] public function uploadChunk(UploadChunkRequest $request): JsonResult { // Rate limiting applied automatically by framework } ``` --- ## Testing ### Unit Tests ```php use function Pest\LiveComponents\mountComponent; use function Pest\LiveComponents\callAction; describe('FileUploaderComponent', function () { it('initializes upload session', function () { $manager = container()->get(ChunkedUploadManager::class); $session = $manager->initializeSession( componentId: 'file-uploader:test', fileName: 'test.pdf', fileSize: 1024 * 1024, // 1MB totalChunks: 2, userId: '123' ); expect($session->fileName)->toBe('test.pdf'); expect($session->totalChunks)->toBe(2); }); it('verifies chunk integrity', function () { $validator = container()->get(IntegrityValidator::class); $chunkPath = __DIR__ . '/fixtures/chunk.bin'; $expectedHash = ChunkHash::fromHex(hash_file('sha256', $chunkPath)); $isValid = $validator->verifyChunkHash($chunkPath, $expectedHash); expect($isValid)->toBeTrue(); }); }); ``` ### Integration Tests ```javascript // JavaScript tests with Jest describe('ChunkedUploader', () => { it('uploads file in chunks', async () => { const file = new File(['test content'], 'test.txt', { type: 'text/plain' }); const uploader = new ChunkedUploader('test:uploader', { chunkSize: 512 }); const result = await uploader.uploadFile(file); expect(result.fileId).toBeDefined(); expect(result.success).toBe(true); }); it('resumes interrupted upload', async () => { const file = new File(['test content'], 'test.txt'); const uploader = new ChunkedUploader('test:uploader'); // Start upload const uploadPromise = uploader.uploadFile(file); // Simulate interruption uploader.abort(); // Resume const result = await uploader.uploadFile(file); expect(result.resumed).toBe(true); }); }); ``` --- ## Production Deployment ### Infrastructure Setup ```bash # Create upload directories mkdir -p /var/www/storage/uploads/{temp,quarantine,files} # Set permissions chown -R www-data:www-data /var/www/storage/uploads chmod -R 755 /var/www/storage/uploads # Install ClamAV apt-get install clamav clamav-daemon systemctl start clamav-daemon # Update virus definitions freshclam ``` ### Environment Configuration ```env # Upload settings UPLOAD_TEMP_DIR=/var/www/storage/uploads/temp UPLOAD_QUARANTINE_DIR=/var/www/storage/uploads/quarantine UPLOAD_FINAL_DIR=/var/www/storage/uploads/files UPLOAD_MAX_FILE_SIZE=2147483648 UPLOAD_CHUNK_SIZE=524288 UPLOAD_SESSION_TTL=3600 # Virus scanning CLAMAV_SOCKET=/var/run/clamav/clamd.ctl VIRUSTOTAL_API_KEY=your-api-key-here # Cleanup UPLOAD_CLEANUP_ENABLED=true UPLOAD_CLEANUP_INTERVAL=3600 # Clean up old sessions every hour ``` ### Monitoring ```php // Track upload metrics $this->metrics->increment('upload.sessions.created'); $this->metrics->increment('upload.chunks.uploaded'); $this->metrics->increment('upload.completed'); $this->metrics->increment('upload.failed'); // Track virus scan results $this->metrics->increment('virus_scan.clean'); $this->metrics->increment('virus_scan.infected'); // Track performance $this->metrics->timing('upload.session.duration', $duration); $this->metrics->timing('upload.chunk.duration', $chunkDuration); ``` --- ## Summary LiveComponents Chunked Upload System: ✅ **Production-Ready** - Battle-tested with large files (>2GB) ✅ **Resumable** - Automatic resume from interruptions ✅ **Secure** - Hash verification + virus scanning + quarantine ✅ **Fast** - Parallel chunk uploads (3 concurrent) ✅ **Reliable** - Retry logic with exponential backoff ✅ **User-Friendly** - Real-time progress via SSE ✅ **Memory-Efficient** - Stream-based chunk assembly Perfect for applications requiring **large file uploads** with **reliability** and **security**.