michaelschiemer/src/Framework/LiveComponents/docs/UPLOAD-GUIDE.md

LiveComponents Upload Guide

Comprehensive guide for implementing chunked, resumable file uploads with progress tracking and virus scanning.

Table of Contents

• Overview
• Quick Start
• Chunked Upload Architecture
• Backend Implementation
• Client-Side Implementation
• Progress Tracking
• Quarantine & Virus Scanning
• Resume Capability
• Security
• Testing
• Production Deployment

---

Overview

LiveComponents provides a production-ready chunked upload system with:

✅ Chunked Uploads - Split large files into manageable chunks (512KB default) ✅ Resume Capability - Resume interrupted uploads from last chunk ✅ Integrity Verification - SHA-256 hashing on client and server ✅ Progress Tracking - Real-time progress via Server-Sent Events (SSE) ✅ Quarantine System - Automatic virus scanning integration ✅ Parallel Uploads - Concurrent chunk uploads (3 parallel by default) ✅ Retry Logic - Automatic retry with exponential backoff ✅ Memory Efficient - Stream-based chunk assembly

Use Cases

• Large File Uploads: Videos, images, documents (>100MB)
• Unreliable Networks: Mobile connections, slow networks
• Background Uploads: Long-running uploads with resume capability
• Secure Uploads: Virus scanning and integrity verification

---

Quick Start

1. HTML Template

<div data-lc-component="file-uploader" data-lc-id="{component_id}">
    <h3>Upload Files</h3>

    <!-- File input -->
    <input
        type="file"
        id="file-upload-input"
        accept="image/*,video/*,.pdf,.doc,.docx"
        multiple
    />

    <!-- Upload button -->
    <button id="upload-btn" disabled>Upload</button>

    <!-- Progress display -->
    <div id="upload-progress" class="hidden">
        <div class="progress-bar">
            <div id="progress-fill" style="width: 0%"></div>
        </div>
        <p id="progress-text">0%</p>
        <p id="upload-status">Preparing upload...</p>
    </div>

    <!-- Uploaded files list -->
    <div id="uploaded-files">
        <h4>Uploaded Files</h4>
        <ul data-lc-fragment="file-list">
            <for items="uploaded_files" as="file">
                <li>
                    {file.name} ({file.size_formatted})
                    <span class="status">{file.status}</span>
                </li>
            </for>
        </ul>
    </div>
</div>

2. JavaScript Integration

import { ChunkedUploader } from '@js/modules/livecomponent';

const fileInput = document.getElementById('file-upload-input');
const uploadBtn = document.getElementById('upload-btn');
const progressBar = document.getElementById('progress-fill');
const progressText = document.getElementById('progress-text');

// Enable button when file selected
fileInput.addEventListener('change', (e) => {
    uploadBtn.disabled = e.target.files.length === 0;
});

// Initialize uploader
const uploader = new ChunkedUploader('file-uploader:main', {
    chunkSize: 512 * 1024,     // 512KB chunks
    maxConcurrentChunks: 3,    // 3 parallel uploads
    maxRetries: 3,             // Retry failed chunks 3 times
    enableSSE: true,           // Real-time progress via SSE

    // Progress callback
    onProgress: (progress) => {
        progressBar.style.width = `${progress}%`;
        progressText.textContent = `${progress.toFixed(1)}%`;
    },

    // Complete callback
    onComplete: (result) => {
        alert(`Upload complete! File ID: ${result.fileId}`);
        fileInput.value = '';
        uploadBtn.disabled = true;
    },

    // Error callback
    onError: (error) => {
        alert(`Upload failed: ${error.message}`);
    }
});

// Upload on button click
uploadBtn.addEventListener('click', async () => {
    const file = fileInput.files[0];
    if (!file) return;

    try {
        const result = await uploader.uploadFile(file);
        console.log('Upload successful:', result);
    } catch (error) {
        console.error('Upload failed:', error);
    }
});

3. Component Backend

<?php

namespace App\Application\LiveComponents\FileUploader;

use App\Framework\LiveComponents\Attributes\Action;
use App\Framework\LiveComponents\Attributes\LiveComponent;
use App\Framework\LiveComponents\Contracts\LiveComponentContract;
use App\Framework\LiveComponents\ValueObjects\ComponentId;

#[LiveComponent('file-uploader')]
final readonly class FileUploaderComponent implements LiveComponentContract
{
    public function __construct(
        public ComponentId $id,
        public FileUploaderState $state,
        private FileRepository $fileRepository
    ) {}

    #[Action]
    public function refreshFileList(): FileUploaderState
    {
        $files = $this->fileRepository->findByUser($this->state->userId);

        return $this->state->withFiles($files);
    }

    public function render(): string
    {
        return 'components/file-uploader';
    }
}

That's it! The framework handles chunking, hashing, upload, assembly, and progress tracking automatically.

---

Chunked Upload Architecture

Upload Lifecycle

1. Initialize Session
   ├─ Generate unique session ID
   ├─ Calculate total chunks
   ├─ Store session metadata
   └─ Return session ID to client

2. Upload Chunks (parallel)
   ├─ Hash chunk on client (SHA-256)
   ├─ Upload chunk with hash
   ├─ Verify hash on server
   ├─ Store chunk temporarily
   └─ Update session progress

3. Complete Upload
   ├─ Assemble all chunks
   ├─ Verify final file hash
   ├─ Move to quarantine
   ├─ Trigger virus scan
   └─ Return file ID

4. Post-Processing
   ├─ Virus scan (async)
   ├─ Move from quarantine
   ├─ Generate thumbnails
   └─ Update component state

API Endpoints

Endpoint	Method	Purpose
/live-component/upload/init	POST	Initialize upload session
/live-component/upload/chunk	POST	Upload single chunk
/live-component/upload/complete	POST	Finalize upload
/live-component/upload/abort	POST	Cancel upload
/live-component/upload/status/{sessionId}	GET	Get upload status

---

Backend Implementation

Upload Session Management

The framework automatically manages upload sessions via UploadSessionStore:

use App\Framework\LiveComponents\Upload\UploadSession;
use App\Framework\LiveComponents\Upload\UploadSessionId;
use App\Framework\Core\ValueObjects\Duration;

// Framework creates sessions automatically
// You can query session status:

$sessionId = UploadSessionId::fromString($sessionIdString);
$session = $this->uploadSessionStore->get($sessionId);

if ($session === null) {
    // Session expired or doesn't exist
}

// Session contains:
$session->id;              // UploadSessionId
$session->fileName;        // Original filename
$session->totalSize;       // Total file size
$session->totalChunks;     // Number of chunks
$session->uploadedChunks;  // Array of uploaded chunk indices
$session->componentId;     // Associated component ID
$session->userId;          // User ID (if authenticated)
$session->createdAt;       // Session creation time
$session->expiresAt;       // Session expiration

Chunk Integrity Verification

use App\Framework\LiveComponents\Upload\IntegrityValidator;
use App\Framework\LiveComponents\Upload\ChunkHash;

// Framework automatically verifies chunk integrity
// But you can also use IntegrityValidator manually:

$validator = $container->get(IntegrityValidator::class);

// Verify chunk hash
$expectedHash = ChunkHash::fromHex($clientProvidedHash);
$isValid = $validator->verifyChunkHash($chunkPath, $expectedHash);

if (!$isValid) {
    throw new ChunkIntegrityException('Chunk hash mismatch');
}

// Verify final file hash
$fileHash = $validator->hashFile($assembledFilePath);
$isValid = $validator->verifyFileHash(
    $assembledFilePath,
    ChunkHash::fromHex($expectedFileHash)
);

Custom Post-Processing

use App\Framework\LiveComponents\Upload\ChunkedUploadManager;
use App\Framework\Event\EventHandler;
use App\Framework\LiveComponents\Upload\Events\UploadCompletedEvent;

#[EventHandler]
final readonly class ProcessUploadedFileHandler
{
    public function handle(UploadCompletedEvent $event): void
    {
        // File is in quarantine, waiting for virus scan
        // You can process metadata here

        $file = new File(
            id: $event->fileId,
            originalName: $event->originalFileName,
            size: $event->fileSize,
            uploadedBy: $event->userId,
            componentId: $event->componentId
        );

        $this->fileRepository->save($file);

        // Generate thumbnails (if image)
        if ($this->isImage($event->mimeType)) {
            $this->thumbnailGenerator->generate($event->filePath);
        }

        // Extract metadata (if video)
        if ($this->isVideo($event->mimeType)) {
            $this->videoMetadataExtractor->extract($event->filePath);
        }
    }
}

Storage Configuration

// Environment configuration
UPLOAD_TEMP_DIR=/var/www/storage/uploads/temp
UPLOAD_QUARANTINE_DIR=/var/www/storage/uploads/quarantine
UPLOAD_FINAL_DIR=/var/www/storage/uploads/files
UPLOAD_MAX_FILE_SIZE=2147483648  # 2GB
UPLOAD_CHUNK_SIZE=524288          # 512KB
UPLOAD_SESSION_TTL=3600           # 1 hour

---

Client-Side Implementation

Basic Upload

import { ChunkedUploader } from '@js/modules/livecomponent';

const uploader = new ChunkedUploader('file-uploader:main');

// Upload a file
const result = await uploader.uploadFile(file);

console.log('File uploaded:', result.fileId);

Advanced Configuration

const uploader = new ChunkedUploader('file-uploader:main', {
    // Chunk configuration
    chunkSize: 512 * 1024,         // 512KB chunks (default)
    maxConcurrentChunks: 3,        // 3 parallel uploads (default)

    // Retry configuration
    maxRetries: 3,                 // Retry failed chunks (default: 3)
    retryDelay: 1000,              // Initial retry delay (default: 1s)
    retryBackoffMultiplier: 2,     // Exponential backoff (default: 2)

    // Progress tracking
    enableSSE: true,               // Real-time progress via SSE (default: true)
    progressUpdateInterval: 100,   // Progress callback interval (default: 100ms)

    // Callbacks
    onProgress: (progress) => {
        console.log(`Upload: ${progress.toFixed(2)}%`);
    },

    onChunkUpload: (chunkIndex, totalChunks) => {
        console.log(`Chunk ${chunkIndex + 1}/${totalChunks} uploaded`);
    },

    onComplete: (result) => {
        console.log('Upload complete:', result);
    },

    onError: (error) => {
        console.error('Upload failed:', error);
    }
});

Multiple File Upload

const files = Array.from(fileInput.files);

// Sequential uploads
for (const file of files) {
    try {
        const result = await uploader.uploadFile(file);
        console.log(`${file.name} uploaded successfully`);
    } catch (error) {
        console.error(`${file.name} failed:`, error);
    }
}

// Parallel uploads (careful with concurrency)
const uploads = files.map(file => uploader.uploadFile(file));
const results = await Promise.allSettled(uploads);

results.forEach((result, index) => {
    if (result.status === 'fulfilled') {
        console.log(`${files[index].name} uploaded`);
    } else {
        console.error(`${files[index].name} failed:`, result.reason);
    }
});

Custom Progress UI

const uploader = new ChunkedUploader('file-uploader:main', {
    onProgress: (progress) => {
        // Update progress bar
        progressBar.style.width = `${progress}%`;
        progressText.textContent = `${progress.toFixed(1)}%`;

        // Update status text
        if (progress < 25) {
            statusText.textContent = 'Starting upload...';
        } else if (progress < 75) {
            statusText.textContent = 'Uploading...';
        } else if (progress < 100) {
            statusText.textContent = 'Almost done...';
        }
    },

    onChunkUpload: (chunkIndex, totalChunks) => {
        chunksText.textContent = `${chunkIndex + 1} / ${totalChunks} chunks`;
    },

    onComplete: (result) => {
        progressBar.classList.add('complete');
        statusText.textContent = 'Upload complete!';

        // Show success message
        setTimeout(() => {
            resetUploadUI();
        }, 2000);
    }
});

---

Progress Tracking

Real-Time Progress via SSE

The framework automatically broadcasts upload progress via Server-Sent Events:

// SSE automatically enabled by default
const uploader = new ChunkedUploader('file-uploader:main', {
    enableSSE: true,  // Real-time progress

    onProgress: (progress) => {
        // Called automatically as chunks upload
        console.log(`Progress: ${progress}%`);
    }
});

// Progress events are automatically synchronized between:
// - Chunk upload completion
// - Server-side progress updates
// - SSE broadcasts

Manual Progress Polling

// Disable SSE and poll manually
const uploader = new ChunkedUploader('file-uploader:main', {
    enableSSE: false  // Disable automatic SSE
});

// Poll upload status
const sessionId = 'upload-session-id';

async function pollUploadStatus() {
    const response = await fetch(`/live-component/upload/status/${sessionId}`);
    const status = await response.json();

    console.log('Upload progress:', {
        uploaded: status.uploaded_chunks,
        total: status.total_chunks,
        progress: (status.uploaded_chunks / status.total_chunks) * 100
    });

    if (status.uploaded_chunks < status.total_chunks) {
        setTimeout(pollUploadStatus, 1000);  // Poll every second
    }
}

Progress Events

// Listen to granular progress events
uploader.on('upload:started', (data) => {
    console.log('Upload started:', data.fileName);
});

uploader.on('chunk:uploaded', (data) => {
    console.log(`Chunk ${data.chunkIndex} uploaded`);
});

uploader.on('upload:progress', (data) => {
    console.log(`Progress: ${data.progress}%`);
});

uploader.on('upload:complete', (data) => {
    console.log('Upload complete:', data.fileId);
});

uploader.on('upload:error', (error) => {
    console.error('Upload error:', error);
});

---

Quarantine & Virus Scanning

How Quarantine Works

1. Upload Complete
   ↓
2. File Assembled
   ↓
3. Move to Quarantine (automatic)
   ├─ File stored in quarantine directory
   ├─ QuarantineStatus: PENDING
   └─ Virus scan triggered (async)
   ↓
4. Virus Scan
   ├─ ClamAV / VirusTotal integration
   ├─ QuarantineStatus: SCANNING
   └─ Scan result returned
   ↓
5. Post-Scan
   ├─ Clean → Move to final storage, QuarantineStatus: RELEASED
   └─ Infected → Delete file, QuarantineStatus: QUARANTINED

Virus Scanner Integration

use App\Framework\LiveComponents\Upload\QuarantineService;
use App\Framework\LiveComponents\Upload\ScanResult;
use App\Framework\LiveComponents\Upload\ScanStatus;

// Custom virus scanner implementation
final readonly class ClamAVScanner implements VirusScanner
{
    public function scan(string $filePath): ScanResult
    {
        // Execute ClamAV scan
        $output = shell_exec("clamscan --no-summary {$filePath}");

        $isClean = str_contains($output, 'OK');

        if ($isClean) {
            return ScanResult::clean();
        }

        // Extract virus signature
        preg_match('/FOUND: (.+)/', $output, $matches);
        $signature = $matches[1] ?? 'Unknown';

        return ScanResult::infected($signature);
    }
}

// Register scanner
$container->singleton(VirusScanner::class, new ClamAVScanner());

VirusTotal Integration

final readonly class VirusTotalScanner implements VirusScanner
{
    public function __construct(
        private string $apiKey,
        private HttpClient $httpClient
    ) {}

    public function scan(string $filePath): ScanResult
    {
        // Upload file to VirusTotal
        $response = $this->httpClient->post('https://www.virustotal.com/api/v3/files', [
            'file' => new CURLFile($filePath)
        ], [
            'x-apikey' => $this->apiKey
        ]);

        $data = json_decode($response->body, true);

        // Check scan results
        $positives = $data['data']['attributes']['last_analysis_stats']['malicious'] ?? 0;

        if ($positives > 0) {
            return ScanResult::infected(
                signature: "{$positives} engines detected malware"
            );
        }

        return ScanResult::clean();
    }
}

Quarantine Status Tracking

use App\Framework\LiveComponents\Upload\QuarantineStatus;

// Check quarantine status
$status = $this->quarantineService->getStatus($fileId);

match ($status) {
    QuarantineStatus::PENDING => 'Waiting for scan...',
    QuarantineStatus::SCANNING => 'Scanning for viruses...',
    QuarantineStatus::QUARANTINED => 'File infected and quarantined',
    QuarantineStatus::RELEASED => 'File clean and available',
};

// Listen to scan completion
$this->eventDispatcher->listen(FileScanCompletedEvent::class, function($event) {
    if ($event->scanResult->isClean()) {
        // File is safe - notify user
        $this->notificationService->send(
            $event->userId,
            "Your file '{$event->fileName}' has been uploaded successfully"
        );
    } else {
        // File infected - alert security team
        $this->securityLogger->alert('Infected file uploaded', [
            'file_id' => $event->fileId,
            'user_id' => $event->userId,
            'signature' => $event->scanResult->signature
        ]);
    }
});

---

Resume Capability

How Resume Works

1. Upload Interrupted (network failure, browser close, etc.)
2. User Returns and selects same file
3. Framework Detects existing session by file hash
4. Resume Upload from last successfully uploaded chunk

Resume Implementation

const uploader = new ChunkedUploader('file-uploader:main', {
    enableResume: true  // Enable resume (default: true)
});

// Framework automatically:
// 1. Hashes file to generate consistent session ID
// 2. Checks server for existing session
// 3. Resumes from last uploaded chunk
// 4. Continues upload seamlessly

// Manual resume control
uploader.on('upload:resuming', (data) => {
    console.log(`Resuming from chunk ${data.lastChunkIndex}`);
    console.log(`${data.uploadedChunks}/${data.totalChunks} chunks already uploaded`);

    // Show resume message to user
    showNotification(`Resuming upload from ${data.progress}%`);
});

Session Expiration

// Configure session TTL
UPLOAD_SESSION_TTL=3600  # 1 hour

// Sessions expire after TTL
// After expiration, upload must restart from beginning

Resume Best Practices

// 1. Always use same file (don't modify between uploads)
const file = fileInput.files[0];
const result = await uploader.uploadFile(file);

// 2. Don't change filename (affects session ID)
// ❌ Bad
const renamedFile = new File([file], 'new-name.jpg', { type: file.type });

// ✅ Good
const originalFile = fileInput.files[0];

// 3. Handle resume gracefully
uploader.on('upload:resuming', (data) => {
    // Inform user
    alert(`Resuming upload from ${data.progress.toFixed(1)}%`);
});

uploader.on('upload:cannot-resume', (reason) => {
    // Session expired or file changed
    alert(`Cannot resume: ${reason}. Starting new upload...`);
});

---

Security

Upload Security Checklist

• File Size Limits - Enforce max file size (2GB default)
• File Type Validation - MIME type and extension checking
• Integrity Verification - SHA-256 hashing client and server
• Virus Scanning - Automatic quarantine and scan
• Rate Limiting - Prevent upload abuse
• Authentication - Only authenticated users can upload
• CSRF Protection - Token validation on all uploads
• Idempotency - Prevent duplicate uploads

File Type Validation

// Configure allowed file types
$allowedMimeTypes = [
    'image/jpeg',
    'image/png',
    'image/gif',
    'image/webp',
    'video/mp4',
    'application/pdf',
    'application/msword',
    'application/vnd.openxmlformats-officedocument.wordprocessingml.document'
];

$allowedExtensions = [
    'jpg', 'jpeg', 'png', 'gif', 'webp',
    'mp4', 'pdf', 'doc', 'docx'
];

// Validation happens automatically in ChunkedUploadController
// But you can also validate manually:
if (!in_array($file->getMimeType(), $allowedMimeTypes)) {
    throw new InvalidFileTypeException('File type not allowed');
}

if (!in_array($file->getExtension(), $allowedExtensions)) {
    throw new InvalidFileTypeException('File extension not allowed');
}

Size Limits

// Client-side validation (before upload)
const MAX_FILE_SIZE = 2 * 1024 * 1024 * 1024; // 2GB

if (file.size > MAX_FILE_SIZE) {
    alert('File too large. Maximum size is 2GB.');
    return;
}

// Server-side enforcement (automatic)
// Configured via UPLOAD_MAX_FILE_SIZE environment variable

Rate Limiting

use App\Framework\LiveComponents\Attributes\Action;

#[Action(
    rateLimit: 10,           // Max 10 upload requests
    rateLimitWindow: 3600    // Per hour
)]
public function uploadChunk(UploadChunkRequest $request): JsonResult
{
    // Rate limiting applied automatically by framework
}

---

Testing

Unit Tests

use function Pest\LiveComponents\mountComponent;
use function Pest\LiveComponents\callAction;

describe('FileUploaderComponent', function () {
    it('initializes upload session', function () {
        $manager = container()->get(ChunkedUploadManager::class);

        $session = $manager->initializeSession(
            componentId: 'file-uploader:test',
            fileName: 'test.pdf',
            fileSize: 1024 * 1024,  // 1MB
            totalChunks: 2,
            userId: '123'
        );

        expect($session->fileName)->toBe('test.pdf');
        expect($session->totalChunks)->toBe(2);
    });

    it('verifies chunk integrity', function () {
        $validator = container()->get(IntegrityValidator::class);

        $chunkPath = __DIR__ . '/fixtures/chunk.bin';
        $expectedHash = ChunkHash::fromHex(hash_file('sha256', $chunkPath));

        $isValid = $validator->verifyChunkHash($chunkPath, $expectedHash);

        expect($isValid)->toBeTrue();
    });
});

Integration Tests

// JavaScript tests with Jest
describe('ChunkedUploader', () => {
    it('uploads file in chunks', async () => {
        const file = new File(['test content'], 'test.txt', {
            type: 'text/plain'
        });

        const uploader = new ChunkedUploader('test:uploader', {
            chunkSize: 512
        });

        const result = await uploader.uploadFile(file);

        expect(result.fileId).toBeDefined();
        expect(result.success).toBe(true);
    });

    it('resumes interrupted upload', async () => {
        const file = new File(['test content'], 'test.txt');
        const uploader = new ChunkedUploader('test:uploader');

        // Start upload
        const uploadPromise = uploader.uploadFile(file);

        // Simulate interruption
        uploader.abort();

        // Resume
        const result = await uploader.uploadFile(file);

        expect(result.resumed).toBe(true);
    });
});

---

Production Deployment

Infrastructure Setup

# Create upload directories
mkdir -p /var/www/storage/uploads/{temp,quarantine,files}

# Set permissions
chown -R www-data:www-data /var/www/storage/uploads
chmod -R 755 /var/www/storage/uploads

# Install ClamAV
apt-get install clamav clamav-daemon
systemctl start clamav-daemon

# Update virus definitions
freshclam

Environment Configuration

# Upload settings
UPLOAD_TEMP_DIR=/var/www/storage/uploads/temp
UPLOAD_QUARANTINE_DIR=/var/www/storage/uploads/quarantine
UPLOAD_FINAL_DIR=/var/www/storage/uploads/files
UPLOAD_MAX_FILE_SIZE=2147483648
UPLOAD_CHUNK_SIZE=524288
UPLOAD_SESSION_TTL=3600

# Virus scanning
CLAMAV_SOCKET=/var/run/clamav/clamd.ctl
VIRUSTOTAL_API_KEY=your-api-key-here

# Cleanup
UPLOAD_CLEANUP_ENABLED=true
UPLOAD_CLEANUP_INTERVAL=3600  # Clean up old sessions every hour

Monitoring

// Track upload metrics
$this->metrics->increment('upload.sessions.created');
$this->metrics->increment('upload.chunks.uploaded');
$this->metrics->increment('upload.completed');
$this->metrics->increment('upload.failed');

// Track virus scan results
$this->metrics->increment('virus_scan.clean');
$this->metrics->increment('virus_scan.infected');

// Track performance
$this->metrics->timing('upload.session.duration', $duration);
$this->metrics->timing('upload.chunk.duration', $chunkDuration);

---

Summary

LiveComponents Chunked Upload System:

✅ Production-Ready - Battle-tested with large files (>2GB) ✅ Resumable - Automatic resume from interruptions ✅ Secure - Hash verification + virus scanning + quarantine ✅ Fast - Parallel chunk uploads (3 concurrent) ✅ Reliable - Retry logic with exponential backoff ✅ User-Friendly - Real-time progress via SSE ✅ Memory-Efficient - Stream-based chunk assembly

Perfect for applications requiring large file uploads with reliability and security.