michaelschiemer/docs/components/waf/feedback.md

WAF Feedback System

The Web Application Firewall (WAF) Feedback System allows users to provide feedback on WAF detections, helping to improve the accuracy of the security system over time. This document explains how to use the feedback system, how it works, and how to integrate it into your application.

Table of Contents

1. User Guide
   • Submitting Feedback
   • Types of Feedback
   • Feedback Dashboard
2. Technical Documentation
   • Architecture
   • Components
   • Integration
3. Learning Algorithms
   • False Positive Reduction
   • False Negative Reduction
   • Severity Adjustment
4. Examples
   • API Examples
   • Dashboard Examples
   • Learning Examples

User Guide

Submitting Feedback

When the WAF detects a potential security threat, it may block the request or display a warning. Users with appropriate permissions can provide feedback on these detections to help improve the system's accuracy.

To submit feedback:

1. Navigate to the security event in the admin dashboard
2. Click on the "Provide Feedback" button
3. Select the type of feedback (false positive, false negative, correct detection, or severity adjustment)
4. Add an optional comment explaining your feedback
5. Click "Submit"

Alternatively, you can use the API to submit feedback programmatically (see API Examples).

Types of Feedback

The WAF Feedback System supports four types of feedback:

1. False Positive: The WAF incorrectly identified a legitimate request as a security threat. This feedback helps reduce false alarms.

2. False Negative: The WAF failed to detect a security threat. This feedback helps improve detection coverage.

3. Correct Detection: The WAF correctly identified a security threat. This feedback confirms the accuracy of the detection.

4. Severity Adjustment: The WAF correctly identified a security threat, but the severity level was incorrect. This feedback helps calibrate the severity assessment.

Feedback Dashboard

The Feedback Dashboard provides an overview of all feedback submitted to the WAF system, along with analytics and trends. To access the dashboard:

1. Navigate to Admin > Security > WAF > Feedback
2. Use the filters to view specific types of feedback, categories, or time periods
3. View charts and graphs showing feedback trends and detection accuracy
4. Access detailed reports for specific detection categories

Technical Documentation

Architecture

The WAF Feedback System consists of several components that work together to collect, store, analyze, and learn from user feedback:

+-------------------+     +-------------------+     +-------------------+
|                   |     |                   |     |                   |
|  Feedback API     | --> |  Feedback Service | --> | Feedback Storage  |
|                   |     |                   |     |                   |
+-------------------+     +-------------------+     +-------------------+
                                   |
                                   v
+-------------------+     +-------------------+     +-------------------+
|                   |     |                   |     |                   |
|  Learning Service | <-- |  Analytics        | --> |  Reporting        |
|                   |     |                   |     |                   |
+-------------------+     +-------------------+     +-------------------+
         |
         v
+-------------------+
|                   |
|  ML Engine        |
|                   |
+-------------------+

Components

The WAF Feedback System includes the following components:

1. Feedback API: Provides endpoints for submitting and retrieving feedback.

   • POST /api/security/waf/feedback: Submit feedback
   • GET /api/security/waf/feedback/{detectionId}: Get feedback for a specific detection
   • GET /api/security/waf/feedback/stats: Get feedback statistics
   • GET /api/security/waf/feedback/recent: Get recent feedback

2. Feedback Service: Handles the business logic for feedback submission and retrieval.

   • FeedbackService: Core service for handling feedback
   • DetectionFeedback: Value object representing feedback
   • FeedbackType: Enum of feedback types

3. Feedback Storage: Stores feedback data for later analysis.

   • FeedbackRepositoryInterface: Interface for feedback storage
   • DatabaseFeedbackRepository: Database implementation

4. Learning Service: Analyzes feedback and adjusts the WAF models.

   • FeedbackLearningService: Core service for learning from feedback
   • ModelAdjustment: Value object representing model adjustments

5. Analytics: Analyzes feedback data to identify patterns and trends.

   • FeedbackReportGenerator: Generates reports from feedback data

6. Reporting: Provides visualizations and reports on feedback data.

   • WafFeedbackDashboardController: Controller for the feedback dashboard

7. ML Engine: Machine learning engine that uses feedback to improve detection.

   • MachineLearningEngine: Core ML engine
   • ThresholdAdjustableInterface: Interface for adjustable thresholds
   • ConfidenceAdjustableInterface: Interface for adjustable confidence
   • FeatureWeightAdjustableInterface: Interface for adjustable feature weights

Integration

To integrate the WAF Feedback System into your application:

1. Register Services: Register the feedback services in your dependency injection container:

// Register feedback services
$container->singleton(FeedbackRepositoryInterface::class, DatabaseFeedbackRepository::class);
$container->singleton(FeedbackService::class, FeedbackService::class);
$container->singleton(FeedbackLearningService::class, FeedbackLearningService::class);
$container->singleton(FeedbackReportGenerator::class, FeedbackReportGenerator::class);

2. Register Controllers: Register the feedback controllers:

// Register feedback controllers
$container->singleton(WafFeedbackController::class, WafFeedbackController::class);
$container->singleton(WafFeedbackDashboardController::class, WafFeedbackDashboardController::class);

3. Schedule Learning: Schedule the learning process to run periodically:

// Schedule learning process to run daily
$scheduler->daily('waf:learn-from-feedback');

4. Add UI Components: Add feedback UI components to your application:

// Add feedback button to WAF detection alerts
$alertComponent->addAction('Provide Feedback', '/admin/security/waf/feedback/submit/{detectionId}');

Learning Algorithms

The WAF Feedback System uses several learning algorithms to improve detection accuracy based on user feedback.

False Positive Reduction

When users report false positives, the system adjusts the detection models to reduce the likelihood of similar false positives in the future:

1. Threshold Adjustment: Increases the detection threshold for the affected category, making it less sensitive.
2. Confidence Reduction: Decreases the confidence score for similar detections, making them less likely to trigger alerts.
3. Feature Weight Adjustment: Reduces the weights of features that contributed to the false positive.

False Negative Reduction

When users report false negatives, the system adjusts the detection models to improve detection coverage:

1. Threshold Adjustment: Decreases the detection threshold for the affected category, making it more sensitive.
2. Confidence Increase: Increases the confidence score for similar detections, making them more likely to trigger alerts.
3. Feature Weight Adjustment: Increases the weights of features that should have contributed to detection.

Severity Adjustment

When users report incorrect severity levels, the system adjusts the severity assessment:

1. Confidence Adjustment: Adjusts the confidence score based on the severity change.
2. Category Default Severity: Updates the default severity for the detection category.

Examples

API Examples

Submitting Feedback

// Example: Submit false positive feedback
fetch('/api/security/waf/feedback', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'X-CSRF-TOKEN': csrfToken
  },
  body: JSON.stringify({
    detection_id: 'abc123',
    feedback_type: 'false_positive',
    category: 'SQL_INJECTION',
    severity: 'HIGH',
    comment: 'This is a legitimate query used by our reporting system',
    context: {
      query: 'SELECT * FROM reports WHERE date > ?'
    }
  })
})
.then(response => response.json())
.then(data => console.log(data));

Getting Feedback Statistics

// Example: Get feedback statistics
fetch('/api/security/waf/feedback/stats')
.then(response => response.json())
.then(data => {
  console.log('Total feedback:', data.stats.total_count);
  console.log('False positives:', data.stats.by_feedback_type.false_positive);
  console.log('False negatives:', data.stats.by_feedback_type.false_negative);
});

Dashboard Examples

Viewing Feedback Trends

The Feedback Dashboard provides visualizations of feedback trends over time:

• Accuracy Trend: Shows how detection accuracy has improved over time
• False Positive Rate: Shows how the false positive rate has decreased
• False Negative Rate: Shows how the false negative rate has decreased
• Category Breakdown: Shows feedback distribution by detection category

Analyzing Detection Categories

The Category Analysis view shows detailed information about each detection category:

• Accuracy: Detection accuracy for the category
• False Positive Rate: False positive rate for the category
• False Negative Rate: False negative rate for the category
• Severity Distribution: Distribution of severity levels for the category
• Common Patterns: Common patterns in false positives and false negatives

Learning Examples

Scheduled Learning

The learning process can be scheduled to run periodically:

# Run learning process daily
php console.php waf:learn-from-feedback

Manual Learning

The learning process can also be triggered manually:

# Run learning process with custom parameters
php console.php waf:learn-from-feedback --threshold=10 --learning-rate=0.5

Learning Results

The learning process provides detailed results:

Starting WAF feedback learning process...
Processing 25 false positives, 10 false negatives, 5 severity adjustments
Created 3 model adjustments for SQL_INJECTION, XSS, COMMAND_INJECTION
Applied 3 model adjustments
Learning process completed successfully in 0.5 seconds