michaelschiemer/docs/deployment/secrets-management.md

Secrets Management with Vault

Comprehensive documentation for secure secrets management using the framework's Vault system for production deployment.

Overview

The Custom PHP Framework includes a fully-featured Vault system for secure secrets storage with:

• Libsodium Authenticated Encryption (XSalsa20-Poly1305)
• Database-backed Storage with encrypted values
• Audit Logging for all operations (read, write, delete, rotate)
• Key Rotation Support for security compliance
• CLI Commands for production management

Architecture

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│  Application    │───▶│  Vault Service  │───▶│  PostgreSQL     │
│  (Get/Set)      │    │  (Encryption)   │    │  (Encrypted)    │
└─────────────────┘    └─────────────────┘    └─────────────────┘
         │                      │                       │
         │                      ▼                       ▼
         │            ┌─────────────────┐    ┌─────────────────┐
         └───────────▶│  Audit Logger   │───▶│  vault_audit    │
                      │  (All Access)   │    │  (Audit Log)    │
                      └─────────────────┘    └─────────────────┘

Security Features

1. Authenticated Encryption

• Algorithm: Libsodium sodium_crypto_secretbox (XSalsa20-Poly1305)
• Key Size: 256-bit (32 bytes)
• Nonce: 192-bit (24 bytes), unique per encryption
• MAC: Poly1305 authentication tag
• Benefits: Encryption + Authentication in one operation, tamper-proof

2. Audit Logging

Every Vault operation is logged:

• Actions: READ, WRITE, DELETE, ROTATE
• Metadata: Timestamp, IP Address, User Agent, User ID
• Status: Success/Failure with error messages
• Usage Tracking: Access count, last accessed timestamp

3. Secure Key Management

• Generation: Cryptographically secure via sodium_crypto_secretbox_keygen()
• Storage: .env.production file (NOT committed to git)
• Rotation: Re-encrypt all secrets with new key
• Backup: Old keys retained until rotation verified

Installation & Setup

1. Generate Encryption Key

# Generate new Vault encryption key
docker exec php php console.php vault:generate-key

# Output:
# 🔐 Generated Vault Encryption Key:
# VAULT_ENCRYPTION_KEY=<base64-encoded-key>

2. Configure Environment

Add to .env.production:

# Vault Configuration
VAULT_ENCRYPTION_KEY=<base64-encoded-key-from-step-1>

CRITICAL:

• NEVER commit .env.production to version control
• Store encryption key in secure password manager as backup
• Losing key = losing ALL secrets permanently

3. Create Vault Tables

# Run Vault migration
docker exec php php console.php db:migrate

# Verifies:
# - vault_secrets table created
# - vault_audit table created

4. Test Vault

# Store test secret
docker exec php php console.php vault:set test_key test_value

# Retrieve test secret
docker exec php php console.php vault:get test_key

# Output should show: test_value

CLI Commands

vault:set - Store Secret

# With value in command
docker exec php php console.php vault:set api_key "sk_live_abc123"

# Interactive (password prompt - more secure)
docker exec php php console.php vault:set api_key
# Prompt: Enter secret value: ****

Output:

✅ Secret 'api_key' stored successfully in vault

vault:get - Retrieve Secret

docker exec php php console.php vault:get api_key

Output:

✅ Secret 'api_key' retrieved:

sk_live_abc123

🔒 This value should be kept secure!

vault:list - List All Secrets

docker exec php php console.php vault:list

Output:

🔐 Vault Secrets:

• api_key
  Accessed: 5 times, Last: 2024-01-15 14:32:45
• database_password
  Accessed: 12 times, Last: 2024-01-15 14:30:12
• oauth_client_secret
  Accessed: Never

Total: 3 secrets

vault:delete - Delete Secret

docker exec php php console.php vault:delete old_api_key

Output:

⚠️  Delete secret 'old_api_key'? (yes/no): yes
✅ Secret 'old_api_key' deleted successfully

vault:audit - View Audit Log

# Show last 50 audit entries (default)
docker exec php php console.php vault:audit

# Show last 100 entries
docker exec php php console.php vault:audit 100

# Show audit log for specific secret
docker exec php php console.php vault:audit 50 api_key

Output:

📊 Vault Audit Log:

✓ [2024-01-15 14:32:45] read - api_key
  IP: 203.0.113.42
✓ [2024-01-15 14:30:12] write - database_password
  User: admin, IP: 203.0.113.42
✗ [2024-01-15 14:28:01] read - missing_key
  IP: 203.0.113.42
  Error: Secret not found

Showing 3 entries

vault:rotate-key - Rotate Encryption Key

docker exec php php console.php vault:rotate-key

Output:

⚠️  KEY ROTATION - CRITICAL OPERATION

This will re-encrypt all secrets with a new key.
Make sure you have a backup before proceeding!

Continue with key rotation? (yes/no): yes

🔄 Rotating encryption key...

✅ Successfully rotated 15 secrets

New encryption key:
VAULT_ENCRYPTION_KEY=<new-base64-encoded-key>

🚨 IMPORTANT:
• Update VAULT_ENCRYPTION_KEY in your .env file immediately
• Restart your application to use the new key
• Keep the old key safe until rotation is verified

Application Integration

Basic Usage

use App\Framework\Vault\Vault;
use App\Framework\Vault\ValueObjects\SecretKey;
use App\Framework\Vault\ValueObjects\SecretValue;

final readonly class PaymentService
{
    public function __construct(
        private Vault $vault
    ) {}

    public function processPayment(Order $order): PaymentResult
    {
        // Retrieve Stripe API key from Vault
        $apiKey = $this->vault->get(SecretKey::from('stripe_api_key'));

        // Use secret (revealed only when needed)
        $stripe = new StripeClient($apiKey->reveal());

        // Process payment
        return $stripe->charge($order->getTotal());
    }
}

Storing Secrets

// Store new secret
$this->vault->set(
    SecretKey::from('shopify_access_token'),
    new SecretValue('shpat_abc123xyz...')
);

// Update existing secret (same method)
$this->vault->set(
    SecretKey::from('shopify_access_token'),
    new SecretValue('shpat_new_token...')
);

Checking Secret Existence

if ($this->vault->has(SecretKey::from('api_key'))) {
    // Secret exists
    $apiKey = $this->vault->get(SecretKey::from('api_key'));
} else {
    // Secret missing
    throw new ConfigurationException('API key not configured in Vault');
}

Deleting Secrets

// Delete deprecated secret
$this->vault->delete(SecretKey::from('old_api_key'));

Metadata & Audit

$metadata = $this->vault->getMetadata(SecretKey::from('api_key'));

// Metadata includes:
// - createdAt: DateTimeImmutable
// - updatedAt: DateTimeImmutable
// - accessCount: int
// - lastAccessedAt: ?DateTimeImmutable
// - createdBy: ?string
// - updatedBy: ?string

echo "Secret accessed {$metadata->accessCount} times";
echo "Last access: {$metadata->lastAccessedAt->format('Y-m-d H:i:s')}";

Production Deployment Workflow

Initial Production Setup

# 1. Generate encryption key
docker exec php php console.php vault:generate-key

# 2. Add to .env.production
echo "VAULT_ENCRYPTION_KEY=<generated-key>" >> .env.production
chmod 600 .env.production

# 3. Run migrations
docker exec php php console.php db:migrate

# 4. Store production secrets
docker exec php php console.php vault:set stripe_api_key
# Enter: sk_live_...

docker exec php php console.php vault:set database_password
# Enter: <strong-production-password>

docker exec php php console.php vault:set redis_password
# Enter: <redis-production-password>

# 5. Verify secrets stored
docker exec php php console.php vault:list

Migration from .env to Vault

# Script to migrate secrets from .env to Vault
#!/bin/bash

# Secrets to migrate (add your secrets here)
SECRETS=(
    "STRIPE_API_KEY"
    "SHOPIFY_ACCESS_TOKEN"
    "RAPIDMAIL_PASSWORD"
    "OAUTH_CLIENT_SECRET"
)

for secret in "${SECRETS[@]}"; do
    # Get value from .env
    VALUE=$(grep "^${secret}=" .env.production | cut -d '=' -f2-)

    if [ -n "$VALUE" ]; then
        echo "Migrating ${secret}..."
        # Store in Vault
        docker exec php php console.php vault:set "${secret}" "${VALUE}"

        # Comment out in .env (keep for reference)
        sed -i "s/^${secret}=/# MIGRATED_TO_VAULT: ${secret}=/" .env.production
    fi
done

echo "✅ Migration complete"
echo "🔒 Update application to use Vault::get() instead of Environment::get()"

Key Rotation Schedule

Recommended Schedule:

• Quarterly: Rotate Vault encryption key
• Annually: Rotate all application secrets (API keys, passwords)
• On Demand: Rotate if key compromise suspected

Rotation Procedure:

# 1. Backup current Vault
docker exec db pg_dump -U postgres michaelschiemer_prod -t vault_secrets > vault_backup_$(date +%Y%m%d).sql

# 2. Rotate encryption key
docker exec php php console.php vault:rotate-key

# 3. Update .env.production with new key
# (shown in command output)

# 4. Restart application
docker-compose -f docker-compose.yml \
               -f docker-compose.production.yml \
               --env-file .env.production \
               restart php

# 5. Verify all secrets accessible
docker exec php php console.php vault:list

# 6. Test application functionality
curl https://your-domain.com/health

Security Best Practices

1. Key Management

DO:

• ✅ Generate key with vault:generate-key command
• ✅ Store key in .env.production (not committed)
• ✅ Backup key in secure password manager
• ✅ Use different keys per environment (dev, staging, prod)
• ✅ Rotate keys quarterly

DON'T:

• ❌ Commit .env.production to version control
• ❌ Share keys via email or Slack
• ❌ Use same key across environments
• ❌ Store key in plain text files
• ❌ Log decrypted secret values

2. Secret Storage

DO:

• ✅ Store ALL sensitive credentials in Vault
• ✅ Use descriptive secret keys (e.g., stripe_live_api_key)
• ✅ Document which secrets are required
• ✅ Use Vault for OAuth tokens, API keys, passwords
• ✅ Set secrets via CLI (not in code)

DON'T:

• ❌ Store secrets in .env files
• ❌ Hardcode secrets in application code
• ❌ Store secrets in database without Vault
• ❌ Share secrets between environments
• ❌ Use weak or guessable secret values

3. Access Control

DO:

• ✅ Monitor audit log regularly (vault:audit)
• ✅ Review secret access patterns
• ✅ Investigate failed access attempts
• ✅ Track who accessed which secrets
• ✅ Revoke secrets on employee offboarding

DON'T:

• ❌ Ignore audit log warnings
• ❌ Share Vault CLI access broadly
• ❌ Allow anonymous secret access
• ❌ Disable audit logging
• ❌ Reuse secrets across services

4. Disaster Recovery

Backup Strategy:

# Weekly backup of Vault tables
0 2 * * 0 docker exec db pg_dump -U postgres michaelschiemer_prod -t vault_secrets -t vault_audit | gzip > /mnt/backups/vault_$(date +\%Y\%m\%d).sql.gz

Recovery Procedure:

# 1. Restore Vault tables from backup
gunzip -c vault_20240115.sql.gz | docker exec -i db psql -U postgres michaelschiemer_prod

# 2. Verify encryption key in .env.production
grep VAULT_ENCRYPTION_KEY .env.production

# 3. Test Vault access
docker exec php php console.php vault:list

# 4. Verify secret retrieval
docker exec php php console.php vault:get <test-key>

Monitoring & Alerting

Health Checks

// Include in /health endpoint
final readonly class VaultHealthCheck
{
    public function check(): HealthCheckResult
    {
        try {
            // Test Vault connectivity
            $testKey = SecretKey::from('_health_check_test');

            if ($this->vault->has($testKey)) {
                return HealthCheckResult::healthy('Vault');
            }

            // Create test secret if not exists
            $this->vault->set($testKey, new SecretValue('healthy'));

            return HealthCheckResult::healthy('Vault');
        } catch (\Throwable $e) {
            return HealthCheckResult::unhealthy('Vault', $e->getMessage());
        }
    }
}

Audit Log Monitoring

# Monitor failed access attempts
docker exec php php console.php vault:audit 100 | grep "✗"

# Alert on suspicious patterns
# - Multiple failed reads for same key
# - Access from unusual IPs
# - High-frequency access attempts

Metrics Collection

// Collect Vault metrics
final readonly class VaultMetricsCollector
{
    public function collect(): array
    {
        return [
            'total_secrets' => count($this->vault->all()),
            'failed_access_24h' => $this->auditLogger->countFailedAccess(Duration::fromHours(24)),
            'most_accessed_secrets' => $this->auditLogger->getMostAccessedSecrets(10),
            'last_rotation' => $this->getLastRotationTimestamp(),
        ];
    }
}

Troubleshooting

Problem: Vault Not Available

Symptoms:

❌ Vault not available. Make sure VAULT_ENCRYPTION_KEY is set in .env

Solution:

# 1. Check .env.production
grep VAULT_ENCRYPTION_KEY .env.production

# 2. Generate key if missing
docker exec php php console.php vault:generate-key

# 3. Add to .env.production and restart
docker-compose -f docker-compose.yml \
               -f docker-compose.production.yml \
               restart php

Problem: Decryption Failed

Symptoms:

❌ Failed to retrieve secret: Decryption failed

Causes:

• Wrong encryption key in .env.production
• Corrupted database entry
• Key was rotated but .env not updated

Solution:

# 1. Verify encryption key matches what was used during encryption
# 2. Check vault_audit for rotation events
docker exec php php console.php vault:audit 100 | grep ROTATE

# 3. If key mismatch, restore correct key or rotate all secrets

Problem: Secret Not Found

Symptoms:

❌ Secret 'api_key' not found in vault

Solution:

# 1. List all secrets
docker exec php php console.php vault:list

# 2. Check audit log for deletions
docker exec php php console.php vault:audit 100 api_key

# 3. Re-create secret
docker exec php php console.php vault:set api_key

Problem: High Access Count

Symptoms: Secret accessed thousands of times per hour

Investigation:

# 1. Check metadata
docker exec php php console.php vault:list

# 2. Review audit log
docker exec php php console.php vault:audit 1000 <secret-key>

# 3. Identify access pattern (IP, User Agent)

Solution:

• Cache secret value in application memory (careful!)
• Review application code for inefficient Vault access
• Consider moving to environment variable if accessed frequently

See Also

• Environment Configuration: docs/deployment/env-production-template.md
• Production Prerequisites: docs/deployment/production-prerequisites.md
• Database Patterns: docs/claude/database-patterns.md
• Security Patterns: docs/claude/security-patterns.md