Staging environment should not expose detailed error messages,
stack traces, or debug information to end users.
Changed default from 'true' to 'false' in all services:
- php
- nginx
- queue-worker
- scheduler
Shell variables like $SECRETS_DIR in docker-compose command blocks
must be escaped as $$SECRETS_DIR. Without escaping, docker-compose
interprets them as environment variable interpolation and expands
them to empty strings, causing:
- mkdir: cannot create directory ''
- Secrets copied to wrong path (/redis_password instead of /var/www/html/storage/secrets/redis_password)
- PHP TypeError: RedisConfig::__construct() argument #3 must be string, null given
The fix applies $$ escaping to all shell variables in the PHP
service entrypoint script.
- Add explicit sed pattern for production-php:9000 → php:9000
- Fix character class [a-f0-9_]* to [a-zA-Z0-9_-]* to match full container names
- Loop over both sites-enabled and sites-available configs
- Add fastcgi_pass replacement for production-php
The --force-recreate flag alone doesn't handle containers that exist
outside the compose project context. Now explicitly:
1. Run docker compose down first
2. Force remove any orphaned containers with known names
3. Then create fresh containers
Fixes deployment error where existing containers with same name
blocked recreation. This ensures clean deployments by:
- Force recreating containers even if unchanged
- Removing orphan containers not in compose file
The Gitea Actions Runner doesn't have Node.js installed, causing
actions/checkout@v3 (a JavaScript action) to fail with
"Cannot find: node in PATH".
Replace with native shell-based git checkout that works without
Node.js and uses Gitea context variables for repository URL.
- Fix Enter key detection: handle multiple Enter key formats (\n, \r, \r\n)
- Reduce flickering: lower render frequency from 60 FPS to 30 FPS
- Fix menu bar visibility: re-render menu bar after content to prevent overwriting
- Fix content positioning: explicit line positioning for categories and commands
- Fix line shifting: clear lines before writing, control newlines manually
- Limit visible items: prevent overflow with maxVisibleCategories/Commands
- Improve CPU usage: increase sleep interval when no events processed
This fixes:
- Enter key not working for selection
- Strong flickering of the application
- Menu bar not visible or being overwritten
- Top half of selection list not displayed
- Lines being shifted/misaligned
- Fix TuiRenderer rendering: correct line positioning for categories
- Fix category item formatting: remove tabs, ensure consistent spacing
- Improve clearContentArea: preserve menu bar (lines 2-3) when clearing content
- Add ConsoleContext: mutable context container for readonly ConsoleOutput
- Add context awareness to ConsoleOutput: setContext/getContext/isInTuiContext
- Auto-detect TUI context in InteractivePrompter: automatically set LayoutAreas
- Set TUI context in TuiFactory and TuiCommandExecutor
- Add tests for TuiRenderer: menu bar preservation, category formatting
This fixes rendering issues where:
- Menu bar was not displayed or overwritten
- Category items had tab/space misalignment
- Content area clearing overwrote the menu bar
- Add comprehensive test suite for CMS and Asset modules using Pest Framework
- Implement ContentTypeService::delete() protection against deletion of in-use content types
- Add CannotDeleteContentTypeInUseException for better error handling
- Fix DerivatPipelineRegistry::getAllPipelines() to handle object uniqueness correctly
- Fix VariantName::getScale() to correctly parse scales with file extensions
- Update CMS module documentation with new features, exceptions, and test coverage
- Add CmsTestHelpers and AssetTestHelpers for test data factories
- Fix BlockTypeRegistry to be immutable after construction
- Update ContentTypeService to check for associated content before deletion
- Improve BlockRendererRegistry initialization
Test coverage:
- Value Objects: All CMS and Asset value objects
- Services: ContentService, ContentTypeService, SlugGenerator, BlockValidator, ContentLocalizationService, AssetService, DeduplicationService, MetadataExtractor
- Repositories: All database repositories with mocked connections
- Rendering: Block renderers and ContentRenderer
- Controllers: API endpoints for both modules
254 tests passing, 38 remaining (mostly image processing pipeline tests)
- Add LayoutAreas and LayoutArea value objects for coordinated screen rendering
- Add ScreenRendererInterface for testable screen operations
- Extend ScreenManager with clearContentArea() for selective clearing
- Refactor InteractiveMenu to support LayoutAreas via setLayoutAreas()
- Add prepareScreen() method that handles both standalone and layout-aware modes
- Fix cursor positioning to prevent menu bar overwriting
- Add comprehensive tests for layout areas and rendering behavior
This fixes rendering issues where InteractiveMenu would overwrite the menu bar
and cause misalignment of menu items when used within TUI layouts.
- Move all Gitea configuration from docker-compose.yml environment variables to app.ini
- Enable Redis cache with proper connection string format (redis://)
- Fix Redis password to use Gitea Redis instance password (gitea_redis_password) instead of application Redis stack password
- Add database connection pool settings to prevent timeout errors
- Configure Redis for cache, session, and queue using app.ini
- Update Ansible task to use correct Redis password for Gitea Redis instance
Benefits:
- Cache now works correctly (environment variables had a bug in Gitea 1.25)
- All settings are versioned in Git
- Better maintainability and reliability
- Configuration follows Gitea documentation recommendations
- Runtime-base job now uses repository artifact instead of cloning (saves 1 git clone per run)
- Health checks are now optional via workflow_dispatch input (default: true)
- Health checks still run automatically on push events
- Reduces additional load on Gitea by ~10-15%
- Use repository artifacts in test and build jobs (reduces 2-3 git clones per run)
- Add comprehensive workflow performance monitoring system
- Add monitoring playbook and Gitea workflow for automated metrics collection
- Add monitoring documentation and scripts
Optimizations:
- Repository artifact caching: changes job uploads repo, test/build jobs download it
- Reduces Gitea load by eliminating redundant git operations
- Faster job starts (artifact download is typically faster than git clone)
Monitoring:
- Script for local workflow metrics collection via Gitea API
- Ansible playbook for server-side system and Gitea metrics
- Automated Gitea workflow that runs every 6 hours
- Tracks workflow durations, system load, Gitea API response times, and more
- Replace http://localhost:8080/api/http/services with traefik show providers docker
- Replace http://localhost:8080/api/http/routers with traefik show providers docker
- Update debug messages to reference CLI command instead of HTTP API
- Fixes false 'NOT_FOUND' errors since api.insecure: false is set in traefik.yml
The Traefik CLI (traefik show providers docker) works without credentials
and is the recommended method for Traefik v3. It shows all Docker providers
including services, routers, and middlewares, so Gitea will be visible if
registered correctly.
- Fix YAML parsing error by quoting task name with colon
- Add PostgreSQL Staging Stack check and auto-start for staging deployments
- Ensures postgres-staging-internal network is created by the stack itself
- Network creation remains as fallback if stack doesn't create them
- Improves deployment reliability by ensuring dependencies are available
This addresses the root cause: PostgreSQL Staging Stack should be running
before the application stack tries to use its network.
- Extract external networks from docker-compose.base.yml and compose override files
- Extract network names from 'name:' fields when external: true
- Create all required external networks before docker compose up
- Fixes error: 'network postgres-staging-internal declared as external, but could not be found'
This ensures all external networks (traefik-public, app-internal, postgres-staging-internal, etc.)
are created before attempting to start containers.
- fix-gitea-timeouts.yml: Add when conditions to wait_for and uri tasks
- Wait for Traefik only if traefik_restart.changed
- Wait for Gitea via Traefik only if traefik_restart or gitea_restart changed
- fix-gitea-complete.yml: Same fixes as fix-gitea-timeouts.yml
- Wait for Traefik only if traefik_restart.changed
- Wait for Gitea and service discovery checks only if restart occurred
- fix-gitea-traefik-connection.yml: Fix wait and test tasks
- Register traefik_restart to track if restart happened
- Wait for Traefik only if traefik_restart.changed
- Test Gitea via Traefik only if traefik_restart.changed
- Update message to reflect actual restart status
- update-gitea-traefik-service.yml: Fix pause block
- Register traefik_restart to track if restart happened
- Wait for Traefik only if traefik_restart.changed
This prevents unnecessary blocking when traefik_auto_restart=false and
ensures wait/healthcheck tasks only run when a restart actually occurred.
- Replace docker_image module with shell command for more reliable pulling
- Add detailed error output from pull command (stdout/stderr)
- Show actual docker pull error messages when pull fails
- Simplify pull logic - always attempt pull regardless of local existence
- Check if image exists before pull to determine if force pull is needed
- Use docker images command to verify image exists locally (more reliable)
- Fix registries_to_login Jinja2 template to ensure it's always a list
- Add better error messages when image pull fails
- Only tag image if it was successfully verified to exist
- Add debug output for image pull process
- Improve error handling: verify image exists after pull before tagging
- Fix registries_to_login Jinja2 template to handle undefined variables
- Add explicit failure if image pull fails
- Only tag image if it was successfully pulled
- Change docker-compose.staging.yml: git.michaelschiemer.de:5000 -> localhost:5000
- Update deploy-image.yml playbook to:
- Pull images from registry.michaelschiemer.de (source registry)
- Tag and push to localhost:5000 (local registry) for local containers
- Remove hardcoded git.michaelschiemer.de:5000 logic
- Use local_registry from compose files for deployment
This ensures:
- Workflow pushes to registry.michaelschiemer.de (external, HTTPS)
- Containers use localhost:5000 (local, faster, no HTTPS overhead)
- Consistent registry usage across staging and production
The playbook was using docker_registry (registry.michaelschiemer.de) but
docker-compose.staging.yml uses git.michaelschiemer.de:5000. Now the playbook:
- Extracts the actual registry URL from docker-compose files
- Uses that registry for deploy_image
- Updates docker-compose file with the correct registry
This ensures the image is pulled from and deployed to the correct registry.
The previous regex matched 'redis:7-alpine' because it saw ':7' as a port.
New regex requires:
- TLD with optional port (e.g. git.michaelschiemer.de:5000)
- Hostname with numeric port only (e.g. localhost:5000)
- localhost with optional port
This excludes image names like 'redis:7-alpine' or 'minio/minio:latest'.
Ansible doesn't allow .append() on lists in Jinja2 templates.
Changed to use select() filter with regex match instead of loop with append.
This is the same filtering logic but using immutable operations.
The previous regex was removing port numbers from registry URLs.
Now using sed to only remove the image name part after the slash,
preserving the full registry URL including port (e.g. git.michaelschiemer.de:5000)
Registry Login Fixes:
- Filter out service names (minio, redis) from registry URL extraction
- Only recognize actual registry URLs (with TLD or port)
- Preserve port numbers in registry URLs (e.g. git.michaelschiemer.de:5000)
- Better error messages for failed logins
Traefik Restart Loop Prevention:
- Set traefik_auto_restart default to false in traefik role
- Add traefik_auto_restart, traefik_ssl_restart, gitea_auto_restart to staging vars
- Add guard to fix-gitea-traefik-connection.yml restart task
- Add guard and deprecation warning to update-gitea-traefik-service.yml
This ensures that:
- CI/CD pipelines won't cause Traefik restart loops
- Staging environment uses same safe defaults as production
- Deprecated playbooks fail by default unless explicitly enabled
- Only actual Docker registries are used for login, not service names
- Extract actual registry URLs from docker-compose files
- Login to all registries found in compose files (e.g. git.michaelschiemer.de:5000)
- This fixes the 'no basic auth credentials' error when pulling images
- The playbook now automatically detects which registry is used in compose files
- Falls back to docker_registry variable if no registry found in compose files
- Add traefik_auto_restart check to fix-gitea-timeouts.yml
- Add traefik_auto_restart check to fix-gitea-ssl-routing.yml
- Add traefik_auto_restart check to fix-gitea-complete.yml
- Set traefik_auto_restart=false in all Gitea workflow Ansible calls
- Set gitea_auto_restart=false in all Gitea workflow Ansible calls
- Add redeploy-traefik-gitea.yml playbook for clean redeployment
This prevents CI/CD pipelines from causing Traefik restart loops by
ensuring all remediation playbooks respect the traefik_auto_restart
flag, which is set to false in group_vars/production/vars.yml.
- Set traefik_auto_restart: false in group_vars to prevent automatic restarts after config deployment
- Set traefik_ssl_restart: false to prevent automatic restarts during SSL certificate setup
- Set gitea_auto_restart: false to prevent automatic restarts when healthcheck fails
- Modify traefik/tasks/ssl.yml to only restart if explicitly requested or acme.json was created
- Modify traefik/tasks/config.yml to respect traefik_auto_restart flag
- Modify gitea/tasks/restart.yml to respect gitea_auto_restart flag
- Add verify-traefik-fix.yml playbook to monitor Traefik stability
This fixes the issue where Traefik was restarting every minute due to
automatic restart mechanisms triggered by config deployments and health checks.
The restart loops caused 504 Gateway Timeouts for Gitea and other services.
Fixes: Traefik restart loop causing service unavailability
- Add deploy-traefik-config.yml to copy updated config files to server
- Deploys docker-compose.yml and traefik.yml
- Shows deployment status and next steps
- Required before restarting Traefik with new configuration
- Change health check to use docker exec traefik healthcheck
- HTTP ping endpoint requires BasicAuth (401), internal check is more reliable
- Improves health check accuracy in restart-traefik.yml playbook
- Add restart-traefik.yml playbook to restart Traefik container
- Verify Traefik health after restart
- Check for ACME challenge errors in logs
- Display status summary with next steps
- Useful after Traefik configuration changes
- Add explicit exclusion of /.well-known/acme-challenge from catch-all redirect
- Ensures ACME challenges are never redirected to HTTPS
- Traefik handles ACME challenges automatically, but explicit exclusion is safer
- Remove explicit ACME challenge router that had no service defined
- Traefik handles ACME challenges automatically when httpChallenge.entryPoint is set
- The router was interfering with automatic challenge handling
- Fixes 'Cannot retrieve the ACME challenge' errors in Traefik logs
- Increase fetch_interval from 2s to 10s to reduce load on Gitea
- Increase fetch_timeout from 5s to 30s for better error handling
- Add documentation about runner overloading Gitea and how to fix it
- Prevents 504 errors caused by runner bombarding Gitea with requests
- Add retry logic (5 retries, 10s delay) to git clone and update tasks
- Handle 504 Gateway Timeout errors from Gitea gracefully
- Fail with clear error message if all retries are exhausted
- Prevents workflow failures due to temporary Gitea unavailability
