Commit Graph

44 Commits

Author SHA1 Message Date
4586f0e9ee feat: further optimize workflows - runtime-base artifact and optional health checks
- Runtime-base job now uses repository artifact instead of cloning (saves 1 git clone per run)
- Health checks are now optional via workflow_dispatch input (default: true)
- Health checks still run automatically on push events
- Reduces additional load on Gitea by ~10-15%
2025-11-09 04:16:00 +01:00
72757954dc feat: optimize workflows with repository artifacts and add performance monitoring
Some checks failed
🚀 Build & Deploy Image / Determine Build Necessity (push) Failing after 33s
🚀 Build & Deploy Image / Build Runtime Base Image (push) Has been skipped
🚀 Build & Deploy Image / Build Docker Image (push) Has been skipped
Security Vulnerability Scan / Check for Dependency Changes (push) Successful in 32s
🚀 Build & Deploy Image / Run Tests & Quality Checks (push) Has been skipped
🚀 Build & Deploy Image / Auto-deploy to Staging (push) Has been skipped
🚀 Build & Deploy Image / Auto-deploy to Production (push) Has been skipped
Security Vulnerability Scan / Composer Security Audit (push) Has been skipped
🧊 Warm Docker Build Cache / Refresh Buildx Caches (push) Failing after 11s
📊 Monitor Workflow Performance / Monitor Workflow Performance (push) Failing after 20s
- Use repository artifacts in test and build jobs (reduces 2-3 git clones per run)
- Add comprehensive workflow performance monitoring system
- Add monitoring playbook and Gitea workflow for automated metrics collection
- Add monitoring documentation and scripts

Optimizations:
- Repository artifact caching: changes job uploads repo, test/build jobs download it
- Reduces Gitea load by eliminating redundant git operations
- Faster job starts (artifact download is typically faster than git clone)

Monitoring:
- Script for local workflow metrics collection via Gitea API
- Ansible playbook for server-side system and Gitea metrics
- Automated Gitea workflow that runs every 6 hours
- Tracks workflow durations, system load, Gitea API response times, and more
2025-11-09 04:03:51 +01:00
f548a0322c fix: prevent CI jobs from restarting Traefik
Some checks failed
🚀 Build & Deploy Image / Determine Build Necessity (push) Successful in 28s
Security Vulnerability Scan / Check for Dependency Changes (push) Successful in 35s
🚀 Build & Deploy Image / Build Runtime Base Image (push) Successful in 18s
🚀 Build & Deploy Image / Run Tests & Quality Checks (push) Has been skipped
Security Vulnerability Scan / Composer Security Audit (push) Has been skipped
🚀 Build & Deploy Image / Build Docker Image (push) Successful in 17s
🚀 Build & Deploy Image / Auto-deploy to Staging (push) Failing after 1m9s
🚀 Build & Deploy Image / Auto-deploy to Production (push) Has been skipped
- Add traefik_auto_restart check to fix-gitea-timeouts.yml
- Add traefik_auto_restart check to fix-gitea-ssl-routing.yml
- Add traefik_auto_restart check to fix-gitea-complete.yml
- Set traefik_auto_restart=false in all Gitea workflow Ansible calls
- Set gitea_auto_restart=false in all Gitea workflow Ansible calls
- Add redeploy-traefik-gitea.yml playbook for clean redeployment

This prevents CI/CD pipelines from causing Traefik restart loops by
ensuring all remediation playbooks respect the traefik_auto_restart
flag, which is set to false in group_vars/production/vars.yml.
2025-11-08 23:47:44 +01:00
48e5179bac fix: Correct deployment order - deploy image before composer install
Some checks failed
🚀 Build & Deploy Image / Determine Build Necessity (push) Successful in 31s
Security Vulnerability Scan / Check for Dependency Changes (push) Successful in 28s
Security Vulnerability Scan / Composer Security Audit (push) Has been skipped
🚀 Build & Deploy Image / Build Docker Image (push) Successful in 11s
🚀 Build & Deploy Image / Build Runtime Base Image (push) Successful in 11s
🚀 Build & Deploy Image / Run Tests & Quality Checks (push) Has been skipped
🚀 Build & Deploy Image / Auto-deploy to Staging (push) Failing after 52s
🚀 Build & Deploy Image / Auto-deploy to Production (push) Has been skipped
- Move deploy-image.yml before install-composer-dependencies.yml
- Containers must be running before composer can install dependencies
- Fixes 'container not running' error in composer install step
- Applied to both staging and production workflows
2025-11-08 15:35:50 +01:00
13d627d351 fix: Use php-ci runner for deploy jobs
Some checks failed
🚀 Build & Deploy Image / Determine Build Necessity (push) Successful in 57s
Security Vulnerability Scan / Check for Dependency Changes (push) Successful in 52s
🚀 Build & Deploy Image / Build Runtime Base Image (push) Successful in 10s
🚀 Build & Deploy Image / Run Tests & Quality Checks (push) Has been skipped
Security Vulnerability Scan / Composer Security Audit (push) Has been skipped
🚀 Build & Deploy Image / Build Docker Image (push) Successful in 15s
🚀 Build & Deploy Image / Auto-deploy to Staging (push) Failing after 40s
🚀 Build & Deploy Image / Auto-deploy to Production (push) Has been skipped
- Change deploy-staging and deploy-production to use php-ci runner
- php-ci image has Ansible pre-installed, fixing 'ansible-playbook: command not found' error
2025-11-08 14:50:25 +01:00
07e92a8709 fix: Install Ansible in docker-build image instead of runtime
Some checks failed
🚀 Build & Deploy Image / Determine Build Necessity (push) Successful in 44s
🚀 Build & Deploy Image / Build Runtime Base Image (push) Successful in 18s
🚀 Build & Deploy Image / Run Tests & Quality Checks (push) Failing after 2s
Security Vulnerability Scan / Composer Security Audit (push) Has been skipped
🚀 Build & Deploy Image / Build Docker Image (push) Failing after 44s
Security Vulnerability Scan / Check for Dependency Changes (push) Successful in 40s
🚀 Build & Deploy Image / Auto-deploy to Staging (push) Has been skipped
🚀 Build & Deploy Image / Auto-deploy to Production (push) Has been skipped
2025-11-08 13:54:01 +01:00
e9e87c9c5e fix: Replace apt-get with apk for Alpine-based docker-build container
Some checks failed
Security Vulnerability Scan / Check for Dependency Changes (push) Successful in 1m1s
🚀 Build & Deploy Image / Determine Build Necessity (push) Successful in 28s
🚀 Build & Deploy Image / Build Runtime Base Image (push) Successful in 10s
🚀 Build & Deploy Image / Run Tests & Quality Checks (push) Has been skipped
Security Vulnerability Scan / Composer Security Audit (push) Has been skipped
🚀 Build & Deploy Image / Build Docker Image (push) Successful in 16s
🚀 Build & Deploy Image / Auto-deploy to Staging (push) Failing after 38s
🚀 Build & Deploy Image / Auto-deploy to Production (push) Has been skipped
2025-11-08 13:45:18 +01:00
1963b10749 feat: Integrate Ansible playbooks into CI/CD workflows
- Add deploy-application-code.yml for Git-based code deployment
- Add install-composer-dependencies.yml for dependency installation
- Add deploy-image.yml for Docker image deployment
- Update build-image.yml to use Ansible playbooks
- Update manual-deploy.yml to use Ansible playbooks
- Add ANSIBLE_VAULT_PASSWORD secret handling
2025-11-07 18:14:11 +01:00
c1d6a71494 fix(deployment): cd to /workspace/repo before scp
The 'Deploy to Production Server' step is separate from the clone step and
doesn't inherit the working directory. The scp command was running from the
default directory where docker-compose files don't exist.

Adding 'cd /workspace/repo' before scp to access the cloned repository files.

This fixes the error:
docker-compose.base.yml: No such file or directory
docker-compose.production.yml: No such file or directory
2025-11-04 16:07:05 +01:00
6e1faabdc1 fix(deployment): deploy docker-compose files via scp before SSH execution
The previous 'always sync' approach failed because it tried to copy files
from /workspace/repo/ which doesn't exist on the production server.

The SSH heredoc (<<EOF) executes commands ON the production server, not in
the Gitea Actions workspace. File paths inside heredoc are relative to the
production server's filesystem.

This commit adds an scp step BEFORE the SSH heredoc to transfer
docker-compose.base.yml and docker-compose.production.yml from the Actions
workspace to the production server.

This ensures the build: null overrides (commit 2e539ed) reach production
and services can restart without build context errors.

Changes:
- Added scp command to deploy docker-compose files before SSH deployment
- Changed file sync check from 'cp' to file existence validation
- Updated comments to clarify rsync-based deployment architecture

Related commits:
- 0b342c6: Sequential push strategy
- 08f6f64: Stable git-SHA IMAGE_TAG
- 2e539ed: build: null overrides
- 0db73df: Always-sync docker-compose (incorrect implementation)
- 3091205: Trigger pipeline with source file change
2025-11-04 16:01:11 +01:00
0db73df2df fix(deployment): always sync docker-compose files from repo
The deployment script was only copying docker-compose files when missing,
preventing configuration updates (like build: null overrides) from being deployed.

Changed from conditional copy to always sync latest files from repository,
ensuring all configuration changes are properly deployed to production.
2025-11-04 15:43:02 +01:00
08f6f64d72 fix(ci): use stable git-SHA tag for deployment instead of timestamp
Changed IMAGE_TAG output from timestamp-based format to stable git-SHA format to ensure
deployment can reliably pull the image that was actually pushed to the registry.

Before: IMAGE_TAG="6c7040e-1762265632" (changes with time)
After: IMAGE_TAG="git-6c7040e" (stable, matches pushed tag)

This fixes deployment manifest not found errors.
2025-11-04 15:20:34 +01:00
0b342c68bb fix(ci): change docker build to load then push tags sequentially
Docker registry was getting overwhelmed with concurrent pushes of
multiple tags and cache layers, resulting in 499 status code
(Client Closed Request).

Changes:
- Build with --load instead of --push to save image locally first
- Push each tag sequentially (latest, timestamp, git-sha) instead of all at once
- Reduce cache targets from 2 to 1 (keep only buildcache)
- Add progress logging for each push operation

This approach:
1. Reduces concurrent write pressure on registry
2. Allows better error handling per tag
3. Provides clearer progress feedback
4. Prevents registry timeouts from concurrent uploads

Related to: Status 499 error during docker push
2025-11-04 15:04:57 +01:00
84a5a3fa21 chore(ci): update build workflow for targeted path triggers and concurrency management
- Refine branch and path filters for efficient CI triggers.
- Add concurrency control to avoid overlapping builds.
- Improve runtime base build logic with conditional evaluation and skipping.
- Enhance image info generation with fallback handling and deployment readiness checks.
2025-11-03 22:41:06 +01:00
7a2cb0b63e fix: staging deployment configuration and redis secrets handling 2025-11-03 00:15:43 +01:00
ff572534e9 ci: update deploy-staging job dependencies and conditions 2025-11-02 23:25:31 +01:00
7f3ce7f969 fix: update build workflow default branch to staging
- Change default branch from 'main' to 'staging'
- Add choice input type for branch selection
- Add debug check for RUNTIME_IMAGE_NAME
- Remove unused env variable
2025-11-02 23:12:18 +01:00
77c656af62 feat(deployment): update Semaphore stack and Traefik configuration
- Add QUICKSTART.md and SETUP_REPOSITORY.md for Semaphore stack
- Add playbooks directory for Semaphore deployment
- Update Semaphore docker-compose.yml, env.example, and README
- Add Traefik local configuration files
- Disable semaphore.yml in Traefik dynamic config
- Update docker-compose.local.yml and build-image workflow
2025-11-02 22:55:51 +01:00
a5cd49bde7 feat(deployment): update semaphore configuration and deployment workflows 2025-11-02 20:46:18 +01:00
24cbbccf4c feat: update deployment configuration and encrypted env loader
- Update Ansible playbooks and roles for application deployment
- Add new Gitea/Traefik troubleshooting playbooks
- Update Docker Compose configurations (base, local, staging, production)
- Enhance EncryptedEnvLoader with improved error handling
- Add deployment scripts (autossh setup, migration, secret testing)
- Update CI/CD workflows and documentation
- Add Semaphore stack configuration
2025-11-02 20:38:06 +01:00
f56d53d873 fix: add automatic nginx upstream fix for staging 502 errors
- Add post-deployment fix in build-image.yml workflow to automatically fix nginx upstream configuration
- nginx sites-available/default uses 127.0.0.1:9000 but PHP-FPM runs in staging-app container
- Fix runs automatically after each staging deployment
- Add troubleshooting documentation for staging 502 errors
- Add Ansible playbooks for quick fix and diagnosis
2025-11-02 02:29:12 +01:00
2defdf2baf chore: update staging branch with current changes 2025-11-02 00:05:26 +01:00
1f2ab358f9 ci: simplify stack path resolution in build-image workflow 2025-11-01 23:13:24 +01:00
25f05c6a17 ci: improve stack path handling in staging deployment
- Simplify local stack path normalization
- Enhance remote shell path resolution to handle user-specific paths (e.g. ~username/path)
- Better handling of escaped tilde paths in SSH heredoc
2025-11-01 23:06:54 +01:00
988e14f3c2 fix(ci): normalize stack path for staging deployment
Normalize stack path locally to avoid ~ confusion when passing to remote shell
2025-11-01 22:54:56 +01:00
ad0598d301 fix: improve stack path handling in staging deployment
- Fix user home directory detection for stack path expansion
- Handle tilde expansion and relative paths correctly
- Add fallback logic for path resolution
2025-11-01 22:47:07 +01:00
ff4b8dd848 fix(ci): expand tilde in staging deployment stack path 2025-11-01 22:32:01 +01:00
888b9cce7f ci: improve build job conditional execution in workflow
- Replace job-level if condition with env variable for better control
- Add conditional checks to all build steps
- Add skip step when build not required
- Simplify deploy-staging conditions
2025-11-01 22:23:18 +01:00
5ec5c41a0a fix(ci): improve image selection and registry handling in deployment workflow
- Add better image selection logic with fallback handling
- Support multiple registry logins for different image sources
- Improve error handling and image URL parsing
- Add proper argument escaping for SSH deployment script
2025-11-01 22:16:23 +01:00
fa28e3580a fix(ci): add image tag fallback logic and code style fixes
- Add fallback to 'latest' tag when build fails
- Add fallback mechanism when pulling specific image tag fails
- Fix code style: move opening brace in ObjectInfo
- Remove unused comment in ObjectStorage
2025-11-01 22:06:12 +01:00
477522bc1e feat: add ObjectStorage and ObjectInfo framework components
- Add ObjectStorage and ObjectInfo classes for framework storage
- Update build-image.yml workflow configuration
2025-11-01 21:53:23 +01:00
9ecc88a0eb ci: improve image tag handling and add staging to security scan 2025-11-01 21:34:40 +01:00
c51e09c6b6 fix(ci): Add runtime-base back to build job needs
- build job accesses needs.runtime-base.outputs.image_ref, so runtime-base must be in needs
- Added always() condition to ensure build runs even if runtime-base is skipped
- Outputs will be empty if runtime-base was skipped, which is handled gracefully
2025-11-01 21:23:08 +01:00
e17469084d fix(ci): Improve runtime-base image reference handling in build job
- Better handling of optional runtime-base outputs
- Added null check for runtime-base image_ref
- Improved logging when using freshly built runtime image
2025-11-01 21:22:26 +01:00
0c7b96e887 fix(ci): Fix workflow job dependencies and execution flow
- Removed runtime-base from build job dependencies (build now only depends on changes)
- runtime-base now only runs when needed (if needs_runtime_build or needs_build is true)
- build job now handles missing runtime-base outputs gracefully
- deploy-staging and deploy-production now only depend on build (not runtime-base)
- Added always() conditions to ensure jobs run even if previous jobs are skipped
- Fixes issue where workflow stopped after runtime-base job when build was skipped
2025-11-01 21:21:58 +01:00
13e12487d7 fix(ci): Fix RUNTIME_IMAGE_NAME variable substitution in runtime-base job
- Added RUNTIME_IMAGE_NAME to env section of Build and push runtime base image step
- Added RUNTIME_IMAGE_NAME to env section of Set runtime base outputs step
- Fixes 'bad substitution' error when variable is used in shell scripts
2025-11-01 21:17:44 +01:00
e68d1917b0 refactor(ci): Consolidate workflows and fix Git sync
- Added explicit Git pull in staging deployment after container restart
- Added production auto-deploy job to build-image.yml (for main branch)
- Removed redundant workflows: deploy-staging.yml, deploy-production.yml, production-deploy.yml
- All deployments now handled by build-image.yml:
  - staging branch → auto-deploy to staging
  - main branch → auto-deploy to production
- Fixed build job dependency (removed test dependency)
- Git sync now explicitly pulls code after deployment to ensure containers are up-to-date
2025-11-01 21:15:00 +01:00
d14d768acd fix(ci): Remove test dependency from build job to fix workflow execution
- Removed 'test' from build job needs array
- Build job now depends only on [changes, runtime-base]
- Fixes issue where build was waiting for skipped test job
- deploy-staging now correctly waits for build completion
2025-11-01 21:07:02 +01:00
9e34c0b474 fix: Verbessere Staging Deployment - Code-Synchronisation
- Aktualisiere docker-compose.yml immer aus Repository
- Starte staging-app Container neu für Git-Pull
- Unterstütze mehrere Image-URL-Formate in sed-Replacements
- Füge Wartezeiten für Git-Sync hinzu
- Behebt Problem, dass Code nicht synchronisiert wurde
2025-11-01 20:57:33 +01:00
478754ab02 chore: clean up and rename workflows 2025-11-01 17:24:20 +01:00
6bf6bf4cbe ci: optimise docker workflows 2025-11-01 17:04:03 +01:00
210e0aa74d feat: add auto-deploy to staging after successful build 2025-11-01 15:50:52 +01:00
77b2dc5dd7 feat: Add staging environment setup
- Create staging stack with separate containers and volumes
- Configure staging.michaelschiemer.de subdomain routing
- Add deploy-staging.yml workflow for auto-deployment
- Extend build-image.yml to support staging branch
- Separate Redis instance and network for staging
- Staging uses staging branch by default

Features:
- Auto-deploy: Push to staging branch → build → deploy to staging
- Separate from production: Different containers, volumes, networks
- Shared Traefik: Uses same SSL certificates (*.michaelschiemer.de)
- Testing environment before production deployment
2025-10-31 23:52:30 +01:00
194bd71257 feat: Split CI/CD pipeline into separate build and deploy workflows
- Add build-image.yml: Automatic image builds on push (5-8 min)
- Add deploy-production.yml: Manual deployment workflow (2-5 min)
- Mark production-deploy.yml as deprecated

Benefits:
- Faster feedback: Images ready in ~5-8 min (vs 10-15 min before)
- Flexible deployment: Deploy when ready, not forced after every build
- Parallel execution: Multiple builds can run simultaneously
- Better separation: Build failures don't block deployments of existing images
2025-10-31 23:43:49 +01:00