Michael Schiemer
36ef2a1e2c
Some checks failed
🚀 Build & Deploy Image / Determine Build Necessity (push) Failing after 10m14s
🚀 Build & Deploy Image / Build Runtime Base Image (push) Has been skipped
🚀 Build & Deploy Image / Build Docker Image (push) Has been skipped
🚀 Build & Deploy Image / Run Tests & Quality Checks (push) Has been skipped
🚀 Build & Deploy Image / Auto-deploy to Staging (push) Has been skipped
🚀 Build & Deploy Image / Auto-deploy to Production (push) Has been skipped
Security Vulnerability Scan / Check for Dependency Changes (push) Failing after 11m25s
Security Vulnerability Scan / Composer Security Audit (push) Has been cancelled
- Remove middleware reference from Gitea Traefik labels (caused routing issues) - Optimize Gitea connection pool settings (MAX_IDLE_CONNS=30, authentication_timeout=180s) - Add explicit service reference in Traefik labels - Fix intermittent 504 timeouts by improving PostgreSQL connection handling Fixes Gitea unreachability via git.michaelschiemer.de
326 lines
15 KiB
YAML
326 lines
15 KiB
YAML
---
|
|
# Diagnose Gitea Timeouts
|
|
# Prüft Gitea-Status, Traefik-Routing, Netzwerk-Verbindungen und behebt Probleme
|
|
- name: Diagnose Gitea Timeouts
|
|
hosts: production
|
|
gather_facts: yes
|
|
become: no
|
|
|
|
tasks:
|
|
- name: Check Gitea container status
|
|
ansible.builtin.shell: |
|
|
cd /home/deploy/deployment/stacks/gitea
|
|
docker compose ps gitea
|
|
register: gitea_status
|
|
changed_when: false
|
|
|
|
- name: Display Gitea container status
|
|
ansible.builtin.debug:
|
|
msg: |
|
|
================================================================================
|
|
Gitea Container Status:
|
|
================================================================================
|
|
{{ gitea_status.stdout }}
|
|
================================================================================
|
|
|
|
- name: Check Gitea health endpoint (direct from container)
|
|
ansible.builtin.shell: |
|
|
cd /home/deploy/deployment/stacks/gitea
|
|
docker compose exec -T gitea curl -f http://localhost:3000/api/healthz 2>&1 || echo "HEALTH_CHECK_FAILED"
|
|
register: gitea_health_direct
|
|
changed_when: false
|
|
failed_when: false
|
|
|
|
- name: Display Gitea health (direct)
|
|
ansible.builtin.debug:
|
|
msg: |
|
|
================================================================================
|
|
Gitea Health Check (direct from container):
|
|
================================================================================
|
|
{% if 'HEALTH_CHECK_FAILED' not in gitea_health_direct.stdout %}
|
|
✅ Gitea is healthy (direct check)
|
|
Response: {{ gitea_health_direct.stdout }}
|
|
{% else %}
|
|
❌ Gitea health check failed (direct)
|
|
Error: {{ gitea_health_direct.stdout }}
|
|
{% endif %}
|
|
================================================================================
|
|
|
|
- name: Check Gitea health endpoint (via Traefik)
|
|
ansible.builtin.uri:
|
|
url: "https://git.michaelschiemer.de/api/healthz"
|
|
method: GET
|
|
status_code: [200]
|
|
validate_certs: false
|
|
timeout: 10
|
|
register: gitea_health_traefik
|
|
failed_when: false
|
|
changed_when: false
|
|
|
|
- name: Display Gitea health (via Traefik)
|
|
ansible.builtin.debug:
|
|
msg: |
|
|
================================================================================
|
|
Gitea Health Check (via Traefik):
|
|
================================================================================
|
|
{% if gitea_health_traefik.status == 200 %}
|
|
✅ Gitea is reachable via Traefik
|
|
Status: {{ gitea_health_traefik.status }}
|
|
{% else %}
|
|
❌ Gitea is NOT reachable via Traefik
|
|
Status: {{ gitea_health_traefik.status | default('TIMEOUT/ERROR') }}
|
|
Message: {{ gitea_health_traefik.msg | default('No response') }}
|
|
{% endif %}
|
|
================================================================================
|
|
|
|
- name: Check Traefik container status
|
|
ansible.builtin.shell: |
|
|
cd /home/deploy/deployment/stacks/traefik
|
|
docker compose ps traefik
|
|
register: traefik_status
|
|
changed_when: false
|
|
|
|
- name: Display Traefik container status
|
|
ansible.builtin.debug:
|
|
msg: |
|
|
================================================================================
|
|
Traefik Container Status:
|
|
================================================================================
|
|
{{ traefik_status.stdout }}
|
|
================================================================================
|
|
|
|
- name: Check Redis container status
|
|
ansible.builtin.shell: |
|
|
cd /home/deploy/deployment/stacks/gitea
|
|
docker compose ps redis
|
|
register: redis_status
|
|
changed_when: false
|
|
|
|
- name: Display Redis container status
|
|
ansible.builtin.debug:
|
|
msg: |
|
|
================================================================================
|
|
Redis Container Status:
|
|
================================================================================
|
|
{{ redis_status.stdout }}
|
|
================================================================================
|
|
|
|
- name: Check PostgreSQL container status
|
|
ansible.builtin.shell: |
|
|
cd /home/deploy/deployment/stacks/gitea
|
|
docker compose ps postgres
|
|
register: postgres_status
|
|
changed_when: false
|
|
|
|
- name: Display PostgreSQL container status
|
|
ansible.builtin.debug:
|
|
msg: |
|
|
================================================================================
|
|
PostgreSQL Container Status:
|
|
================================================================================
|
|
{{ postgres_status.stdout }}
|
|
================================================================================
|
|
|
|
- name: Check Gitea container IP in traefik-public network
|
|
ansible.builtin.shell: |
|
|
docker inspect gitea --format '{{ '{{' }}range .NetworkSettings.Networks{{ '}}' }}{{ '{{' }}if eq .NetworkID (docker network inspect traefik-public --format "{{ '{{' }}.Id{{ '}}' }}"){{ '}}' }}{{ '{{' }}.IPAddress{{ '}}' }}{{ '{{' }}end{{ '}}' }}{{ '{{' }}end{{ '}}' }}' 2>/dev/null || echo "NOT_FOUND"
|
|
register: gitea_ip
|
|
changed_when: false
|
|
failed_when: false
|
|
|
|
- name: Display Gitea IP in traefik-public network
|
|
ansible.builtin.debug:
|
|
msg: |
|
|
================================================================================
|
|
Gitea IP in traefik-public Network:
|
|
================================================================================
|
|
{% if gitea_ip.stdout and gitea_ip.stdout != 'NOT_FOUND' %}
|
|
✅ Gitea IP: {{ gitea_ip.stdout }}
|
|
{% else %}
|
|
❌ Gitea IP not found in traefik-public network
|
|
{% endif %}
|
|
================================================================================
|
|
|
|
- name: Test connection from Traefik to Gitea
|
|
ansible.builtin.shell: |
|
|
cd /home/deploy/deployment/stacks/traefik
|
|
docker compose exec -T traefik wget -qO- --timeout=5 http://gitea:3000/api/healthz 2>&1 || echo "CONNECTION_FAILED"
|
|
register: traefik_gitea_connection
|
|
changed_when: false
|
|
failed_when: false
|
|
|
|
- name: Display Traefik-Gitea connection test
|
|
ansible.builtin.debug:
|
|
msg: |
|
|
================================================================================
|
|
Traefik → Gitea Connection Test:
|
|
================================================================================
|
|
{% if 'CONNECTION_FAILED' in traefik_gitea_connection.stdout %}
|
|
❌ Traefik cannot reach Gitea
|
|
Error: {{ traefik_gitea_connection.stdout }}
|
|
{% else %}
|
|
✅ Traefik can reach Gitea
|
|
Response: {{ traefik_gitea_connection.stdout }}
|
|
{% endif %}
|
|
================================================================================
|
|
|
|
- name: Check Traefik routing configuration for Gitea
|
|
ansible.builtin.shell: |
|
|
docker inspect gitea --format '{{ '{{' }}json .Config.Labels{{ '}}' }}' 2>/dev/null | grep -i "traefik" || echo "NO_TRAEFIK_LABELS"
|
|
register: traefik_labels
|
|
changed_when: false
|
|
failed_when: false
|
|
|
|
- name: Display Traefik labels for Gitea
|
|
ansible.builtin.debug:
|
|
msg: |
|
|
================================================================================
|
|
Traefik Labels for Gitea:
|
|
================================================================================
|
|
{{ traefik_labels.stdout }}
|
|
================================================================================
|
|
|
|
- name: Check Gitea logs for errors
|
|
ansible.builtin.shell: |
|
|
cd /home/deploy/deployment/stacks/gitea
|
|
docker compose logs gitea --tail=50 2>&1 | grep -iE "error|timeout|failed|panic|fatal" | tail -20 || echo "No errors in recent logs"
|
|
register: gitea_errors
|
|
changed_when: false
|
|
failed_when: false
|
|
|
|
- name: Display Gitea errors
|
|
ansible.builtin.debug:
|
|
msg: |
|
|
================================================================================
|
|
Gitea Error Logs (last 50 lines):
|
|
================================================================================
|
|
{{ gitea_errors.stdout }}
|
|
================================================================================
|
|
|
|
- name: Check Traefik logs for Gitea-related errors
|
|
ansible.builtin.shell: |
|
|
cd /home/deploy/deployment/stacks/traefik
|
|
docker compose logs traefik --tail=50 2>&1 | grep -iE "gitea|git\.michaelschiemer\.de|timeout|error" | tail -20 || echo "No Gitea-related errors in Traefik logs"
|
|
register: traefik_gitea_errors
|
|
changed_when: false
|
|
failed_when: false
|
|
|
|
- name: Display Traefik Gitea errors
|
|
ansible.builtin.debug:
|
|
msg: |
|
|
================================================================================
|
|
Traefik Gitea-Related Error Logs (last 50 lines):
|
|
================================================================================
|
|
{{ traefik_gitea_errors.stdout }}
|
|
================================================================================
|
|
|
|
- name: Check if Gitea is in traefik-public network
|
|
ansible.builtin.shell: |
|
|
docker network inspect traefik-public --format '{{ '{{' }}range .Containers{{ '}}' }}{{ '{{' }}.Name{{ '}}' }} {{ '{{' }}end{{ '}}' }}' 2>/dev/null | grep -q gitea && echo "YES" || echo "NO"
|
|
register: gitea_in_traefik_network
|
|
changed_when: false
|
|
failed_when: false
|
|
|
|
- name: Display Gitea network membership
|
|
ansible.builtin.debug:
|
|
msg: |
|
|
================================================================================
|
|
Gitea in traefik-public Network:
|
|
================================================================================
|
|
{% if gitea_in_traefik_network.stdout == 'YES' %}
|
|
✅ Gitea is in traefik-public network
|
|
{% else %}
|
|
❌ Gitea is NOT in traefik-public network
|
|
{% endif %}
|
|
================================================================================
|
|
|
|
- name: Check Redis connection from Gitea
|
|
ansible.builtin.shell: |
|
|
cd /home/deploy/deployment/stacks/gitea
|
|
docker compose exec -T gitea sh -c "redis-cli -h redis -p 6379 -a gitea_redis_password ping 2>&1" || echo "REDIS_CONNECTION_FAILED"
|
|
register: gitea_redis_connection
|
|
changed_when: false
|
|
failed_when: false
|
|
|
|
- name: Display Gitea-Redis connection
|
|
ansible.builtin.debug:
|
|
msg: |
|
|
================================================================================
|
|
Gitea → Redis Connection:
|
|
================================================================================
|
|
{% if 'REDIS_CONNECTION_FAILED' in gitea_redis_connection.stdout %}
|
|
❌ Gitea cannot connect to Redis
|
|
Error: {{ gitea_redis_connection.stdout }}
|
|
{% else %}
|
|
✅ Gitea can connect to Redis
|
|
Response: {{ gitea_redis_connection.stdout }}
|
|
{% endif %}
|
|
================================================================================
|
|
|
|
- name: Check PostgreSQL connection from Gitea
|
|
ansible.builtin.shell: |
|
|
cd /home/deploy/deployment/stacks/gitea
|
|
docker compose exec -T gitea sh -c "pg_isready -h postgres -p 5432 -U gitea 2>&1" || echo "POSTGRES_CONNECTION_FAILED"
|
|
register: gitea_postgres_connection
|
|
changed_when: false
|
|
failed_when: false
|
|
|
|
- name: Display Gitea-PostgreSQL connection
|
|
ansible.builtin.debug:
|
|
msg: |
|
|
================================================================================
|
|
Gitea → PostgreSQL Connection:
|
|
================================================================================
|
|
{% if 'POSTGRES_CONNECTION_FAILED' in gitea_postgres_connection.stdout %}
|
|
❌ Gitea cannot connect to PostgreSQL
|
|
Error: {{ gitea_postgres_connection.stdout }}
|
|
{% else %}
|
|
✅ Gitea can connect to PostgreSQL
|
|
Response: {{ gitea_postgres_connection.stdout }}
|
|
{% endif %}
|
|
================================================================================
|
|
|
|
- name: Summary and recommendations
|
|
ansible.builtin.debug:
|
|
msg: |
|
|
================================================================================
|
|
ZUSAMMENFASSUNG - Gitea Timeout Diagnose:
|
|
================================================================================
|
|
|
|
Gitea Status: {{ gitea_status.stdout | regex_replace('.*(Up|Down|Restarting).*', '\\1') | default('UNKNOWN') }}
|
|
Gitea Health (direct): {% if 'HEALTH_CHECK_FAILED' not in gitea_health_direct.stdout %}✅{% else %}❌{% endif %}
|
|
Gitea Health (via Traefik): {% if gitea_health_traefik.status == 200 %}✅{% else %}❌{% endif %}
|
|
Traefik Status: {{ traefik_status.stdout | regex_replace('.*(Up|Down|Restarting).*', '\\1') | default('UNKNOWN') }}
|
|
Redis Status: {{ redis_status.stdout | regex_replace('.*(Up|Down|Restarting).*', '\\1') | default('UNKNOWN') }}
|
|
PostgreSQL Status: {{ postgres_status.stdout | regex_replace('.*(Up|Down|Restarting).*', '\\1') | default('UNKNOWN') }}
|
|
|
|
Netzwerk:
|
|
- Gitea in traefik-public: {% if gitea_in_traefik_network.stdout == 'YES' %}✅{% else %}❌{% endif %}
|
|
- Traefik → Gitea: {% if 'CONNECTION_FAILED' not in traefik_gitea_connection.stdout %}✅{% else %}❌{% endif %}
|
|
- Gitea → Redis: {% if 'REDIS_CONNECTION_FAILED' not in gitea_redis_connection.stdout %}✅{% else %}❌{% endif %}
|
|
- Gitea → PostgreSQL: {% if 'POSTGRES_CONNECTION_FAILED' not in gitea_postgres_connection.stdout %}✅{% else %}❌{% endif %}
|
|
|
|
Empfohlene Aktionen:
|
|
{% if gitea_health_traefik.status != 200 %}
|
|
1. ❌ Gitea ist nicht über Traefik erreichbar
|
|
→ Führe 'fix-gitea-timeouts.yml' aus um Gitea und Traefik zu restarten
|
|
{% endif %}
|
|
{% if gitea_in_traefik_network.stdout != 'YES' %}
|
|
2. ❌ Gitea ist nicht im traefik-public Netzwerk
|
|
→ Gitea Container neu starten um Netzwerk-Verbindung zu aktualisieren
|
|
{% endif %}
|
|
{% if 'CONNECTION_FAILED' in traefik_gitea_connection.stdout %}
|
|
3. ❌ Traefik kann Gitea nicht erreichen
|
|
→ Beide Container neu starten
|
|
{% endif %}
|
|
{% if 'REDIS_CONNECTION_FAILED' in gitea_redis_connection.stdout %}
|
|
4. ❌ Gitea kann Redis nicht erreichen
|
|
→ Redis Container prüfen und neu starten
|
|
{% endif %}
|
|
{% if 'POSTGRES_CONNECTION_FAILED' in gitea_postgres_connection.stdout %}
|
|
5. ❌ Gitea kann PostgreSQL nicht erreichen
|
|
→ PostgreSQL Container prüfen und neu starten
|
|
{% endif %}
|
|
|
|
================================================================================
|