fix: Gitea Traefik routing and connection pool optimization
Some checks failed
🚀 Build & Deploy Image / Determine Build Necessity (push) Failing after 10m14s
🚀 Build & Deploy Image / Build Runtime Base Image (push) Has been skipped
🚀 Build & Deploy Image / Build Docker Image (push) Has been skipped
🚀 Build & Deploy Image / Run Tests & Quality Checks (push) Has been skipped
🚀 Build & Deploy Image / Auto-deploy to Staging (push) Has been skipped
🚀 Build & Deploy Image / Auto-deploy to Production (push) Has been skipped
Security Vulnerability Scan / Check for Dependency Changes (push) Failing after 11m25s
Security Vulnerability Scan / Composer Security Audit (push) Has been cancelled
Some checks failed
🚀 Build & Deploy Image / Determine Build Necessity (push) Failing after 10m14s
🚀 Build & Deploy Image / Build Runtime Base Image (push) Has been skipped
🚀 Build & Deploy Image / Build Docker Image (push) Has been skipped
🚀 Build & Deploy Image / Run Tests & Quality Checks (push) Has been skipped
🚀 Build & Deploy Image / Auto-deploy to Staging (push) Has been skipped
🚀 Build & Deploy Image / Auto-deploy to Production (push) Has been skipped
Security Vulnerability Scan / Check for Dependency Changes (push) Failing after 11m25s
Security Vulnerability Scan / Composer Security Audit (push) Has been cancelled
- Remove middleware reference from Gitea Traefik labels (caused routing issues) - Optimize Gitea connection pool settings (MAX_IDLE_CONNS=30, authentication_timeout=180s) - Add explicit service reference in Traefik labels - Fix intermittent 504 timeouts by improving PostgreSQL connection handling Fixes Gitea unreachability via git.michaelschiemer.de
This commit is contained in:
325
deployment/ansible/playbooks/diagnose-gitea-timeouts.yml
Normal file
325
deployment/ansible/playbooks/diagnose-gitea-timeouts.yml
Normal file
@@ -0,0 +1,325 @@
|
||||
---
|
||||
# Diagnose Gitea Timeouts
|
||||
# Prüft Gitea-Status, Traefik-Routing, Netzwerk-Verbindungen und behebt Probleme
|
||||
- name: Diagnose Gitea Timeouts
|
||||
hosts: production
|
||||
gather_facts: yes
|
||||
become: no
|
||||
|
||||
tasks:
|
||||
- name: Check Gitea container status
|
||||
ansible.builtin.shell: |
|
||||
cd /home/deploy/deployment/stacks/gitea
|
||||
docker compose ps gitea
|
||||
register: gitea_status
|
||||
changed_when: false
|
||||
|
||||
- name: Display Gitea container status
|
||||
ansible.builtin.debug:
|
||||
msg: |
|
||||
================================================================================
|
||||
Gitea Container Status:
|
||||
================================================================================
|
||||
{{ gitea_status.stdout }}
|
||||
================================================================================
|
||||
|
||||
- name: Check Gitea health endpoint (direct from container)
|
||||
ansible.builtin.shell: |
|
||||
cd /home/deploy/deployment/stacks/gitea
|
||||
docker compose exec -T gitea curl -f http://localhost:3000/api/healthz 2>&1 || echo "HEALTH_CHECK_FAILED"
|
||||
register: gitea_health_direct
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
|
||||
- name: Display Gitea health (direct)
|
||||
ansible.builtin.debug:
|
||||
msg: |
|
||||
================================================================================
|
||||
Gitea Health Check (direct from container):
|
||||
================================================================================
|
||||
{% if 'HEALTH_CHECK_FAILED' not in gitea_health_direct.stdout %}
|
||||
✅ Gitea is healthy (direct check)
|
||||
Response: {{ gitea_health_direct.stdout }}
|
||||
{% else %}
|
||||
❌ Gitea health check failed (direct)
|
||||
Error: {{ gitea_health_direct.stdout }}
|
||||
{% endif %}
|
||||
================================================================================
|
||||
|
||||
- name: Check Gitea health endpoint (via Traefik)
|
||||
ansible.builtin.uri:
|
||||
url: "https://git.michaelschiemer.de/api/healthz"
|
||||
method: GET
|
||||
status_code: [200]
|
||||
validate_certs: false
|
||||
timeout: 10
|
||||
register: gitea_health_traefik
|
||||
failed_when: false
|
||||
changed_when: false
|
||||
|
||||
- name: Display Gitea health (via Traefik)
|
||||
ansible.builtin.debug:
|
||||
msg: |
|
||||
================================================================================
|
||||
Gitea Health Check (via Traefik):
|
||||
================================================================================
|
||||
{% if gitea_health_traefik.status == 200 %}
|
||||
✅ Gitea is reachable via Traefik
|
||||
Status: {{ gitea_health_traefik.status }}
|
||||
{% else %}
|
||||
❌ Gitea is NOT reachable via Traefik
|
||||
Status: {{ gitea_health_traefik.status | default('TIMEOUT/ERROR') }}
|
||||
Message: {{ gitea_health_traefik.msg | default('No response') }}
|
||||
{% endif %}
|
||||
================================================================================
|
||||
|
||||
- name: Check Traefik container status
|
||||
ansible.builtin.shell: |
|
||||
cd /home/deploy/deployment/stacks/traefik
|
||||
docker compose ps traefik
|
||||
register: traefik_status
|
||||
changed_when: false
|
||||
|
||||
- name: Display Traefik container status
|
||||
ansible.builtin.debug:
|
||||
msg: |
|
||||
================================================================================
|
||||
Traefik Container Status:
|
||||
================================================================================
|
||||
{{ traefik_status.stdout }}
|
||||
================================================================================
|
||||
|
||||
- name: Check Redis container status
|
||||
ansible.builtin.shell: |
|
||||
cd /home/deploy/deployment/stacks/gitea
|
||||
docker compose ps redis
|
||||
register: redis_status
|
||||
changed_when: false
|
||||
|
||||
- name: Display Redis container status
|
||||
ansible.builtin.debug:
|
||||
msg: |
|
||||
================================================================================
|
||||
Redis Container Status:
|
||||
================================================================================
|
||||
{{ redis_status.stdout }}
|
||||
================================================================================
|
||||
|
||||
- name: Check PostgreSQL container status
|
||||
ansible.builtin.shell: |
|
||||
cd /home/deploy/deployment/stacks/gitea
|
||||
docker compose ps postgres
|
||||
register: postgres_status
|
||||
changed_when: false
|
||||
|
||||
- name: Display PostgreSQL container status
|
||||
ansible.builtin.debug:
|
||||
msg: |
|
||||
================================================================================
|
||||
PostgreSQL Container Status:
|
||||
================================================================================
|
||||
{{ postgres_status.stdout }}
|
||||
================================================================================
|
||||
|
||||
- name: Check Gitea container IP in traefik-public network
|
||||
ansible.builtin.shell: |
|
||||
docker inspect gitea --format '{{ '{{' }}range .NetworkSettings.Networks{{ '}}' }}{{ '{{' }}if eq .NetworkID (docker network inspect traefik-public --format "{{ '{{' }}.Id{{ '}}' }}"){{ '}}' }}{{ '{{' }}.IPAddress{{ '}}' }}{{ '{{' }}end{{ '}}' }}{{ '{{' }}end{{ '}}' }}' 2>/dev/null || echo "NOT_FOUND"
|
||||
register: gitea_ip
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
|
||||
- name: Display Gitea IP in traefik-public network
|
||||
ansible.builtin.debug:
|
||||
msg: |
|
||||
================================================================================
|
||||
Gitea IP in traefik-public Network:
|
||||
================================================================================
|
||||
{% if gitea_ip.stdout and gitea_ip.stdout != 'NOT_FOUND' %}
|
||||
✅ Gitea IP: {{ gitea_ip.stdout }}
|
||||
{% else %}
|
||||
❌ Gitea IP not found in traefik-public network
|
||||
{% endif %}
|
||||
================================================================================
|
||||
|
||||
- name: Test connection from Traefik to Gitea
|
||||
ansible.builtin.shell: |
|
||||
cd /home/deploy/deployment/stacks/traefik
|
||||
docker compose exec -T traefik wget -qO- --timeout=5 http://gitea:3000/api/healthz 2>&1 || echo "CONNECTION_FAILED"
|
||||
register: traefik_gitea_connection
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
|
||||
- name: Display Traefik-Gitea connection test
|
||||
ansible.builtin.debug:
|
||||
msg: |
|
||||
================================================================================
|
||||
Traefik → Gitea Connection Test:
|
||||
================================================================================
|
||||
{% if 'CONNECTION_FAILED' in traefik_gitea_connection.stdout %}
|
||||
❌ Traefik cannot reach Gitea
|
||||
Error: {{ traefik_gitea_connection.stdout }}
|
||||
{% else %}
|
||||
✅ Traefik can reach Gitea
|
||||
Response: {{ traefik_gitea_connection.stdout }}
|
||||
{% endif %}
|
||||
================================================================================
|
||||
|
||||
- name: Check Traefik routing configuration for Gitea
|
||||
ansible.builtin.shell: |
|
||||
docker inspect gitea --format '{{ '{{' }}json .Config.Labels{{ '}}' }}' 2>/dev/null | grep -i "traefik" || echo "NO_TRAEFIK_LABELS"
|
||||
register: traefik_labels
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
|
||||
- name: Display Traefik labels for Gitea
|
||||
ansible.builtin.debug:
|
||||
msg: |
|
||||
================================================================================
|
||||
Traefik Labels for Gitea:
|
||||
================================================================================
|
||||
{{ traefik_labels.stdout }}
|
||||
================================================================================
|
||||
|
||||
- name: Check Gitea logs for errors
|
||||
ansible.builtin.shell: |
|
||||
cd /home/deploy/deployment/stacks/gitea
|
||||
docker compose logs gitea --tail=50 2>&1 | grep -iE "error|timeout|failed|panic|fatal" | tail -20 || echo "No errors in recent logs"
|
||||
register: gitea_errors
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
|
||||
- name: Display Gitea errors
|
||||
ansible.builtin.debug:
|
||||
msg: |
|
||||
================================================================================
|
||||
Gitea Error Logs (last 50 lines):
|
||||
================================================================================
|
||||
{{ gitea_errors.stdout }}
|
||||
================================================================================
|
||||
|
||||
- name: Check Traefik logs for Gitea-related errors
|
||||
ansible.builtin.shell: |
|
||||
cd /home/deploy/deployment/stacks/traefik
|
||||
docker compose logs traefik --tail=50 2>&1 | grep -iE "gitea|git\.michaelschiemer\.de|timeout|error" | tail -20 || echo "No Gitea-related errors in Traefik logs"
|
||||
register: traefik_gitea_errors
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
|
||||
- name: Display Traefik Gitea errors
|
||||
ansible.builtin.debug:
|
||||
msg: |
|
||||
================================================================================
|
||||
Traefik Gitea-Related Error Logs (last 50 lines):
|
||||
================================================================================
|
||||
{{ traefik_gitea_errors.stdout }}
|
||||
================================================================================
|
||||
|
||||
- name: Check if Gitea is in traefik-public network
|
||||
ansible.builtin.shell: |
|
||||
docker network inspect traefik-public --format '{{ '{{' }}range .Containers{{ '}}' }}{{ '{{' }}.Name{{ '}}' }} {{ '{{' }}end{{ '}}' }}' 2>/dev/null | grep -q gitea && echo "YES" || echo "NO"
|
||||
register: gitea_in_traefik_network
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
|
||||
- name: Display Gitea network membership
|
||||
ansible.builtin.debug:
|
||||
msg: |
|
||||
================================================================================
|
||||
Gitea in traefik-public Network:
|
||||
================================================================================
|
||||
{% if gitea_in_traefik_network.stdout == 'YES' %}
|
||||
✅ Gitea is in traefik-public network
|
||||
{% else %}
|
||||
❌ Gitea is NOT in traefik-public network
|
||||
{% endif %}
|
||||
================================================================================
|
||||
|
||||
- name: Check Redis connection from Gitea
|
||||
ansible.builtin.shell: |
|
||||
cd /home/deploy/deployment/stacks/gitea
|
||||
docker compose exec -T gitea sh -c "redis-cli -h redis -p 6379 -a gitea_redis_password ping 2>&1" || echo "REDIS_CONNECTION_FAILED"
|
||||
register: gitea_redis_connection
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
|
||||
- name: Display Gitea-Redis connection
|
||||
ansible.builtin.debug:
|
||||
msg: |
|
||||
================================================================================
|
||||
Gitea → Redis Connection:
|
||||
================================================================================
|
||||
{% if 'REDIS_CONNECTION_FAILED' in gitea_redis_connection.stdout %}
|
||||
❌ Gitea cannot connect to Redis
|
||||
Error: {{ gitea_redis_connection.stdout }}
|
||||
{% else %}
|
||||
✅ Gitea can connect to Redis
|
||||
Response: {{ gitea_redis_connection.stdout }}
|
||||
{% endif %}
|
||||
================================================================================
|
||||
|
||||
- name: Check PostgreSQL connection from Gitea
|
||||
ansible.builtin.shell: |
|
||||
cd /home/deploy/deployment/stacks/gitea
|
||||
docker compose exec -T gitea sh -c "pg_isready -h postgres -p 5432 -U gitea 2>&1" || echo "POSTGRES_CONNECTION_FAILED"
|
||||
register: gitea_postgres_connection
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
|
||||
- name: Display Gitea-PostgreSQL connection
|
||||
ansible.builtin.debug:
|
||||
msg: |
|
||||
================================================================================
|
||||
Gitea → PostgreSQL Connection:
|
||||
================================================================================
|
||||
{% if 'POSTGRES_CONNECTION_FAILED' in gitea_postgres_connection.stdout %}
|
||||
❌ Gitea cannot connect to PostgreSQL
|
||||
Error: {{ gitea_postgres_connection.stdout }}
|
||||
{% else %}
|
||||
✅ Gitea can connect to PostgreSQL
|
||||
Response: {{ gitea_postgres_connection.stdout }}
|
||||
{% endif %}
|
||||
================================================================================
|
||||
|
||||
- name: Summary and recommendations
|
||||
ansible.builtin.debug:
|
||||
msg: |
|
||||
================================================================================
|
||||
ZUSAMMENFASSUNG - Gitea Timeout Diagnose:
|
||||
================================================================================
|
||||
|
||||
Gitea Status: {{ gitea_status.stdout | regex_replace('.*(Up|Down|Restarting).*', '\\1') | default('UNKNOWN') }}
|
||||
Gitea Health (direct): {% if 'HEALTH_CHECK_FAILED' not in gitea_health_direct.stdout %}✅{% else %}❌{% endif %}
|
||||
Gitea Health (via Traefik): {% if gitea_health_traefik.status == 200 %}✅{% else %}❌{% endif %}
|
||||
Traefik Status: {{ traefik_status.stdout | regex_replace('.*(Up|Down|Restarting).*', '\\1') | default('UNKNOWN') }}
|
||||
Redis Status: {{ redis_status.stdout | regex_replace('.*(Up|Down|Restarting).*', '\\1') | default('UNKNOWN') }}
|
||||
PostgreSQL Status: {{ postgres_status.stdout | regex_replace('.*(Up|Down|Restarting).*', '\\1') | default('UNKNOWN') }}
|
||||
|
||||
Netzwerk:
|
||||
- Gitea in traefik-public: {% if gitea_in_traefik_network.stdout == 'YES' %}✅{% else %}❌{% endif %}
|
||||
- Traefik → Gitea: {% if 'CONNECTION_FAILED' not in traefik_gitea_connection.stdout %}✅{% else %}❌{% endif %}
|
||||
- Gitea → Redis: {% if 'REDIS_CONNECTION_FAILED' not in gitea_redis_connection.stdout %}✅{% else %}❌{% endif %}
|
||||
- Gitea → PostgreSQL: {% if 'POSTGRES_CONNECTION_FAILED' not in gitea_postgres_connection.stdout %}✅{% else %}❌{% endif %}
|
||||
|
||||
Empfohlene Aktionen:
|
||||
{% if gitea_health_traefik.status != 200 %}
|
||||
1. ❌ Gitea ist nicht über Traefik erreichbar
|
||||
→ Führe 'fix-gitea-timeouts.yml' aus um Gitea und Traefik zu restarten
|
||||
{% endif %}
|
||||
{% if gitea_in_traefik_network.stdout != 'YES' %}
|
||||
2. ❌ Gitea ist nicht im traefik-public Netzwerk
|
||||
→ Gitea Container neu starten um Netzwerk-Verbindung zu aktualisieren
|
||||
{% endif %}
|
||||
{% if 'CONNECTION_FAILED' in traefik_gitea_connection.stdout %}
|
||||
3. ❌ Traefik kann Gitea nicht erreichen
|
||||
→ Beide Container neu starten
|
||||
{% endif %}
|
||||
{% if 'REDIS_CONNECTION_FAILED' in gitea_redis_connection.stdout %}
|
||||
4. ❌ Gitea kann Redis nicht erreichen
|
||||
→ Redis Container prüfen und neu starten
|
||||
{% endif %}
|
||||
{% if 'POSTGRES_CONNECTION_FAILED' in gitea_postgres_connection.stdout %}
|
||||
5. ❌ Gitea kann PostgreSQL nicht erreichen
|
||||
→ PostgreSQL Container prüfen und neu starten
|
||||
{% endif %}
|
||||
|
||||
================================================================================
|
||||
Reference in New Issue
Block a user