michaelschiemer/deployment/ansible/playbooks/setup/REDEPLOY_GUIDE.md

# Traefik/Gitea Redeploy Guide

This guide explains how to perform a clean redeployment of Traefik and Gitea stacks.

## Overview

A clean redeploy:
- Stops and removes containers (preserves volumes and SSL certificates)
- Syncs latest configurations
- Redeploys stacks with fresh containers
- Restores configurations
- Verifies service discovery

**Expected downtime**: ~2-5 minutes

## Prerequisites

- Ansible installed locally
- SSH access to production server
- Vault password file: `deployment/ansible/secrets/.vault_pass`

## Step-by-Step Guide

### Step 1: Backup

**Automatic backup (recommended):**
```bash
cd deployment/ansible
ansible-playbook -i inventory/production.yml \
  playbooks/maintenance/backup-before-redeploy.yml \
  --vault-password-file secrets/.vault_pass
```

**Manual backup:**
```bash
# On server
cd /home/deploy/deployment/stacks
docker compose -f gitea/docker-compose.yml exec gitea cat /data/gitea/conf/app.ini > /tmp/gitea-app.ini.backup
cp traefik/acme.json /tmp/acme.json.backup
```

### Step 2: Verify Backup

Check backup contents:
```bash
# Backup location will be shown in output
ls -lh /home/deploy/backups/redeploy-backup-*/
```

Verify:
- `acme.json` exists
- `gitea-app.ini` exists
- `gitea-volume-*.tar.gz` exists (if volumes were backed up)

### Step 3: Redeploy

**With automatic backup:**
```bash
cd deployment/ansible
ansible-playbook -i inventory/production.yml \
  playbooks/setup/redeploy-traefik-gitea-clean.yml \
  --vault-password-file secrets/.vault_pass
```

**With existing backup:**
```bash
cd deployment/ansible
ansible-playbook -i inventory/production.yml \
  playbooks/setup/redeploy-traefik-gitea-clean.yml \
  --vault-password-file secrets/.vault_pass \
  -e "backup_name=redeploy-backup-1234567890" \
  -e "skip_backup=true"
```

### Step 4: Verify Deployment

**Check Gitea accessibility:**
```bash
curl -k https://git.michaelschiemer.de/api/healthz
```

**Check Traefik service discovery:**
```bash
# On server
cd /home/deploy/deployment/stacks/traefik
docker compose exec traefik traefik show providers docker | grep -i gitea
```

**Check container status:**
```bash
# On server
docker ps | grep -E "traefik|gitea"
```

### Step 5: Troubleshooting

**If Gitea is not reachable:**

1. Check Gitea logs:
   ```bash
   cd /home/deploy/deployment/stacks/gitea
   docker compose logs gitea --tail=50
   ```

2. Check Traefik logs:
   ```bash
   cd /home/deploy/deployment/stacks/traefik
   docker compose logs traefik --tail=50
   ```

3. Check service discovery:
   ```bash
   cd /home/deploy/deployment/stacks/traefik
   docker compose exec traefik traefik show providers docker
   ```

4. Run diagnosis:
   ```bash
   cd deployment/ansible
   ansible-playbook -i inventory/production.yml \
     playbooks/diagnose/gitea.yml \
     --vault-password-file secrets/.vault_pass
   ```

**If SSL certificate issues:**

1. Check acme.json permissions:
   ```bash
   ls -l /home/deploy/deployment/stacks/traefik/acme.json
   # Should be: -rw------- (600)
   ```

2. Check Traefik ACME logs:
   ```bash
   cd /home/deploy/deployment/stacks/traefik
   docker compose logs traefik | grep -i acme
   ```

## Rollback Procedure

If something goes wrong, rollback to the backup:

```bash
cd deployment/ansible
ansible-playbook -i inventory/production.yml \
  playbooks/maintenance/rollback-redeploy.yml \
  --vault-password-file secrets/.vault_pass \
  -e "backup_name=redeploy-backup-1234567890"
```

Replace `redeploy-backup-1234567890` with the actual backup name from Step 1.

## What Gets Preserved

- ✅ Gitea data (volumes)
- ✅ SSL certificates (acme.json)
- ✅ Gitea configuration (app.ini)
- ✅ Traefik configuration
- ✅ PostgreSQL data (if applicable)

## What Gets Recreated

- 🔄 Traefik container
- 🔄 Gitea container
- 🔄 Service discovery

## Common Issues

### Issue: Gitea returns 404 after redeploy

**Solution:**
1. Wait 1-2 minutes for service discovery
2. Restart Traefik: `cd /home/deploy/deployment/stacks/traefik && docker compose restart traefik`
3. Check if Gitea is in traefik-public network: `docker network inspect traefik-public | grep gitea`

### Issue: SSL certificate errors

**Solution:**
1. Verify acme.json permissions: `chmod 600 /home/deploy/deployment/stacks/traefik/acme.json`
2. Check Traefik logs for ACME errors
3. Wait 5-10 minutes for certificate renewal

### Issue: Gitea configuration lost

**Solution:**
1. Restore from backup: `playbooks/maintenance/rollback-redeploy.yml`
2. Or manually restore app.ini:
   ```bash
   cd /home/deploy/deployment/stacks/gitea
   docker compose exec gitea sh -c "cat > /data/gitea/conf/app.ini" < /path/to/backup/gitea-app.ini
   docker compose restart gitea
   ```

## Best Practices

1. **Always backup before redeploy** - Use automatic backup
2. **Test in staging first** - If available
3. **Monitor during deployment** - Watch logs in separate terminal
4. **Have rollback ready** - Know backup name before starting
5. **Verify after deployment** - Check all services are accessible

## Related Playbooks

- `playbooks/maintenance/backup-before-redeploy.yml` - Create backup
- `playbooks/setup/redeploy-traefik-gitea-clean.yml` - Perform redeploy
- `playbooks/maintenance/rollback-redeploy.yml` - Rollback from backup
- `playbooks/diagnose/gitea.yml` - Diagnose Gitea issues
- `playbooks/diagnose/traefik.yml` - Diagnose Traefik issues