michaelschiemer/deployment/gitea-runner/README.md

# Gitea Actions Runner (Development Machine)

Self-hosted Gitea Actions runner for executing CI/CD workflows on the development machine.

## Overview

This setup provides a Gitea Actions runner that executes CI/CD workflows triggered by repository events in Gitea. The runner runs in Docker and uses Docker-in-Docker (DinD) for isolated job execution.

**Key Features**:
- **Self-Hosted**: Runs on development machine with full control
- **Docker-Based**: Isolated execution environment for jobs
- **Docker-in-Docker**: Jobs run in separate containers for security
- **Multiple Labels**: Support for different workflow environments
- **Auto-Restart**: Automatically restarts on failure
- **Secure**: Isolated network and resource limits

## Prerequisites

- Docker and Docker Compose installed
- Gitea instance running (Stack 2: Gitea)
- Admin access to Gitea for runner registration
- Network connectivity to Gitea instance

## Directory Structure

```
gitea-runner/
├── docker-compose.yml          # Service definitions
├── .env.example               # Environment template
├── .env                       # Environment configuration (create from .env.example)
├── config.yaml                # Runner configuration
├── register.sh                # Registration script
├── unregister.sh              # Unregistration script
├── data/                      # Runner data (auto-created)
│   └── .runner               # Registration info (auto-generated)
└── README.md                  # This file
```

## Quick Start

### 1. Create Environment File

```bash
cd deployment/gitea-runner
cp .env.example .env
```

### 2. Get Registration Token

1. Go to Gitea admin panel: https://git.michaelschiemer.de/admin
2. Navigate to: **Site Administration > Actions > Runners**
3. Click **"Create New Runner"**
4. Copy the registration token
5. Add token to `.env` file:

```bash
nano .env
# Set GITEA_RUNNER_REGISTRATION_TOKEN=<your-token>
```

### 3. Configure Environment Variables

Edit `.env` and configure:

```bash
# Gitea Instance URL
GITEA_INSTANCE_URL=https://git.michaelschiemer.de

# Registration Token (from step 2)
GITEA_RUNNER_REGISTRATION_TOKEN=<your-token>

# Runner Name (appears in Gitea UI)
GITEA_RUNNER_NAME=dev-runner-01

# Runner Labels (what environments this runner supports)
GITEA_RUNNER_LABELS=ubuntu-latest:docker://node:16-bullseye,ubuntu-22.04:docker://node:16-bullseye
```

### 4. Register Runner

Run the registration script:

```bash
./register.sh
```

This will:
- Start the runner services
- Register the runner with Gitea
- Display runner status

### 5. Verify Registration

Check runner status in Gitea:
- Go to: https://git.michaelschiemer.de/admin/actions/runners
- You should see your runner listed as "Idle" or "Active"

## Configuration

### Runner Labels

Labels define what workflow environments the runner supports. Format: `label:image`

**Common Labels**:
```bash
# Ubuntu with Node.js 16
ubuntu-latest:docker://node:16-bullseye

# Ubuntu 22.04
ubuntu-22.04:docker://node:16-bullseye

# Debian
debian-latest:docker://debian:bullseye

# PHP CI Image (optimized with PHP 8.5, Composer, Ansible pre-installed)
# Build first: ./build-ci-image.sh
php-ci:docker://php-ci:latest

# Custom images from private registry
ubuntu-php:docker://registry.michaelschiemer.de/php:8.3-cli
```

**Using the PHP CI Image**:

The `php-ci` image is pre-built with PHP 8.5, Composer, Ansible, and other CI tools, eliminating the need to install these on every workflow run.

1. Build the CI image:
   ```bash
   ./build-ci-image.sh
   ```

2. Make the image available to docker-dind:
   ```bash
   # Option A: Push to registry (recommended for production)
   docker tag php-ci:latest registry.michaelschiemer.de/ci/php-ci:latest
   docker push registry.michaelschiemer.de/ci/php-ci:latest
   
   # Option B: Load into docker-dind (for local testing)
   docker save php-ci:latest | docker exec -i gitea-runner-dind docker load
   ```

3. Update `.env` with the `php-ci` label (already included in example)

4. Re-register runner:
   ```bash
   ./unregister.sh
   ./register.sh
   ```

**Example Workflow Using Labels**:
```yaml
# .gitea/workflows/test.yml
name: Test
on: [push]

jobs:
  test:
    runs-on: ubuntu-latest  # Uses runner with this label
    steps:
      - uses: actions/checkout@v3
      - run: npm install
      - run: npm test
```

### Runner Capacity

Control how many jobs can run concurrently:

**In `.env`**:
```bash
GITEA_RUNNER_CAPACITY=1  # Max concurrent jobs
```

**In `config.yaml`**:
```yaml
runner:
  capacity: 1  # Max concurrent jobs
  timeout: 3h  # Job timeout
```

### Resource Limits

Configure resource limits in `config.yaml`:

```yaml
container:
  resources:
    memory_limit: 2147483648  # 2GB
    cpu_quota: 100000         # 1 CPU
```

## Usage

### Start Runner

```bash
# Start services
docker compose up -d

# View logs
docker compose logs -f gitea-runner
```

### Stop Runner

```bash
docker compose down
```

### Restart Runner

```bash
docker compose restart gitea-runner
```

### View Logs

```bash
# Follow logs
docker compose logs -f gitea-runner

# View last 100 lines
docker compose logs --tail=100 gitea-runner

# View Docker-in-Docker logs
docker compose logs -f docker-dind
```

### Check Runner Status

```bash
# Check container status
docker compose ps

# View runner info
docker compose exec gitea-runner cat /data/.runner
```

### Unregister Runner

```bash
./unregister.sh
```

This will:
- Stop the runner services
- Remove registration file
- Optionally remove runner data

**Note**: You may need to manually delete the runner from Gitea UI:
- Go to: https://git.michaelschiemer.de/admin/actions/runners
- Find the runner and click "Delete"

## Workflow Examples

### Basic Node.js Test Workflow

Create `.gitea/workflows/test.yml` in your repository:

```yaml
name: Test
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - name: Install dependencies
        run: npm install

      - name: Run tests
        run: npm test
```

### PHP Test Workflow

```yaml
name: PHP Tests
on: [push]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - name: Install PHP dependencies
        run: |
          apt-get update
          apt-get install -y php8.3-cli php8.3-mbstring
          php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
          php composer-setup.php --install-dir=/usr/local/bin --filename=composer
          composer install

      - name: Run tests
        run: ./vendor/bin/pest
```

### Build and Deploy Workflow

```yaml
name: Deploy
on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - name: Build Docker image
        run: |
          docker build -t registry.michaelschiemer.de/app:latest .

      - name: Push to registry
        run: |
          echo "${{ secrets.REGISTRY_PASSWORD }}" | docker login registry.michaelschiemer.de -u admin --password-stdin
          docker push registry.michaelschiemer.de/app:latest

      - name: Deploy via SSH
        run: |
          # Add SSH deployment commands here
```

## Troubleshooting

### Runner Not Appearing in Gitea

**Check registration**:
```bash
# Verify registration file exists
ls -la data/.runner

# Check runner logs
docker compose logs gitea-runner
```

**Re-register**:
```bash
./unregister.sh
./register.sh
```

### Jobs Not Starting

**Check runner status**:
```bash
# View logs
docker compose logs -f gitea-runner

# Check if runner is idle
# In Gitea: Admin > Actions > Runners
```

**Common Issues**:
- Runner is offline: Restart runner (`docker compose restart gitea-runner`)
- No matching labels: Verify workflow `runs-on` matches runner labels
- Capacity reached: Increase `GITEA_RUNNER_CAPACITY` or wait for jobs to finish

### Docker-in-Docker Issues

**Check DinD container**:
```bash
# View DinD logs
docker compose logs docker-dind

# Check DinD is running
docker compose ps docker-dind
```

**Restart DinD**:
```bash
docker compose restart docker-dind
```

### Job Timeout

Jobs timing out after 3 hours? Increase timeout in `config.yaml`:

```yaml
runner:
  timeout: 6h  # Increase to 6 hours
```

### Network Issues

**Cannot reach Gitea**:
```bash
# Test connectivity from runner
docker compose exec gitea-runner wget -O- https://git.michaelschiemer.de

# Check DNS resolution
docker compose exec gitea-runner nslookup git.michaelschiemer.de
```

### Disk Space Issues

**Clean up old job data**:
```bash
# Remove old workspace data
docker compose exec gitea-runner rm -rf /tmp/gitea-runner/*

# Clean up Docker images
docker image prune -a -f
```

## Security Considerations

### 1. Runner Security

- Runner runs with access to Docker socket (required for jobs)
- Jobs execute in isolated containers via Docker-in-Docker
- Network is isolated from other Docker networks
- Resource limits prevent resource exhaustion

### 2. Registration Token

- Registration token has admin privileges
- Store token securely (in `.env` file, not in git)
- Token is only used during registration
- After registration, runner uses generated credentials

### 3. Job Isolation

- Each job runs in a separate container
- Containers are destroyed after job completion
- Docker-in-Docker provides additional isolation layer
- Valid volume mounts are restricted in `config.yaml`

### 4. Secrets Management

**In Gitea**:
- Store secrets in repository settings: Settings > Secrets
- Access in workflows via `${{ secrets.SECRET_NAME }}`
- Secrets are masked in logs

**Example**:
```yaml
steps:
  - name: Deploy
    run: |
      echo "${{ secrets.DEPLOY_KEY }}" > deploy_key
      chmod 600 deploy_key
      ssh -i deploy_key user@server "deploy.sh"
```

### 5. Network Security

- Runner network is isolated
- Only runner and DinD containers share network
- No external access to runner management

## Maintenance

### Daily Tasks

- Monitor runner logs for errors
- Check disk space usage
- Verify runner appears as "Idle" in Gitea when not running jobs

### Weekly Tasks

- Review completed jobs in Gitea
- Clean up old Docker images: `docker image prune -a`
- Check runner resource usage

### Monthly Tasks

- Update runner image: `docker compose pull && docker compose up -d`
- Review and update runner labels
- Audit workflow performance

### Update Runner

```bash
# Pull latest image
docker compose pull

# Restart with new image
docker compose up -d

# Verify update
docker compose logs -f gitea-runner
```

## Performance Optimization

### Reduce Job Startup Time

**Cache dependencies** in workflows:

```yaml
- name: Cache dependencies
  uses: actions/cache@v3
  with:
    path: ~/.npm
    key: ${{ runner.os }}-npm-${{ hashFiles('**/package-lock.json') }}
```

### Optimize Docker Builds

**Use Docker layer caching**:

```yaml
- name: Build with cache
  run: |
    docker build \
      --cache-from registry.michaelschiemer.de/app:cache \
      --tag registry.michaelschiemer.de/app:latest \
      .
```

### Increase Runner Capacity

For more concurrent jobs:

```bash
# In .env
GITEA_RUNNER_CAPACITY=2  # Allow 2 concurrent jobs
```

**Note**: Ensure development machine has sufficient resources (CPU, RAM, disk).

## Integration with Deployment Stacks

### Stack 2: Gitea Integration

- Runner connects to Gitea for job fetching
- Uses Gitea API for workflow definitions
- Reports job status back to Gitea

### Stack 3: Docker Registry Integration

- Push built images to private registry
- Pull base images from registry for jobs
- Use registry for caching layers

### Stack 4: Application Deployment

- Build and test application code
- Deploy to application stack via SSH
- Trigger stack updates via Ansible

### Stack 5: Database Migrations

- Run database migrations in workflows
- Test database changes before deployment
- Backup database before migrations

### Stack 6: Monitoring Integration

- Monitor runner resource usage via cAdvisor
- Track job execution metrics in Prometheus
- Alert on runner failures via Grafana

## Advanced Configuration

### Custom Docker Registry for Jobs

Use private registry for job images:

```bash
# In .env
DOCKER_REGISTRY_MIRROR=https://registry.michaelschiemer.de
```

**In workflows**:
```yaml
jobs:
  test:
    runs-on: ubuntu-latest
    container:
      image: registry.michaelschiemer.de/php:8.3-cli
      credentials:
        username: admin
        password: ${{ secrets.REGISTRY_PASSWORD }}
```

### Multiple Runners

Run multiple runners for different purposes:

```bash
# Production runner
cd deployment/gitea-runner-prod
cp ../gitea-runner/.env.example .env
# Set GITEA_RUNNER_NAME=prod-runner
# Set different labels
./register.sh

# Staging runner
cd deployment/gitea-runner-staging
cp ../gitea-runner/.env.example .env
# Set GITEA_RUNNER_NAME=staging-runner
./register.sh
```

### Custom Job Container Options

In `config.yaml`:

```yaml
container:
  # Custom Docker options
  options: "--dns 8.8.8.8 --add-host git.michaelschiemer.de:94.16.110.151"

  # Custom network mode
  network: host

  # Enable privileged mode (use cautiously)
  privileged: false
```

## Monitoring and Logging

### View Runner Metrics

```bash
# Container resource usage
docker stats gitea-runner

# Detailed metrics
docker compose exec gitea-runner cat /data/metrics
```

### Centralized Logging

Send logs to monitoring stack:

```yaml
# In docker-compose.yml
services:
  gitea-runner:
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"
```

### Health Checks

```bash
# Check runner health
docker compose exec gitea-runner ps aux | grep act_runner

# Check job queue
# In Gitea: Actions > Jobs
```

## Backup and Recovery

### Backup Runner Configuration

```bash
# Backup registration and config
tar czf gitea-runner-backup-$(date +%Y%m%d).tar.gz \
  .env config.yaml data/.runner
```

### Restore Runner

```bash
# Extract backup
tar xzf gitea-runner-backup-YYYYMMDD.tar.gz

# Restart runner
docker compose up -d
```

**Note**: If registration token changed, re-register:
```bash
./unregister.sh
./register.sh
```

## Support

### Documentation

- Gitea Actions: https://docs.gitea.io/en-us/actions/overview/
- Act Runner: https://gitea.com/gitea/act_runner
- GitHub Actions (compatible): https://docs.github.com/en/actions

### Logs

```bash
# Runner logs
docker compose logs -f gitea-runner

# All logs
docker compose logs -f

# Export logs
docker compose logs > runner-logs-$(date +%Y%m%d).log
```

### Health Check

```bash
# Check all components
docker compose ps

# Test runner connection to Gitea
docker compose exec gitea-runner wget -O- https://git.michaelschiemer.de/api/v1/version
```

---

**Setup Status**: ✅ Ready for registration

**Next Steps**:
1. Copy `.env.example` to `.env`
2. Get registration token from Gitea
3. Run `./register.sh`
4. Verify runner appears in Gitea UI
5. Create a test workflow to verify functionality