BREAKING CHANGE: Requires PHP 8.5.0RC3 Changes: - Update Docker base image from php:8.4-fpm to php:8.5.0RC3-fpm - Enable ext-uri for native WHATWG URL parsing support - Update composer.json PHP requirement from ^8.4 to ^8.5 - Add ext-uri as required extension in composer.json - Move URL classes from Url.php85/ to Url/ directory (now compatible) - Remove temporary PHP 8.4 compatibility workarounds Benefits: - Native URL parsing with Uri\WhatWg\Url class - Better performance for URL operations - Future-proof with latest PHP features - Eliminates PHP version compatibility issues
653 lines
16 KiB
Markdown
653 lines
16 KiB
Markdown
# Rsync-Based Deployment
|
|
|
|
**Production-ready Zero-Downtime Deployment** mit Rsync, Release Management und automatischem Rollback.
|
|
|
|
## Übersicht
|
|
|
|
Das Rsync-basierte Deployment Playbook (`deploy-rsync-based.yml`) bietet eine robuste Lösung für Production Deployments ohne externe Git-Server-Abhängigkeiten.
|
|
|
|
**Vorteile**:
|
|
- ✅ Zero-Downtime durch Symlink-Switch
|
|
- ✅ Automatischer Rollback bei Health Check Failure
|
|
- ✅ Git Tag-basiertes Release Management
|
|
- ✅ Keine Gitea/GitHub Abhängigkeit
|
|
- ✅ Schnell für kleine Änderungen
|
|
- ✅ Einfaches Rollback zu vorherigen Releases
|
|
|
|
## Deployment-Architektur
|
|
|
|
### Release Structure
|
|
|
|
```
|
|
/home/deploy/michaelschiemer/
|
|
├── releases/
|
|
│ ├── 1761499893/ # Timestamp-based releases
|
|
│ ├── v1.0.0/ # Git tag-based releases
|
|
│ └── v1.2.3/
|
|
├── shared/ # Shared zwischen Releases
|
|
│ ├── storage/
|
|
│ │ └── sessions/
|
|
│ ├── public/
|
|
│ │ └── uploads/
|
|
│ └── .env.production # Shared config
|
|
├── current -> releases/v1.2.3 # Symlink auf aktives Release
|
|
└── deploy.log # Deployment history
|
|
```
|
|
|
|
### Zero-Downtime Process
|
|
|
|
```
|
|
1. Build Assets (local)
|
|
↓
|
|
2. Rsync to new release directory
|
|
↓
|
|
3. Create symlinks zu shared directories
|
|
↓
|
|
4. Start Docker containers
|
|
↓
|
|
5. Health Check (3 retries)
|
|
├─ Success → Switch 'current' symlink (atomic)
|
|
└─ Failure → Rollback zu previous release
|
|
↓
|
|
6. Cleanup old releases (keep last 5)
|
|
```
|
|
|
|
## Voraussetzungen
|
|
|
|
### 1. SSH Key Setup
|
|
|
|
SSH Keys für Production Server müssen konfiguriert sein:
|
|
|
|
```bash
|
|
# SSH config in ~/.ssh/config
|
|
Host michaelschiemer-prod
|
|
HostName 94.16.110.151
|
|
User deploy
|
|
IdentityFile ~/.ssh/production
|
|
StrictHostKeyChecking no
|
|
```
|
|
|
|
### 2. Production Server Requirements
|
|
|
|
- **User**: `deploy` user mit sudo Rechten
|
|
- **Docker**: Docker und Docker Compose installiert
|
|
- **Directory**: `/home/deploy/michaelschiemer` mit korrekten Permissions
|
|
|
|
### 3. Local Development Setup
|
|
|
|
- **Composer**: Für `composer install`
|
|
- **NPM**: Für `npm run build`
|
|
- **Git**: Für Tag-basiertes Release Management (optional)
|
|
- **Ansible**: Ansible ≥2.13 installiert
|
|
|
|
## Deployment-Workflows
|
|
|
|
### Standard Deployment (Timestamp-based)
|
|
|
|
Deployiert aktuellen Stand ohne Git Tag:
|
|
|
|
```bash
|
|
cd deployment/infrastructure
|
|
ansible-playbook -i inventories/production/hosts.yml playbooks/deploy-rsync-based.yml
|
|
```
|
|
|
|
**Release Name**: Unix Timestamp (z.B. `1761499893`)
|
|
|
|
### Tagged Release Deployment (Recommended)
|
|
|
|
Deployiert spezifischen Git Tag:
|
|
|
|
```bash
|
|
# Option 1: Tag explizit angeben
|
|
ansible-playbook -i inventories/production/hosts.yml playbooks/deploy-rsync-based.yml \
|
|
--extra-vars "release_tag=v1.2.3"
|
|
|
|
# Option 2: Aktuellen Git Tag verwenden (auto-detected)
|
|
git tag v1.2.3
|
|
git push origin v1.2.3
|
|
ansible-playbook -i inventories/production/hosts.yml playbooks/deploy-rsync-based.yml
|
|
```
|
|
|
|
**Release Name**: Git Tag (z.B. `v1.2.3`)
|
|
|
|
### Force Deployment (Override Lock)
|
|
|
|
Wenn ein Deployment Lock existiert:
|
|
|
|
```bash
|
|
ansible-playbook -i inventories/production/hosts.yml playbooks/deploy-rsync-based.yml \
|
|
--extra-vars "force_deploy=true"
|
|
```
|
|
|
|
## Release Management
|
|
|
|
### Git Tag Workflow
|
|
|
|
**Semantic Versioning** wird empfohlen:
|
|
|
|
```bash
|
|
# 1. Create Git tag
|
|
git tag -a v1.2.3 -m "Release v1.2.3: Feature XYZ"
|
|
git push origin v1.2.3
|
|
|
|
# 2. Deploy tagged release
|
|
ansible-playbook -i inventories/production/hosts.yml playbooks/deploy-rsync-based.yml \
|
|
--extra-vars "release_tag=v1.2.3"
|
|
|
|
# 3. Verify deployment
|
|
ssh deploy@94.16.110.151 'ls -la /home/deploy/michaelschiemer/releases/'
|
|
```
|
|
|
|
### Auto-Detection von Git Tags
|
|
|
|
Wenn `release_tag` nicht angegeben wird, versucht das Playbook automatisch den aktuellen Git Tag zu verwenden:
|
|
|
|
```bash
|
|
# Auf einem getaggten Commit
|
|
git describe --tags --exact-match # Zeigt: v1.2.3
|
|
|
|
# Deployment verwendet automatisch v1.2.3 als Release Name
|
|
ansible-playbook -i inventories/production/hosts.yml playbooks/deploy-rsync-based.yml
|
|
```
|
|
|
|
**Fallback**: Wenn kein Git Tag vorhanden → Timestamp als Release Name
|
|
|
|
### Release List anzeigen
|
|
|
|
```bash
|
|
ansible -i inventories/production/hosts.yml web_servers -b -a \
|
|
"ls -lt /home/deploy/michaelschiemer/releases | head -10"
|
|
```
|
|
|
|
**Output**:
|
|
```
|
|
total 20
|
|
drwxr-xr-x 10 deploy deploy 4096 Oct 26 18:50 v1.2.3
|
|
drwxr-xr-x 10 deploy deploy 4096 Oct 25 14:32 v1.2.2
|
|
drwxr-xr-x 10 deploy deploy 4096 Oct 24 10:15 1761499893
|
|
drwxr-xr-x 10 deploy deploy 4096 Oct 23 09:00 v1.2.1
|
|
lrwxrwxrwx 1 deploy deploy 56 Oct 26 18:50 current -> /home/deploy/michaelschiemer/releases/v1.2.3
|
|
```
|
|
|
|
## Rollback Mechanisms
|
|
|
|
### Automatic Rollback
|
|
|
|
Bei Health Check Failure rollback das Playbook automatisch:
|
|
|
|
1. **Stop failed release containers**
|
|
2. **Switch `current` symlink** zurück zu `previous_release`
|
|
3. **Start previous release containers**
|
|
4. **Remove failed release directory**
|
|
5. **Log rollback event**
|
|
|
|
**Trigger**: Health Check Status ≠ 200 (nach 3 Retries mit 5s delay)
|
|
|
|
### Manual Rollback
|
|
|
|
Manueller Rollback zu vorherigem Release:
|
|
|
|
```bash
|
|
# 1. List available releases
|
|
ansible -i inventories/production/hosts.yml web_servers -b -a \
|
|
"ls -lt /home/deploy/michaelschiemer/releases"
|
|
|
|
# 2. Identify target release (z.B. v1.2.2)
|
|
TARGET_RELEASE="v1.2.2"
|
|
|
|
# 3. Manual rollback via Ansible
|
|
ansible -i inventories/production/hosts.yml web_servers -b -m shell -a "
|
|
cd /home/deploy/michaelschiemer && \
|
|
docker compose -f current/docker-compose.yml -f current/docker-compose.production.yml down && \
|
|
ln -sfn releases/${TARGET_RELEASE} current && \
|
|
docker compose -f current/docker-compose.yml -f current/docker-compose.production.yml up -d
|
|
"
|
|
|
|
# 4. Verify rollback
|
|
curl -k https://94.16.110.151/health/summary
|
|
```
|
|
|
|
**Oder**: Erstelle Rollback Playbook:
|
|
|
|
```yaml
|
|
# playbooks/rollback-rsync.yml
|
|
- name: Manual Rollback to Previous Release
|
|
hosts: web_servers
|
|
become: true
|
|
vars:
|
|
app_name: michaelschiemer
|
|
app_user: deploy
|
|
app_base_path: "/home/{{ app_user }}/{{ app_name }}"
|
|
target_release: "{{ rollback_target }}" # --extra-vars "rollback_target=v1.2.2"
|
|
|
|
tasks:
|
|
- name: Stop current release
|
|
command: docker compose down
|
|
args:
|
|
chdir: "{{ app_base_path }}/current"
|
|
become_user: "{{ app_user }}"
|
|
|
|
- name: Switch to target release
|
|
file:
|
|
src: "{{ app_base_path }}/releases/{{ target_release }}"
|
|
dest: "{{ app_base_path }}/current"
|
|
state: link
|
|
force: yes
|
|
|
|
- name: Start target release
|
|
command: docker compose -f docker-compose.yml -f docker-compose.production.yml up -d
|
|
args:
|
|
chdir: "{{ app_base_path }}/current"
|
|
become_user: "{{ app_user }}"
|
|
|
|
- name: Health check
|
|
uri:
|
|
url: "https://{{ ansible_host }}/health/summary"
|
|
method: GET
|
|
status_code: 200
|
|
validate_certs: no
|
|
retries: 3
|
|
delay: 5
|
|
|
|
# Usage:
|
|
# ansible-playbook -i inventories/production/hosts.yml playbooks/rollback-rsync.yml --extra-vars "rollback_target=v1.2.2"
|
|
```
|
|
|
|
## Health Checks
|
|
|
|
### Configured Health Endpoints
|
|
|
|
**Primary Health Check**: `https://{{ ansible_host }}/health/summary`
|
|
|
|
**Retry Strategy**:
|
|
- Retries: 3
|
|
- Delay: 5 seconds
|
|
- Success: HTTP 200 status code
|
|
|
|
### Health Check Flow
|
|
|
|
```yaml
|
|
- name: Health check - Summary endpoint (HTTPS)
|
|
uri:
|
|
url: "https://{{ ansible_host }}/health/summary"
|
|
method: GET
|
|
return_content: yes
|
|
status_code: 200
|
|
validate_certs: no
|
|
follow_redirects: none
|
|
register: health_check
|
|
retries: 3
|
|
delay: 5
|
|
until: health_check.status == 200
|
|
ignore_errors: yes
|
|
|
|
- name: Rollback on health check failure
|
|
block:
|
|
- name: Stop failed release containers
|
|
- name: Switch symlink back to previous release
|
|
- name: Start previous release containers
|
|
- name: Remove failed release
|
|
- name: Log rollback
|
|
- name: Fail deployment
|
|
when: health_check.status != 200
|
|
```
|
|
|
|
### Custom Health Endpoints
|
|
|
|
Füge weitere Health Checks hinzu:
|
|
|
|
```yaml
|
|
# Nach der Primary Health Check in deploy-rsync-based.yml
|
|
- name: Health check - Database connectivity
|
|
uri:
|
|
url: "https://{{ ansible_host }}/health/database"
|
|
method: GET
|
|
status_code: 200
|
|
validate_certs: no
|
|
retries: 2
|
|
delay: 3
|
|
ignore_errors: yes
|
|
register: db_health_check
|
|
|
|
- name: Health check - Cache service
|
|
uri:
|
|
url: "https://{{ ansible_host }}/health/cache"
|
|
method: GET
|
|
status_code: 200
|
|
validate_certs: no
|
|
retries: 2
|
|
delay: 3
|
|
ignore_errors: yes
|
|
register: cache_health_check
|
|
|
|
- name: Aggregate health check results
|
|
set_fact:
|
|
overall_health: "{{ health_check.status == 200 and db_health_check.status == 200 and cache_health_check.status == 200 }}"
|
|
|
|
- name: Rollback on any health check failure
|
|
block:
|
|
# ... rollback steps ...
|
|
when: not overall_health
|
|
```
|
|
|
|
## Monitoring & Logging
|
|
|
|
### Deployment Log
|
|
|
|
Alle Deployments werden geloggt:
|
|
|
|
```bash
|
|
# Deployment log anzeigen
|
|
ssh deploy@94.16.110.151 'tail -50 /home/deploy/michaelschiemer/deploy.log'
|
|
```
|
|
|
|
**Log Format**:
|
|
```
|
|
[2024-10-26T18:50:30Z] Deployment started - Release: v1.2.3 - User: michael
|
|
[2024-10-26T18:50:35Z] Release: v1.2.3 | Git Hash: a1b2c3d | Commit: a1b2c3d4e5f6g7h8i9j0
|
|
[2024-10-26T18:50:50Z] Symlink switched: /home/deploy/michaelschiemer/current -> releases/v1.2.3
|
|
[2024-10-26T18:50:55Z] Health check: 200
|
|
[2024-10-26T18:50:56Z] Cleanup: Kept 5 releases, removed 1
|
|
[2024-10-26T18:50:57Z] Deployment completed successfully - Release: v1.2.3
|
|
```
|
|
|
|
### Docker Logs
|
|
|
|
```bash
|
|
# Application logs
|
|
ssh deploy@94.16.110.151 'cd /home/deploy/michaelschiemer/current && docker compose logs -f'
|
|
|
|
# Specific service
|
|
ssh deploy@94.16.110.151 'cd /home/deploy/michaelschiemer/current && docker compose logs -f php'
|
|
```
|
|
|
|
### System Monitoring
|
|
|
|
```bash
|
|
# Disk usage
|
|
ansible -i inventories/production/hosts.yml web_servers -b -a "df -h /home/deploy/michaelschiemer"
|
|
|
|
# Release directory sizes
|
|
ansible -i inventories/production/hosts.yml web_servers -b -a "du -sh /home/deploy/michaelschiemer/releases/*"
|
|
|
|
# Container status
|
|
ansible -i inventories/production/hosts.yml web_servers -b -a "docker ps"
|
|
```
|
|
|
|
## Configuration
|
|
|
|
### Shared Directories
|
|
|
|
Konfiguriert in `deploy-rsync-based.yml`:
|
|
|
|
```yaml
|
|
shared_dirs:
|
|
- storage/sessions
|
|
- public/uploads
|
|
|
|
shared_files:
|
|
- .env.production
|
|
```
|
|
|
|
**Hinweis**: `storage/logs`, `storage/cache`, `storage/uploads` werden via Docker Volumes verwaltet.
|
|
|
|
### Rsync Exclusions
|
|
|
|
Files/Directories die NICHT deployiert werden:
|
|
|
|
```yaml
|
|
rsync_excludes:
|
|
- .git/
|
|
- .github/
|
|
- node_modules/
|
|
- .env
|
|
- .env.local
|
|
- .env.development
|
|
- storage/
|
|
- public/uploads/
|
|
- tests/
|
|
- .idea/
|
|
- .vscode/
|
|
- "*.log"
|
|
- .DS_Store
|
|
- deployment/
|
|
- database.sqlite
|
|
- "*.cache"
|
|
- .php-cs-fixer.cache
|
|
- var/cache/
|
|
- var/logs/
|
|
```
|
|
|
|
### Keep Releases
|
|
|
|
Anzahl der beibehaltenen Releases:
|
|
|
|
```yaml
|
|
keep_releases: 5 # Standard: 5 Releases
|
|
```
|
|
|
|
Ändere nach Bedarf:
|
|
```bash
|
|
ansible-playbook ... --extra-vars "keep_releases=10"
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
### Problem: Deployment Lock existiert
|
|
|
|
**Error**:
|
|
```
|
|
FAILED! => msg: Deployment already in progress. Lock file exists: /home/deploy/michaelschiemer/.deploy.lock
|
|
```
|
|
|
|
**Ursache**: Vorheriges Deployment wurde unterbrochen
|
|
|
|
**Lösung**:
|
|
```bash
|
|
# Option 1: Force deployment
|
|
ansible-playbook ... --extra-vars "force_deploy=true"
|
|
|
|
# Option 2: Lock manuell entfernen
|
|
ansible -i inventories/production/hosts.yml web_servers -b -m file \
|
|
-a "path=/home/deploy/michaelschiemer/.deploy.lock state=absent"
|
|
```
|
|
|
|
### Problem: Health Check schlägt fehl
|
|
|
|
**Error**:
|
|
```
|
|
FAILED! => Deployment failed - health check returned 503. Rolled back to previous release.
|
|
```
|
|
|
|
**Diagnose**:
|
|
```bash
|
|
# 1. Check application logs
|
|
ssh deploy@94.16.110.151 'cd /home/deploy/michaelschiemer/current && docker compose logs --tail=100'
|
|
|
|
# 2. Check container status
|
|
ssh deploy@94.16.110.151 'docker ps -a'
|
|
|
|
# 3. Manual health check
|
|
curl -k -v https://94.16.110.151/health/summary
|
|
|
|
# 4. Check deployment log
|
|
ssh deploy@94.16.110.151 'tail -100 /home/deploy/michaelschiemer/deploy.log'
|
|
```
|
|
|
|
**Häufige Ursachen**:
|
|
- .env.production fehlt oder fehlerhaft
|
|
- Database migration fehlgeschlagen
|
|
- Docker container starten nicht
|
|
- SSL Zertifikat Probleme
|
|
|
|
### Problem: Rsync zu langsam
|
|
|
|
**Symptom**: Deployment dauert mehrere Minuten
|
|
|
|
**Optimierung**:
|
|
```yaml
|
|
# In deploy-rsync-based.yml - rsync command erweitern
|
|
--compress # Kompression aktiviert
|
|
--delete-after # Löschen nach Transfer
|
|
--delay-updates # Atomic updates
|
|
```
|
|
|
|
**Alternative**: Rsync via lokales Netzwerk statt Internet:
|
|
```yaml
|
|
# Wenn Production Server im gleichen Netzwerk
|
|
ansible_host: 192.168.1.100 # Lokale IP statt öffentliche
|
|
```
|
|
|
|
### Problem: Git Tag nicht erkannt
|
|
|
|
**Symptom**: Deployment verwendet Timestamp statt Git Tag
|
|
|
|
**Diagnose**:
|
|
```bash
|
|
# Check ob auf getaggtem Commit
|
|
git describe --tags --exact-match
|
|
# Sollte: v1.2.3 (ohne Fehler)
|
|
|
|
# Check ob Tag existiert
|
|
git tag -l
|
|
```
|
|
|
|
**Lösung**:
|
|
```bash
|
|
# 1. Tag erstellen falls fehlend
|
|
git tag v1.2.3
|
|
git push origin v1.2.3
|
|
|
|
# 2. Oder Tag explizit angeben
|
|
ansible-playbook ... --extra-vars "release_tag=v1.2.3"
|
|
```
|
|
|
|
## Best Practices
|
|
|
|
### 1. Always Tag Releases
|
|
|
|
```bash
|
|
# Vor Production Deployment immer Git Tag erstellen
|
|
git tag -a v1.2.3 -m "Release v1.2.3: Feature description"
|
|
git push origin v1.2.3
|
|
```
|
|
|
|
**Vorteile**:
|
|
- Klare Release-Historie
|
|
- Einfaches Rollback zu spezifischen Versionen
|
|
- Semantic Versioning tracking
|
|
|
|
### 2. Test Deployment in Staging First
|
|
|
|
```bash
|
|
# Staging deployment (separate inventory)
|
|
ansible-playbook -i inventories/staging/hosts.yml playbooks/deploy-rsync-based.yml \
|
|
--extra-vars "release_tag=v1.2.3"
|
|
|
|
# Nach erfolgreichen Tests → Production
|
|
ansible-playbook -i inventories/production/hosts.yml playbooks/deploy-rsync-based.yml \
|
|
--extra-vars "release_tag=v1.2.3"
|
|
```
|
|
|
|
### 3. Monitor Deployment Log
|
|
|
|
```bash
|
|
# Real-time deployment monitoring
|
|
ssh deploy@94.16.110.151 'tail -f /home/deploy/michaelschiemer/deploy.log'
|
|
```
|
|
|
|
### 4. Backup vor Major Releases
|
|
|
|
```bash
|
|
# Database backup vor Major Release
|
|
ssh deploy@94.16.110.151 'cd /home/deploy/michaelschiemer/current && \
|
|
docker compose exec php php console.php db:backup'
|
|
```
|
|
|
|
### 5. Verify Health Before Release Tag
|
|
|
|
```bash
|
|
# Health check auf Staging
|
|
curl -k https://staging.michaelschiemer.de/health/summary
|
|
|
|
# Bei Erfolg → Production Tag
|
|
git tag v1.2.3
|
|
git push origin v1.2.3
|
|
```
|
|
|
|
## Comparison: Rsync vs Git-based
|
|
|
|
### Rsync-based (Current)
|
|
|
|
**Vorteile**:
|
|
- ✅ Keine Git-Server Abhängigkeit
|
|
- ✅ Funktioniert mit lokalem Development
|
|
- ✅ Schnell für kleine Änderungen
|
|
- ✅ Einfaches Setup
|
|
- ✅ Git Tag Support ohne External Server
|
|
|
|
**Nachteile**:
|
|
- ❌ Keine Git-Historie auf Production Server
|
|
- ❌ Erfordert lokale Build-Steps (Composer, NPM)
|
|
- ❌ Rsync über Internet kann langsam sein
|
|
|
|
### Git-based
|
|
|
|
**Vorteile**:
|
|
- ✅ Git-Historie auf Production Server
|
|
- ✅ Atomare Releases mit Git Commits
|
|
- ✅ Build direkt auf Production Server
|
|
- ✅ Kein lokales Build erforderlich
|
|
|
|
**Nachteile**:
|
|
- ❌ Gitea Server muss öffentlich erreichbar sein
|
|
- ❌ Zusätzliche Infrastruktur (Gitea)
|
|
- ❌ SSH Key Management komplexer
|
|
|
|
## Performance Optimizations
|
|
|
|
### 1. Pre-built Assets
|
|
|
|
Assets werden lokal gebaut → schnelleres Deployment:
|
|
```yaml
|
|
pre_tasks:
|
|
- name: Install Composer dependencies locally
|
|
- name: Build NPM assets locally
|
|
```
|
|
|
|
### 2. Docker Layer Caching
|
|
|
|
Docker Images werden auf Production Server gecached → schnellerer Start.
|
|
|
|
### 3. Shared Directories
|
|
|
|
Shared directories vermeiden unnötiges Kopieren:
|
|
- `storage/sessions`
|
|
- `public/uploads`
|
|
- `.env.production`
|
|
|
|
### 4. Cleanup Old Releases
|
|
|
|
Nur 5 Releases behalten → spart Disk Space:
|
|
```yaml
|
|
keep_releases: 5
|
|
```
|
|
|
|
## Related Files
|
|
|
|
- `deploy-rsync-based.yml` - Rsync-based Deployment Playbook
|
|
- `deploy-git-based.yml` - Git-based Deployment Playbook (Alternative)
|
|
- `rollback-git-based.yml` - Git-based Rollback Playbook
|
|
- `inventories/production/hosts.yml` - Production Server Configuration
|
|
|
|
## Zusammenfassung
|
|
|
|
Das rsync-based Deployment bietet:
|
|
- ✅ **Production-Ready** Zero-Downtime Deployment
|
|
- ✅ **Git Tag Support** für klare Release-Historie
|
|
- ✅ **Automatischer Rollback** bei Failures
|
|
- ✅ **Einfaches Setup** ohne externe Dependencies
|
|
- ✅ **Schnell und Zuverlässig** für Development und Production
|
|
|
|
**Empfehlung**: Ideal für lokale Development → Production Workflows ohne zusätzliche Git-Server-Infrastruktur.
|