Compare commits

...

4 Commits

Author SHA1 Message Date
9aa881d895 Document voice assistant TTS service status
Mark TTS functionality as disabled due to onnxruntime removal
(freed 1.2GB disk space during cleanup)
2025-10-11 18:25:08 +02:00
ab4a4cfc9c Add comprehensive backup system documentation
Document complete backup strategy covering:
- Setup and initialization procedures
- What gets backed up (and what doesn't)
- Multiple restore scenarios (full reinstall, rollback, specific volumes)
- Retention policy explanation
- Troubleshooting and security notes
2025-10-11 18:24:58 +02:00
646d3e59bc Add comprehensive system backup scripts
Implement full system backup with Restic including:
- Docker volume exports (Gitea, Jellyfin, Nextcloud, Portainer)
- System configuration backup (/etc/)
- Package list exports (pacman explicit, all, AUR)
- Automated retention policy (7 daily, 4 weekly, 3 monthly, 1 yearly)
- Separate temporary directories for Docker and system data
2025-10-11 18:24:51 +02:00
ad7270aa74 Add Restic backup repository initialization script
Initialize encrypted backup repository on NAS with:
- Auto-install of Restic if needed
- Secure password generation and storage
- Repository initialization with AES-256 encryption
2025-10-11 18:24:41 +02:00
5 changed files with 533 additions and 0 deletions

233
docs/system-backups.md Normal file
View File

@@ -0,0 +1,233 @@
# System Backups
Complete backup strategy for the homelab using Restic for encrypted, deduplicated backups to NAS.
## Overview
**Backup Tool**: Restic (modern, encrypted, deduplicated)
**Backup Destination**: NAS at `/mnt/nas/backups/homelab-restic`
**Backup Frequency**: Daily (can be automated with systemd timer)
**Encryption**: AES-256, password stored in `/home/hoborg/creds/restic-password.txt`
## What Gets Backed Up
### Full System Backup Includes:
1. **Package Lists**
- All explicitly installed packages (pacman)
- All installed packages (including dependencies)
- AUR/foreign packages list
- Allows exact system recreation after reinstall
2. **System Configuration** (`/etc/`)
- Nginx configuration
- Systemd services
- Network configuration
- All system-wide configs
- Excludes: shadow/gshadow password files
3. **Docker Volumes**
- Gitea (Git repositories and database)
- Jellyfin (media library configuration)
- Nextcloud (cloud storage data and database)
- Portainer (container management data)
### What's NOT Backed Up (Intentionally):
- **User data in `/home`**: Separate partition, safe from system reinstalls
- **Media files**: Already stored on NAS (Music, Videos, Pictures)
- **User documents**: Synced via Nextcloud/Copyparty or on separate partition
## Setup
### Initial Setup (One-Time)
1. **Initialize the backup repository:**
```bash
sudo /home/hoborg/homelab/scripts/backup-init.sh
```
This will:
- Install Restic if needed
- Create backup directory on NAS (`/mnt/nas/backups/homelab-restic`)
- Generate encryption password (saved to `/home/hoborg/creds/restic-password.txt`)
- Initialize the Restic repository
⚠️ **IMPORTANT**: Back up the password file! Without it, backups cannot be restored.
### Running Backups
**Manual backup:**
```bash
sudo /home/hoborg/homelab/scripts/backup-system-full.sh
```
**Automated backups** (optional - setup systemd timer later):
- Create systemd service and timer
- Schedule daily backups (e.g., 3 AM)
- Logs written to `/var/log/homelab-backup.log`
## Backup Scripts
### `backup-init.sh`
Initializes the Restic repository. Run once during setup.
Location: `/home/hoborg/homelab/scripts/backup-init.sh`
### `backup-system-full.sh`
Performs full system backup including packages, configs, and Docker volumes.
Location: `/home/hoborg/homelab/scripts/backup-system-full.sh`
What it does:
1. Exports package lists (pacman explicit, all packages, AUR packages)
2. Exports Docker volumes to compressed tarballs
3. Backs up entire `/etc/` directory
4. Prunes old backups (keeps: 7 daily, 4 weekly, 6 monthly, 2 yearly)
5. Shows backup statistics
## Restore Procedures
### List Available Backups
```bash
export RESTIC_PASSWORD_FILE="/home/hoborg/creds/restic-password.txt"
restic -r /mnt/nas/backups/homelab-restic snapshots
```
### Restore After System Reinstall
1. **Restore backup to temporary location:**
```bash
export RESTIC_PASSWORD_FILE="/home/hoborg/creds/restic-password.txt"
restic -r /mnt/nas/backups/homelab-restic restore latest --target /tmp/restore
```
2. **Reinstall packages:**
```bash
# Reinstall all explicitly installed packages
sudo pacman -S --needed $(cat /tmp/restore/tmp/system-backup-*/pacman-explicit.txt)
# Reinstall AUR packages
yay -S --needed $(cat /tmp/restore/tmp/system-backup-*/aur-packages.txt)
```
3. **Restore system configs:**
```bash
sudo cp -r /tmp/restore/etc/* /etc/
sudo systemctl daemon-reload
```
4. **Restore Docker volumes:**
```bash
# Example: Restore Gitea
docker-compose -f /opt/docker/gitea/docker-compose.yml down
sudo tar xzf /tmp/restore/tmp/docker-backup-*/gitea-data.tar.gz -C /tmp/gitea-restore
docker run --rm -v gitea_gitea:/data -v /tmp/gitea-restore:/backup alpine \
sh -c "rm -rf /data/* && cp -a /backup/* /data/"
docker-compose -f /opt/docker/gitea/docker-compose.yml up -d
```
### Rollback After Broken Update
If a system update breaks something:
1. **Restore specific config:**
```bash
export RESTIC_PASSWORD_FILE="/home/hoborg/creds/restic-password.txt"
# Restore nginx config
restic -r /mnt/nas/backups/homelab-restic restore latest \
--target /tmp/restore --include /etc/nginx/
sudo cp -r /tmp/restore/etc/nginx/* /etc/nginx/
sudo systemctl reload nginx
```
2. **Downgrade packages:**
```bash
# Check which packages were updated
grep -A5 "upgraded" /var/log/pacman.log | tail -20
# Downgrade specific package to cached version
sudo pacman -U /var/cache/pacman/pkg/package-old-version.pkg.tar.zst
```
### Restore Specific Docker Volume
```bash
export RESTIC_PASSWORD_FILE="/home/hoborg/creds/restic-password.txt"
# List files in backup
restic -r /mnt/nas/backups/homelab-restic ls latest | grep docker-backup
# Restore specific volume tarball
restic -r /mnt/nas/backups/homelab-restic restore latest \
--target /tmp/restore \
--include /tmp/docker-backup-*/nextcloud-data.tar.gz
# Extract and restore to Docker volume
mkdir /tmp/nextcloud-data
tar xzf /tmp/restore/tmp/docker-backup-*/nextcloud-data.tar.gz -C /tmp/nextcloud-data
docker run --rm -v nextcloud_nextcloud_data:/data -v /tmp/nextcloud-data:/backup alpine \
sh -c "rm -rf /data/* && cp -a /backup/* /data/"
```
## Backup Retention Policy
The backup script automatically prunes old backups:
- **Daily**: Keep last 7 days
- **Weekly**: Keep last 4 weeks
- **Monthly**: Keep last 6 months
- **Yearly**: Keep last 2 years
This balances storage space with recovery options.
## Troubleshooting
### "Repository not initialized" Error
Run the initialization script:
```bash
sudo /home/hoborg/homelab/scripts/backup-init.sh
```
### "Password incorrect" Error
Check that password file exists and is readable:
```bash
ls -l /home/hoborg/creds/restic-password.txt
```
### Check Backup Size
```bash
export RESTIC_PASSWORD_FILE="/home/hoborg/creds/restic-password.txt"
restic -r /mnt/nas/backups/homelab-restic stats --mode restore-size
```
### Verify Backup Integrity
```bash
export RESTIC_PASSWORD_FILE="/home/hoborg/creds/restic-password.txt"
restic -r /mnt/nas/backups/homelab-restic check
```
## Security Notes
- Backups are encrypted with AES-256
- Password file has `600` permissions (owner read/write only)
- Shadow/gshadow files are excluded from backups
- NAS should have access restrictions
- Consider offsite backup copy of password file
## Future Enhancements
- [ ] Automated systemd timer setup
- [ ] Pre-backup notifications
- [ ] Post-backup success/failure notifications
- [ ] Backup verification script
- [ ] Web dashboard for backup status
- [ ] Offsite backup replication (optional)

View File

@@ -1,5 +1,7 @@
# Voice Assistant Setup # Voice Assistant Setup
⚠️ **STATUS: DISABLED** - onnxruntime package was removed to free disk space (1.2GB). Voice functionality is currently unavailable.
This document describes how to set up AI voice capabilities for Claude Code using local TTS (Text-to-Speech) services. This document describes how to set up AI voice capabilities for Claude Code using local TTS (Text-to-Speech) services.
## Overview ## Overview
@@ -8,6 +10,7 @@ The voice assistant setup uses:
- **Piper TTS**: Local neural text-to-speech engine for generating natural-sounding speech - **Piper TTS**: Local neural text-to-speech engine for generating natural-sounding speech
- **FastAPI**: HTTP server wrapper to make Piper compatible with voice-mode - **FastAPI**: HTTP server wrapper to make Piper compatible with voice-mode
- **Ryan voice model**: Professional male US English voice for AI assistant personality - **Ryan voice model**: Professional male US English voice for AI assistant personality
- **onnxruntime**: ML inference library (removed - required for TTS)
## Prerequisites ## Prerequisites

95
scripts/backup-homelab.sh Executable file
View File

@@ -0,0 +1,95 @@
#!/bin/bash
# Backup Docker volumes and system configs to NAS using Restic
# Can be run manually or via systemd timer
set -e
# Configuration
BACKUP_REPO="/mnt/nas/backups/homelab-restic"
PASSWORD_FILE="/home/hoborg/creds/restic-password.txt"
LOG_FILE="/var/log/homelab-backup.log"
DOCKER_BACKUP_DIR="/tmp/docker-backup-$$"
# Export password
export RESTIC_PASSWORD_FILE="$PASSWORD_FILE"
# Logging function
log() {
echo "[$(date +'%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE"
}
log "========================================"
log "Starting Homelab Backup"
log "========================================"
# Check if repository exists
if ! restic -r "$BACKUP_REPO" snapshots &>/dev/null; then
log "ERROR: Backup repository not initialized. Run backup-init.sh first."
exit 1
fi
# Create temporary directory for Docker volume exports
mkdir -p "$DOCKER_BACKUP_DIR"
log "Created temporary backup directory: $DOCKER_BACKUP_DIR"
# Export Docker volumes
log "Exporting Docker volumes..."
# Gitea
log " - Exporting Gitea data..."
docker run --rm -v gitea_gitea:/data -v "$DOCKER_BACKUP_DIR":/backup alpine \
tar czf /backup/gitea-data.tar.gz -C /data .
# Jellyfin
log " - Exporting Jellyfin config..."
docker run --rm -v jellyfin_jellyfin_config:/data -v "$DOCKER_BACKUP_DIR":/backup alpine \
tar czf /backup/jellyfin-config.tar.gz -C /data .
# Nextcloud data
log " - Exporting Nextcloud data..."
docker run --rm -v nextcloud_nextcloud_data:/data -v "$DOCKER_BACKUP_DIR":/backup alpine \
tar czf /backup/nextcloud-data.tar.gz -C /data .
# Nextcloud database
log " - Exporting Nextcloud database..."
docker run --rm -v nextcloud_nextcloud_db:/data -v "$DOCKER_BACKUP_DIR":/backup alpine \
tar czf /backup/nextcloud-db.tar.gz -C /data .
# Portainer
log " - Exporting Portainer data..."
docker run --rm -v portainer_data:/data -v "$DOCKER_BACKUP_DIR":/backup alpine \
tar czf /backup/portainer-data.tar.gz -C /data .
log "✓ Docker volumes exported"
# Run backup
log "Running Restic backup..."
restic -r "$BACKUP_REPO" backup \
--verbose \
--tag homelab-docker \
--tag homelab-configs \
/etc/nginx/sites-available/homelab \
/etc/systemd/system/copyparty.service \
/home/hoborg/.config/copyparty/ \
"$DOCKER_BACKUP_DIR"
# Cleanup temporary files
log "Cleaning up temporary files..."
rm -rf "$DOCKER_BACKUP_DIR"
# Prune old backups (keep: 7 daily, 4 weekly, 6 monthly, 2 yearly)
log "Pruning old backups..."
restic -r "$BACKUP_REPO" forget \
--keep-daily 7 \
--keep-weekly 4 \
--keep-monthly 6 \
--keep-yearly 2 \
--prune
# Show backup stats
log "Backup statistics:"
restic -r "$BACKUP_REPO" stats --mode restore-size
log "========================================"
log "Backup completed successfully!"
log "========================================"

74
scripts/backup-init.sh Executable file
View File

@@ -0,0 +1,74 @@
#!/bin/bash
# Initialize Restic backup repository on NAS
# Run once to set up the backup system
set -e
# Configuration
BACKUP_REPO="/mnt/nas/backups/homelab-restic"
PASSWORD_FILE="/home/hoborg/creds/restic-password.txt"
echo "========================================"
echo "Restic Backup Repository Initialization"
echo "========================================"
echo ""
echo "This will:"
echo " 1. Install restic if needed"
echo " 2. Create backup repository at: $BACKUP_REPO"
echo " 3. Generate and save encryption password"
echo ""
# Check if restic is installed
if ! command -v restic &> /dev/null; then
echo "Installing restic..."
pacman -S --needed --noconfirm restic
echo "✓ Restic installed"
else
echo "✓ Restic already installed ($(restic version))"
fi
# Create backup directory on NAS
echo ""
echo "Creating backup directory on NAS..."
mkdir -p "$BACKUP_REPO"
echo "✓ Directory created: $BACKUP_REPO"
# Generate random password if doesn't exist
echo ""
if [ -f "$PASSWORD_FILE" ]; then
echo "✓ Password file already exists: $PASSWORD_FILE"
else
echo "Generating encryption password..."
mkdir -p /home/hoborg/creds
openssl rand -base64 32 > "$PASSWORD_FILE"
chmod 600 "$PASSWORD_FILE"
echo "✓ Password saved to: $PASSWORD_FILE"
echo "⚠️ IMPORTANT: Back up this password file! Without it, backups cannot be restored."
fi
# Export password for restic
export RESTIC_PASSWORD_FILE="$PASSWORD_FILE"
# Initialize repository
echo ""
echo "Initializing Restic repository..."
if restic -r "$BACKUP_REPO" snapshots &>/dev/null; then
echo "✓ Repository already initialized"
else
restic -r "$BACKUP_REPO" init
echo "✓ Repository initialized"
fi
echo ""
echo "========================================"
echo "Backup Repository Ready!"
echo "========================================"
echo ""
echo "Repository location: $BACKUP_REPO"
echo "Password file: $PASSWORD_FILE"
echo ""
echo "Next steps:"
echo " 1. Run backup-homelab.sh to create first backup"
echo " 2. Test restore with backup-restore.sh"
echo " 3. Enable automated backups with systemd timer"
echo ""

128
scripts/backup-system-full.sh Executable file
View File

@@ -0,0 +1,128 @@
#!/bin/bash
# Full system backup - packages, configs, and Docker volumes
# Can be run manually or via systemd timer
set -e
# Configuration
BACKUP_REPO="/mnt/nas/backups/homelab-restic"
PASSWORD_FILE="/home/hoborg/creds/restic-password.txt"
LOG_FILE="/var/log/homelab-backup.log"
DOCKER_BACKUP_DIR="/tmp/docker-backup-$$"
SYSTEM_BACKUP_DIR="/tmp/system-backup-$$"
# Export password
export RESTIC_PASSWORD_FILE="$PASSWORD_FILE"
# Logging function
log() {
echo "[$(date +'%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE"
}
log "========================================"
log "Starting Full System Backup"
log "========================================"
# Check if repository exists
if ! restic -r "$BACKUP_REPO" snapshots &>/dev/null; then
log "ERROR: Backup repository not initialized. Run backup-init.sh first."
exit 1
fi
# Create temporary directories
mkdir -p "$DOCKER_BACKUP_DIR"
mkdir -p "$SYSTEM_BACKUP_DIR"
log "Created temporary directories"
# ============================================
# 1. Export Package Lists
# ============================================
log "Exporting package lists..."
pacman -Qqe > "$SYSTEM_BACKUP_DIR/pacman-explicit.txt" # Explicitly installed packages
pacman -Qq > "$SYSTEM_BACKUP_DIR/pacman-all.txt" # All installed packages
pacman -Qqm > "$SYSTEM_BACKUP_DIR/aur-packages.txt" # AUR/foreign packages only
log "✓ Package lists exported ($(wc -l < $SYSTEM_BACKUP_DIR/pacman-explicit.txt) explicit, $(wc -l < $SYSTEM_BACKUP_DIR/aur-packages.txt) AUR)"
# ============================================
# 2. Export Docker Volumes
# ============================================
log "Exporting Docker volumes..."
# Gitea
log " - Exporting Gitea data..."
docker run --rm -v gitea_gitea:/data -v "$DOCKER_BACKUP_DIR":/backup alpine \
tar czf /backup/gitea-data.tar.gz -C /data .
# Jellyfin
log " - Exporting Jellyfin config..."
docker run --rm -v jellyfin_jellyfin_config:/data -v "$DOCKER_BACKUP_DIR":/backup alpine \
tar czf /backup/jellyfin-config.tar.gz -C /data .
# Nextcloud data
log " - Exporting Nextcloud data..."
docker run --rm -v nextcloud_nextcloud_data:/data -v "$DOCKER_BACKUP_DIR":/backup alpine \
tar czf /backup/nextcloud-data.tar.gz -C /data .
# Nextcloud database
log " - Exporting Nextcloud database..."
docker run --rm -v nextcloud_nextcloud_db:/data -v "$DOCKER_BACKUP_DIR":/backup alpine \
tar czf /backup/nextcloud-db.tar.gz -C /data .
# Portainer
log " - Exporting Portainer data..."
docker run --rm -v portainer_data:/data -v "$DOCKER_BACKUP_DIR":/backup alpine \
tar czf /backup/portainer-data.tar.gz -C /data .
log "✓ Docker volumes exported"
# ============================================
# 3. Run Backup
# ============================================
log "Running Restic backup..."
restic -r "$BACKUP_REPO" backup \
--verbose \
--tag full-system \
--exclude '/etc/shadow*' \
--exclude '/etc/gshadow*' \
--exclude '/etc/passwd-' \
--exclude '/etc/group-' \
/etc/ \
"$DOCKER_BACKUP_DIR" \
"$SYSTEM_BACKUP_DIR"
# ============================================
# 4. Cleanup
# ============================================
log "Cleaning up temporary files..."
rm -rf "$DOCKER_BACKUP_DIR"
rm -rf "$SYSTEM_BACKUP_DIR"
# ============================================
# 5. Prune Old Backups
# ============================================
log "Pruning old backups..."
restic -r "$BACKUP_REPO" forget \
--keep-daily 7 \
--keep-weekly 4 \
--keep-monthly 3 \
--keep-yearly 1 \
--prune
# ============================================
# 6. Show Stats
# ============================================
log "Backup statistics:"
restic -r "$BACKUP_REPO" stats --mode restore-size
log "========================================"
log "Full system backup completed!"
log "========================================"
log ""
log "Backed up:"
log " - /etc/ (all system configs)"
log " - Docker volumes (Gitea, Jellyfin, Nextcloud, Portainer)"
log " - Package lists (pacman + AUR)"
log ""
log "To restore packages after reinstall:"
log " pacman -S --needed \$(cat pacman-explicit.txt)"
log " yay -S --needed \$(cat aur-packages.txt)"