homelab/docs/services.md

# Services & Applications

Planning and configuration for self-hosted services and applications.

## Nginx Reverse Proxy

**Status**: ✅ Active
**Port**: 80 (HTTP), 443 (HTTPS)
**Configuration**: `/etc/nginx/sites-available/homelab`

### Features
- Path-based routing to services (`/gitea/`, `/files/`, `/media/`)
- SSL termination with Let's Encrypt certificates
- Security headers (XSS protection, content type sniffing prevention)
- WebDAV support for Copyparty file server
- Custom landing page at domain root with service dashboard
- HTTP to HTTPS redirects
- Custom error pages

### Configuration Notes
- Uses `default_server` directive to handle all requests to port 80/443
- Default nginx server block in `/etc/nginx/nginx.conf` is commented out to prevent conflicts
- Landing page served from `/var/www/homelab/index.html`
- Root location uses `location /` (not `location = /`) for proper index file handling

### Management
```bash
# Deploy configuration from repo
sudo cp config/nginx/homelab.conf /etc/nginx/sites-available/homelab

# Test configuration syntax
sudo nginx -t

# Restart nginx (recommended over reload for major config changes)
sudo systemctl restart nginx

# View logs
sudo journalctl -u nginx -f

# Check active configuration
sudo nginx -T | grep -A20 "server_name ak-homelab"
```

### Landing Page
The homelab landing page (`/var/www/homelab/index.html`) provides a dashboard with:
- Service status indicators
- Direct links to all services
- Responsive design with gradient styling
- Service icons and descriptions

Update the landing page:
```bash
# Copy from repo (if you have a config/www/index.html)
sudo cp config/www/index.html /var/www/homelab/
sudo chown http:http /var/www/homelab/index.html
```

### Troubleshooting
For landing page 404 issues, see the detailed troubleshooting guide in `docs/troubleshooting.md` under "Landing Page Returns 404 Not Found".

Common debugging commands:
```bash
# Check nginx configuration is active
sudo nginx -T | grep -A10 "server_name ak-homelab"

# Test landing page access
curl -I https://ak-homelab.duckdns.org/

# Monitor nginx logs in real-time
sudo journalctl -u nginx -f

# Verify file permissions
sudo ls -la /var/www/homelab/index.html
```

## Git Repository Hosting

### Service Options
- **Gitea**: Lightweight, Go-based, minimal resource usage ✅ *Recommended*
- **Forgejo**: Gitea fork, community-driven development
- **GitLab CE**: Feature-rich but more resource intensive
- **Gogs**: Simple, lightweight alternative

### Gitea Installation (Docker - Recommended)

**Status:** ✅ **Container Running** ⚠️ **Configuration Issues** - UI accessible, config needs debugging

**Prerequisites:**
```bash
# Install Docker and Docker Compose
sudo pacman -S docker docker-compose
sudo systemctl enable docker
sudo systemctl start docker

# Add user to docker group (logout/login required)
sudo usermod -aG docker hoborg
```

**Gitea Docker Setup:**
```bash
# Create directories for persistent data
mkdir -p ~/docker/gitea/{data,config}

# Run Gitea container
docker run -d \
  --name gitea \
  -p 3000:3000 \
  -p 2222:22 \
  -v ~/docker/gitea/data:/data \
  -v ~/docker/gitea/config:/etc/gitea \
  -e USER_UID=1000 \
  -e USER_GID=1000 \
  gitea/gitea:latest

# Alternative: Docker Compose (preferred)
# See docker-compose.yml below
```

**Docker Compose Configuration:**
Create `~/docker/gitea/docker-compose.yml`:
```yaml
version: "3"

networks:
  gitea:
    external: false

services:
  server:
    image: gitea/gitea:latest
    container_name: gitea
    environment:
      - USER_UID=1000
      - USER_GID=1000
    restart: always
    networks:
      - gitea
    volumes:
      - ./data:/data
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
    ports:
      - "3000:3000"
      - "2223:22"  # Gitea SSH (avoid conflict with system SSH on 2222)
```

**Start with Docker Compose:**
```bash
cd ~/docker/gitea
docker-compose up -d
```

### Current Configuration

**Container Status:** ✅ Running at /opt/docker/gitea
**Access URLs:**
- **Local Web**: http://192.168.0.100:3000
- **External Web**: http://ak-homelab.duckdns.org:3000 (requires port forwarding)
- **Git SSH**: ssh://git@ak-homelab.duckdns.org:2223 (requires port forwarding)

**Port Assignments:**
- **System SSH**: 2222 (for server administration)
- **Gitea SSH**: 2223 (for Git operations)
- **Gitea Web**: 3000 (web interface)

**Database**: SQLite (default, stored in container volume)
**Data Location**: /opt/docker/gitea/data (persistent volume)

**Router Port Forwarding Required:**
- 3000 → 192.168.0.100:3000 (web interface)
- 2223 → 192.168.0.100:2223 (Git SSH operations)

**Container Management:**
```bash
cd /opt/docker/gitea
docker-compose logs gitea        # View logs
docker-compose down             # Stop container
docker-compose up -d            # Start container
docker-compose pull && docker-compose up -d  # Update
```

### Gitea Configuration Recommendations

**Database Selection:**
- ✅ **SQLite** (Recommended for homelab)
  - Zero configuration, single file
  - Perfect for personal/small team use
  - Easy backups and migration
- **PostgreSQL** (Production alternative)
  - Better performance for heavy usage
  - Requires additional container setup

**Domain & URL Configuration:**
- **Server Domain**: `ak-homelab.duckdns.org`
- **Gitea Base URL**: `http://ak-homelab.duckdns.org/gitea/`
- **SSH Domain**: `ak-homelab.duckdns.org`
- **SSH Port**: `2223`

**Password Hash Algorithm:**
- ✅ **Argon2** (Recommended)
  - Most secure modern algorithm
  - Resistant to GPU/ASIC attacks
  - Higher CPU usage but excellent security
- **bcrypt** (Alternative)
  - Well-tested, lower resource usage
  - Good balance of security and performance

**Setup Progress:**
1. ✅ Gitea container running
2. ✅ Nginx reverse proxy setup complete
3. ✅ Router port forwarding (80, 443) - **COMPLETE - External access working**
4. ✅ Gitea web configuration - **COMPLETE**
   - Initial setup wizard completed
   - Admin user account created
   - SSH access configured
   - Repository migration completed
5. ✅ SSL certificate setup - **COMPLETE - Let's Encrypt with auto-renewal**

**Current Access:**
- ✅ Local UI working: https://192.168.0.100/gitea/ (HTTPS with SSL)
- ✅ External access: https://ak-homelab.duckdns.org/gitea/ - **WORKING (HTTPS enabled)**
- ✅ Git SSH access: ssh://git@ak-homelab.duckdns.org:2223 - **WORKING**

**Completed Configuration:**
- Router forwards: 80→80, 443→443, 2223→2223
- Removed direct port 3000 forwarding
- Homelab repository successfully migrated to Gitea
- External access confirmed working from Windows PC
- SSL certificates installed with Let's Encrypt
- Automatic HTTP→HTTPS redirect enabled
- Certificate auto-renewal configured

## Cloud Storage Solutions

### Service Options
- **Copyparty**: Lightweight file server with resumable uploads, dedup, WebDAV ✅ **DEPLOYED**
- **Nextcloud**: Full-featured, extensive app ecosystem
- **ownCloud**: Original project, stable and mature
- **Seafile**: Performance-focused file sync
- **Syncthing**: Decentralized sync (no server needed) ✅ **INSTALLED**

### Copyparty Installation and Configuration

**Status:** ✅ **DEPLOYED AND WORKING** - File server with upload/download capabilities

**Installation:**
```bash
# Installed via Arch package
sudo pacman -S copyparty

# Configuration file location
/home/hoborg/.config/copyparty/copyparty.conf

# Systemd service location
/etc/systemd/system/copyparty.service
```

**Current Setup:**
- **Local Access**: https://127.0.0.1/files/ (SSL enabled)
- **External Access**: https://ak-homelab.duckdns.org/files/ (HTTPS with Let's Encrypt SSL)
- **WebDAV Access**: https://ak-homelab.duckdns.org/files/ (X-plore, rclone compatible)
- **Port**: 8082 (behind Nginx reverse proxy on /files/ path)
- **Service**: Managed by systemd, auto-starts on boot
- **SSL**: Let's Encrypt certificates with automatic renewal

**User Accounts:**
- **guest**: Standard user with read/write (`rw`) access to shared areas
- **hoborg**: Admin user with full access (`rwmd` - read/write/move/delete) to all areas including private folder

**Features:**
- ✅ File upload/download via web interface
- ✅ WebDAV support for X-plore File Manager, rclone, etc.
- ✅ File deletion via WebDAV (requires `d` permission)
- ✅ Drag & drop upload in web interface
- ✅ Support for files with spaces/special characters
- ✅ Large file upload support (up to 10GB)
- ✅ Resume interrupted uploads
- ✅ File deduplication and integrity checking

**Volume Structure:**
```
/shared     → /home/hoborg/shared       (guest: rw, hoborg: rwmd)
/documents  → /home/hoborg/Documents    (hoborg: rwmd)
/music      → /home/hoborg/Music        (guest: rw, hoborg: rwmd)
/videos     → /home/hoborg/Videos       (guest: rw, hoborg: rwmd)
/pictures   → /home/hoborg/Pictures     (guest: rw, hoborg: rwmd)
/private    → /home/hoborg/private      (hoborg only: rwmd)
```

**Features Enabled:**
- User-changeable passwords (stored securely in encrypted database)
- Upload deduplication (saves storage space)
- File indexing and search (e2dsa)
- Resumable uploads with up2k
- File integrity verification
- Thumbnail generation for images and videos

**Security:**
- Authentication required for all access
- Passwords stored in encrypted format: `/home/hoborg/.config/copyparty/passwords.json`
- Admin-only private folder isolated from shared areas
- Reverse proxy headers for proper client IP logging

**Service Management:**
```bash
# Check status
sudo systemctl status copyparty

# View logs
journalctl -fu copyparty

# Restart service
sudo systemctl restart copyparty

# Enable/disable autostart
sudo systemctl enable copyparty
sudo systemctl disable copyparty
```

**Configuration Files:**
- **Main config**: `/home/hoborg/.config/copyparty/copyparty.conf`
- **Systemd service**: `/home/hoborg/homelab/config/systemd/copyparty.service`
- **Nginx integration**: Path `/files/` in homelab.conf

**WebDAV Client Setup:**

*X-plore File Manager (Android):*
- Server: `ak-homelab.duckdns.org`
- Path: `/files/shared/` (or other folder paths)
- Protocol: HTTPS (port 443)
- Username: `hoborg`
- Password: [your password]

*rclone configuration:*
```bash
rclone config create homelab-webdav webdav \
  url=https://ak-homelab.duckdns.org/files/ \
  vendor=other \
  user=hoborg \
  pass=$(rclone obscure "your_password")
```

**Troubleshooting:**
- For issues with files containing spaces, see [docs/troubleshooting/webdav-copyparty.md](troubleshooting/webdav-copyparty.md)
- Check nginx WebDAV configuration for URL encoding issues
- Verify copyparty permissions include `d` flag for delete operations

**Testing Confirmed:**
- ✅ File uploads working (including video files)
- ✅ WebDAV folder uploads from Android (X-plore File Manager)
- ✅ User authentication and authorization
- ✅ Private folder access restricted to admin
- ✅ External access through reverse proxy
- ✅ Service auto-starts on boot

### Nextcloud Installation
```bash
# Via snap (recommended)
sudo snap install nextcloud

# Or via Docker
docker run -d \
  --name nextcloud \
  -p 8080:80 \
  -v nextcloud_data:/var/www/html \
  nextcloud
```
Personal notes: Not a fan of snap, isn't there an AUR package?
Go with docker otherwise

### Features
- File synchronization across devices
- Video files, game installers -> high prio
- Self-hosted git mirrors of favorite FOSS projects -> medium prio
- Calendar and contacts (CalDAV/CardDAV) -> low prio
- Document editing (OnlyOffice/Collabora) -> low prio
- Photo management and sharing -> low prio
- Mobile apps available?

## Media Management

### Jellyfin Media Server

**Status:** ✅ **DEPLOYED AND WORKING** - Docker-based media server with hardware acceleration

**Installation:**
```bash
# Deploy configuration files
sudo mkdir -p /opt/docker/jellyfin/{config,cache}
sudo cp config/docker/jellyfin/docker-compose.yml /opt/docker/jellyfin/

# Start container
cd /opt/docker/jellyfin
sudo docker-compose up -d
```

**Current Setup:**
- **Local Access**: http://localhost:8096
- **External Access**: https://ak-homelab.duckdns.org/media/ (HTTPS with Let's Encrypt SSL)
- **Container**: jellyfin/jellyfin:latest with network_mode: host
- **Hardware Acceleration**: VAAPI, QSV, CUDA support enabled

**Media Library Structure:**
```
/media/music    → /home/hoborg/Music        (shared with Copyparty)
/media/videos   → /home/hoborg/Videos       (shared with Copyparty)
/media/pictures → /home/hoborg/Pictures     (shared with Copyparty)
/media/shared   → /home/hoborg/shared       (shared with Copyparty)
/media/private  → /home/hoborg/private      (shared with Copyparty)
```

**Features Enabled:**
- Hardware transcoding (Intel/AMD GPU via /dev/dri)
- Multiple codec support (H.264, H.265, AV1, etc.)
- Hardware acceleration filters and encoders
- Reverse proxy integration with optimized streaming settings
- Resource limits: 2GB max memory, 512MB reserved

**Configuration Management:**
```bash
# Check container status
cd /opt/docker/jellyfin && sudo docker-compose logs jellyfin

# Container management
sudo docker-compose down             # Stop
sudo docker-compose up -d            # Start
sudo docker-compose pull && sudo docker-compose up -d  # Update

# View detailed logs
sudo docker-compose logs -f jellyfin
```

**Integration with Other Services:**
- **Copyparty**: Shares same media folders without duplication
- **Nginx**: Reverse proxy with streaming-optimized configuration
- **SSL**: Secured with Let's Encrypt certificates and auto-renewal

**Initial Setup:**
1. Visit http://localhost:8096 for setup wizard
2. Create admin account and configure server
3. Add media libraries pointing to `/media/music`, `/media/videos`, `/media/pictures`, `/media/shared`, `/media/private`
4. Configure hardware acceleration in Dashboard > Playback settings

**Folder Structure Compatibility: ✅ CONFIRMED WORKING**
- **Music Library**: Successfully works with existing Artist/Album structure
- **Copyparty Interoperability**: ✅ Both services share same folders without conflicts
- **Metadata Support**: Downloads album artwork, descriptions, and artist information from online sources
- **Electronic Music**: Works well with existing organized structure (e.g., "The Hu - Discography/Albums/...")

**Metadata Management:**
- **Auto-Download**: Enable MusicBrainz, TheAudioDB, Last.fm in Dashboard > Metadata > Music
- **Manual Refresh**: Library scan via Dashboard > Libraries > Music > Scan Library
- **Force Refresh**: Use "Replace All Metadata" for complete metadata re-download
- **Individual Items**: Right-click albums/artists for targeted metadata refresh

### Music Collection Management

**Playlist Extraction:**
- **YouTube Music**: Use `ytmusicapi` Python library or Google Takeout
- **SoundCloud**: Use `scdl` tool or SoundCloud API
- **Output Formats**: CSV (artist, title, playlist), JSON metadata, M3U playlists, plain text lists

**Legal Music Sources:**
- **High Quality**: Bandcamp (FLAC), Beatport, 7digital, HDtracks
- **Mainstream**: Amazon Music, iTunes Store, artist websites
- **Subscription Downloads**: Tidal, Qobuz premium tiers
- **Physical Media**: CD ripping, vinyl digitization, cassette conversion

**Organization Tools:**
- **MusicBrainz Picard**: Automatic metadata tagging and correction
- **beets**: Advanced music library management and organization
- **Bulk Scripts**: Custom automation for file organization and import

### Photo Management
- **PhotoPrism**: AI-powered photo management
- **Immich**: Modern photo backup solution
- **LibrePhotos**: Privacy-focused alternative

## Monitoring & Logging

### System Monitoring
```bash
# Prometheus + Grafana stack
docker-compose up -d prometheus grafana node-exporter
```

### Log Management
- **Centralized logging**: rsyslog or journald
- **Log rotation**: logrotate configuration
- **Analysis**: grep, awk, or ELK stack for advanced needs

### Health Checks
- **Uptime monitoring**: Simple HTTP checks
- **Service status**: systemd service monitoring
- **Disk space**: Automated alerts for low space

## Containerization Strategy

### Docker Setup
```bash
# Install Docker
pacman -S docker docker-compose
sudo systemctl enable docker

# Add user to docker group
sudo usermod -aG docker hoborg
```

### Container Management
- **Orchestration**: Docker Compose for multi-service apps
- **Storage**: Named volumes for persistent data
- **Networking**: Custom networks for service isolation
- **Updates**: Watchtower for automated updates

## Reverse Proxy Configuration

### Decision Matrix: Nginx vs Traefik

**Problem:** Multiple services need to be accessible under single domain with path-based routing.
**Solution:** Reverse proxy to route requests based on URL paths.

#### Nginx
✅ **Pros:**
- Mature & battle-tested (decades of production use)
- High performance for static files and caching
- Flexible configuration - handles any routing scenario
- Extensive ecosystem and documentation
- Can handle non-Docker services easily

❌ **Cons:**
- Manual configuration required
- No auto-discovery of services
- Requires config reload for changes

#### Traefik
✅ **Pros:**
- Docker-native auto-discovery via labels
- Automatic HTTPS with Let's Encrypt
- Dynamic configuration (no restarts)
- Modern design for containerized environments

❌ **Cons:**
- Less mature, smaller ecosystem
- Docker-dependent (less flexible)
- Steeper learning curve for complex scenarios

**Decision:** ✅ **Nginx** - Better for homelab due to reliability, documentation, and flexibility for mixed Docker/non-Docker services.

### URL Architecture Decision

**Options Considered:**
1. **Separate subdomains**: `gitea.ak-homelab.duckdns.org` ❌ *DuckDNS doesn't support nested subdomains*
2. **Multiple DuckDNS domains**: `ak-homelab-git.duckdns.org` ❌ *Management overhead*
3. **Path-based routing**: `ak-homelab.duckdns.org/gitea/` ✅ **Selected**

**Chosen Architecture:**
```
https://ak-homelab.duckdns.org/          → Landing page/dashboard
https://ak-homelab.duckdns.org/gitea/    → Gitea Git server ✅ DEPLOYED
https://ak-homelab.duckdns.org/files/    → Copyparty file server (WebDAV) ✅ DEPLOYED
https://ak-homelab.duckdns.org/media/    → Jellyfin media server ✅ DEPLOYED
https://ak-homelab.duckdns.org/monitor/  → System monitoring (planned)
```

### SSL/TLS Configuration

**Status:** ✅ **COMPLETE** - Let's Encrypt SSL with automatic renewal

**SSL Certificate Setup:**
```bash
# Certificate installation (completed)
sudo certbot --nginx -d ak-homelab.duckdns.org

# Auto-renewal service (enabled)
sudo systemctl enable certbot-renew.timer
sudo systemctl start certbot-renew.timer
```

**Certificate Details:**
- **CA**: Let's Encrypt (trusted by all major browsers)
- **Certificate**: `/etc/letsencrypt/live/ak-homelab.duckdns.org/fullchain.pem`
- **Private Key**: `/etc/letsencrypt/live/ak-homelab.duckdns.org/privkey.pem`
- **Renewal**: Automatic every 90 days via systemd timer
- **Security**: Strong TLS configuration with DH parameters

**Security Features:**
- **HTTP→HTTPS Redirect**: All HTTP traffic automatically redirected to HTTPS
- **HSTS Headers**: Included via Let's Encrypt nginx configuration
- **Strong Ciphers**: Modern TLS 1.2/1.3 cipher suites
- **Perfect Forward Secrecy**: Enabled via Diffie-Hellman parameters

**Renewal Management:**
```bash
# Check renewal status
sudo systemctl status certbot-renew.timer

# Manual renewal test (dry run)
sudo certbot renew --dry-run

# View renewal logs
journalctl -u certbot-renew.service
```

**Certificate Verification:**
- Browser shows green padlock for all services
- SSL Labs rating: A+ (expected)
- All services accessible via HTTPS only

### Advanced Nginx Options

**Nginx Plus (Commercial):** $2500+/year
- Advanced load balancing, health checks
- API management, JWT validation
- Real-time monitoring dashboard
- **Verdict:** Overkill for homelab

**Nginx + Lua (OpenResty):**
- Embed Lua scripts for dynamic processing
- Complex routing logic, authentication
- API gateway functionality
- **Verdict:** Powerful but complex, not needed initially

### Nginx Setup

**Status:** ✅ **Complete** - Reverse proxy configured and running

```bash
# Install nginx and SSL tools
sudo pacman -S nginx certbot certbot-nginx

# Create configuration
sudo nano /etc/nginx/sites-available/homelab

# Enable site
sudo ln -s /etc/nginx/sites-available/homelab /etc/nginx/sites-enabled/

# Test and reload
sudo nginx -t
sudo systemctl reload nginx
```

**Configuration Template:**
```nginx
server {
    listen 80;
    server_name ak-homelab.duckdns.org;

    # Main landing page
    location / {
        root /var/www/homelab;
        index index.html;
    }

    # Gitea reverse proxy
    location /gitea/ {
        proxy_pass http://127.0.0.1:3000/;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

    # Future services
    # location /cloud/ {
    #     proxy_pass http://127.0.0.1:8080/;
    #     proxy_set_header Host $host;
    # }
}
```

### Router Port Forwarding Requirements
- **HTTP**: Port 80 → 192.168.0.100:80
- **HTTPS**: Port 443 → 192.168.0.100:443
- **Remove**: Direct port 3000 forwarding (will go through nginx)

### SSL Certificates
```bash
# Let's Encrypt via certbot (after nginx setup)
sudo certbot --nginx -d ak-homelab.duckdns.org

# Auto-renewal
sudo systemctl enable certbot.timer
```

## Backup Strategy

### Configuration Backups
- **Service configs**: Docker volumes, /etc configs
- **Database dumps**: Regular automated backups
- **Storage**: External drive or cloud backup

### Automated Backups
```bash
#!/bin/bash
# backup-services.sh
DATE=$(date +%Y%m%d)

# Backup Gitea
tar -czf /backup/gitea-$DATE.tar.gz /var/lib/gitea/

# Backup Nextcloud data
rsync -av /var/snap/nextcloud/common/nextcloud/data/ /backup/nextcloud-$DATE/

# Database backup
sudo -u postgres pg_dump gitea > /backup/gitea-db-$DATE.sql
```

## Resource Planning

### Hardware Requirements
- **RAM**: 4GB minimum, 8GB recommended
- **Storage**:
  - System: 50GB SSD
  - Data: 1TB+ HDD for media/files
- **Network**: Gigabit Ethernet preferred

### Service Resource Usage
| Service | RAM | CPU | Storage | Port |
|---------|-----|-----|---------|------|
| Gitea | 200MB | Low | 5GB+ | 3000 |
| Nextcloud | 512MB | Medium | 10GB+ | 8080 |
| Jellyfin | 1GB | High* | Media | 8096 |
| Monitoring | 500MB | Low | 2GB | 3000/9090 |

*High during transcoding

## Security Considerations

### Service Hardening
- **Regular updates**: Automated security patches
- **Access control**: VPN-only access when possible
- **Authentication**: Strong passwords, 2FA where available
- **Network isolation**: Separate VLANs or containers

### Data Protection
- **Encryption**: Full disk encryption (LUKS)
- **Backups**: Encrypted offsite backups
- **Access logs**: Monitor service access patterns
- **Fail2ban**: Automatic IP blocking for repeated failures

## Future Expansion

### Additional Services to Consider
- **Home Assistant**: ABSOLUTELY NOT
- **Bitwarden/Vaultwarden**: Password management
  - How is this better than keepassxc + filesync?
- **Pi-hole**: Network-wide ad blocking
- **Wireguard UI**: Web interface for VPN management
- **Bookstack**: Documentation wiki
  - What is this for? How does it compare to Logseq?
- **FreshRSS**: RSS feed aggregator

## System Monitoring & Management

### Overview

**Status:** 🚧 **PLANNED** - Implementing hybrid monitoring and management solution

**Selected Tools:**
- **Portainer** - Docker container management (web UI with built-in auth)
- **Glances** - Real-time system monitoring (web + terminal, nginx basic auth required)
- **Netdata** - Real-time system monitoring with rich dashboards (web UI, nginx basic auth)
- **lazydocker** - Terminal-based Docker management (SSH sessions)

### Architecture Decision

**Hybrid Approach Rationale:**
- **SSH workflow**: lazydocker + glances terminal mode for command-line administration
- **Web overview**: Glances for quick system status checks
- **Real-time monitoring**: Netdata for detailed system metrics and historical data
- **Docker UI**: Portainer for comprehensive container management

### Authentication Strategy

- **Portainer**: ✅ Built-in user authentication and RBAC
- **Glances**: ⚠️ Nginx basic auth required (exposes system metrics)
- **Netdata**: ⚠️ Nginx basic auth required (exposes system metrics, cloud features disabled)
- **Router**: ✅ Has own administrative login
- **NAS Storage**: ✅ Has own administrative login

**Nginx basic auth implemented** for monitoring services that expose system information without built-in authentication.

### Service Details

#### Portainer (Docker Management)
**Status:** 📋 **Planned**
- **Access**: https://ak-homelab.duckdns.org/portainer/
- **Port**: 9000 (behind reverse proxy)
- **Authentication**: Built-in user accounts with role-based permissions
- **Features**: Container lifecycle, image management, volume management, stack deployment

#### Glances (System Monitoring)
**Status:** 📋 **Planned**
- **Access**: https://ak-homelab.duckdns.org/glances/ (nginx basic auth)
- **Port**: 61208 (behind reverse proxy with auth)
- **Authentication**: Nginx basic auth (due to no built-in authentication)
- **Features**: Real-time CPU/RAM/disk metrics, process monitoring, network stats
- **Terminal mode**: Available via SSH for command-line monitoring

#### Netdata (Real-time System Monitoring)
**Status:** ✅ **DEPLOYED**
- **Access**: https://ak-homelab.duckdns.org/netdata/ (nginx basic auth)
- **Port**: 19999 (behind reverse proxy with auth)
- **Authentication**: Nginx basic auth (same credentials as Glances: admin/AdminPass2024!)
- **Configuration**: Privacy-focused local-only setup with cloud features disabled
- **Features**: Real-time system metrics, network monitoring, process tracking, historical data

#### lazydocker (Terminal Docker Tools)
**Status:** 📋 **Planned**
- **Access**: SSH terminal only
- **Installation**: `pacman -S lazydocker`
- **Usage**: Command-line Docker container management for SSH workflows

### URL Architecture

```
Landing Page - Admin Tab:
Server Administration:
├── Glances → https://ak-homelab.duckdns.org/glances/ (nginx basic auth)
├── Netdata → https://ak-homelab.duckdns.org/netdata/ (nginx basic auth)
└── Portainer → https://ak-homelab.duckdns.org/portainer/ (built-in auth)

Local Network:
├── NAS Storage → http://192.168.0.101:5000/ (built-in auth)
└── Router → http://192.168.0.1 (built-in auth)
```

### Implementation Plan

1. **Package Installation**
   ```bash
   sudo pacman -S glances cockpit lazydocker
   ```

2. **Portainer Deployment**
   ```bash
   docker run -d \
     --name portainer \
     -p 9000:9000 \
     -v /var/run/docker.sock:/var/run/docker.sock \
     -v portainer_data:/data \
     portainer/portainer-ce
   ```

3. **Service Configuration**
   - Enable Cockpit: `sudo systemctl enable --now cockpit.socket`
   - Configure Glances web mode: `glances -w -p 61208`
   - Create systemd service for Glances web server

4. **Nginx Configuration**
   - Add reverse proxy configurations for all services
   - Configure basic auth for Glances endpoint
   - SSL termination for all admin services

5. **Landing Page Update**
   - Add all admin service links to Admin tab
   - Include authentication indicators

### Security Considerations

**Data Exposure Analysis (Glances):**
- **Exposed**: System metrics, process names, resource usage, network stats
- **Not Exposed**: File contents, passwords, configuration details, logs
- **Risk Level**: Medium (reconnaissance data for attackers)
- **Mitigation**: Nginx basic auth prevents unauthorized access

**Service Hardening:**
- All services behind HTTPS with SSL certificates
- Each service handles authentication independently
- No shared credentials between services
- Services isolated behind reverse proxy

## Remote Desktop Access

### TigerVNC

**Status:** ✅ **INSTALLED** - VNC server for remote desktop access

**Installation:**
```bash
# Install TigerVNC server and client
sudo pacman -S tigervnc
```

**Service Configuration:**
```bash
# Configure user for VNC display :1
echo ":1=hoborg" | sudo tee -a /etc/tigervnc/vncserver.users

# Set VNC password
vncpasswd

# Configure desktop environment (~/.vnc/xstartup)
#!/bin/bash
xrdb $HOME/.Xresources
startxfce4 &

# Make executable
chmod +x ~/.vnc/xstartup
```

**Service Management:**
```bash
# Enable and start VNC service
sudo systemctl enable vncserver@:1.service
sudo systemctl start vncserver@:1.service

# Check service status
sudo systemctl status vncserver@:1.service

# Service uses vncsession-start for proper X11 session management
```

**Access Details:**
- **Display**: `:1` (port 5901)
- **Local Access**: VNC client to `192.168.0.100:5901`
- **External Access**: Requires router port forwarding 5901→192.168.0.100:5901
- **Security**: Password authentication, consider SSH tunneling for external access

**Client Connection:**
- **Windows**: TigerVNC Viewer to `192.168.0.100:5901`
- **SSH Tunnel**: `ssh -L 5901:localhost:5901 hoborg@192.168.0.100 -p 2222`
- **Tunneled Access**: VNC client to `localhost:5901`

**Service Features:**
- ✅ Systemd integration with proper session management
- ✅ Automatic startup on boot
- ✅ User-specific VNC sessions via `/etc/tigervnc/vncserver.users`
- ✅ Uses `vncsession-start` for robust X11 handling
- ✅ Proper PID file management in `/run/vncsession-:1.pid`

**Security Considerations:**
- VNC traffic is unencrypted - use SSH tunneling for remote access
- Firewall configuration needed for direct external access
- Consider VPN access instead of direct port forwarding