homelab/docs/network-security.md

# Network & Security Configuration

Complete guide for securing and networking your homelab.

## SSH Security Setup

### Initial Configuration
```bash
# Generate SSH key pair
ssh-keygen -t ed25519 -C "homelab-key" -f ~/.ssh/homelab_ed25519

# Copy public key to target
ssh-copy-id -i ~/.ssh/homelab_ed25519.pub user@target
```

### SSH Hardening

**Status:** ✅ **Complete** - Port changed to 2222

Edit `/etc/ssh/sshd_config`:
```
# Disable root login
PermitRootLogin no

# Use key-based authentication only
PasswordAuthentication no
PubkeyAuthentication yes

# Change default port (CRITICAL - currently still on 22)
Port 2222

# Restrict users
AllowUsers hoborg

# Security settings
Protocol 2
X11Forwarding no
AllowTcpForwarding no
ClientAliveInterval 300
ClientAliveCountMax 2
MaxAuthTries 3
MaxStartups 2
```

**Completed:**
1. ✅ Changed SSH port from 22 to 2222
2. ✅ Updated router port forwarding rules
3. ✅ External access via ak-homelab.duckdns.org:2222 working

Restart SSH: `sudo systemctl restart sshd`

### Mosh Alternative (Investigation Needed)

**Issue:** SSH can be unreliable on WiFi connections with packet loss.

**Mosh Benefits:**
- Maintains connection during network switches (ethernet ↔ WiFi)
- Handles poor WiFi connections better
- Local echo for responsive typing
- Roaming support (IP changes don't break connection)

**Installation:**
```bash
# Server side
sudo pacman -S mosh

# Client side
mosh user@server
```

**Requirements:**
- UDP ports 60000-61000 open on router
- SSH still needed for initial authentication

**Status:** ✅ **Local use working** ❌ **External blocked by ISP**

**Key Findings:**
- **Local mosh**: Works perfectly (`mosh localhost`, `mosh 192.168.0.100`)
- **External mosh**: Blocked by ISP UDP port filtering on ports 60000-61000
- **SSH still needed**: Mosh uses SSH for initial authentication, then switches to UDP

**ISP UDP Blocking Issue:**
- Most ISPs block UDP ports 60000-61000 for "security"
- SSH works fine (TCP port 2222) but mosh fails (UDP 60000-61000)  
- Router port forwarding is correct, but ISP drops UDP packets

**Current Recommendation:**
- Use mosh for local/internal network connections
- Stick with SSH for external connections until VPN is set up
- VPN tunnel can bypass ISP UDP blocking

### SSH Client Configuration
Create `~/.ssh/config`:
```
Host homelab
    HostName ak-homelab.duckdns.org
    User hoborg
    Port 2222
    IdentityFile ~/.ssh/homelab_ed25519
    ServerAliveInterval 60
```

**Usage:**
```bash
# Connect via SSH
ssh homelab

# Connect via Mosh (uses SSH config automatically)
mosh homelab
```

## Dynamic DNS with DuckDNS

### Account Setup
1. Create account at duckdns.org
2. Create subdomain: `ak-homelab.duckdns.org`
3. Get token from dashboard

### Automatic IP Updates
Update script at `~/.local/scripts/duckdns.py` (Python implementation)

Cron job for automatic updates:
```bash
# Update every 5 minutes
*/5 * * * * /home/hoborg/.local/scripts/duckdns.py >/dev/null 2>&1
```

### Current Setup (Router-based)

**Status:** ✅ **Migrated from script to router DynDNS**

**Changes made:**
- ✅ Disabled cron job script (`*/5 * * * *` entry removed)
- ✅ Enabled router Dynamic DNS for ak-homelab.duckdns.org
- ⏳ **Testing pending** - Cannot force public IP change to verify

**Router DynDNS Benefits:**
- Immediate updates on IP change (vs 5-minute delay)
- Works when server is down
- Lower resource usage

**Limitations:**
- Likely IPv4-only (Sagemcom router limitation)
- Less control over update process

### Testing
```bash
# Check current IP
curl -s https://ipinfo.io/ip

# Verify DNS resolution
nslookup ak-homelab.duckdns.org

# Check IPv6 (likely not updated by router)
nslookup -type=AAAA ak-homelab.duckdns.org
```

**Testing will occur naturally when ISP changes public IP address.**

## VPN Setup with WireGuard

### What is WireGuard?
WireGuard is a modern, lightweight VPN protocol that creates secure tunnels between devices. It encrypts all network traffic and routes it through a VPN server, making your internet connection private and secure.

**Key benefits:**
- **Privacy**: Hides your IP address and encrypts traffic
- **Security**: Protects against man-in-the-middle attacks on public WiFi
- **Access**: Bypass geo-restrictions and enables remote homelab access
- **Performance**: Much faster than OpenVPN with lower battery drain
- **Simplicity**: Easy to configure compared to other VPN protocols

**When you need VPN:**
- Accessing homelab remotely over internet
- Working from public WiFi frequently
- Need to bypass ISP restrictions
- Running public-facing services

**Costs:** WireGuard itself is free. Self-hosted VPN costs $5-20/month for VPS hosting.

**Use cases:**
- Access homelab services remotely (SSH, web interfaces, file shares)
- Secure connection on public WiFi
- Bypass ISP restrictions or geo-blocks

**Performance:** Much faster and lighter than OpenVPN, better battery life on mobile devices.

### Server Configuration
Install WireGuard: `pacman -S wireguard-tools`

Generate keys:
```bash
wg genkey | tee server_private.key | wg pubkey > server_public.key
```

Server config `/etc/wireguard/wg0.conf`:
```ini
[Interface]
PrivateKey = <SERVER_PRIVATE_KEY>
Address = 10.0.0.1/24
ListenPort = 51820
PostUp = iptables -A FORWARD -i %i -j ACCEPT; iptables -A FORWARD -o %i -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
PostDown = iptables -D FORWARD -i %i -j ACCEPT; iptables -D FORWARD -o %i -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE

[Peer]
PublicKey = <CLIENT_PUBLIC_KEY>
AllowedIPs = 10.0.0.2/32
```

### Client Configuration
Generate client keys:
```bash
wg genkey | tee client_private.key | wg pubkey > client_public.key
```

Client config:
```ini
[Interface]
PrivateKey = <CLIENT_PRIVATE_KEY>
Address = 10.0.0.2/24
DNS = 1.1.1.1

[Peer]
PublicKey = <SERVER_PUBLIC_KEY>
Endpoint = ak-homelab.duckdns.org:51820
AllowedIPs = 0.0.0.0/0
PersistentKeepalive = 25
```

### Enable VPN
```bash
sudo systemctl enable wg-quick@wg0
sudo systemctl start wg-quick@wg0
```

## Firewall Configuration

### UFW (Uncomplicated Firewall)

**What it does:** Controls what network traffic is allowed in/out of your server.

**Key functions:**
- **Default deny**: Blocks all incoming connections by default
- **Port control**: Open only specific ports you need (SSH, HTTP, etc.)
- **Rate limiting**: Prevent brute force attacks
- **Application profiles**: Pre-configured rules for common services

**Why needed:** Without firewall, all services are exposed to network attacks.

### UFW Setup
```bash
# Install and enable UFW
pacman -S ufw
sudo ufw enable

# Default policies
sudo ufw default deny incoming
sudo ufw default allow outgoing

# SSH access
sudo ufw allow 2222/tcp

# WireGuard
sudo ufw allow 51820/udp

# HTTP/HTTPS for services
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
```

### Advanced Rules
```bash
# Rate limiting for SSH
sudo ufw limit 2222/tcp

# Allow from specific networks
sudo ufw allow from 192.168.1.0/24 to any port 22

# Log denied connections
sudo ufw logging on
```

## Network Security Best Practices

### Port Management
- **Change default ports**: SSH (2222), WireGuard (51820)
- **Close unused ports**: Regular port scans with nmap
- **Port forwarding**: Only forward necessary ports

### Access Control
- **VPN-first approach**: Access services through VPN tunnel
- **IP whitelisting**: Restrict access to known IPs when possible
- **Rate limiting**: Prevent brute force attacks

### Monitoring
- **Log analysis**: Monitor `/var/log/auth.log` for SSH attempts
- **Network monitoring**: Use netstat/ss to check listening ports
- **Intrusion detection**: Use fail2ban for automated blocking

## fail2ban - Intrusion Prevention

### What is fail2ban?

**What it does:** Automatically blocks IP addresses that show malicious behavior.

**Key functions:**
- **Log monitoring**: Watches system logs for suspicious activity
- **Pattern detection**: Identifies failed login attempts, scanning, etc.
- **Automatic blocking**: Temporarily bans offending IP addresses
- **Customizable rules**: Configure what triggers a ban and for how long

**Common protections:**
- SSH brute force attempts
- Web server attacks (404 scanning, etc.)
- Email server abuse
- Custom application attacks

**Example:** After 5 failed SSH login attempts in 10 minutes, ban IP for 1 hour.

**Why important:** Reduces server load and prevents automated attacks from succeeding through persistence.

### fail2ban Installation & Setup

```bash
# Install fail2ban
sudo pacman -S fail2ban

# Enable and start service
sudo systemctl enable fail2ban
sudo systemctl start fail2ban

# Create local configuration
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
```

### Basic SSH Protection Configuration

Edit `/etc/fail2ban/jail.local`:
```ini
[sshd]
enabled = true
port = 2222
filter = sshd
logpath = /var/log/auth.log
maxretry = 5
bantime = 3600
findtime = 600
```

**Configuration explained:**
- `maxretry = 5`: Ban after 5 failed attempts
- `bantime = 3600`: Ban for 1 hour (3600 seconds)
- `findtime = 600`: 5 attempts within 10 minutes triggers ban
- `port = 2222`: Monitor custom SSH port

### Restart and Monitor

```bash
# Restart fail2ban to apply changes
sudo systemctl restart fail2ban

# Check status
sudo fail2ban-client status
sudo fail2ban-client status sshd

# View banned IPs
sudo fail2ban-client get sshd banned
```

## Router Configuration

### Port Forwarding
Forward these ports to your homelab server:
- SSH: External port → Internal 2222
- WireGuard: 51820 → 51820
- Web services: 80/443 → 80/443 (if needed)

### Security Settings
- **Disable WPS**: Turn off WiFi Protected Setup
- **Strong WiFi password**: WPA3 with complex passphrase
- **Guest network**: Separate network for guests
- **Firmware updates**: Keep router firmware current

## Network Planning

### Dual Network Interface Issue (Critical)

**Problem:** Server has both ethernet and WiFi interfaces. When switching between connections, IP address changes from ethernet (192.168.0.22) to different WiFi IP, breaking SSH connections and port forwards.

**Limitation:** Most routers don't allow DHCP reservation of same IP for multiple MAC addresses.

**Solutions:**

**Option 1: Static IP Configuration (Recommended)**
Configure both interfaces with same static IP:
```bash
# Check interface names
ip link show

# Configure ethernet interface
sudo systemctl edit --full systemd-networkd
# Create /etc/systemd/network/20-ethernet.network
[Match]
Name=enp*

[Network]
DHCP=no
Address=192.168.0.100/24
Gateway=192.168.0.1
DNS=192.168.0.1

# Create /etc/systemd/network/25-wifi.network
[Match]
Name=wlp*

[Network]
DHCP=no
Address=192.168.0.100/24
Gateway=192.168.0.1
DNS=192.168.0.1
```

**Option 2: Hostname-based Access**
Use local hostname resolution instead of IP:
```bash
# Access via hostname (works for both interfaces)
ssh hoborg@ak-homelab.local
# or configure local DNS/mDNS
```

**Option 3: Bridge Networking** 
Create bridge combining both interfaces for automatic failover:
```bash
# Advanced: Bridge both interfaces
ip link add name br0 type bridge
ip link set enp3s0 master br0
ip link set wlp2s0 master br0
```

**Current Setup:**
- Router: 192.168.0.1
- Ethernet: 192.168.0.100 (static IP achieved)
- WiFi: Static IP needed (same .100)
- External: ak-homelab.duckdns.org ✅
- SSH: Port 2222 ✅

## Network Boot Issues Troubleshooting

### Problem 1: Network not available after reboot until GUI login

**Initial diagnosis**: Thought to be NetworkManager sleep behavior.

**Actual root cause**: System auto-suspend due to closed laptop lid (`HandleLidSwitch=suspend`).

**Symptoms**:
- SSH inaccessible after reboot 
- Network comes up only when laptop lid is opened
- System logs show "Suspending..." followed by "Lid opened"

**Solution Applied**:
1. **systemd-logind configuration**: Disable lid switch handling for headless server operation
2. **Static IP configuration**: Eliminate DHCP negotiation delays  
3. **High connection priority**: Ensure ethernet connects first

**Configuration files created**:

`/etc/systemd/logind.conf.d/01-server-logind.conf`:
```ini
[Login]
# Disable all power management triggers for headless server
HandleLidSwitch=ignore
HandleLidSwitchExternalPower=ignore  
HandleLidSwitchDocked=ignore

# Prevent automatic suspend/hibernate
IdleAction=ignore

# Keep system running even when no users logged in
KillUserProcesses=no
```

**NetworkManager commands**:
```bash
nmcli connection modify "Wired connection 2" connection.autoconnect-priority 10
nmcli connection modify "Wired connection 2" ipv4.method manual ipv4.addresses 192.168.0.100/24 ipv4.gateway 192.168.0.1 ipv4.dns "84.2.44.8 84.2.46.8"
```

**Deployment**:
```bash
sudo cp config/systemd/01-server-logind.conf /etc/systemd/logind.conf.d/
sudo systemctl restart systemd-logind
sudo cp config/networkmanager/01-homelab.conf /etc/NetworkManager/conf.d/
sudo systemctl reload NetworkManager
```

### Diagnostic Commands

**Check system power state**:
```bash
systemctl status systemd-logind  # Check for suspend/lid events
loginctl show-session           # Current power management settings
```

**Check network connectivity**:
```bash
systemctl status NetworkManager
nmcli device status
nmcli connection show "Wired connection 2"
ip addr show enp4s0
```

**Monitor boot process**:
```bash
journalctl -b -u NetworkManager    # Network startup logs
journalctl -b -u sshd             # SSH service logs  
journalctl -b -u systemd-logind   # Power management events
```

**Result**: ✅ **RESOLVED** - SSH accessible immediately on boot, lid closure no longer suspends system.

**Network Interface Identification:**
- **enp3s0f0**: First ethernet port (98:fa:9b:f1:06:d5)
- **enp4s0**: Second ethernet port (98:fa:9b:f1:06:d4) ← **Use this one**
- **wlp1s0**: WiFi interface (0c:dd:24:e6:0f:87)

**Issue Solved:** Dual ethernet ports caused MAC address confusion when cable was moved between ports. Stick to enp4s0 consistently.

### IP Address Scheme
- **Router**: 192.168.0.1
- **Homelab server**: 192.168.0.100 (target static IP)
- **Current ethernet**: 192.168.0.22 (can migrate to .100)
- **DHCP range**: 192.168.0.10-99 (excluding static IPs)
- **VPN subnet**: 10.0.0.0/24

### DNS Configuration
- **Primary DNS**: Router (192.168.1.1)
- **Secondary DNS**: 1.1.1.1, 8.8.8.8
- **Local domain**: homelab.local
- **Dynamic DNS**: ak-homelab.duckdns.org

### Service Architecture
```
Internet → Router → Homelab Server
                 ↓
    ┌─────────────────────────────┐
    │  SSH (2222)                 │
    │  WireGuard VPN (51820)      │
    │  Web Services (80/443)      │
    │  Monitoring & Logging       │
    └─────────────────────────────┘
```

## Backup and Recovery

### Configuration Backups
- **SSH keys**: Store securely, separate from server
- **WireGuard configs**: Document peer configurations
- **Firewall rules**: Export UFW rules with `ufw status numbered`

### Network Documentation
- **IP mappings**: Document static assignments
- **Port forwards**: List all forwarded ports and purposes
- **Access credentials**: Secure storage of passwords/keys