homelab/docs/network-security.md

Network & Security Configuration

Complete guide for securing and networking your homelab.

SSH Security Setup

Initial Configuration

# Generate SSH key pair
ssh-keygen -t ed25519 -C "homelab-key" -f ~/.ssh/homelab_ed25519

# Copy public key to target
ssh-copy-id -i ~/.ssh/homelab_ed25519.pub user@target

SSH Hardening

Status: ✅ Complete - Port changed to 2222

Edit /etc/ssh/sshd_config:

# Disable root login
PermitRootLogin no

# Use key-based authentication only
PasswordAuthentication no
PubkeyAuthentication yes

# Change default port (CRITICAL - currently still on 22)
Port 2222

# Restrict users
AllowUsers hoborg

# Security settings
Protocol 2
X11Forwarding no
AllowTcpForwarding no
ClientAliveInterval 300
ClientAliveCountMax 2
MaxAuthTries 3
MaxStartups 2

Completed:

1. ✅ Changed SSH port from 22 to 2222
2. ✅ Updated router port forwarding rules
3. ✅ External access via ak-homelab.duckdns.org:2222 working

Restart SSH: sudo systemctl restart sshd

Mosh Alternative (Investigation Needed)

Issue: SSH can be unreliable on WiFi connections with packet loss.

Mosh Benefits:

• Maintains connection during network switches (ethernet ↔ WiFi)
• Handles poor WiFi connections better
• Local echo for responsive typing
• Roaming support (IP changes don't break connection)

Installation:

# Server side
sudo pacman -S mosh

# Client side
mosh user@server

Requirements:

• UDP ports 60000-61000 open on router
• SSH still needed for initial authentication

Status: ✅ Local use working ❌ External blocked by ISP

Key Findings:

• Local mosh: Works perfectly (mosh localhost, mosh 192.168.0.100)
• External mosh: Blocked by ISP UDP port filtering on ports 60000-61000
• SSH still needed: Mosh uses SSH for initial authentication, then switches to UDP

ISP UDP Blocking Issue:

• Most ISPs block UDP ports 60000-61000 for "security"
• SSH works fine (TCP port 2222) but mosh fails (UDP 60000-61000)
• Router port forwarding is correct, but ISP drops UDP packets

Current Recommendation:

• Use mosh for local/internal network connections
• Stick with SSH for external connections until VPN is set up
• VPN tunnel can bypass ISP UDP blocking

SSH Client Configuration

Create ~/.ssh/config:

Host homelab
    HostName ak-homelab.duckdns.org
    User hoborg
    Port 2222
    IdentityFile ~/.ssh/homelab_ed25519
    ServerAliveInterval 60

Usage:

# Connect via SSH
ssh homelab

# Connect via Mosh (uses SSH config automatically)
mosh homelab

Dynamic DNS with DuckDNS

Account Setup

1. Create account at duckdns.org
2. Create subdomain: ak-homelab.duckdns.org
3. Get token from dashboard

Automatic IP Updates

Update script at ~/.local/scripts/duckdns.py (Python implementation)

Cron job for automatic updates:

# Update every 5 minutes
*/5 * * * * /home/hoborg/.local/scripts/duckdns.py >/dev/null 2>&1

Current Setup (Router-based)

Status: ✅ Migrated from script to router DynDNS

Changes made:

• ✅ Disabled cron job script (*/5 * * * * entry removed)
• ✅ Enabled router Dynamic DNS for ak-homelab.duckdns.org
• ⏳ Testing pending - Cannot force public IP change to verify

Router DynDNS Benefits:

• Immediate updates on IP change (vs 5-minute delay)
• Works when server is down
• Lower resource usage

Limitations:

• Likely IPv4-only (Sagemcom router limitation)
• Less control over update process

Testing

# Check current IP
curl -s https://ipinfo.io/ip

# Verify DNS resolution
nslookup ak-homelab.duckdns.org

# Check IPv6 (likely not updated by router)
nslookup -type=AAAA ak-homelab.duckdns.org

Testing will occur naturally when ISP changes public IP address.

VPN Setup with WireGuard

What is WireGuard?

WireGuard is a modern, lightweight VPN protocol that creates secure tunnels between devices. It encrypts all network traffic and routes it through a VPN server, making your internet connection private and secure.

Key benefits:

• Privacy: Hides your IP address and encrypts traffic
• Security: Protects against man-in-the-middle attacks on public WiFi
• Access: Bypass geo-restrictions and enables remote homelab access
• Performance: Much faster than OpenVPN with lower battery drain
• Simplicity: Easy to configure compared to other VPN protocols

When you need VPN:

• Accessing homelab remotely over internet
• Working from public WiFi frequently
• Need to bypass ISP restrictions
• Running public-facing services

Costs: WireGuard itself is free. Self-hosted VPN costs $5-20/month for VPS hosting.

Use cases:

• Access homelab services remotely (SSH, web interfaces, file shares)
• Secure connection on public WiFi
• Bypass ISP restrictions or geo-blocks

Performance: Much faster and lighter than OpenVPN, better battery life on mobile devices.

Server Configuration

Install WireGuard: pacman -S wireguard-tools

Generate keys:

wg genkey | tee server_private.key | wg pubkey > server_public.key

Server config /etc/wireguard/wg0.conf:

[Interface]
PrivateKey = <SERVER_PRIVATE_KEY>
Address = 10.0.0.1/24
ListenPort = 51820
PostUp = iptables -A FORWARD -i %i -j ACCEPT; iptables -A FORWARD -o %i -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
PostDown = iptables -D FORWARD -i %i -j ACCEPT; iptables -D FORWARD -o %i -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE

[Peer]
PublicKey = <CLIENT_PUBLIC_KEY>
AllowedIPs = 10.0.0.2/32

Client Configuration

Generate client keys:

wg genkey | tee client_private.key | wg pubkey > client_public.key

Client config:

[Interface]
PrivateKey = <CLIENT_PRIVATE_KEY>
Address = 10.0.0.2/24
DNS = 1.1.1.1

[Peer]
PublicKey = <SERVER_PUBLIC_KEY>
Endpoint = ak-homelab.duckdns.org:51820
AllowedIPs = 0.0.0.0/0
PersistentKeepalive = 25

Enable VPN

sudo systemctl enable wg-quick@wg0
sudo systemctl start wg-quick@wg0

Firewall Configuration

UFW (Uncomplicated Firewall)

What it does: Controls what network traffic is allowed in/out of your server.

Key functions:

• Default deny: Blocks all incoming connections by default
• Port control: Open only specific ports you need (SSH, HTTP, etc.)
• Rate limiting: Prevent brute force attacks
• Application profiles: Pre-configured rules for common services

Why needed: Without firewall, all services are exposed to network attacks.

UFW Setup

# Install and enable UFW
pacman -S ufw
sudo ufw enable

# Default policies
sudo ufw default deny incoming
sudo ufw default allow outgoing

# SSH access
sudo ufw allow 2222/tcp

# WireGuard
sudo ufw allow 51820/udp

# HTTP/HTTPS for services
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp

Advanced Rules

# Rate limiting for SSH
sudo ufw limit 2222/tcp

# Allow from specific networks
sudo ufw allow from 192.168.1.0/24 to any port 22

# Log denied connections
sudo ufw logging on

Network Security Best Practices

Port Management

• Change default ports: SSH (2222), WireGuard (51820)
• Close unused ports: Regular port scans with nmap
• Port forwarding: Only forward necessary ports

Access Control

• VPN-first approach: Access services through VPN tunnel
• IP whitelisting: Restrict access to known IPs when possible
• Rate limiting: Prevent brute force attacks

Monitoring

• Log analysis: Monitor /var/log/auth.log for SSH attempts
• Network monitoring: Use netstat/ss to check listening ports
• Intrusion detection: Use fail2ban for automated blocking

fail2ban - Intrusion Prevention

What is fail2ban?

What it does: Automatically blocks IP addresses that show malicious behavior.

Key functions:

• Log monitoring: Watches system logs for suspicious activity
• Pattern detection: Identifies failed login attempts, scanning, etc.
• Automatic blocking: Temporarily bans offending IP addresses
• Customizable rules: Configure what triggers a ban and for how long

Common protections:

• SSH brute force attempts
• Web server attacks (404 scanning, etc.)
• Email server abuse
• Custom application attacks

Example: After 5 failed SSH login attempts in 10 minutes, ban IP for 1 hour.

Why important: Reduces server load and prevents automated attacks from succeeding through persistence.

fail2ban Installation & Setup

# Install fail2ban
sudo pacman -S fail2ban

# Enable and start service
sudo systemctl enable fail2ban
sudo systemctl start fail2ban

# Create local configuration
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

Basic SSH Protection Configuration

Edit /etc/fail2ban/jail.local:

[sshd]
enabled = true
port = 2222
filter = sshd
logpath = /var/log/auth.log
maxretry = 5
bantime = 3600
findtime = 600

Configuration explained:

• maxretry = 5: Ban after 5 failed attempts
• bantime = 3600: Ban for 1 hour (3600 seconds)
• findtime = 600: 5 attempts within 10 minutes triggers ban
• port = 2222: Monitor custom SSH port

Restart and Monitor

# Restart fail2ban to apply changes
sudo systemctl restart fail2ban

# Check status
sudo fail2ban-client status
sudo fail2ban-client status sshd

# View banned IPs
sudo fail2ban-client get sshd banned

Router Configuration

Port Forwarding

Forward these ports to your homelab server:

• SSH: External port → Internal 2222
• WireGuard: 51820 → 51820
• Web services: 80/443 → 80/443 (if needed)

Security Settings

• Disable WPS: Turn off WiFi Protected Setup
• Strong WiFi password: WPA3 with complex passphrase
• Guest network: Separate network for guests
• Firmware updates: Keep router firmware current

Network Planning

Dual Network Interface Issue (Critical)

Problem: Server has both ethernet and WiFi interfaces. When switching between connections, IP address changes from ethernet (192.168.0.22) to different WiFi IP, breaking SSH connections and port forwards.

Limitation: Most routers don't allow DHCP reservation of same IP for multiple MAC addresses.

Solutions:

Option 1: Static IP Configuration (Recommended) Configure both interfaces with same static IP:

# Check interface names
ip link show

# Configure ethernet interface
sudo systemctl edit --full systemd-networkd
# Create /etc/systemd/network/20-ethernet.network
[Match]
Name=enp*

[Network]
DHCP=no
Address=192.168.0.100/24
Gateway=192.168.0.1
DNS=192.168.0.1

# Create /etc/systemd/network/25-wifi.network
[Match]
Name=wlp*

[Network]
DHCP=no
Address=192.168.0.100/24
Gateway=192.168.0.1
DNS=192.168.0.1

Option 2: Hostname-based Access Use local hostname resolution instead of IP:

# Access via hostname (works for both interfaces)
ssh hoborg@ak-homelab.local
# or configure local DNS/mDNS

Option 3: Bridge Networking Create bridge combining both interfaces for automatic failover:

# Advanced: Bridge both interfaces
ip link add name br0 type bridge
ip link set enp3s0 master br0
ip link set wlp2s0 master br0

Current Setup:

• Router: 192.168.0.1
• Ethernet: 192.168.0.100 (static IP achieved)
• WiFi: Static IP needed (same .100)
• External: ak-homelab.duckdns.org ✅
• SSH: Port 2222 ✅

Network Boot Issues Troubleshooting

Problem 1: Network not available after reboot until GUI login

Initial diagnosis: Thought to be NetworkManager sleep behavior.

Actual root cause: System auto-suspend due to closed laptop lid (HandleLidSwitch=suspend).

Symptoms:

• SSH inaccessible after reboot
• Network comes up only when laptop lid is opened
• System logs show "Suspending..." followed by "Lid opened"

Solution Applied:

1. systemd-logind configuration: Disable lid switch handling for headless server operation
2. Static IP configuration: Eliminate DHCP negotiation delays
3. High connection priority: Ensure ethernet connects first

Configuration files created:

/etc/systemd/logind.conf.d/01-server-logind.conf:

[Login]
# Disable all power management triggers for headless server
HandleLidSwitch=ignore
HandleLidSwitchExternalPower=ignore  
HandleLidSwitchDocked=ignore

# Prevent automatic suspend/hibernate
IdleAction=ignore

# Keep system running even when no users logged in
KillUserProcesses=no

NetworkManager commands:

nmcli connection modify "Wired connection 2" connection.autoconnect-priority 10
nmcli connection modify "Wired connection 2" ipv4.method manual ipv4.addresses 192.168.0.100/24 ipv4.gateway 192.168.0.1 ipv4.dns "84.2.44.8 84.2.46.8"

Deployment:

sudo cp config/systemd/01-server-logind.conf /etc/systemd/logind.conf.d/
sudo systemctl restart systemd-logind
sudo cp config/networkmanager/01-homelab.conf /etc/NetworkManager/conf.d/
sudo systemctl reload NetworkManager

Diagnostic Commands

Check system power state:

systemctl status systemd-logind  # Check for suspend/lid events
loginctl show-session           # Current power management settings

Check network connectivity:

systemctl status NetworkManager
nmcli device status
nmcli connection show "Wired connection 2"
ip addr show enp4s0

Monitor boot process:

journalctl -b -u NetworkManager    # Network startup logs
journalctl -b -u sshd             # SSH service logs  
journalctl -b -u systemd-logind   # Power management events

Result: ✅ RESOLVED - SSH accessible immediately on boot, lid closure no longer suspends system.

Network Interface Identification:

• enp3s0f0: First ethernet port (98:fa:9b:f1:06:d5)
• enp4s0: Second ethernet port (98:fa:9b:f1:06:d4) ← Use this one
• wlp1s0: WiFi interface (0c:dd:24:e6:0f:87)

Issue Solved: Dual ethernet ports caused MAC address confusion when cable was moved between ports. Stick to enp4s0 consistently.

IP Address Scheme

• Router: 192.168.0.1
• Homelab server: 192.168.0.100 (target static IP)
• Current ethernet: 192.168.0.22 (can migrate to .100)
• DHCP range: 192.168.0.10-99 (excluding static IPs)
• VPN subnet: 10.0.0.0/24

DNS Configuration

• Primary DNS: Router (192.168.1.1)
• Secondary DNS: 1.1.1.1, 8.8.8.8
• Local domain: homelab.local
• Dynamic DNS: ak-homelab.duckdns.org

Service Architecture

Internet → Router → Homelab Server
                 ↓
    ┌─────────────────────────────┐
    │  SSH (2222)                 │
    │  WireGuard VPN (51820)      │
    │  Web Services (80/443)      │
    │  Monitoring & Logging       │
    └─────────────────────────────┘

Backup and Recovery

Configuration Backups

• SSH keys: Store securely, separate from server
• WireGuard configs: Document peer configurations
• Firewall rules: Export UFW rules with ufw status numbered

Network Documentation

• IP mappings: Document static assignments
• Port forwards: List all forwarded ports and purposes
• Access credentials: Secure storage of passwords/keys