maarten
/
incus-contrib
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Restructure Hetzner guide: manual install only, hand off to proxmox-setup 

The guide now covers only the manual steps (server selection, QEMU-based
Proxmox install, SSH alias setup) and then hands off to proxmox-setup
for everything else (repos, networking, storage, WireGuard, firewall,
API tokens).

Removed sections 3-11 that duplicated what the script does interactively.
Added section 3 with SSH setup, script invocation, WireGuard verification,
and test deploy. Updated README.md to match the new two-phase workflow.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Maarten 2026-03-02 16:09:58 +01:00
parent 14d27fccc8
commit bb0f516dec
2 changed files with 151 additions and 546 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

46
hetzner/README.md
View File

@ -12,37 +12,49 @@ via a WireGuard tunnel instead of LAN.
- 32+ cores, 128+ GiB RAM - 32+ cores, 128+ GiB RAM
- 2+ disks (ZFS mirror for system, extra disks for VM storage) - 2+ disks (ZFS mirror for system, extra disks for VM storage)
- An SSH key pair on your workstation - An SSH key pair on your workstation
- A VNC client (for the Proxmox installer)
- This repository cloned locally - This repository cloned locally
## Quickstart ## Workflow
### 1. Install Proxmox on the server The setup is split into a manual guide and an automated script:
Order the server, boot the rescue system, and install Proxmox via ### 1. Manual: rent server and install Proxmox
`installimage` or ISO. See [hetzner-setup.md](hetzner-setup.md) section 2
for details.
### 2. Run the setup helper Follow [hetzner-setup.md](hetzner-setup.md) sections 1-2:
- Pick and order a server from the Hetzner auction
- Install Proxmox via QEMU in the rescue system (ZFS RAID1 mirror)
- Fix the network interface name, reboot, verify SSH works
### 2. Manual: set up SSH alias
Follow [hetzner-setup.md](hetzner-setup.md) section 3.1-3.3:
```bash
ssh-copy-id root@<public-ip>
# Add "Host hetzner-lab" entry to ~/.ssh/config
ssh hetzner-lab pvesh get /version # Must succeed
```
### 3. Automated: configure everything else
```bash ```bash
# Add SSH config entry for the server (see hetzner-setup.md section 5)
# Then run the interactive setup:
hetzner/proxmox-setup --host hetzner-lab --dry-run # Preview hetzner/proxmox-setup --host hetzner-lab --dry-run # Preview
hetzner/proxmox-setup --host hetzner-lab # Execute hetzner/proxmox-setup --host hetzner-lab # Execute
``` ```
The script configures repositories, private networking, ZFS storage, The script interactively configures: repos and updates, private bridge
WireGuard tunnel, firewall, and API tokens -- everything needed to (vmbr1), ZFS storage on extra disks, WireGuard tunnel, firewall lockdown,
deploy VMs. and API tokens for `incusos-proxmox`.
### 3. Deploy IncusOS VMs ### 4. Deploy IncusOS VMs
```bash ```bash
# Copy and edit the connection config
cp incusos/targets/hetzner/proxmox.yaml.example incusos/targets/hetzner/proxmox.yaml cp incusos/targets/hetzner/proxmox.yaml.example incusos/targets/hetzner/proxmox.yaml
# Fill in the API token from step 2 # Fill in the API token from step 3
# Deploy a 3-node cluster export PROXMOX_TOKEN_SECRET="<token>"
incusos-proxmox --proxmox incusos/targets/hetzner/proxmox.yaml \ incusos-proxmox --proxmox incusos/targets/hetzner/proxmox.yaml \
incusos/targets/hetzner/lab-cluster.yaml incusos/targets/hetzner/lab-cluster.yaml
``` ```
@ -51,8 +63,8 @@ incusos-proxmox --proxmox incusos/targets/hetzner/proxmox.yaml \
| File | Description | | File | Description |
|------|-------------| |------|-------------|
| [hetzner-setup.md](hetzner-setup.md) | Comprehensive step-by-step guide | | [hetzner-setup.md](hetzner-setup.md) | Manual guide: server selection, Proxmox install, SSH setup |
| [proxmox-setup](proxmox-setup) | Interactive helper script (automates the guide) | | [proxmox-setup](proxmox-setup) | Interactive script: repos, networking, storage, WireGuard, firewall, API tokens |
## Network architecture ## Network architecture

651
hetzner/hetzner-setup.md
View File

@ -1,8 +1,13 @@
# Hetzner Dedicated Server: Proxmox Setup Guide # Hetzner Dedicated Server: Proxmox Setup Guide
Complete guide for turning a Hetzner bare metal server into a Proxmox host This guide covers the manual steps to get a Hetzner dedicated server running
ready for IncusOS lab deployments. Each section can be done manually or Proxmox VE with SSH access from your workstation. Once SSH works, the
automated with `proxmox-setup` (see [proxmox-setup](proxmox-setup)). interactive [`proxmox-setup`](proxmox-setup) script handles everything else
(repos, networking, storage, WireGuard, firewall, API tokens).
**Workflow:**
1. **This guide** -- rent a server, install Proxmox via QEMU, get SSH working
2. **`proxmox-setup --host <alias>`** -- configure everything else interactively
--- ---
@ -61,8 +66,9 @@ strong favourite -- fast and reliable. Note the device names (`/dev/nvme0n1`,
`/dev/nvme1n1`) for the QEMU command. `/dev/nvme1n1`) for the QEMU command.
If you have additional disks beyond the pair (e.g. large SATA drives), If you have additional disks beyond the pair (e.g. large SATA drives),
those can be set up as a separate storage pool later (section 6). Only those can be set up as a separate storage pool later. Only pass the system
pass the system disks to QEMU for now. disks to QEMU for now -- `proxmox-setup` will detect and configure the
rest.
### 2.3 Check BIOS mode ### 2.3 Check BIOS mode
@ -223,9 +229,6 @@ ssh root@<public-ip> pvesh get /version
# Should show Proxmox VE version and API info # Should show Proxmox VE version and API info
``` ```
The web UI is available at `https://<public-ip>:8006` (we'll lock this
down to WireGuard-only in section 9).
> **Alternative method**: You can also install Debian 13 via Hetzner's > **Alternative method**: You can also install Debian 13 via Hetzner's
> `installimage` and then upgrade to Proxmox following the > `installimage` and then upgrade to Proxmox following the
> [official guide](https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_13_Trixie). > [official guide](https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_13_Trixie).
@ -234,63 +237,123 @@ down to WireGuard-only in section 9).
--- ---
## 3. Network configuration ## 3. SSH setup and handoff to proxmox-setup
Hetzner dedicateds get a single public IP with MAC filtering -- you cannot At this point Proxmox is installed and reachable via its public IP.
assign additional public IPs to VMs without ordering extra IPs. Instead, The remaining configuration (repos, private networking, storage,
we create a private bridge and NAT. WireGuard, firewall, API tokens) is handled by the `proxmox-setup`
script. You just need SSH key access and an alias.
### Create the private bridge ### 3.1 Copy your SSH key
Add `vmbr1` to `/etc/network/interfaces`:
```
auto vmbr1
iface vmbr1 inet static
address 10.10.0.1/24
bridge-ports none
bridge-stp off
bridge-fd 0
post-up echo 1 > /proc/sys/net/ipv4/ip_forward
post-up iptables -t nat -A POSTROUTING -s 10.10.0.0/24 -o vmbr0 -j MASQUERADE
post-down iptables -t nat -D POSTROUTING -s 10.10.0.0/24 -o vmbr0 -j MASQUERADE
```
This creates a private bridge where:
- VMs connect to `vmbr1` and get IPs in 10.10.0.0/24
- The Proxmox host (10.10.0.1) is the default gateway
- NAT masquerading gives VMs internet access through the host's public IP
- IP forwarding is enabled automatically on bridge up
### Persist IP forwarding
Also add to `/etc/sysctl.d/99-forward.conf` for boot persistence:
```
net.ipv4.ip_forward = 1
```
### Apply
```bash ```bash
ifreload -a # Apply network changes without reboot ssh-copy-id root@<public-ip>
sysctl -p /etc/sysctl.d/99-forward.conf
``` ```
### Verify ### 3.2 Create an SSH config alias
Add to `~/.ssh/config` on your workstation:
```
Host hetzner-lab
HostName <public-ip>
User root
IdentityFile ~/.ssh/id_ed25519
```
### 3.3 Verify SSH alias works
```bash ```bash
ip addr show vmbr1 # Should show 10.10.0.1/24 ssh hetzner-lab pvesh get /version
cat /proc/sys/net/ipv4/ip_forward # Should show 1
iptables -t nat -L POSTROUTING -n # Should show MASQUERADE rule
``` ```
### Network diagram This must succeed before continuing. If it fails, check your SSH key
and config entry.
### 3.4 Run proxmox-setup
```bash
# Preview what will be configured
hetzner/proxmox-setup --host hetzner-lab --dry-run
# Run for real (interactive, prompts before each step)
hetzner/proxmox-setup --host hetzner-lab
```
The script walks through each step interactively:
1. **Detect** -- reads host info, block devices, existing config
2. **Repos** -- switches to no-subscription repos, removes nag popup
3. **Update** -- `apt update && apt dist-upgrade`
4. **Bridge** -- creates vmbr1 (private bridge, 10.10.0.0/24) with NAT
5. **Storage** -- detects unused disks, creates ZFS pool, registers with PVE
6. **WireGuard** -- installs and configures tunnel, prints client config
7. **Firewall** -- locks down public interface (SSH + WireGuard only)
8. **API token** -- creates role, pool, and token for `incusos-proxmox`
Each step prompts for confirmation. Use `--yes` to skip prompts, or
`--skip-repos`, `--skip-storage`, etc. to skip individual steps.
At the end, the script prints:
- A ready-to-use `proxmox.yaml` config for `incusos/targets/hetzner/`
- An `env` file snippet with the API token
- A WireGuard client config to save on your workstation
### 3.5 Connect WireGuard and verify
Save the WireGuard client config printed by the script:
```bash
# Linux
sudo wg-quick up hetzner-lab
# macOS -- import into the WireGuard app
```
Verify access through the tunnel:
```bash
ping 10.10.0.1 # Proxmox host via WireGuard
curl -sk https://10.10.0.1:8006 # Web UI via WireGuard
```
Optionally add a second SSH alias for access via WireGuard:
```
Host hetzner-lab-wg
HostName 10.10.0.1
User root
IdentityFile ~/.ssh/id_ed25519
```
### 3.6 Test deploy
```bash
# Copy and fill in the connection config
cp incusos/targets/hetzner/proxmox.yaml.example incusos/targets/hetzner/proxmox.yaml
# Set the API token
export PROXMOX_TOKEN_SECRET="<token from proxmox-setup output>"
# Dry run -- verify correct bridge, IPs, storage
incusos-proxmox --dry-run \
--proxmox incusos/targets/hetzner/proxmox.yaml \
incusos/targets/hetzner/lab-cluster.yaml
```
Confirm the output shows: `vmbr1` bridge, `10.10.0.x` IPs, `local-zfs`
storage, 8 cores per VM.
---
## Network architecture (reference)
Once `proxmox-setup` has completed, the network looks like this:
``` ```
Internet Internet
| |
| Public IP (5.9.x.x) | Public IP (e.g. 5.9.x.x)
v v
[vmbr0] ──── Proxmox host ──── [wg0: 10.10.99.1/24] [vmbr0] ──── Proxmox host ──── [wg0: 10.10.99.1/24]
| | | |
@ -304,481 +367,11 @@ Internet
.101 .102 .103 .101 .102 .103
``` ```
--- | Interface | Purpose |
|-----------|---------|
## 4. DNS setup (optional) | vmbr0 | Public (SSH + WireGuard only after firewall) |
| vmbr1 | Private bridge for VMs (10.10.0.0/24, NAT to internet) |
If you have a domain, create an A record pointing to the public IP: | wg0 | WireGuard tunnel (10.10.99.0/24) |
``` AllowedIPs uses `10.10.0.0/16` to leave room for future subnets
pve.example.com → 5.9.x.x (OVN overlay at 10.10.10.0/24, etc.) without updating WireGuard config.
```
This is convenient but not required -- everything is accessed via
WireGuard using private IPs anyway. If you set a hostname, update
`/etc/hosts`:
```
10.10.0.1 pve.example.com pve
```
And set the hostname:
```bash
hostnamectl set-hostname pve
```
---
## 5. SSH hardening
### Copy your SSH key
```bash
ssh-copy-id root@<public-ip>
```
### Disable password authentication
Edit `/etc/ssh/sshd_config`:
```
PermitRootLogin prohibit-password
PasswordAuthentication no
```
Restart:
```bash
systemctl restart sshd
```
### Add a colleague's key
```bash
# From their workstation:
ssh-copy-id root@<public-ip>
# Or manually append to /root/.ssh/authorized_keys
```
### SSH config for convenient access
Add to `~/.ssh/config` on your workstation:
```
Host hetzner-lab
HostName 5.9.x.x # Public IP (or pve.example.com)
User root
IdentityFile ~/.ssh/id_ed25519
```
After WireGuard is set up (section 8), add a second entry:
```
Host hetzner-lab-wg
HostName 10.10.0.1
User root
IdentityFile ~/.ssh/id_ed25519
```
### Verify
```bash
ssh hetzner-lab pvesh get /version
```
---
## 6. Disk and storage setup
### Identify disks
```bash
lsblk -d -o NAME,SIZE,MODEL,ROTA,TYPE
```
Typical Hetzner setup: 2x NVMe for system (already in ZFS mirror from
install), 2+ additional disks for VM storage.
### Check existing pools
```bash
zpool status # System mirror
pvesm status # Proxmox storage backends
```
### Create VM storage pool
If you have additional disks (e.g. `sda`, `sdb`) not used by the system:
```bash
# Mirror (2 disks) -- recommended
zpool create local-zfs mirror /dev/sda /dev/sdb
# RAIDZ1 (3+ disks) -- more space, still redundant
zpool create local-zfs raidz1 /dev/sda /dev/sdb /dev/sdc
# Single disk (no redundancy)
zpool create local-zfs /dev/sda
```
Register with Proxmox:
```bash
pvesm add zfspool local-zfs -pool local-zfs
pvesm set local-zfs -content images,rootdir
```
### Verify
```bash
pvesm status
# Should list local-zfs with type zfspool
```
If the system install already created a suitable ZFS pool, skip pool
creation and just register it with Proxmox if not already visible.
---
## 7. Repositories and system update
### Switch to no-subscription repositories
The default enterprise repos require a paid subscription. Switch to
community repos:
```bash
# Disable enterprise repo
sed -i 's/^deb/# deb/' /etc/apt/sources.list.d/pve-enterprise.list
# Add no-subscription repo
echo "deb http://download.proxmox.com/debian/pve bookworm pve-no-subscription" \
> /etc/apt/sources.list.d/pve-no-subscription.list
# Disable Ceph enterprise repo if present
if [[ -f /etc/apt/sources.list.d/ceph.list ]]; then
sed -i 's/^deb/# deb/' /etc/apt/sources.list.d/ceph.list
fi
```
### Remove subscription nag popup
```bash
# Standard JS patch for the web UI subscription dialog
sed -Ezi.bak \
"s/(Ext\.Msg\.show\(\{[^}]+title: gettext\('No valid sub)/void\(\{ \/\/\1/" \
/usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js
systemctl restart pveproxy
```
### Update
```bash
apt update && apt dist-upgrade -y
```
### Verify
```bash
apt update 2>&1 | tail -5
# Should show no errors, all repos reachable
```
---
## 8. WireGuard tunnel
WireGuard provides secure access to VMs and the Proxmox web UI from
your workstation, without exposing anything on the public interface.
### Install
```bash
apt install -y wireguard
```
### Generate server keys
```bash
wg genkey | tee /etc/wireguard/server-private.key | wg pubkey > /etc/wireguard/server-public.key
chmod 600 /etc/wireguard/server-private.key
```
### Generate client keys (on your workstation)
```bash
wg genkey | tee wg-client-private.key | wg pubkey > wg-client-public.key
```
### Server config
Create `/etc/wireguard/wg0.conf`:
```ini
[Interface]
PrivateKey = <contents of server-private.key>
Address = 10.10.99.1/24
ListenPort = 51820
# Allow forwarding between WireGuard clients and the private VM bridge
PostUp = iptables -A FORWARD -i wg0 -o vmbr1 -j ACCEPT; iptables -A FORWARD -i vmbr1 -o wg0 -j ACCEPT
PostDown = iptables -D FORWARD -i wg0 -o vmbr1 -j ACCEPT; iptables -D FORWARD -i vmbr1 -o wg0 -j ACCEPT
[Peer]
# Workstation
PublicKey = <contents of wg-client-public.key>
AllowedIPs = 10.10.99.2/32
```
### Client config (on your workstation)
Save as `/etc/wireguard/hetzner-lab.conf` (or import into your WireGuard app):
```ini
[Interface]
PrivateKey = <contents of wg-client-private.key>
Address = 10.10.99.2/24
[Peer]
PublicKey = <contents of server-public.key>
Endpoint = <public-ip>:51820
AllowedIPs = 10.10.0.0/16
PersistentKeepalive = 25
```
**AllowedIPs note**: Using `10.10.0.0/16` instead of just the VM subnet
leaves room for future subnets (OVN overlay at 10.10.10.0/24, etc.)
without needing to update the WireGuard config.
### Enable and start
On the server:
```bash
systemctl enable --now wg-quick@wg0
```
On the workstation:
```bash
# Linux
sudo wg-quick up hetzner-lab
# macOS (WireGuard app)
# Import hetzner-lab.conf and activate
```
### Verify
```bash
# On the server
wg show
# From your workstation
ping 10.10.99.1 # WireGuard interface
ping 10.10.0.1 # Private bridge (should work -- routed via WireGuard)
```
### Adding more peers
To add a colleague, generate a new keypair and add a `[Peer]` block to
the server config:
```ini
[Peer]
# Colleague name
PublicKey = <their-public-key>
AllowedIPs = 10.10.99.3/32
```
Then reload:
```bash
systemctl restart wg-quick@wg0
```
Give them a client config with `Address = 10.10.99.3/24` and the
server's public key.
---
## 9. Firewall lockdown
After WireGuard is working, lock down the public interface so only SSH
and WireGuard are accessible from the internet. Proxmox web UI and VM
traffic go through the tunnel only.
### nftables rules
Create `/etc/nftables-hetzner.conf`:
```
#!/usr/sbin/nft -f
flush ruleset
table inet filter {
chain input {
type filter hook input priority 0; policy drop;
# Loopback
iif lo accept
# Established/related connections
ct state established,related accept
# ICMP (ping)
ip protocol icmp accept
ip6 nexthdr icmpv6 accept
# SSH on public interface
iifname "vmbr0" tcp dport 22 accept
# WireGuard on public interface
iifname "vmbr0" udp dport 51820 accept
# Allow everything on private bridge and WireGuard
iifname "vmbr1" accept
iifname "wg0" accept
# Log and drop everything else
log prefix "nft-drop: " limit rate 5/minute counter drop
}
chain forward {
type filter hook forward priority 0; policy drop;
# Established/related
ct state established,related accept
# WireGuard <-> private bridge
iifname "wg0" oifname "vmbr1" accept
iifname "vmbr1" oifname "wg0" accept
# Private bridge -> internet (NAT)
iifname "vmbr1" oifname "vmbr0" accept
}
chain output {
type filter hook output priority 0; policy accept;
}
}
```
### Apply
```bash
nft -f /etc/nftables-hetzner.conf
```
### Persist across reboots
```bash
cp /etc/nftables-hetzner.conf /etc/nftables.conf
systemctl enable nftables
```
### Verify
From an external machine (not through WireGuard):
```bash
# Should work
ssh hetzner-lab echo ok
# Should time out (port 8006 blocked on public interface)
curl -sk --connect-timeout 5 https://<public-ip>:8006
# Through WireGuard -- should work
curl -sk https://10.10.0.1:8006
```
**Warning**: Test SSH access through WireGuard *before* applying firewall
rules. If you lock yourself out, use Hetzner's rescue system to recover.
---
## 10. API token setup
Create a dedicated API token for `incusos-proxmox` automation.
### Create role and user
```bash
# Create role with required privileges
pveum role add IncusOSDeployer -privs \
"VM.Allocate VM.Config.CDROM VM.Config.CPU VM.Config.Disk VM.Config.HWType \
VM.Config.Memory VM.Config.Network VM.Config.Options VM.PowerMgmt \
VM.Monitor VM.Audit VM.Console \
Datastore.AllocateSpace Datastore.Audit \
Pool.Allocate Pool.Audit \
SDN.Use Sys.Modify"
# Create resource pool
pveum pool add IncusLab -comment "IncusOS lab VMs"
# Create API token
pveum user token add automation@pve deploy --privsep 0
# Save the displayed token secret -- it's shown only once!
# Assign role to user on the pool
pveum acl modify /pool/IncusLab -user automation@pve -role IncusOSDeployer
# Assign role on storage
pveum acl modify /storage/local-zfs -user automation@pve -role IncusOSDeployer
pveum acl modify /storage/local -user automation@pve -role IncusOSDeployer
```
### Record credentials
Add to the `env` file at the repo root:
```bash
# Hetzner
HETZNER_PROXMOX_TOKEN_SECRET=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
HETZNER_PROXMOX_ROOT_PASSWORD=your-root-password
```
### Verify
```bash
# Export for incusos-proxmox
export PROXMOX_TOKEN_SECRET="$HETZNER_PROXMOX_TOKEN_SECRET"
# Test API access (update host/node in proxmox-api or use curl directly)
curl -sk "https://10.10.0.1:8006/api2/json/version" \
-H "Authorization: PVEAPIToken=automation@pve!deploy=$PROXMOX_TOKEN_SECRET"
```
---
## 11. Final verification checklist
| Check | Command | Expected |
|-------|---------|----------|
| Proxmox version | `pvesh get /version` | PVE 8.x |
| Private bridge | `ip addr show vmbr1` | 10.10.0.1/24 |
| IP forwarding | `cat /proc/sys/net/ipv4/ip_forward` | 1 |
| NAT | `iptables -t nat -L -n` | MASQUERADE for 10.10.0.0/24 |
| ZFS storage | `pvesm status` | local-zfs available |
| WireGuard | `wg show` | 1+ peer, handshake recent |
| SSH via WG | `ssh hetzner-lab-wg hostname` | Responds |
| Web UI via WG | `curl -sk https://10.10.0.1:8006` | HTML response |
| API token | `curl -sk .../api2/json/version -H Auth...` | JSON with version |
| Public lockdown | `curl --connect-timeout 5 https://<pub-ip>:8006` | Timeout |
### Test deploy
```bash
export PROXMOX_TOKEN_SECRET="$HETZNER_PROXMOX_TOKEN_SECRET"
# Dry run -- verify correct bridge, IPs, storage
incusos-proxmox --dry-run \
--proxmox incusos/targets/hetzner/proxmox.yaml \
incusos/targets/hetzner/lab-cluster.yaml
```
Confirm the output shows: `vmbr1` bridge, `10.10.0.x` IPs, `local-zfs`
storage, 8 cores per VM.