19 KiB

Raw Blame History

Hetzner Dedicated Server: Proxmox Setup Guide

Complete guide for turning a Hetzner bare metal server into a Proxmox host ready for IncusOS lab deployments. Each section can be done manually or automated with proxmox-setup (see proxmox-setup).

1. Server selection

The Hetzner Server Auction offers used dedicated servers at significant discounts. Look for:

CPU: Intel with VT-x and VT-d (AMD works too, but Intel nested virtualization is more proven with Proxmox/IncusOS)
Cores: 32+ (each IncusOS VM wants 4-8 cores)
RAM: 128+ GiB (256 GiB ideal -- 4 VMs at 16 GiB still leaves 192 GiB)
Disks: 2+ NVMe or SSD (ZFS mirror for system, extras for VM storage)
Network: 1 Gbit/s included, single IPv4

Order the server and wait for provisioning (usually minutes for auction servers). Note the assigned IP address and temporary root password.

2. Proxmox installation

We use the official Proxmox VE ISO installer running inside QEMU from the Hetzner rescue system. This gives the full graphical installer with ZFS support, which is not available via Hetzner's installimage. The method is documented in the Hetzner community tutorial.

2.1 Boot the rescue system

In the Hetzner Robot panel, activate the rescue system (Linux 64-bit) and reboot the server. Then SSH in:

ssh root@<public-ip>

2.2 Discover disks

Identify which disks the server has:

lsblk -d -o NAME,SIZE,MODEL,ROTA,TYPE

Example output on a typical auction server with 2 NVMe drives:

NAME    SIZE MODEL                ROTA TYPE
nvme0n1 477G Samsung SSD 970 EVO     0 disk
nvme1n1 477G Samsung SSD 970 EVO     0 disk

For ZFS RAID1 (mirror), you want two matching disks. NVMe pairs are the strong favourite -- fast and reliable. Note the device names (/dev/nvme0n1, /dev/nvme1n1) for the QEMU command.

If you have additional disks beyond the pair (e.g. large SATA drives), those can be set up as a separate storage pool later (section 6). Only pass the system disks to QEMU for now.

2.3 Check BIOS mode

Determine whether the server boots in UEFI or legacy BIOS mode:

[ -d "/sys/firmware/efi" ] && echo "UEFI" || echo "BIOS"

Most modern Hetzner servers are UEFI. The QEMU command below includes the OVMF BIOS line for UEFI -- remove it if your server reports BIOS.

2.4 Download the Proxmox ISO

Get the latest Proxmox VE ISO from the official download page:

wget https://enterprise.proxmox.com/iso/proxmox-ve_9.0-1.iso

Check proxmox.com/en/downloads for the current version and adjust the URL accordingly.

2.5 Set up SSH port forwarding for VNC

On your workstation (not the server), open an SSH tunnel forwarding the VNC port:

ssh -L 5900:localhost:5900 root@<public-ip>

This forwards local port 5900 to the server's localhost:5900, where QEMU will expose its VNC display. Keep this session open.

2.6 Start the QEMU installer

In the SSH session on the server, start QEMU with the ISO and the disks you identified in step 2.2:

qemu-system-x86_64 \
    -enable-kvm \
    -cpu host \
    -m 16G \
    -boot d \
    -cdrom ./proxmox-ve_9.0-1.iso \
    -drive file=/dev/nvme0n1,format=raw,if=virtio \
    -drive file=/dev/nvme1n1,format=raw,if=virtio \
    -bios /usr/share/OVMF/OVMF_CODE.fd \
    -vnc 127.0.0.1:0

Notes:

Remove the -bios /usr/share/OVMF/OVMF_CODE.fd \ line for legacy BIOS servers (step 2.3).
For SATA drives, use /dev/sda, /dev/sdb instead of /dev/nvmeXn1.
Disks appear as /dev/vdX inside the VM because of the virtio interface -- this is normal and expected.
The -vnc 127.0.0.1:0 flag binds VNC to localhost only (safe, no password needed since it's behind the SSH tunnel).

2.7 Connect via VNC and install

Open a VNC client on your workstation and connect to 127.0.0.1:5900 (or just 127.0.0.1 -- most clients default to port 5900).

The Proxmox graphical installer appears. Walk through it:

Accept the EULA.
Target disk: Select the ZFS RAID1 (mirror) option across both drives (/dev/vda and /dev/vdb in the VM -- these are your NVMe drives passed through via virtio).
Country/timezone/keyboard: Set as appropriate.
Root password and email: Set a strong root password. This becomes HETZNER_PROXMOX_ROOT_PASSWORD in your env file.
Network: The installer shows a virtualized NIC. Configure it with the server's public IP, gateway, and hostname. This will be corrected in the next step since the real NIC name differs.
Click Install and wait for completion.

The VNC client may disconnect briefly during install -- just reconnect to 127.0.0.1:5900.

2.8 Predict the real network interface name

After installation completes, stop QEMU with Ctrl+C in the SSH terminal. Do not reboot yet -- the network interface name configured by the installer is wrong (it matches the virtual NIC, not the real one).

Use the Hetzner predict-check tool to discover the real interface name:

predict-check

Example output:

eth0 -> enp0s31f6

Note the predicted name (e.g. enp0s31f6). You can also check the current rescue interface for reference:

netdata

2.9 Fix the network config before first real boot

Boot Proxmox again in QEMU, without the ISO (no -cdrom flag):

qemu-system-x86_64 \
    -enable-kvm \
    -cpu host \
    -m 16G \
    -boot d \
    -drive file=/dev/nvme0n1,format=raw,if=virtio \
    -drive file=/dev/nvme1n1,format=raw,if=virtio \
    -bios /usr/share/OVMF/OVMF_CODE.fd \
    -vnc 127.0.0.1:0

Connect via VNC, log in as root, and edit the network configuration:

nano /etc/network/interfaces

Replace the virtual interface name (e.g. ens18) with the predicted real name (e.g. enp0s31f6). A minimal working config:

auto lo
iface lo inet loopback

auto enp0s31f6
iface enp0s31f6 inet static
    address <public-ip>/32
    gateway <gateway-ip>

Save and shut down the VM (shutdown -h now inside the VNC session, or Ctrl+C in the SSH terminal).

2.10 Reboot into Proxmox

Exit the rescue system and reboot the server from the Hetzner Robot panel (or just reboot from SSH). The server now boots from disk into Proxmox with the correct network configuration.

2.11 Verify

ssh root@<public-ip> pvesh get /version
# Should show Proxmox VE version and API info

The web UI is available at https://<public-ip>:8006 (we'll lock this down to WireGuard-only in section 9).

Alternative method: You can also install Debian 13 via Hetzner's installimage and then upgrade to Proxmox following the official guide. This skips the QEMU/VNC process but does not offer ZFS-on-root from the installer.

3. Network configuration

Hetzner dedicateds get a single public IP with MAC filtering -- you cannot assign additional public IPs to VMs without ordering extra IPs. Instead, we create a private bridge and NAT.

Create the private bridge

Add vmbr1 to /etc/network/interfaces:

auto vmbr1
iface vmbr1 inet static
    address 10.10.0.1/24
    bridge-ports none
    bridge-stp off
    bridge-fd 0
    post-up   echo 1 > /proc/sys/net/ipv4/ip_forward
    post-up   iptables -t nat -A POSTROUTING -s 10.10.0.0/24 -o vmbr0 -j MASQUERADE
    post-down iptables -t nat -D POSTROUTING -s 10.10.0.0/24 -o vmbr0 -j MASQUERADE

This creates a private bridge where:

VMs connect to vmbr1 and get IPs in 10.10.0.0/24
The Proxmox host (10.10.0.1) is the default gateway
NAT masquerading gives VMs internet access through the host's public IP
IP forwarding is enabled automatically on bridge up

Persist IP forwarding

Also add to /etc/sysctl.d/99-forward.conf for boot persistence:

net.ipv4.ip_forward = 1

Apply

ifreload -a        # Apply network changes without reboot
sysctl -p /etc/sysctl.d/99-forward.conf

Verify

ip addr show vmbr1          # Should show 10.10.0.1/24
cat /proc/sys/net/ipv4/ip_forward  # Should show 1
iptables -t nat -L POSTROUTING -n  # Should show MASQUERADE rule

Network diagram

Internet
    |
    | Public IP (5.9.x.x)
    v
[vmbr0] ──── Proxmox host ──── [wg0: 10.10.99.1/24]
                  |                       |
                  |                  WireGuard tunnel
                  |                       |
             [vmbr1: 10.10.0.1/24]   Your workstation
                  |                  (10.10.99.2)
            ┌─────┼─────┐
            |     |     |
          VM-01 VM-02 VM-03
          .101  .102  .103

4. DNS setup (optional)

If you have a domain, create an A record pointing to the public IP:

pve.example.com → 5.9.x.x

This is convenient but not required -- everything is accessed via WireGuard using private IPs anyway. If you set a hostname, update /etc/hosts:

10.10.0.1    pve.example.com pve

And set the hostname:

hostnamectl set-hostname pve

5. SSH hardening

Copy your SSH key

ssh-copy-id root@<public-ip>

Disable password authentication

Edit /etc/ssh/sshd_config:

PermitRootLogin prohibit-password
PasswordAuthentication no

Restart:

systemctl restart sshd

Add a colleague's key

# From their workstation:
ssh-copy-id root@<public-ip>

# Or manually append to /root/.ssh/authorized_keys

SSH config for convenient access

Add to ~/.ssh/config on your workstation:

Host hetzner-lab
    HostName 5.9.x.x           # Public IP (or pve.example.com)
    User root
    IdentityFile ~/.ssh/id_ed25519

After WireGuard is set up (section 8), add a second entry:

Host hetzner-lab-wg
    HostName 10.10.0.1
    User root
    IdentityFile ~/.ssh/id_ed25519

Verify

ssh hetzner-lab pvesh get /version

6. Disk and storage setup

Identify disks

lsblk -d -o NAME,SIZE,MODEL,ROTA,TYPE

Typical Hetzner setup: 2x NVMe for system (already in ZFS mirror from install), 2+ additional disks for VM storage.

Check existing pools

zpool status            # System mirror
pvesm status            # Proxmox storage backends

Create VM storage pool

If you have additional disks (e.g. sda, sdb) not used by the system:

# Mirror (2 disks) -- recommended
zpool create local-zfs mirror /dev/sda /dev/sdb

# RAIDZ1 (3+ disks) -- more space, still redundant
zpool create local-zfs raidz1 /dev/sda /dev/sdb /dev/sdc

# Single disk (no redundancy)
zpool create local-zfs /dev/sda

pvesm add zfspool local-zfs -pool local-zfs
pvesm set local-zfs -content images,rootdir

Verify

pvesm status
# Should list local-zfs with type zfspool

If the system install already created a suitable ZFS pool, skip pool creation and just register it with Proxmox if not already visible.

7. Repositories and system update

Switch to no-subscription repositories

The default enterprise repos require a paid subscription. Switch to community repos:

# Disable enterprise repo
sed -i 's/^deb/# deb/' /etc/apt/sources.list.d/pve-enterprise.list

# Add no-subscription repo
echo "deb http://download.proxmox.com/debian/pve bookworm pve-no-subscription" \
    > /etc/apt/sources.list.d/pve-no-subscription.list

# Disable Ceph enterprise repo if present
if [[ -f /etc/apt/sources.list.d/ceph.list ]]; then
    sed -i 's/^deb/# deb/' /etc/apt/sources.list.d/ceph.list
fi

# Standard JS patch for the web UI subscription dialog
sed -Ezi.bak \
    "s/(Ext\.Msg\.show\(\{[^}]+title: gettext\('No valid sub)/void\(\{ \/\/\1/" \
    /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js
systemctl restart pveproxy

Update

apt update && apt dist-upgrade -y

Verify

apt update 2>&1 | tail -5
# Should show no errors, all repos reachable

8. WireGuard tunnel

WireGuard provides secure access to VMs and the Proxmox web UI from your workstation, without exposing anything on the public interface.

Install

apt install -y wireguard

Generate server keys

wg genkey | tee /etc/wireguard/server-private.key | wg pubkey > /etc/wireguard/server-public.key
chmod 600 /etc/wireguard/server-private.key

Generate client keys (on your workstation)

wg genkey | tee wg-client-private.key | wg pubkey > wg-client-public.key

Server config

Create /etc/wireguard/wg0.conf:

[Interface]
PrivateKey = <contents of server-private.key>
Address = 10.10.99.1/24
ListenPort = 51820

# Allow forwarding between WireGuard clients and the private VM bridge
PostUp = iptables -A FORWARD -i wg0 -o vmbr1 -j ACCEPT; iptables -A FORWARD -i vmbr1 -o wg0 -j ACCEPT
PostDown = iptables -D FORWARD -i wg0 -o vmbr1 -j ACCEPT; iptables -D FORWARD -i vmbr1 -o wg0 -j ACCEPT

[Peer]
# Workstation
PublicKey = <contents of wg-client-public.key>
AllowedIPs = 10.10.99.2/32

Client config (on your workstation)

Save as /etc/wireguard/hetzner-lab.conf (or import into your WireGuard app):

[Interface]
PrivateKey = <contents of wg-client-private.key>
Address = 10.10.99.2/24

[Peer]
PublicKey = <contents of server-public.key>
Endpoint = <public-ip>:51820
AllowedIPs = 10.10.0.0/16
PersistentKeepalive = 25

AllowedIPs note: Using 10.10.0.0/16 instead of just the VM subnet leaves room for future subnets (OVN overlay at 10.10.10.0/24, etc.) without needing to update the WireGuard config.

Enable and start

On the server:

systemctl enable --now wg-quick@wg0

On the workstation:

# Linux
sudo wg-quick up hetzner-lab

# macOS (WireGuard app)
# Import hetzner-lab.conf and activate

Verify

# On the server
wg show

# From your workstation
ping 10.10.99.1   # WireGuard interface
ping 10.10.0.1    # Private bridge (should work -- routed via WireGuard)

Adding more peers

To add a colleague, generate a new keypair and add a [Peer] block to the server config:

[Peer]
# Colleague name
PublicKey = <their-public-key>
AllowedIPs = 10.10.99.3/32

Then reload:

systemctl restart wg-quick@wg0

Give them a client config with Address = 10.10.99.3/24 and the server's public key.

9. Firewall lockdown

After WireGuard is working, lock down the public interface so only SSH and WireGuard are accessible from the internet. Proxmox web UI and VM traffic go through the tunnel only.

nftables rules

Create /etc/nftables-hetzner.conf:

#!/usr/sbin/nft -f

flush ruleset

table inet filter {
    chain input {
        type filter hook input priority 0; policy drop;

        # Loopback
        iif lo accept

        # Established/related connections
        ct state established,related accept

        # ICMP (ping)
        ip protocol icmp accept
        ip6 nexthdr icmpv6 accept

        # SSH on public interface
        iifname "vmbr0" tcp dport 22 accept

        # WireGuard on public interface
        iifname "vmbr0" udp dport 51820 accept

        # Allow everything on private bridge and WireGuard
        iifname "vmbr1" accept
        iifname "wg0" accept

        # Log and drop everything else
        log prefix "nft-drop: " limit rate 5/minute counter drop
    }

    chain forward {
        type filter hook forward priority 0; policy drop;

        # Established/related
        ct state established,related accept

        # WireGuard <-> private bridge
        iifname "wg0" oifname "vmbr1" accept
        iifname "vmbr1" oifname "wg0" accept

        # Private bridge -> internet (NAT)
        iifname "vmbr1" oifname "vmbr0" accept
    }

    chain output {
        type filter hook output priority 0; policy accept;
    }
}

Apply

nft -f /etc/nftables-hetzner.conf

Persist across reboots

cp /etc/nftables-hetzner.conf /etc/nftables.conf
systemctl enable nftables

Verify

From an external machine (not through WireGuard):

# Should work
ssh hetzner-lab echo ok

# Should time out (port 8006 blocked on public interface)
curl -sk --connect-timeout 5 https://<public-ip>:8006

# Through WireGuard -- should work
curl -sk https://10.10.0.1:8006

Warning: Test SSH access through WireGuard before applying firewall rules. If you lock yourself out, use Hetzner's rescue system to recover.

10. API token setup

Create a dedicated API token for incusos-proxmox automation.

Create role and user

# Create role with required privileges
pveum role add IncusOSDeployer -privs \
    "VM.Allocate VM.Config.CDROM VM.Config.CPU VM.Config.Disk VM.Config.HWType \
     VM.Config.Memory VM.Config.Network VM.Config.Options VM.PowerMgmt \
     VM.Monitor VM.Audit VM.Console \
     Datastore.AllocateSpace Datastore.Audit \
     Pool.Allocate Pool.Audit \
     SDN.Use Sys.Modify"

# Create resource pool
pveum pool add IncusLab -comment "IncusOS lab VMs"

# Create API token
pveum user token add automation@pve deploy --privsep 0
# Save the displayed token secret -- it's shown only once!

# Assign role to user on the pool
pveum acl modify /pool/IncusLab -user automation@pve -role IncusOSDeployer

# Assign role on storage
pveum acl modify /storage/local-zfs -user automation@pve -role IncusOSDeployer
pveum acl modify /storage/local -user automation@pve -role IncusOSDeployer

Record credentials

Add to the env file at the repo root:

# Hetzner
HETZNER_PROXMOX_TOKEN_SECRET=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
HETZNER_PROXMOX_ROOT_PASSWORD=your-root-password

Verify

# Export for incusos-proxmox
export PROXMOX_TOKEN_SECRET="$HETZNER_PROXMOX_TOKEN_SECRET"

# Test API access (update host/node in proxmox-api or use curl directly)
curl -sk "https://10.10.0.1:8006/api2/json/version" \
    -H "Authorization: PVEAPIToken=automation@pve!deploy=$PROXMOX_TOKEN_SECRET"

11. Final verification checklist

Check	Command	Expected
Proxmox version	`pvesh get /version`	PVE 8.x
Private bridge	`ip addr show vmbr1`	10.10.0.1/24
IP forwarding	`cat /proc/sys/net/ipv4/ip_forward`	1
NAT	`iptables -t nat -L -n`	MASQUERADE for 10.10.0.0/24
ZFS storage	`pvesm status`	local-zfs available
WireGuard	`wg show`	1+ peer, handshake recent
SSH via WG	`ssh hetzner-lab-wg hostname`	Responds
Web UI via WG	`curl -sk https://10.10.0.1:8006`	HTML response
API token	`curl -sk .../api2/json/version -H Auth...`	JSON with version
Public lockdown	`curl --connect-timeout 5 https://<pub-ip>:8006`	Timeout

Test deploy

export PROXMOX_TOKEN_SECRET="$HETZNER_PROXMOX_TOKEN_SECRET"

# Dry run -- verify correct bridge, IPs, storage
incusos-proxmox --dry-run \
    --proxmox incusos/targets/hetzner/proxmox.yaml \
    incusos/targets/hetzner/lab-cluster.yaml

Confirm the output shows: vmbr1 bridge, 10.10.0.x IPs, local-zfs storage, 8 cores per VM.

19 KiB Raw Blame History

Hetzner Dedicated Server: Proxmox Setup Guide

1. Server selection

2. Proxmox installation

2.1 Boot the rescue system

2.2 Discover disks

2.3 Check BIOS mode

2.4 Download the Proxmox ISO

2.5 Set up SSH port forwarding for VNC

2.6 Start the QEMU installer

2.7 Connect via VNC and install

2.8 Predict the real network interface name

2.9 Fix the network config before first real boot

2.10 Reboot into Proxmox

2.11 Verify

3. Network configuration

Create the private bridge

Persist IP forwarding

Apply

Verify

Network diagram

4. DNS setup (optional)

5. SSH hardening

Copy your SSH key

Disable password authentication

Add a colleague's key

SSH config for convenient access

Verify

6. Disk and storage setup

Identify disks

Check existing pools

Create VM storage pool

Verify

7. Repositories and system update

Switch to no-subscription repositories

Remove subscription nag popup

Update

Verify

8. WireGuard tunnel

Install

Generate server keys

Generate client keys (on your workstation)

Server config

Client config (on your workstation)

Enable and start

Verify

Adding more peers

9. Firewall lockdown

nftables rules

Apply

Persist across reboots

Verify

10. API token setup

Create role and user

Record credentials

Verify

11. Final verification checklist

Test deploy

19 KiB

Raw Blame History