Restructure Hetzner guide: manual install only, hand off to proxmox-setup
The guide now covers only the manual steps (server selection, QEMU-based Proxmox install, SSH alias setup) and then hands off to proxmox-setup for everything else (repos, networking, storage, WireGuard, firewall, API tokens). Removed sections 3-11 that duplicated what the script does interactively. Added section 3 with SSH setup, script invocation, WireGuard verification, and test deploy. Updated README.md to match the new two-phase workflow. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
14d27fccc8
commit
bb0f516dec
|
|
@ -12,37 +12,49 @@ via a WireGuard tunnel instead of LAN.
|
|||
- 32+ cores, 128+ GiB RAM
|
||||
- 2+ disks (ZFS mirror for system, extra disks for VM storage)
|
||||
- An SSH key pair on your workstation
|
||||
- A VNC client (for the Proxmox installer)
|
||||
- This repository cloned locally
|
||||
|
||||
## Quickstart
|
||||
## Workflow
|
||||
|
||||
### 1. Install Proxmox on the server
|
||||
The setup is split into a manual guide and an automated script:
|
||||
|
||||
Order the server, boot the rescue system, and install Proxmox via
|
||||
`installimage` or ISO. See [hetzner-setup.md](hetzner-setup.md) section 2
|
||||
for details.
|
||||
### 1. Manual: rent server and install Proxmox
|
||||
|
||||
### 2. Run the setup helper
|
||||
Follow [hetzner-setup.md](hetzner-setup.md) sections 1-2:
|
||||
|
||||
- Pick and order a server from the Hetzner auction
|
||||
- Install Proxmox via QEMU in the rescue system (ZFS RAID1 mirror)
|
||||
- Fix the network interface name, reboot, verify SSH works
|
||||
|
||||
### 2. Manual: set up SSH alias
|
||||
|
||||
Follow [hetzner-setup.md](hetzner-setup.md) section 3.1-3.3:
|
||||
|
||||
```bash
|
||||
ssh-copy-id root@<public-ip>
|
||||
# Add "Host hetzner-lab" entry to ~/.ssh/config
|
||||
ssh hetzner-lab pvesh get /version # Must succeed
|
||||
```
|
||||
|
||||
### 3. Automated: configure everything else
|
||||
|
||||
```bash
|
||||
# Add SSH config entry for the server (see hetzner-setup.md section 5)
|
||||
# Then run the interactive setup:
|
||||
hetzner/proxmox-setup --host hetzner-lab --dry-run # Preview
|
||||
hetzner/proxmox-setup --host hetzner-lab # Execute
|
||||
```
|
||||
|
||||
The script configures repositories, private networking, ZFS storage,
|
||||
WireGuard tunnel, firewall, and API tokens -- everything needed to
|
||||
deploy VMs.
|
||||
The script interactively configures: repos and updates, private bridge
|
||||
(vmbr1), ZFS storage on extra disks, WireGuard tunnel, firewall lockdown,
|
||||
and API tokens for `incusos-proxmox`.
|
||||
|
||||
### 3. Deploy IncusOS VMs
|
||||
### 4. Deploy IncusOS VMs
|
||||
|
||||
```bash
|
||||
# Copy and edit the connection config
|
||||
cp incusos/targets/hetzner/proxmox.yaml.example incusos/targets/hetzner/proxmox.yaml
|
||||
# Fill in the API token from step 2
|
||||
# Fill in the API token from step 3
|
||||
|
||||
# Deploy a 3-node cluster
|
||||
export PROXMOX_TOKEN_SECRET="<token>"
|
||||
incusos-proxmox --proxmox incusos/targets/hetzner/proxmox.yaml \
|
||||
incusos/targets/hetzner/lab-cluster.yaml
|
||||
```
|
||||
|
|
@ -51,8 +63,8 @@ incusos-proxmox --proxmox incusos/targets/hetzner/proxmox.yaml \
|
|||
|
||||
| File | Description |
|
||||
|------|-------------|
|
||||
| [hetzner-setup.md](hetzner-setup.md) | Comprehensive step-by-step guide |
|
||||
| [proxmox-setup](proxmox-setup) | Interactive helper script (automates the guide) |
|
||||
| [hetzner-setup.md](hetzner-setup.md) | Manual guide: server selection, Proxmox install, SSH setup |
|
||||
| [proxmox-setup](proxmox-setup) | Interactive script: repos, networking, storage, WireGuard, firewall, API tokens |
|
||||
|
||||
## Network architecture
|
||||
|
||||
|
|
|
|||
|
|
@ -1,8 +1,13 @@
|
|||
# Hetzner Dedicated Server: Proxmox Setup Guide
|
||||
|
||||
Complete guide for turning a Hetzner bare metal server into a Proxmox host
|
||||
ready for IncusOS lab deployments. Each section can be done manually or
|
||||
automated with `proxmox-setup` (see [proxmox-setup](proxmox-setup)).
|
||||
This guide covers the manual steps to get a Hetzner dedicated server running
|
||||
Proxmox VE with SSH access from your workstation. Once SSH works, the
|
||||
interactive [`proxmox-setup`](proxmox-setup) script handles everything else
|
||||
(repos, networking, storage, WireGuard, firewall, API tokens).
|
||||
|
||||
**Workflow:**
|
||||
1. **This guide** -- rent a server, install Proxmox via QEMU, get SSH working
|
||||
2. **`proxmox-setup --host <alias>`** -- configure everything else interactively
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -61,8 +66,9 @@ strong favourite -- fast and reliable. Note the device names (`/dev/nvme0n1`,
|
|||
`/dev/nvme1n1`) for the QEMU command.
|
||||
|
||||
If you have additional disks beyond the pair (e.g. large SATA drives),
|
||||
those can be set up as a separate storage pool later (section 6). Only
|
||||
pass the system disks to QEMU for now.
|
||||
those can be set up as a separate storage pool later. Only pass the system
|
||||
disks to QEMU for now -- `proxmox-setup` will detect and configure the
|
||||
rest.
|
||||
|
||||
### 2.3 Check BIOS mode
|
||||
|
||||
|
|
@ -223,9 +229,6 @@ ssh root@<public-ip> pvesh get /version
|
|||
# Should show Proxmox VE version and API info
|
||||
```
|
||||
|
||||
The web UI is available at `https://<public-ip>:8006` (we'll lock this
|
||||
down to WireGuard-only in section 9).
|
||||
|
||||
> **Alternative method**: You can also install Debian 13 via Hetzner's
|
||||
> `installimage` and then upgrade to Proxmox following the
|
||||
> [official guide](https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_13_Trixie).
|
||||
|
|
@ -234,63 +237,123 @@ down to WireGuard-only in section 9).
|
|||
|
||||
---
|
||||
|
||||
## 3. Network configuration
|
||||
## 3. SSH setup and handoff to proxmox-setup
|
||||
|
||||
Hetzner dedicateds get a single public IP with MAC filtering -- you cannot
|
||||
assign additional public IPs to VMs without ordering extra IPs. Instead,
|
||||
we create a private bridge and NAT.
|
||||
At this point Proxmox is installed and reachable via its public IP.
|
||||
The remaining configuration (repos, private networking, storage,
|
||||
WireGuard, firewall, API tokens) is handled by the `proxmox-setup`
|
||||
script. You just need SSH key access and an alias.
|
||||
|
||||
### Create the private bridge
|
||||
|
||||
Add `vmbr1` to `/etc/network/interfaces`:
|
||||
|
||||
```
|
||||
auto vmbr1
|
||||
iface vmbr1 inet static
|
||||
address 10.10.0.1/24
|
||||
bridge-ports none
|
||||
bridge-stp off
|
||||
bridge-fd 0
|
||||
post-up echo 1 > /proc/sys/net/ipv4/ip_forward
|
||||
post-up iptables -t nat -A POSTROUTING -s 10.10.0.0/24 -o vmbr0 -j MASQUERADE
|
||||
post-down iptables -t nat -D POSTROUTING -s 10.10.0.0/24 -o vmbr0 -j MASQUERADE
|
||||
```
|
||||
|
||||
This creates a private bridge where:
|
||||
- VMs connect to `vmbr1` and get IPs in 10.10.0.0/24
|
||||
- The Proxmox host (10.10.0.1) is the default gateway
|
||||
- NAT masquerading gives VMs internet access through the host's public IP
|
||||
- IP forwarding is enabled automatically on bridge up
|
||||
|
||||
### Persist IP forwarding
|
||||
|
||||
Also add to `/etc/sysctl.d/99-forward.conf` for boot persistence:
|
||||
|
||||
```
|
||||
net.ipv4.ip_forward = 1
|
||||
```
|
||||
|
||||
### Apply
|
||||
### 3.1 Copy your SSH key
|
||||
|
||||
```bash
|
||||
ifreload -a # Apply network changes without reboot
|
||||
sysctl -p /etc/sysctl.d/99-forward.conf
|
||||
ssh-copy-id root@<public-ip>
|
||||
```
|
||||
|
||||
### Verify
|
||||
### 3.2 Create an SSH config alias
|
||||
|
||||
Add to `~/.ssh/config` on your workstation:
|
||||
|
||||
```
|
||||
Host hetzner-lab
|
||||
HostName <public-ip>
|
||||
User root
|
||||
IdentityFile ~/.ssh/id_ed25519
|
||||
```
|
||||
|
||||
### 3.3 Verify SSH alias works
|
||||
|
||||
```bash
|
||||
ip addr show vmbr1 # Should show 10.10.0.1/24
|
||||
cat /proc/sys/net/ipv4/ip_forward # Should show 1
|
||||
iptables -t nat -L POSTROUTING -n # Should show MASQUERADE rule
|
||||
ssh hetzner-lab pvesh get /version
|
||||
```
|
||||
|
||||
### Network diagram
|
||||
This must succeed before continuing. If it fails, check your SSH key
|
||||
and config entry.
|
||||
|
||||
### 3.4 Run proxmox-setup
|
||||
|
||||
```bash
|
||||
# Preview what will be configured
|
||||
hetzner/proxmox-setup --host hetzner-lab --dry-run
|
||||
|
||||
# Run for real (interactive, prompts before each step)
|
||||
hetzner/proxmox-setup --host hetzner-lab
|
||||
```
|
||||
|
||||
The script walks through each step interactively:
|
||||
|
||||
1. **Detect** -- reads host info, block devices, existing config
|
||||
2. **Repos** -- switches to no-subscription repos, removes nag popup
|
||||
3. **Update** -- `apt update && apt dist-upgrade`
|
||||
4. **Bridge** -- creates vmbr1 (private bridge, 10.10.0.0/24) with NAT
|
||||
5. **Storage** -- detects unused disks, creates ZFS pool, registers with PVE
|
||||
6. **WireGuard** -- installs and configures tunnel, prints client config
|
||||
7. **Firewall** -- locks down public interface (SSH + WireGuard only)
|
||||
8. **API token** -- creates role, pool, and token for `incusos-proxmox`
|
||||
|
||||
Each step prompts for confirmation. Use `--yes` to skip prompts, or
|
||||
`--skip-repos`, `--skip-storage`, etc. to skip individual steps.
|
||||
|
||||
At the end, the script prints:
|
||||
- A ready-to-use `proxmox.yaml` config for `incusos/targets/hetzner/`
|
||||
- An `env` file snippet with the API token
|
||||
- A WireGuard client config to save on your workstation
|
||||
|
||||
### 3.5 Connect WireGuard and verify
|
||||
|
||||
Save the WireGuard client config printed by the script:
|
||||
|
||||
```bash
|
||||
# Linux
|
||||
sudo wg-quick up hetzner-lab
|
||||
|
||||
# macOS -- import into the WireGuard app
|
||||
```
|
||||
|
||||
Verify access through the tunnel:
|
||||
|
||||
```bash
|
||||
ping 10.10.0.1 # Proxmox host via WireGuard
|
||||
curl -sk https://10.10.0.1:8006 # Web UI via WireGuard
|
||||
```
|
||||
|
||||
Optionally add a second SSH alias for access via WireGuard:
|
||||
|
||||
```
|
||||
Host hetzner-lab-wg
|
||||
HostName 10.10.0.1
|
||||
User root
|
||||
IdentityFile ~/.ssh/id_ed25519
|
||||
```
|
||||
|
||||
### 3.6 Test deploy
|
||||
|
||||
```bash
|
||||
# Copy and fill in the connection config
|
||||
cp incusos/targets/hetzner/proxmox.yaml.example incusos/targets/hetzner/proxmox.yaml
|
||||
|
||||
# Set the API token
|
||||
export PROXMOX_TOKEN_SECRET="<token from proxmox-setup output>"
|
||||
|
||||
# Dry run -- verify correct bridge, IPs, storage
|
||||
incusos-proxmox --dry-run \
|
||||
--proxmox incusos/targets/hetzner/proxmox.yaml \
|
||||
incusos/targets/hetzner/lab-cluster.yaml
|
||||
```
|
||||
|
||||
Confirm the output shows: `vmbr1` bridge, `10.10.0.x` IPs, `local-zfs`
|
||||
storage, 8 cores per VM.
|
||||
|
||||
---
|
||||
|
||||
## Network architecture (reference)
|
||||
|
||||
Once `proxmox-setup` has completed, the network looks like this:
|
||||
|
||||
```
|
||||
Internet
|
||||
|
|
||||
| Public IP (5.9.x.x)
|
||||
| Public IP (e.g. 5.9.x.x)
|
||||
v
|
||||
[vmbr0] ──── Proxmox host ──── [wg0: 10.10.99.1/24]
|
||||
| |
|
||||
|
|
@ -304,481 +367,11 @@ Internet
|
|||
.101 .102 .103
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. DNS setup (optional)
|
||||
|
||||
If you have a domain, create an A record pointing to the public IP:
|
||||
|
||||
```
|
||||
pve.example.com → 5.9.x.x
|
||||
```
|
||||
|
||||
This is convenient but not required -- everything is accessed via
|
||||
WireGuard using private IPs anyway. If you set a hostname, update
|
||||
`/etc/hosts`:
|
||||
|
||||
```
|
||||
10.10.0.1 pve.example.com pve
|
||||
```
|
||||
|
||||
And set the hostname:
|
||||
|
||||
```bash
|
||||
hostnamectl set-hostname pve
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. SSH hardening
|
||||
|
||||
### Copy your SSH key
|
||||
|
||||
```bash
|
||||
ssh-copy-id root@<public-ip>
|
||||
```
|
||||
|
||||
### Disable password authentication
|
||||
|
||||
Edit `/etc/ssh/sshd_config`:
|
||||
|
||||
```
|
||||
PermitRootLogin prohibit-password
|
||||
PasswordAuthentication no
|
||||
```
|
||||
|
||||
Restart:
|
||||
|
||||
```bash
|
||||
systemctl restart sshd
|
||||
```
|
||||
|
||||
### Add a colleague's key
|
||||
|
||||
```bash
|
||||
# From their workstation:
|
||||
ssh-copy-id root@<public-ip>
|
||||
|
||||
# Or manually append to /root/.ssh/authorized_keys
|
||||
```
|
||||
|
||||
### SSH config for convenient access
|
||||
|
||||
Add to `~/.ssh/config` on your workstation:
|
||||
|
||||
```
|
||||
Host hetzner-lab
|
||||
HostName 5.9.x.x # Public IP (or pve.example.com)
|
||||
User root
|
||||
IdentityFile ~/.ssh/id_ed25519
|
||||
```
|
||||
|
||||
After WireGuard is set up (section 8), add a second entry:
|
||||
|
||||
```
|
||||
Host hetzner-lab-wg
|
||||
HostName 10.10.0.1
|
||||
User root
|
||||
IdentityFile ~/.ssh/id_ed25519
|
||||
```
|
||||
|
||||
### Verify
|
||||
|
||||
```bash
|
||||
ssh hetzner-lab pvesh get /version
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Disk and storage setup
|
||||
|
||||
### Identify disks
|
||||
|
||||
```bash
|
||||
lsblk -d -o NAME,SIZE,MODEL,ROTA,TYPE
|
||||
```
|
||||
|
||||
Typical Hetzner setup: 2x NVMe for system (already in ZFS mirror from
|
||||
install), 2+ additional disks for VM storage.
|
||||
|
||||
### Check existing pools
|
||||
|
||||
```bash
|
||||
zpool status # System mirror
|
||||
pvesm status # Proxmox storage backends
|
||||
```
|
||||
|
||||
### Create VM storage pool
|
||||
|
||||
If you have additional disks (e.g. `sda`, `sdb`) not used by the system:
|
||||
|
||||
```bash
|
||||
# Mirror (2 disks) -- recommended
|
||||
zpool create local-zfs mirror /dev/sda /dev/sdb
|
||||
|
||||
# RAIDZ1 (3+ disks) -- more space, still redundant
|
||||
zpool create local-zfs raidz1 /dev/sda /dev/sdb /dev/sdc
|
||||
|
||||
# Single disk (no redundancy)
|
||||
zpool create local-zfs /dev/sda
|
||||
```
|
||||
|
||||
Register with Proxmox:
|
||||
|
||||
```bash
|
||||
pvesm add zfspool local-zfs -pool local-zfs
|
||||
pvesm set local-zfs -content images,rootdir
|
||||
```
|
||||
|
||||
### Verify
|
||||
|
||||
```bash
|
||||
pvesm status
|
||||
# Should list local-zfs with type zfspool
|
||||
```
|
||||
|
||||
If the system install already created a suitable ZFS pool, skip pool
|
||||
creation and just register it with Proxmox if not already visible.
|
||||
|
||||
---
|
||||
|
||||
## 7. Repositories and system update
|
||||
|
||||
### Switch to no-subscription repositories
|
||||
|
||||
The default enterprise repos require a paid subscription. Switch to
|
||||
community repos:
|
||||
|
||||
```bash
|
||||
# Disable enterprise repo
|
||||
sed -i 's/^deb/# deb/' /etc/apt/sources.list.d/pve-enterprise.list
|
||||
|
||||
# Add no-subscription repo
|
||||
echo "deb http://download.proxmox.com/debian/pve bookworm pve-no-subscription" \
|
||||
> /etc/apt/sources.list.d/pve-no-subscription.list
|
||||
|
||||
# Disable Ceph enterprise repo if present
|
||||
if [[ -f /etc/apt/sources.list.d/ceph.list ]]; then
|
||||
sed -i 's/^deb/# deb/' /etc/apt/sources.list.d/ceph.list
|
||||
fi
|
||||
```
|
||||
|
||||
### Remove subscription nag popup
|
||||
|
||||
```bash
|
||||
# Standard JS patch for the web UI subscription dialog
|
||||
sed -Ezi.bak \
|
||||
"s/(Ext\.Msg\.show\(\{[^}]+title: gettext\('No valid sub)/void\(\{ \/\/\1/" \
|
||||
/usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js
|
||||
systemctl restart pveproxy
|
||||
```
|
||||
|
||||
### Update
|
||||
|
||||
```bash
|
||||
apt update && apt dist-upgrade -y
|
||||
```
|
||||
|
||||
### Verify
|
||||
|
||||
```bash
|
||||
apt update 2>&1 | tail -5
|
||||
# Should show no errors, all repos reachable
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. WireGuard tunnel
|
||||
|
||||
WireGuard provides secure access to VMs and the Proxmox web UI from
|
||||
your workstation, without exposing anything on the public interface.
|
||||
|
||||
### Install
|
||||
|
||||
```bash
|
||||
apt install -y wireguard
|
||||
```
|
||||
|
||||
### Generate server keys
|
||||
|
||||
```bash
|
||||
wg genkey | tee /etc/wireguard/server-private.key | wg pubkey > /etc/wireguard/server-public.key
|
||||
chmod 600 /etc/wireguard/server-private.key
|
||||
```
|
||||
|
||||
### Generate client keys (on your workstation)
|
||||
|
||||
```bash
|
||||
wg genkey | tee wg-client-private.key | wg pubkey > wg-client-public.key
|
||||
```
|
||||
|
||||
### Server config
|
||||
|
||||
Create `/etc/wireguard/wg0.conf`:
|
||||
|
||||
```ini
|
||||
[Interface]
|
||||
PrivateKey = <contents of server-private.key>
|
||||
Address = 10.10.99.1/24
|
||||
ListenPort = 51820
|
||||
|
||||
# Allow forwarding between WireGuard clients and the private VM bridge
|
||||
PostUp = iptables -A FORWARD -i wg0 -o vmbr1 -j ACCEPT; iptables -A FORWARD -i vmbr1 -o wg0 -j ACCEPT
|
||||
PostDown = iptables -D FORWARD -i wg0 -o vmbr1 -j ACCEPT; iptables -D FORWARD -i vmbr1 -o wg0 -j ACCEPT
|
||||
|
||||
[Peer]
|
||||
# Workstation
|
||||
PublicKey = <contents of wg-client-public.key>
|
||||
AllowedIPs = 10.10.99.2/32
|
||||
```
|
||||
|
||||
### Client config (on your workstation)
|
||||
|
||||
Save as `/etc/wireguard/hetzner-lab.conf` (or import into your WireGuard app):
|
||||
|
||||
```ini
|
||||
[Interface]
|
||||
PrivateKey = <contents of wg-client-private.key>
|
||||
Address = 10.10.99.2/24
|
||||
|
||||
[Peer]
|
||||
PublicKey = <contents of server-public.key>
|
||||
Endpoint = <public-ip>:51820
|
||||
AllowedIPs = 10.10.0.0/16
|
||||
PersistentKeepalive = 25
|
||||
```
|
||||
|
||||
**AllowedIPs note**: Using `10.10.0.0/16` instead of just the VM subnet
|
||||
leaves room for future subnets (OVN overlay at 10.10.10.0/24, etc.)
|
||||
without needing to update the WireGuard config.
|
||||
|
||||
### Enable and start
|
||||
|
||||
On the server:
|
||||
|
||||
```bash
|
||||
systemctl enable --now wg-quick@wg0
|
||||
```
|
||||
|
||||
On the workstation:
|
||||
|
||||
```bash
|
||||
# Linux
|
||||
sudo wg-quick up hetzner-lab
|
||||
|
||||
# macOS (WireGuard app)
|
||||
# Import hetzner-lab.conf and activate
|
||||
```
|
||||
|
||||
### Verify
|
||||
|
||||
```bash
|
||||
# On the server
|
||||
wg show
|
||||
|
||||
# From your workstation
|
||||
ping 10.10.99.1 # WireGuard interface
|
||||
ping 10.10.0.1 # Private bridge (should work -- routed via WireGuard)
|
||||
```
|
||||
|
||||
### Adding more peers
|
||||
|
||||
To add a colleague, generate a new keypair and add a `[Peer]` block to
|
||||
the server config:
|
||||
|
||||
```ini
|
||||
[Peer]
|
||||
# Colleague name
|
||||
PublicKey = <their-public-key>
|
||||
AllowedIPs = 10.10.99.3/32
|
||||
```
|
||||
|
||||
Then reload:
|
||||
|
||||
```bash
|
||||
systemctl restart wg-quick@wg0
|
||||
```
|
||||
|
||||
Give them a client config with `Address = 10.10.99.3/24` and the
|
||||
server's public key.
|
||||
|
||||
---
|
||||
|
||||
## 9. Firewall lockdown
|
||||
|
||||
After WireGuard is working, lock down the public interface so only SSH
|
||||
and WireGuard are accessible from the internet. Proxmox web UI and VM
|
||||
traffic go through the tunnel only.
|
||||
|
||||
### nftables rules
|
||||
|
||||
Create `/etc/nftables-hetzner.conf`:
|
||||
|
||||
```
|
||||
#!/usr/sbin/nft -f
|
||||
|
||||
flush ruleset
|
||||
|
||||
table inet filter {
|
||||
chain input {
|
||||
type filter hook input priority 0; policy drop;
|
||||
|
||||
# Loopback
|
||||
iif lo accept
|
||||
|
||||
# Established/related connections
|
||||
ct state established,related accept
|
||||
|
||||
# ICMP (ping)
|
||||
ip protocol icmp accept
|
||||
ip6 nexthdr icmpv6 accept
|
||||
|
||||
# SSH on public interface
|
||||
iifname "vmbr0" tcp dport 22 accept
|
||||
|
||||
# WireGuard on public interface
|
||||
iifname "vmbr0" udp dport 51820 accept
|
||||
|
||||
# Allow everything on private bridge and WireGuard
|
||||
iifname "vmbr1" accept
|
||||
iifname "wg0" accept
|
||||
|
||||
# Log and drop everything else
|
||||
log prefix "nft-drop: " limit rate 5/minute counter drop
|
||||
}
|
||||
|
||||
chain forward {
|
||||
type filter hook forward priority 0; policy drop;
|
||||
|
||||
# Established/related
|
||||
ct state established,related accept
|
||||
|
||||
# WireGuard <-> private bridge
|
||||
iifname "wg0" oifname "vmbr1" accept
|
||||
iifname "vmbr1" oifname "wg0" accept
|
||||
|
||||
# Private bridge -> internet (NAT)
|
||||
iifname "vmbr1" oifname "vmbr0" accept
|
||||
}
|
||||
|
||||
chain output {
|
||||
type filter hook output priority 0; policy accept;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Apply
|
||||
|
||||
```bash
|
||||
nft -f /etc/nftables-hetzner.conf
|
||||
```
|
||||
|
||||
### Persist across reboots
|
||||
|
||||
```bash
|
||||
cp /etc/nftables-hetzner.conf /etc/nftables.conf
|
||||
systemctl enable nftables
|
||||
```
|
||||
|
||||
### Verify
|
||||
|
||||
From an external machine (not through WireGuard):
|
||||
|
||||
```bash
|
||||
# Should work
|
||||
ssh hetzner-lab echo ok
|
||||
|
||||
# Should time out (port 8006 blocked on public interface)
|
||||
curl -sk --connect-timeout 5 https://<public-ip>:8006
|
||||
|
||||
# Through WireGuard -- should work
|
||||
curl -sk https://10.10.0.1:8006
|
||||
```
|
||||
|
||||
**Warning**: Test SSH access through WireGuard *before* applying firewall
|
||||
rules. If you lock yourself out, use Hetzner's rescue system to recover.
|
||||
|
||||
---
|
||||
|
||||
## 10. API token setup
|
||||
|
||||
Create a dedicated API token for `incusos-proxmox` automation.
|
||||
|
||||
### Create role and user
|
||||
|
||||
```bash
|
||||
# Create role with required privileges
|
||||
pveum role add IncusOSDeployer -privs \
|
||||
"VM.Allocate VM.Config.CDROM VM.Config.CPU VM.Config.Disk VM.Config.HWType \
|
||||
VM.Config.Memory VM.Config.Network VM.Config.Options VM.PowerMgmt \
|
||||
VM.Monitor VM.Audit VM.Console \
|
||||
Datastore.AllocateSpace Datastore.Audit \
|
||||
Pool.Allocate Pool.Audit \
|
||||
SDN.Use Sys.Modify"
|
||||
|
||||
# Create resource pool
|
||||
pveum pool add IncusLab -comment "IncusOS lab VMs"
|
||||
|
||||
# Create API token
|
||||
pveum user token add automation@pve deploy --privsep 0
|
||||
# Save the displayed token secret -- it's shown only once!
|
||||
|
||||
# Assign role to user on the pool
|
||||
pveum acl modify /pool/IncusLab -user automation@pve -role IncusOSDeployer
|
||||
|
||||
# Assign role on storage
|
||||
pveum acl modify /storage/local-zfs -user automation@pve -role IncusOSDeployer
|
||||
pveum acl modify /storage/local -user automation@pve -role IncusOSDeployer
|
||||
```
|
||||
|
||||
### Record credentials
|
||||
|
||||
Add to the `env` file at the repo root:
|
||||
|
||||
```bash
|
||||
# Hetzner
|
||||
HETZNER_PROXMOX_TOKEN_SECRET=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
|
||||
HETZNER_PROXMOX_ROOT_PASSWORD=your-root-password
|
||||
```
|
||||
|
||||
### Verify
|
||||
|
||||
```bash
|
||||
# Export for incusos-proxmox
|
||||
export PROXMOX_TOKEN_SECRET="$HETZNER_PROXMOX_TOKEN_SECRET"
|
||||
|
||||
# Test API access (update host/node in proxmox-api or use curl directly)
|
||||
curl -sk "https://10.10.0.1:8006/api2/json/version" \
|
||||
-H "Authorization: PVEAPIToken=automation@pve!deploy=$PROXMOX_TOKEN_SECRET"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 11. Final verification checklist
|
||||
|
||||
| Check | Command | Expected |
|
||||
|-------|---------|----------|
|
||||
| Proxmox version | `pvesh get /version` | PVE 8.x |
|
||||
| Private bridge | `ip addr show vmbr1` | 10.10.0.1/24 |
|
||||
| IP forwarding | `cat /proc/sys/net/ipv4/ip_forward` | 1 |
|
||||
| NAT | `iptables -t nat -L -n` | MASQUERADE for 10.10.0.0/24 |
|
||||
| ZFS storage | `pvesm status` | local-zfs available |
|
||||
| WireGuard | `wg show` | 1+ peer, handshake recent |
|
||||
| SSH via WG | `ssh hetzner-lab-wg hostname` | Responds |
|
||||
| Web UI via WG | `curl -sk https://10.10.0.1:8006` | HTML response |
|
||||
| API token | `curl -sk .../api2/json/version -H Auth...` | JSON with version |
|
||||
| Public lockdown | `curl --connect-timeout 5 https://<pub-ip>:8006` | Timeout |
|
||||
|
||||
### Test deploy
|
||||
|
||||
```bash
|
||||
export PROXMOX_TOKEN_SECRET="$HETZNER_PROXMOX_TOKEN_SECRET"
|
||||
|
||||
# Dry run -- verify correct bridge, IPs, storage
|
||||
incusos-proxmox --dry-run \
|
||||
--proxmox incusos/targets/hetzner/proxmox.yaml \
|
||||
incusos/targets/hetzner/lab-cluster.yaml
|
||||
```
|
||||
|
||||
Confirm the output shows: `vmbr1` bridge, `10.10.0.x` IPs, `local-zfs`
|
||||
storage, 8 cores per VM.
|
||||
| Interface | Purpose |
|
||||
|-----------|---------|
|
||||
| vmbr0 | Public (SSH + WireGuard only after firewall) |
|
||||
| vmbr1 | Private bridge for VMs (10.10.0.0/24, NAT to internet) |
|
||||
| wg0 | WireGuard tunnel (10.10.99.0/24) |
|
||||
|
||||
AllowedIPs uses `10.10.0.0/16` to leave room for future subnets
|
||||
(OVN overlay at 10.10.10.0/24, etc.) without updating WireGuard config.
|
||||
|
|
|
|||
Loading…
Reference in New Issue