Add production lab guide with validated output from real deployment
Comprehensive 1775-line guide (notes/production-lab-guide.md) covering full lab lifecycle: OC dashboard, 3-node manual clustering, OVN overlay networking, mixed workloads, live migration, network security, load balancers, and cluster evacuation/restore. Every command was executed on real infrastructure (2026-02-22) and output captured. Key findings incorporated: - IncusOS NIC is named `mgmt`, not `ens18` (UPLINK parent) - OC CLI requires `remote switch`, not `remote:` suffix syntax - LB/forward backends need IP addresses, not instance names - Cluster auto-schedules without --target flag - Migration ~7s at ~140 MB/s with heartbeat continuity Also: lab-production.yaml config, lab-oc.yaml apply_defaults fix, CLAUDE.md and networking-guide.md updates for mgmt/OC/LB findings. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
088baf6807
commit
d8f2a067c4
11
CLAUDE.md
11
CLAUDE.md
|
|
@ -441,6 +441,10 @@ incu-contrib/
|
|||
imported into the browser. Web UI is a React SPA at `/ui/`.
|
||||
- **OC is under active development** (v0.3.0). Commands and APIs may change.
|
||||
Use `operations-center --version` (not `version` subcommand) to check.
|
||||
- **OC CLI does NOT support `remote:` suffix syntax**: unlike the Incus CLI,
|
||||
the OC CLI requires `operations-center remote switch NAME` before running
|
||||
commands. `operations-center admin os show oc-lab:` fails with "Invalid
|
||||
number of arguments".
|
||||
- **v0.3.0 new commands**: `provisioning channel` (add/list/show),
|
||||
`cluster update/update-certificate/rename`, expanded `inventory`
|
||||
(network-acl, address-sets, load-balancers, peers, zones, integrations,
|
||||
|
|
@ -839,7 +843,8 @@ sshpass -p "$PROXMOX_ROOT_PASSWORD" ssh -o StrictHostKeyChecking=no \
|
|||
4. `incus cluster role add <remote>:<member> ovn-chassis` (all nodes)
|
||||
5. Create physical uplink network (two-step cluster pattern)
|
||||
6. Create OVN network with `--type=ovn network=UPLINK`
|
||||
- **Physical uplink network**: uses `parent=ens18` (IncusOS default NIC),
|
||||
- **Physical uplink network**: uses `parent=mgmt` (IncusOS management NIC —
|
||||
NOT `ens18`, which is the underlying device name but not exposed to Incus).
|
||||
`ipv4.ovn.ranges` reserves LAN IPs for OVN router external addresses,
|
||||
`ipv4.gateway` is the LAN gateway in CIDR format.
|
||||
- **OVN network isolation**: multiple OVN networks are fully isolated.
|
||||
|
|
@ -849,6 +854,10 @@ sshpass -p "$PROXMOX_ROOT_PASSWORD" ssh -o StrictHostKeyChecking=no \
|
|||
ACLs (per-source blocking), network peering, L4 load balancers (connection-
|
||||
based hashing, not round-robin), network forwards (port forwarding to LAN
|
||||
IPs), DNS resolution (per-network, hostname.incus domain).
|
||||
- **LB/forward backends require IP addresses**: `incus network load-balancer
|
||||
backend add` and `incus network forward port add` require the target
|
||||
instance's IP address, NOT its name. Using instance names fails with
|
||||
"Invalid target address".
|
||||
- See `notes/networking-guide.md` for full tutorial with test results.
|
||||
|
||||
### Migration into Incus
|
||||
|
|
|
|||
|
|
@ -3,6 +3,11 @@
|
|||
# Deploys 1 Operations Center instance and 3 Incus nodes.
|
||||
# Uses VMID range 910+ to avoid collisions with lab-cluster.yaml (900+).
|
||||
#
|
||||
# The init node (lab-node-01) uses apply_defaults: true for storage pool
|
||||
# and network bridge. Joining nodes use apply_defaults: false -- the cluster
|
||||
# join process creates member-specific pool entries. This follows the upstream
|
||||
# IncusOS recommendation and avoids the 8-command cleanup per joining node.
|
||||
#
|
||||
# Workflow:
|
||||
# 1. Deploy with incusos-proxmox
|
||||
# 2. Access OC web UI (import client.pfx into browser)
|
||||
|
|
@ -33,7 +38,7 @@ vms:
|
|||
|
||||
- name: lab-node-01
|
||||
app: incus
|
||||
apply_defaults: true
|
||||
apply_defaults: true # init node: needs storage pool + network
|
||||
cores: 4
|
||||
memory: 8192
|
||||
disk: 64
|
||||
|
|
@ -41,10 +46,14 @@ vms:
|
|||
|
||||
- name: lab-node-02
|
||||
app: incus
|
||||
apply_defaults: true
|
||||
apply_defaults: false # joining node: cluster join creates pool entry
|
||||
cores: 4
|
||||
memory: 8192
|
||||
ip: 192.168.102.112/22
|
||||
|
||||
- name: lab-node-03
|
||||
app: incus
|
||||
apply_defaults: true
|
||||
apply_defaults: false # joining node: cluster join creates pool entry
|
||||
cores: 4
|
||||
memory: 8192
|
||||
ip: 192.168.102.113/22
|
||||
|
|
|
|||
|
|
@ -0,0 +1,71 @@
|
|||
# lab-production.yaml - Production-quality home lab (OC + 3-node cluster)
|
||||
#
|
||||
# Deploys 1 Operations Center instance and 3 Incus nodes for manual clustering.
|
||||
# Uses VMID range 910-913 (same as lab-oc.yaml -- do not run both simultaneously).
|
||||
#
|
||||
# Architecture:
|
||||
# lab-oc (910) - Operations Center dashboard and monitoring
|
||||
# lab-node-01 (911) - Cluster init node + OVN control plane host (64G disk)
|
||||
# lab-node-02 (912) - Cluster member
|
||||
# lab-node-03 (913) - Cluster member
|
||||
#
|
||||
# Strategy: Hybrid OC + Manual Clustering
|
||||
# - OC deployed with standard ISO (dashboard/monitoring role)
|
||||
# - 3 nodes deployed with standard ISO, static IPs on VLAN 69
|
||||
# - Cluster formed manually via incus CLI (proven reliable)
|
||||
# - OVN overlay networking configured post-clustering
|
||||
# - Mixed workloads: containers + VMs with live migration
|
||||
#
|
||||
# Joining nodes use apply_defaults: false -- the cluster join process creates
|
||||
# member-specific pool entries. This follows the upstream IncusOS recommendation
|
||||
# and avoids the 8-command cleanup per joining node.
|
||||
#
|
||||
# RAM budget: 28 GiB of 64 GiB (44% utilization)
|
||||
# OVN external IPs: 192.168.103.200-210 (excluded from DHCP)
|
||||
#
|
||||
# Usage:
|
||||
# incusos-proxmox --dry-run examples/lab-production.yaml # Preview
|
||||
# incusos-proxmox --yes examples/lab-production.yaml # Deploy
|
||||
# incusos-proxmox --status examples/lab-production.yaml # Check status
|
||||
# incusos-proxmox --cleanup examples/lab-production.yaml # Tear down VMs
|
||||
# incusos-proxmox --cleanup --deep examples/lab-production.yaml # Full cleanup
|
||||
#
|
||||
# After deployment, follow notes/production-lab-guide.md for:
|
||||
# - Cluster formation (section 4)
|
||||
# - OVN networking (section 6)
|
||||
# - Workload deployment (sections 7-9)
|
||||
# - Live migration testing (section 10)
|
||||
# - Cluster lifecycle (section 11)
|
||||
#
|
||||
# Connection settings from proxmox.yaml (see examples/proxmox.yaml.example).
|
||||
# Add a proxmox: section here to override per-deployment.
|
||||
|
||||
defaults:
|
||||
cores: 4
|
||||
memory: 8192
|
||||
disk: 50
|
||||
start_vmid: 910
|
||||
|
||||
vms:
|
||||
- name: lab-oc
|
||||
app: operations-center
|
||||
apply_defaults: true
|
||||
cores: 2
|
||||
memory: 4096
|
||||
ip: 192.168.102.110/22
|
||||
|
||||
- name: lab-node-01
|
||||
app: incus
|
||||
apply_defaults: true # init node: needs storage pool + network
|
||||
disk: 64 # extra space for OVN control plane container
|
||||
ip: 192.168.102.111/22
|
||||
|
||||
- name: lab-node-02
|
||||
app: incus
|
||||
apply_defaults: false # joining node: cluster join creates pool entry
|
||||
ip: 192.168.102.112/22
|
||||
|
||||
- name: lab-node-03
|
||||
app: incus
|
||||
apply_defaults: false # joining node: cluster join creates pool entry
|
||||
ip: 192.168.102.113/22
|
||||
|
|
@ -380,7 +380,10 @@ incus network create net-node-01:UPLINK --type=physical \
|
|||
dns.nameservers=192.168.1.1
|
||||
```
|
||||
|
||||
- `parent=ens18`: the node's physical NIC (IncusOS default on Proxmox)
|
||||
- `parent=ens18`: the node's physical NIC. **Note**: newer IncusOS versions
|
||||
name this interface `mgmt`. If `ens18` fails with "Parent interface not
|
||||
found", use `parent=mgmt` instead. Check with:
|
||||
`incus query REMOTE:/os/1.0/system/network`
|
||||
- `ipv4.ovn.ranges`: pool of LAN IPs for OVN router external addresses
|
||||
- `ipv4.gateway`: LAN gateway in CIDR format
|
||||
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load Diff
Loading…
Reference in New Issue