Safety fix: --cleanup-all, --resources, --labs silently failed when
PROXMOX_TOKEN_SECRET was not set, reporting "No VMs found" instead of
an auth error. Added require_api_auth() gate and error propagation in
pve_list_vms() to fail loudly on API auth failures.
Also:
- Standardize role name to IncusOSDeployer across all docs and scripts
- Add Sys.Audit privilege to all setup command examples
- Document /nodes/pve ACL requirement for scoped permissions
- Update lab-cluster.yaml: apply_defaults: false for joining nodes
(upstream IncusOS recommendation to avoid 8-command cleanup)
- Add full cluster automation to lab-test phase_cluster:
core.https_address setup, remote cert fixup, automated join prompts,
conditional defaults cleanup for apply_defaults: true nodes
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Volume testing with live Proxmox infrastructure proved that guest reboot
does NOT break standalone IncusOS nodes. Tested:
- 4 consecutive Proxmox stop/start cycles: all clean, data intact
- Single-node guest reboot: recovered in ~50s, containers auto-started
- Simultaneous 3-node guest reboot: all recovered, data integrity verified
The "reboot breaks IncusOS" issue is OC-specific. Console errors showed:
- Node 1: "failed to parse update frequency" (OC pushed bad crontab)
- Node 2: stuck at "starting application" (waiting for peers in loops)
- Node 3: "constraint violation" (OC re-registering existing server)
Updated all 8 references across CLAUDE.md, operations-center-guide.md,
and TESTING.md to reflect the corrected diagnosis.
Also fixed one more bare return bug in do_list_labs (empty pool path).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- New lab-advanced.yaml (3-node 6/4/4 core heterogeneous cluster) and
lab-replace.yaml (single replacement node) for advanced migration and
cluster lifecycle testing
- New notes/clustering-guide.md with full clustering walkthrough: cluster
formation, storage pool conflict fix, join workflow, VM live migration
with limits.cpu range fix, multi-vCPU test results (2/3/4 vCPUs),
edge cases (I/O, memory, concurrent), and node replacement lifecycle
- Updated CLAUDE.md with clustering docs, rebalancing config keys,
cluster enable syntax fix (TWO args not one), cluster remove syntax,
agent reconnect timing (3-4s not 1-2s), and lifecycle procedure
- proxmox.yaml.example for Proxmox connection config reference
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Major changes to incusos-proxmox and incusos-seed to achieve fully automated
IncusOS deployment to Proxmox VE. Fixes multiple blockers discovered during
testing: FAT→ISO 9660 seed format (CD-ROM label detection), preseed YAML
structure (Go yaml:",inline"), disk target auto-detection, install completion
monitoring via blockstat disk I/O, install media ejection, and ARP-based IP
detection for the agent-less IncusOS.
Key changes:
- incusos-seed: add --format iso using genisoimage, fix preseed structure
- incusos-proxmox: ISO seeds, blockstat install detection, media ejection,
ARP-based IP lookup, force_reboot requirement
- Documentation: comprehensive updates to CLAUDE.md, README.md, TESTING.md
- Examples: fix preseed.certificates path in all YAML examples
- Notes: full session progress with all bugs found and fixed
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Maarten