Maarten
52d8dbdc89
OC v0.3.0 comprehensive test: retry logic, auto-heal, apply_defaults validation
...
Phase 0-10 of OC test plan completed:
- Add --retries N flag to incusos-proxmox (default 3, configurable boot retries)
- Add fix_scrub_schedule() auto-heal for IncusOS crontab bug via REST API
- Validate apply_defaults: false for OC-managed nodes (recommended approach)
- Test full OC lifecycle: deploy, provision, cluster, workloads, migration,
evacuation, node failure/recovery, inventory query, REST API
- Update CLAUDE.md: OC v0.3.0 commands, --retries, scrub fix, node recovery
- Update operations-center-guide.md: v0.3.0, apply_defaults, scrub fix,
node failure simulation, IncusOS REST API docs, inventory query
Key findings:
- apply_defaults: false lets OC Terraform handle everything cleanly
- fix_scrub_schedule() proactively heals ~50% crontab boot failure
- OC does not detect node failures (always shows "ready")
- inventory query (v0.3.0) provides cross-resource tree views
- Node crash recovery: auto-rejoin in ~60s, containers auto-start
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 21:41:34 +01:00
Maarten
ca660585dc
Correct reboot diagnosis: OC-specific, not dqlite/TPM corruption
...
Volume testing with live Proxmox infrastructure proved that guest reboot
does NOT break standalone IncusOS nodes. Tested:
- 4 consecutive Proxmox stop/start cycles: all clean, data intact
- Single-node guest reboot: recovered in ~50s, containers auto-started
- Simultaneous 3-node guest reboot: all recovered, data integrity verified
The "reboot breaks IncusOS" issue is OC-specific. Console errors showed:
- Node 1: "failed to parse update frequency" (OC pushed bad crontab)
- Node 2: stuck at "starting application" (waiting for peers in loops)
- Node 3: "constraint violation" (OC re-registering existing server)
Updated all 8 references across CLAUDE.md, operations-center-guide.md,
and TESTING.md to reflect the corrected diagnosis.
Also fixed one more bare return bug in do_list_labs (empty pool path).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 00:48:06 +01:00
Maarten
f812e04746
Add macOS support, multi-lab lifecycle, extended testing, and migration docs
...
Fix OC reboot diagnosis across all docs (dqlite corruption, not TPM PCR).
Add cross-platform support: genisoimage/mkisofs detection, portable dd/grep/
date usage. Add --lab-up/--lab-down/--resources/--labs to incusos-proxmox for
multi-lab coexistence. Add 7 new lab-test phases (storage, network, projects,
backup, limits, security, resilience). Fix VMID collision in lab-advanced
(910→930). Create migration guide and UTM design doc.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 23:41:06 +01:00
Maarten
8ae8700346
Add Operations Center hands-on testing, hybrid deployment, and comprehensive guide
...
Deployed and tested the full OC provisioning pipeline on Proxmox:
- Hybrid approach: incusos-proxmox --iso with OC-provisioned ISOs
- Token seeds, auto-registration, single-command cluster formation
- Inventory sync, live migration visibility, evacuation, node replacement
- Discovered OC reboot breaks TPM encryption on Proxmox VMs
Key findings documented: dual-seed coexistence, apply_defaults conflict
with Terraform, inventory requires explicit resync, stale server entries
from out-of-band changes, meshbr0 network added by OC clustering.
New files:
- notes/operations-center-guide.md: full tested OC reference
- incusos/examples/lab-oc-deploy.yaml: OC server deployment config
- incusos/examples/lab-oc-nodes.yaml: 3-node OC cluster config
Updated: CLAUDE.md (OC section rewrite), TESTING.md (hybrid workflow),
.gitignore (env file with Proxmox secrets)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 22:55:48 +01:00
Maarten