Maarten
6d193158d5
Rewrite production lab guide for OC-managed provisioning
...
Rewrites sections 0-6 from scratch with OC-managed clustering (tokens,
OC ISOs, auto-registration) replacing manual cluster formation. Updates
sections 7-15 with correct naming (oc-node-* instead of lab-node-*),
fixes device override bug, and adds sequential launch guidance.
Key changes:
- Architecture: new VMID scheme (920/400-402), IP scheme (.120/.140-142)
- Node-01 disk increased to 250 GiB for Aether golden image
- OC provisioning flow: token → older-channel ISO → auto-update
- Fixed root device config: device override instead of device set
- Added image download timing notes (sequential, not concurrent)
- Updated OC Dashboard section for OC-managed deployment
- Updated node replacement procedure with correct VMIDs/IPs
- Updated cleanup to use --cleanup-all --deep --yes
Full lab validated from scratch: deploy, cluster, OVN, workloads,
isolation, peering, load balancing, live migration, evacuation/restore.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 22:19:03 +01:00
Maarten
fd30ef27d4
Expand Aether guide: deploy forms, blueprints, instance mgmt, OVN recovery
...
Document the full Aether platform after hands-on testing:
- Deploy form fields, validation, IP reservation, AWX integration
- Blueprint design (cluster-agnostic, component structure, JSON API)
- Blueprint deployment lifecycle (3-phase deploy, decommission workflow)
- Instance management (state control, resources, snapshots, console, ACLs)
- OVN recovery procedure after ovn-central container move (5-phase)
- Node resize procedure (Proxmox disk grow + ZFS pool expansion)
- Update lab-oc-nodes defaults to match resized nodes (20G RAM, 100G disk)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 22:59:30 +01:00
Maarten
b9f1894681
OC-managed cluster with OVN and HA nginx: full validated deployment
...
Deploy a 3-node OC-managed Incus cluster with OVN overlay networking
and an HA nginx workload behind an OVN load balancer. Key discoveries:
- OC needs_update blocker: OC tracks update delivery through its pipeline,
not version comparison. Nodes deployed from the latest ISO are stuck at
needs_update=true. Fix: generate ISO from older channel (--channel
old-stable), let OC push the real update after registration.
- ISO upload skip bug: incusos-proxmox silently reused stale ISOs with the
same filename. Fixed: --iso now replaces existing same-named ISOs.
- Empty app-seed-config ({}): avoids "Certificate already in trust store"
Terraform error when cert is already injected via SEED_DATA.
Files changed:
- notes/operations-center-guide.md: full rewrite with real captured output
- CLAUDE.md: needs_update findings, ISO fix, OVN LB behavior
- incusos/examples/lab-oc-deploy.yaml: static IP for VLAN 69
- incusos/examples/lab-oc-nodes.yaml: static IPs + disk sizing
- incusos/incusos-proxmox: fix stale ISO reuse when --iso provided
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 11:08:28 +01:00
Maarten
52d8dbdc89
OC v0.3.0 comprehensive test: retry logic, auto-heal, apply_defaults validation
...
Phase 0-10 of OC test plan completed:
- Add --retries N flag to incusos-proxmox (default 3, configurable boot retries)
- Add fix_scrub_schedule() auto-heal for IncusOS crontab bug via REST API
- Validate apply_defaults: false for OC-managed nodes (recommended approach)
- Test full OC lifecycle: deploy, provision, cluster, workloads, migration,
evacuation, node failure/recovery, inventory query, REST API
- Update CLAUDE.md: OC v0.3.0 commands, --retries, scrub fix, node recovery
- Update operations-center-guide.md: v0.3.0, apply_defaults, scrub fix,
node failure simulation, IncusOS REST API docs, inventory query
Key findings:
- apply_defaults: false lets OC Terraform handle everything cleanly
- fix_scrub_schedule() proactively heals ~50% crontab boot failure
- OC does not detect node failures (always shows "ready")
- inventory query (v0.3.0) provides cross-resource tree views
- Node crash recovery: auto-rejoin in ~60s, containers auto-start
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 21:41:34 +01:00
Maarten
8ae8700346
Add Operations Center hands-on testing, hybrid deployment, and comprehensive guide
...
Deployed and tested the full OC provisioning pipeline on Proxmox:
- Hybrid approach: incusos-proxmox --iso with OC-provisioned ISOs
- Token seeds, auto-registration, single-command cluster formation
- Inventory sync, live migration visibility, evacuation, node replacement
- Discovered OC reboot breaks TPM encryption on Proxmox VMs
Key findings documented: dual-seed coexistence, apply_defaults conflict
with Terraform, inventory requires explicit resync, stale server entries
from out-of-band changes, meshbr0 network added by OC clustering.
New files:
- notes/operations-center-guide.md: full tested OC reference
- incusos/examples/lab-oc-deploy.yaml: OC server deployment config
- incusos/examples/lab-oc-nodes.yaml: 3-node OC cluster config
Updated: CLAUDE.md (OC section rewrite), TESTING.md (hybrid workflow),
.gitignore (env file with Proxmox secrets)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 22:55:48 +01:00
Maarten