Commit Graph

11 Commits

Author SHA1 Message Date
Maarten 4617de8869 Update all metadata and documentation for new repo structure
- .gitignore: envs/*/env, envs/*/proxmox.yaml instead of old incusos/ paths
- .claude/rules/*.md: all paths: directives updated from incusos/ to bin/
- .claude/skills/*.md: helper script paths updated
- CLAUDE.md: structure diagram, capabilities paths, context loading table
- README.md: structure, quick start examples with new paths
- bin/TESTING.md, bin/README.md: command examples updated
- envs/hetzner/setup/: all guide and script path references updated
- bin/examples/proxmox.yaml.example: updated copy instructions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 18:53:26 +01:00
Maarten 855cbfa073 Make deploy scripts environment-agnostic, update paths for new structure
- deploy-haproxy: clear hardcoded beelink IPs from defaults, add
  require_config() validation, update load_password() for envs/*/env
- deploy-awx: clear hardcoded IPs, add require_config(), update
  MANIFESTS_DIR to data/awx-manifests
- deploy-observability: clear hardcoded IPs, add --config/parse_config()
  support, add require_config(), update DASHBOARDS_DIR to
  data/observability-dashboards
- manage-dashboards: update DASHBOARDS_DIR path
- incusos-proxmox: update load_env_file() to search envs/*/env
- helpers/proxmox-api, proxmox-screenshot: update load_env() and
  find_proxmox_yaml() for new envs/ layout
- diagnostic/*: update load_env() and proxmox.yaml discovery
- .claude/skills: update helper script paths

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 18:45:49 +01:00
Maarten a7c856121d Add Hetzner bare metal setup guide, proxmox-setup script, and multi-target config layout
New hetzner/ directory with step-by-step Proxmox setup guide (networking,
WireGuard, firewall, storage, API tokens) and an interactive helper script
that automates all steps over SSH.

New incusos/targets/ layout with per-host connection configs and lab
definitions for beelink and hetzner targets. No changes to incusos-proxmox
itself -- uses existing --proxmox flag.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 15:52:57 +01:00
Maarten 360fba5680 Execute break-fix exercises, document results and fix VMIDs
Exercises executed on 2026-02-24:
- Exercise 1 (Update Observation): observed download, A/B switch, DB-leader
  transfer during reboot. All 3 nodes updated to 202602240349.
- Exercise 3 (Network Isolation): OVN gateway is SPOF (no failover),
  east-west traffic survives, MAC change causes boot hang.
- Exercise 4 (Full Node Failure): 2/3 quorum held, ~20s detection,
  ~50s recovery, instances auto-restart.
- Exercise 2 (Simulated Failed Update): blocked (no update available).

Fixes: VMIDs corrected from 900-902 to 400-402, added MAC safety rule,
documented Proxmox API --data-urlencode requirement for MAC addresses.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 16:51:26 +01:00
Maarten 95316a5bee Add IncusOS break-fix guide, incusos-health helper, immutability rules
Phase 4: Explore IncusOS immutability via the /os/1.0 API:
- incusos-health script with --status/--partitions/--tpm/--services/--network/--update
- Validated: TPM encryption (root+swap), Secure Boot with 4 certs, ZFS pool
- Documented A/B partition scheme, update mechanics, security chain
- Defined 4 break-fix exercises (update observation, failed update, network
  isolation, full node failure) with safety rules and pickup points

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 15:57:43 +01:00
Maarten cc017b648d Add observability stack: Prometheus, Grafana, Loki with dashboards
Deploy and validate full monitoring stack on the Incus cluster:
- Prometheus scraping 9 targets (3 Incus nodes, 2 HAProxy, 3 node-exporters, self)
- Grafana with 3 dashboards (cluster overview, HAProxy traffic, host resources)
- Loki + Promtail for log aggregation
- OVN network forward for LAN access (192.168.103.201:3000/9090)
- deploy-observability script with --deploy/--status/--cleanup/--doctor
- Solved OVN ACL challenges: Aether default-deny requires monitoring-allow ACL

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 15:44:59 +01:00
Maarten 91a87456a5 Add API interception guide, incus-mitm helper, and payload rules
Phase 2: captured Aether↔Incus API traffic using the event stream.
Identified callers by TLS certificate fingerprint (CLI vs Aether vs
cluster nodes). Documented API patterns for stop/start/snapshot/exec.
Key finding: Aether subscribes to lifecycle events and reactively
queries affected resources.

Deliverables:
- notes/api-interception-guide.md: capture setup + all API patterns
- incusos/helpers/incus-mitm: --capture/--live/--analyze/--callers
- .claude/rules/aether-api-payloads.md: context for AI assistants
- .mcp.json: add --ignore-https-errors for Playwright

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 15:01:56 +01:00
Maarten 94eb3c7cc4 Add OVN deep-dive guide, ovn-inspect helper, and internals rules
Phase 1 of observability plan: mapped the complete OVN topology at
logical (NB), physical (SB), and OVS layers. Traced packet path from
LAN through VIP LB to HAProxy backends. Documented naming conventions,
gateway failover, Geneve tunnels, and MTU implications.

Deliverables:
- notes/ovn-deep-dive.md: comprehensive OVN internals guide
- incusos/helpers/ovn-inspect: --nb/--sb/--ovs/--full/--trace actions
- .claude/rules/ovn-internals.md: context for AI assistants

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 14:52:37 +01:00
Maarten c047bc6ed4 Add HAProxy LB guide, deploy script, Playwright helper; validate end-to-end
Tear down and fully re-deploy HAProxy infrastructure from scratch to
validate the guide reads correctly. Key fixes from re-validation:

- Fix session API field names: lb_vip (not vip), target_ip/target_port
  (not ip/port), health_check_enabled (not health_check)
- Document that Aether does NOT create the OVN load balancer — must be
  created manually via incus network load-balancer
- Fix internal VIP test: OVN LB doesn't hairpin, use localhost instead
- Document OVN LB has no health checking (HAProxy failover not instant)
- Add cleanup step for manually-created OVN LB
- Add aether-browser Playwright helper and .mcp.json config
- Add deploy-haproxy script, haproxy-lb rules, full guide

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 14:08:43 +01:00
Maarten 0100182dd8 Validate AWX guide, document Aether API, enforce API-first policy
- AWX guide: fix wrong project ID (6→9), add dynamic ID lookups,
  add missing Step 5 push commands, add migration pod to expected
  output, replace PostgreSQL workaround with Aether UI instructions,
  reference deploy-awx as automated path
- Aether guide: document JWT REST API (auth, clusters, ACLs),
  swagger UI at /api/docs, rate limits, coverage vs UI-only features
- CLAUDE.md: add Aether API capability, add interaction hierarchy
  (API first → UI fallback → never root/DB in guides)
- deploy-awx: fix PLAYBOOK_DIR from "ansible/playbooks" to "playbooks"
- post-deploy.yml: clarify cert push path vs EE runtime mount path

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 10:33:46 +01:00
Maarten c4efcbe30d Modularize CLAUDE.md: rules, helpers, skills (866→112 lines)
Split monolithic CLAUDE.md into three layers:
- CLAUDE.md (112 lines): behavioral focus, capabilities, safety rules
- .claude/rules/ (8 files, 503 lines): topic-specific context with
  paths: frontmatter for conditional loading
- .claude/skills/ (2 files): /screenshot and /proxmox-api commands

Add helper scripts (incusos/helpers/):
- proxmox-screenshot: SSH→screendump→PPM→PNG pipeline with VMID safety
- proxmox-api: authenticated API calls handling ! in token quoting

Fix git workflow (master branch, both origin+aether remotes).
Add capabilities section for screenshots, API calls, live AWX output.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 09:48:25 +01:00