incus-contrib/CLAUDE.md

CLAUDE.md - Project context for AI assistants

What this repository is

A collection of peripheral tools, scripts, and snippets for working with Incus, IncusOS, and the broader ecosystem (Operations Center, Migration Manager). Primarily targeting home lab environments but aiming for production-quality scripts.

Repository structure

incu-contrib/
├── CLAUDE.md              # This file -- project context
├── README.md              # Main overview
├── .gitignore
├── incusos/               # IncusOS installation tooling
│   ├── README.md          # Detailed usage docs
│   ├── incusos-iso        # ISO/IMG builder (wraps flasher-tool)
│   ├── incusos-seed       # Seed archive generator
│   ├── incusos-proxmox    # Declarative Proxmox VM deployment
│   ├── TESTING.md         # Testing guide for incusos-proxmox
│   └── examples/          # Example seed + Proxmox YAML files
└── notes/                 # Research notes and reference material

Key technical context

Incus version differences

• Debian stable ships Incus 6.0 LTS which is significantly behind upstream. The Zabbly repo (https://github.com/zabbly/incus) provides latest on Debian/Ubuntu.
• macOS (Homebrew) and Arch Linux both track latest upstream (currently 6.21). macOS is client-only by design; Arch has no client-only split package.
• incus remote get-client-certificate was added in Incus 6.3+ and does not exist in 6.0 LTS. Scripts must never depend on it as the only cert path.
• Always prefer reading ~/.config/incus/client.crt directly from disk. Fall back to the CLI command only as a secondary option.
• See notes/incus-version-compatibility.md for full platform matrix and install instructions.

IncusOS flasher-tool

• Install: go install github.com/lxc/incus-os/incus-osd/cmd/flasher-tool@latest
• Actual CLI flags: -f/--format, -s/--seed, -c/--channel, -i/--image, -v/--version
• There is NO --seed-tar flag -- it's just --seed (or -s).
• There is NO --arch flag -- architecture is determined by the downloaded image. For cross-arch builds, download the image manually and pass via --image.
• CDN index: https://images.linuxcontainers.org/os/index.json
• CDN images: https://images.linuxcontainers.org/os/{version}/{arch}/IncusOS_{version}.{format}.gz

Seed archives

• Tar archives containing YAML files at the root level.
• Written to byte offset 2148532224 (the seed partition) in the image.
• Alternative: external boot media labeled SEED_DATA attached as CD-ROM.
  • ISO 9660 (genisoimage -V SEED_DATA -J -r): preferred for CD-ROM devices. Volume labels are correctly detected by the kernel on /dev/sr*.
  • FAT image (mkfs.fat -n SEED_DATA): works for USB/block devices but NOT for CD-ROM -- the Linux kernel's sr_mod driver does not expose FAT filesystem labels, so IncusOS cannot find the seed.
• Key files: install.yaml, applications.yaml, incus.yaml, operations-center.yaml, network.yaml, update.yaml.

Client certificates

• Stored at ~/.config/incus/client.crt and ~/.config/incus/client.key.
• Running incus remote list triggers auto-generation if no keypair exists.
• For Incus seed: injected under preseed.certificates[] (NOT preseed.server.certificates[]). The InitPreseed.Server field uses yaml:",inline" so its fields (including certificates) are promoted to the top level of the preseed object.
• For Operations Center seed: injected under trusted_client_certificates[].
• Operations Center requires at least one trusted certificate -- without it, you are locked out after installation.

Proxmox VE deployment

• incusos-proxmox reads a YAML config, generates per-VM SEED_DATA images via incusos-seed --format iso, uploads the ISO + seeds to Proxmox, creates VMs with IncusOS-correct settings, and boots them through installation.
• Connection methods: SSH (default, ssh root@host qm ...) or API (curl -k https://host:8006/api2/json/... with PVEAPIToken header).
• Minimum API privileges for token-based access:

  VM.Allocate VM.Config.Disk VM.Config.CPU VM.Config.Memory
  VM.Config.Network VM.Config.CDROM VM.Config.Options VM.Config.HWType
  VM.PowerMgmt VM.Audit Datastore.AllocateSpace Datastore.AllocateTemplate
  Datastore.Audit SDN.Use
  

• Required VM settings (getting any wrong causes IncusOS install failure):
  • bios=ovmf, machine=q35 -- UEFI boot required
  • efidisk0: pre-enrolled-keys=0 -- IncusOS enrolls its own Secure Boot keys
  • tpmstate0: version=v2.0 -- required for disk encryption
  • cpu=host -- needed for x86_64_v3 instruction set requirement
  • scsihw=virtio-scsi-pci + scsi0 -- VirtIO-blk is broken with IncusOS
  • balloon=0 -- IncusOS manages memory internally
  • ide3 -- SEED_DATA ISO 9660 image attached as second CD-ROM
  • Minimum 50 GiB disk, minimum 4096 MiB RAM
• Disk target: do NOT specify disk-target in the seed for Proxmox VMs. IncusOS does literal string matching (not glob) on disk device IDs. scsi-* does NOT match scsi-0QEMU_QEMU_HARDDISK_drive-scsi0. Omit disk-target entirely and let IncusOS auto-detect (works for single-disk VMs).
• Install flow (automated by incusos-proxmox):
  1. Boot VM with ISO (ide2) + SEED_DATA (ide3) + force_reboot: true in seed
  2. IncusOS reads seed, installs to disk (scsi0), auto-reboots
  3. Detect install completion by polling blockstat.scsi0.wr_bytes via API -- when disk writes start then stop for 15s (3 stable polls), install is done
  4. Stop the VM (Proxmox stop, not guest shutdown)
  5. Delete ide2 and ide3 -- IncusOS checks for install media at every boot and refuses to start if found, regardless of boot order
  6. Set boot order to order=scsi0 and start from disk
• IP detection: IncusOS is immutable and has no QEMU guest agent. Use ARP-based lookup: get MAC from Proxmox VM config → flush stale ARP → ping broadcast → look up MAC in ARP table. Verify with direct ping before trusting the result.
• force_reboot is required: without force_reboot: true in the seed, IncusOS sits at "please remove installation media" and waits indefinitely. It does NOT halt automatically. force_reboot triggers a guest-level reboot (note: this does NOT reset QEMU VM uptime -- only a Proxmox stop/start does).

Coding conventions for scripts

• Shell: bash with set -euo pipefail
• Arithmetic: use var=$((var + 1)) instead of ((var++)) to avoid false exits under set -e when the value is 0.
• Colors: support NO_COLOR=1 and TERM=dumb; use setup_colors() pattern.
• Flags: support both short (-d) and long (--defaults) options.
• Defaults: sane defaults so the script does something useful with zero flags.
• Dry run: all scripts should support --dry-run to preview actions.
• Cert detection order: files on disk first, CLI command second.
• Error messages: include actionable remediation steps, not just "failed".
• No hardcoded package managers: say "install the Incus client" with a link, not "sudo apt install incus".

Git workflow

• Main branch: main
• Development happens on feature branches
• Remote: private Gitea at ssh://git@192.168.1.200:2222/maarten/incu-contrib.git