incus-contrib/incusos

IncusOS Tools

Scripts for building IncusOS installation media and seed configurations.

Overview

Script	Purpose
incusos-iso	Build customized IncusOS ISO/IMG images using the flasher-tool
incusos-seed	Generate seed archives (tar, FAT, or ISO 9660) for installation customization
incusos-proxmox	Declarative IncusOS deployment on Proxmox VE
lab-test	Guided lab validation (workloads, clustering, migration)

All scripts support automatic client certificate injection from the local Incus installation, so the freshly installed system is immediately manageable.

Prerequisites

Required:

• flasher-tool -- the official IncusOS image builder (used by incusos-iso)

  # Requires Go 1.22+
  go install github.com/lxc/incus-os/incus-osd/cmd/flasher-tool@latest
  export PATH="${GOPATH:-$HOME/go}/bin:$PATH"
  

  The incusos-iso script will offer to install it automatically if Go is available.

Recommended:

• Incus client -- for automatic client certificate detection. Any version works (certificates are read from disk), but 6.3+ has the most complete CLI. See notes/incus-version-compatibility.md for install instructions per platform.

  Platform	Install command	Notes
  macOS	brew install incus	Client only (no daemon on macOS)
  Debian/Ubuntu	apt install incus-client	Use Zabbly repo for latest
  Arch Linux	pacman -S incus	Full daemon+client (no client-only split)
  Any (source)	go install github.com/lxc/incus/v6/cmd/incus@latest	Requires Go 1.22+

Optional:

• genisoimage -- for creating ISO 9660 seed images (incusos-seed --format iso). Required by incusos-proxmox (Proxmox CD-ROM devices need ISO 9660, not FAT).
• mtools / dosfstools -- for creating FAT seed images (incusos-seed --format fat). Only for USB/block device use; FAT labels are NOT detected on CD-ROM devices.
• jq or python3 -- for parsing the CDN index (cross-architecture builds)
• curl -- for downloading images from the CDN

incusos-iso

Wrapper around the official flasher-tool that adds:

• Cross-architecture builds -- build aarch64 images on x86_64 and vice versa
• Automatic seed generation -- no need to manually create seed files
• Client certificate injection -- auto-detects your local Incus certificate
• Batch mode -- generate all common ISO variants in one command
• Sensible defaults -- minimal flags needed for common use cases

Quick Start

# Build a default IncusOS ISO for the current architecture
./incusos-iso

# Incus with default configuration (ZFS pool, bridge, port 8443)
./incusos-iso --defaults

# Operations Center for aarch64
./incusos-iso --arch aarch64 --app operations-center --defaults

# Build a USB image instead of ISO
./incusos-iso --format img --defaults

# Generate all variants (2 arches x 2 apps x 2 default modes = 8 images)
./incusos-iso --batch

# Preview what would be built
./incusos-iso --batch --dry-run

Options Reference

-a, --arch ARCH          x86_64 or aarch64 (default: auto-detect)
-A, --app APP            incus or operations-center (default: incus)
-d, --defaults           Apply default config (ZFS pool, bridge, port 8443)
-c, --cert FILE          Client certificate PEM file (default: auto-detect)
    --no-cert            Skip certificate injection
-F, --format FORMAT      iso or img (default: iso)
-o, --output FILE        Output filename (default: auto-generated)
-C, --channel CHANNEL    stable or testing (default: stable)
-s, --seed FILE          Use pre-built seed tar (skip seed generation)
-i, --image FILE         Use local base image (skip CDN download)
-b, --batch              Generate all common variants
    --batch-arches LIST  Architectures for batch (default: x86_64,aarch64)
    --batch-apps LIST    Apps for batch (default: incus,operations-center)
    --output-dir DIR     Output directory for batch (default: .)
    --disk-target ID     Target disk pattern (e.g. "virtio-*")
    --force-install      Overwrite existing installation
    --force-reboot       Auto-reboot after installation
    --missing-tpm        Allow install without TPM 2.0
    --missing-secureboot Allow install without Secure Boot
    --hostname NAME      Set system hostname
    --dry-run            Show what would happen
-q, --quiet              Suppress info output
-h, --help               Full help text
-V, --version            Show version

Output Naming Convention

Auto-generated filenames follow the pattern:

incusos-{app}-{defaults|bare}-{arch}.{iso|img}

Examples:

• incusos-incus-defaults-x86_64.iso
• incusos-ops-center-bare-aarch64.iso
• incusos-incus-defaults-x86_64.img

Batch Mode Matrix

With --batch, the following variants are generated:

#	Architecture	Application	Defaults	Filename
1	x86_64	Incus	yes	incusos-incus-defaults-x86_64.iso
2	x86_64	Incus	no	incusos-incus-bare-x86_64.iso
3	x86_64	Ops Center	yes	incusos-ops-center-defaults-x86_64.iso
4	x86_64	Ops Center	no	incusos-ops-center-bare-x86_64.iso
5	aarch64	Incus	yes	incusos-incus-defaults-aarch64.iso
6	aarch64	Incus	no	incusos-incus-bare-aarch64.iso
7	aarch64	Ops Center	yes	incusos-ops-center-defaults-aarch64.iso
8	aarch64	Ops Center	no	incusos-ops-center-bare-aarch64.iso

Customize with --batch-arches and --batch-apps:

# Only x86_64 Incus variants
./incusos-iso --batch --batch-arches x86_64 --batch-apps incus

incusos-seed

Generates IncusOS seed archives containing the YAML configuration files that customize an IncusOS installation. Output can be:

• tar archive (default) -- for use with flasher-tool --seed
• ISO 9660 image -- for use as a CD-ROM SEED_DATA device (e.g. Proxmox ide3)
• FAT image -- for use as a USB/block SEED_DATA device (NOT for CD-ROM)

Quick Start

# Standalone Incus with defaults and auto-detected certificate
./incusos-seed --defaults

# Operations Center seed
./incusos-seed --app operations-center --defaults --cert ~/.config/incus/client.crt

# Cluster node (no defaults) with forced install and reboot
./incusos-seed --force-install --force-reboot

# FAT image for external boot media
./incusos-seed --format fat --defaults --output seed.img

# Preview generated YAML without creating files
./incusos-seed --defaults --dry-run

Options Reference

-A, --app APP            incus or operations-center (default: incus)
-d, --defaults           Apply default configuration
-c, --cert FILE          Client certificate PEM file (default: auto-detect)
    --no-cert            Skip certificate injection
-o, --output FILE        Output file (default: incusos-seed.tar)
-f, --format FORMAT      tar, fat, or iso (default: tar)
    --fat-size SIZE      FAT image size in MiB (default: 10)
    --disk-target ID     Target disk pattern
    --force-install      Overwrite existing installation
    --force-reboot       Auto-reboot after installation
    --missing-tpm        Allow install without TPM 2.0
    --missing-secureboot Allow install without Secure Boot
    --hostname NAME      Set hostname
    --domain NAME        Set domain
    --dns SERVERS        Comma-separated DNS servers
    --channel CHANNEL    stable or testing (default: stable)
    --auto-reboot        Auto-reboot for updates
    --dry-run            Preview YAML output
-q, --quiet              Suppress info output
-h, --help               Full help text
-V, --version            Show version

Seed Files Reference

The following YAML files are generated inside the archive:

File	Always	Description
install.yaml	yes	Installation target, security, reboot settings
applications.yaml	yes	Application selection
incus.yaml	when app=incus	Incus preseed (defaults, certificates)
operations-center.yaml	when app=ops-center	Operations Center config
network.yaml	when network opts set	Hostname, DNS, static IP
update.yaml	when non-default	Update channel, auto-reboot

Using ISO 9660 Images (SEED_DATA for CD-ROM)

The ISO output format creates a small ISO 9660 image with volume label SEED_DATA. This is the required format when attaching seed data as a CD-ROM device (e.g. Proxmox ide3). FAT filesystem labels are NOT detected by the Linux kernel on CD-ROM (/dev/sr*) devices — only ISO 9660 volume labels work.

# Generate ISO seed for Proxmox deployment
./incusos-seed --format iso --defaults --output seed-node-1.iso

incusos-proxmox uses this format automatically.

Using FAT Images (SEED_DATA for USB/Block Devices)

The FAT output format creates a small disk image labeled SEED_DATA that can be attached to a VM as a secondary block device or written to USB. IncusOS detects this label at boot and reads the seed configuration from it.

Note: FAT images do NOT work when attached as CD-ROM devices. Use ISO 9660 format instead for CD-ROM use cases.

This is useful for the "pristine image" workflow on bare metal:

1. Download one uncustomized IncusOS ISO
2. Generate per-node seed images with incusos-seed --format fat
3. Boot each machine with both the ISO and the node-specific seed USB

# Generate seeds for three cluster nodes (USB/block device use)
for i in 1 2 3; do
    ./incusos-seed --format fat \
        --hostname "node-${i}" \
        --force-install \
        --output "seed-node-${i}.img"
done

incusos-proxmox

Declarative deployment of IncusOS VMs on Proxmox VE. Define your lab in a YAML file and deploy with a single command. Handles seed generation, ISO upload, VM creation with IncusOS-correct settings, and automated installation.

IncusOS requires very specific Proxmox VM settings (UEFI without pre-enrolled keys, TPM 2.0, VirtIO-SCSI, host CPU type, minimum 50 GiB disk, 4 GiB RAM). This script ensures all settings are correct, preventing install failures.

Quick Start

# Preview what would be deployed
./incusos-proxmox --dry-run examples/proxmox-lab.yaml

# Validate config and check Proxmox connectivity
./incusos-proxmox --phase validate examples/proxmox-lab.yaml

# Deploy everything
./incusos-proxmox examples/proxmox-lab.yaml

# Check deployment status
./incusos-proxmox --status examples/proxmox-lab.yaml

# Deploy a single VM from the config
./incusos-proxmox --vm ops-center examples/proxmox-lab.yaml

# Tear down all VMs defined in config
./incusos-proxmox --cleanup examples/proxmox-lab.yaml

Options Reference

-p, --phase PHASE       validate, prepare, upload, create, install, status, all (default)
-m, --method METHOD     Override connection method: ssh (default), api
-H, --host HOST         Override Proxmox host from config
-i, --iso FILE          Use local ISO (skip CDN download)
-c, --cert FILE         Client certificate PEM file (default: auto-detect)
    --no-cert           Skip certificate injection
    --vm NAME           Operate on a single VM only
    --retries N         Boot retries for first-boot failures (default: 3)
-y, --yes               Skip confirmation prompts
    --proxmox FILE      Proxmox connection config (default: auto-discover)
    --doctor            Check tool versions and environment health
    --status            Check deployment status (shortcut for --phase status)
    --cleanup           Destroy VMs defined in config
    --cleanup --deep    Also remove deployment ISO, seeds, remotes, and cache
    --cleanup-all       Destroy ALL managed VMs in pool (no config file needed)
    --cleanup-all --deep  Also remove ALL ISOs and seeds from storage
    --force-cleanup     Destroy VMs even without management marker
    --dry-run           Preview without executing
-v, --verbose           Show detailed output (tool paths, API calls)
-q, --quiet             Suppress info output
-h, --help              Full help text
-V, --version           Show version

Doctor Check

Run --doctor to verify your environment without a config file:

./incusos-proxmox --doctor
./incusos-proxmox --doctor -v   # verbose: show tool paths

This checks: incus client, flasher-tool, operations-center CLI (optional), python3, curl, genisoimage, jq, incusos-seed sibling, IncusOS CDN availability, and proxmox.yaml discovery.

If proxmox.yaml is found or a config file is provided, additionally checks Proxmox connectivity, pool existence, and API token validity:

./incusos-proxmox --doctor examples/lab-cluster.yaml

Deep Cleanup

Use --deep with --cleanup or --cleanup-all to remove artifacts beyond VMs:

# VMs + the IncusOS ISO used by this deployment + seeds + remotes + cache
./incusos-proxmox --cleanup --deep examples/lab-cluster.yaml

# Pool-wide: destroy all managed VMs (uses proxmox.yaml, no config needed)
./incusos-proxmox --cleanup-all
./incusos-proxmox --cleanup-all --deep   # also nuke ALL ISOs + seeds

--cleanup --deep:

• Deletes per-VM seed ISOs from Proxmox storage
• Deletes the specific IncusOS ISO used by this deployment (not all of them)
• Removes matching incus remote entries
• Clears the local ISO cache (~/.cache/incusos-proxmox/iso-cache/)

--cleanup-all --deep:

• Deletes ALL IncusOS_*.iso and seed-*.iso from storage (intentionally aggressive)
• Removes remotes for the destroyed VMs
• Clears the local ISO cache

Deployment Phases

Phase	What it does
validate	Parse YAML, check fields, verify Proxmox connectivity, allocate VMIDs
prepare	Download IncusOS ISO from CDN, generate per-VM SEED_DATA images
upload	Upload ISO and seed images to Proxmox ISO storage
create	Create VMs with IncusOS-correct settings (UEFI, TPM, VirtIO-SCSI, etc.)
install	Start VMs, detect install completion via disk I/O monitoring, detach CD-ROMs, boot from disk
status	Check deployment status: Proxmox state, IP, port 8443, incus remotes, post-deploy checks

Proxmox Connection Config

Proxmox connection settings live in a separate proxmox.yaml file to avoid duplicating credentials across lab configs. Auto-discovered in the script directory and current directory. Override with --proxmox FILE.

Copy the template to get started:

cp incusos/examples/proxmox.yaml.example incusos/proxmox.yaml
# Edit with your Proxmox host, credentials, and storage settings

# proxmox.yaml (flat structure, no wrapping key)
host: 192.168.1.10
method: api
api_token_id: automation@pve!deploy
node: pve
storage: local-lvm
iso_storage: local
bridge: vmbr0
vlan: 69
pool: IncusLab
ssh_user: root

Lab configs can override any setting by including a proxmox: section. Merge priority: proxmox.yaml → lab config proxmox: → CLI flags.

Lab Config File Format

# proxmox: section is optional — overrides proxmox.yaml if present
proxmox:
  storage: local-zfs        # Override storage for this deployment

defaults:
  cores: 4
  memory: 8192              # MiB, min 4096
  disk: 64                  # GiB, min 50
  channel: stable
  start_vmid: 200           # Auto-assign VMIDs from here

vms:
  - name: my-vm
    app: incus              # incus or operations-center
    apply_defaults: true
    cores: 4                # Override defaults per VM
    memory: 8192
    disk: 64
    vmid: 200               # Optional explicit VMID
    cluster: my-cluster     # Future: cluster formation
    cluster_role: init      # init or join

Connection Methods

SSH (default): Requires key-based SSH authentication to the Proxmox host. Simplest for homelab use.

# Test connectivity
ssh root@192.168.1.10 pvesh get /version

API: Requires an API token. Set up minimal privileges:

pveum role add IncusOSDeployer --privs "VM.Allocate VM.Config.Disk \
    VM.Config.CPU VM.Config.Memory VM.Config.Network VM.Config.CDROM \
    VM.Config.Options VM.Config.HWType VM.PowerMgmt VM.Audit \
    Datastore.AllocateSpace Datastore.AllocateTemplate Datastore.Audit SDN.Use Sys.Audit"
pveum user add automation@pve
pveum aclmod / -user automation@pve -role IncusOSDeployer
pveum user token add automation@pve deploy --privsep 0

export PROXMOX_TOKEN_SECRET="<secret-from-above>"

Resource Pool Isolation

For tighter access control, create a Proxmox resource pool and scope the API token to it instead of /:

# On Proxmox (one-time setup)
pveum pool add IncusLab --comment "IncusOS Lab VMs"
pveum pool modify IncusLab --storage local-lvm,local
pveum aclmod /pool/IncusLab -user automation@pve -role IncusOSDeployer

Then in your config file:

proxmox:
  pool: IncusLab

When a pool is configured, the script only "sees" VMs in that pool — both for collision detection and cleanup. The VM management marker includes the pool name for traceability.

Re-runs, Status, and Reconcile

Status check: Use --status to verify deployment health without modifying anything:

./incusos-proxmox --status examples/proxmox-lab.yaml

Shows per-VM: Proxmox state, install status, IP address, port 8443 reachability, and incus remote connectivity.

Re-run safety: When re-running a full deployment (--phase all), the script detects existing VMs and presents an interactive menu:

1. Run status checks and post-deployment verification
2. Continue deployment (skip existing, install incomplete VMs)
3. Destroy and redeploy cleanly
4. Abort

With --yes, defaults to option 1 (safe — never auto-destroys).

Install idempotency: The install phase checks each VM's state before acting: already-running VMs are skipped, stopped-but-installed VMs are started from disk, and only VMs with install media still attached proceed through installation.

Multi-Lab Management

Multiple lab environments can coexist on the same Proxmox host. Each lab uses a non-overlapping VMID range (see start_vmid in example configs). Only one lab needs to be running at a time — the others can be stopped to save RAM.

# Stop a lab (VMs stay on disk, ready to restart)
./incusos-proxmox --lab-down examples/lab-cluster.yaml

# Start a different lab
./incusos-proxmox --lab-up examples/lab-advanced.yaml

# See all deployed labs and their status
./incusos-proxmox --labs

# Check Proxmox host resource usage (RAM, storage, per-pool summary)
./incusos-proxmox --resources

# Permanently destroy a lab
./incusos-proxmox --cleanup examples/lab-cluster.yaml

Proxmox VM Settings

Every VM is created with these IncusOS-specific settings:

Setting	Value	Reason
BIOS	OVMF (UEFI)	IncusOS requires UEFI boot
Machine	q35	Modern chipset for UEFI
EFI disk	pre-enrolled-keys=0	IncusOS enrolls its own Secure Boot keys
TPM	v2.0	Required for disk encryption
CPU	host	Needs x86_64_v3 instruction set
SCSI controller	virtio-scsi-pci	VirtIO-blk is broken with IncusOS
Balloon	0 (disabled)	IncusOS manages memory internally
SEED_DATA	ISO 9660 image on ide3 CD-ROM	Per-VM seed configuration

Prerequisites

• python3 + PyYAML (python3-yaml) -- YAML config parsing
• genisoimage (Linux) or mkisofs from cdrtools (macOS) -- for ISO 9660 seed images
• ssh + scp (for SSH method) or curl (for API method)
• incusos-seed -- sibling script (auto-detected in same directory)
• Bash 4+ -- associative arrays
• Incus client (recommended) -- for client certificate auto-detection

Example Configurations

• examples/proxmox-lab.yaml -- Full 4-VM lab (ops-center + 3-node cluster)
• examples/proxmox-minimal.yaml -- Single standalone Incus VM

lab-test

Guided lab validation script for IncusOS deployments. Walks through workload testing, cluster formation, and migration tests step by step. Interactive by default — prints what it will do, confirms, executes, and reports pass/fail.

Quick Start

# Run all phases (deploy, single-node tests, cluster, workloads, migration)
./incusos/lab-test examples/lab-cluster.yaml

# Deploy and verify only
./incusos/lab-test --phase deploy examples/lab-cluster.yaml

# Single-node workload tests
./incusos/lab-test --phase single examples/lab-cluster.yaml

# Form cluster from deployed nodes
./incusos/lab-test --phase cluster examples/lab-cluster.yaml

# Test migration and evacuation
./incusos/lab-test --phase migrate examples/lab-cluster.yaml

# Full run without prompts
./incusos/lab-test --yes examples/lab-cluster.yaml

# Clean up test instances (keeps VMs)
./incusos/lab-test --cleanup examples/lab-cluster.yaml

Phases

Phase	What it does
deploy	Deploy VMs (via incusos-proxmox), verify incus remotes, check connectivity
single	Launch a test container and VM on the first node, exec, cleanup
cluster	Enable clustering on node 1, join remaining nodes, verify membership
workload	Launch instances on the cluster (scheduler-placed + targeted), cleanup
migrate	Stop/move container between nodes, evacuate/restore a node

Options

-p, --phase PHASE     deploy, single, cluster, workload, migrate, all (default)
    --skip-deploy     Skip deploy/status check (assume VMs already running)
    --cleanup         Remove test instances without destroying VMs
-y, --yes             Skip confirmation prompts
    --dry-run         Preview actions
-q, --quiet           Suppress info output
-h, --help            Full help text
-V, --version         Show version

Client Certificate Auto-Detection

Both scripts automatically look for a client certificate in this order:

1. ~/.config/incus/client.crt (on-disk, works with all Incus versions)
2. ~/snap/incus/common/config/client.crt (snap installations)
3. ~/.config/lxc/client.crt (legacy LXC path)
4. incus remote get-client-certificate (CLI fallback, requires Incus >= 6.3)

The file-based detection is preferred because incus remote get-client-certificate was added in Incus 6.3 and is not available in the 6.0 LTS shipped by Debian stable.

If no certificate exists yet, run incus remote list to trigger auto-generation of a client keypair at ~/.config/incus/client.{crt,key}.

Override with --cert FILE or disable with --no-cert.

Incus Version Compatibility

Feature	6.0 LTS (Debian stable)	6.3+ (Zabbly, Homebrew, Arch)
Certificate on disk	~/.config/incus/client.crt	~/.config/incus/client.crt
get-client-certificate CLI	not available	available
incus admin os commands	not available	available
Script cert auto-detection	works (reads file directly)	works (reads file directly)

These scripts work with any Incus version because they read the certificate file from disk rather than relying on newer CLI commands.

For full installation instructions per platform, see notes/incus-version-compatibility.md.

After Installation

With an injected certificate, the remote is already trusted — no authentication prompt appears:

incus remote add my-server <ip-address> --accept-certificate
incus remote list

For web UI access, export a PKCS#12 bundle and import it into your browser:

openssl pkcs12 -export \
    -inkey ~/.config/incus/client.key \
    -in ~/.config/incus/client.crt \
    -out ~/.config/incus/client.pfx

Examples

See the examples/ directory for sample seed configurations:

Seed templates:

• incus-standalone.yaml -- Single Incus server with defaults
• incus-cluster-node.yaml -- Cluster node (no defaults)
• ops-center.yaml -- Operations Center
• network-static.yaml -- Static network configuration

Proxmox connection:

• proxmox.yaml.example -- Proxmox connection config template

Lab configs (Proxmox deployment):

• proxmox-lab.yaml -- 4-VM lab (self-contained, includes proxmox: section)
• proxmox-minimal.yaml -- Single standalone VM (self-contained)
• lab-single.yaml -- Single standalone node (uses proxmox.yaml)
• lab-cluster.yaml -- 3-node cluster (uses proxmox.yaml)
• lab-advanced.yaml -- 3-node heterogeneous migration cluster
• lab-production.yaml -- Production-like cluster (OVN + HA)
• lab-replace.yaml -- Node replacement testing
• lab-networking.yaml -- OVN networking lab
• lab-oc.yaml -- Operations Center + 3 nodes (uses proxmox.yaml)
• lab-oc-deploy.yaml -- OC server only (for hybrid OC workflow)
• lab-oc-nodes.yaml -- 3 OC-managed nodes (requires --iso)
• lab-iscsi-target.yaml -- iSCSI target reference config

Platform Support

Script	Linux	macOS	Notes
incusos-iso	yes	yes	Requires flasher-tool (Go binary, compiles on both) and genisoimage/mkisofs
incusos-seed	yes	yes	Requires genisoimage (Linux) or mkisofs from cdrtools (macOS) for ISO format
incusos-proxmox	yes	yes	Uses incusos-seed for seed generation; SSH/API transport is cross-platform
lab-test	yes	yes	Only needs the incus CLI client

macOS prerequisites (when running tools against a remote Proxmox host):

brew install cdrtools    # provides mkisofs (replaces genisoimage)
brew install mtools      # provides mcopy (for FAT seed images)
brew install incus       # Incus CLI client

Scripts auto-detect the platform and use the appropriate tools (e.g. mkisofs on macOS vs genisoimage on Linux).

Future: UTM Support

A future incusos-utm script could deploy IncusOS VMs on macOS using UTM as the hypervisor. The seed generation (incusos-seed) already works on macOS. The main challenge is VM creation (UTM's utmctl CLI doesn't support it -- AppleScript or .utm bundle generation would be needed).

See notes/utm-support.md for the full design document.

Security Notes

• TPM 2.0 and Secure Boot are required by default. Use --missing-tpm and --missing-secureboot only for lab environments or VMs that lack these features.
• All storage pools are automatically encrypted by IncusOS.
• Retrieve encryption recovery keys after installation with: incus admin os system security show
• Keep recovery keys safe -- they are needed if TPM state changes.