From e2909c914916637fe8fc66015cb3ad9b99bd2093 Mon Sep 17 00:00:00 2001 From: Maarten Date: Wed, 18 Feb 2026 14:49:31 +0100 Subject: [PATCH] version and certificate refinement --- CLAUDE.md | 84 ++++++++++++++ README.md | 17 ++- incusos/README.md | 67 ++++++++--- incusos/incusos-iso | 55 +++++---- incusos/incusos-seed | 54 +++++---- notes/incus-version-compatibility.md | 164 +++++++++++++++++++++++++++ 6 files changed, 377 insertions(+), 64 deletions(-) create mode 100644 CLAUDE.md create mode 100644 notes/incus-version-compatibility.md diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..9cfddda --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,84 @@ +# CLAUDE.md - Project context for AI assistants + +## What this repository is + +A collection of peripheral tools, scripts, and snippets for working with +Incus, IncusOS, and the broader ecosystem (Operations Center, Migration Manager). +Primarily targeting home lab environments but aiming for production-quality scripts. + +## Repository structure + +``` +incu-contrib/ +├── CLAUDE.md # This file -- project context +├── README.md # Main overview +├── .gitignore +├── incusos/ # IncusOS installation tooling +│ ├── README.md # Detailed usage docs +│ ├── incusos-iso # ISO/IMG builder (wraps flasher-tool) +│ ├── incusos-seed # Seed archive generator +│ └── examples/ # Example seed YAML files +└── notes/ # Research notes and reference material +``` + +## Key technical context + +### Incus version differences + +- **Debian stable ships Incus 6.0 LTS** which is significantly behind upstream. + The Zabbly repo (https://github.com/zabbly/incus) provides latest on Debian/Ubuntu. +- **macOS (Homebrew)** and **Arch Linux** both track latest upstream (currently 6.21). + macOS is client-only by design; Arch has no client-only split package. +- `incus remote get-client-certificate` was added in **Incus 6.3+** and does + not exist in 6.0 LTS. Scripts must never depend on it as the only cert path. +- Always prefer reading `~/.config/incus/client.crt` directly from disk. + Fall back to the CLI command only as a secondary option. +- See `notes/incus-version-compatibility.md` for full platform matrix and + install instructions. + +### IncusOS flasher-tool + +- Install: `go install github.com/lxc/incus-os/incus-osd/cmd/flasher-tool@latest` +- Actual CLI flags: `-f/--format`, `-s/--seed`, `-c/--channel`, `-i/--image`, `-v/--version` +- There is NO `--seed-tar` flag -- it's just `--seed` (or `-s`). +- There is NO `--arch` flag -- architecture is determined by the downloaded image. + For cross-arch builds, download the image manually and pass via `--image`. +- CDN index: `https://images.linuxcontainers.org/os/index.json` +- CDN images: `https://images.linuxcontainers.org/os/{version}/{arch}/IncusOS_{version}.{format}.gz` + +### Seed archives + +- Tar archives containing YAML files at the root level. +- Written to byte offset 2148532224 (the seed partition) in the image. +- Alternative: FAT image labeled `SEED_DATA` as external boot media. +- Key files: `install.yaml`, `applications.yaml`, `incus.yaml`, + `operations-center.yaml`, `network.yaml`, `update.yaml`. + +### Client certificates + +- Stored at `~/.config/incus/client.crt` and `~/.config/incus/client.key`. +- Running `incus remote list` triggers auto-generation if no keypair exists. +- For Incus seed: injected under `preseed.server.certificates[]`. +- For Operations Center seed: injected under `trusted_client_certificates[]`. +- Operations Center **requires** at least one trusted certificate -- without it, + you are locked out after installation. + +## Coding conventions for scripts + +- **Shell**: bash with `set -euo pipefail` +- **Arithmetic**: use `var=$((var + 1))` instead of `((var++))` to avoid + false exits under `set -e` when the value is 0. +- **Colors**: support `NO_COLOR=1` and `TERM=dumb`; use setup_colors() pattern. +- **Flags**: support both short (`-d`) and long (`--defaults`) options. +- **Defaults**: sane defaults so the script does something useful with zero flags. +- **Dry run**: all scripts should support `--dry-run` to preview actions. +- **Cert detection order**: files on disk first, CLI command second. +- **Error messages**: include actionable remediation steps, not just "failed". +- **No hardcoded package managers**: say "install the Incus client" with a link, + not "sudo apt install incus". + +## Git workflow + +- Main branch: `main` +- Development happens on feature branches +- Remote: private Gitea at `ssh://git@192.168.1.200:2222/maarten/incu-contrib.git` diff --git a/README.md b/README.md index e0f2f0b..914843a 100644 --- a/README.md +++ b/README.md @@ -23,12 +23,18 @@ See [`incusos/README.md`](incusos/README.md) for full documentation. ### [`notes/`](notes/) -Research notes and reference material. +Research notes and reference material: + +- [`incus-version-compatibility.md`](notes/incus-version-compatibility.md) -- + Incus package versions across platforms (macOS, Debian/Ubuntu, Arch), + install instructions, and certificate management notes. +- [`iso-download-methods.md`](notes/iso-download-methods.md) -- + Original research on ISO download/customization approaches. ## Quick Start ```bash -# Install the flasher-tool (requires Go 1.21+) +# Install the flasher-tool (requires Go 1.22+) go install github.com/lxc/incus-os/incus-osd/cmd/flasher-tool@latest # Build a default IncusOS ISO with your client cert injected @@ -49,11 +55,14 @@ go install github.com/lxc/incus-os/incus-osd/cmd/flasher-tool@latest ## Requirements -- **Go 1.21+** -- for installing the flasher-tool +- **Go 1.22+** -- for installing the flasher-tool - **curl** -- for downloading images from the CDN - **tar**, **gzip** -- for seed archive creation and image decompression +- **Incus client** (recommended) -- for automatic client certificate detection. + Works with any version; certificates are read directly from disk. + See [version compatibility notes](notes/incus-version-compatibility.md) + for install instructions per platform (macOS, Debian/Ubuntu, Arch). - **mtools** (optional) -- for creating FAT seed images -- **incus** (optional) -- for automatic client certificate detection ## License diff --git a/incusos/README.md b/incusos/README.md index 570d405..f4803ae 100644 --- a/incusos/README.md +++ b/incusos/README.md @@ -14,19 +14,37 @@ installation, so the freshly installed system is immediately manageable. ## Prerequisites -- **Go 1.21+** -- for installing the flasher-tool -- **flasher-tool** -- the official IncusOS image builder +**Required:** -```bash -go install github.com/lxc/incus-os/incus-osd/cmd/flasher-tool@latest -export PATH="${GOPATH:-$HOME/go}/bin:$PATH" -``` +- **flasher-tool** -- the official IncusOS image builder (used by `incusos-iso`) -Optional: -- **incus** -- for automatic client certificate detection -- **mtools** / **dosfstools** -- for creating FAT seed images + ```bash + # Requires Go 1.22+ + go install github.com/lxc/incus-os/incus-osd/cmd/flasher-tool@latest + export PATH="${GOPATH:-$HOME/go}/bin:$PATH" + ``` + + The `incusos-iso` script will offer to install it automatically if Go is available. + +**Recommended:** + +- **Incus client** -- for automatic client certificate detection. + Any version works (certificates are read from disk), but 6.3+ has the + most complete CLI. See [`notes/incus-version-compatibility.md`](../notes/incus-version-compatibility.md) + for install instructions per platform. + + | Platform | Install command | Notes | + |----------|----------------|-------| + | macOS | `brew install incus` | Client only (no daemon on macOS) | + | Debian/Ubuntu | `apt install incus-client` | Use [Zabbly repo](https://github.com/zabbly/incus) for latest | + | Arch Linux | `pacman -S incus` | Full daemon+client (no client-only split) | + | Any (source) | `go install github.com/lxc/incus/v6/cmd/incus@latest` | Requires Go 1.22+ | + +**Optional:** + +- **mtools** / **dosfstools** -- for creating FAT seed images (`incusos-seed --format fat`) - **jq** or **python3** -- for parsing the CDN index (cross-architecture builds) -- **curl** -- for downloading images +- **curl** -- for downloading images from the CDN ## incusos-iso @@ -217,13 +235,34 @@ done Both scripts automatically look for a client certificate in this order: -1. `incus remote get-client-certificate` (if the `incus` CLI is installed) -2. `~/.config/incus/client.crt` -3. `~/snap/incus/common/config/client.crt` -4. `~/.config/lxc/client.crt` +1. `~/.config/incus/client.crt` (on-disk, works with all Incus versions) +2. `~/snap/incus/common/config/client.crt` (snap installations) +3. `~/.config/lxc/client.crt` (legacy LXC path) +4. `incus remote get-client-certificate` (CLI fallback, requires Incus >= 6.3) + +The file-based detection is preferred because `incus remote get-client-certificate` +was added in Incus 6.3 and is not available in the 6.0 LTS shipped by Debian stable. + +If no certificate exists yet, run `incus remote list` to trigger auto-generation +of a client keypair at `~/.config/incus/client.{crt,key}`. Override with `--cert FILE` or disable with `--no-cert`. +### Incus Version Compatibility + +| Feature | 6.0 LTS (Debian stable) | 6.3+ (Zabbly, Homebrew, Arch) | +|---------|-------------------------|-------------------------------| +| Certificate on disk | `~/.config/incus/client.crt` | `~/.config/incus/client.crt` | +| `get-client-certificate` CLI | not available | available | +| `incus admin os` commands | not available | available | +| Script cert auto-detection | works (reads file directly) | works (reads file directly) | + +These scripts work with **any** Incus version because they read the certificate +file from disk rather than relying on newer CLI commands. + +For full installation instructions per platform, see +[`notes/incus-version-compatibility.md`](../notes/incus-version-compatibility.md). + ### After Installation With an injected certificate, the remote is already trusted: diff --git a/incusos/incusos-iso b/incusos/incusos-iso index c29adae..29aca24 100755 --- a/incusos/incusos-iso +++ b/incusos/incusos-iso @@ -470,21 +470,11 @@ detect_client_cert() { return fi - # Auto-detection + # Auto-detection: prefer reading cert files directly (works with all + # Incus versions including 6.0 LTS which lacks 'get-client-certificate'). step "Auto-detecting client certificate" - # Try incus CLI - if command -v incus &>/dev/null; then - local cert - if cert=$(incus remote get-client-certificate 2>/dev/null); then - CERT_CONTENT="$cert" - success "Certificate obtained from Incus CLI" - return - fi - detail "Incus CLI available but could not retrieve certificate" - fi - - # Try common file locations + # 1) Check common on-disk certificate locations first local cert_paths=( "${HOME}/.config/incus/client.crt" "${HOME}/snap/incus/common/config/client.crt" @@ -494,29 +484,48 @@ detect_client_cert() { for path in "${cert_paths[@]}"; do if [[ -f "$path" ]]; then CERT_CONTENT=$(<"$path") - success "Certificate loaded from ${path}" - return + if [[ "$CERT_CONTENT" == *"BEGIN CERTIFICATE"* ]]; then + success "Certificate loaded from ${path}" + return + fi + detail "File exists but does not contain a valid PEM certificate: ${path}" fi done - # Determine how serious the missing cert is + # 2) Try the incus CLI (only works with Incus >= 6.3) + if command -v incus &>/dev/null; then + local cert + if cert=$(incus remote get-client-certificate 2>/dev/null); then + CERT_CONTENT="$cert" + success "Certificate obtained via Incus CLI" + return + fi + # CLI is installed but cert retrieval failed -- check why + local incus_version + incus_version=$(incus version 2>/dev/null | grep -oP 'Client version: \K[0-9.]+' || echo "unknown") + detail "Incus CLI v${incus_version} found but no certificate available" + detail "Generate one with: incus remote list (triggers auto-generation)" + fi + + # 3) Nothing found -- report based on context if [[ "$APP" == "operations-center" ]]; then error "No client certificate found" error "Operations Center requires at least one trusted certificate for access" - error "" + echo "" >&2 error "Options:" - error " --cert FILE Provide a certificate file" - error " Install the Incus client: sudo apt install incus-client" - error " --no-cert Skip certificate (you will be locked out!)" + error " --cert FILE Provide a certificate file directly" + error " Generate one: incus remote list (creates ~/.config/incus/client.crt)" + error " --no-cert Skip certificate injection (you will be locked out!)" exit 1 fi warn "No client certificate found" - if ! command -v incus &>/dev/null; then + if command -v incus &>/dev/null; then + warn "Run 'incus remote list' to generate a client certificate, then re-run this script" + else warn "The Incus client is not installed on this machine" - warn "Install it to enable auto-detection: sudo apt install incus-client" + warn "Install it to enable auto-detection (see: https://github.com/lxc/incus)" fi - warn "The installed system will require manual certificate trust setup" warn "Use --cert FILE to provide a certificate, or --no-cert to suppress this warning" } diff --git a/incusos/incusos-seed b/incusos/incusos-seed index 9bc5b69..7d99148 100755 --- a/incusos/incusos-seed +++ b/incusos/incusos-seed @@ -324,21 +324,11 @@ detect_client_cert() { return fi - # Auto-detection + # Auto-detection: prefer reading cert files directly (works with all + # Incus versions including 6.0 LTS which lacks 'get-client-certificate'). step "Auto-detecting client certificate" - # Try incus CLI first - if command -v incus &>/dev/null; then - local cert - if cert=$(incus remote get-client-certificate 2>/dev/null); then - CERT_CONTENT="$cert" - success "Certificate obtained from Incus CLI" - return - fi - detail "Incus CLI available but could not retrieve certificate" - fi - - # Try common file locations + # 1) Check common on-disk certificate locations first local cert_paths=( "${HOME}/.config/incus/client.crt" "${HOME}/snap/incus/common/config/client.crt" @@ -348,29 +338,47 @@ detect_client_cert() { for path in "${cert_paths[@]}"; do if [[ -f "$path" ]]; then CERT_CONTENT=$(<"$path") - success "Certificate loaded from ${path}" - return + if [[ "$CERT_CONTENT" == *"BEGIN CERTIFICATE"* ]]; then + success "Certificate loaded from ${path}" + return + fi + detail "File exists but does not contain a valid PEM certificate: ${path}" fi done - # Determine how serious the missing cert is + # 2) Try the incus CLI (only works with Incus >= 6.3) + if command -v incus &>/dev/null; then + local cert + if cert=$(incus remote get-client-certificate 2>/dev/null); then + CERT_CONTENT="$cert" + success "Certificate obtained via Incus CLI" + return + fi + local incus_version + incus_version=$(incus version 2>/dev/null | grep -oP 'Client version: \K[0-9.]+' || echo "unknown") + detail "Incus CLI v${incus_version} found but no certificate available" + detail "Generate one with: incus remote list (triggers auto-generation)" + fi + + # 3) Nothing found -- report based on context if [[ "$APP" == "operations-center" ]]; then error "No client certificate found" error "Operations Center requires at least one trusted certificate for access" - error "" + echo "" >&2 error "Options:" - error " --cert FILE Provide a certificate file" - error " Install the Incus client: sudo apt install incus-client" - error " --no-cert Skip certificate (you will be locked out!)" + error " --cert FILE Provide a certificate file directly" + error " Generate one: incus remote list (creates ~/.config/incus/client.crt)" + error " --no-cert Skip certificate injection (you will be locked out!)" exit 1 fi warn "No client certificate found" - if ! command -v incus &>/dev/null; then + if command -v incus &>/dev/null; then + warn "Run 'incus remote list' to generate a client certificate, then re-run this script" + else warn "The Incus client is not installed on this machine" - warn "Install it to enable auto-detection: sudo apt install incus-client" + warn "Install it to enable auto-detection (see: https://github.com/lxc/incus)" fi - warn "The installed system will require manual certificate trust setup" warn "Use --cert FILE to provide a certificate, or --no-cert to suppress this warning" } diff --git a/notes/incus-version-compatibility.md b/notes/incus-version-compatibility.md new file mode 100644 index 0000000..2e86eb7 --- /dev/null +++ b/notes/incus-version-compatibility.md @@ -0,0 +1,164 @@ +# Incus Version Compatibility Notes + +## Package landscape + +| Platform | Package | Current version | Type | Notes | +|----------|---------|-----------------|------|-------| +| Debian stable (apt) | `incus` / `incus-client` | 6.0 LTS | Frozen | Security fixes only until June 2029 | +| Debian/Ubuntu (Zabbly) | `incus` / `incus-client` | 6.21 | Rolling | Official upstream repo; separate client-only package | +| macOS (Homebrew) | `incus` | 6.21 | Rolling | Client only by design (no daemon on macOS) | +| Arch Linux (extra) | `incus` | 6.21 | Rolling | Full daemon + client; no client-only split | +| From source (Go) | -- | latest | Manual | Build just the client binary | + +Incus releases monthly. LTS (6.0.x) gets bug + security fixes for 2 years, +then security-only for 3 more years after Incus 7.0 LTS ships. + +## CLI differences between 6.0 LTS and 6.3+ + +### Commands added after 6.0 + +| Command | Added in | Purpose | +|---------|----------|---------| +| `incus remote get-client-certificate` | 6.3 | Export client cert via CLI | +| `incus admin os system ...` | 6.x | IncusOS system management | + +### Client certificate locations (all versions) + +The client certificate and key are always stored on disk regardless of version: + +``` +~/.config/incus/client.crt # PEM certificate +~/.config/incus/client.key # Private key +``` + +On snap installations: +``` +~/snap/incus/common/config/client.crt +~/snap/incus/common/config/client.key +``` + +Legacy LXC path: +``` +~/.config/lxc/client.crt +~/.config/lxc/client.key +``` + +Running any remote-related command (e.g. `incus remote list`) triggers +auto-generation of the keypair if it doesn't exist yet. + +## Installing the Incus client + +These instructions cover installing the **client only** (the `incus` CLI +for managing remote servers). If you also need the daemon, see the +platform-specific notes below. + +### macOS (Homebrew) + +Homebrew provides the client only (macOS cannot run the daemon). + +```bash +brew install incus +``` + +Verify: + +```bash +incus version +``` + +Certificate location: `~/.config/incus/client.crt` + +### Debian / Ubuntu (Zabbly upstream repo) + +Distribution packages ship 6.0 LTS which is missing newer CLI commands. +Use the Zabbly repo for the latest version. + +**Remove distro package first** (if installed): + +```bash +sudo apt remove --purge incus incus-client +sudo apt autoremove +``` + +**Add the Zabbly repository:** + +```bash +sudo mkdir -p /etc/apt/keyrings + +curl -fsSL https://pkgs.zabbly.com/key.asc | \ + sudo gpg --dearmor -o /etc/apt/keyrings/zabbly.gpg + +echo "deb [signed-by=/etc/apt/keyrings/zabbly.gpg] \ + https://pkgs.zabbly.com/incus/stable \ + $(lsb_release -cs) main" | \ + sudo tee /etc/apt/sources.list.d/incus-stable.list +``` + +**Install client only:** + +```bash +sudo apt update +sudo apt install incus-client +``` + +Or install the full daemon + client with `sudo apt install incus`. + +Supported distributions: Debian 11+, Ubuntu 22.04+. + +### Arch Linux + +Arch ships the latest Incus in the `extra` repository, but only as a +combined daemon + client package. There is no client-only split. + +**Full package (daemon + client):** + +```bash +sudo pacman -S incus +``` + +**Client only from source** (if you don't want the daemon): + +```bash +go install github.com/lxc/incus/v6/cmd/incus@latest +``` + +This puts the binary in `$GOPATH/bin` (typically `~/go/bin`). + +### From source (any platform with Go) + +Build just the client binary directly: + +```bash +go install github.com/lxc/incus/v6/cmd/incus@latest +export PATH="${GOPATH:-$HOME/go}/bin:$PATH" +``` + +Requires Go 1.22+. + +### Verify installation + +```bash +incus version +# Client version: 6.21 +``` + +Generate a client certificate (if one doesn't exist yet): + +```bash +incus remote list +ls ~/.config/incus/client.crt +``` + +## Multi-machine certificate identity + +When managing IncusOS nodes from multiple machines, use the **same** +client certificate on all management machines. Copy the keypair: + +```bash +# From source machine to target machine +scp ~/.config/incus/client.crt ~/.config/incus/client.key user@target:~/.config/incus/ +``` + +This ensures all machines are recognized as the same trusted client. +Using different certificates means each must be individually trusted +on every managed node.