Go to file
Maarten 2e2447de06 Boot failure investigation: 67% crontab bug rate confirmed, version mismatch ruled out
Parallel investigation of IncusOS issue #843 across 20 VM deploys confirms
the "invalid crontab expression" error hits 67% of first boots. The bug is
a race condition in the daemon startup — NOT related to ISO/CDN version
mismatch, update downloads, or seed config. Sysext downloads occur on every
first boot even with matching versions.

New tools: investigate-parallel (parallel VM testing), pve-api (Python
Proxmox API helper avoiding bash ! quoting), investigate-boot (sequential
testing), test-boot-reliability (batch reliability testing).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 23:19:35 +01:00
incusos Boot failure investigation: 67% crontab bug rate confirmed, version mismatch ruled out 2026-02-21 23:19:35 +01:00
notes Boot failure investigation: 67% crontab bug rate confirmed, version mismatch ruled out 2026-02-21 23:19:35 +01:00
.gitignore Add observe-deploy diagnostics, boot failure analysis, and deploy reliability fixes 2026-02-21 13:42:51 +01:00
CLAUDE.md Boot failure investigation: 67% crontab bug rate confirmed, version mismatch ruled out 2026-02-21 23:19:35 +01:00
README.md Add macOS support, multi-lab lifecycle, extended testing, and migration docs 2026-02-20 23:41:06 +01:00

README.md

incu-contrib

Peripheral tools, snippets, and scripts for working with Incus, IncusOS, and the broader ecosystem.

Primarily aimed at home lab environments but useful in production as well.

Contents

incusos/

Scripts for building IncusOS installation media and seed configurations:

  • incusos-iso -- Build customized IncusOS ISO/IMG images using the official flasher-tool. Supports both architectures (x86_64, aarch64), multiple applications (Incus, Operations Center), automatic client certificate injection, and batch generation.

  • incusos-seed -- Generate IncusOS seed archives (tar or FAT image) for installation customization. Covers all seed file types: install, applications, Incus preseed, Operations Center, network, and update configuration.

  • incusos-proxmox -- Declarative IncusOS deployment on Proxmox VE. Define VMs in YAML, and the script handles seed generation, ISO upload, VM creation with correct settings (UEFI, TPM 2.0, VirtIO-SCSI), and automated installation.

See incusos/README.md for full documentation.

notes/

Research notes and reference material:

Quick Start

# Install the flasher-tool (requires Go 1.22+)
go install github.com/lxc/incus-os/incus-osd/cmd/flasher-tool@latest

# Build a default IncusOS ISO with your client cert injected
./incusos/incusos-iso

# Build for a specific architecture and application
./incusos/incusos-iso --arch aarch64 --app operations-center --defaults

# Generate all common ISO variants at once
./incusos/incusos-iso --batch

# Generate a standalone seed archive
./incusos/incusos-seed --app incus --defaults --output my-seed.tar

# Generate a SEED_DATA FAT image for external boot media
./incusos/incusos-seed --format fat --app incus --defaults --output seed.img

Requirements

  • Go 1.22+ -- for installing the flasher-tool
  • curl -- for downloading images from the CDN
  • tar, gzip -- for seed archive creation and image decompression
  • Incus client (recommended) -- for automatic client certificate detection. Works with any version; certificates are read directly from disk. See version compatibility notes for install instructions per platform (macOS, Debian/Ubuntu, Arch).
  • mtools (optional) -- for creating FAT seed images

License

MIT