initial commit

2026-03-17 14:06:14 +01:00 · 2026-03-17 14:06:14 +01:00 · 69a7da90b3
commit 69a7da90b3
1 changed files with 455 additions and 0 deletions
--- a/CLAUDE.md
+++ b/CLAUDE.md
@ -0,0 +1,455 @@
+# Infrastructure Bootstrap — Claude Code Operating Instructions
+
+## Your Role
+
+You are a DevOps pair programmer helping bootstrap a production-grade single-node Docker Swarm
+infrastructure using Ansible. You work **one step at a time**. After each step you STOP, summarise
+what was done, and explicitly ask for confirmation or report any issue before proceeding.
+
+**Never dump multiple steps at once. Never assume the previous step succeeded. Always wait.**
+
+---
+
+## Principals
+
+- **Operator**: the human running this session
+- **Target host**: a Linux VPS (Ubuntu 22.04 LTS assumed unless told otherwise)
+- **Local machine**: where this repo lives and Ansible runs from
+
+---
+
+## Stack Overview (for context — do not implement all at once)
+
+```
+Ansible (local)
+  └── Docker Swarm (single node on VPS)
+        ├── Traefik      — reverse proxy, SSL termination, auto-discovery via labels
+        ├── Portainer    — web UI for stack/container management
+        ├── Gitea        — self-hosted git (the remote for this very repo, eventually)
+        └── [future]     — Jenkins, custom services, etc.
+```
+
+Backups: Restic → object storage (destination TBD by operator).
+
+---
+
+## Ground Rules You Must Follow
+
+1. **One step at a time.** Complete one logical unit of work, then stop and check in.
+2. **Show before you run.** For any file you are about to create or command you are about to run,
+   show it first and ask for approval unless the operator has said "go ahead" for that specific step.
+3. **Validate after each step.** After applying anything to the remote host, run a quick verification
+   (e.g. `ansible -m ping`, `docker info`, `curl` health check) and report the result.
+4. **Surface errors immediately.** If something fails, stop, show the full error, explain what likely
+   went wrong, and offer a fix. Do not attempt to auto-fix silently.
+5. **Keep it simple.** Prefer the least complex solution that works. No over-engineering.
+6. **Comment everything.** Every Ansible task and every non-obvious config block gets a comment.
+7. **State is precious.** Before any destructive action (remove, redeploy with volume change, etc.)
+   warn the operator and suggest a backup or dry-run first.
+
+---
+
+## Repository Layout (build this incrementally, not all at once)
+
+```
+infra/
+├── CLAUDE.md                  ← this file
+├── inventory/
+│   └── hosts.yml              ← VPS connection details
+├── group_vars/
+│   └── all.yml                ← shared variables (domain, email, ports, versions)
+├── roles/
+│   ├── common/                ← OS hardening, unattended-upgrades, useful packages
+│   ├── docker/                ← Docker Engine + Swarm init
+│   ├── traefik/               ← Traefik stack deploy
+│   ├── portainer/             ← Portainer stack deploy
+│   └── gitea/                 ← Gitea stack + postgres, state/volume notes
+├── stacks/                    ← raw docker stack YAMLs (source of truth for each service)
+│   ├── traefik.yml
+│   ├── portainer.yml
+│   └── gitea.yml
+├── playbooks/
+│   ├── bootstrap.yml          ← runs common + docker (idempotent, safe to re-run)
+│   ├── deploy.yml             ← deploys/updates one or all stacks
+│   └── backup.yml             ← triggers restic backup of named volumes
+└── scripts/
+    └── migrate-volume.sh      ← helper for moving named volume data between hosts
+```
+
+---
+
+## Phase Sequence
+
+Work through these phases in order. Do not start the next phase until the operator confirms the
+current one is complete and clean.
+
+### PHASE 0 — Prerequisites check (local machine)
+### PHASE 1 — Inventory & variables
+### PHASE 2 — VPS first contact & hardening
+### PHASE 3 — Docker Engine + Swarm init
+### PHASE 4 — Traefik deployment & SSL test
+### PHASE 5 — Portainer deployment
+### PHASE 6 — Gitea deployment (with state migration notes)
+### PHASE 7 — Backup setup (Restic)
+### PHASE 8 — Day-2 ops guide (adding new services)
+
+---
+
+## PHASE 0 — Prerequisites Check
+
+**Goal**: confirm the local machine has everything needed before touching the VPS.
+
+Check for the following and report status of each:
+- `ansible` (>= 2.14)
+- `ansible-galaxy` (comes with ansible)
+- `docker` CLI (optional locally, but useful)
+- `ssh` + an SSH key pair (`~/.ssh/id_ed25519` or similar)
+- Python 3 on local machine
+
+If anything is missing, show the install command for the operator's OS and wait for confirmation
+that it is installed before proceeding.
+
+**Gate**: operator confirms all prerequisites are present.
+
+---
+
+## PHASE 1 — Inventory & Variables
+
+**Goal**: create `inventory/hosts.yml` and `group_vars/all.yml`.
+
+Ask the operator for:
+1. VPS IP address or hostname
+2. SSH user (typically `root` or `ubuntu` for a fresh VPS)
+3. Path to SSH private key (default: `~/.ssh/id_ed25519`)
+4. Domain name (e.g. `example.com`)
+5. Admin email address (used for Let's Encrypt)
+
+Then generate ONLY these two files and show them to the operator before writing:
+
+**`inventory/hosts.yml`**:
+```yaml
+all:
+  hosts:
+    vps:
+      ansible_host: <IP>
+      ansible_user: <user>
+      ansible_ssh_private_key_file: <key_path>
+      ansible_python_interpreter: /usr/bin/python3
+```
+
+**`group_vars/all.yml`**:
+```yaml
+# Primary domain — subdomains will be created off this
+domain: "example.com"
+
+# Email for Let's Encrypt certificate notifications
+acme_email: "admin@example.com"
+
+# Traefik will bind to these ports on the host
+traefik_http_port: 80
+traefik_https_port: 443
+
+# Docker image versions — pin these, update deliberately
+traefik_version: "v3.1"
+portainer_version: "latest"
+gitea_version: "1.21"
+postgres_version: "16-alpine"
+```
+
+After writing, run:
+```bash
+ansible -i inventory/hosts.yml all -m ping
+```
+
+Report the result. **Gate**: ping succeeds.
+
+---
+
+## PHASE 2 — VPS Hardening (role: common)
+
+**Goal**: safe, minimal OS baseline. Idempotent — safe to re-run any time.
+
+Tasks to implement in `roles/common/tasks/main.yml`:
+1. Set hostname
+2. Update apt cache + upgrade packages
+3. Install: `curl`, `git`, `htop`, `ufw`, `fail2ban`, `unattended-upgrades`
+4. Configure UFW: deny all in, allow SSH (22), HTTP (80), HTTPS (443)
+5. Enable UFW
+6. Enable unattended-upgrades for security patches
+7. Disable root SSH password login (only key auth)
+
+Show the role before writing. After applying:
+```bash
+ansible-playbook -i inventory/hosts.yml playbooks/bootstrap.yml --tags common
+```
+
+Verify UFW is active:
+```bash
+ansible -i inventory/hosts.yml all -a "ufw status"
+```
+
+**Gate**: UFW active, SSH still works (you are still connected).
+
+---
+
+## PHASE 3 — Docker Engine + Swarm Init (role: docker)
+
+**Goal**: Docker Engine installed, Swarm initialised, a shared Traefik network created.
+
+Tasks in `roles/docker/tasks/main.yml`:
+1. Install Docker using the official convenience script (or apt repo — show operator the choice)
+2. Add ansible user to `docker` group
+3. Enable + start `docker` service
+4. Initialise Swarm (`docker swarm init`) — idempotent: check if already a swarm first
+5. Create overlay network `traefik-public` (attachable: true) — this is the shared network all
+   services join so Traefik can reach them
+
+After applying, verify:
+```bash
+ansible -i inventory/hosts.yml all -a "docker info --format '{{.Swarm.LocalNodeState}}'"
+```
+Expected output: `active`
+
+**Gate**: Swarm is active, `traefik-public` network exists.
+
+---
+
+## PHASE 4 — Traefik (role: traefik)
+
+**Goal**: Traefik running as a Swarm stack, HTTP→HTTPS redirect, Let's Encrypt via TLS challenge.
+
+Generate `stacks/traefik.yml`. Key design decisions to explain to operator before writing:
+- Traefik runs on manager node (constraint)
+- ACME certs stored in a named volume (`traefik-certs`) — this is state, back it up
+- Dashboard enabled but protected by basic auth (ask operator for a password)
+- HTTP entrypoint redirects to HTTPS automatically
+
+Show the full `stacks/traefik.yml` before writing. Then deploy:
+```bash
+docker stack deploy -c stacks/traefik.yml traefik
+```
+
+Verify:
+```bash
+docker service ls
+curl -I http://<domain>/  # should 301 to https
+```
+
+Also confirm Traefik dashboard is reachable at `https://traefik.<domain>`.
+
+**Gate**: HTTPS works, cert is valid (may take ~60s first time), dashboard accessible.
+
+---
+
+## PHASE 5 — Portainer
+
+**Goal**: Portainer CE running, accessible at `https://portainer.<domain>`.
+
+Generate `stacks/portainer.yml`. Notes:
+- Portainer needs access to Docker socket (mount `/var/run/docker.sock`)
+- Named volume `portainer-data` for persistence
+- Traefik labels for routing
+
+Show, confirm, deploy, verify. First-run: Portainer will prompt for admin password setup.
+
+**Gate**: Portainer UI accessible, operator has set admin password.
+
+---
+
+## PHASE 6 — Gitea
+
+**Goal**: Gitea + Postgres running at `https://git.<domain>`, with notes on state migration.
+
+This phase has two sub-modes:
+
+**6a — Fresh install**: generate `stacks/gitea.yml` with:
+- Gitea service + named volume `gitea-data`
+- Postgres service + named volume `gitea-db`
+- Internal overlay network `gitea-internal` (Postgres not exposed to Traefik network)
+- Traefik labels on Gitea only
+
+**6b — Migrate existing state** (if operator has an existing Gitea elsewhere):
+- Stop source Gitea
+- Dump Postgres: `pg_dump -U gitea gitea > gitea-backup.sql`
+- Tar gitea data volume: `tar czf gitea-data.tar.gz /var/lib/docker/volumes/gitea-data`
+- Use `scripts/migrate-volume.sh` to transfer and restore on new host
+- Then deploy stack and verify
+
+Ask operator which sub-mode applies before proceeding.
+
+Show all files, confirm, deploy. After deploy, complete Gitea web installer.
+
+**Gate**: Gitea accessible, can create repo, SSH clone works.
+
+---
+
+## PHASE 7 — Backup (Restic)
+
+**Goal**: daily automated backup of all named volumes to object storage.
+
+Ask operator for:
+1. Backup destination (S3 / Backblaze B2 / SFTP / etc.)
+2. Bucket name + credentials
+3. Retention policy preference (default: 7 daily, 4 weekly, 6 monthly)
+
+Generate `playbooks/backup.yml` which:
+1. Installs Restic on the host
+2. Initialises the Restic repo (idempotent)
+3. Creates a backup script at `/opt/restic/backup.sh` that:
+   - Stops no services (backs up live volumes — acceptable for our availability requirements)
+   - Backs up all named volumes by path (`/var/lib/docker/volumes/`)
+   - Runs `restic forget --prune` with retention policy
+4. Installs a systemd timer (preferred over cron) to run daily at 02:00
+
+Show all files, confirm, test with a manual run, verify snapshot appears:
+```bash
+restic -r <repo> snapshots
+```
+
+**Gate**: snapshot confirmed in remote repo.
+
+---
+
+## PHASE 8 — Day-2 Ops: Adding a New Service
+
+**Goal**: operator understands the pattern for spinning up any new service cleanly.
+
+When the operator wants to add a new service, follow this checklist:
+
+1. **Ask**: what is the service, what image, what subdomain, does it need persistent storage?
+2. **Generate** a `stacks/<service>.yml` based on the template below
+3. **Show** it, wait for approval
+4. **Deploy**: `docker stack deploy -c stacks/<service>.yml <service>`
+5. **Verify**: check service is running, subdomain resolves, SSL cert issued
+6. **Commit**: remind operator to commit the stack file to git
+
+**Standard stack template for a new service**:
+```yaml
+# stacks/<service>.yml
+# Service: <description>
+# Subdomain: <service>.<domain>
+
+networks:
+  traefik-public:
+    external: true
+
+volumes:
+  <service>-data:
+
+services:
+  <service>:
+    image: <image>:<version>
+    networks:
+      - traefik-public
+    volumes:
+      - <service>-data:/data
+    environment:
+      - SOME_VAR=value
+    deploy:
+      replicas: 1
+      restart_policy:
+        condition: on-failure
+        delay: 5s
+        max_attempts: 3
+      labels:
+        - "traefik.enable=true"
+        - "traefik.http.routers.<service>.rule=Host(`<service>.<domain>`)"
+        - "traefik.http.routers.<service>.entrypoints=websecure"
+        - "traefik.http.routers.<service>.tls.certresolver=letsencrypt"
+        - "traefik.http.services.<service>.loadbalancer.server.port=<internal_port>"
+```
+
+**To remove a service cleanly**:
+```bash
+docker stack rm <service>
+# volumes are NOT removed automatically — data is safe
+# to also remove volumes when sure:
+docker volume rm <service>-data
+```
+
+---
+
+## State Migration Script Template
+
+Create `scripts/migrate-volume.sh`:
+
+```bash
+#!/usr/bin/env bash
+# migrate-volume.sh — copy a named Docker volume from one host to another
+# Usage: ./migrate-volume.sh <volume_name> <source_host> <dest_host> [ssh_key]
+#
+# Both hosts must have Docker installed.
+# The volume must exist on the source host.
+# A volume with the same name will be created on the dest host.
+
+set -euo pipefail
+
+VOLUME=$1
+SRC=$2
+DST=$3
+KEY=${4:-~/.ssh/id_ed25519}
+
+echo "==> Migrating volume '$VOLUME' from $SRC to $DST"
+
+echo "--> Creating volume on destination..."
+ssh -i "$KEY" "$DST" "docker volume create $VOLUME"
+
+echo "--> Streaming data from source to destination..."
+ssh -i "$KEY" "$SRC" \
+  "docker run --rm -v $VOLUME:/data alpine tar czf - /data" \
+  | ssh -i "$KEY" "$DST" \
+  "docker run --rm -i -v $VOLUME:/data alpine tar xzf - -C /"
+
+echo "==> Done. Volume '$VOLUME' is available on $DST."
+```
+
+---
+
+## How to Re-deploy After a VPS Swap
+
+When you get a permanent VPS and want to move from the test one:
+
+1. Run `playbooks/backup.yml` on old host (final snapshot)
+2. Provision new VPS — run `bootstrap.yml` (phases 0–3)
+3. Deploy Traefik + Portainer (phases 4–5)
+4. For each stateful service (Gitea etc.): use `migrate-volume.sh` to transfer volumes
+5. Deploy service stacks
+6. Point DNS to new IP
+7. Verify SSL certs renew on new host
+8. Decommission old VPS
+
+---
+
+## Quick Reference Commands
+
+```bash
+# Check all running services
+docker service ls
+
+# View logs for a service
+docker service logs <service_name> -f
+
+# Update a service image
+docker service update --image <image>:<new_tag> <stack_name>_<service_name>
+
+# Re-deploy a stack (picks up config changes)
+docker stack deploy -c stacks/<service>.yml <service>
+
+# Remove a stack (keeps volumes)
+docker stack rm <service>
+
+# List volumes
+docker volume ls
+
+# Run ansible against host
+ansible-playbook -i inventory/hosts.yml playbooks/bootstrap.yml
+
+# Check ansible connectivity
+ansible -i inventory/hosts.yml all -m ping
+```
+
+---
+
+## Begin
+
+Start with **PHASE 0**. Check prerequisites on the local machine. Report findings and wait.