infra setup
This commit is contained in:
parent
69a7da90b3
commit
72da841c2b
498
CLAUDE.md
498
CLAUDE.md
|
|
@ -1,8 +1,8 @@
|
||||||
# Infrastructure Bootstrap — Claude Code Operating Instructions
|
# Infrastructure — Claude Code Operating Instructions
|
||||||
|
|
||||||
## Your Role
|
## Your Role
|
||||||
|
|
||||||
You are a DevOps pair programmer helping bootstrap a production-grade single-node Docker Swarm
|
You are a DevOps pair programmer helping maintain and extend a production single-node Docker Compose
|
||||||
infrastructure using Ansible. You work **one step at a time**. After each step you STOP, summarise
|
infrastructure using Ansible. You work **one step at a time**. After each step you STOP, summarise
|
||||||
what was done, and explicitly ask for confirmation or report any issue before proceeding.
|
what was done, and explicitly ask for confirmation or report any issue before proceeding.
|
||||||
|
|
||||||
|
|
@ -13,443 +13,173 @@ what was done, and explicitly ask for confirmation or report any issue before pr
|
||||||
## Principals
|
## Principals
|
||||||
|
|
||||||
- **Operator**: the human running this session
|
- **Operator**: the human running this session
|
||||||
- **Target host**: a Linux VPS (Ubuntu 22.04 LTS assumed unless told otherwise)
|
- **Target host**: Debian 13 (Trixie) VPS on DigitalOcean, hostname: `box.cloud-elves.eu` (188.166.60.12)
|
||||||
- **Local machine**: where this repo lives and Ansible runs from
|
- **Local machine**: macOS, where this repo lives and Ansible runs from
|
||||||
|
- **SSH key**: `~/.ssh/id_ed25519_ce`
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Stack Overview (for context — do not implement all at once)
|
## Stack Overview
|
||||||
|
|
||||||
```
|
```
|
||||||
Ansible (local)
|
Ansible (local machine)
|
||||||
└── Docker Swarm (single node on VPS)
|
└── Docker Compose (single node on box.cloud-elves.eu)
|
||||||
├── Traefik — reverse proxy, SSL termination, auto-discovery via labels
|
├── Caddy — reverse proxy, automatic HTTPS via Let's Encrypt
|
||||||
├── Portainer — web UI for stack/container management
|
├── LLDAP — lightweight LDAP user directory (ldap.cloud-elves.eu)
|
||||||
├── Gitea — self-hosted git (the remote for this very repo, eventually)
|
├── Dex — OIDC provider, backed by LLDAP (auth.cloud-elves.eu)
|
||||||
└── [future] — Jenkins, custom services, etc.
|
├── Gitea — git hosting (git.cloud-elves.eu), SSH on port 2222
|
||||||
|
├── Outline — wiki/docs (docs.cloud-elves.eu)
|
||||||
|
└── Static site — cloud-elves.eu / www.cloud-elves.eu (rsync deploy)
|
||||||
```
|
```
|
||||||
|
|
||||||
Backups: Restic → object storage (destination TBD by operator).
|
Supporting containers (internal only): Gitea Postgres, Outline Postgres, Outline Redis.
|
||||||
|
|
||||||
|
Backups: not yet configured (Restic → object storage planned).
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Ground Rules You Must Follow
|
## Ground Rules You Must Follow
|
||||||
|
|
||||||
1. **One step at a time.** Complete one logical unit of work, then stop and check in.
|
1. **One step at a time.** Complete one logical unit of work, then stop and check in.
|
||||||
2. **Show before you run.** For any file you are about to create or command you are about to run,
|
2. **Research before writing.** Verify image versions, OS compatibility, and config options against
|
||||||
show it first and ask for approval unless the operator has said "go ahead" for that specific step.
|
current documentation. Do not guess at configuration.
|
||||||
3. **Validate after each step.** After applying anything to the remote host, run a quick verification
|
3. **Everything through Ansible.** No manual SSH fixes, no ad-hoc commands. If it needs to happen
|
||||||
(e.g. `ansible -m ping`, `docker info`, `curl` health check) and report the result.
|
on the server, it goes in a playbook or role.
|
||||||
4. **Surface errors immediately.** If something fails, stop, show the full error, explain what likely
|
4. **Validate after each step.** After applying anything, run a verification and report the result.
|
||||||
went wrong, and offer a fix. Do not attempt to auto-fix silently.
|
5. **Surface errors immediately.** If something fails, stop, show the error, explain, and offer a fix.
|
||||||
5. **Keep it simple.** Prefer the least complex solution that works. No over-engineering.
|
6. **Keep it simple.** Prefer the least complex solution. We moved away from Docker Swarm + Traefik
|
||||||
6. **Comment everything.** Every Ansible task and every non-obvious config block gets a comment.
|
to Docker Compose + Caddy for exactly this reason.
|
||||||
7. **State is precious.** Before any destructive action (remove, redeploy with volume change, etc.)
|
7. **State is precious.** Before any destructive action, warn the operator and suggest a backup first.
|
||||||
warn the operator and suggest a backup or dry-run first.
|
8. **Comment everything.** Every Ansible task and every non-obvious config block gets a comment.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Repository Layout (build this incrementally, not all at once)
|
## Repository Layout
|
||||||
|
|
||||||
```
|
```
|
||||||
infra/
|
ce-infra/
|
||||||
├── CLAUDE.md ← this file
|
├── CLAUDE.md ← this file
|
||||||
|
├── ansible.cfg ← sets roles_path and default inventory
|
||||||
|
├── credentials.md ← all service credentials (DO NOT commit to public repo)
|
||||||
|
├── docs/ ← infrastructure documentation
|
||||||
|
│ ├── overview.md ← architecture and service overview
|
||||||
|
│ ├── runbook.md ← operational procedures
|
||||||
|
│ └── future-plans.md ← planned work and improvements
|
||||||
├── inventory/
|
├── inventory/
|
||||||
│ └── hosts.yml ← VPS connection details
|
│ ├── hosts.yml ← VPS connection details
|
||||||
├── group_vars/
|
│ └── group_vars/
|
||||||
│ └── all.yml ← shared variables (domain, email, ports, versions)
|
│ └── all.yml ← all variables (domain, secrets, versions)
|
||||||
├── roles/
|
├── roles/
|
||||||
│ ├── common/ ← OS hardening, unattended-upgrades, useful packages
|
│ ├── common/ ← OS hardening, packages, UFW, cloud-init disable
|
||||||
│ ├── docker/ ← Docker Engine + Swarm init
|
│ ├── docker/ ← Docker Engine install
|
||||||
│ ├── traefik/ ← Traefik stack deploy
|
│ └── services/ ← Docker Compose stack (all services)
|
||||||
│ ├── portainer/ ← Portainer stack deploy
|
│ ├── tasks/main.yml
|
||||||
│ └── gitea/ ← Gitea stack + postgres, state/volume notes
|
│ └── templates/
|
||||||
├── stacks/ ← raw docker stack YAMLs (source of truth for each service)
|
│ ├── docker-compose.yml.j2 ← all containers
|
||||||
│ ├── traefik.yml
|
│ ├── Caddyfile.j2 ← reverse proxy config
|
||||||
│ ├── portainer.yml
|
│ └── dex-config.yml.j2 ← Dex OIDC config
|
||||||
│ └── gitea.yml
|
└── playbooks/
|
||||||
├── playbooks/
|
├── bootstrap.yml ← runs common + docker (idempotent, safe to re-run)
|
||||||
│ ├── bootstrap.yml ← runs common + docker (idempotent, safe to re-run)
|
└── deploy.yml ← deploys/updates all services
|
||||||
│ ├── deploy.yml ← deploys/updates one or all stacks
|
|
||||||
│ └── backup.yml ← triggers restic backup of named volumes
|
|
||||||
└── scripts/
|
|
||||||
└── migrate-volume.sh ← helper for moving named volume data between hosts
|
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Phase Sequence
|
## Key Technical Decisions
|
||||||
|
|
||||||
Work through these phases in order. Do not start the next phase until the operator confirms the
|
| Decision | Rationale |
|
||||||
current one is complete and clean.
|
|---|---|
|
||||||
|
| Docker Compose over Swarm | Single node — Swarm adds overlay network complexity for zero benefit |
|
||||||
### PHASE 0 — Prerequisites check (local machine)
|
| Caddy over Traefik | Automatic HTTPS with zero config, simple Caddyfile, no label machinery |
|
||||||
### PHASE 1 — Inventory & variables
|
| LLDAP + Dex over Keycloak/Authentik | Lightweight identity stack, minimal resource usage on 2GB RAM |
|
||||||
### PHASE 2 — VPS first contact & hardening
|
| Local file storage for Outline | No S3 dependency, simpler backup story |
|
||||||
### PHASE 3 — Docker Engine + Swarm init
|
| `server_hostname` variable name | `hostname` collides with Ansible built-in |
|
||||||
### PHASE 4 — Traefik deployment & SSL test
|
| Cloud-init disabled on first run | Prevents SSH key regeneration on hostname change (DigitalOcean) |
|
||||||
### PHASE 5 — Portainer deployment
|
| `group_vars/` inside `inventory/` | Ansible auto-discovers it there without extra config |
|
||||||
### PHASE 6 — Gitea deployment (with state migration notes)
|
|
||||||
### PHASE 7 — Backup setup (Restic)
|
|
||||||
### PHASE 8 — Day-2 ops guide (adding new services)
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## PHASE 0 — Prerequisites Check
|
## Authentication Flow
|
||||||
|
|
||||||
**Goal**: confirm the local machine has everything needed before touching the VPS.
|
```
|
||||||
|
User → Outline / Gitea → Dex (OIDC) → LLDAP (user directory)
|
||||||
|
```
|
||||||
|
|
||||||
Check for the following and report status of each:
|
- Manage users in LLDAP web UI (ldap.cloud-elves.eu)
|
||||||
- `ansible` (>= 2.14)
|
- All services authenticate via Dex OIDC
|
||||||
- `ansible-galaxy` (comes with ansible)
|
- Gitea has auto-registration enabled — first SSO login creates the account
|
||||||
- `docker` CLI (optional locally, but useful)
|
- Outline creates accounts on first SSO login automatically
|
||||||
- `ssh` + an SSH key pair (`~/.ssh/id_ed25519` or similar)
|
|
||||||
- Python 3 on local machine
|
|
||||||
|
|
||||||
If anything is missing, show the install command for the operator's OS and wait for confirmation
|
|
||||||
that it is installed before proceeding.
|
|
||||||
|
|
||||||
**Gate**: operator confirms all prerequisites are present.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## PHASE 1 — Inventory & Variables
|
## DNS Records
|
||||||
|
|
||||||
**Goal**: create `inventory/hosts.yml` and `group_vars/all.yml`.
|
All subdomains are CNAMEs pointing to `box.cloud-elves.eu`:
|
||||||
|
|
||||||
Ask the operator for:
|
| Subdomain | Service |
|
||||||
1. VPS IP address or hostname
|
|---|---|
|
||||||
2. SSH user (typically `root` or `ubuntu` for a fresh VPS)
|
| `box.cloud-elves.eu` | A record → 188.166.60.12 |
|
||||||
3. Path to SSH private key (default: `~/.ssh/id_ed25519`)
|
| `cloud-elves.eu` | A record → 188.166.60.12 |
|
||||||
4. Domain name (e.g. `example.com`)
|
| `www` | Static site |
|
||||||
5. Admin email address (used for Let's Encrypt)
|
| `git` | Gitea |
|
||||||
|
| `auth` | Dex OIDC |
|
||||||
Then generate ONLY these two files and show them to the operator before writing:
|
| `ldap` | LLDAP web UI |
|
||||||
|
| `docs` | Outline wiki |
|
||||||
**`inventory/hosts.yml`**:
|
|
||||||
```yaml
|
|
||||||
all:
|
|
||||||
hosts:
|
|
||||||
vps:
|
|
||||||
ansible_host: <IP>
|
|
||||||
ansible_user: <user>
|
|
||||||
ansible_ssh_private_key_file: <key_path>
|
|
||||||
ansible_python_interpreter: /usr/bin/python3
|
|
||||||
```
|
|
||||||
|
|
||||||
**`group_vars/all.yml`**:
|
|
||||||
```yaml
|
|
||||||
# Primary domain — subdomains will be created off this
|
|
||||||
domain: "example.com"
|
|
||||||
|
|
||||||
# Email for Let's Encrypt certificate notifications
|
|
||||||
acme_email: "admin@example.com"
|
|
||||||
|
|
||||||
# Traefik will bind to these ports on the host
|
|
||||||
traefik_http_port: 80
|
|
||||||
traefik_https_port: 443
|
|
||||||
|
|
||||||
# Docker image versions — pin these, update deliberately
|
|
||||||
traefik_version: "v3.1"
|
|
||||||
portainer_version: "latest"
|
|
||||||
gitea_version: "1.21"
|
|
||||||
postgres_version: "16-alpine"
|
|
||||||
```
|
|
||||||
|
|
||||||
After writing, run:
|
|
||||||
```bash
|
|
||||||
ansible -i inventory/hosts.yml all -m ping
|
|
||||||
```
|
|
||||||
|
|
||||||
Report the result. **Gate**: ping succeeds.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## PHASE 2 — VPS Hardening (role: common)
|
## Deployment
|
||||||
|
|
||||||
**Goal**: safe, minimal OS baseline. Idempotent — safe to re-run any time.
|
From a **fresh droplet** (Debian 13, SSH key injected):
|
||||||
|
|
||||||
Tasks to implement in `roles/common/tasks/main.yml`:
|
|
||||||
1. Set hostname
|
|
||||||
2. Update apt cache + upgrade packages
|
|
||||||
3. Install: `curl`, `git`, `htop`, `ufw`, `fail2ban`, `unattended-upgrades`
|
|
||||||
4. Configure UFW: deny all in, allow SSH (22), HTTP (80), HTTPS (443)
|
|
||||||
5. Enable UFW
|
|
||||||
6. Enable unattended-upgrades for security patches
|
|
||||||
7. Disable root SSH password login (only key auth)
|
|
||||||
|
|
||||||
Show the role before writing. After applying:
|
|
||||||
```bash
|
|
||||||
ansible-playbook -i inventory/hosts.yml playbooks/bootstrap.yml --tags common
|
|
||||||
```
|
|
||||||
|
|
||||||
Verify UFW is active:
|
|
||||||
```bash
|
|
||||||
ansible -i inventory/hosts.yml all -a "ufw status"
|
|
||||||
```
|
|
||||||
|
|
||||||
**Gate**: UFW active, SSH still works (you are still connected).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## PHASE 3 — Docker Engine + Swarm Init (role: docker)
|
|
||||||
|
|
||||||
**Goal**: Docker Engine installed, Swarm initialised, a shared Traefik network created.
|
|
||||||
|
|
||||||
Tasks in `roles/docker/tasks/main.yml`:
|
|
||||||
1. Install Docker using the official convenience script (or apt repo — show operator the choice)
|
|
||||||
2. Add ansible user to `docker` group
|
|
||||||
3. Enable + start `docker` service
|
|
||||||
4. Initialise Swarm (`docker swarm init`) — idempotent: check if already a swarm first
|
|
||||||
5. Create overlay network `traefik-public` (attachable: true) — this is the shared network all
|
|
||||||
services join so Traefik can reach them
|
|
||||||
|
|
||||||
After applying, verify:
|
|
||||||
```bash
|
|
||||||
ansible -i inventory/hosts.yml all -a "docker info --format '{{.Swarm.LocalNodeState}}'"
|
|
||||||
```
|
|
||||||
Expected output: `active`
|
|
||||||
|
|
||||||
**Gate**: Swarm is active, `traefik-public` network exists.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## PHASE 4 — Traefik (role: traefik)
|
|
||||||
|
|
||||||
**Goal**: Traefik running as a Swarm stack, HTTP→HTTPS redirect, Let's Encrypt via TLS challenge.
|
|
||||||
|
|
||||||
Generate `stacks/traefik.yml`. Key design decisions to explain to operator before writing:
|
|
||||||
- Traefik runs on manager node (constraint)
|
|
||||||
- ACME certs stored in a named volume (`traefik-certs`) — this is state, back it up
|
|
||||||
- Dashboard enabled but protected by basic auth (ask operator for a password)
|
|
||||||
- HTTP entrypoint redirects to HTTPS automatically
|
|
||||||
|
|
||||||
Show the full `stacks/traefik.yml` before writing. Then deploy:
|
|
||||||
```bash
|
|
||||||
docker stack deploy -c stacks/traefik.yml traefik
|
|
||||||
```
|
|
||||||
|
|
||||||
Verify:
|
|
||||||
```bash
|
|
||||||
docker service ls
|
|
||||||
curl -I http://<domain>/ # should 301 to https
|
|
||||||
```
|
|
||||||
|
|
||||||
Also confirm Traefik dashboard is reachable at `https://traefik.<domain>`.
|
|
||||||
|
|
||||||
**Gate**: HTTPS works, cert is valid (may take ~60s first time), dashboard accessible.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## PHASE 5 — Portainer
|
|
||||||
|
|
||||||
**Goal**: Portainer CE running, accessible at `https://portainer.<domain>`.
|
|
||||||
|
|
||||||
Generate `stacks/portainer.yml`. Notes:
|
|
||||||
- Portainer needs access to Docker socket (mount `/var/run/docker.sock`)
|
|
||||||
- Named volume `portainer-data` for persistence
|
|
||||||
- Traefik labels for routing
|
|
||||||
|
|
||||||
Show, confirm, deploy, verify. First-run: Portainer will prompt for admin password setup.
|
|
||||||
|
|
||||||
**Gate**: Portainer UI accessible, operator has set admin password.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## PHASE 6 — Gitea
|
|
||||||
|
|
||||||
**Goal**: Gitea + Postgres running at `https://git.<domain>`, with notes on state migration.
|
|
||||||
|
|
||||||
This phase has two sub-modes:
|
|
||||||
|
|
||||||
**6a — Fresh install**: generate `stacks/gitea.yml` with:
|
|
||||||
- Gitea service + named volume `gitea-data`
|
|
||||||
- Postgres service + named volume `gitea-db`
|
|
||||||
- Internal overlay network `gitea-internal` (Postgres not exposed to Traefik network)
|
|
||||||
- Traefik labels on Gitea only
|
|
||||||
|
|
||||||
**6b — Migrate existing state** (if operator has an existing Gitea elsewhere):
|
|
||||||
- Stop source Gitea
|
|
||||||
- Dump Postgres: `pg_dump -U gitea gitea > gitea-backup.sql`
|
|
||||||
- Tar gitea data volume: `tar czf gitea-data.tar.gz /var/lib/docker/volumes/gitea-data`
|
|
||||||
- Use `scripts/migrate-volume.sh` to transfer and restore on new host
|
|
||||||
- Then deploy stack and verify
|
|
||||||
|
|
||||||
Ask operator which sub-mode applies before proceeding.
|
|
||||||
|
|
||||||
Show all files, confirm, deploy. After deploy, complete Gitea web installer.
|
|
||||||
|
|
||||||
**Gate**: Gitea accessible, can create repo, SSH clone works.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## PHASE 7 — Backup (Restic)
|
|
||||||
|
|
||||||
**Goal**: daily automated backup of all named volumes to object storage.
|
|
||||||
|
|
||||||
Ask operator for:
|
|
||||||
1. Backup destination (S3 / Backblaze B2 / SFTP / etc.)
|
|
||||||
2. Bucket name + credentials
|
|
||||||
3. Retention policy preference (default: 7 daily, 4 weekly, 6 monthly)
|
|
||||||
|
|
||||||
Generate `playbooks/backup.yml` which:
|
|
||||||
1. Installs Restic on the host
|
|
||||||
2. Initialises the Restic repo (idempotent)
|
|
||||||
3. Creates a backup script at `/opt/restic/backup.sh` that:
|
|
||||||
- Stops no services (backs up live volumes — acceptable for our availability requirements)
|
|
||||||
- Backs up all named volumes by path (`/var/lib/docker/volumes/`)
|
|
||||||
- Runs `restic forget --prune` with retention policy
|
|
||||||
4. Installs a systemd timer (preferred over cron) to run daily at 02:00
|
|
||||||
|
|
||||||
Show all files, confirm, test with a manual run, verify snapshot appears:
|
|
||||||
```bash
|
|
||||||
restic -r <repo> snapshots
|
|
||||||
```
|
|
||||||
|
|
||||||
**Gate**: snapshot confirmed in remote repo.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## PHASE 8 — Day-2 Ops: Adding a New Service
|
|
||||||
|
|
||||||
**Goal**: operator understands the pattern for spinning up any new service cleanly.
|
|
||||||
|
|
||||||
When the operator wants to add a new service, follow this checklist:
|
|
||||||
|
|
||||||
1. **Ask**: what is the service, what image, what subdomain, does it need persistent storage?
|
|
||||||
2. **Generate** a `stacks/<service>.yml` based on the template below
|
|
||||||
3. **Show** it, wait for approval
|
|
||||||
4. **Deploy**: `docker stack deploy -c stacks/<service>.yml <service>`
|
|
||||||
5. **Verify**: check service is running, subdomain resolves, SSL cert issued
|
|
||||||
6. **Commit**: remind operator to commit the stack file to git
|
|
||||||
|
|
||||||
**Standard stack template for a new service**:
|
|
||||||
```yaml
|
|
||||||
# stacks/<service>.yml
|
|
||||||
# Service: <description>
|
|
||||||
# Subdomain: <service>.<domain>
|
|
||||||
|
|
||||||
networks:
|
|
||||||
traefik-public:
|
|
||||||
external: true
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
<service>-data:
|
|
||||||
|
|
||||||
services:
|
|
||||||
<service>:
|
|
||||||
image: <image>:<version>
|
|
||||||
networks:
|
|
||||||
- traefik-public
|
|
||||||
volumes:
|
|
||||||
- <service>-data:/data
|
|
||||||
environment:
|
|
||||||
- SOME_VAR=value
|
|
||||||
deploy:
|
|
||||||
replicas: 1
|
|
||||||
restart_policy:
|
|
||||||
condition: on-failure
|
|
||||||
delay: 5s
|
|
||||||
max_attempts: 3
|
|
||||||
labels:
|
|
||||||
- "traefik.enable=true"
|
|
||||||
- "traefik.http.routers.<service>.rule=Host(`<service>.<domain>`)"
|
|
||||||
- "traefik.http.routers.<service>.entrypoints=websecure"
|
|
||||||
- "traefik.http.routers.<service>.tls.certresolver=letsencrypt"
|
|
||||||
- "traefik.http.services.<service>.loadbalancer.server.port=<internal_port>"
|
|
||||||
```
|
|
||||||
|
|
||||||
**To remove a service cleanly**:
|
|
||||||
```bash
|
|
||||||
docker stack rm <service>
|
|
||||||
# volumes are NOT removed automatically — data is safe
|
|
||||||
# to also remove volumes when sure:
|
|
||||||
docker volume rm <service>-data
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## State Migration Script Template
|
|
||||||
|
|
||||||
Create `scripts/migrate-volume.sh`:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
#!/usr/bin/env bash
|
# 1. Accept new host keys
|
||||||
# migrate-volume.sh — copy a named Docker volume from one host to another
|
ssh-keygen -R box.cloud-elves.eu
|
||||||
# Usage: ./migrate-volume.sh <volume_name> <source_host> <dest_host> [ssh_key]
|
ssh-keyscan box.cloud-elves.eu >> ~/.ssh/known_hosts
|
||||||
#
|
|
||||||
# Both hosts must have Docker installed.
|
|
||||||
# The volume must exist on the source host.
|
|
||||||
# A volume with the same name will be created on the dest host.
|
|
||||||
|
|
||||||
set -euo pipefail
|
# 2. Bootstrap (OS hardening, Docker install)
|
||||||
|
ansible-playbook -i inventory/hosts.yml playbooks/bootstrap.yml
|
||||||
|
|
||||||
VOLUME=$1
|
# 3. Deploy all services
|
||||||
SRC=$2
|
ansible-playbook -i inventory/hosts.yml playbooks/deploy.yml
|
||||||
DST=$3
|
|
||||||
KEY=${4:-~/.ssh/id_ed25519}
|
|
||||||
|
|
||||||
echo "==> Migrating volume '$VOLUME' from $SRC to $DST"
|
|
||||||
|
|
||||||
echo "--> Creating volume on destination..."
|
|
||||||
ssh -i "$KEY" "$DST" "docker volume create $VOLUME"
|
|
||||||
|
|
||||||
echo "--> Streaming data from source to destination..."
|
|
||||||
ssh -i "$KEY" "$SRC" \
|
|
||||||
"docker run --rm -v $VOLUME:/data alpine tar czf - /data" \
|
|
||||||
| ssh -i "$KEY" "$DST" \
|
|
||||||
"docker run --rm -i -v $VOLUME:/data alpine tar xzf - -C /"
|
|
||||||
|
|
||||||
echo "==> Done. Volume '$VOLUME' is available on $DST."
|
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
To **update services** after config changes:
|
||||||
|
|
||||||
## How to Re-deploy After a VPS Swap
|
```bash
|
||||||
|
ansible-playbook -i inventory/hosts.yml playbooks/deploy.yml
|
||||||
When you get a permanent VPS and want to move from the test one:
|
```
|
||||||
|
|
||||||
1. Run `playbooks/backup.yml` on old host (final snapshot)
|
|
||||||
2. Provision new VPS — run `bootstrap.yml` (phases 0–3)
|
|
||||||
3. Deploy Traefik + Portainer (phases 4–5)
|
|
||||||
4. For each stateful service (Gitea etc.): use `migrate-volume.sh` to transfer volumes
|
|
||||||
5. Deploy service stacks
|
|
||||||
6. Point DNS to new IP
|
|
||||||
7. Verify SSL certs renew on new host
|
|
||||||
8. Decommission old VPS
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Quick Reference Commands
|
## Quick Reference Commands
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Check all running services
|
# Check Ansible connectivity
|
||||||
docker service ls
|
|
||||||
|
|
||||||
# View logs for a service
|
|
||||||
docker service logs <service_name> -f
|
|
||||||
|
|
||||||
# Update a service image
|
|
||||||
docker service update --image <image>:<new_tag> <stack_name>_<service_name>
|
|
||||||
|
|
||||||
# Re-deploy a stack (picks up config changes)
|
|
||||||
docker stack deploy -c stacks/<service>.yml <service>
|
|
||||||
|
|
||||||
# Remove a stack (keeps volumes)
|
|
||||||
docker stack rm <service>
|
|
||||||
|
|
||||||
# List volumes
|
|
||||||
docker volume ls
|
|
||||||
|
|
||||||
# Run ansible against host
|
|
||||||
ansible-playbook -i inventory/hosts.yml playbooks/bootstrap.yml
|
|
||||||
|
|
||||||
# Check ansible connectivity
|
|
||||||
ansible -i inventory/hosts.yml all -m ping
|
ansible -i inventory/hosts.yml all -m ping
|
||||||
|
|
||||||
|
# Check running containers on box
|
||||||
|
ansible -i inventory/hosts.yml all -m ansible.builtin.shell \
|
||||||
|
-a "cd /opt/services && docker compose ps"
|
||||||
|
|
||||||
|
# View logs for a specific service
|
||||||
|
ansible -i inventory/hosts.yml all -m ansible.builtin.shell \
|
||||||
|
-a "cd /opt/services && docker compose logs <service> --tail 50"
|
||||||
|
|
||||||
|
# Deploy static site
|
||||||
|
rsync -avz --delete site/ root@box.cloud-elves.eu:/opt/www/
|
||||||
|
|
||||||
|
# Restart a single service
|
||||||
|
ansible -i inventory/hosts.yml all -m ansible.builtin.shell \
|
||||||
|
-a "cd /opt/services && docker compose restart <service>"
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Begin
|
## Future Work
|
||||||
|
|
||||||
Start with **PHASE 0**. Check prerequisites on the local machine. Report findings and wait.
|
- **Backup setup** — Restic to object storage, systemd timer, retention policy
|
||||||
|
- **Security review** — firewall audit, container hardening, secret management (move secrets out of group_vars)
|
||||||
|
- **CI/CD** — automated deployments, documentation pipeline
|
||||||
|
- **Migration plan** — move to permanent VPS once company incorporation is complete
|
||||||
|
- **Portainer** — optional web UI for container management (low priority)
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,3 @@
|
||||||
|
[defaults]
|
||||||
|
roles_path = roles
|
||||||
|
inventory = inventory/hosts.yml
|
||||||
Binary file not shown.
|
|
@ -0,0 +1,46 @@
|
||||||
|
# Credentials
|
||||||
|
|
||||||
|
## LLDAP (User Directory)
|
||||||
|
|
||||||
|
- URL: https://ldap.cloud-elves.eu
|
||||||
|
- Admin username: `admin`
|
||||||
|
- Admin password: `cio3LFvUwUzLhO6iMXTAbQPq`
|
||||||
|
- Base DN: `dc=cloud-elves,dc=eu`
|
||||||
|
- LDAP port: 3890 (internal only)
|
||||||
|
|
||||||
|
## Dex (OIDC Provider)
|
||||||
|
|
||||||
|
- URL: https://auth.cloud-elves.eu
|
||||||
|
- Discovery: https://auth.cloud-elves.eu/.well-known/openid-configuration
|
||||||
|
|
||||||
|
### Dex Clients
|
||||||
|
| Client | ID | Secret |
|
||||||
|
|---|---|---|
|
||||||
|
| Outline | `outline` | `6edfe427944e50856e65e87ebf62217389f72963e8e6fe16d19536f1d31ffb56` |
|
||||||
|
| Gitea | `gitea` | `nWotT1z2AziFbmzscORURXlZ` |
|
||||||
|
|
||||||
|
## Gitea
|
||||||
|
|
||||||
|
- URL: https://git.cloud-elves.eu
|
||||||
|
- SSH clone: `ssh://git@git.cloud-elves.eu:2222`
|
||||||
|
- Auth: via Dex SSO (or local admin account)
|
||||||
|
|
||||||
|
### Gitea Postgres Database
|
||||||
|
- Host: `gitea-db` (internal)
|
||||||
|
- Database / User: `gitea`
|
||||||
|
- Password: `0cGpEZ0Ph2mdLbPJjA4kbR`
|
||||||
|
|
||||||
|
## Outline (Wiki)
|
||||||
|
|
||||||
|
- URL: https://docs.cloud-elves.eu
|
||||||
|
- Auth: via Dex SSO (login button: "Cloud Elves SSO")
|
||||||
|
|
||||||
|
### Outline Postgres Database
|
||||||
|
- Host: `outline-db` (internal)
|
||||||
|
- Database / User: `outline`
|
||||||
|
- Password: `oZEdt71Jd2zkto2qWKj1MI7K`
|
||||||
|
|
||||||
|
## Static Site
|
||||||
|
|
||||||
|
- URL: https://cloud-elves.eu (also https://www.cloud-elves.eu)
|
||||||
|
- Deploy: `rsync -avz --delete site/ root@box.cloud-elves.eu:/opt/www/`
|
||||||
|
|
@ -0,0 +1,87 @@
|
||||||
|
# Future Plans
|
||||||
|
|
||||||
|
## Priority Overview
|
||||||
|
|
||||||
|
| Priority | Item | Blocked on |
|
||||||
|
|---|---|---|
|
||||||
|
| High | Backup system (Restic) | Choosing backup destination |
|
||||||
|
| High | Security review | — |
|
||||||
|
| Medium | Migration to permanent VPS | Company incorporation |
|
||||||
|
| Medium | SMTP / Email | ProtonMail setup after incorporation |
|
||||||
|
| Medium | CI/CD pipeline | — |
|
||||||
|
| Low | Monitoring | — |
|
||||||
|
| Low | Group-based access control | Not needed yet (4 people) |
|
||||||
|
| Low | Move SSH to non-standard port | — |
|
||||||
|
| Low | Outline branding (remove badge) | Cosmetic, parked |
|
||||||
|
|
||||||
|
## High Priority
|
||||||
|
|
||||||
|
### Backup System (Restic)
|
||||||
|
- Daily automated backup of all Docker volumes to object storage
|
||||||
|
- Systemd timer (not cron)
|
||||||
|
- Retention policy: 7 daily, 4 weekly, 6 monthly
|
||||||
|
- Test restore procedure documented
|
||||||
|
- **Blocked on**: choosing a backup destination (S3 / Backblaze B2 / SFTP)
|
||||||
|
|
||||||
|
### Security Review
|
||||||
|
- Firewall audit — verify only intended ports are exposed
|
||||||
|
- Container hardening — run containers as non-root where possible
|
||||||
|
- Secret management — move passwords/secrets out of `group_vars/all.yml` into Ansible Vault or environment files
|
||||||
|
- Review Dex/LLDAP security configuration
|
||||||
|
- Fail2ban rules review
|
||||||
|
- Check for unnecessary services or open ports
|
||||||
|
- HTTPS headers audit (HSTS, CSP, etc.)
|
||||||
|
|
||||||
|
## Medium Priority
|
||||||
|
|
||||||
|
### Migration to Permanent VPS
|
||||||
|
|
||||||
|
Once company incorporation is complete and bank account is set up.
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
sequenceDiagram
|
||||||
|
participant Old as Old VPS
|
||||||
|
participant Backup as Backup Storage
|
||||||
|
participant New as New VPS
|
||||||
|
participant DNS
|
||||||
|
|
||||||
|
Old->>Backup: Final backup snapshot
|
||||||
|
Note over New: Provision Debian 13
|
||||||
|
New->>New: Run bootstrap.yml
|
||||||
|
New->>New: Run deploy.yml
|
||||||
|
Backup->>New: Restore volumes
|
||||||
|
DNS->>New: Update A record to new IP
|
||||||
|
Note over New: Verify SSL certs renew
|
||||||
|
Note over Old: Decommission
|
||||||
|
```
|
||||||
|
|
||||||
|
### CI/CD Pipeline
|
||||||
|
- Automated deployment on git push
|
||||||
|
- Documentation pipeline (Markdown to Outline, or similar)
|
||||||
|
- Options to evaluate: Gitea Actions, Drone CI, or simple webhook-based deploy
|
||||||
|
|
||||||
|
### SMTP / Email
|
||||||
|
- Set up ProtonMail for cloud-elves.eu (after incorporation)
|
||||||
|
- Configure Outline and Gitea to send transactional emails
|
||||||
|
- Update Let's Encrypt contact email from personal to company address
|
||||||
|
|
||||||
|
## Low Priority
|
||||||
|
|
||||||
|
### Monitoring
|
||||||
|
- Resource monitoring (CPU, memory, disk)
|
||||||
|
- Uptime monitoring for all services
|
||||||
|
- Options: simple solution like Uptime Kuma, or external service
|
||||||
|
|
||||||
|
### Group-Based Access Control
|
||||||
|
- Configure Dex to pass LLDAP groups to applications
|
||||||
|
- Restrict certain services to specific groups (e.g., admin-only tools)
|
||||||
|
- Not needed now (4 people, everyone needs access everywhere)
|
||||||
|
|
||||||
|
### Move SSH to Non-Standard Port
|
||||||
|
- Move host sshd from 22 to a high port (e.g., 2200)
|
||||||
|
- Reduces log noise from automated scanners
|
||||||
|
- Free up port 22 for Gitea SSH (nicer clone URLs)
|
||||||
|
|
||||||
|
### Outline Branding
|
||||||
|
- Remove "Built with Outline" badge on public share pages
|
||||||
|
- Parked for now — cosmetic only
|
||||||
|
|
@ -0,0 +1,47 @@
|
||||||
|
# Cloud Elves — Our Tools
|
||||||
|
|
||||||
|
## What do we have?
|
||||||
|
|
||||||
|
We run our own private server with a set of tools for the team. Everything is accessible from anywhere with a browser — just log in with your Cloud Elves account.
|
||||||
|
|
||||||
|
## Your account
|
||||||
|
|
||||||
|
You have one account that works across all our tools. Your login is managed centrally — if you can log into one tool, you can log into all of them.
|
||||||
|
|
||||||
|
If you need an account or forgot your password, ask an admin.
|
||||||
|
|
||||||
|
## Our tools
|
||||||
|
|
||||||
|
### Documentation — https://docs.cloud-elves.eu
|
||||||
|
|
||||||
|
This is where you're probably reading this. Our wiki for everything: internal docs, guides, meeting notes, public documentation — whatever we need to write down and share.
|
||||||
|
|
||||||
|
- Log in to create and edit pages
|
||||||
|
- Some pages are public (anyone on the internet can read them)
|
||||||
|
- Private collections are only visible to the team
|
||||||
|
|
||||||
|
### Git — https://git.cloud-elves.eu
|
||||||
|
|
||||||
|
Our code lives here. This is like GitHub, but hosted by us. If you work with code, repositories, or anything version-controlled, this is where it goes.
|
||||||
|
|
||||||
|
- Web interface for browsing code and pull requests
|
||||||
|
- SSH clone: `ssh://git@git.cloud-elves.eu:2222`
|
||||||
|
|
||||||
|
### Website — https://cloud-elves.eu
|
||||||
|
|
||||||
|
Our public website. Also available at https://www.cloud-elves.eu.
|
||||||
|
|
||||||
|
### User Management (admin only) — https://ldap.cloud-elves.eu
|
||||||
|
|
||||||
|
This is where admins create, edit, and remove team member accounts. Regular team members don't need to visit this — it's just for account administration.
|
||||||
|
|
||||||
|
## How to log in
|
||||||
|
|
||||||
|
1. Visit any of the tools above
|
||||||
|
2. Click the sign-in / SSO button
|
||||||
|
3. Enter your Cloud Elves username and password
|
||||||
|
4. You're in
|
||||||
|
|
||||||
|
## Need help?
|
||||||
|
|
||||||
|
Ask in the team chat, or reach out to whoever set up your account.
|
||||||
|
|
@ -0,0 +1,232 @@
|
||||||
|
# Gitea Actions — CI/CD Guide
|
||||||
|
|
||||||
|
## What is this?
|
||||||
|
|
||||||
|
Gitea Actions is our built-in CI/CD system. It lets you automate tasks when things happen in a git repository — like running tests on every push, deploying a site when you merge to main, or running a script on a schedule.
|
||||||
|
|
||||||
|
If you've used GitHub Actions before, it's the same syntax.
|
||||||
|
|
||||||
|
## How it works
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
sequenceDiagram
|
||||||
|
participant Dev as Developer
|
||||||
|
participant Gitea as Gitea
|
||||||
|
participant Runner as Runner (box-runner)
|
||||||
|
participant Container as Job Container
|
||||||
|
|
||||||
|
Dev->>Gitea: Push code
|
||||||
|
Gitea->>Runner: "There's a job to run"
|
||||||
|
Runner->>Container: Spin up temporary container
|
||||||
|
Container->>Container: Execute workflow steps
|
||||||
|
Container-->>Runner: Results
|
||||||
|
Runner-->>Gitea: Report status (pass/fail)
|
||||||
|
Note over Container: Container is cleaned up
|
||||||
|
Gitea-->>Dev: Show result in UI
|
||||||
|
```
|
||||||
|
|
||||||
|
Our runner (`box-runner`) runs on the same server as everything else. When a job triggers, it spins up a temporary Docker container, runs the steps, reports back, and cleans up. The runner itself is always running in the background, waiting for work.
|
||||||
|
|
||||||
|
## Quick start
|
||||||
|
|
||||||
|
### 1. Create a workflow file
|
||||||
|
|
||||||
|
In any Gitea repository, create a file at `.gitea/workflows/` with a `.yml` extension:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# .gitea/workflows/hello.yml
|
||||||
|
name: Hello World
|
||||||
|
on: [push]
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
greet:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- run: echo "Hello from CI!"
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Push it
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add .gitea/workflows/hello.yml
|
||||||
|
git commit -m "Add CI workflow"
|
||||||
|
git push
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Check the result
|
||||||
|
|
||||||
|
Go to your repository on Gitea → **Actions** tab. You'll see the workflow run with its status.
|
||||||
|
|
||||||
|
## Workflow syntax
|
||||||
|
|
||||||
|
### When to run (triggers)
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# Run on every push
|
||||||
|
on: [push]
|
||||||
|
|
||||||
|
# Run on push to main only
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
branches: [main]
|
||||||
|
|
||||||
|
# Run on pull request
|
||||||
|
on: [pull_request]
|
||||||
|
|
||||||
|
# Run on a schedule (cron syntax)
|
||||||
|
on:
|
||||||
|
schedule:
|
||||||
|
- cron: "0 2 * * *" # Every day at 2 AM
|
||||||
|
|
||||||
|
# Run manually from the UI
|
||||||
|
on:
|
||||||
|
workflow_dispatch:
|
||||||
|
```
|
||||||
|
|
||||||
|
### Available runners
|
||||||
|
|
||||||
|
| Label | What it is |
|
||||||
|
|---|---|
|
||||||
|
| `ubuntu-latest` | Ubuntu container (default choice) |
|
||||||
|
| `ubuntu-24.04` | Ubuntu 24.04 specifically |
|
||||||
|
| `ubuntu-22.04` | Ubuntu 22.04 specifically |
|
||||||
|
|
||||||
|
Use these in `runs-on`:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
jobs:
|
||||||
|
my-job:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
```
|
||||||
|
|
||||||
|
### Common steps
|
||||||
|
|
||||||
|
**Check out the code:**
|
||||||
|
```yaml
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
```
|
||||||
|
|
||||||
|
**Run a shell command:**
|
||||||
|
```yaml
|
||||||
|
steps:
|
||||||
|
- run: echo "Hello"
|
||||||
|
```
|
||||||
|
|
||||||
|
**Run multiple commands:**
|
||||||
|
```yaml
|
||||||
|
steps:
|
||||||
|
- run: |
|
||||||
|
echo "Step 1"
|
||||||
|
echo "Step 2"
|
||||||
|
ls -la
|
||||||
|
```
|
||||||
|
|
||||||
|
**Use environment variables:**
|
||||||
|
```yaml
|
||||||
|
jobs:
|
||||||
|
deploy:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
env:
|
||||||
|
MY_VAR: "some value"
|
||||||
|
steps:
|
||||||
|
- run: echo $MY_VAR
|
||||||
|
```
|
||||||
|
|
||||||
|
**Use secrets (set in Gitea UI under repo Settings → Actions → Secrets):**
|
||||||
|
```yaml
|
||||||
|
steps:
|
||||||
|
- run: echo ${{ secrets.MY_SECRET }}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Practical examples
|
||||||
|
|
||||||
|
### Run tests on every push
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
name: Test
|
||||||
|
on: [push]
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
test:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
- run: npm install
|
||||||
|
- run: npm test
|
||||||
|
```
|
||||||
|
|
||||||
|
### Deploy static site on push to main
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
name: Deploy Site
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
branches: [main]
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
deploy:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
- name: Deploy via rsync
|
||||||
|
run: |
|
||||||
|
mkdir -p ~/.ssh
|
||||||
|
echo "${{ secrets.DEPLOY_KEY }}" > ~/.ssh/id_ed25519
|
||||||
|
chmod 600 ~/.ssh/id_ed25519
|
||||||
|
ssh-keyscan box.cloud-elves.eu >> ~/.ssh/known_hosts
|
||||||
|
rsync -avz --delete site/ root@box.cloud-elves.eu:/opt/www/
|
||||||
|
```
|
||||||
|
|
||||||
|
> For this to work, add a `DEPLOY_KEY` secret in the repo settings containing a private SSH key that has access to the server.
|
||||||
|
|
||||||
|
### Scheduled task (glorified cron)
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
name: Nightly Cleanup
|
||||||
|
on:
|
||||||
|
schedule:
|
||||||
|
- cron: "0 3 * * *" # 3 AM daily
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
cleanup:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- run: echo "Running nightly task..."
|
||||||
|
# Add your actual commands here
|
||||||
|
```
|
||||||
|
|
||||||
|
### Build and push a Docker image
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
name: Build Image
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
branches: [main]
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
build:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
- run: docker build -t my-app:latest .
|
||||||
|
- run: docker tag my-app:latest git.cloud-elves.eu/myorg/my-app:latest
|
||||||
|
# Push to Gitea's container registry if enabled
|
||||||
|
```
|
||||||
|
|
||||||
|
## Where to find things in the UI
|
||||||
|
|
||||||
|
| What | Where |
|
||||||
|
|---|---|
|
||||||
|
| Workflow runs | Repository → **Actions** tab |
|
||||||
|
| Workflow files | Repository → `.gitea/workflows/` directory |
|
||||||
|
| Secrets | Repository → **Settings** → **Actions** → **Secrets** |
|
||||||
|
| Runner status | **Site Administration** → **Actions** → **Runners** (admin only) |
|
||||||
|
|
||||||
|
## Tips
|
||||||
|
|
||||||
|
- Workflows live in your repo as code — version controlled like everything else
|
||||||
|
- Each push shows a status check (green tick or red cross) next to the commit
|
||||||
|
- You can re-run failed workflows from the Actions tab
|
||||||
|
- Use `workflow_dispatch` trigger if you want a "run manually" button in the UI
|
||||||
|
- Keep workflows simple — if it's getting complex, it probably belongs in a script that the workflow just calls
|
||||||
|
|
@ -0,0 +1,121 @@
|
||||||
|
# Cloud Elves Infrastructure Overview
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
All services run on a single DigitalOcean VPS (`box.cloud-elves.eu`, Debian 13) managed by Ansible and deployed as Docker Compose containers.
|
||||||
|
|
||||||
|
### Services
|
||||||
|
|
||||||
|
| Service | URL | Purpose |
|
||||||
|
|---|---|---|
|
||||||
|
| Caddy | (internal) | Reverse proxy, automatic HTTPS via Let's Encrypt |
|
||||||
|
| Gitea | https://git.cloud-elves.eu | Git hosting, SSH clone on port 2222 |
|
||||||
|
| Outline | https://docs.cloud-elves.eu | Wiki and documentation |
|
||||||
|
| Dex | https://auth.cloud-elves.eu | OIDC identity provider |
|
||||||
|
| LLDAP | https://ldap.cloud-elves.eu | User directory with web UI |
|
||||||
|
| Static site | https://cloud-elves.eu | Company website (also www) |
|
||||||
|
|
||||||
|
### Supporting Services (internal only)
|
||||||
|
|
||||||
|
| Service | Used by |
|
||||||
|
|---|---|
|
||||||
|
| Postgres (gitea-db) | Gitea |
|
||||||
|
| Postgres (outline-db) | Outline |
|
||||||
|
| Redis | Outline |
|
||||||
|
|
||||||
|
### How a request reaches your app
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
sequenceDiagram
|
||||||
|
participant Browser
|
||||||
|
participant Caddy
|
||||||
|
participant App as Gitea / Outline / etc.
|
||||||
|
|
||||||
|
Browser->>Caddy: HTTPS request (port 443)
|
||||||
|
Note over Caddy: Terminates TLS<br/>Auto-issued Let's Encrypt cert
|
||||||
|
Caddy->>App: Forward to internal port
|
||||||
|
App-->>Caddy: Response
|
||||||
|
Caddy-->>Browser: HTTPS response
|
||||||
|
```
|
||||||
|
|
||||||
|
## Authentication
|
||||||
|
|
||||||
|
All authentication flows through a centralized identity stack.
|
||||||
|
|
||||||
|
### SSO Login Flow
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
sequenceDiagram
|
||||||
|
participant User
|
||||||
|
participant App as Gitea / Outline
|
||||||
|
participant Dex as Dex (OIDC)
|
||||||
|
participant LLDAP as LLDAP (Directory)
|
||||||
|
|
||||||
|
User->>App: Visit app
|
||||||
|
App->>Dex: Redirect to SSO
|
||||||
|
Dex->>User: Show login form
|
||||||
|
User->>Dex: Enter credentials
|
||||||
|
Dex->>LLDAP: Verify credentials
|
||||||
|
LLDAP-->>Dex: OK
|
||||||
|
Dex-->>App: Auth token
|
||||||
|
App-->>User: Logged in
|
||||||
|
```
|
||||||
|
|
||||||
|
### Onboarding a new team member
|
||||||
|
|
||||||
|
1. Create their account in LLDAP (https://ldap.cloud-elves.eu)
|
||||||
|
2. They can immediately log into all services via SSO — accounts are auto-created on first login
|
||||||
|
|
||||||
|
### Removing a team member
|
||||||
|
|
||||||
|
1. Delete or disable the user in LLDAP
|
||||||
|
2. Their SSO sessions will expire (existing sessions may persist until token expiry)
|
||||||
|
|
||||||
|
## Networking
|
||||||
|
|
||||||
|
### Exposed Ports
|
||||||
|
|
||||||
|
| Port | Protocol | Service |
|
||||||
|
|---|---|---|
|
||||||
|
| 22 | TCP | SSH (host access) |
|
||||||
|
| 80 | TCP | HTTP (redirects to HTTPS) |
|
||||||
|
| 443 | TCP + UDP | HTTPS (all web services) |
|
||||||
|
| 2222 | TCP | Gitea SSH (git clone) |
|
||||||
|
|
||||||
|
All other communication happens over internal Docker networks. UFW firewall blocks everything else.
|
||||||
|
|
||||||
|
### Internal Networks
|
||||||
|
|
||||||
|
| Network | Connects |
|
||||||
|
|---|---|
|
||||||
|
| frontend | Caddy to all web-facing services |
|
||||||
|
| identity | Dex to LLDAP |
|
||||||
|
| gitea-backend | Gitea to its Postgres |
|
||||||
|
| outline-backend | Outline to its Postgres and Redis |
|
||||||
|
|
||||||
|
## DNS Records
|
||||||
|
|
||||||
|
| Record | Type | Points to |
|
||||||
|
|---|---|---|
|
||||||
|
| `cloud-elves.eu` | A | 188.166.60.12 |
|
||||||
|
| `box.cloud-elves.eu` | A | 188.166.60.12 |
|
||||||
|
| `www.cloud-elves.eu` | CNAME | box.cloud-elves.eu |
|
||||||
|
| `git.cloud-elves.eu` | CNAME | box.cloud-elves.eu |
|
||||||
|
| `auth.cloud-elves.eu` | CNAME | box.cloud-elves.eu |
|
||||||
|
| `ldap.cloud-elves.eu` | CNAME | box.cloud-elves.eu |
|
||||||
|
| `docs.cloud-elves.eu` | CNAME | box.cloud-elves.eu |
|
||||||
|
|
||||||
|
## Data & Persistence
|
||||||
|
|
||||||
|
All persistent data is in Docker named volumes under `/var/lib/docker/volumes/` on the host.
|
||||||
|
|
||||||
|
| Volume | Contains |
|
||||||
|
|---|---|
|
||||||
|
| `caddy-data` | TLS certificates |
|
||||||
|
| `caddy-config` | Caddy runtime config |
|
||||||
|
| `lldap-data` | User directory (SQLite) |
|
||||||
|
| `dex-data` | Dex database (SQLite) |
|
||||||
|
| `gitea-data` | Git repositories, Gitea config |
|
||||||
|
| `gitea-db` | Gitea Postgres data |
|
||||||
|
| `outline-data` | Uploaded files/attachments |
|
||||||
|
| `outline-db` | Outline Postgres data |
|
||||||
|
|
@ -0,0 +1,189 @@
|
||||||
|
# Operational Runbook
|
||||||
|
|
||||||
|
## Day-to-Day Operations
|
||||||
|
|
||||||
|
### Check service health
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ansible -i inventory/hosts.yml all -m ansible.builtin.shell \
|
||||||
|
-a "cd /opt/services && docker compose ps"
|
||||||
|
```
|
||||||
|
|
||||||
|
### View logs for a service
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Replace <service> with: caddy, gitea, outline, dex, lldap, gitea-db, outline-db, outline-redis
|
||||||
|
ansible -i inventory/hosts.yml all -m ansible.builtin.shell \
|
||||||
|
-a "cd /opt/services && docker compose logs <service> --tail 100"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Restart a single service
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ansible -i inventory/hosts.yml all -m ansible.builtin.shell \
|
||||||
|
-a "cd /opt/services && docker compose restart <service>"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Deploy config changes
|
||||||
|
|
||||||
|
After editing any template in `roles/services/templates/`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ansible-playbook -i inventory/hosts.yml playbooks/deploy.yml
|
||||||
|
```
|
||||||
|
|
||||||
|
### Deploy static site
|
||||||
|
|
||||||
|
From your local `cloud-elves.eu` repo:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
rsync -avz --delete site/ root@box.cloud-elves.eu:/opt/www/
|
||||||
|
```
|
||||||
|
|
||||||
|
## User Management
|
||||||
|
|
||||||
|
### Onboarding flow
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
sequenceDiagram
|
||||||
|
participant Admin
|
||||||
|
participant LLDAP
|
||||||
|
participant NewUser as New Team Member
|
||||||
|
participant App as Gitea / Outline
|
||||||
|
|
||||||
|
Admin->>LLDAP: Create user account
|
||||||
|
LLDAP-->>Admin: Done
|
||||||
|
Admin->>NewUser: Share login details
|
||||||
|
NewUser->>App: Visit any app
|
||||||
|
App->>NewUser: Redirect to SSO
|
||||||
|
NewUser->>App: Log in with LLDAP credentials
|
||||||
|
Note over App: Account auto-created on first login
|
||||||
|
```
|
||||||
|
|
||||||
|
1. Go to https://ldap.cloud-elves.eu and log in as admin
|
||||||
|
2. Click "Create user" — fill in username, email, display name, password
|
||||||
|
3. Share credentials with the new team member
|
||||||
|
4. They can immediately log into Gitea and Outline via SSO
|
||||||
|
|
||||||
|
### Removing a team member
|
||||||
|
|
||||||
|
1. Go to https://ldap.cloud-elves.eu
|
||||||
|
2. Delete or disable the user
|
||||||
|
|
||||||
|
## Adding a New Service
|
||||||
|
|
||||||
|
### Checklist
|
||||||
|
|
||||||
|
| Step | What | Where |
|
||||||
|
|---|---|---|
|
||||||
|
| 1 | Add container definition | `roles/services/templates/docker-compose.yml.j2` |
|
||||||
|
| 2 | Add reverse proxy entry | `roles/services/templates/Caddyfile.j2` |
|
||||||
|
| 3 | Add OIDC client (if SSO needed) | `roles/services/templates/dex-config.yml.j2` |
|
||||||
|
| 4 | Add variables (passwords, versions) | `inventory/group_vars/all.yml` |
|
||||||
|
| 5 | Create DNS CNAME | DNS provider: `<subdomain>` → `box.cloud-elves.eu` |
|
||||||
|
| 6 | Deploy | `ansible-playbook -i inventory/hosts.yml playbooks/deploy.yml` |
|
||||||
|
| 7 | Verify | Caddy auto-issues TLS cert on first request |
|
||||||
|
|
||||||
|
### How it connects
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
sequenceDiagram
|
||||||
|
participant Dev as Developer
|
||||||
|
participant Ansible
|
||||||
|
participant Box as box.cloud-elves.eu
|
||||||
|
participant Caddy
|
||||||
|
participant LE as Let's Encrypt
|
||||||
|
|
||||||
|
Dev->>Ansible: Run deploy.yml
|
||||||
|
Ansible->>Box: Template configs + docker compose up
|
||||||
|
Box-->>Ansible: Services running
|
||||||
|
Note over Caddy: New subdomain in Caddyfile
|
||||||
|
Caddy->>LE: Request certificate
|
||||||
|
LE-->>Caddy: Certificate issued
|
||||||
|
Note over Caddy: Service live with HTTPS
|
||||||
|
```
|
||||||
|
|
||||||
|
## Full Rebuild from Scratch
|
||||||
|
|
||||||
|
### When you need this
|
||||||
|
|
||||||
|
- Droplet is destroyed
|
||||||
|
- Migrating to a new server
|
||||||
|
- Starting fresh
|
||||||
|
|
||||||
|
### Procedure
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Provision new Debian 13 droplet with SSH key (id_ed25519_ce)
|
||||||
|
|
||||||
|
# 2. Update DNS: point box.cloud-elves.eu A record to new IP
|
||||||
|
# Flush local DNS: sudo dscacheutil -flushcache && sudo killall -HUP mDNSResponder
|
||||||
|
|
||||||
|
# 3. Accept new host keys
|
||||||
|
ssh-keygen -R box.cloud-elves.eu
|
||||||
|
ssh-keyscan box.cloud-elves.eu >> ~/.ssh/known_hosts
|
||||||
|
|
||||||
|
# 4. Bootstrap (OS hardening, Docker install)
|
||||||
|
ansible-playbook -i inventory/hosts.yml playbooks/bootstrap.yml
|
||||||
|
|
||||||
|
# 5. Deploy all services
|
||||||
|
ansible-playbook -i inventory/hosts.yml playbooks/deploy.yml
|
||||||
|
|
||||||
|
# 6. Restore data from backup (once backup system is in place)
|
||||||
|
```
|
||||||
|
|
||||||
|
### After a fresh deploy you'll need to
|
||||||
|
|
||||||
|
- Set up a Gitea admin account (first registered user via SSO becomes admin)
|
||||||
|
- Re-create users in LLDAP
|
||||||
|
- All Outline content will be empty (unless restored from backup)
|
||||||
|
|
||||||
|
## Upgrading a Service
|
||||||
|
|
||||||
|
### Procedure
|
||||||
|
|
||||||
|
1. Check the current version in `inventory/group_vars/all.yml`
|
||||||
|
2. Look up the latest version on Docker Hub
|
||||||
|
3. Update the version variable
|
||||||
|
4. Run: `ansible-playbook -i inventory/hosts.yml playbooks/deploy.yml`
|
||||||
|
5. Check logs for migration output: `docker compose logs <service> --tail 50`
|
||||||
|
6. Verify the service is healthy
|
||||||
|
|
||||||
|
### Example: upgrading Outline
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# In inventory/group_vars/all.yml, change:
|
||||||
|
# outline_version: "1.6.0"
|
||||||
|
# to:
|
||||||
|
# outline_version: "1.7.0"
|
||||||
|
|
||||||
|
ansible-playbook -i inventory/hosts.yml playbooks/deploy.yml
|
||||||
|
```
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### Service won't start
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ansible -i inventory/hosts.yml all -m ansible.builtin.shell \
|
||||||
|
-a "cd /opt/services && docker compose logs <service> --tail 50"
|
||||||
|
```
|
||||||
|
|
||||||
|
### SSL certificate not issued
|
||||||
|
|
||||||
|
Caddy issues certs on first request. If a cert is missing:
|
||||||
|
1. Verify DNS resolves: `dig +short <subdomain>.cloud-elves.eu`
|
||||||
|
2. Check Caddy logs for ACME errors
|
||||||
|
3. Restart Caddy: `docker compose restart caddy`
|
||||||
|
|
||||||
|
### Can't SSH to the box
|
||||||
|
|
||||||
|
1. Check if DNS is stale: `dig +short box.cloud-elves.eu`
|
||||||
|
2. Flush DNS: `sudo dscacheutil -flushcache && sudo killall -HUP mDNSResponder`
|
||||||
|
3. If host key changed (rebuild): `ssh-keygen -R box.cloud-elves.eu`
|
||||||
|
|
||||||
|
### Ansible can't connect
|
||||||
|
|
||||||
|
1. Verify SSH works: `ssh -i ~/.ssh/id_ed25519_ce root@box.cloud-elves.eu`
|
||||||
|
2. Check known_hosts for stale entries
|
||||||
|
3. Run: `ansible -i inventory/hosts.yml all -m ping`
|
||||||
|
|
@ -0,0 +1,34 @@
|
||||||
|
# Primary domain — subdomains will be created off this
|
||||||
|
domain: "cloud-elves.eu"
|
||||||
|
|
||||||
|
# Hostname for the server
|
||||||
|
server_hostname: "box"
|
||||||
|
|
||||||
|
# Email for Let's Encrypt certificate notifications (used by Caddy automatically)
|
||||||
|
acme_email: "maarten@lambda-m.nl"
|
||||||
|
|
||||||
|
# Docker image versions — pin these, update deliberately
|
||||||
|
gitea_version: "1.21"
|
||||||
|
postgres_version: "16-alpine"
|
||||||
|
outline_version: "1.6.0"
|
||||||
|
redis_version: "7-alpine"
|
||||||
|
|
||||||
|
# LDAP base DN
|
||||||
|
ldap_base_dn: "dc=cloud-elves,dc=eu"
|
||||||
|
|
||||||
|
# Gitea database password
|
||||||
|
gitea_db_password: "0cGpEZ0Ph2mdLbPJjA4kbR"
|
||||||
|
|
||||||
|
# LLDAP secrets
|
||||||
|
lldap_jwt_secret: "371bc1a40f93dc2229fbaa5e080ac323"
|
||||||
|
lldap_key_seed: "1a5873dd243b29f60043d2e69ec9b7f7"
|
||||||
|
lldap_admin_password: "cio3LFvUwUzLhO6iMXTAbQPq"
|
||||||
|
|
||||||
|
# Dex OIDC client secrets
|
||||||
|
dex_outline_client_secret: "6edfe427944e50856e65e87ebf62217389f72963e8e6fe16d19536f1d31ffb56"
|
||||||
|
dex_gitea_client_secret: "nWotT1z2AziFbmzscORURXlZ"
|
||||||
|
|
||||||
|
# Outline secrets
|
||||||
|
outline_secret_key: "72a81e9516e29c249ad8543866b1275a34b6a15014527f81d1becab363c82fb4"
|
||||||
|
outline_utils_secret: "9219100ad29e330c0a7ea8032adc0c5831de14e487a7c3c5e050b757d022f5da"
|
||||||
|
outline_db_password: "oZEdt71Jd2zkto2qWKj1MI7K"
|
||||||
|
|
@ -0,0 +1,7 @@
|
||||||
|
all:
|
||||||
|
hosts:
|
||||||
|
box:
|
||||||
|
ansible_host: box.cloud-elves.eu
|
||||||
|
ansible_user: root
|
||||||
|
ansible_ssh_private_key_file: ~/.ssh/id_ed25519_ce
|
||||||
|
ansible_python_interpreter: /usr/bin/python3
|
||||||
|
|
@ -0,0 +1,9 @@
|
||||||
|
---
|
||||||
|
- name: Bootstrap and harden the VPS
|
||||||
|
hosts: all
|
||||||
|
become: true
|
||||||
|
roles:
|
||||||
|
- role: common
|
||||||
|
tags: [common]
|
||||||
|
- role: docker
|
||||||
|
tags: [docker]
|
||||||
|
|
@ -0,0 +1,7 @@
|
||||||
|
---
|
||||||
|
- name: Deploy services
|
||||||
|
hosts: all
|
||||||
|
become: true
|
||||||
|
roles:
|
||||||
|
- role: services
|
||||||
|
tags: [services]
|
||||||
|
|
@ -0,0 +1,5 @@
|
||||||
|
---
|
||||||
|
- name: Restart sshd
|
||||||
|
ansible.builtin.service:
|
||||||
|
name: ssh
|
||||||
|
state: restarted
|
||||||
|
|
@ -0,0 +1,127 @@
|
||||||
|
---
|
||||||
|
# Disable cloud-init so it never regenerates SSH keys or resets config on reboot
|
||||||
|
- name: Disable cloud-init
|
||||||
|
ansible.builtin.file:
|
||||||
|
path: /etc/cloud/cloud-init.disabled
|
||||||
|
state: touch
|
||||||
|
mode: "0644"
|
||||||
|
|
||||||
|
- name: Find active cloud-init services
|
||||||
|
ansible.builtin.shell: systemctl list-unit-files 'cloud-*' --no-legend | awk '{print $1}'
|
||||||
|
register: cloud_init_services
|
||||||
|
changed_when: false
|
||||||
|
|
||||||
|
- name: Stop cloud-init services
|
||||||
|
ansible.builtin.service:
|
||||||
|
name: "{{ item }}"
|
||||||
|
state: stopped
|
||||||
|
enabled: false
|
||||||
|
loop: "{{ cloud_init_services.stdout_lines }}"
|
||||||
|
when: cloud_init_services.stdout_lines | length > 0
|
||||||
|
|
||||||
|
# Set the hostname to match our inventory name
|
||||||
|
- name: Set hostname
|
||||||
|
ansible.builtin.hostname:
|
||||||
|
name: "{{ server_hostname }}"
|
||||||
|
|
||||||
|
# Update /etc/hosts to include the new hostname
|
||||||
|
- name: Update /etc/hosts with hostname
|
||||||
|
ansible.builtin.lineinfile:
|
||||||
|
path: /etc/hosts
|
||||||
|
regexp: '^127\.0\.1\.1'
|
||||||
|
line: "127.0.1.1 {{ server_hostname }}.{{ domain }} {{ server_hostname }}"
|
||||||
|
state: present
|
||||||
|
|
||||||
|
# Refresh apt cache before installing anything
|
||||||
|
- name: Update apt cache
|
||||||
|
ansible.builtin.apt:
|
||||||
|
update_cache: true
|
||||||
|
cache_valid_time: 3600
|
||||||
|
|
||||||
|
# Upgrade all installed packages to latest
|
||||||
|
- name: Upgrade all packages
|
||||||
|
ansible.builtin.apt:
|
||||||
|
upgrade: dist
|
||||||
|
autoremove: true
|
||||||
|
|
||||||
|
# Install essential tools and security packages
|
||||||
|
- name: Install base packages
|
||||||
|
ansible.builtin.apt:
|
||||||
|
name:
|
||||||
|
- curl
|
||||||
|
- git
|
||||||
|
- htop
|
||||||
|
- rsync
|
||||||
|
- ufw
|
||||||
|
- fail2ban
|
||||||
|
- unattended-upgrades
|
||||||
|
- apt-listchanges
|
||||||
|
state: present
|
||||||
|
|
||||||
|
# Allow SSH before enabling the firewall (don't lock ourselves out)
|
||||||
|
- name: UFW — allow SSH
|
||||||
|
community.general.ufw:
|
||||||
|
rule: allow
|
||||||
|
port: "22"
|
||||||
|
proto: tcp
|
||||||
|
|
||||||
|
# Allow HTTP for Caddy
|
||||||
|
- name: UFW — allow HTTP
|
||||||
|
community.general.ufw:
|
||||||
|
rule: allow
|
||||||
|
port: "80"
|
||||||
|
proto: tcp
|
||||||
|
|
||||||
|
# Allow HTTPS for Caddy
|
||||||
|
- name: UFW — allow HTTPS
|
||||||
|
community.general.ufw:
|
||||||
|
rule: allow
|
||||||
|
port: "443"
|
||||||
|
proto: tcp
|
||||||
|
|
||||||
|
# Allow Gitea SSH
|
||||||
|
- name: UFW — allow Gitea SSH
|
||||||
|
community.general.ufw:
|
||||||
|
rule: allow
|
||||||
|
port: "2222"
|
||||||
|
proto: tcp
|
||||||
|
|
||||||
|
# Default deny incoming, allow outgoing, then enable
|
||||||
|
- name: UFW — enable with default deny incoming
|
||||||
|
community.general.ufw:
|
||||||
|
state: enabled
|
||||||
|
default: deny
|
||||||
|
direction: incoming
|
||||||
|
|
||||||
|
- name: UFW — default allow outgoing
|
||||||
|
community.general.ufw:
|
||||||
|
default: allow
|
||||||
|
direction: outgoing
|
||||||
|
|
||||||
|
# Enable automatic security updates
|
||||||
|
- name: Enable unattended-upgrades
|
||||||
|
ansible.builtin.copy:
|
||||||
|
dest: /etc/apt/apt.conf.d/20auto-upgrades
|
||||||
|
content: |
|
||||||
|
APT::Periodic::Update-Package-Lists "1";
|
||||||
|
APT::Periodic::Unattended-Upgrade "1";
|
||||||
|
APT::Periodic::AutocleanInterval "7";
|
||||||
|
mode: "0644"
|
||||||
|
|
||||||
|
# Disable root password login — key-only auth
|
||||||
|
- name: Disable SSH password authentication
|
||||||
|
ansible.builtin.lineinfile:
|
||||||
|
path: /etc/ssh/sshd_config
|
||||||
|
regexp: '^#?PasswordAuthentication'
|
||||||
|
line: 'PasswordAuthentication no'
|
||||||
|
state: present
|
||||||
|
notify: Restart sshd
|
||||||
|
|
||||||
|
# Disable root login via password (permit key-based only)
|
||||||
|
- name: Set SSH to permit root login with key only
|
||||||
|
ansible.builtin.lineinfile:
|
||||||
|
path: /etc/ssh/sshd_config
|
||||||
|
regexp: '^#?PermitRootLogin'
|
||||||
|
line: 'PermitRootLogin prohibit-password'
|
||||||
|
state: present
|
||||||
|
notify: Restart sshd
|
||||||
|
|
@ -0,0 +1,50 @@
|
||||||
|
---
|
||||||
|
# Install Docker prerequisites
|
||||||
|
- name: Install Docker dependencies
|
||||||
|
ansible.builtin.apt:
|
||||||
|
name:
|
||||||
|
- ca-certificates
|
||||||
|
- curl
|
||||||
|
- gnupg
|
||||||
|
state: present
|
||||||
|
|
||||||
|
# Add Docker's official GPG key
|
||||||
|
- name: Create keyrings directory
|
||||||
|
ansible.builtin.file:
|
||||||
|
path: /etc/apt/keyrings
|
||||||
|
state: directory
|
||||||
|
mode: "0755"
|
||||||
|
|
||||||
|
- name: Add Docker GPG key
|
||||||
|
ansible.builtin.get_url:
|
||||||
|
url: https://download.docker.com/linux/debian/gpg
|
||||||
|
dest: /etc/apt/keyrings/docker.asc
|
||||||
|
mode: "0644"
|
||||||
|
|
||||||
|
# Add Docker apt repository for Debian
|
||||||
|
- name: Add Docker apt repository
|
||||||
|
ansible.builtin.apt_repository:
|
||||||
|
repo: >-
|
||||||
|
deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.asc]
|
||||||
|
https://download.docker.com/linux/debian
|
||||||
|
{{ ansible_facts['distribution_release'] }} stable
|
||||||
|
state: present
|
||||||
|
filename: docker
|
||||||
|
|
||||||
|
# Install Docker Engine + Compose plugin
|
||||||
|
- name: Install Docker Engine
|
||||||
|
ansible.builtin.apt:
|
||||||
|
name:
|
||||||
|
- docker-ce
|
||||||
|
- docker-ce-cli
|
||||||
|
- containerd.io
|
||||||
|
- docker-compose-plugin
|
||||||
|
state: present
|
||||||
|
update_cache: true
|
||||||
|
|
||||||
|
# Ensure Docker service is running and enabled at boot
|
||||||
|
- name: Enable and start Docker
|
||||||
|
ansible.builtin.service:
|
||||||
|
name: docker
|
||||||
|
state: started
|
||||||
|
enabled: true
|
||||||
|
|
@ -0,0 +1,157 @@
|
||||||
|
---
|
||||||
|
# Create the services directory on the host
|
||||||
|
- name: Create services directory
|
||||||
|
ansible.builtin.file:
|
||||||
|
path: /opt/services
|
||||||
|
state: directory
|
||||||
|
mode: "0755"
|
||||||
|
|
||||||
|
# Create a placeholder index page for the static site
|
||||||
|
- name: Create www directory
|
||||||
|
ansible.builtin.file:
|
||||||
|
path: /opt/www
|
||||||
|
state: directory
|
||||||
|
mode: "0755"
|
||||||
|
|
||||||
|
- name: Create placeholder index.html
|
||||||
|
ansible.builtin.copy:
|
||||||
|
dest: /opt/www/index.html
|
||||||
|
content: |
|
||||||
|
<!DOCTYPE html>
|
||||||
|
<html>
|
||||||
|
<head><title>Cloud Elves</title></head>
|
||||||
|
<body><h1>cloud-elves.eu</h1><p>Coming soon.</p></body>
|
||||||
|
</html>
|
||||||
|
mode: "0644"
|
||||||
|
force: false
|
||||||
|
|
||||||
|
# Template config files
|
||||||
|
- name: Deploy Caddyfile
|
||||||
|
ansible.builtin.template:
|
||||||
|
src: Caddyfile.j2
|
||||||
|
dest: /opt/services/Caddyfile
|
||||||
|
mode: "0644"
|
||||||
|
register: caddyfile_changed
|
||||||
|
|
||||||
|
- name: Deploy Dex config
|
||||||
|
ansible.builtin.template:
|
||||||
|
src: dex-config.yml.j2
|
||||||
|
dest: /opt/services/dex-config.yml
|
||||||
|
mode: "0644"
|
||||||
|
|
||||||
|
# Check if we already have a runner token saved
|
||||||
|
- name: Check for existing runner token
|
||||||
|
ansible.builtin.stat:
|
||||||
|
path: /opt/services/.runner-token
|
||||||
|
register: runner_token_file
|
||||||
|
|
||||||
|
# Use a placeholder token for first deploy (runner won't start yet)
|
||||||
|
- name: Set runner token from file if exists
|
||||||
|
ansible.builtin.slurp:
|
||||||
|
src: /opt/services/.runner-token
|
||||||
|
register: runner_token_content
|
||||||
|
when: runner_token_file.stat.exists
|
||||||
|
|
||||||
|
- name: Set runner token variable
|
||||||
|
ansible.builtin.set_fact:
|
||||||
|
gitea_runner_token: "{{ runner_token_content.content | b64decode | trim }}"
|
||||||
|
when: runner_token_file.stat.exists
|
||||||
|
|
||||||
|
- name: Set placeholder runner token for first deploy
|
||||||
|
ansible.builtin.set_fact:
|
||||||
|
gitea_runner_token: "placeholder"
|
||||||
|
when: not runner_token_file.stat.exists
|
||||||
|
|
||||||
|
# Deploy compose file
|
||||||
|
- name: Deploy docker-compose.yml
|
||||||
|
ansible.builtin.template:
|
||||||
|
src: docker-compose.yml.j2
|
||||||
|
dest: /opt/services/docker-compose.yml
|
||||||
|
mode: "0644"
|
||||||
|
|
||||||
|
# Start everything (runner will fail to register with placeholder token — that's OK)
|
||||||
|
- name: Start all services
|
||||||
|
ansible.builtin.command:
|
||||||
|
cmd: docker compose up -d --remove-orphans
|
||||||
|
chdir: /opt/services
|
||||||
|
changed_when: true
|
||||||
|
|
||||||
|
# Reload Caddy if Caddyfile changed (zero-downtime)
|
||||||
|
- name: Reload Caddy config
|
||||||
|
ansible.builtin.command:
|
||||||
|
cmd: docker compose exec -T caddy caddy reload --config /etc/caddy/Caddyfile
|
||||||
|
chdir: /opt/services
|
||||||
|
when: caddyfile_changed.changed
|
||||||
|
changed_when: true
|
||||||
|
|
||||||
|
# Wait for Gitea to be ready
|
||||||
|
- name: Wait for Gitea to be ready
|
||||||
|
ansible.builtin.command:
|
||||||
|
cmd: docker compose exec -T -u git gitea gitea admin user list
|
||||||
|
chdir: /opt/services
|
||||||
|
register: gitea_ready
|
||||||
|
retries: 10
|
||||||
|
delay: 5
|
||||||
|
until: gitea_ready.rc == 0
|
||||||
|
changed_when: false
|
||||||
|
|
||||||
|
# --- Gitea OIDC setup ---
|
||||||
|
|
||||||
|
- name: Check if Dex auth source exists in Gitea
|
||||||
|
ansible.builtin.shell:
|
||||||
|
cmd: docker compose exec -T -u git gitea gitea admin auth list 2>&1 | grep -q dex
|
||||||
|
chdir: /opt/services
|
||||||
|
register: gitea_auth_check
|
||||||
|
changed_when: false
|
||||||
|
failed_when: false
|
||||||
|
|
||||||
|
- name: Add Dex OIDC auth source to Gitea
|
||||||
|
ansible.builtin.command:
|
||||||
|
cmd: >
|
||||||
|
docker compose exec -T -u git gitea gitea admin auth add-oauth
|
||||||
|
--name dex
|
||||||
|
--provider openidConnect
|
||||||
|
--key gitea
|
||||||
|
--secret {{ dex_gitea_client_secret }}
|
||||||
|
--auto-discover-url https://auth.{{ domain }}/.well-known/openid-configuration
|
||||||
|
--scopes "openid profile email"
|
||||||
|
chdir: /opt/services
|
||||||
|
when: gitea_auth_check.rc != 0
|
||||||
|
changed_when: true
|
||||||
|
|
||||||
|
# --- Gitea Actions runner setup ---
|
||||||
|
|
||||||
|
- name: Generate runner registration token
|
||||||
|
ansible.builtin.shell:
|
||||||
|
cmd: docker compose exec -T -u git gitea gitea actions generate-runner-token 2>/dev/null
|
||||||
|
chdir: /opt/services
|
||||||
|
register: runner_token_result
|
||||||
|
changed_when: false
|
||||||
|
when: not runner_token_file.stat.exists
|
||||||
|
|
||||||
|
- name: Save runner token
|
||||||
|
ansible.builtin.copy:
|
||||||
|
content: "{{ runner_token_result.stdout | trim }}"
|
||||||
|
dest: /opt/services/.runner-token
|
||||||
|
mode: "0600"
|
||||||
|
when: not runner_token_file.stat.exists
|
||||||
|
|
||||||
|
# Re-template compose with real token and restart runner
|
||||||
|
- name: Set real runner token
|
||||||
|
ansible.builtin.set_fact:
|
||||||
|
gitea_runner_token: "{{ runner_token_result.stdout | trim }}"
|
||||||
|
when: not runner_token_file.stat.exists
|
||||||
|
|
||||||
|
- name: Re-deploy docker-compose with real runner token
|
||||||
|
ansible.builtin.template:
|
||||||
|
src: docker-compose.yml.j2
|
||||||
|
dest: /opt/services/docker-compose.yml
|
||||||
|
mode: "0644"
|
||||||
|
when: not runner_token_file.stat.exists
|
||||||
|
|
||||||
|
- name: Restart runner with real token
|
||||||
|
ansible.builtin.command:
|
||||||
|
cmd: docker compose up -d --remove-orphans
|
||||||
|
chdir: /opt/services
|
||||||
|
when: not runner_token_file.stat.exists
|
||||||
|
changed_when: true
|
||||||
|
|
@ -0,0 +1,29 @@
|
||||||
|
{
|
||||||
|
email {{ acme_email }}
|
||||||
|
}
|
||||||
|
|
||||||
|
# Gitea
|
||||||
|
git.{{ domain }} {
|
||||||
|
reverse_proxy gitea:3000
|
||||||
|
}
|
||||||
|
|
||||||
|
# LLDAP web UI
|
||||||
|
ldap.{{ domain }} {
|
||||||
|
reverse_proxy lldap:17170
|
||||||
|
}
|
||||||
|
|
||||||
|
# Dex OIDC provider
|
||||||
|
auth.{{ domain }} {
|
||||||
|
reverse_proxy dex:5556
|
||||||
|
}
|
||||||
|
|
||||||
|
# Outline wiki
|
||||||
|
docs.{{ domain }} {
|
||||||
|
reverse_proxy outline:3000
|
||||||
|
}
|
||||||
|
|
||||||
|
# Static site
|
||||||
|
{{ domain }}, www.{{ domain }} {
|
||||||
|
root * /srv/www
|
||||||
|
file_server
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,51 @@
|
||||||
|
# Dex OIDC provider configuration
|
||||||
|
issuer: https://auth.{{ domain }}
|
||||||
|
|
||||||
|
storage:
|
||||||
|
type: sqlite3
|
||||||
|
config:
|
||||||
|
file: /var/dex/dex.db
|
||||||
|
|
||||||
|
web:
|
||||||
|
http: 0.0.0.0:5556
|
||||||
|
|
||||||
|
connectors:
|
||||||
|
- type: ldap
|
||||||
|
id: lldap
|
||||||
|
name: LLDAP
|
||||||
|
config:
|
||||||
|
host: lldap:3890
|
||||||
|
insecureNoSSL: true
|
||||||
|
bindDN: uid=admin,ou=people,{{ ldap_base_dn }}
|
||||||
|
bindPW: "{{ lldap_admin_password }}"
|
||||||
|
userSearch:
|
||||||
|
baseDN: ou=people,{{ ldap_base_dn }}
|
||||||
|
filter: "(objectClass=person)"
|
||||||
|
username: uid
|
||||||
|
idAttr: uid
|
||||||
|
emailAttr: mail
|
||||||
|
nameAttr: displayName
|
||||||
|
preferredUsernameAttr: uid
|
||||||
|
groupSearch:
|
||||||
|
baseDN: ou=groups,{{ ldap_base_dn }}
|
||||||
|
filter: "(objectClass=groupOfUniqueNames)"
|
||||||
|
userMatchers:
|
||||||
|
- userAttr: DN
|
||||||
|
groupAttr: uniqueMember
|
||||||
|
nameAttr: cn
|
||||||
|
|
||||||
|
staticClients:
|
||||||
|
- id: outline
|
||||||
|
name: Outline
|
||||||
|
secret: "{{ dex_outline_client_secret }}"
|
||||||
|
redirectURIs:
|
||||||
|
- https://docs.{{ domain }}/auth/oidc.callback
|
||||||
|
|
||||||
|
- id: gitea
|
||||||
|
name: Gitea
|
||||||
|
secret: "{{ dex_gitea_client_secret }}"
|
||||||
|
redirectURIs:
|
||||||
|
- https://git.{{ domain }}/user/oauth2/dex/callback
|
||||||
|
|
||||||
|
oauth2:
|
||||||
|
skipApprovalScreen: true
|
||||||
|
|
@ -0,0 +1,182 @@
|
||||||
|
# /opt/services/docker-compose.yml
|
||||||
|
# All services for box.cloud-elves.eu
|
||||||
|
|
||||||
|
services:
|
||||||
|
|
||||||
|
# --- Reverse Proxy ---
|
||||||
|
|
||||||
|
caddy:
|
||||||
|
image: caddy:2
|
||||||
|
restart: unless-stopped
|
||||||
|
ports:
|
||||||
|
- "80:80"
|
||||||
|
- "443:443"
|
||||||
|
- "443:443/udp"
|
||||||
|
volumes:
|
||||||
|
- ./Caddyfile:/etc/caddy/Caddyfile:ro
|
||||||
|
- caddy-data:/data
|
||||||
|
- caddy-config:/config
|
||||||
|
# Static site served from host directory
|
||||||
|
- /opt/www:/srv/www:ro
|
||||||
|
networks:
|
||||||
|
- frontend
|
||||||
|
|
||||||
|
# --- Identity ---
|
||||||
|
|
||||||
|
lldap:
|
||||||
|
image: lldap/lldap:stable
|
||||||
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
- LLDAP_JWT_SECRET={{ lldap_jwt_secret }}
|
||||||
|
- LLDAP_KEY_SEED={{ lldap_key_seed }}
|
||||||
|
- LLDAP_LDAP_USER_PASS={{ lldap_admin_password }}
|
||||||
|
- LLDAP_LDAP_BASE_DN={{ ldap_base_dn }}
|
||||||
|
- LLDAP_HTTP_URL=https://ldap.{{ domain }}
|
||||||
|
volumes:
|
||||||
|
- lldap-data:/data
|
||||||
|
networks:
|
||||||
|
- frontend
|
||||||
|
- identity
|
||||||
|
|
||||||
|
dex:
|
||||||
|
image: dexidp/dex:latest
|
||||||
|
restart: unless-stopped
|
||||||
|
command: ["dex", "serve", "/etc/dex/config.yaml"]
|
||||||
|
volumes:
|
||||||
|
- ./dex-config.yml:/etc/dex/config.yaml:ro
|
||||||
|
- dex-data:/var/dex
|
||||||
|
networks:
|
||||||
|
- frontend
|
||||||
|
- identity
|
||||||
|
depends_on:
|
||||||
|
- lldap
|
||||||
|
|
||||||
|
# --- Gitea ---
|
||||||
|
|
||||||
|
gitea:
|
||||||
|
image: gitea/gitea:{{ gitea_version }}
|
||||||
|
restart: unless-stopped
|
||||||
|
ports:
|
||||||
|
# Gitea SSH on host port 2222
|
||||||
|
- "2222:22"
|
||||||
|
environment:
|
||||||
|
- GITEA__database__DB_TYPE=postgres
|
||||||
|
- GITEA__database__HOST=gitea-db:5432
|
||||||
|
- GITEA__database__NAME=gitea
|
||||||
|
- GITEA__database__USER=gitea
|
||||||
|
- GITEA__database__PASSWD={{ gitea_db_password }}
|
||||||
|
- GITEA__server__DOMAIN=git.{{ domain }}
|
||||||
|
- GITEA__server__SSH_DOMAIN=git.{{ domain }}
|
||||||
|
- GITEA__server__SSH_PORT=2222
|
||||||
|
- GITEA__server__ROOT_URL=https://git.{{ domain }}/
|
||||||
|
# Auto-create Gitea accounts on first SSO login
|
||||||
|
- GITEA__service__ALLOW_ONLY_EXTERNAL_REGISTRATION=true
|
||||||
|
- GITEA__oauth2_client__ENABLE_AUTO_REGISTRATION=true
|
||||||
|
- GITEA__oauth2_client__ACCOUNT_LINKING=auto
|
||||||
|
# Enable Gitea Actions
|
||||||
|
- GITEA__actions__ENABLED=true
|
||||||
|
volumes:
|
||||||
|
- gitea-data:/data
|
||||||
|
- /etc/localtime:/etc/localtime:ro
|
||||||
|
networks:
|
||||||
|
- frontend
|
||||||
|
- gitea-backend
|
||||||
|
depends_on:
|
||||||
|
- gitea-db
|
||||||
|
|
||||||
|
gitea-db:
|
||||||
|
image: postgres:{{ postgres_version }}
|
||||||
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
- POSTGRES_DB=gitea
|
||||||
|
- POSTGRES_USER=gitea
|
||||||
|
- POSTGRES_PASSWORD={{ gitea_db_password }}
|
||||||
|
volumes:
|
||||||
|
- gitea-db:/var/lib/postgresql/data
|
||||||
|
networks:
|
||||||
|
- gitea-backend
|
||||||
|
|
||||||
|
# --- Gitea Runner ---
|
||||||
|
|
||||||
|
gitea-runner:
|
||||||
|
image: gitea/act_runner:latest
|
||||||
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
- GITEA_INSTANCE_URL=http://gitea:3000
|
||||||
|
- GITEA_RUNNER_REGISTRATION_TOKEN={{ gitea_runner_token }}
|
||||||
|
- GITEA_RUNNER_NAME=box-runner
|
||||||
|
volumes:
|
||||||
|
# Runner needs Docker socket to spin up job containers
|
||||||
|
- /var/run/docker.sock:/var/run/docker.sock
|
||||||
|
- gitea-runner-data:/data
|
||||||
|
networks:
|
||||||
|
- frontend
|
||||||
|
depends_on:
|
||||||
|
- gitea
|
||||||
|
|
||||||
|
# --- Outline ---
|
||||||
|
|
||||||
|
outline:
|
||||||
|
image: outlinewiki/outline:{{ outline_version }}
|
||||||
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
- URL=https://docs.{{ domain }}
|
||||||
|
- SECRET_KEY={{ outline_secret_key }}
|
||||||
|
- UTILS_SECRET={{ outline_utils_secret }}
|
||||||
|
- DATABASE_URL=postgres://outline:{{ outline_db_password }}@outline-db:5432/outline
|
||||||
|
- PGSSLMODE=disable
|
||||||
|
- REDIS_URL=redis://outline-redis:6379
|
||||||
|
- FILE_STORAGE=local
|
||||||
|
- FILE_STORAGE_LOCAL_ROOT_DIR=/var/lib/outline/data
|
||||||
|
- FORCE_HTTPS=false
|
||||||
|
- OIDC_CLIENT_ID=outline
|
||||||
|
- OIDC_CLIENT_SECRET={{ dex_outline_client_secret }}
|
||||||
|
- OIDC_AUTH_URI=https://auth.{{ domain }}/auth
|
||||||
|
- OIDC_TOKEN_URI=https://auth.{{ domain }}/token
|
||||||
|
- OIDC_USERINFO_URI=https://auth.{{ domain }}/userinfo
|
||||||
|
- OIDC_DISPLAY_NAME=Cloud Elves SSO
|
||||||
|
- OIDC_SCOPES=openid profile email offline_access
|
||||||
|
volumes:
|
||||||
|
- outline-data:/var/lib/outline/data
|
||||||
|
networks:
|
||||||
|
- frontend
|
||||||
|
- outline-backend
|
||||||
|
depends_on:
|
||||||
|
- outline-db
|
||||||
|
- outline-redis
|
||||||
|
- dex
|
||||||
|
|
||||||
|
outline-db:
|
||||||
|
image: postgres:{{ postgres_version }}
|
||||||
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
- POSTGRES_DB=outline
|
||||||
|
- POSTGRES_USER=outline
|
||||||
|
- POSTGRES_PASSWORD={{ outline_db_password }}
|
||||||
|
volumes:
|
||||||
|
- outline-db:/var/lib/postgresql/data
|
||||||
|
networks:
|
||||||
|
- outline-backend
|
||||||
|
|
||||||
|
outline-redis:
|
||||||
|
image: redis:{{ redis_version }}
|
||||||
|
restart: unless-stopped
|
||||||
|
networks:
|
||||||
|
- outline-backend
|
||||||
|
|
||||||
|
networks:
|
||||||
|
frontend:
|
||||||
|
identity:
|
||||||
|
gitea-backend:
|
||||||
|
outline-backend:
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
caddy-data:
|
||||||
|
caddy-config:
|
||||||
|
lldap-data:
|
||||||
|
dex-data:
|
||||||
|
gitea-data:
|
||||||
|
gitea-db:
|
||||||
|
gitea-runner-data:
|
||||||
|
outline-data:
|
||||||
|
outline-db:
|
||||||
|
|
@ -0,0 +1,75 @@
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
# backup.sh — Create a full backup of all Cloud Elves services
|
||||||
|
#
|
||||||
|
# Usage: ./scripts/backup.sh [destination_dir]
|
||||||
|
#
|
||||||
|
# Creates a timestamped tarball containing volume-level backups of all services.
|
||||||
|
# Run from the ce-infra repo root directory.
|
||||||
|
# Requires SSH access to box.cloud-elves.eu as root.
|
||||||
|
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
HOST="root@box.cloud-elves.eu"
|
||||||
|
SSH_KEY="${SSH_KEY:-$HOME/.ssh/id_ed25519_ce}"
|
||||||
|
SSH="ssh -i $SSH_KEY $HOST"
|
||||||
|
SCP="scp -i $SSH_KEY"
|
||||||
|
TIMESTAMP=$(date +%Y%m%d-%H%M%S)
|
||||||
|
DEST_DIR="${1:-./backups}"
|
||||||
|
BACKUP_NAME="ce-backup-${TIMESTAMP}"
|
||||||
|
REMOTE_TMP="/tmp/${BACKUP_NAME}"
|
||||||
|
|
||||||
|
echo "=== Cloud Elves Backup — ${TIMESTAMP} ==="
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
# Create local destination
|
||||||
|
mkdir -p "${DEST_DIR}"
|
||||||
|
|
||||||
|
# Create remote temp directory
|
||||||
|
echo "[1/7] Preparing..."
|
||||||
|
$SSH "mkdir -p ${REMOTE_TMP}"
|
||||||
|
|
||||||
|
# Stop services for consistent backup (databases stay up for pg_dump)
|
||||||
|
echo "[2/7] Stopping application services for consistent backup..."
|
||||||
|
$SSH "cd /opt/services && docker compose stop gitea outline dex lldap gitea-runner caddy"
|
||||||
|
|
||||||
|
# Gitea: pg_dump + data volume
|
||||||
|
echo "[3/7] Backing up Gitea (database + data)..."
|
||||||
|
$SSH "cd /opt/services && docker compose exec -T gitea-db pg_dump -U gitea gitea | gzip > ${REMOTE_TMP}/gitea-db.sql.gz"
|
||||||
|
$SSH "docker run --rm -v services_gitea-data:/data -v ${REMOTE_TMP}:/backup alpine tar czf /backup/gitea-data.tar.gz -C /data ."
|
||||||
|
|
||||||
|
# Outline: pg_dump + data volume
|
||||||
|
echo "[4/7] Backing up Outline (database + uploads)..."
|
||||||
|
$SSH "cd /opt/services && docker compose exec -T outline-db pg_dump -U outline outline | gzip > ${REMOTE_TMP}/outline-db.sql.gz"
|
||||||
|
$SSH "docker run --rm -v services_outline-data:/data -v ${REMOTE_TMP}:/backup alpine tar czf /backup/outline-data.tar.gz -C /data ."
|
||||||
|
|
||||||
|
# LLDAP: SQLite volume
|
||||||
|
echo "[5/7] Backing up LLDAP (user directory)..."
|
||||||
|
$SSH "docker run --rm -v services_lldap-data:/data -v ${REMOTE_TMP}:/backup alpine tar czf /backup/lldap-data.tar.gz -C /data ."
|
||||||
|
|
||||||
|
# Dex: SQLite volume
|
||||||
|
echo "[6/7] Backing up Dex (auth database)..."
|
||||||
|
$SSH "docker run --rm -v services_dex-data:/dex -v ${REMOTE_TMP}:/backup alpine tar czf /backup/dex-data.tar.gz -C /dex ."
|
||||||
|
|
||||||
|
# Restart services
|
||||||
|
echo "[7/7] Restarting services..."
|
||||||
|
$SSH "cd /opt/services && docker compose up -d"
|
||||||
|
|
||||||
|
# Bundle everything and download
|
||||||
|
echo ""
|
||||||
|
echo "Downloading backup..."
|
||||||
|
$SSH "cd /tmp && tar cf ${BACKUP_NAME}.tar -C ${REMOTE_TMP} ."
|
||||||
|
$SCP "${HOST}:/tmp/${BACKUP_NAME}.tar" "${DEST_DIR}/${BACKUP_NAME}.tar"
|
||||||
|
|
||||||
|
# Clean up remote
|
||||||
|
$SSH "rm -rf ${REMOTE_TMP} /tmp/${BACKUP_NAME}.tar"
|
||||||
|
|
||||||
|
# Show result
|
||||||
|
BACKUP_SIZE=$(du -h "${DEST_DIR}/${BACKUP_NAME}.tar" | cut -f1)
|
||||||
|
echo ""
|
||||||
|
echo "=== Backup complete ==="
|
||||||
|
echo "File: ${DEST_DIR}/${BACKUP_NAME}.tar (${BACKUP_SIZE})"
|
||||||
|
echo ""
|
||||||
|
echo "Contents:"
|
||||||
|
tar tf "${DEST_DIR}/${BACKUP_NAME}.tar" | sed 's/^/ /'
|
||||||
|
echo ""
|
||||||
|
echo "Downtime was ~30 seconds while services were stopped for consistency."
|
||||||
|
|
@ -0,0 +1,113 @@
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
# restore.sh — Restore Cloud Elves services from a backup
|
||||||
|
#
|
||||||
|
# Usage: ./scripts/restore.sh <backup_file>
|
||||||
|
#
|
||||||
|
# Prerequisites:
|
||||||
|
# - Target server already bootstrapped: ansible-playbook playbooks/bootstrap.yml
|
||||||
|
# - Services already deployed: ansible-playbook playbooks/deploy.yml
|
||||||
|
# - This script stops services, replaces data, and restarts
|
||||||
|
#
|
||||||
|
# Run from the ce-infra repo root directory.
|
||||||
|
# Requires SSH access to box.cloud-elves.eu as root.
|
||||||
|
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
if [ $# -eq 0 ]; then
|
||||||
|
echo "Usage: $0 <backup_file>"
|
||||||
|
echo "Example: $0 ./backups/ce-backup-20260317-180000.tar"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
BACKUP_FILE="$1"
|
||||||
|
HOST="root@box.cloud-elves.eu"
|
||||||
|
SSH_KEY="${SSH_KEY:-$HOME/.ssh/id_ed25519_ce}"
|
||||||
|
SSH="ssh -i $SSH_KEY $HOST"
|
||||||
|
SCP="scp -i $SSH_KEY"
|
||||||
|
REMOTE_TMP="/tmp/ce-restore"
|
||||||
|
|
||||||
|
if [ ! -f "${BACKUP_FILE}" ]; then
|
||||||
|
echo "Error: Backup file not found: ${BACKUP_FILE}"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "=== Cloud Elves Restore ==="
|
||||||
|
echo "Backup: ${BACKUP_FILE}"
|
||||||
|
echo ""
|
||||||
|
echo "Contents:"
|
||||||
|
tar tf "${BACKUP_FILE}" | sed 's/^/ /'
|
||||||
|
echo ""
|
||||||
|
echo "WARNING: This will stop all services and replace their data."
|
||||||
|
echo "Press Enter to continue, Ctrl+C to abort."
|
||||||
|
read -r
|
||||||
|
|
||||||
|
# Upload backup
|
||||||
|
echo "[1/7] Uploading backup to server..."
|
||||||
|
$SSH "mkdir -p ${REMOTE_TMP}"
|
||||||
|
$SCP "${BACKUP_FILE}" "${HOST}:${REMOTE_TMP}/backup.tar"
|
||||||
|
$SSH "cd ${REMOTE_TMP} && tar xf backup.tar && rm backup.tar"
|
||||||
|
|
||||||
|
# Stop all services
|
||||||
|
echo "[2/7] Stopping all services..."
|
||||||
|
$SSH "cd /opt/services && docker compose down"
|
||||||
|
|
||||||
|
# Restore LLDAP volume
|
||||||
|
echo "[3/7] Restoring LLDAP (user directory)..."
|
||||||
|
$SSH "docker volume rm services_lldap-data 2>/dev/null || true && docker volume create services_lldap-data"
|
||||||
|
$SSH "docker run --rm -v services_lldap-data:/data -v ${REMOTE_TMP}:/backup alpine tar xzf /backup/lldap-data.tar.gz -C /data"
|
||||||
|
|
||||||
|
# Restore Dex volume
|
||||||
|
echo "[4/7] Restoring Dex (auth database)..."
|
||||||
|
$SSH "docker volume rm services_dex-data 2>/dev/null || true && docker volume create services_dex-data"
|
||||||
|
$SSH "docker run --rm -v services_dex-data:/dex -v ${REMOTE_TMP}:/backup alpine tar xzf /backup/dex-data.tar.gz -C /dex"
|
||||||
|
|
||||||
|
# Restore Gitea data volume
|
||||||
|
echo "[5/7] Restoring Gitea (data + database)..."
|
||||||
|
$SSH "docker volume rm services_gitea-data 2>/dev/null || true && docker volume create services_gitea-data"
|
||||||
|
$SSH "docker run --rm -v services_gitea-data:/data -v ${REMOTE_TMP}:/backup alpine tar xzf /backup/gitea-data.tar.gz -C /data"
|
||||||
|
|
||||||
|
# Start databases only so we can restore into them
|
||||||
|
$SSH "cd /opt/services && docker compose up -d gitea-db outline-db"
|
||||||
|
echo " Waiting for databases to start..."
|
||||||
|
sleep 5
|
||||||
|
|
||||||
|
# Restore Gitea database
|
||||||
|
$SSH "cd /opt/services && docker compose exec -T gitea-db dropdb -U gitea --if-exists gitea"
|
||||||
|
$SSH "cd /opt/services && docker compose exec -T gitea-db createdb -U gitea gitea"
|
||||||
|
$SSH "gunzip -c ${REMOTE_TMP}/gitea-db.sql.gz | docker compose -f /opt/services/docker-compose.yml exec -T gitea-db psql -U gitea gitea"
|
||||||
|
|
||||||
|
# Restore Outline data volume
|
||||||
|
echo "[6/7] Restoring Outline (uploads + database)..."
|
||||||
|
$SSH "docker volume rm services_outline-data 2>/dev/null || true && docker volume create services_outline-data"
|
||||||
|
$SSH "docker run --rm -v services_outline-data:/data -v ${REMOTE_TMP}:/backup alpine tar xzf /backup/outline-data.tar.gz -C /data"
|
||||||
|
|
||||||
|
# Restore Outline database
|
||||||
|
$SSH "cd /opt/services && docker compose exec -T outline-db dropdb -U outline --if-exists outline"
|
||||||
|
$SSH "cd /opt/services && docker compose exec -T outline-db createdb -U outline outline"
|
||||||
|
$SSH "gunzip -c ${REMOTE_TMP}/outline-db.sql.gz | docker compose -f /opt/services/docker-compose.yml exec -T outline-db psql -U outline outline"
|
||||||
|
|
||||||
|
# Start everything
|
||||||
|
echo "[7/7] Starting all services..."
|
||||||
|
$SSH "cd /opt/services && docker compose up -d"
|
||||||
|
|
||||||
|
# Wait for Gitea and regenerate hooks
|
||||||
|
echo " Waiting for services to start..."
|
||||||
|
sleep 10
|
||||||
|
|
||||||
|
echo " Regenerating Gitea hooks..."
|
||||||
|
$SSH "cd /opt/services && docker compose exec -T -u git gitea gitea admin regenerate hooks" || echo " (no repos to regenerate hooks for)"
|
||||||
|
|
||||||
|
# Clean up
|
||||||
|
$SSH "rm -rf ${REMOTE_TMP}"
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "=== Restore complete ==="
|
||||||
|
echo ""
|
||||||
|
echo "Please verify:"
|
||||||
|
echo " - https://ldap.cloud-elves.eu — LLDAP (users should be there)"
|
||||||
|
echo " - https://auth.cloud-elves.eu — Dex (SSO should work)"
|
||||||
|
echo " - https://git.cloud-elves.eu — Gitea (repos, users)"
|
||||||
|
echo " - https://docs.cloud-elves.eu — Outline (documents)"
|
||||||
|
echo " - https://cloud-elves.eu — Static site (deploy via rsync separately)"
|
||||||
|
echo ""
|
||||||
|
echo "Note: Users will need to log in again (SSO sessions were reset)."
|
||||||
Loading…
Reference in New Issue