HAProxy Load Balancing (Aether-Managed)

Overview

Aether manages HAProxy deployment: image builds, HA pair, VIP management, L7 service configuration, health monitoring.
Lab cluster: oc-lab-cluster (ID: 52), OVN network: net-prod (10.10.10.0/24).
VIP range: 192.168.103.200-210 (from UPLINK).
deploy-haproxy script: --deploy, --status, --cleanup, --doctor.

Use Playwright for Aether session auth, NOT curl

HAProxy management endpoints use session-authenticated routes with CSRF protection. curl cannot reliably handle this — use Playwright instead.

Playwright MCP: .mcp.json configures @playwright/mcp@latest. When loaded, use browser_navigate, browser_click, browser_fill tools.
Helper: incusos/helpers/aether-browser for standalone automation.
Login page: GET / (root, NOT /login). Form action: POST /login.
CSRF token: hidden <input name="csrf_token"> in HTML form, NOT cookie.

Key UI element IDs (for Playwright automation)

Element	ID	Purpose
Cluster dropdown	`#clusterSelect`	Select cluster to manage
Deploy mode	`#deployMode`	OVN or Keepalived
OVN network	`#deployNetwork`	Select network (populates IP dropdowns)
LB VIP select	`#deployLBVIPSelect`	Choose VIP from UPLINK range
HP01 IP select	`#deployHP01IPSelect`	Choose HAProxy 01 IP
HP02 IP select	`#deployHP02IPSelect`	Choose HAProxy 02 IP
CPU limit	`#deployCPU`	Default: 2
Memory limit	`#deployMemory`	Default: 1 GB
Service VIP	`#serviceVIP`	VIP by ID (e.g., value="41")
Service mode	`#serviceMode`	tcp, http, https-passthrough, https-termination
Service port	`#servicePort`	Listen port
Service balance	`#serviceBalance`	roundrobin, leastconn, source, first
Health check	`#serviceHealthCheck`	Checkbox, default: checked
Backend name	`input[name="backend_name_N"]`	Nth backend name
Backend IP	`input[name="backend_ip_N"]`	Nth backend IP
Backend port	`input[name="backend_port_N"]`	Nth backend port

Key JS functions: showEditServiceModal(serviceId), showServiceHealthStats(serviceId, name), reloadHAProxyConfig(), showAddServiceModal(), showAddVIPModalWithVIPs().

Key API routes (session-authenticated, use via Playwright page.evaluate)

Route	Method	Purpose
`/haproxy/infrastructure/{cluster_id}`	GET	Deployment state (JSON)
`/haproxy/infrastructure/deploy`	POST	Deploy infrastructure
`/haproxy/service`	POST	Create service
`/haproxy/service/{id}`	PUT	Update service
`/haproxy/service/{id}`	DELETE	Delete service
`/haproxy/reload/{cluster_id}`	POST	Reload HAProxy config
`/haproxy/infrastructure/{cluster_id}`	DELETE	Remove infrastructure

API field names (validated 2026-02-24)

Deploy infrastructure (POST /haproxy/infrastructure/deploy): cluster_id (int), ovn_network, lb_vip (NOT vip), haproxy_01_ip, haproxy_02_ip, cpu_limit (string), memory_limit (string, e.g. "1GB")

Create service (POST /haproxy/service): vip_id (int, NOT VIP address), name, description, hostname, listen_port (int), mode, balance_method, health_check_enabled (bool), session_persistence (bool), backends array with objects: { name, target_ip, target_port (int), weight (int) }

Container naming

ffsdn-haproxy-{clusterID}-01 and ffsdn-haproxy-{clusterID}-02 For cluster 52: ffsdn-haproxy-52-01, ffsdn-haproxy-52-02.

Lab IP assignments (live, validated 2026-02-24)

Component	IP
VIP	192.168.103.200
HAProxy 01	10.10.10.50 (oc-node-01)
HAProxy 02	10.10.10.51 (oc-node-02)
nginx-lb-01	10.10.10.60
nginx-lb-02	10.10.10.61
nginx-lb-03	10.10.10.62

Validated gotchas

OVN LB not created by Aether: Aether deploys HAProxy containers and ACLs but does NOT create the OVN load balancer (incus network load-balancer). You must create it manually after deployment. See guide Step 3.
HTTP mode health checks: Aether generates option httpchk (sends OPTIONS /). Nginx returns 405 → backends marked DOWN. Use TCP mode for nginx backends, or configure nginx to accept OPTIONS.
HP-01 may not auto-start: After deploy, ffsdn-haproxy-52-01 was sometimes found Stopped. Check both containers post-deploy. Not always reproducible — both started on the 2026-02-24 re-validation.
OVN VIP external access: Aether ACLs only allow ingress from source: VIP_IP. External traffic preserves original client IP → rejected. Fix: add broad ingress rule to each HAProxy ACL: incus network acl rule add <remote>:<acl> ingress action=allow protocol=tcp destination_port=80,443 Must re-apply after infrastructure redeploy.
Internal VIP not reachable: The manually-created OVN LB handles external-to-internal traffic but does NOT hairpin for internal-to-VIP requests from the same logical switch. Use HAProxy localhost for internal tests.
OVN LB has no health checking: When one HAProxy instance is stopped, the OVN LB continues routing ~50% of connections to the dead backend, causing timeouts. Failover is not instant. Consider Keepalived mode for production deployments requiring true active/standby failover.
Static IPs on OVN: Use ipv4.address device key on the NIC device, not systemd-networkd: incus launch ... -d eth0,ipv4.address=10.10.10.60.

Key differences from deploy-awx

No VM creation: HAProxy uses Aether-managed containers, not a standalone VM.
Playwright auth: Uses browser automation, not JWT or basic auth.
No --heal action: HAProxy is fully Aether-managed. Recovery = --cleanup + --deploy.
Backends via incus CLI: Test nginx backends created directly with ipv4.address device key.

5.9 KiB Raw Blame History