maarten
/
incus-contrib
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
incus-contrib/notes/ovn-deep-dive.md

605 lines
22 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# OVN Deep-Dive: Incus Cluster Networking Internals
How Incus translates its network abstractions into OVN logical objects,
OVS physical flows, and Geneve tunnels. Based on a live 3-node IncusOS
cluster running OVS 3.6.1 with an OVN overlay network.
## Lab Topology
```
LAN (192.168.100.0/22)
│
├── oc-node-01 192.168.102.140 (Geneve endpoint)
├── oc-node-02 192.168.102.141 (Geneve endpoint)
├── oc-node-03 192.168.102.142 (Geneve endpoint, gateway chassis)
│
└── UPLINK physical network
└── OVN range: 192.168.103.200-210
└── net-prod (10.10.10.0/24) ← OVN overlay
```
**Instances on net-prod:**
| Instance | IP | Node | Role |
|---|---|---|---|
| ffsdn-haproxy-52-01 | 10.10.10.50 | oc-node-01 | HAProxy LB |
| ffsdn-haproxy-52-02 | 10.10.10.51 | oc-node-02 | HAProxy LB |
| nginx-lb-01 | 10.10.10.60 | oc-node-01 | Nginx backend |
| nginx-lb-02 | 10.10.10.61 | oc-node-02 | Nginx backend |
| nginx-lb-03 | 10.10.10.62 | oc-node-03 | Nginx backend |
| test-app-manual-web-tier-app-1 | 10.10.10.4 | oc-node-03 | App server |
| test-app-manual-web-tier-web-1 | 10.10.10.2 | oc-node-01 | Web server |
| test-app-manual-web-tier-web-2 | 10.10.10.3 | oc-node-02 | Web server |
## OVN Northbound Database (Logical Layer)
The NB database describes the desired network topology. Incus creates and
manages all objects here — users never touch OVN directly.
### Incus → OVN Naming Convention
Incus uses a consistent naming scheme. For a network named `net-prod`
with internal ID `net8`:
| Incus Concept | OVN Object | OVN Name |
|---|---|---|
| OVN network | Logical Switch (internal) | `incus-net8-ls-int` |
| OVN uplink | Logical Switch (external) | `incus-net8-ls-ext` |
| OVN gateway | Logical Router | `incus-net8-lr` |
| Instance NIC | Logical Switch Port | `incus-net8-instance-<UUID>-eth0` |
| Router↔int switch | Router Port + Switch Port | `incus-net8-lr-lrp-int` / `incus-net8-ls-int-lsp-router` |
| Router↔ext switch | Router Port + Switch Port | `incus-net8-lr-lrp-ext` / `incus-net8-ls-ext-lsp-router` |
| Provider connection | Localnet Port | `incus-net8-ls-ext-lsp-provider` |
| Network forward LB | Load Balancer | `incus-net8-lb-<VIP>-<proto>` |
The numeric `net8` suffix is an internal Incus identifier for the network.
It increments as networks are created.
### Logical Switches
**Internal switch** (`incus-net8-ls-int`): The overlay network where all
instances connect. Subnet 10.10.10.0/24, IPv6 fd42:842b:1e5f:b27::/64.
```
incus-net8-ls-int
├── 8 instance ports (one per container on net-prod)
├── 1 router port (incus-net8-ls-int-lsp-router)
├── 1 load balancer attached (VIP 192.168.103.200:80)
├── 15 ACL rules (Incus baseline security)
├── 8 DNS records (forward + reverse per instance)
└── DHCP options (IPv4 lease=3600, MTU=1442, DNS=192.168.100.1)
```
Key config in `other_config`:
- `subnet=10.10.10.0/24` — Incus IPAM range
- `exclude_ips=10.10.10.1 10.10.10.254 ...` — reserved IPs (gateway, broadcast, static assignments)
- `ipv6_prefix=fd42:842b:1e5f:b27::/64` — SLAAC prefix
**External switch** (`incus-net8-ls-ext`): Bridges the logical router to
the physical UPLINK network.
```
incus-net8-ls-ext
├── 1 localnet port (incus-net8-ls-ext-lsp-provider)
│ └── options: network_name=UPLINK
└── 1 router port (incus-net8-ls-ext-lsp-router)
└── nat_addresses: 10:66:6a:d4:30:c7 192.168.103.200
is_chassis_resident("cr-incus-net8-lr-lrp-ext")
```
The `localnet` port type is special — it tells OVN to bridge this logical
switch to a physical network via `ovn-bridge-mappings` on each chassis.
The mapping `UPLINK:incusovn7` connects it to the OVS provider bridge.
### Logical Router
**Router** (`incus-net8-lr`): Connects internal overlay to external UPLINK.
```
incus-net8-lr
├── lrp-int: 10.10.10.1/24, fd42:842b:1e5f:b27::1/64
│ MAC: 10:66:6a:d4:30:c7, MTU: 1442
│ IPv6 RA: periodic, DHCPv6 stateless, DNSSL=incus
│
├── lrp-ext: 192.168.103.200/22
│ MAC: 10:66:6a:d4:30:c7, MTU: 1442
│ HA Chassis Group: incus-net8 (gateway on oc-node-03)
│
├── NAT:
│ └── SNAT: 10.10.10.0/24 → 192.168.103.200 (stateful)
│
├── Static Routes:
│ ├── 0.0.0.0/0 → 192.168.100.1 via lrp-ext (default gateway)
│ └── 192.168.103.200/32 → 10.10.10.1 (hairpin VIP route)
│
├── Load Balancer: (same LB as on ls-int)
│ └── 192.168.103.200:80 → 10.10.10.50:80, 10.10.10.51:80
│
├── Policies:
│ ├── priority 600: allow ip4.src == $incus_net8_routes_ip4
│ ├── priority 600: allow ip6.src == $incus_net8_routes_ip6
│ └── priority 500: drop (inport == lrp-int) ← deny-by-default
│
└── Options:
├── always_learn_from_arp_request=false
└── dynamic_neigh_routers=true
```
**Router policies explained:** The priority-500 drop rule blocks all
traffic from the internal network by default. The priority-600 allow rules
create exceptions for the network's own IP ranges. This prevents instances
from spoofing source IPs outside their assigned ranges.
**Hairpin VIP route:** The `192.168.103.200/32 → 10.10.10.1` route
handles the case where an instance on net-prod accesses the VIP from
inside the overlay. Without this, the router would try to route the VIP
out the external port, but the packet originated internally.
### Load Balancer
```
Name: incus-net8-lb-192.168.103.200-tcp
Protocol: tcp
VIPs: 192.168.103.200:80 → 10.10.10.50:80, 10.10.10.51:80
```
Incus creates this LB from `incus network forward` or from Aether's
HAProxy deployment. The LB is attached to both the logical switch
(ls-int) and the logical router (lr). This dual attachment ensures
the LB intercepts traffic regardless of where it enters:
- **On the router**: Catches external traffic (LAN → VIP)
- **On the switch**: Catches internal traffic (instance → VIP)
### DHCP and DNS
**DHCP Options** (IPv4):
- Server: 10.10.10.1, MAC: 10:66:6a:d4:30:c7
- Lease time: 3600s, MTU: 1442 (Geneve overhead: 1500 - 58 = 1442)
- DNS: 192.168.100.1 (upstream resolver from UPLINK config)
- Domain: incus, search list: incus
**DHCP Options** (IPv6):
- Stateless DHCPv6 (SLAAC for addresses, DHCPv6 for DNS)
- DNS: fd42:842b:1e5f:b27::1
**DNS Records**: Incus creates forward and reverse DNS entries for every
instance. The DNS server runs inside the OVN router (lrp-int port).
```
ffsdn-haproxy-52-01.incus → 10.10.10.50, fd42:842b:1e5f:b27:1266:6aff:fe67:3482
ffsdn-haproxy-52-02.incus → 10.10.10.51, fd42:842b:1e5f:b27:1266:6aff:fe5f:418f
nginx-lb-01.incus → 10.10.10.60, fd42:842b:1e5f:b27:1266:6aff:fe2a:da10
nginx-lb-02.incus → 10.10.10.61, fd42:842b:1e5f:b27:1266:6aff:fe69:888e
nginx-lb-03.incus → 10.10.10.62, fd42:842b:1e5f:b27:1266:6aff:fe90:4ab8
```
### ACLs (Baseline Security)
Incus creates a default ACL set on the internal switch. These are not
user-configurable — they form the baseline security policy:
| Priority | Direction | Match | Action | Purpose |
|---|---|---|---|---|
| 200 | to-lport | `arp \|\| nd` | allow | ARP/ND always allowed |
| 200 | to-lport | `icmp4.type == {3,11,12} && ip.ttl == 255` | allow | ICMP errors |
| 200 | to-lport | `igmp && ip.ttl == 1 && ip4.mcast` | allow | IGMP |
| 200 | to-lport | Router port ping echo reply | allow | Router ping |
| 200 | to-lport | DHCP to router port | allow | DHCP relay |
| 200 | to-lport | DNS to router port | allow | DNS queries |
| 200 | to-lport | `nd_ra` from router | allow | IPv6 RA |
| 200 | to-lport | `nd_rs` to router | allow | IPv6 RS |
| 200 | to-lport | `tcp.flags == 0x014` | allow | TCP RST+ACK |
All ACLs are `to-lport` (ingress to the logical port), which means they
filter traffic arriving at a port. There are no explicit `from-lport`
rules — the default for egress is allow.
## OVN Southbound Database (Physical Layer)
The SB database maps logical topology to physical infrastructure. It's
computed by `ovn-northd` from the NB database and consumed by
`ovn-controller` on each chassis.
### Chassis
Each Incus cluster member registers as a chassis:
| Chassis | Hostname | Geneve IP | OVS UUID |
|---|---|---|---|
| b840b5b2-... | oc-node-01 | 192.168.102.140 | 298b76a9-... |
| 4652b51f-... | oc-node-02 | 192.168.102.141 | 5ee7906c-... |
| 3f7400f9-... | oc-node-03 | 192.168.102.142 | fd0c3d36-... |
All chassis use Geneve encapsulation with checksum enabled (`csum=true`).
Each chassis advertises `ovn-bridge-mappings=UPLINK:incusovn7`.
Key `other_config` values:
- `datapath-type=system` — kernel datapath (not DPDK)
- `ovn-bridge-mappings=UPLINK:incusovn7` — maps logical "UPLINK" network to OVS bridge
- `ovn-monitor-all=false` — each chassis only gets flows relevant to its local ports
- `ct-commit-nat-v2=true`, `ct-next-zone=true` — modern conntrack features
### Datapath Bindings (Tunnel Keys)
Each logical switch/router gets a unique tunnel key used inside Geneve:
| Tunnel Key | Datapath | OVN Object |
|---|---|---|
| 1 | incus-net8-lr | Logical Router |
| 2 | incus-net8-ls-ext | External Switch |
| 3 | incus-net8-ls-int | Internal Switch |
When a packet traverses a Geneve tunnel, the tunnel key (VNI) identifies
which datapath it belongs to.
### Port Bindings
Port bindings map logical ports to physical chassis:
| Logical Port | Type | Chassis | Tunnel Key |
|---|---|---|---|
| Instance ports (8 total) | VIF | Respective chassis | 2-10 |
| cr-incus-net8-lr-lrp-ext | chassisredirect | oc-node-03 | 2 |
| incus-net8-lr-lrp-ext | patch | (distributed) | 1 |
| incus-net8-lr-lrp-int | patch | (distributed) | 3 |
| incus-net8-ls-ext-lsp-provider | localnet | (all chassis) | 2 |
| incus-net8-ls-ext-lsp-router | patch | (distributed) | 1 |
**Key port types:**
- **VIF (Virtual Interface)**: Regular instance ports, bound to a specific
chassis. Each has a MAC+IP pair for the instance.
- **patch**: Connects two OVN datapaths (switch↔router). Distributed —
processed locally on every chassis.
- **chassisredirect**: The gateway port. Centralizes external-facing
traffic on a single chassis for SNAT/DNAT. Bound to oc-node-03.
- **localnet**: Maps to a physical network. Present on every chassis
via `ovn-bridge-mappings`.
### HA Chassis Group (Gateway Failover)
The gateway port uses an HA chassis group for failover:
```
HA Chassis Group: incus-net8
├── oc-node-03 priority 22372 ← active gateway
├── oc-node-02 priority 18011 ← first failover
└── oc-node-01 priority 12213 ← second failover
```
Incus assigns random-looking priorities (likely based on a hash). If
oc-node-03 goes down, the `cr-incus-net8-lr-lrp-ext` binding migrates
to oc-node-02, then oc-node-01 if needed. BFD (Bidirectional Forwarding
Detection) between chassis enables fast failure detection.
### Logical Flows (Compiled Pipeline)
The lflow pipeline is the compiled form of all NB rules. For ls-int ingress:
| Table | Name | Key Rules |
|---|---|---|
| 0 | ls_in_check_port_sec | Drop multicast src, VLAN tagged |
| 1 | ls_in_apply_port_sec | Enforce port security |
| 4 | ls_in_pre_acl | Skip ACL for router port traffic |
| 5 | ls_in_pre_lb | Pre-process for LB (set reg0[2] for CT) |
| 6 | ls_in_pre_stateful | **LB intercept**: dst=192.168.103.200:80 → `ct_lb_mark` |
| 7 | ls_in_acl_hint | Compute ACL hints from conntrack state |
| 8 | ls_in_acl | Apply ACLs, track connections |
The critical LB flow in table 6:
```
priority=120, match=(reg0[2] == 1 && ip4.dst == 192.168.103.200 && tcp.dst == 80)
action=(reg1 = 192.168.103.200; reg2[0..15] = 80; ct_lb_mark;)
```
This intercepts packets to the VIP and sends them through conntrack
load balancing, which selects a backend and rewrites the destination.
## OVS Physical Layer
Each IncusOS node runs OVS 3.6.1 with two bridges.
### Bridge Architecture
```
Per-node OVS layout:
incusovn7 (provider bridge) br-int (integration bridge)
├── incusovn7 ← physical NIC ├── veth* ← instance NICs
├── incusovn7b ← internal port ├── ovn-* ← Geneve tunnels
└── patch-...-to-br-int ←──patch──→ ├── patch-br-int-to-...
└── br-int ← internal port
```
**`br-int` (integration bridge)**:
- `fail_mode: secure` — drops all traffic if OVN controller disconnects
- Contains all instance veth ports, Geneve tunnel ports, and the patch port
to the provider bridge
- All OVN logical processing (ACLs, NAT, LB, routing) happens here via
OpenFlow rules installed by `ovn-controller`
**`incusovn7` (provider bridge)**:
- Named after the Incus-managed OVS bridge (`incusovn` + network ID)
- Contains the physical NIC port, an internal port, and a patch port to br-int
- Single OpenFlow rule: `priority=0 actions=NORMAL` (standard L2 switching)
- This bridge is the on-ramp/off-ramp between OVN and the physical network
### Geneve Tunnels (Full Mesh)
Every pair of chassis has a Geneve tunnel with BFD health monitoring:
```
oc-node-01 ←── Geneve (UDP 6081) ──→ oc-node-02
↑ ↑
└──────── Geneve (UDP 6081) ──────────┘
↓
oc-node-03
```
| Source | Destination | OVS Port Name | BFD State |
|---|---|---|---|
| .140 (node-01) | .141 (node-02) | ovn-4652b5-0 | forwarding=true |
| .140 (node-01) | .142 (node-03) | ovn-3f7400-0 | forwarding=true |
| .141 (node-02) | .140 (node-01) | ovn-b840b5-0 | forwarding=true |
| .141 (node-02) | .142 (node-03) | ovn-3f7400-0 | forwarding=true |
| .142 (node-03) | .140 (node-01) | ovn-b840b5-0 | forwarding=true |
| .142 (node-03) | .141 (node-02) | ovn-4652b5-0 | forwarding=true |
Tunnel port names use the first 6 hex chars of the remote chassis name.
Options: `key=flow` (tunnel key set per-packet from OVN datapath), `csum=true`.
### Veth Port Mapping
Each instance on net-prod gets a veth pair: one end in the container's
network namespace, the other plugged into br-int. The OVS `external_ids`
field links the veth to its OVN logical port.
**oc-node-01:**
| veth | OVN Port (iface-id) | Instance | OVS ofport |
|---|---|---|---|
| veth8bd9abf3 | ...c446bd6a...-eth0 | nginx-lb-01 (10.10.10.60) | 8 |
| vethfc99b1ec | ...b90abddd...-eth0 | test-web-tier-web-1 (10.10.10.2) | 5 |
| veth07cf17cf | ...0880f911...-eth0 | ffsdn-haproxy-52-01 (10.10.10.50) | 9 |
**oc-node-02:**
| veth | OVN Port (iface-id) | Instance |
|---|---|---|
| veth3352cb4f | ...a2cad635...-eth0 | ffsdn-haproxy-52-02 (10.10.10.51) |
| vethc0b4d30a | ...07270515...-eth0 | nginx-lb-02 (10.10.10.61) |
| vethd2307cca | ...b2cbf869...-eth0 | test-web-tier-web-2 (10.10.10.3) |
**oc-node-03:**
| veth | OVN Port (iface-id) | Instance |
|---|---|---|
| veth9306f6ef | ...292ce9ce...-eth0 | nginx-lb-03 (10.10.10.62) |
| veth4701597a | ...1f056d1a...-eth0 | test-web-tier-app-1 (10.10.10.4) |
## Packet Trace: LAN → VIP → HAProxy → Nginx → Return
Complete path for an HTTP request from a LAN client to
`http://192.168.103.200/` (the HAProxy VIP).
### 1. Client → Gateway Chassis
```
Client (192.168.1.x) sends TCP SYN to 192.168.103.200:80
→ LAN switch forwards to oc-node-03 (gateway chassis)
192.168.103.200 is announced by oc-node-03 via the
cr-incus-net8-lr-lrp-ext chassisredirect port
→ Packet enters physical NIC → incusovn7 bridge
→ OVS NORMAL action → patch port → br-int
```
Why oc-node-03? The external router port (`lrp-ext`) uses HA chassis
scheduling, and oc-node-03 has the highest priority (22372). OVN makes
oc-node-03's OVS respond to ARP for 192.168.103.200, directing all
external traffic to this node.
### 2. br-int → OVN Router (External Processing)
```
br-int receives packet on patch port
→ OpenFlow table 0: classify as localnet traffic (metadata=0x2, ls-ext)
→ Pipeline: ls-ext ingress → router pipeline
→ Router receives on lrp-ext (192.168.103.200/22)
```
### 3. OVN Load Balancer DNAT
```
Router detects dst=192.168.103.200:80 matches LB
→ ct_lb_mark: conntrack creates new entry
→ DNAT: rewrite dst to 10.10.10.50:80 or 10.10.10.51:80
(round-robin selection, conntrack-aware)
→ Packet now has dst=10.10.10.50:80 (say, haproxy-01)
```
The LB is processed in the router pipeline because it's attached to both
the router and the internal switch. For external traffic, the router
processes it first.
### 4. Router → Internal Switch
```
Router forwards via lrp-int (10.10.10.1)
→ Enters ls-int pipeline
→ ACL check (baseline allows established connections)
→ Destination lookup: 10.10.10.50 → ffsdn-haproxy-52-01
→ Port binding: haproxy-01 is on oc-node-01
```
### 5. Geneve Tunnel (Cross-Chassis)
Since the gateway is on oc-node-03 but the destination (haproxy-01) is
on oc-node-01:
```
br-int on oc-node-03:
→ Output action: tunnel to oc-node-01
→ Geneve encapsulate:
- Outer: src=192.168.102.142, dst=192.168.102.140, UDP:6081
- VNI (tunnel key): 3 (ls-int datapath)
- TUN_METADATA0: encodes reg14 (destination port) + reg15
→ Physical NIC sends to LAN
oc-node-01 receives Geneve packet:
→ br-int decapsulates
→ Restores metadata from tunnel headers
→ Delivers to veth07cf17cf (haproxy-01's veth, ofport 9)
→ Packet enters container's network namespace
```
### 6. HAProxy Processing
```
HAProxy receives HTTP request on 10.10.10.50:80
→ HAProxy selects backend: nginx-lb-01 (10.10.10.60)
→ Opens new TCP connection to 10.10.10.60:80
→ Proxies request
```
### 7. HAProxy → Nginx (Same or Different Chassis)
If nginx-lb-01 is on the same node (oc-node-01):
```
→ br-int local delivery (no tunnel needed)
→ veth8bd9abf3 (nginx-lb-01, ofport 8)
```
If HAProxy chose nginx-lb-02 (oc-node-02) or nginx-lb-03 (oc-node-03):
```
→ Geneve tunnel to remote node
→ Same encap/decap as step 5
```
### 8. Return Path
```
Nginx response → HAProxy (reverse of step 7)
→ HAProxy response → OVN (enters ls-int as src=10.10.10.50)
→ ls-int → router (dst is LAN client, matches default route)
→ Router SNAT: src 10.10.10.50 → 192.168.103.200
→ lrp-ext → ls-ext → provider bridge → physical NIC → LAN
→ Client receives response from 192.168.103.200
```
**Important**: The return path from HAProxy to the LAN client goes through
the gateway chassis (oc-node-03) because SNAT is centralized there. If
HAProxy is on oc-node-01, the reply tunnels to oc-node-03 for SNAT, then
exits to the LAN.
## MTU: Why 1442?
Standard Ethernet MTU is 1500 bytes. Geneve adds 58 bytes of overhead:
```
Outer Ethernet: 14 bytes
Outer IP: 20 bytes
Outer UDP: 8 bytes
Geneve header: 16 bytes (8 base + 8 metadata)
─────────
Total overhead: 58 bytes
Inner MTU: 1442 bytes (1500 - 58)
```
Incus sets `bridge.mtu=1442` on net-prod and propagates this via:
- DHCP option: `mtu=1442`
- Router port option: `gateway_mtu=1442`
- IPv6 RA: `mtu=1442`
## Inspection Tools
### Using ovn-inspect
The `incusos/helpers/ovn-inspect` script provides structured inspection:
```bash
# NB database (logical topology)
incusos/helpers/ovn-inspect --nb
# SB database (physical bindings)
incusos/helpers/ovn-inspect --sb
# OVS on each node (deploys temp containers, cleans up after)
incusos/helpers/ovn-inspect --ovs
# Everything
incusos/helpers/ovn-inspect --full
# Trace a VIP's packet path
incusos/helpers/ovn-inspect --trace 192.168.103.200
# Dry run (show commands without executing)
incusos/helpers/ovn-inspect --nb --dry-run
```
### Manual Commands
All NB/SB commands go through the `ovn-central` container:
```bash
# NB: Logical topology
incus exec oc-node-01:ovn-central -- ovn-nbctl ls-list
incus exec oc-node-01:ovn-central -- ovn-nbctl lr-list
incus exec oc-node-01:ovn-central -- ovn-nbctl lsp-list incus-net8-ls-int
incus exec oc-node-01:ovn-central -- ovn-nbctl lr-nat-list incus-net8-lr
incus exec oc-node-01:ovn-central -- ovn-nbctl lb-list
incus exec oc-node-01:ovn-central -- ovn-nbctl acl-list incus-net8-ls-int
# SB: Physical bindings
incus exec oc-node-01:ovn-central -- ovn-sbctl show
incus exec oc-node-01:ovn-central -- ovn-sbctl list Chassis
incus exec oc-node-01:ovn-central -- ovn-sbctl list Port_Binding
incus exec oc-node-01:ovn-central -- ovn-sbctl lflow-list incus-net8-ls-int
# OVS: Physical layer (requires privileged container with /run/openvswitch)
ovs-vsctl show
ovs-ofctl dump-flows br-int
ovs-vsctl get Interface <veth> external_ids
```
### Incus CLI Cross-Reference
```bash
# See OVN network config
incus network show oc-node-01:net-prod --target oc-node-01
# See UPLINK config
incus network show oc-node-01:UPLINK --target oc-node-01
# See which instances use net-prod
incus list oc-node-01: -f csv -c n4l | grep "10.10.10"
# Network forwards (LB VIPs managed via Incus)
incus network forward list oc-node-01:net-prod
```
## Key Architectural Insights
1. **Incus fully manages OVN**: Users never interact with OVN directly.
`incus network create`, `incus network forward`, and instance NIC
configs translate to OVN objects automatically.
2. **Single gateway chassis**: All external traffic (SNAT, DNAT, LB for
external VIPs) is centralized on one node. This is a potential
bottleneck but simplifies state management. HA failover handles node
failures.
3. **Distributed routing for internal traffic**: East-west traffic
between instances on the same switch is fully distributed. No traffic
goes through the gateway unless it needs SNAT/DNAT.
4. **LB is in OVN, not HAProxy**: The VIP load balancing between HAProxy
instances is done by OVN's built-in L4 LB (conntrack-based). HAProxy
then does L7 load balancing to nginx backends. This is a two-tier LB
architecture.
5. **BFD for fast failover**: All Geneve tunnels have BFD enabled, which
detects chassis failures in ~3×detection-interval (typically <1s),
much faster than relying on OVN cluster heartbeats.
6. **MTU must be consistent**: The 1442 byte MTU is critical. If any
path (physical switch, hypervisor NIC) has MTU < 1500, Geneve
encapsulated packets will be fragmented or dropped.
Reference in New Issue View Git Blame Copy Permalink