Skip to content

VM-to-Operator Migration

Cutting a client over from the hand-provisioned Ansible-managed VM to this operator, without a client-side firewall change.

This runbook is unverified against a live cutover

Everything below follows directly from the reconciler's documented contract (EnsureFIP's pinning behavior, spec.endpoint.floatingIP), and the underlying FIP-pinning mechanism is covered by v0.1.0's live end-to-end validation — but the specific VM→operator cutover sequence below has not itself been exercised against a real cutover. Treat this as the plan, not a track record. See Known Limitations.

Why a pinned FIP makes this safe

The client's firewall/parameter sheet points at our floating IP, not at any internal address. As long as the same FIP answers IKE afterward, the client's own configuration needs zero changes — the cutover is entirely on our side. spec.endpoint.floatingIP exists precisely for this: it pins an existing floating IP instead of allocating a new one from a pool.

EnsureFIP's pinning contract (see internal/openstack.Client):

  • The named IP must already exist in the project — an error if not found.
  • It must not be associated with a different port — an error if so (this is what makes "pre-stage, then cutover" the right shape below, rather than trying to create-and-associate in one step while the VM still holds it).
  • A pinned FIP is treated as customer-owned: the operator's Cleanup never releases it, only disassociates — deleting the IPsecGateway later does not return the address to the pool.

Runbook

1. Pre-stage the CR without the FIP association taking effect yet

Create the IPsecGateway CR with spec.endpoint.floatingIP set to the VM's current floating IP, but do not disassociate it from the VM yet. EnsureFIP will fail (association error: already associated with a different port) until the VM releases it — that's expected and safe.

Reconcile runs its steps strictly sequentially and returns on the first failure (see internal/controller's Reconcile), in this fixed order: security group → port → FIP → NAD/render/ConfigMaps → ServiceAccount/ RoleBinding/NetworkPolicy → Deployment. EnsureFIP is the third step, so while it fails, only the security group and the port exist — the NAD, the rendered swanctl.conf/NAT ConfigMaps, and the Deployment (hence the pod) are not created yet. This is safe (nothing here touches the client's traffic path, which is still on the VM) but it means there is no pod to exec into during pre-stage.

kubectl apply -f client-gateway.yaml   # spec.endpoint.floatingIP: <the VM's existing FIP>
kubectl get ipsecgateway <name> -w

Confirm PortReady=True and FIPAttached=False/EnsureFIPFailed — that's the expected, safe pre-stage state, not a fault to chase down:

kubectl get ipsecgateway <name> -o jsonpath='{.status.conditions}' | jq .

ConfigRendered and DeploymentReady will show as absent (never reached yet) rather than False — don't expect a pod, a rendered ConfigMap, or a swanctl --list-conns to check at this stage; there is nothing to exec into until EnsureFIP succeeds in step 2 below.

Future improvement: render ahead of FIP attach

spec.endpoint.floatingIP is already known at pre-stage time (it's the VM's existing FIP, passed in the CR). A future change could render swanctl.conf's local_id from that pinned spec value directly, rather than waiting on EnsureFIP's return value as RenderSwanctl's fip argument does today — letting the NAD, rendered config, and Deployment materialize during pre-stage so their content could be reviewed before cutover. This is not implemented; today rendering only happens after EnsureFIP succeeds.

2. Cutover — disassociate from the VM, let the operator associate

openstack floating ip unset --port <vm-port-id> <the-fip>

EnsureFIPFailed is a failStep return (a non-nil reconcile error), so controller-runtime's own exponential-backoff requeue keeps retrying EnsureFIP automatically — no manual nudge needed, though a trivial no-op edit (kubectl label ipsecgateway <name> cutover=$(date +%s) --overwrite) forces an immediate retry instead of waiting out the backoff. Once it succeeds, the same FIP is associated with the gateway pod's port — FIPAttached and (if reverse exposures/routed mode apply) RoutingReady go True.

3. Verify the tunnel re-establishes

kubectl get ipsecgateway <name> -o jsonpath='{.status.tunnels}' | jq .
kubectl exec -n <tenant-ns> deploy/<name> -c strongswan -- swanctl --list-sas

The client's peer should re-negotiate against the same public IP within its own DPD/retry window — no action needed on their end if peer.address and the CR's tunnel config match what was running on the VM. See Tunnel-down triage if it doesn't come up within a few minutes.

4. Decommission the VM

Only after the tunnel is confirmed ESTABLISHED and traffic is flowing through the new gateway pod — stop the VM (don't delete it immediately; keep it available for rollback until the cutover has soaked).

Rollback

If the operator-side gateway doesn't come up cleanly:

# 1. Take the FIP back off the gateway's port
openstack floating ip unset --port <gateway-port-id> <the-fip>

# 2. Re-associate it with the VM
openstack floating ip set --port <vm-port-id> <the-fip>

# 3. Restart the VM's ipsec service if it was stopped

The client's firewall never needed to change in either direction — the same FIP just points at whichever backend currently holds the port association. The IPsecGateway CR can be left in place (still trying EnsureFIP, harmlessly failing with "already associated") while you diagnose, or deleted if you want a clean slate for a retry — deleting it does not release the FIP back to the pool (pinned FIPs are never released by Cleanup), so the VM re-association in step 2 is unaffected either way.