chore(bootstrap): upgrade kubespray to release-2.28 and kubernetes to v1.32.13

- Bump Kubespray baseline to release-2.28 (last release supporting Ubuntu 20.04)
- Upgrade Kubernetes control plane and node to v1.32.13
- Configure single-node cluster settings (drain_nodes: false)
- Add patch for Kubespray v1beta4 kubeadm preflight error handling
- Update init.sh, AGENTS.md, and README.md with two-phase upgrade instructions

bootstrap/README.md

@@ -103,12 +103,60 @@ This runs everything and is kind of idempotent:
 ansible-playbook -i inventory/prod/inventory.ini cluster.yml
 ```
 
-## Upgrade to 2.31.3
+## Upgrade to Kubernetes 1.31 (Previous)
 
 Required to execute: 
 
-```
+```sh
 ansible-playbook -i inventory/prod/inventory.ini -e upgrade_cluster_setup=true -e drain_nodes=false upgrade-cluster.yml
 ```
 
 + set a feature flag: https://github.com/kubernetes-sigs/kubespray/issues/11887
+
+## Upgrade to Kubespray release-2.28 & Kubernetes v1.32.13
+
+> [!NOTE]
+> Upgrade philosophy: decoupled two-phase upgrade.
+> 1. Upgrade Kubespray tooling baseline (keeping k8s version unchanged).
+> 2. Upgrade Kubernetes control plane and worker components.
+>
+> **Important**:
+> - `haumdaucher` is a **single-node cluster**. Always use `-e drain_nodes=false` when running `upgrade-cluster.yml` to prevent the upgrade from hanging on draining.
+> - **Ubuntu 20.04 LTS constraint**: Kubespray `release-2.28` is the **last** release officially supporting Ubuntu 20.04. Upstream Kubespray `release-2.29`+ requires Ubuntu 22.04 or 24.04.
+
+### Phase 1: Kubespray Baseline Upgrade (Retaining k8s v1.31.9)
+1. Ensure `init.sh` has `VERSION="release-2.28"`.
+2. Initialize environment:
+   ```sh
+   . ./init.sh
+   ```
+3. Run cluster playbook to reconcile Kubespray baseline:
+   ```sh
+   ansible-playbook -i inventory/prod/inventory.ini cluster.yml
+   ```
+4. Verify cluster health:
+   ```sh
+   kubectl get nodes -o wide
+   kubectl get pods -A | grep -v -E "Running|Completed"
+   ```
+
+### Phase 2: Kubernetes Upgrade to v1.32.13
+1. In `inventory/prod/group_vars/k8s_cluster/k8s-cluster.yml`, update:
+   ```yaml
+   kube_version: 1.32.13
+   ```
+2. Re-sync inventory to ephemeral `kubespray/`:
+   ```sh
+   rsync -a ./inventory/ ./kubespray/inventory/
+   ```
+3. Run upgrade playbook:
+   ```sh
+   cd kubespray
+   ansible-playbook -i inventory/prod/inventory.ini -e upgrade_cluster_setup=true cluster.yml
+   ```
+4. Verify upgraded cluster:
+   ```sh
+   kubectl version
+   kubectl get nodes -o wide
+   kubectl get pods -A | grep -v -E "Running|Completed"
+   ```