Set Kubernetes control-plane upgrade timeout to 210s

In the case of a rare intermittent failure behaviour during the
upgrading control plane step where puppet hits timeout first before
the upgrade is completed or kubeadm hits its own Upgrade Manifest
timeout (at 5m).

This change sets puppet timeouts slightly larger than the
engineered kubeadm timeout settings. Typical puppet apply times
are less than 90 seconds, though we have seen infrequent outliers
hit the default 5m timeout.

We engineer the timeout for kubeadm-upgrade-apply and
kubeadm-upgrade-node to 210 seconds,  based on setting 3 minute
kubeadm UpgrademManifestTimeout and 30 second buffer.

Note: 'kubeadm-upgrade-apply' and 'kubeadm-upgrade-node' take the
      same amount of time for the control-plane upgrade.

TEST PLAN:
PASS: Perform k8s upgrade and verify puppet does not timeout
      before kubeadm-upgrade-apply and kubeadm-upgrade-node .

Partial-Bug: 2056326

Change-Id: Iec60476c964140f7b717c6d4dcdb266b0229b556
Signed-off-by: Saba Touheed Mujawar <sabatouheed.mujawar@windriver.com>
This commit is contained in:
Saba Touheed Mujawar 2024-03-07 11:39:15 -05:00
parent d298294244
commit 6c15b7a41b
1 changed files with 2 additions and 2 deletions

View File

@ -849,7 +849,7 @@ class platform::kubernetes::upgrade_first_control_plane
--allow-experimental-upgrades --allow-release-candidate-upgrades -y",
logname => 'kubeadm-upgrade-apply.log',
environment => 'KUBECONFIG=/etc/kubernetes/admin.conf',
timeout => 600,
timeout => 210,
}
-> exec { 'purge all kubelet-config except most recent':
environment => [ 'KUBECONFIG=/etc/kubernetes/admin.conf' ],
@ -895,7 +895,7 @@ class platform::kubernetes::upgrade_control_plane
command => 'kubeadm -v6 upgrade node',
logname => 'kubeadm-upgrade-node.log',
environment => 'KUBECONFIG=/etc/kubernetes/admin.conf:/etc/kubernetes/kubelet.conf',
timeout => 300,
timeout => 210,
}
}