This argument has been defined for containerd clusters in Magnum, and is set to
the default (and only valid) value of 'remote'.
Kubelet warning in 1.26:
* Flag --container-runtime has been deprecated, will be removed in 1.27 as the only valid value is 'remote'
Kubelet error in 1.27:
* E0801 03:10:26.723998 8889 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --container-runtime"
Change-Id: I072fab1342593941414b86e28b8a76edf2b19a6f
klog args have been removed from kubernetes in 1.26, and
deprecated since 1.23. https://github.com/kubernetes/kubernetes/pull/112120
The argument --logtostderr has defaulted to true for a long time, so
this removal on older versions should have no impact.
Change-Id: I64f934a9bbc39c5e054d8a83b3f6edee061469e6
If the kubelet container is restarted on a host (during upgrades, or manually)
the bind mounts duplicate into /rootfs and kubelet cannot unmount these.
This leads to stuck terminating pods that must be resolved with either --force
or restart of kubelet container.
Adding 'rslave' means that when the kubelet unmounts volumes at /var/lib/kubelet/pods
this propogates to the host (using 'rshared'), and back into the container in /rootfs.
This bug was likely introduced when mounting of /rootfs was added[0].
[0] 1994e9448a
Change-Id: I44f80ccc97c0eeab98f1edbe4a22763732b7f4da
Only specify dockershim options when container runtime is not containerd.
Those options were ignored in the past when using containerd but since 1.24
kubelet refuses to start.
Task: 45282
Story: 2010028
Signed-off-by: Daniel Meyerholt <dxm523@gmail.com>
Change-Id: Ib44cc30285c8bd4219d4a45dc956696505ddd570
Fedora CoreOS 34 has switched from cgroups v1 to
cgroups v2 by default, which changes the sysfs hierarchy.
Task: 42809
Story: 2009045
Change-Id: I2f9651421370ba44e2f0ddc7bb6526745b62ad40
* in 1.20 8080 is not supported anymore
** use only 6443
** change all probes for health to use kubectl and 6443
* configure the signing key in API
story: 2008524
task: 41731
Change-Id: Ibaf1840214016d2dd6ac15e2137eb3cd3d767889
Signed-off-by: Spyros Trigazis <spyridon.trigazis@cern.ch>
At present, insecure registry doesn't work as expected when Podman is
used. This patch addresses the issue by fixing the ignition user data so
that Podman is configured correctly. Then it ensures that
--insecure-registry flag is provided to Docker in /etc/sysconfig/docker.
Story: 2008479
Task: 41519
Change-Id: I2e1c86e0c88ab5b59185fd523e9c9696ce0f951e
Create certificates for kubelet and kube-proxy on control-plane
nodes similar to worker nodes. Use the secure kube-apiserver
port on control-plane nodes.
story: 2008524
task: 41602
Change-Id: Ibeb32a24ca25914cab32c63a9ccafaf711148a84
Signed-off-by: Spyros Trigazis <spyridon.trigazis@cern.ch>
There are two issues with current k8s admission controller list:
1. The default existing list is not consistent when user passes
in extra controller or not
2. The existing list is out of date.
The new list are based on below consideration:
1. Get the default list based on k8s v1.16.x[1] because it's the
supported oldest version.
2. Keep it consistent when user passes in extra controllers or not
3. Keep all the admission controllers we has used in the code
[1] https://v1-16.docs.kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#which-plugins-are-enabled-by-default
Task: 40767
Story: 2008076
Change-Id: Ie5b89b97710d2e2d41c9ce4f3ec30046390acbeb
From k8s v1.19.x, kube-apiserver binary can't accept any parameter,
and actually we're not using the pass-in KUBE_API_PORT. So it's
safe to drop it.
Change-Id: I12a0bb3441d18c3b68a8db4ab3234e04e5218cd2
Without this, heat container agents using kubectl version
1.18.x (e.g. ussuri-dev) fail because they do not have the correct
KUBECONFIG in the environment.
Task: 39938
Story: 2007591
Change-Id: Ifc212478ae09c658adeb6ba4c8e8afc8943e3977
In the heat-agent we use kubectl to install
several deployments, it is better if we use
matching versions of kubectl and apiserver
to minimize errors. Additionally, the
heat-agent won't need kubectl anymore.
story: 2007591
task: 39536
Change-Id: If8f6d84efc70606ac0d888c084c82d8c7eff54f8
Signed-off-by: Spyros Trigazis <strigazi@gmail.com>
To mount nfs volumes with the embedded volume
pkg [0], rpc-statd is required and should be
started by mount.nfs. When running kubelet
in a chroot this fails. With atomic containers
it used to work.
[0] https://github.com/kubernetes/kubernetes/tree/master/pkg/volume/nfs
story: 2005201
task: 39403
Change-Id: Ib64efe7ecbe9a24e86fa9d9a35a4d90c0e8bbf2e
Signed-off-by: Spyros Trigazis <strigazi@gmail.com>
Improve the taint of master node kubelet to get the conformance
test passed and update the OCCM and Helm/Tiller tolerations accordingly.
Task: 39223
Story: 2007256
Change-Id: Ief452e05ddf13a1d1ee77641311c3ae7abbe90f2
Kubelet fails to handle SELinux labelling of Cinder PV without
presenting the rootfs to Kubelet and as a result, an unprivileged
container lacks the ability to access the path.
With this patch, Kubelet handles the correct labelling automatically
when a Cinder PV is attached to a pod.
The default behaviour using system containers in Fedora Atomic is to
mount rootfs [1] but we did not implement the same behaviour in Fedora
CoreOS which was a mistake as this was a missing piece of code.
[1] https://github.com/openstack/magnum/blob/master/dockerfiles/kubernetes-kubelet/config.json.template#L335
Story: 2007413
Task: 39129
Change-Id: Id59c604928244bf49773b7519fa756d5b2814b69
Set the max-size for container/pod logs to 10m
and max of 5 rotated files. The values relay
the default of kubernetes when it is using
a remote container runtime [0] (container-log-max-files
and container-log-max-size) This defaults cover the
case of containerd.
[0] https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/
story: 2007402
task: 39031
Change-Id: Ie3106b40b4d1c6866761c507122047e88e513651
Signed-off-by: Spyros Trigazis <strigazi@gmail.com>
Add support for out of tree Cinder CSI. This is installed when the
cinder_csi_enabled=true label is added. This will allow us to eventually
deprecate in-tree Cinder.
story: 2007048
task: 37868
Change-Id: I8305b9f8c9c37518ec39198693adb6f18542bf2e
Signed-off-by: Bharat Kunwar <brtknr@bath.edu>
Adding the volume mount for /etc/machine-id so that the kubelet
boostraped by podman can access the correct instance ID. Without
this, autoscaler will fail to delete empty node. This issue is
reported on autoscaler repo[1].
[1] https://github.com/kubernetes/autoscaler/issues/2819
Task: 38743
Story: 2007286
Change-Id: I2852f4b255e782bb65b13571502194ee9f455ae3
To display the node OS-IMAGE in k8s properly
we need to mount /usr/lib/os-release,
/ets/os-release is just a symlink.
story: 2006459
task: 38505
Change-Id: I0c850126c7299cb7a4fe201efee311d76bc14ce6
Signed-off-by: Spyros Trigazis <spyridon.trigazis@cern.ch>
Upstream k8s images changed the entrypoint to
/hyperkube instead of shell.
Set the entrypoint to /hyperkube which works
for v1.17.x and v1.16.x.
podman inspect k8s.gcr.io/hyperkube:v1.16.0 | grep Entrypoint -A 2
podman inspect k8s.gcr.io/hyperkube:v1.17.0 | grep Entrypoint -A 2
"Entrypoint": [
"/hyperkube"
]
story: 2007031
task: 37834
Change-Id: I021aeeef9f39dd426c1f335161a3d4b3f51670e8
Signed-off-by: Spyros Trigazis <strigazi@gmail.com>
Now Magnum is using podman and systemd to manage the k8s components.
In cases where the nodes pull images from docker.io or another
mirror registry with high latency, some of the components may take long
time to start, which is causing timeout when bootstraping k8s
cluster for fedora atomic/coreos drivers. This patch fixes it by
adding TimeoutStartSec for the systemd services.
Task: 37251
Story: 2006459
Change-Id: I709bac620e4ceec1858672076eb0aef997704b62
Choose whether system containers etcd, kubernetes and the heat-agent will be
installed with podman or atomic. This label is relevant for k8s_fedora drivers.
k8s_fedora_atomic_v1 defaults to use_podman=false, meaning atomic will be used
pulling containers from docker.io/openstackmagnum. use_podman=true is accepted
as well, which will pull containers by k8s.gcr.io.
k8s_fedora_coreos_v1 defaults and accepts only use_podman=true.
Fix upgrade for k8s_fedora_coreos_v1 and magnum-cordon systemd unit.
Task: 37242
Story: 2005201
Change-Id: I0d5e4e059cd4f0458746df7c09d2fd47c389c6a0
Signed-off-by: Spyros Trigazis <spyridon.trigazis@cern.ch>
With this change each node will be labeled with the following:
* --node-labels=magnum.openstack.org/role=${NODEGROUP_ROLE}
* --node-labels=magnum.openstack.org/nodegroup=${NODEGROUP_NAME}
Change-Id: Ic410a059b19a1252cdf6eed786964c5c7b03d01c
Add fedora coreos driver. To deploy clusters with fedora coreos operators
or users need to add os_distro=fedora-coreos to the image. The scripts
to deploy kubernetes on top are the same with fedora atomic. Note that
this driver has selinux enabled.
The startup of the heat-container-agent uses a workaround to copy the
SoftwareDeployment credentials to /var/lib/cloud/data/cfn-init-data.
The fedora coreos driver requires heat train to support ignition.
Task: 29968
Story: 2005201
Signed-off-by: Spyros Trigazis <spyridon.trigazis@cern.ch>
Change-Id: Iffcaa68d385b1b829b577ebce2df465073dfb5a1
Using the atomic cli to install kubelet breaks mount
propagation of secrets, configmaps and so on. Using podman
in a systemd unit works.
Additionally, with this change all atomic commands are dropped,
containers are pulled from gcr.io (ofiicial kubernetes containers).
Finally, after this patch only by starting the heat-agent with
ignition, we can use fedora coreos as a drop-in replacement.
* Drop del of docker0
This command to remove docker0 is carried from
earlier versions of docker. This is not an issue
anymore.
story: 2006459
task: 36871
Change-Id: I2ed8e02f5295e48d371ac9e1aff2ad5d30d0c2bd
Signed-off-by: Spyros Trigazis <spyridon.trigazi@cern.ch>
Due to [0], we can not label nodes with
node-role.kubernetes.io/master="". We need to do it with the kubernetes
API.
[0] https://github.com/kubernetes/kubernetes/issues/75457
story: 2006459
task: 36872
Change-Id: I2dc2a125c49f9fc33aa02d3d0c99a5bb0eec1156
Signed-off-by: Spyros Trigazis <spyridon.trigazi@cern.ch>
Pass the node name to kube-proxy and not repy
on the cloud provider to set it. Kube-proxy needs
to start before the cloud-provider.
Without it kube-proxy fail to find the node
in the kubernete api.
story: 2006459
task: 36873
Change-Id: Ie04d8d99e68ee43c9d407dbd6f746f6249337ba2
Dale Smith