This option was deprecated in 4.0.0[1] in favor of the new subnet"s"
option. The latest release is 9.0.0, so we can assume enough times has
been given to users so that they can switch to the new option.
[1] b3814a33d6
Change-Id: Ie86c019bbb560cca9b5a3a77319ed639a2245a2d
Seems like the set of resources we gather from K8s for CI gates
debugging purposes was outdated. This commit updates it to make sure
we're also getting NPs and KLBs, while removing Ingress, KuryrNet and
KuryrNetPolicy that are obsolete today.
Besides that KuryrNet and KuryrNetPolicy CRD definitions are removed as
we're not using them for a long time now.
Change-Id: Ia7ecd8f79b6ac0a8172cf108e228e2ea1cb7a0d5
Before switch to kubeadm we used to use manual approach for installing
all the bits with option for switching container runtimes between docker
and cri-o. With this patch we re-gain that ability with simply setting
appropriate container engine variable for devstack-plugin-contaier to
use either CRI-O or default Docker.
Depends-On: https://review.opendev.org/c/openstack/devstack-plugin-container/+/817231
Change-Id: I273888a7428611b40802dc5dd53fcee864ce43da
Till now, for installing kuryr-kubernetes and one of the crucial service
- kubernetes, there has been used manual method for installing it in
specified version. Over time it became a burden to follow requirements
and constraints, therefore decision has been made to use recommended way
of installing Kubernetes - kubeadm. In this patch devstack installation
of the kuryr-kubernetes and its dependences has been heavily reworked.
Other than that, OpenShift related functions has been removed, since
they were all outdated and non-working for the long time.
Change-Id: Ife21874c0a71ba07723094c0f880aabcf5825b77
This commit deprecates `[pod_vif_nested]worker_nodes_subnet` in favor of
`[pod_vif_nested]worker_nodes_subnets` that will accept a list instead.
All the code using the deprecated options is updated to expect a list
and iterate over possible nodes subnets.
Change-Id: I7671fb06863d58b58905bec43555d8f21626f640
This commit is a huge refactoring of how we handle network policies. In
general:
* KuryrNetPolicy is replaced by KuryrNetworkPolicy. The upgrade path
is handled in the constructor of KuryrNetworkPolicyHandler.
* New CRD has spec and status properties. spec is always populated by
NetworkPolicyHandler. status is handled by KuryrNetworkPolicyHandler.
This means that in order to trigger SG rules recalculation on Pod ang
Service events, the NetworkPolicy is "bumped" with a dummy annotation.
* NetworkPolicyHandler injects finalizers onto NetworkPolicy and
KuryrNetworkPolicy objects, so that objects cannot get removed before
KuryrNetworkPolicyHandler won't process deletion correctly.
Depends-On: https://review.opendev.org/742209
Change-Id: Iafc982e590ada0cd9d82e922c103583e4304e9ce
Till now, we were using pod annotations to store information regarding
state of the associated VIFs to pod. This alone have its own issues and
it's prone to the inconsistency in case of controller failures.
In this patch we propose new CRD called KuryrPort for storage the
information about VIFs.
Depends-On: If639b63dcf660ed709623c8d5f788026619c895c
Change-Id: I1e76ea949120f819dcab6d07714522a576e426f2
This patch moves the namespace handling to be more aligned
with the k8s style.
Depends-on: If0aaf748d13027b3d660aa0f74c4f6653e911250
Change-Id: Ia2811d743f6c4791321b05977118d0b4276787b5
We save logs of all containers running on our K8s, but sometimes those
containers are restarted. To increase debugging abilities this commit
makse sure to also save the logs of previous run of the service.
Change-Id: Ic6a0d72de1b64b1217164f2e75b5a3e018f5a4ce
From time to time in the gate we suffer from Kubernetes/Docker bug [1].
As it seems to still be open, we can work it around by removing usage of
subPath property of volumeMounts attached to Kuryr pods and this commit
does so. Besides that it removes possibility of providing different
kuryr.conf for kuryr-controller and kuryr-daemon as this shouldn't be
required as we don't support running without kuryr-daemon anymore.
[1] https://github.com/kubernetes/kubernetes/issues/68211
Closes-Bug: 1833228
Change-Id: I2465bc45324482cc4ab32a1367ab08f34ce28b1c
This commit does several cleanups to the Dockerfiles that we have:
* git is removed from the images after Kuryr packages installation
* jq and wget is removed from kuryr-cni image as those ar no longer used
* explicit setuptools installation is no longer required
* raw Kuryr code is removed from images after it's `pip install`ed
* unnecessary VOLUME line is removed from kuryr-cni Dockerfile
* CNI_CONFIG_DIR and CNI_BIN_DIR build arguments are removed from
kuryr-cni Dockerfile as they are not used anywhere. Initially we've
kept them to allow deployer to tell where host's /etc/cni/net.d and
/opt/cni/bin will be mounted, but one of the refactorings of
cni_ds_init must have stopped depending on them and we simply started
to expect the mounts to be in the same paths as on host. We can
continue to do that.
The build_cni_daemonset_image script was created back in the time when
we have had multi-stage build of the kuryr-cni image. This is no longer
the case and building the image is as easy as:
`docker build -f cni.Dockerfile .`
Given that this commit removes the script and updates documentation to
recommend using `docker build` directly.
Change-Id: Ib1807344ede11ec6845e5f09c5a87c29a779af03
-D/--no-daemon option in tools/build_cni_daemonset_image should not
require a parameter. This commit fixes that mistake and the HA gate.
Change-Id: I439d75436e4fdf3671e54d979fb45cf8d6193ee1
Closes-Bug: 1807380
This commit adds support for cri-o by changing the binary initially used
to run CNI plugin to runc and falling back to docker only in case it's
not available.
Also DevStack support for installing and configuring Kubernetes with
cri-o is added.
Implements: blueprint crio-support
Depends-On: Ib049d66058429e499f5d0932c4a749820bec73ff
Depends-On: Ic3c7d355a455298f43e37fb2aceddfd1e7eefaf2
Change-Id: I081edf0dbd4eb57826399c4820376381950080ed
This patch adds dockerfiles and helper scripts to build the Kuryr-Kubernetes
containers using Python3 and get it automated in devstack.
Implements: blueprint goal-python36
Change-Id: I2066aacfebc7339d5a86b717327bdad428b0e54c
Signed-off-by: Antoni Segura Puimedon <celebdor@gmail.com>
This patch adds a new subnet driver that creates a new network
for each created k8s namespace. It makes use of K8s CRDs to store
the information about the network resources created for each
namespace
Partially Implements: blueprint network-namespace
Change-Id: I7988e1da7a9ed57f29c85ddcd99bb2c87808010e
Enable the docker_mode and set the netns_proc_dir with '/host_proc'
as the mount point defined in the cni ds definition in configmap.
Closes-Bug: #1769815
Change-Id: I7bc0e92fb69118b9a1e9b58f9cf9cb7a3ea63a39
Signed-off-by: Peng Liu <pliu@redhat.com>
Due to assigning CA_CERTIFICATE_PATH as the CA_CERTIFICATE_PATH if a
path is not passed, if the value was unset, we'd get a null value which
would be passed to the devstack instead of the intended "".
Closes-Bug: #1765639
Change-Id: I6f51ebd0ac7879786a3827fd37fe4953e410a843
Signed-off-by: Antoni Segura Puimedon <antonisp@celebdor.com>
When Kuryr is run in pods, logs of its services are not available in the
gate. This commit adds a post-run playbook that will copy information
about Kubernetes resources and logs of all pods into the results.
Change-Id: Idac654854f0b968fe0c3024fd4f0436279a945a5
Partial-Implements: enhance-upstream-gates
Our containerized gates started failing recently. Turns out some default
configuation was changed and `tls-proxy` service was added. This option
makes all OpenStack endpoints use HTTPS. This includes creation of a
DevStack CA certificates bundle that then will be configured to be
verified when connecting to OpenStack APIs. This works well with
non-containerized deployment as the bundle is available locally in
/opt/stack/data and our `[neutron]` section sets `cafile` option to
point there.
Things are different in containerized deployment use case as we need a
way to pass those certificates into the container. Effectively - we had
no CA certificates support for containerized deployments either in
DevStack or production.
This commit adds that support by including new Kuryr Kubernetes resource
definition - `kuryr-certificates` Secret. It is supposed to hold CA
certificate under `kuryr-ca-bundle.crt` key. kuryr-controller DaemonSet
definition was modified to mount the certificate into /etc/ssl/certs.
Changes also include implementing support for that in DevStack plugin
(placing the certificate in the secret and setting the `[neutron]cafile`
config option to point to that certificate).
Closes-Bug: 1758061
Change-Id: I7ac9d05868994cfc2a1aef4a8cd6c2148895e9c8
This commit implements what was discussed on the PTG, i.e. deprecation
of running Kuryr-Kubernetes without kuryr-daemon services. This commit
includes changes in configuration defaults, sample local.conf files,
documentation, gates and a release note explaining the change.
Change-Id: I152c81797cb83237af4917a4487cb1f1918270aa
This patch adds readiness and liveness to CNI. It checks presence
of NET_ADMIN capabilities, IPDB in working order, connection to
Kubernetes API, quantity of CNI add failures, health of CNI
components and existence of memory leaks.
Implements: blueprint cni-daemon-readiness-liveness
Change-Id: I9a4b871d196dbadfed687df93bb3cad97c957bfb
This commit changes the way we produce kuryr-cni Docker image. Previously we've
distributed the kuryr-driver as pyinstaller binary that contained Python 3
interpreter and all the dependencies. This binary was called from CNI. That
approach had some disadvantages, the major being complicated build procedure
and having to see false-positive BrokenPipeError tracebacks in kubelet
logs.
This commit implements distributing kuryr-driver as a virtualenv with
kuryr-kubernetes and all the dependecies installed. That virtualenv is then
copied onto the host system and CNI can easily activate it and run kuryr-cni
binary. This should solve issues caused by pyinstaller.
Closes-Bug: 1747058
Change-Id: I65b01ba27cbe39b66f0a972d12f3abc166934e62
This patch checks the health of k8s, Keystone and Neutron,
by using a server that combines all the verifications. Also, checks
if ports are loaded into the pools when required.
Partially Implements: blueprint controller-readiness-liveness-probes
Change-Id: I09121a61d23fb64b326dae3947d5d24b1329cde3
This file is completely unused and the new PTI docs job check whether it
exists - and if it does, use it and execute it.
so, now this file is used - and fails to setup.
Remove it to get everything working again.
Change-Id: I863b32e5bccc03cd2ef91f2590b59e24543ba353
This commit implements kuryr-daemon support when
KURYR_K8S_CONTAINERIZED_DEPLOYMENT=True. It's done by:
* CNI docker image installs Kuryr-Kubernetes pip package and adds
exectution of kuryr-daemon into entrypoint script.
* Hosts /proc and /var/run/openvswitch are mounted into the CNI
container.
* Code is changed to use /host_proc instead of /proc when in a container
(it's impossible to mount host's /proc into container's /proc).
Implements: blueprint cni-split-exec-daemon
Change-Id: I9155a2cba28f578cee129a4c40066209f7ab543d
This patch add a readiness probe to the kuryr controller when
the ports pool functionality is enabled. This ensures the
controller pod is not set to ready until all the precreated ports
have been loaded into their respective pools. This helps admins
to know when the kuryr-controller pod is prepared to start serving
requests.
Note the kuryr-controller will reply to request even if it is not
on ready status. However, that will lead to trigger port creation
for new pods as the already existing ports may not be on their
respective pools yet.
Change-Id: Id47d3e7450551c19cb19d9278e459bd32bf364cf
Containerized deployment through DevStack had two bugs related to
mismatches in handling environment variables in Dockerfiles:
1. cni.Dockerfile was using ENV vars to define CNI bin and conf
directories, but when DevStack was building them it wasn't setting them
correctly. This resulted in CNI binaries and configs ending up
in wrong directories when deploying through DevStack. This is fixed by
passing $CNI_BIN_DIR and $CNI_CONF_DIR into the build function.
2. cni_builder script used $CNI_BIN_DIR_PATH, but it was only defined in
cni.Dockerfile and was is missing from cni_builder.Dockerfile. This
resulted in malformed kuryr-cni script, that pointed to non-existing
"/kuryr-cni-bin" file. This is fixed by adding those ENV vars to
cni_builder.Dockerfile
Change-Id: I4833124231f256b74f80bd5fee732686bffab77e
Closes-Bug: 1718137
This makes tox use the latest version of the tool from os-testr repo,
which contains some valuable changes, like the one making the gate fail
if for some reason no tests are executed because all of them are
skipped.
Change-Id: If88be4b310d746ccc885e3d0c1db8a4dfabb2a28
The config generation was from back when we did not have keystone v3
support, which moved around how it all is generated. This patch puts
kuryr-kubernetes in line with the rest of Kuryr.
Change-Id: I877fa57308aa4c2128bb5d12e801e7e566aef108
Closes-bug: #1626014
Signed-off-by: Antoni Segura Puimedon <antonisp@celebdor.com>
Takashi Kajinami