Current usage of pytz can be easily replaced by the built-in datetime
library and this allows reducing dependency on 3rd party libraries.
Change-Id: I74c5b8ebce7600cc5986f48a9874ab1882a49ed4
This commit bumps openstacksdk to at least 0.59.0 in requirements.txt
and removes the workarounds that we carried to actually support 0.36.0.
Change-Id: I5a5a37b621521cab9d7a8b7634b9392fc70b3c39
This got decided at the PTG. The code is old, not maintained, not tested
and most likely doesn't work anymore. Moreover it gave us a hard
dependency on grpcio and protobuf, which is fairly problematic in Python
and gave us all sorts of headaches.
Change-Id: I0c8c91cdd3e1284e7a3c1e9fe04b4c0fbbde7e45
We've discovered that running kuryr-daemon with [cni_daemon]worker_num=1
breaks pyroute2.IPDB's ability to correctly close threads, leading to a
process leak. This commit makes sure kuryr-daemon will fail to start
when worker_num <= 1.
This required a few more changes in order to make sure that when any
kuryr-daemon subservice dies, kuryr-daemon will shutdown too.
Change-Id: I41afc6fa67abfff62d2f0017db508051a1e7edf4
- update grpcio in requirements.txt
- update cffi, greenlet and grpcio in l-c to avoid problems
when trying to build the wheels.
- adds .stestr to gitignore
Change-Id: Id2d1c3ea60c33803a35082b9064ccba982b014a7
We had restricted version for <0.8 and >0.7.2 (which is 8 years old
btw), which probably breaks the interface, although current version,
which is 2.2.1 works just fine with kuryr-status command. With this
patch, we allow to install latest version, so that it will not break
requirements-check test.
Change-Id: I5542db6829e01b4767fe1df11b421446bc26b6a3
Bump requirements and lower-constraints libs to allow kuryr to work with
the new pip resolver.
Also update the DevStack plugin to query for docker's cgroup driver
correctly, because the old way suddenly stopped working.
Change-Id: I4f176e2eda194b060abc33e7d1541078cb78d97f
Kuryr-Kubernetes relies on watching resources in K8s API using an HTTP
stream served by kube-apiserver. In such a distributed system this is
sometimes unstable and e.g. etcd issues can cause some events to be
omitted. To prevent controller from such situations this patch makes
sure that periodically a full list of resources is fetched and injected
as events into the handlers.
We should probably do the same for kuryr-daemon watcher, but that case
is less problematic as it'll be restarted in event of ADD requests
timing out.
Change-Id: I67874d086043071de072420df9ea5e86b3f2582e
Since we already migrated fully to Python3, it's time to also remove
bits needed for Python2. One of those libs is six.
Change-Id: Ib984d7b4b3c1048ed091c78986c634689a8ace8c
Add DPDK support for nested K8s pods. Patch includes a new VIF driver on
the controller and a new CNI binding driver.
This patch introduces dependency from os-vif v.1.12.0, since there
a new vif type.
Change-Id: I6be9110192f524325e24fb97d905faff86d0cfef
Implements: blueprint nested-dpdk-support
Co-Authored-By: Kural Ramakrishnan <kuralamudhan.ramakrishnan@intel.com>
Co-Authored-By: Marco Chiappero <marco.chiappero@intel.com>
Signed-off-by: Alexey Perevalov <a.perevalov@samsung.com>
Signed-off-by: Danil Golov <d.golov@samsung.com>
We have numerous issues with openstacksdk compatibility, let's bump it
to Train version = 0.36.0. Also some other deps need to be bumped too.
Change-Id: Ibdae122c565b294622ebdcadf2e562a37cccd4d1
We need at least 0.17, since there is a lot of improvements since 0.13,
which we need for smooth transition from clients use to OpenStackSDK.
Change-Id: Ia86fe9ae075a7feba2f15c0804f6b9b37af12e61
Implements: blueprint switch-to-openstacksdk
To make lower-constraints gate work with Python 3 we need to upgrade
eventlet to 0.22.0. Also it's now safe to drop Python 2 entry for
Sphinx.
Change-Id: I50663c428cf5524526055374e53964b86609f653
Depends-On: https://review.opendev.org/#/c/693631/
Since possibility to operate with virtual functions
was introduced in pyroute2 it is not necessary
to use ip program in kuryr-kubernetes anymore.
This commit adds a using of pyroute2 library
in sriov binding driver.
Partially implements: bp use-pyroute2-to-tweak-vf-in-sriov
Change-Id: I3cb2da476a6948dc98b2312a5779e47a410832bf
Signed-off-by: Danil Golov <d.golov@samsung.com>
Numerous times we hit issues when testing packaged versions of Kuryr
that use dependencies with versions between our lower-constraints.txt
and global requirements/upper-constraints.txt. Each time we found
issues and that's because our lower-constraints gate was not functional.
To further increase test coverage of lower-constraints this commit adds
a functional gate that will run Kuryr with old dependencies.
This is implemented by simply overwriting UPPER_CONSTRAINTS_FILE
build-arg when building Kuryr containers.
Besides that cotyledon's minimum version is raised to 1.5.0 as we're
actively using a feature added back then (it was in late 2016, so we
should be safe here).
Change-Id: Ic148020d1a0ec98150d8c61b21bbbe44b7be2075
We keep 0.13.0 as minimum supported version of openstacksdk in
lower-constraints.txt, but we've found an issue running with 0.17.2. The
issue is fixed by switching from dict notation to object notation when
accessing properties of calls to openstacksdk as only newer versions of
openstacksdk support dict notation.
Another side of the story is that our lower-constraints job doesn't
really install versions from lower-constraints.txt due to a mistake in
tox.ini. Maybe we could have avoided the aformentioned bug if we had
noticed that before. This is fixed by this commit and that also required
me to bump up the pyroute2 to 0.5.3 in upper-constraints.txt as
apparently 0.5.1 isn't py3-compatible. protobuf is bumped to 3.6.0 as
3.5.2 version apparently wasn't compatible with our own code.
The unit tests are now updated to make sure using dict notation to
access openstacksdk objects will fail on lower-constraints job in the
future.
Change-Id: I5113f7574f4d2e450de95494c9287bb7427e67d4
Closes-Bug: 1830398
PodResources client could be used by sriov cni to obtain devices
allocated for container by sriov device-plugin.
KubeletPodResources service is still in alpha, so it should be
explicitly enabled in kubelet feature-gates:
kubelet --feature-gates KubeletPodResources=true
New config option 'kubelet_root_dir' added to 'sriov' section
that defaults to kubelet default root-dir '/var/lib/kulelet'.
In case kubelet started with non-default root directory passed
via '--root-dir' option, the same value should be configured
in 'kubelet_root_dir'.
Note that if sriov binding driver will be used inside container
'kubelet_root_dir'/pod-resources directory should be mounted
to this container in order to allow communication with kubelet
via gRPC protocol over the unix domain socket.
Partial-Bug: 1826865
Depends-On: https://review.openstack.org/#/c/652629
Change-Id: Icf088b839db079efe9c7647c31be4ead867ed32b
Signed-off-by: Ilya Maximets <i.maximets@samsung.com>
This commit adds support for ipBlocks when using Network Policies with
Kuryr-Kubernetes.
Partially Implements: blueprint k8s-network-policies
Change-Id: I4f9078420190521fcba7bbc02540b616c479c0d3
For a while we were using a dirty hack to call Octavia API - i.e. we
were mocking python-neutronclient to use a session pointing to Octavia
endpoint. This was a workaround to save some effort that was needed to
introduce python-octaviaclient dependency.
As Kuryr is strongly tied to Kubernetes version it is serving, we should
try to limit its dependency on OpenStack versions. E.g. we should be
able to cooperate with various versions of Octavia. openstacksdk is a
tool that is designed to do exactly that, adding abstraction layer that
hides differences between APIs and as last resort allowing doing bare
HTTP calls, while taking care of all the Keystone stuff for us.
This commit removes the ugly workaround mentioned in first paragraph and
switches all the LBaaS v2 calls to use openstacksdk's Octavia proxy.
Also some leftovers from LBaaS v2 removal are cleaned up, e.g. logic
behind not using cascade deletion or workaround for LbaaS v2 issues with
pools.
Implements: blueprint switch-to-openstacksdk
Change-Id: Ic0bb56f90fe9effcdcb2ae5db96b8a1ec19738df
This commit adds kuryr-status utility that can be used to check if
upgrade is possible, convert annotations to new format and rollback
those changes if needed.
Implements: blueprint upgrade-checkers
Change-Id: I7a40a68518d7fbba18146b64befb6f585176ec8d
As Flask version was updated we need to update our own requirements.txt
to avoid reqirements-check job to fail on every patch that changes
anything related to requirements.
Hopefully that's the last issue introduced by Flask upgrade.
Change-Id: Ief90d241104fe2d42b3c9504a827ea640c304a7a
Seems like we've did something wrong when applying initial
lower-constraints.txt and now the job fails with errors about that file
being incompatible with requirements.txt and test-requirements.txt.
This commit attempts to fix that by updating lower-constraints.txt to
match requirements.
Change-Id: I0a10a408a6ae6e015ce5f5243d61cfcff230df44
Closes-Bug: 1766898
The CNI daemon should always be run in its own cgroup. That typically
can take two forms:
- Running inside a container
- Running as a systemd service
This patch changes the way the memory usage is tracked so that both
of the cgroup memberships listed above are supported.
Thanks to using cgroups for tracking the memory usage, we will finally
take into account the CNI daemon children memory usage.
Change-Id: I0ef48742653d5c17ea0cc787ae3a997d5d315c5a
Closes-Bug: 1752939
Signed-off-by: Antoni Segura Puimedon <antonisp@celebdor.com>
This patch adds readiness and liveness to CNI. It checks presence
of NET_ADMIN capabilities, IPDB in working order, connection to
Kubernetes API, quantity of CNI add failures, health of CNI
components and existence of memory leaks.
Implements: blueprint cni-daemon-readiness-liveness
Change-Id: I9a4b871d196dbadfed687df93bb3cad97c957bfb
This patch adds oslo_cache to the default subnet driver to
avoid unneed calls to neutron when creating pods (or when
getting subnet information)
Change-Id: I93b6bed424757e4138ba656251ae5da46b857da1
This commit implements basic CNI daemon service. The aim of this new
entity is to increase scalability of CNI operations by moving watching
for VIF to a separate process.
This commit:
* Introduces kuryr-daemon service
* Implements communication between CNI driver and CNI daemon using HTTP
* Consolidates watching for VIF on CNI side to a single Watcher that
looks for all the pods on the node it is running on.
* Solves bug 1731485 when running with CNI daemon.
* Enables new service in DevStack plugin
* Provides unit tests for new code.
Follow up patches will include:
- Documentation.
- Support for running in containerized mode.
To test the patch add `enable_service kuryr-daemon` to your DevStack's
local.conf file.
Partial-Bug: 1731485
Co-Authored-By: Janonymous <janonymous.codevulture@gmail.com>
Implements: blueprint cni-split-exec-daemon
Change-Id: I1bd6406dacab0735a94474e146645c63d933be16
Guru Meditation report can log runtime configuration of a given process,
along with thread status and greenthread status.
It greatly helps to check the process runtime status.
The usage is simple:
kill -SIGUSR2 {process-id}
With this we can show report of kuryr-kubernetes
Change-Id: I4cb2314bd25d3200781582b1e188139751666fd3
Implements: blueprint oslo-gmr
This has worked so far because kuryr-lib, which is a requirement of
kuryr-kubernetes has pyroute2 as a requirement. However, since
kuryr-kubernetes is a direct consumer of pyroute2 with its binding code,
we can't pretend this is an indirect dependency. We must include it
here.
Change-Id: I882fa9b7159e01c159a77eb949a0bf1e76f51a81
Signed-off-by: Antoni Segura Puimedon <antonisp@celebdor.com>
This patch provides an experimental CNI driver. It's primary purpose
is to enable development of other components (e.g. functional tests,
service/LBaaSv2 support). It is expected to be replaced with daemon
to configure VIF and connect it to the pods and a small lightweight
client to serve as CNI driver called by Kubernetes.
NOTE: unit tests are not provided as part of this patch as it is yet
unclear what parts of it will be reused in daemon-based
implementation.
Change-Id: Iacc8439dd3aee910d542e48ed013d6d3f354786e
Partially-Implements: blueprint kuryr-k8s-integration