Commit Graph

56 Commits

Author SHA1 Message Date
Takashi Kajinami 52ae921e28 Drop dependency on pytz
Current usage of pytz can be easily replaced by the built-in datetime
library and this allows reducing dependency on 3rd party libraries.

Change-Id: I74c5b8ebce7600cc5986f48a9874ab1882a49ed4
2024-02-28 11:35:24 +09:00
Roman Dobosz a3095c0485 Added pytz to requirements.
Change-Id: Ife9e645d6d9dbf298f08201366a2075d74f80258
2024-02-26 09:53:57 +01:00
Zuul 3da4cce87e Merge "Remove SR-IOV support" 2022-07-21 13:11:07 +00:00
Michał Dulko e478f1a5aa Update minimum openstacksdk version to 0.59.0
This commit bumps openstacksdk to at least 0.59.0 in requirements.txt
and removes the workarounds that we carried to actually support 0.36.0.

Change-Id: I5a5a37b621521cab9d7a8b7634b9392fc70b3c39
2022-07-12 14:42:28 +02:00
Michał Dulko 04d4439606 Remove SR-IOV support
This got decided at the PTG. The code is old, not maintained, not tested
and most likely doesn't work anymore. Moreover it gave us a hard
dependency on grpcio and protobuf, which is fairly problematic in Python
and gave us all sorts of headaches.

Change-Id: I0c8c91cdd3e1284e7a3c1e9fe04b4c0fbbde7e45
2022-06-29 12:49:37 +02:00
Ghanshyam Mann 6ac2b4a813 Drop lower-constraints.txt and its testing
As discussed in TC PTG[1] and TC resolution[2], we are
dropping the lower-constraints.txt file and its testing.
We will keep lower bounds in the requirements.txt file but
with a note that these are not tested lower bounds and we
try our best to keep them updated.

[1] https://etherpad.opendev.org/p/tc-zed-ptg#L326
[2] https://governance.openstack.org/tc/resolutions/20220414-drop-lower-constraints.html#proposal

Change-Id: I91c271d46a2b596c1c007b741363ba51f0b1690a
2022-04-30 20:20:07 -05:00
Michał Dulko 87981d0652 Do not start kuryr-daemon when worker_num <= 1
We've discovered that running kuryr-daemon with [cni_daemon]worker_num=1
breaks pyroute2.IPDB's ability to correctly close threads, leading to a
process leak. This commit makes sure kuryr-daemon will fail to start
when worker_num <= 1.

This required a few more changes in order to make sure that when any
kuryr-daemon subservice dies, kuryr-daemon will shutdown too.

Change-Id: I41afc6fa67abfff62d2f0017db508051a1e7edf4
2021-11-05 14:25:22 +01:00
Iury Gregory Melo Ferreira 2cb2f6a049 Fix l-c and requirements job
- update grpcio in requirements.txt
- update cffi, greenlet and grpcio in l-c to avoid problems
when trying to build the wheels.
- adds .stestr to gitignore

Change-Id: Id2d1c3ea60c33803a35082b9064ccba982b014a7
2021-10-05 21:44:48 +02:00
Zuul dfe6daf2e6 Merge "Export Prometheus metrics" 2021-10-05 17:11:35 +00:00
Maysa Macedo 935d917d48 Export Prometheus metrics
This commit export kuryr_cni_requests_duration, kuryr_quota_free_count,
kuryr_port_quota_per_subnet, kuryr_pod_creation_latency metrics to
Prometheus.

Depends-On: https://review.opendev.org/c/openstack/kuryr-kubernetes/+/811557

Change-Id: I8c8c98153eaf6dd2418efe124c875c1ff9c66205
2021-10-04 09:26:31 +00:00
Roman Dobosz c70a81aa90 Drop constrain of versions <0.8 for PrettyTable.
We had restricted version for <0.8 and >0.7.2 (which is 8 years old
btw), which probably breaks the interface, although current version,
which is 2.2.1 works just fine with kuryr-status command. With this
patch, we allow to install latest version, so that it will not break
requirements-check test.

Change-Id: I5542db6829e01b4767fe1df11b421446bc26b6a3
2021-09-29 08:41:44 +02:00
Maysa Macedo ccc220bbbd Fix CI issues
Bump requirements and lower-constraints libs to allow kuryr to work with
the new pip resolver.

Also update the DevStack plugin to query for docker's cgroup driver
correctly, because the old way suddenly stopped working.

Change-Id: I4f176e2eda194b060abc33e7d1541078cb78d97f
2020-12-10 18:59:13 +01:00
Michał Dulko 9f722e6200 Periodically fetch full list of watched resources
Kuryr-Kubernetes relies on watching resources in K8s API using an HTTP
stream served by kube-apiserver. In such a distributed system this is
sometimes unstable and e.g. etcd issues can cause some events to be
omitted. To prevent controller from such situations this patch makes
sure that periodically a full list of resources is fetched and injected
as events into the handlers.

We should probably do the same for kuryr-daemon watcher, but that case
is less problematic as it'll be restarted in event of ADD requests
timing out.

Change-Id: I67874d086043071de072420df9ea5e86b3f2582e
2020-06-30 12:31:32 +02:00
Roman Dobosz ded6b6debc Removing six library.
Since we already migrated fully to Python3, it's time to also remove
bits needed for Python2. One of those libs is six.

Change-Id: Ib984d7b4b3c1048ed091c78986c634689a8ace8c
2020-02-28 14:45:46 +01:00
Gary Loughnane edc6597fe2 Add DPDK support for nested pods
Add DPDK support for nested K8s pods. Patch includes a new VIF driver on
the controller and a new CNI binding driver.

This patch introduces dependency from os-vif v.1.12.0, since there
a new vif type.

Change-Id: I6be9110192f524325e24fb97d905faff86d0cfef
Implements: blueprint nested-dpdk-support
Co-Authored-By: Kural Ramakrishnan <kuralamudhan.ramakrishnan@intel.com>
Co-Authored-By: Marco Chiappero <marco.chiappero@intel.com>
Signed-off-by: Alexey Perevalov <a.perevalov@samsung.com>
Signed-off-by: Danil Golov <d.golov@samsung.com>
2020-02-04 10:59:45 +03:00
Michał Dulko fb602cd350 Bump openstacksdk to 0.36.0
We have numerous issues with openstacksdk compatibility, let's bump it
to Train version = 0.36.0. Also some other deps need to be bumped too.

Change-Id: Ibdae122c565b294622ebdcadf2e562a37cccd4d1
2019-12-20 12:04:13 +01:00
Roman Dobosz 7bc2cd23d3 Bump OpenStackSDK version to 0.17
We need at least 0.17, since there is a lot of improvements since 0.13,
which we need for smooth transition from clients use to OpenStackSDK.

Change-Id: Ia86fe9ae075a7feba2f15c0804f6b9b37af12e61
Implements: blueprint switch-to-openstacksdk
2019-11-28 16:46:32 +01:00
Michał Dulko 322394050e Fix lower-constraints gate after switch to py36
To make lower-constraints gate work with Python 3 we need to upgrade
eventlet to 0.22.0. Also it's now safe to drop Python 2 entry for
Sphinx.

Change-Id: I50663c428cf5524526055374e53964b86609f653
Depends-On: https://review.opendev.org/#/c/693631/
2019-11-19 12:37:52 +01:00
Danil Golov b53ea8760e Use pyroute2 to tweak vf in sriov
Since possibility to operate with virtual functions
was introduced in pyroute2 it is not necessary
to use ip program in kuryr-kubernetes anymore.

This commit adds a using of pyroute2 library
in sriov binding driver.

Partially implements: bp use-pyroute2-to-tweak-vf-in-sriov
Change-Id: I3cb2da476a6948dc98b2312a5779e47a410832bf
Signed-off-by: Danil Golov <d.golov@samsung.com>
2019-10-22 17:15:47 +09:00
Michał Dulko 1ec05b5581 Add Tempest gate with lower-constraints
Numerous times we hit issues when testing packaged versions of Kuryr
that use dependencies with versions between our lower-constraints.txt
and global requirements/upper-constraints.txt. Each time we found
issues and that's because our lower-constraints gate was not functional.
To further increase test coverage of lower-constraints this commit adds
a functional gate that will run Kuryr with old dependencies.

This is implemented by simply overwriting UPPER_CONSTRAINTS_FILE
build-arg when building Kuryr containers.

Besides that cotyledon's minimum version is raised to 1.5.0 as we're
actively using a feature added back then (it was in late 2016, so we
should be safe here).

Change-Id: Ic148020d1a0ec98150d8c61b21bbbe44b7be2075
2019-06-14 09:32:52 +02:00
Michał Dulko 533ab7cff5 Fix incombatilibity with openstacksdk <= 0.17.2
We keep 0.13.0 as minimum supported version of openstacksdk in
lower-constraints.txt, but we've found an issue running with 0.17.2. The
issue is fixed by switching from dict notation to object notation when
accessing properties of calls to openstacksdk as only newer versions of
openstacksdk support dict notation.

Another side of the story is that our lower-constraints job doesn't
really install versions from lower-constraints.txt due to a mistake in
tox.ini. Maybe we could have avoided the aformentioned bug if we had
noticed that before. This is fixed by this commit and that also required
me to bump up the pyroute2 to 0.5.3 in upper-constraints.txt as
apparently 0.5.1 isn't py3-compatible. protobuf is bumped to 3.6.0 as
3.5.2 version apparently wasn't compatible with our own code.

The unit tests are now updated to make sure using dict notation to
access openstacksdk objects will fail on lower-constraints job in the
future.

Change-Id: I5113f7574f4d2e450de95494c9287bb7427e67d4
Closes-Bug: 1830398
2019-05-28 09:08:11 +02:00
Ilya Maximets 685f4c456a Add PodResources service client
PodResources client could be used by sriov cni to obtain devices
allocated for container by sriov device-plugin.

KubeletPodResources service is still in alpha, so it should be
explicitly enabled in kubelet feature-gates:
    kubelet --feature-gates KubeletPodResources=true

New config option 'kubelet_root_dir' added to 'sriov' section
that defaults to kubelet default root-dir '/var/lib/kulelet'.
In case kubelet started with non-default root directory passed
via '--root-dir' option, the same value should be configured
in 'kubelet_root_dir'.
Note that if sriov binding driver will be used inside container
'kubelet_root_dir'/pod-resources directory should be mounted
to this container in order to allow communication with kubelet
via gRPC protocol over the unix domain socket.

Partial-Bug: 1826865
Depends-On: https://review.openstack.org/#/c/652629
Change-Id: Icf088b839db079efe9c7647c31be4ead867ed32b
Signed-off-by: Ilya Maximets <i.maximets@samsung.com>
2019-05-20 14:16:04 +03:00
Daniel Mellado 9b3182cfeb Add ipBlock support to NP
This commit adds support for ipBlocks when using Network Policies with
Kuryr-Kubernetes.

Partially Implements: blueprint k8s-network-policies
Change-Id: I4f9078420190521fcba7bbc02540b616c479c0d3
2019-04-16 09:22:36 +02:00
Michał Dulko b90ca1b9b9 Switch Octavia API calls to openstacksdk
For a while we were using a dirty hack to call Octavia API - i.e.  we
were mocking python-neutronclient to use a session pointing to Octavia
endpoint. This was a workaround to save some effort that was needed to
introduce python-octaviaclient dependency.

As Kuryr is strongly tied to Kubernetes version it is serving, we should
try to limit its dependency on OpenStack versions. E.g. we should be
able to cooperate with various versions of Octavia. openstacksdk is a
tool that is designed to do exactly that, adding abstraction layer that
hides differences between APIs and as last resort allowing doing bare
HTTP calls, while taking care of all the Keystone stuff for us.

This commit removes the ugly workaround mentioned in first paragraph and
switches all the LBaaS v2 calls to use openstacksdk's Octavia proxy.
Also some leftovers from LBaaS v2 removal are cleaned up, e.g. logic
behind not using cascade deletion or workaround for LbaaS v2 issues with
pools.

Implements: blueprint switch-to-openstacksdk
Change-Id: Ic0bb56f90fe9effcdcb2ae5db96b8a1ec19738df
2019-03-07 17:38:27 +01:00
Luis Tomas Bolivar d8c04167e4 Upgrade Flask version due to security concerns
Change-Id: I080716f5fcdea5d50b4a5c30ffb9c83ab501da46
2019-02-14 10:09:40 +01:00
Michał Dulko 5358702c26 Add kuryr-status utility for upgrade-checkers
This commit adds kuryr-status utility that can be used to check if
upgrade is possible, convert annotations to new format and rollback
those changes if needed.

Implements: blueprint upgrade-checkers

Change-Id: I7a40a68518d7fbba18146b64befb6f585176ec8d
2018-10-03 15:09:53 +02:00
Michał Dulko 6e30059e14 Try pyroute2 0.5.1
Newer pyroute2 includes a few fixes and improvements, let's see if it's
compatible with us.

Change-Id: I7d92f94decfc65d35d27077b2eb931632b9d53a4
2018-07-16 17:34:16 +00:00
Michał Dulko d78ea128e6 Sync up Flask version with global-requirements
As Flask version was updated we need to update our own requirements.txt
to avoid reqirements-check job to fail on every patch that changes
anything related to requirements.

Hopefully that's the last issue introduced by Flask upgrade.

Change-Id: Ief90d241104fe2d42b3c9504a827ea640c304a7a
2018-07-10 16:35:51 +02:00
Michał Dulko 5b9bb03e77 Update lower-constraints to match requirements.txt
Seems like we've did something wrong when applying initial
lower-constraints.txt and now the job fails with errors about that file
being incompatible with requirements.txt and test-requirements.txt.

This commit attempts to fix that by updating lower-constraints.txt to
match requirements.

Change-Id: I0a10a408a6ae6e015ce5f5243d61cfcff230df44
Closes-Bug: 1766898
2018-04-25 18:01:27 +02:00
OpenStack Proposal Bot 7a73ca0496 Updated from global requirements
Change-Id: I03bd9ca6d9b3fcb48b83763b1ba6ceb5676e3276
2018-03-15 07:06:09 +00:00
Antoni Segura Puimedon 8f453a2dda
cni health: track all cgroup memory usage
The CNI daemon should always be run in its own cgroup. That typically
can take two forms:

- Running inside a container
- Running as a systemd service

This patch changes the way the memory usage is tracked so that both
of the cgroup memberships listed above are supported.

Thanks to using cgroups for tracking the memory usage, we will finally
take into account the CNI daemon children memory usage.

Change-Id: I0ef48742653d5c17ea0cc787ae3a997d5d315c5a
Closes-Bug: 1752939
Signed-off-by: Antoni Segura Puimedon <antonisp@celebdor.com>
2018-03-06 22:24:58 +01:00
Maysa Macedo 1e4b7f1109 Add readiness and liveness checks to CNI.
This patch adds readiness and liveness to CNI. It checks presence
of NET_ADMIN capabilities, IPDB in working order, connection to
Kubernetes API, quantity of CNI add failures, health of CNI
components and existence of memory leaks.

Implements: blueprint cni-daemon-readiness-liveness
Change-Id: I9a4b871d196dbadfed687df93bb3cad97c957bfb
2018-02-20 14:58:53 +00:00
OpenStack Proposal Bot ad72c97f6f Updated from global requirements
Change-Id: I5f60b4b366a9d12bc0fc0a1165f962aca9be39f8
2018-01-27 17:47:08 +00:00
Luis Tomas Bolivar bcc851aacf Add oslo_cache to default_subnet driver
This patch adds oslo_cache to the default subnet driver to
avoid unneed calls to neutron when creating pods (or when
getting subnet information)

Change-Id: I93b6bed424757e4138ba656251ae5da46b857da1
2017-11-27 11:41:00 +01:00
Michał Dulko 2f65d993f3 CNI split - introducing CNI daemon
This commit implements basic CNI daemon service. The aim of this new
entity is to increase scalability of CNI operations by moving watching
for VIF to a separate process.

This commit:
* Introduces kuryr-daemon service
* Implements communication between CNI driver and CNI daemon using HTTP
* Consolidates watching for VIF on CNI side to a single Watcher that
  looks for all the pods on the node it is running on.
* Solves bug 1731485 when running with CNI daemon.
* Enables new service in DevStack plugin
* Provides unit tests for new code.

Follow up patches will include:
- Documentation.
- Support for running in containerized mode.

To test the patch add `enable_service kuryr-daemon` to your DevStack's
local.conf file.

Partial-Bug: 1731485
Co-Authored-By: Janonymous <janonymous.codevulture@gmail.com>
Implements: blueprint cni-split-exec-daemon
Change-Id: I1bd6406dacab0735a94474e146645c63d933be16
2017-11-21 08:50:04 +01:00
OpenStack Proposal Bot e74debff62 Updated from global requirements
Change-Id: I5c19809d0188200a393506c380b6311751627768
2017-09-27 13:41:21 +00:00
OpenStack Proposal Bot ae22fe6ee5 Updated from global requirements
Change-Id: Ia180dc15524a8bf325954f6a97fd8c263b1aba0e
2017-08-24 05:51:17 +00:00
OpenStack Proposal Bot 4437b6542d Updated from global requirements
Change-Id: I00073207c78758c1c1f571addf2b8d2ea268ecf3
2017-07-23 13:43:37 +00:00
OpenStack Proposal Bot de7c47a90c Updated from global requirements
Change-Id: I374165d8682951f9783c1f81b0ffdab3dc331156
2017-06-29 02:19:39 +00:00
OpenStack Proposal Bot 1e2a4af65d Updated from global requirements
Change-Id: I49fb755006365d8630bde1848ce4bf35ae8a1d51
2017-05-25 12:52:17 +00:00
OpenStack Proposal Bot e7bb8e6514 Updated from global requirements
Change-Id: Idf5c9ceaeac924babb547d1dabc220f3509565d0
2017-04-20 16:18:51 +00:00
Janonymous bb46102a03 Implement Guru meditation reports
Guru Meditation report can log runtime configuration of a given process,
along with thread status and greenthread status.
It greatly helps to check the process runtime status.

The usage is simple:
kill -SIGUSR2 {process-id}

With this we can show report of kuryr-kubernetes

Change-Id: I4cb2314bd25d3200781582b1e188139751666fd3
Implements: blueprint oslo-gmr
2017-04-13 03:19:08 +00:00
OpenStack Proposal Bot 3d58376b20 Updated from global requirements
Change-Id: Ief73cee711353c1a75e667df56b04b84803be7fd
2017-03-28 21:30:02 +00:00
OpenStack Proposal Bot ca3ed9b378 Updated from global requirements
Change-Id: Ia800b56977ddd85f2d5c16e62c7a75a3a3c673b7
2017-03-15 12:44:19 +00:00
Antoni Segura Puimedon 5bd2d40f1d
requirements: add pyroute2 to the runtime reqs
This has worked so far because kuryr-lib, which is a requirement of
kuryr-kubernetes has pyroute2 as a requirement. However, since
kuryr-kubernetes is a direct consumer of pyroute2 with its binding code,
we can't pretend this is an indirect dependency. We must include it
here.

Change-Id: I882fa9b7159e01c159a77eb949a0bf1e76f51a81
Signed-off-by: Antoni Segura Puimedon <antonisp@celebdor.com>
2017-03-08 00:50:02 +01:00
OpenStack Proposal Bot 4214037855 Updated from global requirements
Change-Id: If0c3e2b5aff2f7107aeea65cfae7f006ff65ad56
2017-02-11 17:43:48 +00:00
OpenStack Proposal Bot 43d9730e9d Updated from global requirements
Change-Id: If81b7606373a423a2286b3002c617c1ae18f378a
2017-01-16 16:44:21 +00:00
Dongcan Ye 1d25da9008 Update requirements
Remove oslo.i18n and ddt from requirements and test-requirements.

Change-Id: Id3db0ac4d9fdd57f631e2bc4b40dc9f0d3885917
2017-01-12 13:21:29 +08:00
OpenStack Proposal Bot 47880b1abf Updated from global requirements
Change-Id: Ib361081a8545f343d28a1ec1b77994c8126f9893
2016-12-07 13:20:25 +00:00
Ilya Chukhnakov fa03953aff Experimental CNI & VIFBridge binding
This patch provides an experimental CNI driver. It's primary purpose
is to enable development of other components (e.g. functional tests,
service/LBaaSv2 support). It is expected to be replaced with daemon
to configure VIF and connect it to the pods and a small lightweight
client to serve as CNI driver called by Kubernetes.

NOTE: unit tests are not provided as part of this patch as it is yet
unclear what parts of it will be reused in daemon-based
implementation.

Change-Id: Iacc8439dd3aee910d542e48ed013d6d3f354786e
Partially-Implements: blueprint kuryr-k8s-integration
2016-12-05 18:05:22 +00:00