Commit Graph

397 Commits

Author SHA1 Message Date
Tadas Sutkaitis 30094ecc85
Cinder: Enable custom annotations
Enable custom annotations for pods [deployments, daemonsets]

Change-Id: I7e3605d521e6b8e9485c76a06def866fff7c6029
2024-03-26 00:25:27 +02:00
ricolin f65cb36c84 Cinder: Support uWSGI for API server
Currently Cinder API server still using eventlet-based HTTP servers,
it is generally considered more performant and flexible to run them
using a generic HTTP server that supports WSGI.

Change-Id: Iccdfaba66af6e814b7c28bf682b0d2bface2d982
2024-03-05 16:21:52 +08:00
Stephen Taylor ef049413fd [ceph] Update Ceph images to Jammy and Reef 18.2.1
This change updates all Ceph images for Jammy-based deployments in
openstack-helm to latest-ubuntu_jammy.

Change-Id: Id80f0fc074da01548006fc37c2629b27fbddbd25
2024-01-30 08:02:02 -07:00
Vladimir Kozhukalov 82a6aa8ce9 Add 2023.2 (Bobcat) jobs
Change-Id: Iea2a16db8acaa94259aeb3e21097bb771b70c38e
2023-10-13 12:40:12 -05:00
Vladimir Kozhukalov 5aadee0dc8 Add Ubuntu Jammy overrides
Change-Id: Icabf43efee2e64c856ae14a69881d96b380d7751
2023-09-07 00:12:04 +03:00
Sadegh Hayeri 91c8a5baf2 Use service tokens
Change-Id: If81d59cb848ae7e07eb7bcb8d594b5005a7d5528
2023-07-27 07:01:00 +00:00
Vladimir Kozhukalov b1f74a351a Add 2023.1 test jobs
- Also run last two test scripts in compute-kit job
  sequentially. This is handy since it allows to see
  what is happening during the test run. Both these
  test scripts usually take just few minutes. But if
  we run them using ansible async feature and one of
  the scripts fails then we are forced to wait for
  a long timeout.

Change-Id: I75b8fde3ec4e3355319b1c3f257e2d76c36f6aa4
2023-07-04 18:34:31 +03:00
Vladimir Kozhukalov 02a9e1e0ed Enable Zed compute-kit and cinder jobs
Also a new nodeset was temporarily added.
The aio compute-kit jobs for recent releases require
a huge node to work reliably. We'll remove the temporary nodeset
once this is merged
https://review.opendev.org/c/openstack/openstack-helm-infra/+/884989

Change-Id: I7572fc39a8f6248ff7dac44f20076ba74a3499fc
2023-06-01 16:15:50 +03:00
okozachenko 423d91d6c3 Define service_type in keystone_authtoken
If application credentials with access rules are required,
an OpenStack service using keystonemiddleware to authenticate
with keystone, needs to define service_type in its configuration
file.

Change-Id: I7034e82837d724f12d57969857f79d67c962cebe
2023-05-12 23:09:58 +10:00
Samuel Liu 73e696b3fb Replace node-role.kubernetes.io/master with control-plane
The master label is no longer present on kubeadm control plane nodes(v1.24). For new clusters, the label 'node-role.kubernetes.io/master' will no longer be added to control plane nodes, only the label 'node-role.kubernetes.io/control-plane' will be added. For more information, refer to KEP-2067[https://github.com/kubernetes/enhancements/tree/master/keps/sig-cluster-lifecycle/kubeadm/2067-rename-master-label-taint]: Rename the kubeadm "master" label and taint.

the kubernetes pr: https://github.com/kubernetes/kubernetes/pull/107533

Change-Id: Iad8c833371efb3ec35149c89eb8fafdf1150fa87
2023-03-21 09:02:00 +08:00
Stephen Taylor 884a734833 [ceph] Update all Ceph images to Focal
This change updates all Ceph image references to use Focal images
for all charts in openstack-helm.

Change-Id: I67cd294e2aabf3c3af404da42204f9b6157b06f7
2023-03-17 07:50:10 -06:00
Stephen Taylor 46beb87574 Allow Ceph pools to use 1x replication
Beginning with the Pacific release, Ceph pools are not allowed to
use 1x replication by default. This is problematic for the
openstack-helm gate scripts, which frequently use 1x replication
for automated testing. This change adds Ceph configuration and
command overrides to allow those gate scripts to continue to use
1x replication for testing.

Change-Id: I21ed3e43f3773d5ea830959f1b66b35f38185ca7
2023-03-15 07:10:10 -06:00
Mohammed Naser 99cf85550b fix(cinder): solve keyring placement for uppercased backends
Change-Id: Ic384a86f74751dae7ff964688aef30e411fe4b8e
2023-02-23 18:36:50 +00:00
Mohammed Naser 6b504ecd0f fix(cinder): add endpoint details for nova
This change adds the endpoint details for Nova to allow for online
resizes however a very key thing in this is that it is actually
using the Cinder credentials to talk to Nova.

OpenStack projects have historically arbitrarily decided to use
the user credentials of the _target_ service rather than the
source service which does not make sense, a mailing list discussion
seemed to have not brought up any negatives but only positives.

For the future, we can continue to do this which will simplify
our endpoints section but this is a start.

Change-Id: Ib9b500ef9a9bc34c8b64215bee57451494735573
2023-02-23 18:26:59 +00:00
Mohammed Naser a0bb05f41e fix: add hooks for cinder bootstrap jobs
Change-Id: Ib457934ff539e4c7e19690a696415394634766d0
2023-01-31 17:29:06 -05:00
Terekhin, Alexey (at4945) 9216563be2 Fix for creation endpoins and services when v1/v2 are disabled.
Change-Id: I187e882c653c93ad4e1ef83a88ac4fcc3e60f763
2023-01-19 14:09:43 -08:00
ricolin b72f3d0f3c Avoid unrequired policy setup
OpenStack services already moved to use policy in code.
No need to have policy file at this point, at least no need to put
default policy rule to policy.yaml file anymore.
To put in duplicate rules, will cause unnecessay logs and process.
Also not healthy for policy in code maintain as the `default` rules in
openstack-helm might override actual default rules in code which we
might not even mean to change it at all.

Change-Id: I29ea57aa80444ed64673818e597c9ca346ba7b2f
2022-11-23 22:43:10 +08:00
Ritchie, Frank (fr801x) 5ad407ab0e Change ceph-config-helper image tag
Change ceph-config-helper image tag to bionic from focal. There is no
focal based image available.

Change-Id: I6c4039161d09a55cac84fc047080603953a1deb5
2022-11-22 13:13:06 -05:00
Gage Hugo 5ffefb60c1 Remove train and ussuri overrides
We dropped train support a long time ago now, and our latest efforts
are to drop ussuri/bionic images. This change removes any leftover
train overrides as well as any ussuri overrides. This also changes
any image defaults to use wallaby.

Change-Id: I818a3a79faa631ec1b7de625f2113c6f19610760
2022-10-24 16:00:59 -05:00
Mohammed Naser 6a5466e9a4 Revert "Fix cinder host name"
This reverts commit d554d74bf0.

This change is very problematic, because then the volumes will no longer be able to have API operations work on them, since the volumes are tied to a host, so when that host it down, things are bad.

We should keep this old (non-clean behaviour) and switch towards using Cinder clusters instead https://docs.openstack.org/cinder/latest/contributor/high_availability.html

This means configuring a few other things, but this is a really bad change and will break production deployments.  I suggest we revert ASAP.

Change-Id: Iee54ef5feca5c8bee80aded75d2fd182a431adb3
2022-09-12 17:41:00 +00:00
okozachenko d554d74bf0 Fix cinder host name
Now `cinder-volume-worker` is set as DEFAULT.host. It results only
one host per volume service type registered regardless of replica
numbers.

To fix this, removed DEFAULT.host from default values so hostname
of pods will be used as host name.
It will produce garbage services with down state because pod name
changes every creation. To avoid this, added a cronjob for service
clean.

Change-Id: I9ec6f62105124f9088afdb231099b532fc83bb34
2022-09-06 19:47:50 +10:00
josebb b5ae94c0f7 Specify configmap name for external ceph configuration in cinder
Now we need to add external ceph configuration in values as yaml
format, then it is converted to ini format and added in cinder-etc
configmap.

Instead, we can just specify the pre-existing configmap name.
Configmap name takes precedence over plain configuration.

Change-Id: Ica1973798223207f6a453613a600d121db25edea
2022-09-02 19:07:20 +03:00
josebb 227eac2a77 Add SYS_ADMIN cap in cinder-volume container in default
Retyping volume with on-deman migration policy requires SYS_ADMIN
cap in c-vol

Change-Id: Ie002de788926879c7fd02b6213762a40da2b30ac
2022-09-02 19:01:25 +03:00
okozachenko f3ed56cc18 Use HTTP probe instead of TCP probe
Strictly speaking, open socket doesn't mean working API.
We experienced API stopped responding and the socket was still
open so API was unhealthy actually but kubernetes did not restart.

HTTP probe will fix this issue.

Change-Id: I95bb3ad3123d8a4a784d260477f037fa5506d290
2022-09-01 15:54:07 +10:00
josebb f16ac05866 Support TLS endpoints in cinder
This allows cinder to consume TLS openstack endpoints.
Jobs consume openstack endpoints, typically identity endpoints.
And cinder itself interact with other openstack services via
endpoints.

Change-Id: Id5668f9dde1f63fe472fef639571936de831e217
2022-08-18 13:17:12 +03:00
josebb d2be022999 Distinguish between port number of internal endpoint and binding
port number in cinder

Now binding ports of service and pod spec are configured using
internal endpoint values.
To support reverse proxy for internalUrl, need to distinguish
between binding ports and internal endpoint ports.

I added `service` section in endpoint items apart from admin,public
,internal and default.

Change-Id: I40c3fc822e8ea9ef0eca8c15afe0d12b8bc926ff
2022-08-18 11:02:24 +03:00
okozachenko 830755a3c3 Fix volume list in cinder-volume deployment
Volumes (pod-shared and cinder-conversion) should be created
regardless of ceph backend existence but those are created
conditionally.
This bug has been existing but never revealed because conditional
check for ceph backend was wrong before. After this change
https://review.opendev.org/c/openstack/openstack-helm/+/852809,
started happening.

Change-Id: Ifc90e40bc201245d3d9e2b472573948ff64e0c61
2022-08-17 17:09:16 +10:00
okozachenko 93bad61c4e Fix conditional check for ceph backend
Current check uses cinder.utils.has_ceph_backend template which
returns bool-like string values("true" or "false"), and consider it
as bool type. So it is always true regardless of whether there is
ceph backend or not.

To fix this, this change uses string comparision.

Change-Id: Ie2e54c00d536874562eb93e70a2836cac102c992
2022-08-12 00:21:02 +10:00
Brian Haley ced30abead Support image registries with authentication
Based on spec
support-OCI-image-registry-with-authentication-turned-on.rst

Each Helm chart can configure an OCI image registry and
credentials to use. A Kubernetes secret is then created with this
info. Service Accounts then specify an imagePullSecret specifying
the Secret with creds for the registry. Then any pod using one
of these ServiceAccounts may pull images from an authenticated
container registry.

Related OSH-infra change:
https://review.opendev.org/c/openstack/openstack-helm-infra/+/848142

Change-Id: I54540f14fed29622bc5af8d18939afd06d65e2d8
2022-08-11 00:18:37 +00:00
Gage Hugo 89addfd4e1 Add Xena and Yoga values overrides
This change adds the overrides needed to run both the Xena and
Yoga releases in the OSH zuul jobs.

Change-Id: I65e016a4cb3fd52707ab29c37f025818fcb6c405
2022-06-08 17:21:57 +00:00
Schubert Anselme 8d5ddc9035
Migrate CronJob resources to batch/v1 and PodDisruptionBudget resources to policy/v1
This change updates the following charts to migrate CronJob resources to the batch/v1 API version, available since v1.21. [0]
and to migrate PodDisruptionBudget to the policy/v1 API version, also available since v1.21. [1]

- aodh (CronJob & PodDisruptionBudget)
- barbican (PodDisruptionBudget)
- ceilometer (PodDisruptionBudget)
- cinder (CronJob & PodDisruptionBudget)
- cyborg (PodDisruptionBudget)
- designate (PodDisruptionBudget)
- glance (PodDisruptionBudget)
- heat (CronJob & PodDisruptionBudget)
- horizon (PodDisruptionBudget)
- Ironic (PodDisruptionBudget)
- Keystone (CronJob & PodDisruptionBudget)
- magnum (PodDisruptionBudget)
- masakari (PodDisruptionBudget)
- mistral (PodDisruptionBudget)
- neutron (PodDisruptionBudget)
- nova (CronJob & PodDisruptionBudget)
- octavia (PodDisruptionBudget)
- placement (PodDisruptionBudget)
- rally (PodDisruptionBudget)
- senlin (CronJob & PodDisruptionBudget)

0: https://kubernetes.io/docs/reference/using-api/deprecation-guide/#cronjob-v125
1: https://kubernetes.io/docs/reference/using-api/deprecation-guide/#poddisruptionbudget-v125

Change-Id: I2fc0692e1c8e2c4fa4d4ca1da96b5c6a832343fa
2022-05-19 10:08:18 -04:00
Phil Sphicas 3e63be0682 Allow cinder v1/v2 endpoint creation if needed
An earlier change [0] removed the older cinder v1 and v2 endpoint
definitions from values.yaml. If older APIs are required, they can be
enabled using overrides, but the ks-service job only creates a volumev3
endpoint.

This change updates the ks-service job to create all endpoints defined
in .Values.endpoints that have "volume" in their name. Note that several
settings are hardcoded to use volumev3, so this change would mostly
useful in enabling v2 in addition to v3.

0: https://review.opendev.org/c/openstack/openstack-helm/+/817310

Change-Id: I26594668ff26ed3f28e207f341b28a139b514e1c
2022-05-04 10:03:20 -07:00
Vladimir Sigunov (vs422h) 48625ad984 [Cinder] Add visibilty settings to volume types
This is to add public/private  visibility option
and project level access list to a volume type while creating.

Change-Id: Id33c8c9f10e60fcdb4b6c49e69f3b5d8f11850c6
2022-05-03 12:34:06 -04:00
josebb 9d7bd69147 Add helm hook in cinder bootstrap job
Change-Id: I3a556d6deb57dc1b4c374fa997b0d37c22f1b066
2022-04-20 10:17:07 +03:00
Gage Hugo f5647770fa Remove unsupported values overrides - cinder
THe cinder chart contains several values overrides for older
releases of openstack that are no longer supported by
openstack-helm. This change removes these overrides from the
cinder chart.

Change-Id: I9d506e2cc1eebaeb6d2eacff5bd47113d069dbb0
2022-04-07 16:00:13 -05:00
Thiago Brito a9987be64c Enable taint toleration for cinder
This changes use the helm-toolkit template for toleration
in openstack services

Signed-off-by: Lucas Cavalcante <lucasmedeiros.cavalcante@windriver.com>
Story: 2009276
Task: 43531
Depends-On: I168837f962465d1c89acc511b7bf4064ac4b546c
Change-Id: Id63d0950fd8b7ff9592cbfe196b95739dc0b1380
2022-03-22 18:47:23 +00:00
Gage Hugo 2f89f1d020 Remove glance registry
Glance registry was deprecated in Queens and removed in Stein.
This change removes glance-registry settings and templates
from the glance chart. Also removed the overrides from older
releases that are no longer actively supported and tested.

Change-Id: I704d844b9ab96daa73ec42e29cded31fbbe3f720
2022-02-21 21:32:08 +00:00
Mohammed Naser 909824ac2b Fix notifications for Cinder
At the moment, the Cinder usage audit runs every 5 minutes which
is excessive and causes load on the system.  Also, it defaults to
auditing an entire month which can take ages for large systems.

This patch makes it run sanely at the 5th minute of ever hour and
also runs the audit for the past hour only.

Change-Id: I59d1230fa4d33a2cf0364ade1a710e65ef449057
Signed-off-by: Mohammed Naser <mnaser@vexxhost.com>
2021-11-30 16:11:11 +04:00
Gupta, Sangeet (sg774j) 44914c4ef1 cinder: Upgrade default images to ussuri
After volumev1 and volumev2 were removed, openstack-helm-infra gates
started failing because they are deploying defult cinder image which
is currently stein. The python-openstackclient for stein sets volumev2
as default volume type. This was failing volume commands in cinder
bootstrap job for openstack-helm-infra gates

Change-Id: Ifcb3c813f132c9deedaba9a11f9ef721efcb92b0
2021-11-18 21:33:29 +00:00
Nafiz Haider 6ec2773c95 Remove cinder v1/v2 defaults
This change removes the cinder v1 and v2 endpoint definitions
from the default values in the cinder chart.

Change-Id: I0ee35ad71c76df157e2c670a7899e4b6c1b91e46
2021-11-15 15:59:06 +00:00
Gage Hugo c20c1e4400 Update htk requirements repo
As part of the move to helm v3, all the charts in the OSH repos
will no longer lint/build properly due to a lack of helm serve
in helm v3.

This change modifies the helm-toolkit repo location to the
osh-infra repo in order to account for the removal oh helm serve.

This work is part of the migration to helm v3 and will be utilized
in future changes.

Change-Id: I90d25943d69ad6c76455f7778a4894f00c525c46
2021-10-10 18:45:28 -05:00
Gage Hugo 1e651dc3c3 Helm 3 - Fix Job Labels
If labels are not specified on a Job, kubernetes defaults them
to include the labels of their underlying Pod template. Helm 3
injects metadata into all resources [0] including a
`app.kubernetes.io/managed-by: Helm` label. Thus when kubernetes
sees a Job's labels they are no longer empty and thus do not get
defaulted to the underlying Pod template's labels. This is a
problem since Job labels are depended on by
- Armada pre-upgrade delete hooks
- Armada wait logic configurations
- kubernetes-entrypoint dependencies

Thus for each Job template this adds labels matching the
underlying Pod template to retain the same labels that were
present with Helm 2.

[0]: https://github.com/helm/helm/pull/7649

Change-Id: Ib5a7eb494fb776d74e1edc767b9522b02453b19d
2021-10-06 13:54:58 -05:00
Darren DeJaeger e783450071 Add rabbitmq secret volume + mount for audit usage cronjob
This PS adds the rabbitmq secret volume + mount for the audit
usage cronjob, as it was previously missing and the job's command(s)
were failing when run.

In addition, add labels to the CronJob's metadata, so that it can
be picked up for pre-delete hooks.

Change-Id: I0a2ed0655702b4e41cc12d3908b9aed141e6f0d2
2021-09-28 13:43:51 -05:00
Sophie Huang 8ea0e27892 Add logic to Cinder bootstrap to handle timing issue
During upgrade, the Cinder pods go through the upgrade
process. Sometimes, the pods are unavailable to handle
the requests in bootstrap even the Cinder services are
up. This patchset gives the bootstrap job additional
attempt to finish the tasks

Change-Id: Ie7bd8909f1c93b76b2242748318f892a6ff9c53d
2021-08-24 21:04:47 +00:00
Andrii Ostapenko 3ac3caa013 Add support for Victoria and Wallaby
Defines compute kit and cinder jobs for new releases with
corresponding values overrides.

Disables compute agent list test for Wallaby since related API
is removed [0].

Since Wallaby with switch of osc to sdk '--id auto' is no longer
treated specially in 'openstack flavor create'. The same behavior
can be achieved w/o specifying --id flag for flavor creation [1].

Starting Wallaby 'nova-manage api_db version' returns init version
for empty database greater than 0 [2]. _db-sync.sh.tpl logic prior to
this commit does not work due to this. We need to either remove
(done in current commit) or justify and alter previous logic.

[0] https://review.opendev.org/749309
[1] https://review.opendev.org/750151
[2] https://opendev.org/openstack/nova/src/branch/stable/wallaby/nova/db/sqlalchemy/migration.py#L32

Change-Id: I361431d9aa8c1a06c5d59f479fb161ecd87e2ee2
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2021-08-02 15:46:07 +00:00
Susanta Gautam 3c9cf1539b Added post-install and post-upgrade hook for jobs.
Chart upgrading was failing due to some immutable fields are needed to upgrade before the jobs can be upgraded. For solving this issue, we
have added the helm.sh/hook annotations with post-install and post-upgrade values.
As for hook-weight annotations, we have added these to control the flow of the jobs with hook creation as the jobs are dependent. Like,
db-init jobs need to run before db-sync and so on.
helm3_hook values is added so that hook can be disabled from the values files.

Change-Id: Idfcc0479d152286ecd144502d80732094c9e43ea
2021-07-20 17:58:46 +05:45
Sophie Huang a711de8195 Add volume QoS support
In this patchset, volume QoS creation and association are added
to the bootstrap if configured.

Change-Id: I8c4bd4909f06e9db4fbc6a6daad0932babe94bcf
2021-07-16 13:59:16 +00:00
Kabanov, Dmitrii b1abce9a75 Add Ussuri release support
The PS adds the set of overrides for Ussuri release.

Change-Id: I6b3055e376aa14d0c2ecbea638e6e9ba3b03bde5
2021-06-30 16:47:22 -07:00
Gupta, Sangeet (sg774j) 5028aa8de1 Mount rabbitmq TLS secret
Mount rabbitmq TLS secret to openstack services which support internal
TLS. Once internal TLS support is added to other service, the TLSed 
rabbitmq support should be added.

Depends-on: https://review.opendev.org/c/openstack/openstack-helm-infra/+/795188

Change-Id: I9aa272e365f846746f2e06aa7b7010db730e17df
2021-06-10 14:12:57 +00:00
Haider, Nafiz (nh532m) c900712f30 feat(tls): Make openstack services compatible with rabbitmq TLS
Depends-on: https://review.opendev.org/c/openstack/openstack-helm-infra/+/770678

Co-authored-by: Sangeet Gupta <sg774j@att.com>

Change-Id: I11e9ad3f4079b0e12e498f9ed57e5b87ae9dc66a
2021-05-21 01:27:18 +00:00