Add the ability for charts that use helm-toolkit to allow the users to
set custom annotations on jobs. Use the snippet in a generic way in the
job templates provided by helm-toolkit.
Change-Id: I5d60fe849e172c19d865b614c3c44ea618f92f20
Depends-On: I3991d6984563813d5a3a776eabd52e2e89933bd8
Signed-off-by: Doug Goldstein <doug.goldstein@rackspace.com>
Due to CVE-2022-4886 the default pathType for an ingress should be
either "Exact" or "Prefix". This allows for more strict path validation by
the admission controller. This PS changes the default pathType to Prefix.
This value can be overridden.
In a separate PS I will add the pathType parameter to the ingressOpts
for all helm charts that create an ingress.
See:
https://github.com/kubernetes/ingress-nginx/issues/10570
Change-Id: I8f1df594f0c86f2de6cdd7cf2ee56637bd508565
This PS adds a possibility to limit (to throttle) the number of
simultaneously uploaded backups while keeping the logic on the client
side using flag files on remote side. The main idea is to have an
ability to limit number of simultaneous remote backups upload sessions.
Change-Id: I5464004d4febfbe20df9cd41ca62ceb9fd6f0c0d
This PS removes mariadb-verify-server sidecar container from
mariadb-backup cronjob in order to make backup process more resilient.
Change-Id: I2517c2de435ead34397ca0483610f511c8035bdf
This PS adds staggered backups possibility by adding anti-affinity rules
to backups cronjobs that can be followed across several namespaces to
decrease load on remote backup destination server making sure that at
every moment in time there is only one backup upload is in progress.
Change-Id: If49791f866a73a08fb98fa0e0b4854042d079c66
When using Rook for managing Ceph we can use
Rook CRDs to create S3 buckets and users.
This PR adds bucket claim template to the
elasticsearch chart. Rook creates a bucket for
a bucket claim and also creates a secret
containing the credentials to get access to this
bucket. So we also add a snippet to expose
these credentials via environment variables to
containers where they are needed.
Change-Id: Ic5cd35a5c64a914af97d2b3cfec21dbe399c0f14
This PS replaces deprecated kubernetes.io/ingress.class annotation with
spec.ingressClassName field that is a reference to an IngressClass
resource that contains additional Ingress configuration, including the
name of the Ingress controller.
https://kubernetes.io/blog/2020/04/02/improvements-to-the-ingress-api-in-kubernetes-1.18/#deprecating-the-ingress-class-annotation
Change-Id: I9953d966b4f9f7b1692b39f36f434f5055317025
Co-authored-by: Sergiy Markin <smarkin@mirantis.com>
Co-authored-by: Leointii Istomin <listomin@mirantis.com>
Signed-off-by: Anselme, Schubert (sa246v) <sa246v@att.com>
There is a condition check for dependencyKey when dependencyMixinParam
is a string value, but not when a slice value.
It requires to add an empty section in dependencies.dynamic.targeted
even if there is no dynamic dependency requirements.
This patch adds a condition check to avoid the dummy values.
Change-Id: I1db9156741959acb074d86a3ae900e8be31170f7
Using GRANT to create users was deprecated in 5.7. The current
query to create user fails with new versions of mysql.
Change-Id: If991778763dc0961508e8466244955fd71b47591
This PS resolves several issues in database backup script in HTK chart:
- decreases random delay before uploading remote backup to up to 30s
- removes additional random delay before remote backup verification
- switches remote backup verification protocol from sha256 to md5
The main goal for the changes above is decreasing network load on remote
backup storages by eliminating the need of remote file download right
after uploading in order to be able to calculate sha256 checksum.
Change-Id: Ic01a37d8814283a2e9a11dac94d6909d34edc937
This PS adds a random delay up to 300 seconds to remote backup upload
and download actions to spread the network load in time. Backup process
failure may happen if many sites are pushing their backups at the same
time. It was OK previously but now with added remote bakup sha256
checksum verification we need to download the backup we just uploaded.
So the network load already doubled. And this PS mitigates the impact
of that.
Change-Id: Ibc2a8f8287e20aeb56ad1f9c604b47db2d0eb06c
HTK - added verify_databases_backup_in_directory function that is
going to be defined inside mariadb/postgresql/etcd charts.
Mariadb chart - added verify_databases_backup_archives function
implementation.
Added mariadb-verify container to mariadb-backup cronjob to run
verification process.
Added remove backup verification pocess - comparition of local and remote file md5 hashes.
PostgreSQL chart - added empty implementation of verify_databases_backup_archives() function. This is a subject for future realization.
Change-Id: I361cdb92c66b0b27539997d697adfd1e93c9a29d
Having the "use_external_ingress_controller" field in
"network.server.ingress" yaml path is not a good choice as there are
services such neutron that use this path to define backend service,
named "server", options. We propose moving it to the root of the
path "network".
Change-Id: If98d6555a9c012872d3fb1a38b370a3195ea49ab
Based on spec in openstack-helm repo,
support-OCI-image-registry-with-authentication-turned-on.rst
Each Helm chart can configure an OCI image registry and
credentials to use. A Kubernetes secret is then created with these
info. Service Accounts then specify an imagePullSecret specifying
the Secret with creds for the registry. Then any pod using one
of these ServiceAccounts may pull images from an authenticated
container registry.
Change-Id: Iebda4c7a861aa13db921328776b20c14ba346269
This change allows creating a single ingress resource using the
public fqdn of the service, instead of two (cluster and namespace)
that is currently the case. Every openstack-helm chart can have a
network.server.ingress.use_external_ingress_controller boolean
field to choose the creation of a single ingress resource
(ingressName-namespace-fqdn).
Signed-off-by: Yanos Angelopoulos <yanos@admin.grnet.gr>
Change-Id: I46da850fccc3fee76595a2e6c49d51197a282c3e
Fixes minor issue with naming of variables which prevents the script to
be compliant the backup retention policy.
Change-Id: Ic241310a66af92ee423f5c762c413af7d6d53f0b
Added a parser for archive names to cover the situation when an archive
name could be represented in two different formats
1) <database name>.<namespace>.<table name | all>.<date-time>.tar.gz
2) <database name>.<namespace>.<table name | all>.<backup mode>.<date-time>.tar.gz
The first format is what is using at the moment,
the second format is recommended for future use.
Change-Id: I6b631b3b938c0a0242c5a8870284995b2cd8f27b
Minor change to list archive directory with files in sub-directory
as below. Without the change, only the directory name 'quarantine'
is displayed.
All Local Archives
==============================================
mariadb.openstack.all.2022-03-20T18:00:17Z.tar.gz
mariadb.openstack.all.2022-03-21T00:00:16Z.tar.gz
mariadb.openstack.all.2022-03-21T06:00:12Z.tar.gz
mariadb.openstack.all.2022-03-21T12:00:13Z.tar.gz
mariadb.openstack.all.2022-03-21T18:00:11Z.tar.gz
quarantine/mariadb.openstack.all.2022-03-23T00:00:12Z.tar.gz
quarantine/mariadb.openstack.all.2022-03-23T06:00:11Z.tar.gz
quarantine/mariadb.openstack.all.2022-03-23T12:00:14Z.tar.gz
quarantine/mariadb.openstack.all.2022-03-23T14:24:04Z.tar.gz
Change-Id: Ic47a30884b82cdecedbfff8ddf1d85fc00d89acc
This adds taint toleration support for openstack jobs
Signed-off-by: Lucas Cavalcante <lucasmedeiros.cavalcante@windriver.com>
Change-Id: I168837f962465d1c89acc511b7bf4064ac4b546c
This is to cover some relatively rare sutuation, when backups
of different databases can share the same storage.
Change-Id: I0770e1baf3d33e2d56c34558a9a97a99a01e5e04
Modifies the backup script in the way that there will always be
a minimum given number of days of backups in both local, and remote
(if applicable) locations, regardless the date that the backups
are taken.
Change-Id: I19d5e592905ce83acdba043f68ca4d0b042de065
The set -x has produced 6 identical log strings every time the
log_backup_error_exit function is called. Prometheus is using
the occurrence and number of some logs over a period of time to
evaluate database backup failure or not. Only one log should be
generated when a particular database backup scenario failed.
Upon discussion with database backup and restore SME, it is
recommended to remove the set -x once and for all.
Change-Id: I846b5c16908f04ac40ee8f4d87d3b7df86036512
This is a code improvement to reuse ceph monitor doscovering function
in different templates. Calling the mentioned above function from
a single place (helm-infra snippets) allows less code maintenance
and simlifies further development.
Rev. 0.1 Charts version bump for ceph-client, ceph-mon, ceph-osd,
ceph-provisioners and helm-toolkit
Rev. 0.2 Mon endpoint discovery functionality added for
the rados gateway. ClusterRole and ClusterRoleBinding added.
Rev. 0.3 checkdns is allowed to correct ceph.conf for RGW deployment.
Rev. 0.4 Added RoleBinding to the deployment-rgw.
Rev. 0.5 Remove _namespace-client-ceph-config-manager.sh.tpl and
the appropriate job, because of duplicated functionality.
Related configuration has been removed.
Rev. 0.6 RoleBinding logic has been changed to meet rules:
checkdns namespace - HAS ACCESS -> RGW namespace(s)
Change-Id: Ie0af212bdcbbc3aa53335689deed9b226e5d4d89
At the moment it is very difficult to pull images from a private
registry that hasn't been configured on Kubernetes nodes as there
is no way to specify imagePullSecrets on pods.
This change introduces a snippet that can return a set of image
pull secrets using either a default or a per pod value. It also
adds this new snippet to the manifests for standard job types.
Change-Id: I710e1feffdf837627b80bc14320751f743e048cb
* Add capability to retry uploading backup to remote server configured
number of times and delay the retires randomly between configured
minimum/maximum seconds.
* Enhanced error checking, logging and retrying logic.
Change-Id: Ida3649420bdd6d39ac6ba7412c8c7078a75e0a10
We need flexibility to add securityContext to ks-user job at pod and containerlevel,
so that it can be executed without elevated privileges.
Change-Id: Ibd8abdc10906ca4648bfcaa91d0f122e56690606
In cert-manager v1 API, the private key size "keySize" was updated to "size"
under "privateKey".
Support of minor (less than v1) API version is also removed for certificates.
Change-Id: If3fa0e296b8a1c2ab473e67b24d4465fe42a5268
This reverts commit 5407b547bb.
Reason for revert: This outputs duplicate securityContext entries,
breaking the yamllinter in osh. This needs a slight rework.
Change-Id: I0c892be5aba7ccd6e3c378e4e45a79d2df03c06a
We need flexibility to add securityContext to ks-user job , so that it can be executed without elevated privileges.
Change-Id: I24544015816d57d86c1e69f44b90b6b0271e76a4
If labels are not specified on a Job, kubernetes defaults them
to include the labels of their underlying Pod template. Helm 3
injects metadata into all resources [0] including a
`app.kubernetes.io/managed-by: Helm` label. Thus when kubernetes
sees a Job's labels they are no longer empty and thus do not get
defaulted to the underlying Pod template's labels. This is a
problem since Job labels are depended on by
- Armada pre-upgrade delete hooks
- Armada wait logic configurations
- kubernetes-entrypoint dependencies
Thus for each Job template this adds labels matching the
underlying Pod template to retain the same labels that were
present with Helm 2.
[0]: https://github.com/helm/helm/pull/7649
Change-Id: I3b6b25fcc6a1af4d56f3e2b335615074e2f04b6d
Currently it isn't possible to set extra labels on pods that use
the labels snippet. This means users are required to fork the helm
repository for OpenStack services to add custom labels. Use cases
for this are for example injecting Istio sidecars.
This change introduces the ability to set one set of labels on all
resources that use the labels snippet.
Change-Id: Iefc8465300f434b89c07b18ba75260fee0a05ef5
The return code from the send_to_remote_server function are
being eaten by an if statement and thus we never hit the elif
section of code.
Change-Id: Id3e256c991421ad6624713f65212abb4881240c1
In the process of secondary development, we found
that we often need to access secrets from pod.
However, it seems that helm-tookit does not support
adding resource of secrets to role. This commit
try to fix that.
Change-Id: If384d6ccb7672a8da5a5e1403733fa655dfe40dd
There is an additional error status 'Service Unavailable' which can
indicate the service is temporary unavailable. Adding that error
status to the retry list in case the issue is resolved during the
backup timeframe.
Change-Id: I9e2fc1a9b33dea3858de06b10d512da98a635015
Tadas Sutkaitis