Commit Graph

210 Commits

Author SHA1 Message Date
Sergiy Markin 0e086e4c12 [mariadb] Switch to ingress-less mariadb
This PS switches mariadb to use primary service by
default instead of ingress based deployment. The
primary service that is getting created and
automatically updated based on the leader election
process in start.py entrypoint script.

Mariadb primary service was introduced by this PS:

https://review.opendev.org/c/openstack/openstack-helm-infra/+/905797

Change-Id: I4992276d0902d277a7a81f2730c22635b15794b0
2024-02-26 18:59:22 +00:00
Sergiy Markin 07bd8c92a2 [mariadb] Add mariadb-server-primary service
This PS adds mariadb-server-primary service that is getting created
and automatically updated based on the leader election process in
start.py entrypoint script.

Change-Id: I1d8a8db0ce8102e5e23f7efdeedd139726ffff28
Signed-off-by: Sergiy Markin <smarkin@mirantis.com>
2024-01-18 00:17:47 +00:00
Ritchie, Frank (fr801x) f66bb53509 Update template for ingress 1.9
The names of a few configuration variables have changed in version 1.9.

EnableRealIp to EnableRealIP
HttpAccessLogPath to HTTPAccessLogPath
whitelist to allowlist
Whitelist to Allowlist

Additionally,

ajp_temp_path

is no longer valid.

Change-Id: I2ebb658bd237216c43306dab6cd7f7a1ca6388ac
2024-01-04 18:32:56 -05:00
Sergiy Markin 13c1d8cd38 [backups] Add throttlling of remote backups
This PS adds a possibility to limit (to throttle) the number of
simultaneously uploaded backups while keeping the logic on the client
side using flag files on remote side. The main idea is to have an
ability to limit number of simultaneous remote backups upload sessions.

Change-Id: I5464004d4febfbe20df9cd41ca62ceb9fd6f0c0d
2023-12-18 20:39:45 +00:00
Sergiy Markin f66c924b2f [backups] Mariadb backups improvements
This PS removes mariadb-verify-server sidecar container from
mariadb-backup cronjob in order to make backup process more resilient.

Change-Id: I2517c2de435ead34397ca0483610f511c8035bdf
2023-12-15 16:18:35 +00:00
Sergiy Markin 4a95f75b6b [backups] Added staggered backups
This PS adds staggered backups possibility by adding anti-affinity rules
to backups cronjobs that can be followed across several namespaces to
decrease load on remote backup destination server making sure that at
every moment in time there is only one backup upload is in progress.

Change-Id: If49791f866a73a08fb98fa0e0b4854042d079c66
2023-12-05 04:10:22 +00:00
Sergiy Markin 29f2b616cc [mariadb-operator] Mariadb-cluster chart
This PS adds mariadb-cluster chart based on mariadb-operator. Also for
some backward compartibility this PS adds mariadb-backup chart and
prometheus-mysql-exporter chart as a separate ones.

Change-Id: I3f652375cce2e3b45e095e08d2e6f4ae73b8d8f0
2023-11-29 21:51:48 -06:00
Sergiy Markin eae5c6d69a Uplift nginx ingress controller to v1.8.2
Change-Id: I4223f3f859833447f4045e7acea81bf4c7a8948a
2023-10-04 20:00:33 +00:00
Vladimir Kozhukalov ae91cf3fc3 Use deploy-env role for all deployment jobs
To make it easier to maintain the jobs all experimental
jobs (those which are not run in check and gate pipelines)
are moved to a separate file. They will be revised later
to use the same deploy-env role.

Also many charts use Openstack images for testing this
PR adds 2023.1 Ubuntu Focal overrides for all these charts.

Change-Id: I4a6fb998c7eb1026b3c05ddd69f62531137b6e51
2023-09-22 15:02:07 -05:00
Mosher, Jaymes (jm616v) 5358aed591 Prevent liveness probe from killing mariadb pods during SST
Update liveness probe script to accept pods either sending
or receiving a SST, and avoid killing them.

Change-Id: I4ad95c45a7ab7e5e1cec2b4696671b6055cc10e7
2023-09-20 14:12:24 -06:00
Vladimir Kozhukalov 97ce6d7d8e Update kubernetes registry to registry.k8s.io
See this link for details
https://kubernetes.io/blog/2023/03/10/image-registry-redirect/

Change-Id: Ifc8b64825751933def16a1784fae987a1d7250ad
2023-04-11 04:54:26 +03:00
Samuel Liu 6034a00bf7 Replace node-role.kubernetes.io/master with control-plane
The master label is no longer present on kubeadm control plane nodes(v1.24). For new clusters, the label 'node-role.kubernetes.io/master' will no longer be added to control plane nodes, only the label 'node-role.kubernetes.io/control-plane' will be added. For more information, refer to KEP-2067[https://github.com/kubernetes/enhancements/tree/master/keps/sig-cluster-lifecycle/kubeadm/2067-rename-master-label-taint]: Rename the kubeadm "master" label and taint.

the kubernetes pr: https://github.com/kubernetes/kubernetes/pull/107533

Change-Id: I3056b642db0a1799089998e3c020b4203c9a93ab
2023-03-20 13:38:07 +08:00
Anselme, Schubert (sa246v) d30bbfbfe7
Uplift nginx ingress controller to v1.5.1
Signed-off-by: Anselme, Schubert (sa246v) <sa246v@att.com>
Change-Id: I8e3eb2ebd52c0dae1d0cc0ebaa23885b8c0cf83f
2022-12-14 15:19:51 -05:00
Markin, Sergiy (sm515x) 5c4056ad34 [DATABASE] Add verify databases backup
HTK - added verify_databases_backup_in_directory function that is
going to be defined inside mariadb/postgresql/etcd charts.

Mariadb chart - added verify_databases_backup_archives function
implementation.

Added mariadb-verify container to mariadb-backup cronjob to run
verification process.

Added remove backup verification pocess - comparition of local and remote file md5 hashes.

PostgreSQL chart - added empty implementation of verify_databases_backup_archives() function. This is a subject for future realization.

Change-Id: I361cdb92c66b0b27539997d697adfd1e93c9a29d
2022-09-09 01:41:00 +00:00
Thales Elero Cervi 111f41edf6 Fixing broken mariadb helmrelease for helmv3
In an environment with helmv3, it was noticed that the mariadb
helmrelease is failing to render properly due to unsupported map key
type (int).

This change quickly fix this problem by quoting the value, forcing it to
be rendered as a string.

Signed-off-by: Thales Elero Cervi <thaleselero.cervi@windriver.com>
Change-Id: I2f2be87d0f79ca439e731d07354bcd5f149790d5
2022-09-01 13:31:09 -03:00
Brian Haley f31cfb2ef9 support image registries with authentication
Based on spec in openstack-helm repo,
support-OCI-image-registry-with-authentication-turned-on.rst

Each Helm chart can configure an OCI image registry and
credentials to use. A Kubernetes secret is then created with these
info. Service Accounts then specify an imagePullSecret specifying
the Secret with creds for the registry. Then any pod using one
of these ServiceAccounts may pull images from an authenticated
container registry.

Change-Id: Iebda4c7a861aa13db921328776b20c14ba346269
2022-07-20 14:28:47 -05:00
Markin, Sergiy (sm515x) 931ba39e87 [MariaDB] Add liveness probe to restart a pod that got stuck in a transfer wsrep_local_state_comment
Readiness probe that we currently have does not help with restarting a
pod that got stuck in a transfer state reported by
wsrep_local_state_comment.

root@mariadb-server-2:/# mysql_status_query wsrep_ready
OFF
root@mariadb-server-2:/# mysql_status_query wsrep_connected
ON
root@mariadb-server-2:/# mysql_status_query wsrep_cluster_status
non-Primary
root@mariadb-server-2:/# mysql_status_query wsrep_local_state_comment
Transfer

So the idea is to add a liveness probe that will take care of this.

Change-Id: I2ccecc75349667fe19c6f7f9dccc2dbbd17d0a5e
2022-06-21 20:32:52 +00:00
Schubert Anselme fce7ca38ae
Uplift Mariadb-ingress image to v1.2.0
Change-Id: Ic368517f893c5016793ce5d65b882a43fb2381ec
2022-06-06 09:14:56 -04:00
Markin, Sergiy (sm515x) 1f1a2ff527 [MariaDB] Fix backup/restore scripts for MariaDB 10.6
This patch adds database sys to the list of databases
to be ignored by backup/restore scripts in mariadb chart.

Change-Id: Ida7965bc583ada2c7ca4800c8ff5d6761fb3913a
2022-05-19 00:26:49 +00:00
Markin, Sergiy (sm515x) 322e5b8ccb [MariaDB] Fix ingress cluster role privileges
This patchset is adding update priviledge to ingress cluster role in order to let it to update mariadb state configmap. The problem appeared after upgrading nginx controller up to v1.1.3 in https://review.opendev.org/c/openstack/openstack-helm-infra/+/840691

Change-Id: I962ac336bf6b3588db88b04e2259de1aa20b1221
2022-05-13 17:42:32 -05:00
Markin, Sergiy (sm515x) 9d9edbded5 [MariaDB] Fix privileges for mysql-exporter user used by prometheus exporter
Change-Id: I1a2ba8d2525d28d1179a64d5c815e2f32ef56744
2022-05-12 17:35:55 -05:00
Schubert Anselme 753a32c33d
Migrate CronJob resources to batch/v1 and PodDisruptionBudget resources to policy/v1
This change updates the following charts to migrate CronJob resources to the batch/v1 API version, available since v1.21. [0]
and to migrate PodDisruptionBudget to the policy/v1 API version, also available since v1.21. [1]

This also uplift ingress controller to 1.1.3

- ceph-client (CronJob)
- cert-rotation (CronJob)
- elasticsearch (CronJob)
- mariadb (CronJob & PodDisruptionBudget)
- postgresql (CronJob)

0: https://kubernetes.io/docs/reference/using-api/deprecation-guide/#cronjob-v125
1: https://kubernetes.io/docs/reference/using-api/deprecation-guide/#poddisruptionbudget-v125

Change-Id: Ia6189b98a86b3f7575dc4678bb3a0cce69562c93
2022-05-10 15:12:53 -04:00
Gage Hugo 20d7d55f33 Update default image values for mariadb
This change updates the default image values in the mariadb chart
up to using Wallaby for the ones that use openstack images.

Change-Id: Id28da22932362c0400766a564b382ddbcada8c61
2022-04-28 17:22:09 +00:00
Graham Steffaniak 7b93d86fa6 Updated chart naming for subchart compatibility
CHG: Updated naming variable to change based on global values
     subchart_release_name for the following:
       * mariadb
       * rabbitmq
       * memcached

     This is a required change for the chart to be included
     as a subchart. if subchart_release_name is not present the
     yaml will render the same as prior to this change, leaving
     existing deployments unaffected.

Change-Id: Ib7a449f3b21d5169b8003cf4464f3ed95e942c14
2022-04-01 09:32:39 -05:00
Thiago Brito 6dcc7f8f89 Enable taint toleration for mariadb
This adds taint toleration support for openstack jobs

Signed-off-by: Lucas Cavalcante <lucasmedeiros.cavalcante@windriver.com>
Change-Id: Iab78370182b15b48df964eb2dfdc957a9868c708
2022-03-29 11:34:48 -03:00
Graham Steffaniak b5c285ae98 Revert chart naming to .Release.Name expression
CHG required for existing deployments to be
	 upgraded in place.

Change-Id: Ife4278f17601358dcd853c29977f5e2e88e521dc
2022-03-28 19:35:14 +00:00
Graham Steffaniak 8e3c97714b Updated chart naming for subchart compatibility
CHG: - reno-check script to ignore subchart
     - .Release.Name to match .Chart.Name instead:
       - mariadb
       - rabbitmq
       - memcached

Change-Id: Ieaecd5537c2843357b2787f6f59405b672ce8b8a
2022-03-23 17:24:05 -05:00
Gage Hugo a1bd832b0f Fix comparison error with mariadb and helm v3
The mariadb chart currently fails to deploy due to
differences in handling comparison between helm v2
and v3. This change updates the comparison to work
in both versions.

Change-Id: I9143a16f3011c0c0ae5420e6ec41ad7745a28cab
2022-03-19 01:21:26 +00:00
Markin, Sergiy (sm515x) 848f392b3a [DATABASE] MariaDB de-clustering
Adjust chart behavior in case only one mariadb instance is present and replication is disabled.

Change-Id: Ifa540580cf9d5755b83dbb949555ec814dda2744
2022-03-17 17:34:42 +00:00
Lo, Chi (cl566n) 2fc1ce4a14 Removing -x from database backup script
The set -x has produced 6 identical log strings every time the
log_backup_error_exit function is called.  Prometheus is using
the occurrence and number of some logs over a period of time to
evaluate database backup failure or not.  Only one log should be
generated when a particular database backup scenario failed.

Upon discussion with database backup and restore SME, it is
recommended to remove the set -x once and for all.

Change-Id: I846b5c16908f04ac40ee8f4d87d3b7df86036512
2022-02-23 16:42:29 -08:00
Gupta, Sangeet (sg774j) 47795919cb Mariadb: Enhance mariadb backup
* Add capability to retry uploading backup to remote server configured
  number of times and delay the retires randomly between configured
  minimum/maximum seconds.
* Enhanced error checking, logging and retrying logic.

Change-Id: Ida3649420bdd6d39ac6ba7412c8c7078a75e0a10
2021-11-20 02:06:28 +00:00
jinyuanliu 6d808ceb47 Fix Python exceptions
If thread launch_cluster_Monitor() and launch_leader_election() operates on the configmap at the same time, Will cause a error 'Exception in thread "Thread-1"'.
This error will cause the thread to get stuck. Configmap will not be updated and the error "data too old" will be reported.
Just passing kubernetes_API exceptions is not enough, all are more appropriate.

Change-Id: I6baa9ece474f9c937fe9bce2231ef500562e0406
2021-11-01 05:57:33 +00:00
Gage Hugo 22e50a5569 Update htk requirements
This change updates the helm-toolkit path in each chart as part
of the move to helm v3. This is due to a lack of helm serve.

Change-Id: I011e282616bf0b5a5c72c1db185c70d8c721695e
2021-10-06 01:02:28 +00:00
Sean Eagan b1a247e7f5 Helm 3 - Fix Job labels
If labels are not specified on a Job, kubernetes defaults them
to include the labels of their underlying Pod template. Helm 3
injects metadata into all resources [0] including a
`app.kubernetes.io/managed-by: Helm` label. Thus when kubernetes
sees a Job's labels they are no longer empty and thus do not get
defaulted to the underlying Pod template's labels. This is a
problem since Job labels are depended on by
- Armada pre-upgrade delete hooks
- Armada wait logic configurations
- kubernetes-entrypoint dependencies

Thus for each Job template this adds labels matching the
underlying Pod template to retain the same labels that were
present with Helm 2.

[0]: https://github.com/helm/helm/pull/7649

Change-Id: I3b6b25fcc6a1af4d56f3e2b335615074e2f04b6d
2021-09-30 16:01:31 -05:00
Samuel Liu b7b2048b35 add ingress resources
The current ingress deployment does not add resource, we need to add it.

Change-Id: I9d610f13235c431ffdfa1d29b71660b3c1261e37
2021-09-09 19:43:47 +08:00
root 45b50160f6 Update log format stream for mariadb
It is usefule for troubleshooting.

Change-Id: Ief9fb0c700e64717fe3a7f62b7b7c22ec1f84179
2021-08-20 16:43:40 +02:00
xuxant02@gmail.com 9133218e83 Added the helm hook for create user job for exporter
exporter-jpb-create-user was failing due to the field immutability
which was resulting in the manual delete of the job for every helm
upgrade to be successful. Reason being job being upgraded before the
other manifest that are required been updated. It can be avoided by
using helm-hook post-install and post-upgrade which will force the
job manifest to be applied only after all other manifest are applied.
Hook annotation is provided "5" so that the if other jobs are annotated,
exporter job will be last to created.
helm3_hook value is used for the condition which will enable the disable
of the hook.

Change-Id: I2039abb5bad07a19fd09fc5e245485c3c772beca
2021-07-08 22:04:36 +05:45
Thiago Brito 5a0ba49d50 Prepending library/ to docker official images
This will ease mirroring capabilities for the docker official images.

Signed-off-by: Thiago Brito <thiago.brito@windriver.com>
Change-Id: I0f9177b0b83e4fad599ae0c3f3820202bf1d450d
2021-06-02 15:04:38 -03:00
jinyuanliu 7baceae82f Remove panko residue
About panko chart,It's been removed.

Change-Id: I781f42f11e1bfc26537d393c527e34c66d29d0cf
2021-05-18 11:01:40 +08:00
Samuel Liu 0f1974f1c0 Remove deprecated svc annotation tolerate-unready-endpoints
Since k8s v1.11+, the annotation `service.alpha.kubernetes.io/tolerate-unready-endpoints` is deprecated.  we should use Service.spec.publishNotReadyAddresses instead.

Change-Id: Ic4f82b8e78770ff29637937c4bcb9af71b53f8d3
2021-05-11 07:10:10 +00:00
Roy Tang 5a8aabaee3 Prevent mariadb from split brain while cluster is in reboot state
The current start logic when existing cluster state is reboot can
lead to a split brain condition under certain circumstances.  This
patchset adds some additional step to ensure cluster is set to
live state once leader node is ready to start, instead of relying
on slave nodes to handle.  Also add some simple retry when there
is collision detected while trying to write to configmap.

The existing hair-trigger that will put the cluster state from
"live" into "reboot" can use some fine tuning, but updating it
properly should require additional investigation and testing,
hence should be done as a separate activity outside the scope
of this patchset.

Change-Id: Ieb2861d6fbc435e24e20d13c7b358c751890b4c4
2021-05-05 17:33:20 -04:00
Gage Hugo 44947cc80b Uplift mariadb version and ubuntu release
This change updates the default images for mariadb, both the version
to 10.5.9 and the ubuntu release to focal.

Change-Id: Iff99ebe78554197db4d459bef0dda01b6b2710b7
2021-04-22 21:08:44 +00:00
Lo, Chi (cl566n) fbc9564016 Updated mysqld-exporter image
Updated mysqld-exporter image version to v0.12.1.

Change-Id: I2add0a7fa668a59fafdcd939c5830f7d78094bdc
2021-04-22 11:26:12 -07:00
Neely, Travis M f33a629086 Fix race condition for grastate.dat
There seems to be a race condition involving the grastate.dat file.
Upon creation of a new mariad-server pod the file would exist however,
it is not populated for a short period of time. It seems to take
around 15-20 seconds for this file to be populated. However there is
a separate thread which is attempting to read the file and tends to
end in an IndexError exception killing the thread which maintains the
grastate.dat file until the pod is restarted. This patchset adds a
loop to check for up to 60 seconds for the file to be populated
before attempting to continue, thus giving the file time to be
populated.

Change-Id: I2f2a801aa4528a7af61797419422572be1c82e75
2021-04-19 19:57:49 +00:00
Huang, Sophie (sh879n) 6eec615b39 Set strict permission on mariadb data dir
For security reasons, strict access permission is given to
the mariadb data directory /var/lib/mysql

Change-Id: I9e55a7e564d66874a35a54a72817fa1237a162e9
2021-03-24 20:20:03 +00:00
Huang, Sophie (sh879n) 87429ebb86 Disable mariadb mysql history client logging
Environment variable MYSQL_HISTFILE is added to mariadb container
to disable storing client mysql history to ~/.mysql_history file.

Change-Id: Ie95bc1f830fbf34d30c73de07513299115d8e8c5
2021-03-12 20:50:15 +00:00
okozachenko 5db88a5fb4 Rename mariadb backup identities
Challenge:
Now remote_ks_admin and remote_rgw_user are using for user labels
of backup target openstack cloud.
When the backup user doesn't exist and we can enable job_ks_user
manifest.
But job_ks_user uses .Vaules.secrets.identity.admin and mariadb,
while secret-rgw and cron-job-backup-mariadb use .Values.secrets.
identity.remote_ks_admin and remote_rgw_user.
It requires to use same values for admin and remote_ks_admin,
and for mariadb and remote_rgw_user.
Seems it isbreaking values consistency.

Suggestion:
Now providing 2 kinds of backup - pvc and swift.
"remote_" means the swift backup.
In fact, mariadb chart has no case to access to keystone except
swift backup. So we can remove remote_xx_* prefix and there is
no confusion.

Change-Id: Ib82120611659bd36bae35f2e90054642fb8ee31f
2021-03-03 20:46:51 +02:00
anthony.bellino dcd77ceba3 [mariadb-ingress] Uplift Mariadb-ingress to 0.42.0
- Uplifts the image to nginx 0.42.0 to address CVEs
- Updates nginx.tmpl accordingly for nginx 0.42.0
- Adds CLusterRole and labels needed for nginx 0.42.0
- Updates release notes for mariadb

Change-Id: Ie4e2a66873bc130c547ff8f30d8e1b2ee9a62186
2021-03-01 18:07:23 -08:00
Nafiz Haider 6ee06562c8 Re-enable "feat(tls): Change Issuer to ClusterIssuer""
This reverts commit 8a79d7c51b.

Reason for revert: resolved bug with cluster issuer versioning

Co-authored-by: Sangeet Gupta <sg774j@att.com>

Change-Id: I047cbfaa5aa9e7285a23e603074429180495557d
2021-02-24 20:50:24 +00:00
Travis Neely 8a79d7c51b Revert "feat(tls): Change Issuer to ClusterIssuer"
This reverts commit f60c94fc16.

Reason for revert: This introduced a bug:
https://cert-manager.io/docs/installation/upgrading/upgrading-0.15-0.16/#issue-with-older-versions-of-kubectl

Older versions of kubectl will have issues with the nested CRDs.

Change-Id: I322fc1382fe3d0a4517e4c7c5982ea50a721a1f7
2021-01-27 16:59:01 -06:00