openstack-helm-infra chart rebase

Each patch included in this commit contains a commit message that
describes the required purpose of the patch.

Change-Id: Ia92158b77478c602e65280b09a744414c1bb31aa
Depends-On: Ic788a2c86edfbceca1f1ff18dd0344472546c81b
Story: 2004520
Task: 29966
Signed-off-by: Robert Church <robert.church@windriver.com>
This commit is contained in:
Robert Church 2019-03-22 04:24:35 -04:00
parent 0b04175782
commit f1901be75a
10 changed files with 147 additions and 645 deletions

View File

@ -1,8 +1,8 @@
TAR_NAME=openstack-helm-infra
SHA=5d356f9265b337b75f605dee839faa8cd0ed3ab2
SHA=aae64213c95fbcea7a0a7671dcb9d8a366f16fa5
VERSION=1.0.0
TAR="$TAR_NAME-$SHA.tar.gz"
COPY_LIST="${CGCS_BASE}/downloads/$TAR $PKG_BASE/files/*"
TIS_PATCH_VER=8
TIS_PATCH_VER=9

View File

@ -1,4 +1,4 @@
%global sha 5d356f9265b337b75f605dee839faa8cd0ed3ab2
%global sha aae64213c95fbcea7a0a7671dcb9d8a366f16fa5
%global helm_folder /usr/lib/helm
Summary: Openstack-Helm-Infra charts
@ -15,12 +15,11 @@ Source1: repositories.yaml
BuildArch: noarch
Patch01: 0001-gnocchi-chart-updates.patch
Patch02: Mariadb-Support-adoption-of-running-single-node-mari.patch
Patch03: 0004-Allow-multiple-containers-per-daemonset-pod.patch
Patch04: fix-type-error-to-streamline-single-replica-mariadb-.patch
Patch05: Add-imagePullSecrets-in-service-account.patch
Patch06: 0006-Set-Min-NGINX-handles.patch
Patch01: 0001-Allow-multiple-containers-per-daemonset-pod.patch
Patch02: 0002-Add-imagePullSecrets-in-service-account.patch
Patch03: 0003-Set-Min-NGINX-handles.patch
Patch04: 0004-Enable-Ceph-Jewel-support-for-gnocchi.patch
Patch05: 0005-Partial-revert-of-31e3469d28858d7b5eb6355e88b6f49fd6.patch
BuildRequires: helm
@ -34,7 +33,6 @@ Openstack Helm Infra charts
%patch03 -p1
%patch04 -p1
%patch05 -p1
%patch06 -p1
%build
# initialize helm and build the toolkit

View File

@ -1,22 +1,27 @@
From 26844aac43f76afc65ed907fc94ab83ca93c86ae Mon Sep 17 00:00:00 2001
From 47315e28d44cff586f6fff026dd00e61c2c77bcd Mon Sep 17 00:00:00 2001
From: Gerry Kopec <Gerry.Kopec@windriver.com>
Date: Wed, 9 Jan 2019 20:11:33 -0500
Subject: [PATCH] Allow multiple containers per daemonset pod
Subject: [PATCH 1/5] Allow multiple containers per daemonset pod
Remove code that restricted daemonset pods to single containers.
Container names will default to name from helm chart template without
hostname and sha though the pod will still have them.
Container names will default to name from helm chart template.
Required for nova cold migrations to work.
May require further refinement before this can be upstreamed.
Story: 2003876
Task: 26735
Change-Id: Icce660415d43baefbbf768a785c5dedf04ea2930
Signed-off-by: Gerry Kopec <Gerry.Kopec@windriver.com>
(cherry picked from commit 7ca30319f418cd39db5ecf44cce5fb5fe39c458e)
Signed-off-by: Robert Church <robert.church@windriver.com>
---
helm-toolkit/templates/utils/_daemonset_overrides.tpl | 7 -------
1 file changed, 7 deletions(-)
diff --git a/helm-toolkit/templates/utils/_daemonset_overrides.tpl b/helm-toolkit/templates/utils/_daemonset_overrides.tpl
index 8ba2241..b960a84 100644
index 743bd6b..c02de9e 100644
--- a/helm-toolkit/templates/utils/_daemonset_overrides.tpl
+++ b/helm-toolkit/templates/utils/_daemonset_overrides.tpl
@@ -217,13 +217,6 @@ limitations under the License.
@@ -223,13 +223,6 @@ limitations under the License.
{{- if not $context.Values.__daemonset_yaml.metadata.name }}{{- $_ := set $context.Values.__daemonset_yaml.metadata "name" dict }}{{- end }}
{{- $_ := set $context.Values.__daemonset_yaml.metadata "name" $current_dict.dns_1123_name }}
@ -31,5 +36,5 @@ index 8ba2241..b960a84 100644
{{- $_ := set $context.Values "__volume_list" list }}
{{- range $current_volume := $context.Values.__daemonset_yaml.spec.template.spec.volumes }}
--
1.8.3.1
2.16.5

View File

@ -1,255 +0,0 @@
From da5bfc668bf5ccfa384bb91f9b933526f33c3492 Mon Sep 17 00:00:00 2001
From: Angie Wang <angie.wang@windriver.com>
Date: Wed, 26 Sep 2018 17:01:28 +0000
Subject: [PATCH 1/1] gnocchi chart updates
- Remove the gnocchi upgrade option "--create-legacy-resource-types" as
it is deprecated since gnocchi 4.0.0
- Set the default coordination driver of gnocchi to memcached
- Add the cron job for purging the deleted openstack resources
---
gnocchi/templates/bin/_db-sync.sh.tpl | 2 +-
.../bin/_gnocchi-resources-cleaner.sh.tpl | 14 ++++
gnocchi/templates/configmap-bin.yaml | 2 +
gnocchi/templates/configmap-etc.yaml | 6 ++
gnocchi/templates/cron-job-resources-cleaner.yaml | 86 ++++++++++++++++++++++
gnocchi/values.yaml | 34 +++++++++
6 files changed, 143 insertions(+), 1 deletion(-)
create mode 100755 gnocchi/templates/bin/_gnocchi-resources-cleaner.sh.tpl
create mode 100755 gnocchi/templates/cron-job-resources-cleaner.yaml
diff --git a/gnocchi/templates/bin/_db-sync.sh.tpl b/gnocchi/templates/bin/_db-sync.sh.tpl
index a32db4e..0693ee2 100644
--- a/gnocchi/templates/bin/_db-sync.sh.tpl
+++ b/gnocchi/templates/bin/_db-sync.sh.tpl
@@ -18,4 +18,4 @@ limitations under the License.
set -ex
-exec gnocchi-upgrade --create-legacy-resource-types
+exec gnocchi-upgrade
diff --git a/gnocchi/templates/bin/_gnocchi-resources-cleaner.sh.tpl b/gnocchi/templates/bin/_gnocchi-resources-cleaner.sh.tpl
new file mode 100755
index 0000000..5ef2fba
--- /dev/null
+++ b/gnocchi/templates/bin/_gnocchi-resources-cleaner.sh.tpl
@@ -0,0 +1,14 @@
+#!/bin/bash
+
+{{/*
+Copyright (c) 2018 Wind River Systems, Inc.
+
+SPDX-License-Identifier: Apache-2.0
+*/}}
+
+set -ex
+
+echo "Purging the deleted resources with its associated metrics which have lived more than ${DELETED_RESOURCES_TTL}"
+gnocchi resource batch delete "ended_at < '-${DELETED_RESOURCES_TTL}'"
+
+exit 0
diff --git a/gnocchi/templates/configmap-bin.yaml b/gnocchi/templates/configmap-bin.yaml
index 0ea4646..dc1a85f 100644
--- a/gnocchi/templates/configmap-bin.yaml
+++ b/gnocchi/templates/configmap-bin.yaml
@@ -46,6 +46,8 @@ data:
{{ tuple "bin/_gnocchi-metricd.sh.tpl" . | include "helm-toolkit.utils.template" | indent 4 }}
gnocchi-statsd.sh: |
{{ tuple "bin/_gnocchi-statsd.sh.tpl" . | include "helm-toolkit.utils.template" | indent 4 }}
+ gnocchi-resources-cleaner.sh: |
+{{ tuple "bin/_gnocchi-resources-cleaner.sh.tpl" . | include "helm-toolkit.utils.template" | indent 4 }}
ks-service.sh: |
{{- include "helm-toolkit.scripts.keystone_service" . | indent 4 }}
ks-endpoints.sh: |
diff --git a/gnocchi/templates/configmap-etc.yaml b/gnocchi/templates/configmap-etc.yaml
index 83d3f14..bdf6acc 100644
--- a/gnocchi/templates/configmap-etc.yaml
+++ b/gnocchi/templates/configmap-etc.yaml
@@ -50,6 +50,12 @@ limitations under the License.
{{- $_ := set .Values.conf.gnocchi.keystone_authtoken "password" .Values.endpoints.identity.auth.gnocchi.password -}}
{{- end -}}
+{{- if empty .Values.conf.gnocchi.DEFAULT.coordination_url -}}
+{{- $endpointUrl := tuple "oslo_cache" "internal" "memcache" . | include "helm-toolkit.endpoints.host_and_port_endpoint_uri_lookup" }}
+{{- $driver := .Values.endpoints.oslo_cache.hosts.default -}}
+{{- $_ := printf "%s://%s" $driver $endpointUrl | set .Values.conf.gnocchi.DEFAULT "coordination_url" -}}
+{{- end -}}
+
{{- if empty .Values.conf.gnocchi.database.connection -}}
{{- $_ := tuple "oslo_db" "internal" "gnocchi" "mysql" . | include "helm-toolkit.endpoints.authenticated_endpoint_uri_lookup" | set .Values.conf.gnocchi.database "connection" -}}
{{- end -}}
diff --git a/gnocchi/templates/cron-job-resources-cleaner.yaml b/gnocchi/templates/cron-job-resources-cleaner.yaml
new file mode 100755
index 0000000..0235454
--- /dev/null
+++ b/gnocchi/templates/cron-job-resources-cleaner.yaml
@@ -0,0 +1,86 @@
+{{/*
+Copyright (c) 2018 Wind River Systems, Inc.
+
+SPDX-License-Identifier: Apache-2.0
+*/}}
+
+{{- if .Values.manifests.cron_job_resources_cleaner }}
+{{- $envAll := . }}
+
+{{- $mounts_gnocchi_resources_cleaner := .Values.pod.mounts.gnocchi_resources_cleaner.gnocchi_resources_cleaner }}
+{{- $mounts_gnocchi_resources_cleaner_init := .Values.pod.mounts.gnocchi_resources_cleaner.init_container }}
+
+{{- $serviceAccountName := "gnocchi-resources-cleaner" }}
+{{ tuple $envAll "resources_cleaner" $serviceAccountName | include "helm-toolkit.snippets.kubernetes_pod_rbac_serviceaccount" }}
+---
+apiVersion: batch/v1beta1
+kind: CronJob
+metadata:
+ name: gnocchi-resources-cleaner
+ annotations:
+ {{ tuple $envAll | include "helm-toolkit.snippets.release_uuid" }}
+spec:
+ schedule: {{ .Values.jobs.resources_cleaner.cron | quote }}
+ successfulJobsHistoryLimit: {{ .Values.jobs.resources_cleaner.history.success }}
+ failedJobsHistoryLimit: {{ .Values.jobs.resources_cleaner.history.failed }}
+ concurrencyPolicy: Forbid
+ jobTemplate:
+ metadata:
+ labels:
+{{ tuple $envAll "gnocchi" "resources-cleaner" | include "helm-toolkit.snippets.kubernetes_metadata_labels" | indent 8 }}
+ spec:
+ template:
+ spec:
+ serviceAccountName: {{ $serviceAccountName }}
+ restartPolicy: OnFailure
+ nodeSelector:
+ {{ .Values.labels.job.node_selector_key }}: {{ .Values.labels.job.node_selector_value }}
+ initContainers:
+{{ tuple $envAll "resources_cleaner" $mounts_gnocchi_resources_cleaner_init | include "helm-toolkit.snippets.kubernetes_entrypoint_init_container" | indent 12 }}
+ containers:
+ - name: gnocchi-resources-cleaner
+{{ tuple $envAll "gnocchi_resources_cleaner" | include "helm-toolkit.snippets.image" | indent 14 }}
+{{ tuple $envAll $envAll.Values.pod.resources.jobs.resources_cleaner | include "helm-toolkit.snippets.kubernetes_resources" | indent 14 }}
+ env:
+{{- with $env := dict "ksUserSecret" .Values.secrets.identity.admin }}
+{{- include "helm-toolkit.snippets.keystone_openrc_env_vars" $env | indent 16 }}
+{{- end }}
+ - name: OS_AUTH_TYPE
+ valueFrom:
+ secretKeyRef:
+ name: {{ $.Values.secrets.identity.admin }}
+ key: OS_AUTH_TYPE
+ - name: OS_TENANT_NAME
+ valueFrom:
+ secretKeyRef:
+ name: {{ $.Values.secrets.identity.admin }}
+ key: OS_TENANT_NAME
+ - name: DELETED_RESOURCES_TTL
+ value: {{ .Values.jobs.resources_cleaner.deleted_resources_ttl | quote }}
+ command:
+ - /tmp/gnocchi-resources-cleaner.sh
+ volumeMounts:
+ - name: gnocchi-bin
+ mountPath: /tmp/gnocchi-resources-cleaner.sh
+ subPath: gnocchi-resources-cleaner.sh
+ readOnly: true
+ - name: pod-etc-gnocchi
+ mountPath: /etc/gnocchi
+ - name: gnocchi-etc
+ mountPath: /etc/gnocchi/gnocchi.conf
+ subPath: gnocchi.conf
+ readOnly: true
+{{ if $mounts_gnocchi_resources_cleaner.volumeMounts }}{{ toYaml $mounts_gnocchi_resources_cleaner.volumeMounts | indent 14 }}{{ end }}
+ volumes:
+ - name: pod-etc-gnocchi
+ emptyDir: {}
+ - name: gnocchi-etc
+ secret:
+ secretName: gnocchi-etc
+ defaultMode: 0444
+ - name: gnocchi-bin
+ configMap:
+ name: gnocchi-bin
+ defaultMode: 0555
+{{ if $mounts_gnocchi_resources_cleaner.volumes }}{{ toYaml $mounts_gnocchi_resources_cleaner.volumes | indent 10 }}{{ end }}
+{{- end }}
diff --git a/gnocchi/values.yaml b/gnocchi/values.yaml
index e32693a..2eae14c 100644
--- a/gnocchi/values.yaml
+++ b/gnocchi/values.yaml
@@ -50,6 +50,7 @@ images:
gnocchi_api: quay.io/attcomdev/ubuntu-source-gnocchi-api:3.0.3
gnocchi_statsd: quay.io/attcomdev/ubuntu-source-gnocchi-statsd:3.0.3
gnocchi_metricd: quay.io/attcomdev/ubuntu-source-gnocchi-metricd:3.0.3
+ gnocchi_resources_cleaner: quay.io/attcomdev/ubuntu-source-gnocchi-base:3.0.3
image_repo_sync: docker.io/docker:17.07.0
pull_policy: "IfNotPresent"
local_registry:
@@ -58,6 +59,15 @@ images:
- dep_check
- image_repo_sync
+jobs:
+ resources_cleaner:
+ # daily
+ cron: "0 */24 * * *"
+ deleted_resources_ttl: '1day'
+ history:
+ success: 3
+ failed: 1
+
network:
api:
ingress:
@@ -155,6 +165,19 @@ dependencies:
service: oslo_db_postgresql
- endpoint: internal
service: metric
+ resources_cleaner:
+ jobs:
+ - gnocchi-storage-init
+ - gnocchi-db-sync
+ - gnocchi-ks-user
+ - gnocchi-ks-endpoints
+ services:
+ - endpoint: internal
+ service: oslo_db
+ - endpoint: internal
+ service: identity
+ - endpoint: internal
+ service: metric
storage_init:
services: null
tests:
@@ -193,6 +216,9 @@ pod:
gnocchi_metricd:
init_container: null
gnocchi_metricd:
+ gnocchi_resources_cleaner:
+ init_container: null
+ gnocchi_resources_cleaner:
gnocchi_tests:
init_container: null
gnocchi_tests:
@@ -288,6 +314,13 @@ pod:
limits:
memory: "1024Mi"
cpu: "2000m"
+ resources_cleaner:
+ requests:
+ memory: "128Mi"
+ cpu: "100m"
+ limits:
+ memory: "1024Mi"
+ cpu: "2000m"
tests:
requests:
memory: "124Mi"
@@ -577,6 +610,7 @@ endpoints:
manifests:
configmap_bin: true
configmap_etc: true
+ cron_job_resources_cleaner: true
daemonset_metricd: true
daemonset_statsd: true
deployment_api: true
--
1.8.3.1

View File

@ -1,8 +1,9 @@
From c432facb61964b1f5f3a0522083ef9e830697f4c Mon Sep 17 00:00:00 2001
From ac3f9db5ac1a19af71136752f5709ba1da55d201 Mon Sep 17 00:00:00 2001
From: Angie Wang <angie.wang@windriver.com>
Date: Mon, 11 Feb 2019 11:29:03 -0500
Subject: [PATCH] Add imagePullSecrets in service account
Subject: [PATCH 2/5] Add imagePullSecrets in service account
Signed-off-by: Robert Church <robert.church@windriver.com>
---
helm-toolkit/templates/snippets/_kubernetes_pod_rbac_serviceaccount.tpl | 2 ++
1 file changed, 2 insertions(+)
@ -20,6 +21,6 @@ index b4cf1a6..2f4113b 100644
{{- range $k, $v := $deps -}}
{{- if eq $k "services" }}
{{- range $serv := $v }}
--
1.8.3.1
--
2.16.5

View File

@ -1,14 +1,15 @@
From dda42ea9d18cacb7059652e95fb1b689f175f6ac Mon Sep 17 00:00:00 2001
From 93ec2454cba41bf3de1419bada1f145f1ca9dbd9 Mon Sep 17 00:00:00 2001
From: Al Bailey <Al.Bailey@windriver.com>
Date: Wed, 20 Feb 2019 13:56:27 -0600
Subject: [PATCH 6/6] Set Min NGINX handles
Subject: [PATCH 3/5] Set Min NGINX handles
Signed-off-by: Robert Church <robert.church@windriver.com>
---
mariadb/files/nginx.tmpl | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/mariadb/files/nginx.tmpl b/mariadb/files/nginx.tmpl
index b74b2b6..9775d89 100644
index 5ec3d0d..07b7cc5 100644
--- a/mariadb/files/nginx.tmpl
+++ b/mariadb/files/nginx.tmpl
@@ -23,7 +23,9 @@ daemon off;
@ -20,8 +21,8 @@ index b74b2b6..9775d89 100644
+worker_rlimit_nofile 2048;
+{{else}}
worker_rlimit_nofile {{ .MaxOpenFiles }};
{{ end}}
{{ end }}
--
1.8.3.1
2.16.5

View File

@ -0,0 +1,50 @@
From 71972bb0f45ca2bdd1aefa70061d087400b5251c Mon Sep 17 00:00:00 2001
From: Robert Church <robert.church@windriver.com>
Date: Fri, 22 Mar 2019 13:02:23 -0400
Subject: [PATCH 4/5] Enable Ceph Jewel support for gnocchi
Current upstream charts align to Ceph Luminous/Mimic. Revert
functionality that is not present in Jewel.
Drop this after the Ceph rebase to Mimic is complete.
Signed-off-by: Robert Church <robert.church@windriver.com>
---
gnocchi/templates/bin/_storage-init.sh.tpl | 20 ++++++++------------
1 file changed, 8 insertions(+), 12 deletions(-)
diff --git a/gnocchi/templates/bin/_storage-init.sh.tpl b/gnocchi/templates/bin/_storage-init.sh.tpl
index 328d27b..39a0f8c 100644
--- a/gnocchi/templates/bin/_storage-init.sh.tpl
+++ b/gnocchi/templates/bin/_storage-init.sh.tpl
@@ -36,19 +36,15 @@ function ensure_pool () {
ensure_pool ${RBD_POOL_NAME} ${RBD_POOL_CHUNK_SIZE} "gnocchi-metrics"
if USERINFO=$(ceph auth get client.${RBD_POOL_USER}); then
- echo "Cephx user client.${RBD_POOL_USER} already exist."
- echo "Update its cephx caps"
- ceph auth caps client.${RBD_POOL_USER} \
- mon "profile r" \
- osd "profile rwx pool=${RBD_POOL_NAME}" \
- mgr "allow r"
- ceph auth get client.${RBD_POOL_USER} -o ${KEYRING}
+ KEYSTR=$(echo $USERINFO | sed 's/.*\( key = .*\) caps mon.*/\1/')
+ echo $KEYSTR > ${KEYRING}
else
- ceph auth get-or-create client.${RBD_POOL_USER} \
- mon "profile r" \
- osd "profile rwx pool=${RBD_POOL_NAME}" \
- mgr "allow r" \
- -o ${KEYRING}
+ #NOTE(Portdirect): Determine proper privs to assign keyring
+ ceph auth get-or-create client.${RBD_POOL_USER} \
+ mon "allow *" \
+ osd "allow *" \
+ mgr "allow *" \
+ -o ${KEYRING}
fi
ENCODED_KEYRING=$(sed -n 's/^[[:blank:]]*key[[:blank:]]\+=[[:blank:]]\(.*\)/\1/p' ${KEYRING} | base64 -w0)
--
2.16.5

View File

@ -0,0 +1,65 @@
From b3829fef30e76fdf498fa1d0d35185f642dce5f6 Mon Sep 17 00:00:00 2001
From: Robert Church <robert.church@windriver.com>
Date: Mon, 8 Apr 2019 02:12:39 -0400
Subject: [PATCH 5/5] Partial revert of
31e3469d28858d7b5eb6355e88b6f49fd62032be
Suspect that new use of mergeOverwrite vs. merge is breaking the
per-host DaemonSet overrides.
Signed-off-by: Robert Church <robert.church@windriver.com>
---
helm-toolkit/templates/utils/_daemonset_overrides.tpl | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/helm-toolkit/templates/utils/_daemonset_overrides.tpl b/helm-toolkit/templates/utils/_daemonset_overrides.tpl
index c02de9e..ef52592 100644
--- a/helm-toolkit/templates/utils/_daemonset_overrides.tpl
+++ b/helm-toolkit/templates/utils/_daemonset_overrides.tpl
@@ -48,10 +48,10 @@ limitations under the License.
{{/* apply overrides */}}
{{- $override_conf_copy := $host_data.conf }}
{{- $root_conf_copy := omit $context.Values.conf "overrides" }}
- {{- $merged_dict := mergeOverwrite $root_conf_copy $override_conf_copy }}
+ {{- $merged_dict := merge $override_conf_copy $root_conf_copy }}
{{- $root_conf_copy2 := dict "conf" $merged_dict }}
{{- $context_values := omit $context.Values "conf" }}
- {{- $root_conf_copy3 := mergeOverwrite $context_values $root_conf_copy2 }}
+ {{- $root_conf_copy3 := merge $context_values $root_conf_copy2 }}
{{- $root_conf_copy4 := dict "Values" $root_conf_copy3 }}
{{- $_ := set $current_dict "nodeData" $root_conf_copy4 }}
@@ -87,10 +87,10 @@ limitations under the License.
{{/* apply overrides */}}
{{- $override_conf_copy := $label_data.conf }}
{{- $root_conf_copy := omit $context.Values.conf "overrides" }}
- {{- $merged_dict := mergeOverwrite $root_conf_copy $override_conf_copy }}
+ {{- $merged_dict := merge $override_conf_copy $root_conf_copy }}
{{- $root_conf_copy2 := dict "conf" $merged_dict }}
{{- $context_values := omit $context.Values "conf" }}
- {{- $root_conf_copy3 := mergeOverwrite $context_values $root_conf_copy2 }}
+ {{- $root_conf_copy3 := merge $context_values $root_conf_copy2 }}
{{- $root_conf_copy4 := dict "Values" $root_conf_copy3 }}
{{- $_ := set $context.Values.__current_label "nodeData" $root_conf_copy4 }}
@@ -185,7 +185,7 @@ limitations under the License.
{{- $root_conf_copy1 := omit $context.Values.conf "overrides" }}
{{- $root_conf_copy2 := dict "conf" $root_conf_copy1 }}
{{- $context_values := omit $context.Values "conf" }}
- {{- $root_conf_copy3 := mergeOverwrite $context_values $root_conf_copy2 }}
+ {{- $root_conf_copy3 := merge $context_values $root_conf_copy2 }}
{{- $root_conf_copy4 := dict "Values" $root_conf_copy3 }}
{{- $_ := set $context.Values.__default "nodeData" $root_conf_copy4 }}
@@ -196,7 +196,7 @@ limitations under the License.
{{- range $current_dict := $context.Values.__daemonset_list }}
{{- $context_novalues := omit $context "Values" }}
- {{- $merged_dict := mergeOverwrite $context_novalues $current_dict.nodeData }}
+ {{- $merged_dict := merge $current_dict.nodeData $context_novalues }}
{{- $_ := set $current_dict "nodeData" $merged_dict }}
{{/* Deep copy original daemonset_yaml */}}
{{- $_ := set $context.Values "__daemonset_yaml" ($daemonset_yaml | toYaml | fromYaml) }}
--
2.16.5

View File

@ -1,329 +0,0 @@
From 896385354e535d68f7ee06074bb8266c0f1b7055 Mon Sep 17 00:00:00 2001
From: Pete Birley <pete@port.direct>
Date: Sat, 1 Dec 2018 18:52:39 -0600
Subject: [PATCH] Mariadb: Support adoption of running single node mariadb
deployment
This PS updates the mariadb chart to both support adoption of a
single instance of mariadb running the bash driven chart, which
did not support reforming a galera cluster by tracking state using
a configmap. Additionally basic logic is added for upgrading the
database as part of the normal rolling update flow.
Change-Id: I412de507112b38d6d2534e89f2a02f84bef3da63
Signed-off-by: Pete Birley <pete@port.direct>
---
mariadb/templates/bin/_start.py.tpl | 168 +++++++++++++++++++++++----------
mariadb/templates/etc/_00-base.cnf.tpl | 8 +-
2 files changed, 123 insertions(+), 53 deletions(-)
diff --git a/mariadb/templates/bin/_start.py.tpl b/mariadb/templates/bin/_start.py.tpl
index 8a0236b..4d4428c 100644
--- a/mariadb/templates/bin/_start.py.tpl
+++ b/mariadb/templates/bin/_start.py.tpl
@@ -48,6 +48,10 @@ logger.addHandler(ch)
local_hostname = socket.gethostname()
logger.info("This instance hostname: {0}".format(local_hostname))
+# Get the instance number
+instance_number = local_hostname.split("-")[-1]
+logger.info("This instance number: {0}".format(instance_number))
+
# Setup k8s client credentials and check api version
kubernetes.config.load_incluster_config()
kubernetes_version = kubernetes.client.VersionApi().get_code().git_version
@@ -109,6 +113,7 @@ def ensure_state_configmap(pod_namespace, configmap_name, configmap_body):
except:
k8s_api_instance.create_namespaced_config_map(
namespace=pod_namespace, body=configmap_body)
+
return False
@@ -351,13 +356,36 @@ def get_cluster_state():
except:
logger.info("The cluster configmap \"{0}\" does not exist.".format(
state_configmap_name))
+ time.sleep(default_sleep)
+ leader_expiry_raw = datetime.utcnow() + timedelta(
+ seconds=cluster_leader_ttl)
+ leader_expiry = "{0}Z".format(leader_expiry_raw.isoformat("T"))
+ if check_for_active_nodes():
+ # NOTE(portdirect): here we make the assumption that the 1st pod
+ # in an existing statefulset is the one to adopt as leader.
+ leader = "{0}-0".format("-".join(
+ local_hostname.split("-")[:-1]))
+ state = "live"
+ logger.info(
+ "The cluster is running already though unmanaged \"{0}\" will be declared leader in a \"{1}\" state".
+ format(leader, state))
+ else:
+ leader = local_hostname
+ state = "new"
+ logger.info(
+ "The cluster is new \"{0}\" will be declared leader in a \"{1}\" state".
+ format(leader, state))
+
initial_configmap_body = {
"apiVersion": "v1",
"kind": "ConfigMap",
"metadata": {
"name": state_configmap_name,
"annotations": {
- "openstackhelm.openstack.org/cluster.state": "new"
+ "openstackhelm.openstack.org/cluster.state": state,
+ "openstackhelm.openstack.org/leader.node": leader,
+ "openstackhelm.openstack.org/leader.expiry":
+ leader_expiry
}
},
"data": {}
@@ -369,14 +397,11 @@ def get_cluster_state():
return state
-def declare_myself_cluser_leader(ttl):
- """Declare the current pod as the cluster leader.
-
- Keyword arguments:
- ttl -- the ttl for the leader period
- """
+def declare_myself_cluser_leader():
+ """Declare the current pod as the cluster leader."""
logger.info("Declaring myself current cluster leader")
- leader_expiry_raw = datetime.utcnow() + timedelta(seconds=120)
+ leader_expiry_raw = datetime.utcnow() + timedelta(
+ seconds=cluster_leader_ttl)
leader_expiry = "{0}Z".format(leader_expiry_raw.isoformat("T"))
set_configmap_annotation(
key='openstackhelm.openstack.org/leader.node', value=local_hostname)
@@ -393,10 +418,10 @@ def deadmans_leader_election():
if iso8601.parse_date(leader_expiry).replace(
tzinfo=None) < datetime.utcnow().replace(tzinfo=None):
logger.info("Current cluster leader has expired")
- declare_myself_cluser_leader(ttl=cluster_leader_ttl)
+ declare_myself_cluser_leader()
elif local_hostname == leader_node:
logger.info("Renewing cluster leader lease")
- declare_myself_cluser_leader(ttl=cluster_leader_ttl)
+ declare_myself_cluser_leader()
def get_grastate_val(key):
@@ -452,43 +477,47 @@ def update_grastate_configmap():
def update_grastate_on_restart():
"""Update the grastate.dat on node restart."""
logger.info("Updating grastate info for node")
- if get_grastate_val(key='seqno') == '-1':
- logger.info(
- "Node shutdown was not clean, getting position via wsrep-recover")
-
- def recover_wsrep_position():
- """Extract recoved wsrep position from uncleanly exited node."""
- wsrep_recover = subprocess.Popen(
- [
- 'mysqld', '--bind-address=127.0.0.1',
- '--wsrep_cluster_address=gcomm://', '--wsrep-recover'
- ],
- stdout=subprocess.PIPE,
- stderr=subprocess.PIPE)
- out, err = wsrep_recover.communicate()
- for item in err.split("\n"):
- if "WSREP: Recovered position:" in item:
- line = item.strip().split()
- wsrep_rec_pos = line[-1].split(':')[-1]
- return wsrep_rec_pos
-
- set_grastate_val(key='seqno', value=recover_wsrep_position())
- else:
- logger.info("Node shutdown was clean, using grastate.dat")
+ if os.path.exists('/var/lib/mysql/grastate.dat'):
+ if get_grastate_val(key='seqno') == '-1':
+ logger.info(
+ "Node shutdown was not clean, getting position via wsrep-recover"
+ )
+
+ def recover_wsrep_position():
+ """Extract recoved wsrep position from uncleanly exited node."""
+ wsrep_recover = subprocess.Popen(
+ [
+ 'mysqld', '--bind-address=127.0.0.1',
+ '--wsrep_cluster_address=gcomm://', '--wsrep-recover'
+ ],
+ stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE)
+ out, err = wsrep_recover.communicate()
+ for item in err.split("\n"):
+ if "WSREP: Recovered position:" in item:
+ line = item.strip().split()
+ wsrep_rec_pos = line[-1].split(':')[-1]
+ return wsrep_rec_pos
+
+ set_grastate_val(key='seqno', value=recover_wsrep_position())
+ else:
+ logger.info("Node shutdown was clean, using grastate.dat")
- update_grastate_configmap()
+ update_grastate_configmap()
+ else:
+ logger.info("No grastate.dat exists I am a new node")
-def check_for_active_nodes(endpoints_name=direct_svc_name,
- namespace=pod_namespace):
- """Check K8s endpoints to see if there are active Mariadb Instances.
+
+def get_active_endpoints(endpoints_name=direct_svc_name,
+ namespace=pod_namespace):
+ """Returns a list of active endpoints.
Keyword arguments:
endpoints_name -- endpoints to check for active backends
(default direct_svc_name)
namespace -- namespace to check for endpoints (default pod_namespace)
"""
- logger.info("Checking for active nodes")
endpoints = k8s_api_instance.read_namespaced_endpoints(
name=endpoints_name, namespace=pod_namespace)
endpoints_dict = endpoints.to_dict()
@@ -496,6 +525,20 @@ def check_for_active_nodes(endpoints_name=direct_svc_name,
i for i, s in enumerate(endpoints_dict['subsets']) if 'addresses' in s
][0]
active_endpoints = endpoints_dict['subsets'][addresses_index]['addresses']
+ return active_endpoints
+
+
+def check_for_active_nodes(endpoints_name=direct_svc_name,
+ namespace=pod_namespace):
+ """Check K8s endpoints to see if there are active Mariadb Instances.
+
+ Keyword arguments:
+ endpoints_name -- endpoints to check for active backends
+ (default direct_svc_name)
+ namespace -- namespace to check for endpoints (default pod_namespace)
+ """
+ logger.info("Checking for active nodes")
+ active_endpoints = get_active_endpoints()
if active_endpoints and len(active_endpoints) >= 1:
return True
else:
@@ -608,7 +651,11 @@ def launch_leader_election():
def run_mysqld(cluster='existing'):
- """Launch the mysqld instance for the pod.
+ """Launch the mysqld instance for the pod. This will also run mysql upgrade
+ if we are the 1st replica, and the rest of the cluster is already running.
+ This senario will be triggerd either following a rolling update, as this
+ works in reverse order for statefulset. Or restart of the 1st instance, in
+ which case the comand should be a no-op.
Keyword arguments:
cluster -- whether we going to form a cluster 'new' or joining an existing
@@ -621,18 +668,28 @@ def run_mysqld(cluster='existing'):
mysqld_cmd = ['mysqld']
if cluster == 'new':
mysqld_cmd.append('--wsrep-new-cluster')
+ else:
+ if int(instance_number) == 0:
+ active_endpoints = get_active_endpoints()
+ if active_endpoints and len(active_endpoints) == (
+ int(mariadb_replicas) - 1):
+ run_cmd_with_logging([
+ 'mysql_upgrade',
+ '--defaults-file=/etc/mysql/admin_user.cnf'
+ ], logger)
+
run_cmd_with_logging(mysqld_cmd, logger)
def mysqld_reboot():
"""Reboot a mysqld cluster."""
- declare_myself_cluser_leader(ttl=cluster_leader_ttl)
+ declare_myself_cluser_leader()
set_grastate_val(key='safe_to_bootstrap', value='1')
run_mysqld(cluster='new')
def sigterm_shutdown(x, y):
- """Shutdown the instnace of mysqld on shutdown signal."""
+ """Shutdown the instance of mysqld on shutdown signal."""
logger.info("Got a sigterm from the container runtime, time to go.")
stop_mysqld()
@@ -642,15 +699,26 @@ signal.signal(signal.SIGTERM, sigterm_shutdown)
# Main logic loop
if get_cluster_state() == 'new':
- set_configmap_annotation(
- key='openstackhelm.openstack.org/cluster.state', value='init')
- declare_myself_cluser_leader(ttl=cluster_leader_ttl)
- launch_leader_election()
- mysqld_bootstrap()
- update_grastate_configmap()
- set_configmap_annotation(
- key='openstackhelm.openstack.org/cluster.state', value='live')
- run_mysqld(cluster='new')
+ leader_node = get_configmap_value(
+ type='annotation', key='openstackhelm.openstack.org/leader.node')
+ if leader_node == local_hostname:
+ set_configmap_annotation(
+ key='openstackhelm.openstack.org/cluster.state', value='init')
+ declare_myself_cluser_leader()
+ launch_leader_election()
+ mysqld_bootstrap()
+ update_grastate_configmap()
+ set_configmap_annotation(
+ key='openstackhelm.openstack.org/cluster.state', value='live')
+ run_mysqld(cluster='new')
+ else:
+ logger.info("Waiting for cluster to start running")
+ while not get_cluster_state() == 'live':
+ time.sleep(default_sleep)
+ while not check_for_active_nodes():
+ time.sleep(default_sleep)
+ launch_leader_election()
+ run_mysqld()
elif get_cluster_state() == 'init':
logger.info("Waiting for cluster to start running")
while not get_cluster_state() == 'live':
diff --git a/mariadb/templates/etc/_00-base.cnf.tpl b/mariadb/templates/etc/_00-base.cnf.tpl
index fc0b079..949d867 100644
--- a/mariadb/templates/etc/_00-base.cnf.tpl
+++ b/mariadb/templates/etc/_00-base.cnf.tpl
@@ -21,7 +21,7 @@ collation_server=utf8_unicode_ci
skip-character-set-client-handshake
# Logging
-slow_query_log=on
+slow_query_log=off
slow_query_log_file=/var/log/mysql/mariadb-slow.log
log_warnings=2
@@ -75,9 +75,11 @@ table_definition_cache=1024
# TODO(tomasz.paszkowski): This needs to by dynamic based on available RAM.
innodb_buffer_pool_size=1024M
innodb_doublewrite=0
+innodb_file_format=Barracuda
innodb_file_per_table=1
innodb_flush_method=O_DIRECT
innodb_io_capacity=500
+innodb_locks_unsafe_for_binlog=1
innodb_log_file_size=128M
innodb_old_blocks_time=1000
innodb_read_io_threads=8
@@ -93,9 +95,9 @@ wsrep_on=1
wsrep_provider=/usr/lib/galera/libgalera_smm.so
wsrep_provider_options="gmcast.listen_addr=tcp://0.0.0.0:{{ tuple "oslo_db" "direct" "wsrep" . | include "helm-toolkit.endpoints.endpoint_port_lookup" }}"
wsrep_slave_threads=12
-# FIX_ME(portdirect): https://mariadb.com/kb/en/library/mariabackup-overview/#granting-privileges-for-ssts
wsrep_sst_auth=root:{{ .Values.endpoints.oslo_db.auth.admin.password }}
-wsrep_sst_method=mariabackup
+# FIXME(portdirect): use rsync for compatibility between image variations
+wsrep_sst_method=rsync
[mysqldump]
max-allowed-packet=16M
--
1.8.3.1

View File

@ -1,34 +0,0 @@
From d983c89dbce840fad50f49e4253ecc7930f15338 Mon Sep 17 00:00:00 2001
From: Chris Friesen <chris.friesen@windriver.com>
Date: Wed, 6 Feb 2019 17:19:39 -0600
Subject: [PATCH] fix type error to streamline single-replica mariadb startup
The mariadb startup script was trying to optimize the single-replica
case but missed the fact that the variable it was checking was a
string rather than an int.
Converting it to an int before doing the comparison makes it work
as expected.
Change-Id: I8612e9e8ef5ec8ff61ecf0782f262a5feafd501a
Signed-off-by: Chris Friesen <chris.friesen@windriver.com>
---
mariadb/templates/bin/_start.py.tpl | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/mariadb/templates/bin/_start.py.tpl b/mariadb/templates/bin/_start.py.tpl
index 4d4428c..d0b9c8e 100644
--- a/mariadb/templates/bin/_start.py.tpl
+++ b/mariadb/templates/bin/_start.py.tpl
@@ -729,7 +729,7 @@ elif get_cluster_state() == 'init':
run_mysqld()
elif get_cluster_state() == 'live':
logger.info("Cluster has been running starting restore/rejoin")
- if not mariadb_replicas > 1:
+ if not int(mariadb_replicas) > 1:
logger.info(
"There is only a single node in this cluster, we are good to go")
update_grastate_on_restart()
--
1.8.3.1