Make selenium v4 syntax optional using the same pattern as
https://review.opendev.org/c/openstack/openstack-helm-infra/+/892708
See:
https: //review.opendev.org/c/openstack/openstack-helm-infra/+/883894/5/grafana/templates/bin/_selenium-tests.py.tpl
Change-Id: I744b721750c474db9fecbd46280d30cfb8347a6f
The run migrator job should be deployed
if .Values.manifests.job_run_migrator is true
story: 2010696
task: 47809
Change-Id: I5717d8c6c2de270268a0e14ee4b8abb7969b07bb
The main goal of this PS is to make sure the migrator can complete the migrations even if mariadb galera cluster dropped the migrator connection leaving the database in inconsistent state. It may happen that migration_log has a record of a successfully performed migration while the database scheme misses an entity so any further attempts to re-run the migrator fail because of missed entity the migrator expects to be present.
Also the migrator is running mariadb image as a main one and grafana binaries are mounted as /usr/share/grafana. Migrator job container is running under nobody user uid.
This PS runs migrator in a safe way:
- prepares database backup
- runs a single instance of grafana as migrator with log file as a background process in a loop
- constantly checks the log file in the main process
- in case of the migrations completed it stops grafana-server process and completed the job
- in case of a migration error it restores the previously prepared backup so the grafana-server that is running in a background loop can re-try the migration
- the database operations are prefixed with code that makes sure the database is reachable.
Change-Id: I4e1542b62777f25c08ddd2cb74f0a0e7bfea5145
This PS updates Grafana to 8.5.10 and enables unified alerting
which will be the default in 9.x.
The following directories are required for unified alerting:
/var/lib/grafana/alerting
/var/lib/grafana/csv
If a user is upgrading from Grafana 7.x and wants to opt out of unified
alerting they will need to set:
[alerting]
enabled = true
[unified_alerting]
enabled = false
in grafana.ini. Additionally, to roll back, the env var
GF_DEFAULT_FORCE_MIGRATION: true
needs to be set for the grafana_run_migrator pod.
Unified alerting doc:
https://grafana.com/docs/grafana/v9.0/alerting/migrating-alerts/
Change-Id: I3a6ca005b9d9433e958802e7e978b81479a16fb8
Based on spec in openstack-helm repo,
support-OCI-image-registry-with-authentication-turned-on.rst
Each Helm chart can configure an OCI image registry and
credentials to use. A Kubernetes secret is then created with these
info. Service Accounts then specify an imagePullSecret specifying
the Secret with creds for the registry. Then any pod using one
of these ServiceAccounts may pull images from an authenticated
container registry.
Change-Id: Iebda4c7a861aa13db921328776b20c14ba346269
During the first run, grafana will run migrator job, which populates
necessary fields in database. Previously, if there are two or more
grafana pods which start up simultaneously, it causes the race condition
for database access and finally one of the pods will fail (in some
cases both of them), leaving the grafana database in incomplete state.
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: I5a7993b3cad2d48af3f73218d6c61c216520e1c5
The metric ceph_pool_bytes_used has changed to ceph_pool_stored.
https: //tracker.ceph.com/issues/39932
Change-Id: Iab5cf2b318ce538e72b4592dedd8f0e489741797
This patchset also refactor the handling of dashboards yaml
files so that multiple configmaps, grouped by functionality
will be created.
Change-Id: I9849e2a2744e1d2ae895d3e18647b9b3a1c38b12
This change updates the helm-toolkit path in each chart as part
of the move to helm v3. This is due to a lack of helm serve.
Change-Id: I011e282616bf0b5a5c72c1db185c70d8c721695e
If labels are not specified on a Job, kubernetes defaults them
to include the labels of their underlying Pod template. Helm 3
injects metadata into all resources [0] including a
`app.kubernetes.io/managed-by: Helm` label. Thus when kubernetes
sees a Job's labels they are no longer empty and thus do not get
defaulted to the underlying Pod template's labels. This is a
problem since Job labels are depended on by
- Armada pre-upgrade delete hooks
- Armada wait logic configurations
- kubernetes-entrypoint dependencies
Thus for each Job template this adds labels matching the
underlying Pod template to retain the same labels that were
present with Helm 2.
[0]: https://github.com/helm/helm/pull/7649
Change-Id: I3b6b25fcc6a1af4d56f3e2b335615074e2f04b6d
This patch set adds a new Alertmanager dashboard to Grafana. Note
that a new configmap is created for this instead of using the
same configmap which includes all the dashboards. Using the same
configmap will eventually run into issue with configmap size limitation.
Change-Id: I10561c0b0b464c3b67d4a738f9f2cb70ef601b3d
This will ease mirroring capabilities for the docker official images.
Signed-off-by: Thiago Brito <thiago.brito@windriver.com>
Change-Id: I0f9177b0b83e4fad599ae0c3f3820202bf1d450d
This ps updates the version to 7.4.5
The css element in GUI has changed in the new Grafana version , so updated
the selenium script to find the right element.
Change-Id: I972fddf73719fb5bef821679a5ab07491edf55ff
This patchset enables TLS path between Prometheus and Grafana.
Grafana pull data from Prometheus. As such, Prometheus is the
server and Grafana is the client for TLS handshake.
Change-Id: I50cb6f59472155415cff16a81ebaebd192064d65
This patchset updates grafana chart manifests to enable TLS with
osh-infra mariadb. TLS for osh-infra mariadb is completed as part
of TLS phase 1, no additional mariadb chart work is required to
work with this patch.
Change-Id: Ic01e7c012ab2167d59b2117eb985386666f2bb2a
Grafana v7.3 has ability to add the customized dashboard as homepage.
This removes the necessity to have explicit job to add a homepage.
Clean up the explicit add_home-dashboard job.
Change-Id: I68baa7693f545e2d91dba01ae61b4bdae8a26bee
This brings the Grafana version up to the current version
and fixes the selenium helm and gate test for the new login
dashboard.
Change-Id: I0b65412f4689c763b3f035055ecbb4ca63c21048
Since we introduced chart version check in gates, requirements are not
satisfied with strict check of 0.1.0
Change-Id: I15950b735b4f8566bc0018fe4f4ea9ba729235fc
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
Added chart lint in zuul CI to enhance the stability for charts.
Fixed some lint errors in the current charts.
Change-Id: I9df4024c7ccf8b3510e665fc07ba0f38871fcbdb
Implement helm-toolkit snippet for grafana add-home-dashboard
which adds security context template at pod/container
Change-Id: I12a5fd6c5043079f830eb36043f5b0ca495a3e93
Unrestrict octal values rule since benefits of file modes readability
exceed possible issues with yaml 1.2 adoption in future k8s versions.
These issues will be addressed when/if they occur.
Also ensure osh-infra is a required project for lint job, that matters
when running job against another project.
Change-Id: Ic5e327cf40c4b09c90738baff56419a6cef132da
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
This commit rewrites lint job to make template linting available.
Currently yamllint is run in warning mode against all templates
rendered with default values. Duplicates detected and issues will be
addressed in subsequent commits.
Also all y*ml files are added for linting and corresponding code changes
are made. For non-templates warning rules are disabled to improve
readability. Chart and requirements yamls are also modified in the name
of consistency.
Change-Id: Ife6727c5721a00c65902340d95b7edb0a9c77365
The current copyright refers to a non-existent group
"openstack helm authors" with often out-of-date references that
are confusing when adding a new file to the repo.
This change removes all references to this copyright by the
non-existent group and any blank lines underneath.
Change-Id: I1882738cf9757c5350a8533876fd37b5920b5235
Adding a loop to wait for the grafana dashboard to be up and contain the
OSH Home dashboard before running the script.
This should resolve the job completing before the OSH Home dashboard is
in the grafana pod.
Change-Id: I7ab20fad3ce7f7216e2b2679d863f02f97ef1ff4
This code change is to enable grafana to use the raw json format
dashboards . This is to avoid the conversion of dashboards from json to
yaml and back to json during which the format is encountering issue.
Also this will help in adding new dashboards and maintaining the old
ones.
All the exisiting dashboards under values_override folder are updated to
use raw json format.
Change-Id: I48a7db1514857e082cecbb3b57deff9174509601
This code will help to add any customized dashboard as a Home Page for
Grafana. The add_home_dashboard script will be executed after the
Grafana is deployed which sets a new Dashboard(OSH Home) as a landing
Page for a specific Organization.
Change-Id: I32b6b9cad4eaefe7d153cae797d3b3143be5c49b
In Ceph Cluster Dashboard the OSDs In, OSDs Out, OSDs Down Panel was
showing wrong values. Updated
the expression from "count" to "sum" to show the correct values.
Change-Id: I1959eeb445bf297c1ec696f3867315f05552b03e
This removes the default dashboards from the Grafana chart and
instead places them in the values_overrides directory, similar to
what was done for the Prometheus rules. As Grafana dashboards
will likely be heavily dependent upon end-user needs, the old
default dashboard configs should only be used as a reference
instead of opinionated defaults that are difficult to override.
The previous defaults made using specialized labels for dashboard
variables difficult, as they were making dangerous assumptions
about deployed namespaces and host fqdns. By removing the defaults
entirely, end users can define their own dashboards to meet their
specialized needs
Change-Id: I7def8df68371deda0b75a685363c8a73b818dd45
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This updates the Grafana chart to support the definition of
multiple datasources. This moves to defining a template in the
chart's values.yaml file that allows for inline gotpl for
defining an arbitrary number of datasources. This also updates the
grafana dashboards to include a selector for the Prometheus
datasource to use via a drop down selector. This is vetted out in
the federated monitoring job
Change-Id: I55171fed5c2b343130d135d0b42bc96ff11c4712
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This updates charts that consume images built from osh-images to
use tags other than the :latest tags. This will be followed up
with the definition of jobs to allow for vetting out of updated
images, as reliance on :latest tags assumes any change merged into
osh-images will result in functionally correct behavior (which has
shown to not be the case traditionally)
Change-Id: I181aa56ed187604dc7583d8081e53cc69eb27310
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This updates the Grafana chart to support the definition of
arbitrary environment variables to support scenarios where
additional information may be required at runtime for things like
datasource and dashboard provisioning
Change-Id: I95e4abe9030116a440c6d78a1d14dbcaaf743b40
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This updates the Grafana version deployed by default from 5.0.0 to
6.2.0
Change-Id: I39b5405cc3f3fe7754ed6544a8388ff912a4ef58
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This patch set will implement the grafana metrics related changes
required for kubernetes version upgrade to 1.16. Updates are mostly
specific to cadvisor metric labels. It is to make sure all
existing metrics are scraped and available in Prometheus so that
these can be consumed by Grafana & Nagios.
Change-Id: I74369ac49dd3f7d9f3682dd5318a3818a4d3f178
This change is to fix the issue where Kubernetes Calico dashboard
is not loading in grafana dashboard.
Found the behavior that YAML reserved keys like yes/no/y/n evaluate
to true and false in Ruby hash,so the gridPos key like 'y:' converts
to 'true:' in charts json causing the dashbaord not loading.
Hence updating gridPos panel key (y:) by wrapping it with quotes
like ('y':) to resolve the issue.
There's actually a long list of YAML reserved words with this
behavior like below-
y|Y|yes|Yes|YES|n|N|no|No|NO
|true|True|TRUE|false|False|FALSE
|on|On|ON|off|Off|OFF
Change-Id: I56b01f1312efa8de90def349518c618affb5de85
This updates the kubernetes-entrypoint image reference to consume
the publicly available kubernetes-entrypoint image that is built
and maintained under the airshipit namespace, as the stackanetes
image is no longer actively maintained
Change-Id: I5bfdc156ae228ab16da57569ac6b05a9a125cb6a
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This change updates the selenium_tests container image
to one which installs python3.
The selenium-test.py template file has been refactored
to match the structure of the selenium tests in
/tools/gate/selenium
Depends on: https://review.opendev.org/688436
Change-Id: I4ece5c71df18c21f0cdff536140f63881ff24e30
Ritchie, Frank (fr801x)