Commit Graph

674 Commits

Author SHA1 Message Date
Vladimir Kozhukalov 67fae419b8 Remove ingress chart
We have not been using it for a while since some
time ago we switched to the upstream ingress-nginx.

Change-Id: I2afe101cec2ddc562190812fc27bb3fad11469f1
2024-04-20 16:15:42 -05:00
Vladimir Kozhukalov a23312374d Clean up outdated deploy k8s scripts
Change-Id: I8481869a6547feae2ac057b65c8c4aecc2c1f505
2024-04-14 20:46:11 -05:00
Stephen Taylor 1e84d3f714 [rook-ceph] Add a script to migrate Ceph clusters to Rook
This change adds a deployment script that can be used to migrate a
Ceph cluster deployed with the legacy openstack-helm-infra Ceph
charts to Rook. This process is disruptive. The Ceph cluster goes
down and comes back up multiple times during the migration, but the
end result is a Rook-deployed Ceph cluster with the original
cluster FSID and all OSD data intact.

Change-Id: Ied8ff94f25cd792a9be9f889bb6fdabc45a57f2e
2024-03-21 19:36:37 +00:00
Vladimir Kozhukalov cbbeebb5a1 Fix registry bootstrap values
The quay.io/airshipit/kubernetes-entrypoint:v1.0.0 image format is
deprecated and not supported any more by the docker registry.

This is temporary fix to download the image from third party repo
until we update the quay.io/airshipit/kubernetes-entrypoint:v1.0.0.

The deprecation message is as follows:

[DEPRECATION NOTICE] Docker Image Format v1 and Docker
Image manifest version 2, schema 1 support is disabled
by default and will be removed in an upcoming release.
Suggest the author of quay.io/airshipit/kubernetes-entrypoint:v1.0.0
to upgrade the image to the OCI Format or Docker Image
manifest v2, schema 2. More information at
https://docs.docker.com/go/deprecated-image-specs/

The docker-registry container must start not
earlier than docker-images PVC is bound.

Change-Id: I6bff98aa7d0b23e13a17a038f3039b7956703d40
2024-03-20 22:01:26 -05:00
Zuul 4df05848f5 Merge "Update Ceph images to patched 18.2.2 and restore debian-reef repo" 2024-03-13 16:51:25 +00:00
Stephen Taylor 2fd438b4b1 Update Ceph images to patched 18.2.2 and restore debian-reef repo
This change updates the Ceph images to 18.2.2 images patched with a
fix for https://tracker.ceph.com/issues/63684. It also reverts the
package repository in the deployment scripts to use the debian-reef
directory on download.ceph.com instead of debian-18.2.1. The issue
with the repo that prompted the previous change to debian-18.2.1
has been resolved and the more generic debian-reef directory may
now be used again.

Change-Id: I85be0cfa73f752019fc3689887dbfd36cec3f6b2
2024-03-12 13:45:42 -06:00
astebenkova 1953f869ad Include values_overrides for OpenStack components
Fixes issue where override files for OS charts were
missing due to specifying the wrong project directory.

Change-Id: I4af6715a33c7de43068ed76a8115c12a2c0969ed
2024-03-11 17:21:04 +02:00
Sergiy Markin 3a2399c99f Workaround for debian-reef folder issue
This PS changes ceph repo to debian-18.2.1 from
debian-reef due to some issues with debian-reef
folder at https://download.ceph.com/

Change-Id: I31c501541b54d9253c334b56df975bddb13bbaeb
2024-03-04 19:35:38 +00:00
Zuul c48dffaaa2 Merge "Use upstream ingress-nginx chart" 2024-02-01 21:42:47 +00:00
Vladimir Kozhukalov 88ad17a84b Use upstream ingress-nginx chart
Change-Id: I90a1a1e27f0b821bbecfe493057eada81d4f9424
2024-02-01 10:45:05 -06:00
Vladimir Kozhukalov 03225aad49 Use containerized Openstack client
Change-Id: I17c841b74bf92fc3ac375404b27fa2562603604f
2024-01-31 13:42:43 -06:00
Stephen Taylor 2d15cb4fdf [ceph-rook] Update Rook and increase ceph-mon memory limit
This change updates Rook to the 1.13.3 release. It also increases
the memory limit for ceph-mon pods deployed by Rook to prevent
pod restarts due to liveness probe failures that sometimes result
from probes causing ceph-mon pods to hit their memory limit.

Change-Id: Ib7d28fd866a51cbc5ad0d7320ae2ef4a831276aa
2024-01-26 13:15:11 -07:00
Vladimir Kozhukalov bba74aefde Fix ceph-adapter-rook.sh script
Change-Id: I6ebcceb105781e2ca2a39ca84d4e4bc9171a5f15
2023-12-05 22:24:15 -06:00
Vladimir Kozhukalov 978507351f Add ceph-adapter-rook chart
When using Rook for managing Ceph clusters we have
to provision a minimal set of assets (keys, endpoints, etc.)
to make Openstack-Helm charts work with these Ceph clusters.

Rook provides CRDs that can be used for managing Ceph assets
like pools/keyrings/buckets etc. but Openstack-Helm can not
utilize these CRDs. To support these CRDs in OSH would
require having lots of conditionals in OSH templates since
we still want OSH to work with OSH ceph-* charts.

Change-Id: If7fe29052640e48c37b653e13a74d95e360a6d16
2023-12-05 14:27:57 -06:00
Sergiy Markin 29f2b616cc [mariadb-operator] Mariadb-cluster chart
This PS adds mariadb-cluster chart based on mariadb-operator. Also for
some backward compartibility this PS adds mariadb-backup chart and
prometheus-mysql-exporter chart as a separate ones.

Change-Id: I3f652375cce2e3b45e095e08d2e6f4ae73b8d8f0
2023-11-29 21:51:48 -06:00
Vladimir Kozhukalov 3d64d4c832 Update get-values-overrides.sh script
The PR synchronized this script with that
used in the openstack-helm repo.

Let's use the same script in both repos.
The related PR for the openstack-helm repo
is coming.

Change-Id: I5cfaad8ebfd08790ecabb3e8fa480a7bf2bb7e1e
2023-11-29 22:56:42 +00:00
Vladimir Kozhukalov 730488ca53 Disable metrics gathering for Rook Ceph cluster
We don't need this for tests and it is better to
keep the test env minimal since the test hardware
is limited.

Change-Id: I0b3f663408c1ef57ad25a4d031b706cb6abc87a9
2023-11-29 15:47:20 -06:00
Vladimir Kozhukalov 7f783dba51 Update elasticsearch chart to work with Rook Ceph
When using Rook for managing Ceph we can use
Rook CRDs to create S3 buckets and users.

This PR adds bucket claim template to the
elasticsearch chart. Rook creates a bucket for
a bucket claim and also creates a secret
containing the credentials to get access to this
bucket. So we also add a snippet to expose
these credentials via environment variables to
containers where they are needed.

Change-Id: Ic5cd35a5c64a914af97d2b3cfec21dbe399c0f14
2023-11-26 19:34:42 -06:00
Vladimir Kozhukalov 145e9df9b7 Run Rook job on multi-node env
Change-Id: Idce9fd9f4817e0dd07b49c291fa6a0a887384073
2023-11-24 19:14:37 -06:00
Vladimir Kozhukalov 510cea0c23 Deploy Ceph on multi-node envs
- In case we deploy Ceph on a multi-node env we have
  to prepare the loop devices on all nodes. For this
  we moved loop devices setup to the deploy-env
  Ansible role.

  For simplicity we need the same device on all nodes,
  so we create a loop device with a big
  minor number (/dev/loop100 by default) hoping
  that only low minor numbers could be busy.

- For test jobs we don't need to use different devices
  for OSD data and metadata. There is no
  any benefit from this for the test environment.
  So let's keep it simple and put both OSD data and metadata
  on the same device.

- On multi-node env Ceph cluster needs cluster members
  see each other, so let's use pod network CIDR.

Change-Id: I493b6c31d97ff2fc4992c6bb1994d0c73320cd7b
2023-11-24 19:06:08 -06:00
Vladimir Kozhukalov c047fce569 Fix path for setup-client.sh script
Change-Id: Ieb7549d2f00d981efa1d4bc2d6d8a57a067ef6c7
2023-11-06 10:34:20 -06:00
Zuul ff552d5969 Merge "Create osh-bandit role" 2023-10-27 22:12:27 +00:00
Sergiy Markin a430d16bd5 Control ceph loopback devices path
This PS adds control over location of Ceph loopback devices path.

Change-Id: Ib0738c1127ff37633cdd035b3978cc137c5eaf71
2023-10-26 17:28:45 +00:00
Vladimir Kozhukalov 45b209ac79 Create osh-bandit role
The motivation is to reduce the code base and get rid
of unnecessary duplications. This PR is moves bandit
tasks from the osh-infra-bandit.yaml playbook
to the osh-bandit role. Then we can use this role for the
same job in OSH.

Change-Id: I9489a8c414e6679186e6c399243a7c0838df812a
2023-10-25 17:59:56 -05:00
Stephen Taylor cdfb3ce6a4 Downgrade Rook to the 1.12.4 release
Roll back Rook in the openstack-support-rook Zuul job to the 1.12.4
release to work around a problem with ceph-rook-exporter resource
conflicts while the issue is investigated further.

Change-Id: Idabc1814e9b8665c0ce63e2efd5ad94bf193f97a
2023-10-24 21:22:13 +00:00
Stephen Taylor 5e5a52cc04 Update Rook to 1.12.5 and Ceph to 18.2.0
This change updates Rook to the 1.12.5 release and Ceph to the
18.2.0 (Reef) release.

Change-Id: I546780ce33b6965aa699f1578d1db9790dc4e002
2023-10-13 12:58:56 -06:00
Stephen Taylor a58f80599b [ceph] Add support for deploying and managing Ceph with Rook
This change adds an openstack-support-rook zuul job to test
deploying Ceph using the upstream Rook helm charts found in the
https://charts.rook.io/release repository. Minor changes to the
storage keyring manager job and the mon discovery service in the
ceph-mon chart are also included to allow the ceph-mon chart to be
used to generate auth keys and deploy the mon discovery service
necessary for OpenStack.

Change-Id: Iee4174dc54b6a7aac6520c448a54adb1325cccab
2023-10-03 07:16:02 -06:00
Vladimir Kozhukalov ae91cf3fc3 Use deploy-env role for all deployment jobs
To make it easier to maintain the jobs all experimental
jobs (those which are not run in check and gate pipelines)
are moved to a separate file. They will be revised later
to use the same deploy-env role.

Also many charts use Openstack images for testing this
PR adds 2023.1 Ubuntu Focal overrides for all these charts.

Change-Id: I4a6fb998c7eb1026b3c05ddd69f62531137b6e51
2023-09-22 15:02:07 -05:00
Zuul adcab364d2 Merge "Deprecating the Ingress Class Annotation" 2023-09-16 04:32:02 +00:00
Vladimir Kozhukalov b0d659f9b2 Use Ceph Reef repo while deploying K8s
Reef is compatible with Focal and Jammy

Change-Id: Ic98a5824a319e4835aa3df2a8e68a1daef97392c
2023-09-06 19:17:17 +03:00
Anselme, Schubert 51c70e48df
Deprecating the Ingress Class Annotation
This PS replaces deprecated kubernetes.io/ingress.class annotation  with
spec.ingressClassName field that is a reference to an IngressClass
resource that contains additional Ingress configuration, including the
name of the Ingress controller.

https://kubernetes.io/blog/2020/04/02/improvements-to-the-ingress-api-in-kubernetes-1.18/#deprecating-the-ingress-class-annotation

Change-Id: I9953d966b4f9f7b1692b39f36f434f5055317025
Co-authored-by: Sergiy Markin <smarkin@mirantis.com>
Co-authored-by: Leointii Istomin <listomin@mirantis.com>
Signed-off-by: Anselme, Schubert (sa246v) <sa246v@att.com>
2023-08-26 00:39:34 +00:00
Leontii Istomin 4a74ff2ba9 Upgrade ElasticSearch and Kibana to v8.9.0
Change-Id: I5ce965a2abf40bad14f0a8a505c8f3000f110d37
2023-08-24 11:09:19 -05:00
astebenkova 6b6ca9e26c [osh-selenium] Migrate deprecated functions
There are some leftovers from the following commit:
https://review.opendev.org/c/openstack/openstack-helm-infra/+/883894

Change-Id: If167646b088b361d49d33400abab131c79afedc9
2023-06-30 16:44:09 +03:00
Vladimir Kozhukalov 3d8935a536 Use --ignore-installed while install pip packages for dev env
story: 2010785
task: 48210
There were a bunch of stories like this 2010785 and in most
cases users face the conflict of pip and apt package
management systems. We can either use --ignore-installed
or use python virtualenv. The second option does not contradict
to the first one.

Change-Id: I345e887b3f35f1d1d6c86cc40a29ff0b1920a1f1
2023-06-15 06:27:22 +03:00
Samuel Liu 664f4c9dfb Remove PodSecurityPolicy
PodSecurityPolicy was deprecated in Kubernetes v1.21, and removed from Kubernetes in v1.25.[1]

In Kubernetes 1.21, PodSecurityPolicy is deprecated. As with all Kubernetes feature deprecations, PodSecurityPolicy will continue to be available and fully-functional for several more releases. PodSecurityPolicy, previously in the beta stage, is planned for removal in Kubernetes 1.25.[2]

[1] https://kubernetes.io/docs/concepts/security/pod-security-policy/
[2] https://kubernetes.io/blog/2021/04/08/kubernetes-1-21-release-announcement/#podsecuritypolicy-deprecation

Change-Id: Ic060d925b6e97e5651e74a1a1161906aef740a8c
2023-06-08 21:53:29 +03:00
Zuul b9954058bc Merge "[osh-selenium] Upgrade image to latest-ubuntu_focal" 2023-05-25 02:48:48 +00:00
astebenkova 92d16f3a29 [osh-selenium] Upgrade image to latest-ubuntu_focal
+ migrate all Python tests to use Selenium v4 (bionic image had v3 installed):
https://www.selenium.dev/documentation/webdriver/getting_started/upgrade_to_selenium_4/
+ amend selenium role in order to install ChromeDriver compatible with Google Chrome:
https://chromedriver.chromium.org/downloads/version-selection
+ run selenium tests AFTER the charts are deployed

Change-Id: I46200b7dc173bd0e1e6bf3545d9a26c252a21927
2023-05-23 18:09:16 +03:00
Vladimir Kozhukalov ebbf659e35 Revert "Update kubernetes to 1.27.1"
This reverts commit 8e96a91ffa.

Reason for revert: The change broke the compute-kit tests.
The deployment of all Openstack components is successful but then when we create networks and a VM, neutron-dhcp-agent crashes. It is still not clear why it happens. Let's revert this change and figure out what is going on.

Change-Id: I07082511cd168560c8fe8dce3421e37fc402a1ae
2023-05-23 09:20:47 +00:00
SPEARS, DUSTIN (ds443n) 8e96a91ffa Update kubernetes to 1.27.1
Update mini-kube to 1.30.1
Update crictl to 1.27.0

Change-Id: I528bad131cac4922b5663a8f7522657d26d1e020
2023-05-18 16:08:01 -04:00
Vladimir Kozhukalov 97ce6d7d8e Update kubernetes registry to registry.k8s.io
See this link for details
https://kubernetes.io/blog/2023/03/10/image-registry-redirect/

Change-Id: Ifc8b64825751933def16a1784fae987a1d7250ad
2023-04-11 04:54:26 +03:00
Sergiy Markin e6dfa15c26 Upgrade of Kubernetes
This PS upgrades the following components:
- minikube to 1.29.0
- kubernetes to 1.26.3
- calico to 3.25
- coredns to 1.9.4

Also this PS adds cri-dockerd required for kubernetes newer than 1.24
and adds recirsive response to coredns.

Change-Id: Ie8aa43642de5dfa69ed72fadbfd943b578a80a74
2023-03-30 00:37:51 +00:00
Zuul 728641cce3 Merge "[ceph] Update all Ceph images to Focal" 2023-03-17 16:46:18 +00:00
Zuul d3acf392cd Merge "Use free loop devices for ceph deployment" 2023-03-17 05:49:57 +00:00
Stephen Taylor fc92933346 [ceph] Update all Ceph images to Focal
This change updates all Ceph image references to use Focal images
for all charts in openstack-helm-infra.

Change-Id: I759d3bdcf1ff332413e14e367d702c3b4ec0de44
2023-03-16 16:39:37 -06:00
Zuul 0e4af3f0e7 Merge "Fix typo in loops-setup systemd unit definition" 2023-03-16 22:12:38 +00:00
Stephen Taylor f80049faa1 [ceph] Allow gate scripts to use 1x replication in Ceph
The Pacific release of Ceph disabled 1x replication by default, and
some of the gate scripts are not updated to allow this explicitly.
Some gate jobs fail in some configurations as a result, so this
change adds 'mon_allow_pool_size_one = true' to those Ceph gate
scripts that don't already have it, along with
--yes-i-really-mean-it added to commands that set pool size.

Change-Id: I5fb08d3bb714f1b67294bb01e17e8a5c1ddbb73a
2023-03-16 05:19:30 -06:00
Vladimir Kozhukalov c0ca3b5c99 Fix typo in loops-setup systemd unit definition
Change-Id: Ib069e1ea70d011e5167440f02b62cb91688f92e4
2023-03-09 20:28:30 +03:00
Vladimir Kozhukalov 66fb4a2b26 Use free loop devices for ceph deployment
This is for convenience when running deployment scripts
manually. On ubuntu loop0 and loop1 could in use
for snaps so we can find free loop devices before trying
to use them.

Change-Id: Iec54c0decd3a401c99f4770187d81f370bcee24c
2023-03-09 20:18:45 +03:00
Vladimir Kozhukalov 42752cca63 Remove unnecessary ceph provisioners templates
Change-Id: Ia12a99e7c97f7af701b17e1f783d772ab44b5cd7
2023-03-09 19:20:55 +03:00
Vladimir Kozhukalov c925341518 Fix ceph deployment
When deploy ceph on loop devices we need lvm2
to be installed on the host to create necessary
device links like /dev/<vgname>/<lvname>

Change-Id: I5dabbc080aa45b28c1dd5e1d883f9d45affdf60f
2023-02-15 21:39:59 +03:00