Commit Graph

204 Commits

Author SHA1 Message Date
Vladimir Kozhukalov d31027cfb4 Install OSH Helm plugin
Depends-On: I71ab6ad104beb491b5b15b7750e2fc0988db82bf
Change-Id: I8f30fbdf94d76ef9fa2985a25c033df290995326
2024-04-17 19:06:48 -05:00
Vladimir Kozhukalov cdbecfb7f4 Update test jobs
- Remove openstack-helm-infra-openstack-support* jobs.
  Instead of these jobs we run compute-kit, cinder and tls
  jobs defined in the openstack-helm repo.
- Remove all experimental jobs since they are outdated and
  do not work. We will later add some of the test cases
  including apparmor, network policy, tenant Ceph and others.

Change-Id: I8f3379c06b4595ed90de025d32c89de29614057d
2024-04-14 20:47:46 -05:00
astebenkova ed4a8acf85 Enable job for DPDK
Depends-On: I3ad5b63a0813761a23573166c5024e17d87f775d
Change-Id: I4851767a79bc4571a0f38622fe309807b53a7504
2024-04-12 17:12:17 +03:00
Vladimir Kozhukalov 51c7affc72 Run more test jobs when helm-toolkit updated
Specifically we would like at least the following
deployments to be tested when helm-toolkit is updated
- compute-kit
- cinder
- tls

Change-Id: I3991d6984563813d5a3a776eabd52e2e89933bd8
2024-04-03 14:07:23 -05:00
Vladimir Kozhukalov cfed816a9a Fix coredns resolver
Forward requests for unknown names to 8.8.8.8

NOTE: Temporarily disable DPDK job which turned to
be incompatible with this PR
https://review.opendev.org/c/openstack/openstack-helm/+/914399
It wasn't tested with the DPDK job.

Change-Id: I936fb1032a736f7b09ad50b749d37095cce4c392
2024-03-28 20:34:23 -05:00
Vladimir Kozhukalov 6ca83be780 Rename dpdk job name to reflect Openstack version
Change-Id: I9c04a60ae8b7fde35a8a970e3b74bcaad7bd564f
2024-03-26 11:22:04 -05:00
Vladimir Kozhukalov 81828430e5 Bump RabbitMQ version 3.9.0 -> 3.13.0
Also
- Update default Heat image to 2023.2 used for
  init and test jobs
- Add overrides for
  - yoga-ubuntu_focal
  - zed-ubuntu_focal
  - zed-ubuntu_jammy
  - 2023.1-ubuntu_focal
  - 2023.1-ubuntu_jammy
  - 2023.2-ubuntu_jammy

Change-Id: I516c655ea1937f9bd1d363ea86d35e05e3d54eed
2024-03-25 17:56:06 -05:00
Vladimir Kozhukalov 5b1879aa09 Refactor deploy-env role
- Make it less mixed. Each task file
  deploys one feature.
- Deploy Metallb
- Deploy Openstack provider network gateway

Change-Id: I41f0353b286f817cb562b3bd59992e4baa473568
2024-03-25 14:45:00 -05:00
Vladimir Kozhukalov fb90642b18 Update ovn controller init script
- OVN init script must be able to attach an interface
  to the provider network bridge and migrate IP from the
  interface to the bridge exactly like Neutron OVS agent
  init script does it.

- OVN init script sets gateway option to those OVN controller
  instances which are running on nodes with l3-agent=enabled
  label.

Change-Id: I24345c1f85c1e75af6e804f09d35abf530ddd6b4
2024-03-21 16:03:51 -05:00
Sergiy Markin 0e086e4c12 [mariadb] Switch to ingress-less mariadb
This PS switches mariadb to use primary service by
default instead of ingress based deployment. The
primary service that is getting created and
automatically updated based on the leader election
process in start.py entrypoint script.

Mariadb primary service was introduced by this PS:

https://review.opendev.org/c/openstack/openstack-helm-infra/+/905797

Change-Id: I4992276d0902d277a7a81f2730c22635b15794b0
2024-02-26 18:59:22 +00:00
Zuul 973041f378 Merge "Remove unused nodesets" 2024-02-22 00:34:57 +00:00
astebenkova 2216cbfec4 Add compute-kit job with DPDK enabled
+ add role for enabling hugepages

Change-Id: I89d3c09ea3bedcba6cb51178c8d1ac482a57af01
Depends-On: I2f9d954258451f64eb87d03affc079b71b00f7bd
2024-02-14 13:23:34 -06:00
Vladimir Kozhukalov cab8491389 Remove unused nodesets
Change-Id: Ifc5ea6a83729fc2313c209f683ef7476d6a14272
2024-02-05 14:31:44 -08:00
Vladimir Kozhukalov 6a452ecb49 Remove some aio jobs
These two jobs openstack-helm-infra-aio-monitoring and
openstack-helm-infra-aio-logging were only needed for
backward compatibility.

Depends-On: I9c3b8cd18178aa57ce44564490ef1b61f275ae29
Change-Id: I09d0e48128a3fd98fa9148b8e520df75d6e5be50
2024-02-05 14:17:39 -08:00
Zuul 010c21cfc9 Merge "Bump Calico version to v3.27.0" 2024-02-03 00:06:33 +00:00
Vladimir Kozhukalov cf2cdd7821 Fix prevent trailing whitespace lint command
Recently we added a jpg file to OSH documentation
but the lint job didn't run due to the job configuration.

But then for the next PR link job did run and failed
due to trailing whitespace in the jpg file.

Change-Id: I9abf8f93a4566411076190965f282375846dc5db
2024-02-02 14:12:26 -06:00
Vladimir Kozhukalov cfff60ec10 Bump Calico version to v3.27.0
Change-Id: I8daa54e70c66cec41733d6b9fd5c9dd4597ff9c1
2024-02-02 13:54:22 -06:00
Zuul c48dffaaa2 Merge "Use upstream ingress-nginx chart" 2024-02-01 21:42:47 +00:00
Vladimir Kozhukalov 88ad17a84b Use upstream ingress-nginx chart
Change-Id: I90a1a1e27f0b821bbecfe493057eada81d4f9424
2024-02-01 10:45:05 -06:00
Vladimir Kozhukalov 03225aad49 Use containerized Openstack client
Change-Id: I17c841b74bf92fc3ac375404b27fa2562603604f
2024-01-31 13:42:43 -06:00
Sergiy Markin 07bd8c92a2 [mariadb] Add mariadb-server-primary service
This PS adds mariadb-server-primary service that is getting created
and automatically updated based on the leader election process in
start.py entrypoint script.

Change-Id: I1d8a8db0ce8102e5e23f7efdeedd139726ffff28
Signed-off-by: Sergiy Markin <smarkin@mirantis.com>
2024-01-18 00:17:47 +00:00
Vladimir Kozhukalov 9e256fd8b0 Update k8s packages repo
For details see the announcement
https://kubernetes.io/blog/2023/08/31/legacy-package-repository-deprecation/

Also bump K8s version up to 1.28.4

Change-Id: Ic6b3478e53504622804b6f003ca176a679573d5b
2023-12-12 17:58:20 -06:00
Vladimir Kozhukalov 978507351f Add ceph-adapter-rook chart
When using Rook for managing Ceph clusters we have
to provision a minimal set of assets (keys, endpoints, etc.)
to make Openstack-Helm charts work with these Ceph clusters.

Rook provides CRDs that can be used for managing Ceph assets
like pools/keyrings/buckets etc. but Openstack-Helm can not
utilize these CRDs. To support these CRDs in OSH would
require having lots of conditionals in OSH templates since
we still want OSH to work with OSH ceph-* charts.

Change-Id: If7fe29052640e48c37b653e13a74d95e360a6d16
2023-12-05 14:27:57 -06:00
Sergiy Markin 29f2b616cc [mariadb-operator] Mariadb-cluster chart
This PS adds mariadb-cluster chart based on mariadb-operator. Also for
some backward compartibility this PS adds mariadb-backup chart and
prometheus-mysql-exporter chart as a separate ones.

Change-Id: I3f652375cce2e3b45e095e08d2e6f4ae73b8d8f0
2023-11-29 21:51:48 -06:00
Vladimir Kozhukalov 1c5610c2c0 Uncomment erroneously commented jobs in check pipeline
Change-Id: Icae3903cb3818e5eb5a15e93b751b3ba4ccad32e
2023-11-27 17:13:03 -06:00
Vladimir Kozhukalov 7f783dba51 Update elasticsearch chart to work with Rook Ceph
When using Rook for managing Ceph we can use
Rook CRDs to create S3 buckets and users.

This PR adds bucket claim template to the
elasticsearch chart. Rook creates a bucket for
a bucket claim and also creates a secret
containing the credentials to get access to this
bucket. So we also add a snippet to expose
these credentials via environment variables to
containers where they are needed.

Change-Id: Ic5cd35a5c64a914af97d2b3cfec21dbe399c0f14
2023-11-26 19:34:42 -06:00
Vladimir Kozhukalov 145e9df9b7 Run Rook job on multi-node env
Change-Id: Idce9fd9f4817e0dd07b49c291fa6a0a887384073
2023-11-24 19:14:37 -06:00
Vladimir Kozhukalov 510cea0c23 Deploy Ceph on multi-node envs
- In case we deploy Ceph on a multi-node env we have
  to prepare the loop devices on all nodes. For this
  we moved loop devices setup to the deploy-env
  Ansible role.

  For simplicity we need the same device on all nodes,
  so we create a loop device with a big
  minor number (/dev/loop100 by default) hoping
  that only low minor numbers could be busy.

- For test jobs we don't need to use different devices
  for OSD data and metadata. There is no
  any benefit from this for the test environment.
  So let's keep it simple and put both OSD data and metadata
  on the same device.

- On multi-node env Ceph cluster needs cluster members
  see each other, so let's use pod network CIDR.

Change-Id: I493b6c31d97ff2fc4992c6bb1994d0c73320cd7b
2023-11-24 19:06:08 -06:00
Vladimir Kozhukalov c047fce569 Fix path for setup-client.sh script
Change-Id: Ieb7549d2f00d981efa1d4bc2d6d8a57a067ef6c7
2023-11-06 10:34:20 -06:00
Zuul ff552d5969 Merge "Create osh-bandit role" 2023-10-27 22:12:27 +00:00
Sergiy Markin a430d16bd5 Control ceph loopback devices path
This PS adds control over location of Ceph loopback devices path.

Change-Id: Ib0738c1127ff37633cdd035b3978cc137c5eaf71
2023-10-26 17:28:45 +00:00
Vladimir Kozhukalov 45b209ac79 Create osh-bandit role
The motivation is to reduce the code base and get rid
of unnecessary duplications. This PR is moves bandit
tasks from the osh-infra-bandit.yaml playbook
to the osh-bandit role. Then we can use this role for the
same job in OSH.

Change-Id: I9489a8c414e6679186e6c399243a7c0838df812a
2023-10-25 17:59:56 -05:00
Sergiy Markin f9b0360418 Mount extra 80Gb volume
This PS mounts extra 80Gb volume if available and mounts it to
/opt/ext_vol. It also alters docker and containerd configs to move their
root folder to that extra volume. This helps zuul gates to succeed when
a node with 40Gb volume is assigned to a zuul gate.

Change-Id: I1c91b13c233bac5ebfe6e3cb16d4288df2c2fe80
2023-10-24 18:39:51 +00:00
Stephen Taylor a58f80599b [ceph] Add support for deploying and managing Ceph with Rook
This change adds an openstack-support-rook zuul job to test
deploying Ceph using the upstream Rook helm charts found in the
https://charts.rook.io/release repository. Minor changes to the
storage keyring manager job and the mon discovery service in the
ceph-mon chart are also included to allow the ceph-mon chart to be
used to generate auth keys and deploy the mon discovery service
necessary for OpenStack.

Change-Id: Iee4174dc54b6a7aac6520c448a54adb1325cccab
2023-10-03 07:16:02 -06:00
Vladimir Kozhukalov ae91cf3fc3 Use deploy-env role for all deployment jobs
To make it easier to maintain the jobs all experimental
jobs (those which are not run in check and gate pipelines)
are moved to a separate file. They will be revised later
to use the same deploy-env role.

Also many charts use Openstack images for testing this
PR adds 2023.1 Ubuntu Focal overrides for all these charts.

Change-Id: I4a6fb998c7eb1026b3c05ddd69f62531137b6e51
2023-09-22 15:02:07 -05:00
Samuel Liu 664f4c9dfb Remove PodSecurityPolicy
PodSecurityPolicy was deprecated in Kubernetes v1.21, and removed from Kubernetes in v1.25.[1]

In Kubernetes 1.21, PodSecurityPolicy is deprecated. As with all Kubernetes feature deprecations, PodSecurityPolicy will continue to be available and fully-functional for several more releases. PodSecurityPolicy, previously in the beta stage, is planned for removal in Kubernetes 1.25.[2]

[1] https://kubernetes.io/docs/concepts/security/pod-security-policy/
[2] https://kubernetes.io/blog/2021/04/08/kubernetes-1-21-release-announcement/#podsecuritypolicy-deprecation

Change-Id: Ic060d925b6e97e5651e74a1a1161906aef740a8c
2023-06-08 21:53:29 +03:00
astebenkova 92d16f3a29 [osh-selenium] Upgrade image to latest-ubuntu_focal
+ migrate all Python tests to use Selenium v4 (bionic image had v3 installed):
https://www.selenium.dev/documentation/webdriver/getting_started/upgrade_to_selenium_4/
+ amend selenium role in order to install ChromeDriver compatible with Google Chrome:
https://chromedriver.chromium.org/downloads/version-selection
+ run selenium tests AFTER the charts are deployed

Change-Id: I46200b7dc173bd0e1e6bf3545d9a26c252a21927
2023-05-23 18:09:16 +03:00
Gage Hugo 74652cb181 Bump ubuntu and openstack version in jobs
The current zuul jobs definitions still use an older release and
distro version of ubuntu. This change modifies the versions to run
ubuntu focal and the Xena release of openstack.

Change-Id: I653fd9ed42972c7bba5fa94519cd413c0d15b2c9
2023-02-16 06:02:48 +00:00
Gage Hugo 4d10d7e04d Remove fedora nodesets
osh-infra still have references to fedora 27 which is quite old now,
this change removes those definitions since they are unused.

Change-Id: I4f127113f0014ec2ed11f21e230facd08820af6e
2022-10-22 04:22:50 +00:00
Gage Hugo 6be6d638b3 Update ubuntu nodesets to focal
The openstack-single-node nodeset still is using ubuntu-bionic,
which is nearly 4 years old now. This change updates it to use
the newer ubuntu focal release.

Depends-on: https://review.opendev.org/c/openstack/openstack-helm-infra/+/839996
Depends-on: https://review.opendev.org/c/openstack/openstack-helm-infra/+/840370

Change-Id: Ia43cb31e13bc059541116064aa2092526186b831
2022-05-03 19:57:15 +00:00
Gage Hugo d37fd936bf Remove keystone-auth job
The kubernetes-keystone-auth check job has not been ran in a long
time and has not been maintained. This change removes it from the
list of jobs defined and jobs ran in the osh-infra project.

Change-Id: If0275524fda92d8fd8baa689521e2e841210ce51
2022-03-31 17:00:16 -05:00
Gage Hugo ddb377df6d Test linting osh on helm-toolkit changes
Since most of the charts in both openstack-helm and
this repo use helm-toolkit, changes in helm-toolkit
have the possibility of impacting charts in the
openstack-helm repo and will not be caught in testing
here.

This change adds a conditional linter to lint the
charts in the openstack-helm repo if any changes
to helm-toolkit are made.

Change-Id: I0f6a935eca53d966c01e0902e546ea132a636a9d
2021-11-02 22:46:43 +00:00
Zuul 6fedc626da Merge "Add gate script to deploy kubernetes using kubeadm." 2021-11-01 16:45:23 +00:00
Gage Hugo 57c452154e Remove fedora and centos jobs
The fedora and centos jobs have not been used or maintained for
quite some time. This change removes them and the related notes.

Also removed an outdate note about disabling all the experimental
and periodic jobs.

Change-Id: Ic8eb628e21c49957bdcd10a8d69d850ec921b6d6
2021-11-01 00:48:27 +00:00
Gupta, Sangeet (sg774j) 6c044362d9 Add gate script to deploy kubernetes using kubeadm.
Change-Id: I164d0aa03f420cf263832e31c30807e9fccd8495
2021-10-29 01:05:29 +00:00
Gage Hugo e3203bd7fe Improve osh-infra-deploy helm v3 job
This change improves the osh-infra-deploy job to
successfully deploy minikube with helm v3 along with
the necessary namespaces. Future changes will modify
the install scripts for each job to make them helm
v3 compatible.

Change-Id: I08a94046f86f7c92be7580fbf10751150d2fcecc
2021-10-11 17:02:06 +00:00
Tin Lam 5f75ffa180 fix(ssl): fixes libvirt ssl job
Changes the override to use dynamically generated certs for the
libvirt-ssl jobs so they don't expire in the future. Also, changes it so
it is voting again like before.

Signed-off-by: Tin Lam <t@lam.wtf>
Change-Id: If7215961b0b9a7cad75afd7f78592515b74a7b58
2021-09-27 12:45:29 -05:00
Tin Lam 418143f3e4 fix(gate): disable ssl job
This patch sets temporary disables the ssl gate job and makes the check
job non-voting to unblock osh-infra. The certificate hardcoded in [0]
has expired.

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            5f:61:31:9d:0f:ff:99:81:ba:6d:50:1a
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN = libvirt.org
        Validity
            Not Before: Sep 15 21:26:53 2020 GMT
            Not After : Sep 15 21:26:53 2021 GMT

This will need to be updated or better, unhardcode this at the gate.

[0] https://opendev.org/openstack/openstack-helm-infra/src/branch/master/tools/deployment/openstack-support/051-libvirt-ssl.sh#L27-L51

Signed-off-by: Tin Lam <t@lam.wtf>
Change-Id: I5ea58490c4fe4b65fec7bd3f11b4684cdc1a3e8b
2021-09-24 12:31:17 -05:00
Gage Hugo 9030ff05da Remove unused jobs and related files
This change removes a bunch of unused and unmaintained files
and job declarations related to deploying osh-infra with armada.

Change-Id: I158a255132cd6b02607b6e1e77b8b9525cc8a3d5
2021-09-06 22:23:33 -05:00
Gage Hugo 21ada44f59 Add base helm3 job
This change adds a new script and job to deploy minikube with
helm3. This job will be improved upon in later changes as
part of the movement to helm3.

Change-Id: Ia7ef30a4e2af77508ad95191e5241d2c1b83a7c4
2021-09-02 04:54:52 +00:00