Commit Graph

144 Commits

Author SHA1 Message Date
Vladimir Kozhukalov d31027cfb4 Install OSH Helm plugin
Depends-On: I71ab6ad104beb491b5b15b7750e2fc0988db82bf
Change-Id: I8f30fbdf94d76ef9fa2985a25c033df290995326
2024-04-17 19:06:48 -05:00
astebenkova efea7f5fd0 [chromedriver] Change json api endpoint
Choose a more reliable json file from the upstream to refer to.
"Stable" versions of Chrome and Chromedriver became unsynchronized for some reason.

Change-Id: I1688a867ea1987105e7a79c89ba7ea797819a12f
2024-04-17 18:17:56 +03:00
Vladimir Kozhukalov 4d203b2274 Update deploy-env role README.md
Change-Id: Ia2ace3541be97577f1225d54417f6a287b7a8eb2
2024-04-03 21:47:01 -05:00
Vladimir Kozhukalov cfed816a9a Fix coredns resolver
Forward requests for unknown names to 8.8.8.8

NOTE: Temporarily disable DPDK job which turned to
be incompatible with this PR
https://review.opendev.org/c/openstack/openstack-helm/+/914399
It wasn't tested with the DPDK job.

Change-Id: I936fb1032a736f7b09ad50b749d37095cce4c392
2024-03-28 20:34:23 -05:00
Vladimir Kozhukalov 5b1879aa09 Refactor deploy-env role
- Make it less mixed. Each task file
  deploys one feature.
- Deploy Metallb
- Deploy Openstack provider network gateway

Change-Id: I41f0353b286f817cb562b3bd59992e4baa473568
2024-03-25 14:45:00 -05:00
Vladimir Kozhukalov d33e9bd50c Bump containerd sandbox image from 3.6 to 3.9
Fixes the following kubeadm warning:

W0321 01:33:46.409134   14953 checks.go:835] detected that the
sandbox image "registry.k8s.io/pause:3.6" of the container
runtime is inconsistent with that used by kubeadm.
It is recommended that using "registry.k8s.io/pause:3.9"
as the CRI sandbox image.

Change-Id: I8129a6e9ad3acdf314e2853851cd5274855e3209
2024-03-21 19:37:28 +00:00
Stephen Taylor 2fd438b4b1 Update Ceph images to patched 18.2.2 and restore debian-reef repo
This change updates the Ceph images to 18.2.2 images patched with a
fix for https://tracker.ceph.com/issues/63684. It also reverts the
package repository in the deployment scripts to use the debian-reef
directory on download.ceph.com instead of debian-18.2.1. The issue
with the repo that prompted the previous change to debian-18.2.1
has been resolved and the more generic debian-reef directory may
now be used again.

Change-Id: I85be0cfa73f752019fc3689887dbfd36cec3f6b2
2024-03-12 13:45:42 -06:00
Sergiy Markin 3a2399c99f Workaround for debian-reef folder issue
This PS changes ceph repo to debian-18.2.1 from
debian-reef due to some issues with debian-reef
folder at https://download.ceph.com/

Change-Id: I31c501541b54d9253c334b56df975bddb13bbaeb
2024-03-04 19:35:38 +00:00
astebenkova 2216cbfec4 Add compute-kit job with DPDK enabled
+ add role for enabling hugepages

Change-Id: I89d3c09ea3bedcba6cb51178c8d1ac482a57af01
Depends-On: I2f9d954258451f64eb87d03affc079b71b00f7bd
2024-02-14 13:23:34 -06:00
Sergiy Markin 1c83e3a9ae [deploy-env] Docker env setup
This PS adds connection reset for ansible session
letting zuul user to use newly installed docker
environment without sudo

Change-Id: I37a2570f1dd58ec02338e07c32ec15eacbfaf4b6
2024-02-08 17:57:48 +00:00
Vladimir Kozhukalov cfff60ec10 Bump Calico version to v3.27.0
Change-Id: I8daa54e70c66cec41733d6b9fd5c9dd4597ff9c1
2024-02-02 13:54:22 -06:00
Vladimir Kozhukalov 2b7563f5de Add zuul user to docker group
We are going to use containerized Openstack client
in test scripts. Adding zuul to the docker group
allows running docker command directly not using sudo.

Change-Id: Iee77e7f2b8801743f95535d31d0b909dcea50bf3
2024-01-16 13:38:19 -06:00
Vladimir Kozhukalov e45cbaf088 Add license headers to deploy-env tasks files
Change-Id: Ic0ed6d2cdc02e5f55019f9f38a3811af6b39a5ea
2023-12-13 14:19:50 -06:00
Vladimir Kozhukalov 9e256fd8b0 Update k8s packages repo
For details see the announcement
https://kubernetes.io/blog/2023/08/31/legacy-package-repository-deprecation/

Also bump K8s version up to 1.28.4

Change-Id: Ic6b3478e53504622804b6f003ca176a679573d5b
2023-12-12 17:58:20 -06:00
Vladimir Kozhukalov 10a171eb18 Increase the number of inotify instances
For TLS test jobs on Ubuntu Jammy when we run
dnsmasq on the master node needed for testing
we get the error:

"failed to create inotify: Too many open files"

By default the number of inotify instances on Jammy
is 128. We increase this up to 256.

Change-Id: I07c8a0f909608b6e44040ffeefc6ab576236c93f
2023-12-11 13:07:09 -06:00
Stephen Taylor 426c71f753 Initialize registry_namespaces unconditionally if not initialized
The deploy-env playbook can fail with an error stating that
registry_namespaces is not defined in some cases. This change moves
the initialization of registry_namespaces so that buildset_registry
is not required for it to be set when other conditions are not met.

Change-Id: I160e7d479008fd3afd460382691673b92bd042c9
2023-12-06 12:15:57 -07:00
Vladimir Kozhukalov 7f783dba51 Update elasticsearch chart to work with Rook Ceph
When using Rook for managing Ceph we can use
Rook CRDs to create S3 buckets and users.

This PR adds bucket claim template to the
elasticsearch chart. Rook creates a bucket for
a bucket claim and also creates a secret
containing the credentials to get access to this
bucket. So we also add a snippet to expose
these credentials via environment variables to
containers where they are needed.

Change-Id: Ic5cd35a5c64a914af97d2b3cfec21dbe399c0f14
2023-11-26 19:34:42 -06:00
Vladimir Kozhukalov 510cea0c23 Deploy Ceph on multi-node envs
- In case we deploy Ceph on a multi-node env we have
  to prepare the loop devices on all nodes. For this
  we moved loop devices setup to the deploy-env
  Ansible role.

  For simplicity we need the same device on all nodes,
  so we create a loop device with a big
  minor number (/dev/loop100 by default) hoping
  that only low minor numbers could be busy.

- For test jobs we don't need to use different devices
  for OSD data and metadata. There is no
  any benefit from this for the test environment.
  So let's keep it simple and put both OSD data and metadata
  on the same device.

- On multi-node env Ceph cluster needs cluster members
  see each other, so let's use pod network CIDR.

Change-Id: I493b6c31d97ff2fc4992c6bb1994d0c73320cd7b
2023-11-24 19:06:08 -06:00
Vladimir Kozhukalov b5b66f1489 Fix deploy-env when buildset_registry is defined
It used to configure /etc/hosts in two different places.
The buildset registry record was added while configuing
Containerd and then this record was removed while
configuring Kubernetes.

The PR adds the buildset registry record to the /etc/hosts
template and the task is moved to the tasks/main.yaml.

Change-Id: I7d1ae6c7d33a33d8ca80b63ef9d69decb283e0a6
2023-10-31 17:34:12 -05:00
Zuul ff552d5969 Merge "Create osh-bandit role" 2023-10-27 22:12:27 +00:00
Zuul 1fcb3f1b23 Merge "Fix deploy-env role" 2023-10-27 21:48:05 +00:00
Vladimir Kozhukalov f9f487ce4a Fix deploy-env role
The role tried to include non-existing file
which was forgotten while we moved the role to this repo.
This inclusion is only actual for cases when we
consume images from a buildset registry.

Change-Id: I1510edf7bdc78f9c61f7722e2c7848e152edf892
2023-10-27 13:59:19 -05:00
Vladimir Kozhukalov 0cd77664e0 Fix private key paths in the deploy-env role README.md
Change-Id: Ie240f79fbd4162961f28e5c9094bb5b70bae8a3f
2023-10-26 22:20:17 +00:00
Sergiy Markin a430d16bd5 Control ceph loopback devices path
This PS adds control over location of Ceph loopback devices path.

Change-Id: Ib0738c1127ff37633cdd035b3978cc137c5eaf71
2023-10-26 17:28:45 +00:00
Vladimir Kozhukalov 45b209ac79 Create osh-bandit role
The motivation is to reduce the code base and get rid
of unnecessary duplications. This PR is moves bandit
tasks from the osh-infra-bandit.yaml playbook
to the osh-bandit role. Then we can use this role for the
same job in OSH.

Change-Id: I9489a8c414e6679186e6c399243a7c0838df812a
2023-10-25 17:59:56 -05:00
Sergiy Markin f9b0360418 Mount extra 80Gb volume
This PS mounts extra 80Gb volume if available and mounts it to
/opt/ext_vol. It also alters docker and containerd configs to move their
root folder to that extra volume. This helps zuul gates to succeed when
a node with 40Gb volume is assigned to a zuul gate.

Change-Id: I1c91b13c233bac5ebfe6e3cb16d4288df2c2fe80
2023-10-24 18:39:51 +00:00
Zuul adcab364d2 Merge "Deprecating the Ingress Class Annotation" 2023-09-16 04:32:02 +00:00
Vladimir Kozhukalov 63cbad8f42 Fix deploy-env role to work on Ubuntu Jammy
Change-Id: Idfddfa48a7078ca63efa085191a5a07cdb8c2e61
2023-09-13 21:59:42 +03:00
Vladimir Kozhukalov bda43dfff8 Add deploy-env role
This role works both for singlenode and multinode
inventories. The role installs all necessary prerequisites
and deploys K8s with Containerd as a container runtime.
The idea is to use this role to deploy
all test singlenode/multinode environments for all test jobs.

This PR wraps into a role playbooks that
we are currently using for multinode compute-kit tests.

Change-Id: I41bbe80d806e614a155e6775c4505a4d81a086e8
2023-09-06 13:19:44 +03:00
Anselme, Schubert 51c70e48df
Deprecating the Ingress Class Annotation
This PS replaces deprecated kubernetes.io/ingress.class annotation  with
spec.ingressClassName field that is a reference to an IngressClass
resource that contains additional Ingress configuration, including the
name of the Ingress controller.

https://kubernetes.io/blog/2020/04/02/improvements-to-the-ingress-api-in-kubernetes-1.18/#deprecating-the-ingress-class-annotation

Change-Id: I9953d966b4f9f7b1692b39f36f434f5055317025
Co-authored-by: Sergiy Markin <smarkin@mirantis.com>
Co-authored-by: Leointii Istomin <listomin@mirantis.com>
Signed-off-by: Anselme, Schubert (sa246v) <sa246v@att.com>
2023-08-26 00:39:34 +00:00
astebenkova c84fd9a993 [chromedriver] Amend upstream download link
The upstream changed the structure of the JSON API endpoints
(now they contain both chrome and chromedriver binaries)
https://github.com/GoogleChromeLabs/chrome-for-testing#json-api-endpoints

Change-Id: Idaa38f3c2522f4709c396e99090b7fa6d9790c8d
2023-08-10 14:37:07 +03:00
astebenkova 027bcefbd4 [chromedriver] Fix package installation
This commit alllows us to switch to JSON endpoints for chromedriver
since upstream changed the way of installation for version 115 or newer:
https://chromedriver.chromium.org/downloads#h.p_ID_32
https://github.com/GoogleChromeLabs/chrome-for-testing#json-api-endpoints

Change-Id: I4a432ec36fe9e3f794cc6b7788bbdc04db3c8cf6
2023-07-20 13:35:46 +03:00
Stephen Taylor bf95238dc1 [helm] Upgrade Helm to version 3.12.0
Helm version 3 is needed for upcoming features, notably the
rook-helm charts to manage Ceph via Rook require Helm v3. This
change updates Helm in openstack-helm-infra to 3.12.0.

Change-Id: I13b1671121658b1390d89beabacfd15a24b19afe
2023-06-01 08:22:31 -06:00
astebenkova 92d16f3a29 [osh-selenium] Upgrade image to latest-ubuntu_focal
+ migrate all Python tests to use Selenium v4 (bionic image had v3 installed):
https://www.selenium.dev/documentation/webdriver/getting_started/upgrade_to_selenium_4/
+ amend selenium role in order to install ChromeDriver compatible with Google Chrome:
https://chromedriver.chromium.org/downloads/version-selection
+ run selenium tests AFTER the charts are deployed

Change-Id: I46200b7dc173bd0e1e6bf3545d9a26c252a21927
2023-05-23 18:09:16 +03:00
Wahlstedt, Walter (ww229g) d132d708e9 Docker log collection
Collect logs for zuul jobs that use docker

Change-Id: I8da05be5e84c0d565655721c4545ca7446794858
2023-02-22 12:38:07 -05:00
Gage Hugo ad133069aa Remove kubeadm-aio
As part of the move to helm v3, we migrated how we setup
deploying kubernetes from a series of playbooks to using
a bash script. This had the advantage of being easier to
follow and deploy locally, as well as easier to debug
when an issue arose. While the kubeadm-aio was very useful
in the past to help lower run-times, currently it appears
that optimizations in our jobs have seem to mitigate a lot
of the issues that were lessened with the AIO.

Change-Id: If0c6a97aed4083307a9b6f5beb4ec525e8103e01
2022-05-19 16:13:12 +00:00
Dustin Specker 367bed952f fix helm-release-status role to use helm3 commands
Helm3 requires specifying a namespace to retrieve status and values for
a given release.

Before this role would not retrieve any values or statuses. Now values
and statuses are retrieved in saved in Zuul archive correctly.

Change-Id: If8cf28349b0c14ba6e1de74d49e46ab4a61b93c7
2022-05-10 11:17:53 -05:00
Gage Hugo d200c97464 Update several default ubuntu release defaults
Several roles and scripts in the osh-infra repo have defaults for
ubuntu that are still on xenial when we mostly run bionic or newer.

This change updates these references to default to focal.

Change-Id: I84bca2d685cf1c67b10eee3bb7c05deb2cc4670b
2022-04-27 11:29:27 -05:00
Gage Hugo 83b8dc8616 Remove tiller chart
Now that openstack-helm has migrated all of its jobs over, we
can now actually delete the tiller chart.

Change-Id: I4a6b201906fb6840222c1b082febedff2e51ce97
2022-03-31 16:52:30 -05:00
Phil Sphicas 05f2a42330 Use Kubernetes v1.19.15 in kubeadm-aio image
Update Kubernetes version to v1.19.15, the latest patch release of the
earliest supported version (as of 2021-09-15).

Change-Id: Ia8f398098dfafa7fc029c982c71bce4a876668de
2021-10-07 22:14:24 -07:00
zhaoleilc e81a86d574 Fix an attribute error
The corresponding attribute in roles/build-images/defaults/main.yml
is helm_repo instead of google_helm_repo.

Change-Id: Id1be29773224ea496a3550642d7ba194fd1e83c2
2021-08-23 22:42:47 +08:00
Gage Hugo 67ac5da9ed Update helm repo url
The googleapi repo has been causing issues and the latest
one is giving an unauthorized error when trying to download
helm tarball.

This change moves the repo to use the official helm one.

Change-Id: I52607b0ca6d650d5f5e4a95045389970faa08cfb
2021-08-11 16:18:44 +00:00
Gage Hugo a4f300e3da Update helm 2 version to latest
The version of helm 2 that OSH has been using was older and seems
to have been removed from the googleapi repo that the jobs are
setup to use, this was causing job failures.

This change updates the version to the latest v2 release.

Change-Id: I675f539b24ea9c2355ac9eacc7dd8122c5236e5f
2021-08-06 10:56:50 -05:00
Zuul 3ded481794 Merge "Fix openvswitch gate issue for multinode" 2020-12-29 02:18:56 +00:00
jh629g 63f0bc364e Update hardcoded Google Resource URLs
Kubernetes charts from google are
deprecated resources. Updated to helm
repositories for kubernetes charts per [0]

[0] https://helm.sh/blog/new-location-stable-incubator-charts/

Change-Id: I31f29d8576b3d7e8a5ac1d14faa26f0fd6ba77a1
2020-12-23 15:16:09 +00:00
Gupta, Sangeet (sg774j) 1f3fe0cb45 Fix openvswitch gate issue for multinode
Add openvswitch gate issue with systemd 237-3ubuntu10.43 to
multinode also. Added code from [0].
Additionally, made changes to support 1.18.9 version of kubeadm.

[0] https://review.opendev.org/c/openstack/openstack-helm-infra/+/763619

Change-Id: I2681feb1029e5535f3f278513e8aece821c715f1
2020-12-11 17:10:55 +00:00
Andrii Ostapenko 7be813374f
Collect dpkg -l for host
Change-Id: I8886e2bacb74f95ac117aad07c831c5c3803d5c0
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-12-03 15:21:29 -06:00
Gupta, Sangeet (sg774j) c988632091 Changing the kube version to 1.18.9
Change-Id: I216d16de1f4fb1438534c9362b57499ec3d6725b
2020-11-09 23:15:33 +00:00
Andrii Ostapenko 7a0558bd78 Override images provided as artifacts from buildset registry
Switch from using images from defined in docker_images to provided as
zuul artifacts. Currently to be used in conjunction with [0] in
openstack-helm-images pipelines.

[0] https://review.opendev.org/741551

Change-Id: I43dbd38906e8854c87a361f2e5e479f57850252f
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-09-28 17:38:29 +00:00
KAVVA, JAGAN MOHAN REDDY (jk330k) dc58ef9ddd Move Tiller version to 2.16.9
Update Helm chart for Armada to use Tiller version 2.16.9.

Change-Id: I6556a6e2a1fad3946a92da5e8dd97e0da5803c62
2020-09-02 08:05:00 -05:00