Commit Graph

365 Commits

Author SHA1 Message Date
Zuul 9ebd51f799 Merge "Revert "CentOS/Rocky: use CentOS Cloud SIG repo instead of Delorean (Bobcat)"" 2024-01-03 15:58:27 +00:00
Michal Nasiadka 559955d5a6 Revert "CentOS/Rocky: use CentOS Cloud SIG repo instead of Delorean (Bobcat)"
This reverts commit 2ed66effa6.

Reason for revert: Let's use RDO trunk

Change-Id: I3945fe5fd0aef2e12a7f6c60d714e69ea496083b
2023-12-20 17:05:25 +00:00
Michal Nasiadka 316bafaf07 CI: Use newer podman/buildah on Ubuntu Jammy
Change-Id: Ifce506b49571950ee16e26aae06f20aa94567722
2023-12-14 08:08:03 +00:00
Michal Nasiadka 8f863beb45 CI: Pin docker installation to <7
Docker 7.0.0 introduced a pre build check for tag regex, that is most
probably the culprit - see [1] and [2].

Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/903364

[1]: a9b5494fd0
[2]: https://github.com/docker/docker-py/issues/3195

Change-Id: I147c9715eb922abb6675a2786beaec85971718a8
2023-12-12 12:35:25 +00:00
Michal Nasiadka 0cfc5c7c09 podman: install "rich" dependency
This dependency was added to podman-py in version 4.8.0, but not added
properly to their requirements. Install it explicitly for our tox
and integration testing as a workaround.

[0] https://github.com/containers/podman-py/issues/350

Change-Id: Ic0d9b2c26fe5474f579b72d6779243b7ad39b420
2023-11-29 19:07:31 +01:00
Michal Nasiadka 5a259fdafe Drop docker-py from requirements.txt
Since addition of podman - users need to install only one from docker|podman

Change-Id: I622eefad3b2e93f334028d9c0775a2cca558a39c
2023-11-28 11:08:01 +00:00
Zuul d86cba116e Merge "CentOS/Rocky: use CentOS Cloud SIG repo instead of Delorean (Bobcat)" 2023-11-18 11:40:10 +00:00
Michal Nasiadka 2ed66effa6 CentOS/Rocky: use CentOS Cloud SIG repo instead of Delorean (Bobcat)
Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/900536

Change-Id: I1bc19f8198da3f9ab2ae2a8864c3349b21b0249e
2023-11-09 18:11:16 +01:00
Michal Nasiadka 213e53e18a CI: Move ephemeral disk config to a role
Change-Id: Ieee71aa22f4ec45fe03f50fd697c85dda952b4d0
2023-11-08 09:57:32 +00:00
Zuul 80a16bfc6e Merge "CI: speed up podman on el9" 2023-10-25 17:52:19 +00:00
Zuul e770021469 Merge "Revert "CentOS/Rocky: use CentOS Cloud SIG repo instead of Delorean"" 2023-10-24 16:16:23 +00:00
Jan Gutter c4b56a4b84
CI: speed up podman on el9
Before this commit the following warning message appeared
in the podman log:
  Not using native diff for overlay, this may cause degraded
  performance for building images: kernel has
  CONFIG_OVERLAY_FS_REDIRECT_DIR enabled

It appears that the default mount options for the overlay
graph driver is badly optimised for Kolla builds on el9:
* Enable native overlay diff
  https://mikeshade.com/posts/docker-native-overlay-diff/
  09480532cf
  https://github.com/containers/podman/blob/main/docs/tutorials/performance.md

Things that did not appear to make a difference:
* volume mounting /var/lib/container
  https://github.com/containers/buildah/issues/3666#issuecomment-1351992335
* enable transientStore
  https://www.redhat.com/sysadmin/speed-containers-podman-raspberry-pi
  note: currently incompatible with debian

Change-Id: I002e88b4b6162f6732e622fd97639d543c66784c
2023-10-24 09:37:43 +01:00
Jan Gutter bd2b69e6f8 CI: fix podman info log collection
* capture podman config
* podman info needs sudo permissions
* before this fix, the podman info command failed with:
  Error: cannot set up namespace using "/usr/bin/newuidmap":
  should have setuid or have filecaps setuid: exit status 1

Change-Id: Ib3fe039a8c35421b56eedbd2d5197217b4a15a9a
2023-10-24 00:26:24 +01:00
Michal Nasiadka 43d50b7c8b Revert "CentOS/Rocky: use CentOS Cloud SIG repo instead of Delorean"
This reverts commit 881e3925c7.

Reason for revert: switch back to master, to be followed by switch to bobcat when available

Change-Id: Ib8fdad41a669fc8404b8de2de5361db9547c4ea2
2023-10-05 09:57:38 +00:00
Michal Arbet 899545f4e9 [CI] Add podman build
Change-Id: I41b89ec87a4849668102ca9c9cc79f0ef9944b85
2023-09-21 11:01:21 +02:00
Zuul 9a97600b0c Merge "Add comprehensive checks for container restarts" 2023-08-29 16:12:56 +00:00
Zuul 58030f8677 Merge "ubuntu: use Antelope UCA" 2023-05-18 20:37:31 +00:00
Michal Nasiadka 665357248d ubuntu: use Antelope UCA
Skip using the mirror for now, given the OpenDev AFS
space issues.

Change-Id: I8e55c699e493494b3e5c908c7e8c370ce992fdf5
2023-05-18 14:54:56 +00:00
Michal Nasiadka 881e3925c7 CentOS/Rocky: use CentOS Cloud SIG repo instead of Delorean
Antelope was released, landed in CentOS repo, time to use it.

Change-Id: I9fa8fd5dd9b75251363f1957d1e249e7cb4beb84
2023-05-17 16:44:33 +00:00
Michal Nasiadka 1cf4e75aaa CI: Fix visibility publish task
Change-Id: I41bab698e17254d8c92e621ba9b88b6fe20c2300
2023-05-08 13:02:17 +02:00
Michal Nasiadka 58b265f46d CI: Ensure quay repo is public on publish
Change-Id: Ib49e7c51be552a9aad0526841e7cffe588189371
2023-03-27 12:19:08 +00:00
Dawud M d9a6c5f390 Add comprehensive checks for container restarts
When adding a dashboard to grafana the containers aren't restarted when
they should be. This is due to a bug in Kolla where the logic to
determine whether or the container needs to be restarted fails in the
case where the file does not exist in the container. This patch adds
more comprehensive checks for container restarts in the set_configs.py
file. This patch also adds a test to ensure that the functions work as
expected.

Closes-Bug: #1997984
Co-Authored-By: Will Szumski <will@stackhpc.com>
Change-Id: I67f5f12700d7b55f26bff81e9b54559303da6d83
2023-03-09 18:03:58 +00:00
Zuul 7bbf36cca0 Merge "CI: Additional disk followup" 2023-02-21 17:39:17 +00:00
Zuul c91513d3eb Merge "CI: update docker hub publish creds" 2023-02-21 17:26:37 +00:00
Zuul fd952d8832 Merge "CI: Retry docker push on publish jobs" 2023-02-21 17:26:34 +00:00
Michal Nasiadka 5fe335d143 CI: Additional disk followup
state present only defines mount in /etc/fstab, does
not do the actual mount
Need to symlink /var/lib/docker to separate disk,
work-dir only templates out dockerfiles and downloaded
content

Change-Id: I06e262cd48d33ccfc0634589c82dcac80dff51af
2023-02-21 13:51:26 +00:00
Michal Nasiadka a0d758019a CI: Configure additional disk when available
It seems that on some nodepool providers (rax for sure),
we get 40G boot disk and 80G additional disk, while
on others we get 80G boot disk only.

Change-Id: I179c41032239f5f0ce6055c79295b32c33fe67a5
2023-02-20 13:24:06 +00:00
Michal Nasiadka e43317bc4e CI: update docker hub publish creds
Change-Id: Ieceadb1f56908ce752f48b0884ed5cf764bc1ecd
2023-02-17 17:28:41 +00:00
Michal Nasiadka 0887d2e789 CI: Retry docker push on publish jobs
We have a high number of POST_FAILURE jobs failing on at least
one container push - let's fix that.

Change-Id: I0015dd4d3ffe5b3e1202ebcfde2b2c72c4655b41
2023-02-16 08:24:16 +01:00
Michal Nasiadka c45a742e6f CI: store only templated out Dockerfiles in work_dir
Change-Id: I0832d7997536f5f0dc195d5038545b6cc220f0fa
2023-02-07 14:23:32 +00:00
Michal Nasiadka 16f27c5496 CI: Install Apparmor
Docker 23.* is out and it started failing with apparmor_parser
related messages - it seems apparmor is missing in the OpenDev image.

Change-Id: I2a6d98e3c1d2d1b7c97a8e6172decbf56e77042d
2023-02-07 11:18:10 +00:00
Michal Nasiadka c8596ddda4 CI: Add work dir
It's useful to have a templated out version of the Dockerfiles
in order to diagnose errors like "error on line xxx" after
tests/templates/template-override.j2 gets applied.

Change-Id: I5443454d8b020aa1c5552e11f6039e582a002431
2023-01-31 01:04:37 +00:00
Marcin Juszkiewicz c9e60c0b21 CI: CentOS/Rocky: revert Delorean repo url at the end
We revert all repos to their normal versions. This change takes care of
forgotten Delorean repos.

Change-Id: I8bcddbd6ebf3119f4f56d86e47377f27b8671b1a
2023-01-10 13:34:09 +00:00
Marcin Juszkiewicz 0c523e7f1b CI/centos/rocky: use repos from opendev mirror
Repo definitions in RHEL 9 rebuilds use metalink method for most of
entries which makes our use of sed obsolete.

This patch adds file with definitions of all CS9 repos we use and
moves all official definitions a side at start of build. After build our
definitions get removed and official ones are restored.

Similar is done for RockyLinux 9 builds but there RL9 repos are used
from official mirrors while CS9/EPEL ones are from OpenDev mirrors.

Final images will have repos in wrong enabled/disabled state.

Change-Id: I04121fd5ca9894b6deeda3808808cbffd6b97811
2022-10-03 15:29:05 +02:00
Michal Nasiadka 0883532434 Add RockyLinux 9 support
Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/859570

Change-Id: I7af51d07b8a939c908aa072294061def57dd67de
2022-09-30 14:17:34 +00:00
Marcin Juszkiewicz 878b00013b Move to CentOS Stream 9
OpenStack 'zed' requires Python 3.8+ so RHEL 8 family has to go.

This changeset moves to CentOS Stream 9 while move to RockyLinux 9 is
planned as final solution.

CI moved to CentOS Stream 9 nodes.

Depends-on: https://review.opendev.org/c/openstack/kolla-ansible/+/839715

Change-Id: I113b9984294cf8663d3fc0c8840320e1d40ea731
2022-09-27 07:40:06 +00:00
Radosław Piliszek 5b1da01798 Fix CVE-2022-38060
Closes-Bug: #1985784
Change-Id: I66476a2b396e2cbe41e68ac51f57aae1806b2ed8
2022-08-29 10:05:05 +00:00
Zuul ee02fc807a Merge "[CI] Drop bindep" 2022-07-26 12:23:10 +00:00
Pierre Riteau 2da76f78de Fix container image publish to Docker Hub
Publishing container images to Docker Hub was failing with:

    state is present but all of the following are missing: source

This is caused by the recent Ansible upgrade by Zuul [1]: the source
parameter is mandatory since Ansible 2.12.

[1] https://review.opendev.org/c/openstack/project-config/+/849120

Change-Id: I232fe835f5a650c258243f1ee4720480f2e6e7b1
2022-07-25 13:10:15 +02:00
Radosław Piliszek 8477e86db8 [CI] Drop bindep
It is not necessary and it may be confusing.

Change-Id: Ie84bce6a90de622dc76a2309fc579d881325b956
2022-06-23 10:24:14 +02:00
Radosław Piliszek 9d5bdbf36d Use the new image naming scheme by default
Adds a relevant reno to inform users about the change and the
possibility to customise the prefix.

Additionally, this patch includes the fix to Ubuntu image builds
regarding rabbitmq and erlang.

Both the changes are required to fix master CI.

Change-Id: I6bc8b489a810849744c8bead6b9e350c5d3e36b7
Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/843752
2022-05-29 18:14:01 +00:00
Radosław Piliszek 3889952454 [CI] Fix tags in publishing
The original patch did not insert dashes as promised in the commit
message. [1]

[1] https://review.opendev.org/c/openstack/kolla/+/840164

Change-Id: Ibd6e06502ce56f5e0b44dea604ffee2fd9b7b8be
2022-05-21 17:41:37 +00:00
Radosław Piliszek 1e40f08e6c [publishing] use a new naming scheme
This change switches published images names from the current
format <base_distro>-<image_name>:<openstack_release>-<arch_suffix>
to a preciser and image-oriented (for better variant listing) format
<image_name>:<openstack_release>-<base_distro>-<base_distro_version>-<arch_suffix>

Change-Id: I6a4b5c2087b94f96f72aa9d0d0781ff14d3ee6bb
2022-05-02 11:00:45 +02:00
Marcin Juszkiewicz 08b80e34da switch to short image names by default
We can drop temporary 'use-short-names' argument and make short
(kolla/debian-nova-libvirt:master) image names default.

Kolla Ansible is using short names now.

Change-Id: I153355353cd12aa25adaa9c87438a478748dfb28
2022-04-21 17:01:08 +02:00
Marcin Juszkiewicz e877556aaf add a way to not have 'install_type' in image name (temporary)
--use-short-names argument drops 'install_type' from image names:

kolla/debian-openstack-base:14.1.0

This should allow to get rid of '-source' in image names without
disrupting CI jobs.

Switch publish jobs to use short names so k-a tests will have ready to
use images for both long and short names.

Change-Id: I29c9d51f4b518a4e2d3157ab5d6cc434fb83f2ef
2022-04-19 16:23:12 +02:00
Marcin Juszkiewicz 0cf5b1d4e6 enable logging to file for quiet mode
We have 'quiet' mode where only minimal data is printed to the console.
But there were no logs at all then.

This change generates log files (if 'logs-dir' argument is used) during
quiet build.

Also enables 'quiet' mode for CI so Zuul will not have to parse 29MB
JSON file each time.

Change-Id: If7d5c2807f0947a8bbbc1ceb8531c9b9c9287c1f
2022-04-15 13:37:25 +00:00
Radosław Piliszek f33ef03977 Revert "CI: add templated Dockerfiles to build logs"
This reverts commit 09c890b124.

Reason for revert: That change caused all Kolla's work items to be
transferred. This includes tarballs, their extracted contents, cloned
git repositories, etc. to also be transferred to swift which takes
extra time, pollutes logs and decreases the stability of jobs slightly
(the mechanism is not designed for this kind of load).
The collection of templated Dockerfiles in CI has to be reimplemented
if it is wanted.

Change-Id: Ideaf66ce18c993802f4d20f490caa47cec0dde5e
2022-04-15 13:37:18 +00:00
Marcin Juszkiewicz 41fa71d2c2 CI: drop install_type from build jobs
Change-Id: If524e45dc262efb253fc06329d4567412282926d
2022-04-11 19:40:01 +02:00
Maksim Malchuk 83efe32546 Unify curl use (customisation options)
This change adds and unifies an ability to override hardcoded URLs for
air-gapped environments via the usual blocks mechanism. Also, this
change replaces ENV with ARG instruction for the variables used only
in building images, and uses bash variable expansion where it possible.

This change is a continuation of the I46b77978926fc2b578a68d1aaa944b2198af0685

Change-Id: I1fbad333b1bc95484e6f4c5145d5936a0e2db84f
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>
2022-03-11 15:58:05 +03:00
Radosław Piliszek b4b206de42 [CI] Drop support for publishing versioned image tags
Follow-up on I3259c0736a296751129aa349e8d378c0123957e6

Change-Id: I6b8935b16390a301602218ce35edfe132b140a4c
2021-09-08 14:54:13 +00:00