Commit Graph

46 Commits

Author SHA1 Message Date
Michal Arbet 47ddac4131 Bump ansible-lint version
The version that we were capping to is no longer compatible with latest
upper-constraints.txt, so let us free float again.

The resulting linting errors are included for now to unblock the gate,
these will still need to be discussed or fixed later.

NOTE(kevko): Temporarily disabling horizon deployment, as it's not
possible to unblock gates without it

Co-Authored-By: Michal Arbet <michal.arbet@ultimum.io>
Change-Id: Ib7f72b2663199ef80844a412bc436c6ef09322cc
2024-01-22 22:49:46 +01:00
Michal Nasiadka 9bc99b9434 Test haproxy single external frontend
Change-Id: Id25b4407a8170f69e4cd7278e0aff64c609ace7d
2024-01-03 08:31:14 +00:00
Michal Nasiadka 58f7ad3112 CI: Configure ephemeral disk using Kolla role
Depends-On: https://review.opendev.org/c/openstack/kolla/+/899465
Change-Id: I876536263865077b7ecb2288ad95061292babcce
2023-10-30 14:56:26 +00:00
Martin Hiner 53e8b80ed3 Add container engine option to scripts
This patch add a way to choose container engine inside tool and test
scripts. This is in preparation for Podman introduction but still
leaves Docker as default container engine.

Signed-off-by: Martin Hiner <m.hiner@partner.samsung.com>
Change-Id: I395d2bdb0dfb4b325b6ad197c8893c8a0f768324
2023-04-28 16:16:55 +02:00
Michal Nasiadka 654577646f CI: Drop apparmor installation
Depends-On: https://review.opendev.org/c/openstack/ansible-collection-kolla/+/872590
Change-Id: Ia8283f28147fbbd2d24028d01e021f95598b86fb
2023-02-07 17:12:07 +00:00
Michal Nasiadka ef49c7440f CI: Install lvm on setup_disk scenarios
Change-Id: I99145322f65468e9926b1412844ad4ccaa6829d7
2023-02-06 16:06:02 +00:00
Michal Nasiadka f253f99c12 Do not support dimensions:kernel_memory on Docker API 1.42
It is deprecated in 20.10 and removed in 23.0 (and 23.0 is out) [1], [2].

[1]: https://docs.docker.com/engine/deprecated/#kernel-memory-limit
[2]: https://docs.docker.com/engine/api/version-history/#v142-api-changes

Change-Id: Ia6fa85172aad7bcd5f958922d3c224ef79882e6c
2023-02-03 11:32:32 +00:00
Radosław Piliszek d096103e2b [CI] Do not install wheel
We do not need to build wheels here.

Change-Id: I1e3641f83a16cdc367c2643d643076a25034ffe4
2022-09-13 09:36:16 +00:00
Marcin Juszkiewicz 67607c679e Add CentOS Stream 9 / Rocky Linux 9 host support
Added c9s jobs are non voting, as agreed on PTG to focus on Rocky Linux 9.
Since both CS9 and RL9 have higher default fd limit (1073741816 vs
1048576 in CS8) - lowering that for:
* RMQ - because Erlang allocates memory based on this (see [1], [2], [3]).
* MariaDB - because Galera cluster bootstrap failed

Changed openvswitch_db healthcheck, because for unknown reason
the usual check (using lsof on /run/openvswitch/db.sock) is hanging
on "Bad file descriptor" (even with privileged: true).

[1]: https://github.com/docker-library/rabbitmq/issues/545
[2]: https://github.com/rabbitmq/cluster-operator/issues/959#issuecomment-1043280324
[3]: a8b627aaed

Depends-On: https://review.opendev.org/c/openstack/tenks/+/856296
Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/856328
Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/856443
Needed-By: https://review.opendev.org/c/openstack/kolla/+/836664
Co-Authored-By: Michał Nasiadka <mnasiadka@gmail.com>
Change-Id: I3f7b480519aea38c3927bee7fb2c23eea178554d
2022-09-08 16:33:25 +00:00
Zuul b0dc2423bc Merge "Upgrade openEuler Distro" 2022-09-01 08:13:16 +00:00
Radosław Piliszek 37d9cf0d19 [CI] Drop bindep
It is not necessary and it may be confusing.

Change-Id: If7a727861883efcae55bb70eb9c223a5870f9a4f
2022-08-27 20:00:11 +00:00
wangxiyuan bafdab171d Upgrade openEuler Distro
openEuler 20.03 LTS SP2 is out of date. This patch:
1. Upgrade openEuler to 22.03 TLS for host OS.
2. Switch guest OS from centOS 8 to ubuntu

Change-Id: If2ff036e965def141f67240945802611e1f4dc4e
2022-08-22 00:51:19 +00:00
wangxiyuan 01ed75acef Add openEuler Distro support
openEuler is the newest open source operating system, which is quite
famous and widely used in China. This patch aims to add the OS support
in kolla-ansible for the host OS.

bp: support-openeuler-os

Depends-On: https://review.opendev.org/c/openstack/ansible-collection-kolla/+/830320

Change-Id: I66dc84f02c324dbc0787ec25d4bd92ada9362e94
2022-03-28 07:54:49 +00:00
Zuul 9a1bd06098 Merge "[CI] Collect info about environment before deploy" 2022-03-03 16:35:34 +00:00
Michal Nasiadka 7080ccfc3d Add Rocky Linux support as Host OS
Depends-On: https://review.opendev.org/c/openstack/ansible-collection-kolla/+/831642
Change-Id: I70dcd2d0cade52a23b3e219b7e0aaa31193ec938
2022-03-03 09:59:16 +00:00
Radosław Piliszek e63bbed18d Clean up chrony cleanup
In the Yoga cycle we no longer need kolla chrony container removal
procedures.

Change-Id: I4dc246cf0fd68838470bf9e9bf749fa9be4d6670
2022-01-18 15:37:51 +00:00
Radosław Piliszek 9ef6bb2d96 [CI] Stop adding the DROP workaround
Since Wallaby, we default to disabling Docker's iptables
management, thus making the code being removed here obsolete.

Change-Id: Ieb7774f2380a811070aea27964a39e4c8cb02083
2021-09-30 16:47:22 +00:00
Radosław Piliszek e6edec78e5 [CI] Collect info about environment before deploy
And also collect lsmod listing.

These both are useful to have a clearer picture of the original
environment and the effect kolla ansible had on it.

Change-Id: I5d87cfd45e4369df40b8195124535e59d24700c3
2021-09-25 20:10:15 +00:00
Mark Goddard 84ac7b3096 chrony: allow to remove the container
The chrony container is deprecated in Wallaby, and disabled by default.
This change allows to remove the container if chrony is disabled.

Change-Id: I1c4436072c2d47a95625e64b731edb473384b395
2021-06-02 17:28:35 +00:00
Michał Nasiadka 337771143b CI: Configure IP on a linux bridge instead of OVS br-ex
OVS upgrade jobs failed due to assigning an IP on the br-ex interface
(which is recycled during OVS upgrade). This change introduces a bridge
and veth pair for Neutron to use.

Change-Id: Ib3bee6e810fb8d31552d4c72c2a1ccae382c51f0
2021-05-24 14:40:30 +00:00
Michał Nasiadka e83b5869a3 CI: Wait for NTP synced status in systemd
Debian's ntpd often needs more time to settle and we fail on prechecks.

Change-Id: Id23f40ab7ef75d06bb92973042c9af580ddffeb0
2021-05-14 11:18:30 +00:00
Radosław Piliszek e7247dbd9a CI: use venv only (and not virtualenv)
It seems we used virtualenv for ceph-ansible only.

Change-Id: I7f9002283462dbe4bae3c1d7ff1dedcc4e7d01f2
2020-06-17 18:24:07 +00:00
Marcin Juszkiewicz 09ab5fd0ba CI: drop pre-ussuri upgrade support
Ussuri is Py3 everywhere. We are free to drop any py2 leftovers.

Depends-On: https://review.opendev.org/731805
Change-Id: I1a9c9a14af351cd3e8b01a40f323a82ffa673d35
2020-05-30 15:23:47 +00:00
Marcin Juszkiewicz 30b8eed1cd CI: Fix Debian aarch64 jobs
Debian defaults to Python2 which is not complete in aarch64 images.
This patch changes CI to always use Python3.

We need to install several Python modules to have working ussuri jobs.

"Failed to import the required Python library (setuptools) on primary's Python /usr/bin/python3."

And then several Python2 ones for train->ussuri upgrade jobs:

"Unable to find any of pip2 to use. pip needs to be installed."

Change-Id: Ia0d3ff15d97d1cabbb0b8e7f32e8712ca3f94732
2020-04-28 19:15:06 +02:00
Radosław Piliszek 29e4ef1374 CI: Debug init-runonce
Also adds gawk for timestamping.

This helps to correlate init events with failures elsewhere.

Change-Id: I22fdb683ecf9870b2d66fedd6b40b7004317130a
2020-03-01 09:57:35 +01:00
Zuul 438a611375 Merge "CI: Test floating ip allocation and connectivity to the instance" 2020-02-28 20:06:48 +00:00
Radosław Piliszek 1f0070db6c CI: Test floating ip allocation and connectivity to the instance
Inspired by pending work of mnasiadka [1] and mgoddard [2].

[1] https://review.opendev.org/708250
[2] https://review.opendev.org/707604

Depends-on: https://review.opendev.org/709362
Depends-on: https://review.opendev.org/709671
Depends-on: https://review.opendev.org/709830
Change-Id: Ifd7bde60881a4b644fc3f18dd9fcb7f8b02d3a00
2020-02-28 10:47:45 +01:00
Zuul 0317de44cc Merge "CI: Add addressing on external network" 2020-02-27 21:26:52 +00:00
Radosław Piliszek 195813dd60 CI: Add addressing on external network
This is to allow CI testing of network connectivity.

Note only "primary" node gets an address on it.
Hence, "primary" becomes our fake external router.

Depends-on: https://review.opendev.org/709361

Change-Id: I05592888796107d6de95b940c42b2bff73ac0669
2020-02-26 13:27:00 +01:00
Zuul c3f7b6cce3 Merge "CI: Remove useless pre task" 2020-02-24 16:07:58 +00:00
Radosław Piliszek dc0c762ab5 CI: Remove useless pre task
This was never necessary because C7 IPv6 CI was not enabled at
the time and later we fixed IPv6 in C7 to avoid this issue.
In Ussuri C7 is going away so even more reasons to drop it. :-)

Change-Id: I4066c9cd86ff892d78f6713589f9afffc611dcc1
2020-02-21 10:36:01 +01:00
Radosław Piliszek 4ac7f6f3a0 CI: Replace dummy interface (fake for neutron external) with VXLAN
This allows for some real testing, especially in a multinode
environment.

Change-Id: Ic96819fefe460e14c8460e52a78d9b0d034edc80
2020-02-21 08:56:26 +00:00
Radosław Piliszek 72a2543568 Revert "Upgrade virtualenv in pre"
Reverts temporary.

This reverts commit 241e34748d.

Change-Id: I4af6b9c57e594fce407b9524f04cfa83479e16f0
2020-02-20 17:58:13 +01:00
Michal Nasiadka 220b21a844 CI: Remove dbus from bindep and playbooks
Change-Id: I8b4e649e24871b1191dba39d766b6c133c7e8f24
2020-02-20 16:50:43 +00:00
Radosław Piliszek b356762b94 CI: Refactor VXLAN overlay setup
This makes it cleaner, allows reuse and outsourcing to zuul jobs
and enables us to create multiple of these overlay networks for
testing of more advanced scenarios.

Change-Id: Id557c81f68a7f34556854e7d6efc6eddfd2e7216
2020-02-20 12:24:05 +01:00
Michal Nasiadka 241e34748d Upgrade virtualenv in pre
Since virtualenv 20.0 (amongst other changes) six version >1.12.0 is required.
This change adds upgrade of virtualenv and six in pre - to be reverted once
infra CentOS images are sorted out.

Change-Id: I0ca0347bb6ebc5d8f5d22f708211e01221165262
2020-02-12 13:48:25 +01:00
Radosław Piliszek bc053c09c1 Implement IPv6 support in the control plane
Introduce kolla_address filter.
Introduce put_address_in_context filter.

Add AF config to vars.

Address contexts:
- raw (default): <ADDR>
- memcache: inet6:[<ADDR>]
- url: [<ADDR>]

Other changes:

globals.yml - mention just IP in comment

prechecks/port_checks (api_intf) - kolla_address handles validation

3x interface conditional (swift configs: replication/storage)

2x interface variable definition with hostname
(haproxy listens; api intf)

1x interface variable definition with hostname with bifrost exclusion
(baremetal pre-install /etc/hosts; api intf)

neutron's ml2 'overlay_ip_version' set to 6 for IPv6 on tunnel network

basic multinode source CI job for IPv6

prechecks for rabbitmq and qdrouterd use proper NSS database now

MariaDB Galera Cluster WSREP SST mariabackup workaround
(socat and IPv6)

Ceph naming workaround in CI
TODO: probably needs documenting

RabbitMQ IPv6-only proto_dist

Ceph ms switch to IPv6 mode

Remove neutron-server ml2_type_vxlan/vxlan_group setting
as it is not used (let's avoid any confusion)
and could break setups without proper multicast routing
if it started working (also IPv4-only)

haproxy upgrade checks for slaves based on ipv6 addresses

TODO:

ovs-dpdk grabs ipv4 network address (w/ prefix len / submask)
not supported, invalid by default because neutron_external has no address
No idea whether ovs-dpdk works at all atm.

ml2 for xenapi
Xen is not supported too well.
This would require working with XenAPI facts.

rp_filter setting
This would require meddling with ip6tables (there is no sysctl param).
By default nothing is dropped.
Unlikely we really need it.

ironic dnsmasq is configured IPv4-only
dnsmasq needs DHCPv6 options and testing in vivo.

KNOWN ISSUES (beyond us):

One cannot use IPv6 address to reference the image for docker like we
currently do, see: https://github.com/moby/moby/issues/39033
(docker_registry; docker API 400 - invalid reference format)
workaround: use hostname/FQDN

RabbitMQ may fail to bind to IPv6 if hostname resolves also to IPv4.
This is due to old RabbitMQ versions available in images.
IPv4 is preferred by default and may fail in the IPv6-only scenario.
This should be no problem in real life as IPv6-only is indeed IPv6-only.
Also, when new RabbitMQ (3.7.16/3.8+) makes it into images, this will
no longer be relevant as we supply all the necessary config.
See: https://github.com/rabbitmq/rabbitmq-server/pull/1982

For reliable runs, at least Ansible 2.8 is required (2.8.5 confirmed
to work well). Older Ansible versions are known to miss IPv6 addresses
in interface facts. This may affect redeploys, reconfigures and
upgrades which run after VIP address is assigned.
See: https://github.com/ansible/ansible/issues/63227

Bifrost Train does not support IPv6 deployments.
See: https://storyboard.openstack.org/#!/story/2006689

Change-Id: Ia34e6916ea4f99e9522cd2ddde03a0a4776f7e2c
Implements: blueprint ipv6-control-plane
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-10-16 10:24:35 +02:00
Mark Goddard e3e5f7f20f Fix CI failures
1. Fix yamllint errors in .yamllint file(!)

YAML lint is currently failling on its own configuration file,
.yamllint. This change fixes the issues.

2. Run bindep role in Zuul jobs

This fixes an issue where libffi is not available.

Change-Id: Ic08a8e53a6905a68f0fe26d4b28184e62a64324f
2019-10-15 13:27:55 +01:00
Mark Goddard fac1670433 CI: Use any_errors_fatal in pre.yml and run.yml
This ensures that failure of a single host fails the whole play at that
task. This can avoid confusing errors such as when the task
"Assert that the nodepool private IPv4 address is assigned" fails on one
host, causing subsequent errors on other hosts.

Note that this only affects the Zuul playbooks, not Kolla Ansible's
playbooks.

Change-Id: I77a6534dd2ddd188f795e17d17a44be249d01f31
2019-10-07 09:33:41 +00:00
Mark Goddard 8e40629161 CI: Use VXLAN overlay network
VXLAN is necessary to run HA in CI (due to floating VIP
address handled by keepalived).
It also turned out to be required to have private
IPv6 address assignments.
This patch is based on linux bridge rather than OVS
to avoid problems with OVS deployed in containers.

This patch enables haproxy in multinode jobs.

Includes saving of linux networking details.

Makes DASHBOARD_URL agree with OS_AUTH_URL - properly uses the
pre-upgrade value for testing.

Co-authored-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
Depends-on: https://review.opendev.org/683068
Depends-on: https://review.opendev.org/682957
Change-Id: I66888712da80c3d6f84ee4949762961664d3adea
2019-09-19 11:07:02 +02:00
Mark Goddard eac1e479b7 CI: Sanity check that nodepool.private_ipv4 is assigned
During the MariaDB testing we saw a number of cases where this IP
address was not assigned to one or more hosts, which caused various
issues later on.

Change-Id: I61b54483e4553b926e9ddc0a8848b2daa6bc49f1
2019-08-06 19:03:05 +01:00
Eduardo Gonzalez ea1a1dee0d Verify YAML syntax in gates
This patchset implements yamllint test to all *.yml
files.

Also fixes syntax errors to make jobs to pass.

Change-Id: I3186adf9835b4d0cada272d156b17d1bc9c2b799
2018-03-26 17:56:22 +02:00
Paul Bourke 404d4d0a50 Use zuul firewall rules in gate
Till now we've been flusing iptables in the gates to allow cross node
communication in the multi node ceph jobs. This raised security
concerns, in particular it exposed memcached to the external net.

This patch uses the infra provided role 'multi-node-firewall' in order
to correctly configure iptables. Thanks to Jeremy Stanley and Jeffrey
for help with this.

Closes-Bug: #1749326
Change-Id: Iafaf1cf1d9b0227b0f869969d0bd52fbde3791a0
2018-03-08 16:37:13 +00:00
Michal (inc0) Jastrzebski 478e2cf9c9 Enable multinode scenarios in zuulv3
Another step to migrate to new CI

Change-Id: Id9bf9b4b1114f5be66f6c028a0db16d869efeceb
Depends-On: Iaacdca31b27b7033ec08499bcea7061b9d3ec279
2017-12-27 09:21:35 -08:00
Jeffrey Zhang cdd125117f Optimize zuul v3 jobs
- move check container failure from post.yml to run.yml
- add binary related jobs
- use static kolla-ansible src dir, which is helpful for kolla project
  to use.
- generate correct /etc/hosts by using private ip address and hostname
- fix the wrong api interface in global.yml file

Change-Id: Idfdee6dfe18f0fa2d4f984df59b57553122ce298
2017-10-26 09:58:29 +08:00
Jeffrey Zhang baa9319a75 Move to zuul v3 in project jobs
Partial-Bug: #1720601
Change-Id: Ibc20a6ae8c645ff82f3c14a6286073dffd4cfae2
2017-10-18 12:31:52 -07:00