Commit Graph

2638 Commits

Author SHA1 Message Date
Dmitriy Rabotyagov 3e2d7afa05 Remove Jinja from conditions
Due to CVE-2023-5764 conditional statements should not include
jinja2 templating anymore and result in warnings/failures

This patch replaces Jinja tags with slightly different format that leads to the
same result/logic.

Change-Id: I049ac770b32152866194190e54f5947fe7589b39
2024-03-27 10:42:53 +00:00
Zuul 30879b1e64 Merge "Add check_hostname option to db healthcheck tasks" 2024-03-14 17:44:51 +00:00
Jimmy McCrory 6240c769c0 Add check_hostname option to db healthcheck tasks
To allow encrypting connections of db healthcheck tasks, include the
check_hostname option to verify a server host name when an SSL
connection is required.

Also enable galera_require_secure_transport during TLS test jobs.

Change-Id: I23d839e75b202d0400aeefe6e98c429e16ecd37e
2024-03-11 11:07:00 -07:00
Jonathan Rosser 563159f49c Always use physnet1 as external network name in AIO and examples
Previously this was either 'vlan' or 'flat' depending on the external
network type, and there were also cases when the name and type were
mismatched - particularly when the flat network was untagged traffic
on a vlan bridge.

This patch removes that confusion and always names the external
network 'physnet1' to align with the upstream neutron examples.

Change-Id: I3cd8b93b42777b787552051bcdc9a90347f1e03d
2024-02-20 17:26:00 +00:00
Jonathan Rosser 03e9c6b69e Add user defined hooks that can run extra playbooks
This patch adds variables which when can be used to extend
openstack ansible by calling additional user defined playbooks
at the start and end of the main setup-* playbooks.

Change-Id: Ic55dd6447f603d91beaeea28beb04e4c1393d6af
2024-02-05 12:32:37 +00:00
Zuul 55e9baecac Merge "Add openstack-resources playbook" 2024-02-02 22:07:36 +00:00
Dmitriy Rabotyagov 9694ae8c23 Ensure disable/enable haproxy backends exists for all services
Right now we ensure that services are enabled/disabled while running
playbooks only for core services. At the same time some services still do
not have this mechanism, that might result in unexpected outages.

So we ensure that all service playbooks will behave in the same way and
disable backends in advance before playbook will do any modifications.

With that, setting variable `haproxy_drain: true` will ensure that moving
backend to the MAINT state will be graceful and all current connections
will close normally unless a timeout is reached, which is 2 min by default.

Closes-Bug: #2047017
Change-Id: I8554defec4df54d14be72ae9a1560907ff1aaddf
2024-01-04 15:33:53 +00:00
Dmitriy Rabotyagov 0c1c9a8485 Add openstack-resources playbook
This aims to add a user-facing interface to interact with
openstack_resources role.

Change-Id: I1c2688f469cfaef02e6e925fa7a99573f61efbd6
2023-11-20 14:54:01 +01:00
Jonathan Rosser 2302883430 Use haproxy_service_setup playbook from plugins collection
Move the common playbook to the OSA plugins collection and use it
from there.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/900431
Change-Id: I34c4900cf34635e46b8faec06dfc7cd24885eb3d
2023-11-08 16:11:58 +00:00
Dmitriy Rabotyagov 3ff832bca4 Drop ssh_keypairs_install_authorized_keys reference
This variable does not exist outside of the contenxt of this playbook
so can be safely removed.

Change-Id: Icdf8c2cafe145a9edab8211443a8c4414fe823ea
2023-10-19 11:18:39 +02:00
Dmitriy Rabotyagov 607d237e50 Fix vars-file include for os-nova-install
This is a follow-up patch to [1]. Somehow, this issue was not catched by
our CI on master, likely due to some safeguard logic in ansible-core 2.15

Though, it's still worth to align vars_files include and do that properly.

[1] https://review.opendev.org/c/openstack/openstack-ansible/+/897570

Related-Bug: #2034583
Change-Id: I130aab09610f594e0d67db5082c8ff28c9298661
2023-10-17 17:06:31 +02:00
Zuul de65a235ae Merge "Run nova db post setup from nova playbook" 2023-10-16 18:11:22 +00:00
Zuul e8fe3944f1 Merge "Remove common nova playbook" 2023-10-16 18:10:10 +00:00
Dmitriy Rabotyagov f05c90103c Define install_method default when hosts resolution depend on it
In some playbooks we try to fetch openstack_service_setup_host based on
install_method variable. However, if variable is not defined,
and it is not by default, playbooks will run against localhost,
which will result in failure.

Closes-Bug: #2031647
Change-Id: I247ce2a808076ff55347db54327a1e4913ea3006
2023-10-10 09:57:42 +00:00
Dmitriy Rabotyagov a44f1212c3 Run nova db post setup from nova playbook
We need to run specific tasks, like online migrations or cells discovery
after all tasks have finished against nova conductor hosts.

This can't be done with the role logic, as we run computes the last,
and delegation to conductors does not work nicely since handlers are
not delegated.

Closes-Bug: #2034583
Change-Id: Ic4486cf90310dc81af15b9297e84c078e612c0c2
2023-10-09 11:15:50 +00:00
Dmitriy Rabotyagov 61ea7a8207 Remove common nova playbook
Code of os-nova-install has been refactored to include content from the
common nova playbook. This allows us to be more flexible in executed
tasks and simplify logic.

Related-Bug: #2034583
Change-Id: I21fe061d93cf77c97f8fa6d0003219595459e1c3
2023-10-09 13:15:31 +02:00
Zuul 68643fb110 Merge "Allow deployment on debian bookworm hosts" 2023-10-04 19:13:55 +00:00
Dmitriy Rabotyagov ba9a248550 Gather extra networking facts for keepalived
By default keepalived pingable check tries to verify the gateway which
in it's turn requires to have appropriate facts defined.

At the moment re-running haproxy playbook as a standalone instance
when facts are outdated will lead to re-configuration of keepalived
to check 127.0.0.1 as a track script.

So we gather extra network facts to identify the gateway for
pingable script.

Change-Id: I4e72e81d05d8d70e76730a46687dd06f3e66ebc6
2023-09-27 11:31:41 +02:00
Jonathan Rosser b00b4db3e2 Allow deployment on debian bookworm hosts
Change-Id: I1a2d8e6ecbc12168971b3983308ffaf537b87d19
2023-09-11 17:23:22 +01:00
Jonathan Rosser 33e59f154f Apply deployment env vars during keystone main_pre
This will be required for tasks that need external http
access, such as package installations as part of main_pre
tasks file.

Change-Id: I285f34f2b657f1d7e6436505bd39e025e504bfa9
2023-09-07 07:07:32 +01:00
Dmitriy Rabotyagov 5ea6f419c7 Fix linters to satisfy ansible-lint 6.18
With update of ansible-lint to version >=6.0.0 a lot of new
linters were added, that enabled by default. In order to comply
with linter rules we're applying changes to the role.

Update of linters version is required for bumping ansible-core
version.

Change-Id: Icb7d9906bdc3dcb50c44c67881c2078ade8b72f2
2023-08-22 15:47:38 +00:00
Jonathan Rosser c19c11c53c Allow bind mount of types other than directory
It is possible to bind mount files as well as directories. Allow
this to happen in the LXC container config.

Change-Id: I8b9acd2cdd8cd6e7e77f2286a2c48ed4facdcfd9
2023-08-17 13:02:00 +01:00
Andrew Bonney d0aeb6c5bb Remove dynamic-address-fact call which is no longer required
I don't believe this is necessary anymore since 'management_ip'
was introduced into openstack_user_config.

Whether 'management_ip' is set or not, 'management_address' is
already set in the inventory, and if dynamic-address-fact runs
here it can confusingly override the value of 'management_address'.

Change-Id: I0565eeffd599cc90119420edc18ed37aea80b0db
2023-08-02 09:30:56 +01:00
Zuul 49738ba852 Merge "Remove Ubuntu 20.04 support" 2023-07-19 09:34:55 +00:00
Damian Dabrowski 60750a2796 Gather facts before including common-playbooks
For nova, neutron and cinder, haproxy service is configured before
playbooks from common-playbooks/ are included(and facts for these hosts
are gathered). Eventually, haproxy config tasks are executed without
common facts.

It may lead to several corner cases. For example, after facts cache
expires and user runs os-nova-install.yml, haproxy config('Create
haproxy service config files' task) will fail because
`haproxy_service_configs` uses `nova_console_type` variable which needs
`ansible_facts['architecture']`. It can be easily reproduced with:
```
rm -rf /etc/openstack_deploy/ansible_facts/ && \
openstack-ansible /opt/openstack-ansible/playbooks/os-nova-install.yml
```

As a solution, this change gathers facts at the beginning of
os-<service>-install.yml playbooks, instead of doing this inside
common-playbooks/<service>.yml.

This bug was not detected by CI because during deployment process,
hardware facts are gathered for all hosts (at least) by
security-hardening.yml. As long as they exists in cache everything works
fine.

Change-Id: I27073a1bf85294ff65bde24cae939f28f5c69bd7
2023-07-11 21:00:19 +02:00
Zuul ea0bbd2c58 Merge "Use include_role in task to avoid lack of access to vars" 2023-07-08 17:42:42 +00:00
Dmitriy Rabotyagov d72b3394a2 Remove Ubuntu 20.04 support
With ansible-core>=2.14 python 3.8 support has been dropped. Moreover,
nova has bumped minimal required version for libvirt/qemu, which makes
us to remove support for Ubuntu 20.04 Focal Fossa.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-os_zun/+/884362
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/884363
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/884361
Change-Id: I3e6e22553248a9199113a65b0dbe992c38ccb22e
2023-07-07 12:39:23 +02:00
Zuul 4dd73c3ebf Merge "Add TLS support to ceph-rgw backends" 2023-07-07 06:53:33 +00:00
root 9690b34193 Use include_role in task to avoid lack of access to vars
This patch updates the security hardening playbook to use include_role
within a task versus using 'roles' directly to fix cases where
apply_security_hardening is set to False. Some change to Ansible
appears to limit access to vars when the role is skipped, resulting
in failures. The side effect of this change is the role is skipped
entirely (when applicable) versus the individual tasks being skipped,
which speeds up deployment times.

Change-Id: Iaf81f778ffd9fcf6572e03e8ce73ffef46bd98aa
2023-06-30 14:19:19 +00:00
Zuul 42b7c85528 Merge "Ensure management_address is used instead of ansible_host" 2023-06-19 16:16:25 +00:00
Damian Dabrowski 14f69fbb5d Add TLS support to ceph-rgw backends
By overriding the variable `ceph_rgw_backend_ssl: True` HTTPS will
be enabled, disabling HTTP support on the ceph-rgw backend api.

The ansible-role-pki is used to generate the required TLS
certificates if this functionality is enabled.

Change-Id: I276ccd7e49db7e7ffe4f6f6c22ab1a82edc34688
2023-05-29 17:30:11 +02:00
Zuul 802cef2aa6 Merge "Fix repo url in healthcheck-infrastructure.yml" 2023-05-26 21:25:33 +00:00
Damian Dabrowski 724caeffd8 Fix repo url in healthcheck-infrastructure.yml
Repository may be also available over https.
That is why `openstack_repo_protocol` variable should be used instead
of hardcoded http://.

Change-Id: Ic2d829a6a87746dc746efe54e5e77e0f694cdf3b
2023-05-26 14:24:00 +02:00
Zuul 91bd608395 Merge "Run healthcheck-openstack from utility host" 2023-05-26 00:18:36 +00:00
Dmitriy Rabotyagov 555ec6abb7 Remove unreadable unicode symbols
There was No-Break space present in the file (U+00a0) which is now
removed with a regular space.

Change-Id: I9e7270c2236dfea7716d3efb787459ea947c2a17
2023-05-24 14:44:35 +02:00
Dmitriy Rabotyagov 86d1bdff55 Rename container_address to management_address
This patch aims to reduce confusion caused by a variable
`container_address` that's applicable for bare metal hosts. With that
it renames  `is_container_address` to `is_management_address`
to be aligned with the purpose of the variable, as `container` part
raised confusion.

Change-Id: I314224f3376cf91e05680b11d225fdaf81ec32ab
2023-05-22 09:57:17 +02:00
Zuul 41965140c3 Merge "Move repo_packages to group_vars" 2023-05-19 21:52:56 +00:00
Dmitriy Rabotyagov f3d58b5d83 Bump down etcd version for zun
Docker libkv [1] does not support etcd of version >=3.4,
as otherwise ErrInvalidJSON exception is raised. Since
docker cluster-store is deprecated and removed in 23.0.0,
support of etcd won't be fixed.

At the same time while zookeeper is also a supported driver,
it's usage causes docker to crash on attempt to use it.

So bumping etcd version down is the only solution at the
moment.

[1] https://github.com/docker/libkv

Needed-By: https://review.opendev.org/c/openstack/openstack-ansible-os_zun/+/880288
Change-Id: I9a13d836f7f266c54283125dc09d11feeea23d3e
2023-05-18 16:52:05 +00:00
Dmitriy Rabotyagov 70c48b47b2 Run healthcheck-openstack from utility host
Instead of running healtcheck-openstack from localhost, we launch it
from utility host now which has required env and credentials to
access cloud.

Change-Id: If0c11958bbe3ab64f4f535c7e795f3e035f52a60
2023-05-18 14:30:05 +02:00
Dmitriy Rabotyagov 9fca8555df Move repo_packages to group_vars
At the moment it's not possible to apply different versions of
services to the different groups due to playbook vars having
prescedence over group_vars. However, it can be quite important
to  such use cases, for example for phased rollouts of newer versions.

This will also reduce amount of unnecessary variables that are included
for each host, since only required git details will be loaded.

Closes-Bug: #2007296
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-os_rally/+/881954
Change-Id: Icaa3a958926d9f9aa6cb649bd9f3da9449dd7490
2023-05-18 08:49:51 +00:00
Dmitriy Rabotyagov d1d68f8f30 Add drain option to haproxy-endpoint-manage
Drain option allows to wait until all connections will close and then
move backend to the MAINT state. Task will wait until MAINT is reached,
so drain is disabled by default.

Change-Id: I531f2def83df9779f656ccb308d9b234f110107a
2023-05-03 09:37:24 +02:00
Zuul da7862de0c Merge "Stop gathering local software_versions for services" 2023-04-29 14:33:51 +00:00
Dmitriy Rabotyagov 61041a53d5 Switch spice-html5 source to freedesktop gitlab
Repository has been removed from github and it's not available there
anymore.

Change-Id: I8c5ef6efadcf2a8deb48bbdf88f8672433ae939c
2023-04-24 09:57:45 +02:00
Zuul 3c201f8c71 Merge "Implement separated haproxy service config" 2023-04-21 22:00:01 +00:00
Damian Dabrowski 930236d7d2 Implement separated haproxy service config
All services placed behind haproxy include an additional playbook
responsible for creating their haproxy services.

Scope for some variables was changed to be more optimal.

Change-Id: I0eec358d982cc09dcb7e2c0045b9684c65876faf
2023-04-18 21:07:41 +02:00
Dmitriy Rabotyagov bb4f1c7b2a Gather generic masakari facts
With commit [1] we moved extra facts gathering to pre_tasks
but with that we did not enable generic facts gathering, which
led to regression. So we cover this by ensuring that generic
facts are also gathered and not only extra ones.

[1] 8bc9b167ab
Closes-Bug: #1979145

Change-Id: Ibb9dc3377a4de06af25281bf777b16faad16d261
2023-04-14 10:16:29 +02:00
Dmitriy Rabotyagov e26b56adfa Stop gathering local software_versions for services
With latest ansible-core playbooks started failing on adding extra compute
or controller nodes, when cinder/nova playbooks run with limits.

This happens as we're trying to reply on local facts for hosts that are
expired. At the same time, it's not always possible to collect them, as some computes
can be down while adding another one.

With that we're simplifying flow and avoid old process of
restarting services or executing migrations based on local facts.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/880147
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-os_cinder/+/880210
Closes-Bug: #2009834
Change-Id: I44dc8567e9a93f91327202de1bf88a067266711d
2023-04-12 19:12:04 +02:00
Zuul fa225523c9 Merge "Swith to tracking stable/2023.1 for upstream services" 2023-04-12 12:45:46 +00:00
Damian Dabrowski f600f995e4 Enable TLS frontend for repo_server by default
We aim to secure traffic from haproxy to repo backends with TLS[1].
But there is no point in doing that unless the traffic to haproxy
frontend is encrypted.

This patch enables TLS on repo frontend when `haproxy_ssl_all_vips` is
enabled.

[1] https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/876429

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/877429
Change-Id: I9f0f40977b13a150e30f000dee8d7309bbffe74f
2023-04-07 20:26:49 +00:00
Dmitriy Rabotyagov f274b472f4 Swith to tracking stable/2023.1 for upstream services
Since Antelope (2023.1) release is official now, it's worth switch to it
in openstack_services, to ensure we're testing Antelope rather then
Bobcat already.

Change-Id: If99611310cb5795a27c89a90ec4ca2d067b2165f
2023-04-04 09:26:55 +00:00