Commit Graph

546 Commits

Author SHA1 Message Date
Zuul 506d3bae49 Merge "[Feature] Add skyline deployment capability" 2024-04-16 10:45:07 +00:00
James Denton 2c06a65a75 [Feature] Add skyline deployment capability
This change adds Skyline deployment options to OSA.

New files have been added / integrated into the normal deployment
process to support the use of skyline.

Co-Authored-By: Jonathan Rosser <jonathan.rosser@rd.bbc.co.uk>
Signed-off-by: Kevin Carter <kevin@cloudnull.com>
Needed-By: https://review.opendev.org/c/openstack/openstack-ansible-os_skyline/+/912333
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-os_skyline/+/914439
Change-Id: Ia5c8c60fb152e3eb27e2719f6d7032ca62690601
2024-04-05 10:56:31 +00:00
Zuul f4bf8419e8 Merge "Fix inventory defenition for Cloudkitty" 2024-04-02 17:25:13 +00:00
Dmitriy Rabotyagov 959da8cd03 Fix inventory defenition for Cloudkitty
At the moment Cloudkitty is targeted at all LXC hosts along with containers
which is not needed nor intended.

Unfortunatelly there's no really good compatible fix exist, so action from operator is required to handle transition to new naming for
the service.

Change-Id: I9360495e3b3347568969e36e0e96bb1325efd59f
2024-03-22 10:57:51 +00:00
Dmitriy Rabotyagov 8d46ed8842 Allow env.d to contain underscores in physical_skel
At the moment our dynamic_inventory does have assumption that group names
defined in env.d will not contain underscores, except when it's ending
with `_hosts` or `_containers` since inventory script uses split on `_` and taking
the last argument. So in cases when underscore was used elsewhere in the
group name it will result in unexpected behaviour.

Instead of this approach we now use regexp which replaces the last octet separated with underscore with intended suffix to preserve
rest behaviour.

Change-Id: Id9ba56292972b8b52b4786c78684f2d6f289d88a
2024-03-05 09:17:44 +00:00
Dmitriy Rabotyagov d74d038032 Do not use underscores in container names
According to RFC1034 [1], underscores are not legitimate symbols for
hostnames. We have a naming convention that historically uses underscores
in names of containers which leads to a complications in some cases/logic
Moreover, container actual hostnames are always different from their
inventory_hostnames, which might break some logic of external dependencies.

To change this historical issue, all new containers will have different
naming convention, where only dash will be used as a separator.
This will not touch already existing in inventory containers and they will
preserve their names for the time being.

[1] http://www.faqs.org/rfcs/rfc1034.html

Change-Id: Iedebf9935059ecfe8370f2a84ad52516cc93320e
2024-03-05 09:17:37 +00:00
Jonathan Rosser 03e9c6b69e Add user defined hooks that can run extra playbooks
This patch adds variables which when can be used to extend
openstack ansible by calling additional user defined playbooks
at the start and end of the main setup-* playbooks.

Change-Id: Ic55dd6447f603d91beaeea28beb04e4c1393d6af
2024-02-05 12:32:37 +00:00
Zuul 55e9baecac Merge "Add openstack-resources playbook" 2024-02-02 22:07:36 +00:00
Dmitriy Rabotyagov 5b57f10eec Add support for extra Python packages inside Ansible runtime
At the moment there is no well defined way to supply list of extra Python
requirements for Ansible venv. However, some collections for their work might
require presence of extra libraries inside the execution environment.

While PIP_OPTS might be used right for that, it's usage is not transparent
or documented.

In order to handle such need, `user-ansible-venv-requirements.txt ` is being implemented
that reside inside OSA_CONFIG_DIR and contains list of required packages
for installation when running bootstrap-ansible.sh

Change-Id: Ic99f7eff200e2e672dcc3edd875b155af84232b7
2024-01-11 14:35:09 +00:00
Dmitriy Rabotyagov 4203aa26c6 Modify RGW client format
With changes to config_template module that restored usage of {% raw %} tags [1]
renderring of mapping keys, if they're defined as variables, was broken.

Ansible, by design [2], does not render mapping keys. Moreover, it was not
working as intended anyway, since renderring happened in post-copy stage
so same records were not merged together, which resulted in #1812245

As such behaviour is expected by Ansible design, instead of adding some
workaround in config_template module, I suggest working around issue
by defining troublesome mapping with Jinja, that will allow it to render properly.

[1] https://review.opendev.org/c/openstack/ansible-config_template/+/881887
[2] https://github.com/ansible/ansible/issues/17324#issuecomment-685102595

Closes-Bug: #2048036
Related-Bug: #1812245
Change-Id: I8a32736239c6326d817c620451799c13d5d8938c
2024-01-04 14:41:59 +00:00
OpenStack Release Bot 916968afbd Update master for stable/2023.2
Add file to the reno documentation build to show release notes for
stable/2023.2.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/2023.2.

Sem-Ver: feature
Change-Id: I182d9ac70b3149dd035aaec89a64db9321a514f9
2023-12-12 12:34:55 +00:00
Zuul c9f50fd949 Merge "Bump roles and upstream version for 28.0.0.rc1" 2023-12-11 11:01:35 +00:00
Dmitriy Rabotyagov 1403962a83 Bump roles and upstream version for 28.0.0.rc1
Change-Id: Icd2a692cd0077d1fb1b9cf9ae0572dc7878c70b4
2023-12-08 15:52:21 +00:00
Dmitriy Rabotyagov 7b4640d5fb Add variable to control queues redundancy
Since we have 2 redundancy scenarios that are controlled independnetly
and are exclusive, we add variable to disable any queues redundancy.
This also allows to adjust logic of classic queues versioning, since usage
of CQv2 with classic mirrored queues may result in reduced performance
as CQv2 do not have optimizations for mirrored queues [1]

[1] https://blog.rabbitmq.com/posts/2023/05/rabbitmq-3.12-performance-improvements/#classic-queues-massively-improved-classic-queues-v2-cqv2

Change-Id: I81c7e9ce0ed090d7b87ba865c7dd5b4b706701d5
2023-12-08 15:52:10 +00:00
Dmitriy Rabotyagov f33a8b1836 Disable RabbitMQ quorum queues by default
During PTG we agreed to disable quorum queues by default during this
cycle and wait for improvements proposed as part of [1] before enabling
it by default.

This also adds a separate job that will test scenario with enabled quorum
queues.

[1] https://review.opendev.org/q/topic:bug-2031497

Change-Id: I0807cc1ed991fd85f9f74d4a360d3fd23cde227c
2023-11-27 10:23:49 +00:00
Dmitriy Rabotyagov 0c1c9a8485 Add openstack-resources playbook
This aims to add a user-facing interface to interact with
openstack_resources role.

Change-Id: I1c2688f469cfaef02e6e925fa7a99573f61efbd6
2023-11-20 14:54:01 +01:00
Damian Dabrowski 8f3c02d2da Explicitly add localhost to inventory.ini
Because implicit localhost is not a part of 'all' or any other group,
playbooks executed with '--limit' may not take it into account.
The problem was extensively described in bug #2041717.

This change explicitly adds localhost to OSA inventory to avoid
unexpected behavior.

Closes-Bug: #2041717
Change-Id: Ib44ed22d7132b42a4185a91f12c66ced5a1a6209
2023-11-01 00:32:57 +00:00
Dmitriy Rabotyagov 61ea7a8207 Remove common nova playbook
Code of os-nova-install has been refactored to include content from the
common nova playbook. This allows us to be more flexible in executed
tasks and simplify logic.

Related-Bug: #2034583
Change-Id: I21fe061d93cf77c97f8fa6d0003219595459e1c3
2023-10-09 13:15:31 +02:00
Zuul dd3e54cc82 Merge "Replace HA policies for RabbitMQ with quorum" 2023-09-03 11:15:10 +00:00
Dmitriy Rabotyagov cc9e72b7f0 Do not add all computes as OVN gateways
At the moment all compute nodes are explicitly added as
OVN gateway nodes. At the same time one of recommended setups
are to not pass public networks to compute hosts and have
standalone network nodes that are running ovn gateways which is
not possible to configure with current setup.

Change-Id: If99ddc47d32acf41cdb542b4e56d90b6e3589a56
2023-08-23 16:13:24 +02:00
Dmitriy Rabotyagov 62fb57b2d7 Replace HA policies for RabbitMQ with quorum
HA policies were replaced with quorum queues [1] and discuouraged and
marked for removal in 4.0 [2]

Based on that we perform migration from HA queues to quorum,
since they're already supported in oslo.messaging.

Patches per-service are required to enable quorum queues in service
configuration.

This also adjusts upgrade doc to contain a variable required for
proper nova cell update on changed vhost.

[1] https://www.rabbitmq.com/quorum-queues.html
[2] https://blog.rabbitmq.com/posts/2021/08/4.0-deprecation-announcements/

Change-Id: Icd5eabcad4801b454f29b388613d7241bb9b0ad0
2023-08-23 13:39:47 +02:00
Zuul e44a274f7c Merge "Enable S3 API by default" 2023-07-07 06:56:28 +00:00
Zuul 0df5f09755 Merge "Update master for stable/2023.1" 2023-05-31 16:15:55 +00:00
Zuul ebf9f3ae1b Merge "Bump SHAs for Antelope RC2" 2023-05-31 10:33:50 +00:00
Dmitriy Rabotyagov 29709994f4 Bump SHAs for Antelope RC2
Change-Id: Ib76379c5fdc9318e622d7d8596e4a79c56ab0d8b
2023-05-30 19:47:56 +02:00
OpenStack Release Bot c5fa98690a Update master for stable/2023.1
Add file to the reno documentation build to show release notes for
stable/2023.1.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/2023.1.

Sem-Ver: feature
Change-Id: I0ef888fc91bf3888851f6251a21d546f62333195
2023-05-30 15:34:43 +00:00
Zuul 377c12c87d Merge "Implement support for haproxy_accept_both_protocols" 2023-05-30 13:55:48 +00:00
Damian Dabrowski e00689d50f Enable S3 API by default
When only 'swift' is specified in `rgw_enable_apis`, sending a http
request to the base RadosGW API URL('/') returns '405 Method Not
Allowed'.
It causes an important issue, because when any change is made to RadosGW
configuration via ceph-ansible, the 'restart ceph rgws' handler is
triggered that use restart_rgw_daemon.sh[1] script to restart radosgw
service.
Both curl and wget used by this script return non-zero return code on
'405 Method Not Allowed' response, causing ceph-ansible playbook to fail.

As a solution 's3' api can be enabled by default. With S3 API enabled,
base RadosGW API URL('/') returns 200 instead of 405 RC.
This change affects only environments using integrated ceph-ansible.

[1] https://github.com/ceph/ceph-ansible/blob/stable-7.0/roles/ceph-handler/templates/restart_rgw_daemon.sh.j2#L68

Change-Id: Ief8759e19d935aec9d8cfa855b1b0ba2b0c83424
2023-05-30 00:11:08 +02:00
Dmitriy Rabotyagov 7437b7641b Bump SHAs for OpenStack-Ansible 27.0.0.rc1
Change-Id: I20a5049e14fae2da25a87a97efa58692e561b1be
2023-05-26 10:34:19 +02:00
Damian Dabrowski b75a9d0dd0 Implement support for haproxy_accept_both_protocols
Enabling TLS on the internal VIP for existing deployments will cause
downtime until each client is configured to use HTTPS instead of HTTP.

To avoid downtime, it is recommended to enable
`openstack_service_accept_both_protocols` until all services are
configured correctly.
It allows haproxy frontends to accept both HTTP and HTTPS.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/864784
Change-Id: Ie6f5b73c54b0a6d1f661a9d4f33b8a301d8c4170
2023-05-26 00:33:18 +02:00
Dmitriy Rabotyagov c8ecc9fa10 Add management_ip option for metal hosts
In cases when SSH and mgmt networks are different, it might be important
to have valid management_address that services are relying on when
listening on interfaces. At the moment for bare metal hosts
management_address will be equal to ansible_host which leads to
unpredictable behaviour under some scenarios. With management_ip we allow
to define another IP address that will be used as container/management
address for bare metal host, while `ip` will still represent
ansible_host.

Related-Bug: #2002645
Change-Id: I3152ae7985319e85b9ea520021f9eea6f5850341
2023-05-22 11:53:55 +02:00
Dmitriy Rabotyagov 86d1bdff55 Rename container_address to management_address
This patch aims to reduce confusion caused by a variable
`container_address` that's applicable for bare metal hosts. With that
it renames  `is_container_address` to `is_management_address`
to be aligned with the purpose of the variable, as `container` part
raised confusion.

Change-Id: I314224f3376cf91e05680b11d225fdaf81ec32ab
2023-05-22 09:57:17 +02:00
Dmitriy Rabotyagov 9fca8555df Move repo_packages to group_vars
At the moment it's not possible to apply different versions of
services to the different groups due to playbook vars having
prescedence over group_vars. However, it can be quite important
to  such use cases, for example for phased rollouts of newer versions.

This will also reduce amount of unnecessary variables that are included
for each host, since only required git details will be loaded.

Closes-Bug: #2007296
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-os_rally/+/881954
Change-Id: Icaa3a958926d9f9aa6cb649bd9f3da9449dd7490
2023-05-18 08:49:51 +00:00
Damian Dabrowski 930236d7d2 Implement separated haproxy service config
All services placed behind haproxy include an additional playbook
responsible for creating their haproxy services.

Scope for some variables was changed to be more optimal.

Change-Id: I0eec358d982cc09dcb7e2c0045b9684c65876faf
2023-04-18 21:07:41 +02:00
Damian Dabrowski 73a8bdde58 Serve security.txt directly from haproxy
Currently security.txt file is stored and served from keystone.
It's not necessary as haproxy is able to serve static files[1].

This patch creates security_txt haproxy service responsible for
serving security.txt static file using recently implemented maps feature.

[1] https://sleeplessbeastie.eu/2020/05/11/how-to-serve-single-file-using-haproxy/

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/880088

Change-Id: If0ec375055abedceb13a035b9c8f5107f4659f86
2023-04-12 14:17:56 +02:00
Damian Dabrowski 1a3c3f5603 Update letsencrypt docs
After haproxy base service was added in [1], dummy service is not needed
for deployments without horizon.

[1] https://review.opendev.org/c/openstack/openstack-ansible/+/876160/

Change-Id: I8258b3e2905a6d5b2ae435a3449ee834483e6908
2023-03-31 21:20:18 +02:00
Jonathan Rosser 1694b142bc Split haproxy horizon config into 'base' frontend and 'horizon' backend
Several new features have been added to openstack-ansible which
require special handling on port 80/443, such as support for
LetsEncrypt and security.txt. This causes a confusing situation
when several different features are served by the 'horizon' haproxy
frontend/backend when horizon itself may not necessarily be deployed.

This patch splits the haproxy config for port 80/443 into a 'base'
frontend which is always deployed and is responsible for handling
requests for LetsEncrypt and security.txt with all other traffic being
handled by a default 'horizon' backend.

The 'horizon' backend is only deployed when the horizon service is
enabled, i.e. when the horizon_all ansible group has members.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/876157
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/876436
Change-Id: I7bdf71faa761897cc2a0e01d1af1ccb4946f10f5
2023-03-15 23:16:48 +01:00
Jonathan Rosser 3c779ac916 Add releasenote for ANSIBLE_INJECT_FACT_VARS defaulting to false
Change-Id: Ibfadbc054152c9de91a2ecb989939dbff63eeecd
2023-03-08 08:29:31 +00:00
Jonathan Rosser dc1f76c823 Remove support for calico ml2 driver.
The deployment of etcd as a service on the control plane is no
longer needed as calico was the only user of this.

The etcd-server role remains in our requirements as it is used
internally as part of the Zun playbook.

Change-Id: I2a158fd2b85ec0e637071ed4ef7c123a6583ecc0
2023-02-23 12:13:55 +01:00
Jonathan Rosser df4758ab1b Allow git servers for openstack services and tempest to be overridden
This will allow the services to be cloned from github or a local
mirror by setting a small number of variables rather than overriding
every git repo URL individually.

Change-Id: I750d897e9e3c8ca161c0740c73cdc4c6e42b6440
2023-02-01 14:57:14 +00:00
OpenStack Proposal Bot 36919af266 Imported Translations from Zanata
For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

Change-Id: Ib358d0df3a5e7ded711d3e36f5c1a9bb87416f59
2022-12-17 02:46:36 +00:00
OpenStack Release Bot 7f8f6327d0 Update master for stable/zed
Add file to the reno documentation build to show release notes for
stable/zed.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/zed.

Sem-Ver: feature
Change-Id: I683fb98c47a3894b2d66df44a7dd6681ae02c6b2
2022-12-16 08:45:44 +00:00
Dmitriy Rabotyagov b2b61f792e Bump OpenStack-Ansible for Zed release
Change-Id: Ib97bdd1c77649ce8d3a767da6d29edbe9f7c2888
2022-12-14 13:03:44 +00:00
James Denton 79fc2a604d Implement OVN inventory changes and deploy by default
This patch updates the OSA inventory to include a new group,
network-gateway_hosts, which operators can define to dictate which
nodes can be considered OVN gateway nodes. In addition, the default
mechanism driver is no longer ML2/LXB and must be specified by the
operator.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/865961
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/866249
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/866159
Needed-By: https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/865973/4
Change-Id: I7be6520b338b5578b04631cfa04a9122c735959c
2022-12-05 16:11:25 +00:00
Zuul 031ae9f727 Merge "Add zookeeper deployment" 2022-12-01 21:34:58 +00:00
Dmitriy Rabotyagov e315e2e327 Allow to define user.rc file for OSA
At the moment there is no handy functionality to override ENV variables
and store them in git near OSA config. As while we take into account
that ENV vars can already exist and we should use them, these VARs must
be defined in /etc/environement or bashrc files.

This patch aims to look for user.rc file inside OSA_CONFIG_DIR.
If it's present - it will be sourced with all it's content. This way
deployers can store environemnt variables overrides in git and they will
be loaded during on openstack-ansible startup.

Change-Id: Ie24ada54a0e0dc064be028929b416d983fdb5b49
2022-11-28 16:54:28 +01:00
Dmitriy Rabotyagov 65ce2753fa Add zookeeper deployment
Zookeper is being added as a coordination backend which is
required for some service features to work properly.

Change-Id: I2d464ae339f057ea4ba08bd343b6a51c029a74f3
2022-11-25 09:13:22 +00:00
Zuul 5e1e8913c4 Merge "Add release note about used ansible and ceph versions" 2022-11-08 17:43:19 +00:00
Zuul 73db9d6575 Merge "Remove usage of rsyslog roles" 2022-10-31 17:13:17 +00:00
Zuul 7be4cf6f89 Merge "Switch Ceph for Ubuntu Jammy to distro" 2022-10-31 15:09:09 +00:00