918 Commits

Author SHA1 Message Date
Dmitriy Rabotyagov
d1f662225d Disable v3 endpoints by default
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/942775
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/945476
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-os_trove/+/945477
Change-Id: If9b81df824d04ef6b13443441acd0af0241cbe78
2025-03-25 15:50:49 +00:00
Dmitriy Rabotyagov
a76535685f Align on cinder service naming
With an effort [1][2] to align cinder service type naming to
`block-storage` we revert deprecation of some variables, which
were annoinced as deprecated in [3] and dperecate versioned
ones instead.

[1] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/thread/QQ7L44VE4MOWOQVOFCZQPTIVI35A6SCK/#X7BYL6AYQDMXAG7XXDNTSASWZCGEHQ37
[2] https://review.opendev.org/q/topic:%22remove-cinder-v2-crud%22
[3] https://review.opendev.org/c/openstack/openstack-ansible-os_cinder/+/934593

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/942580
Change-Id: I7a8e563fa60a62bdc24d5a3c9d7fa1a43f324b76
2025-02-26 12:47:02 +00:00
Dmitriy Rabotyagov
4416d82c89 Auto-fix usage of modules via FQCN
Since ansible-core 2.10 it is recommended to use modules via FQCN
In order to align with recommendation, we perform migration
by applying suggestions made by `ansible-lint --fix=fqcn`

Change-Id: I7220eec2957ea2ef8acfa9dba3e443ab39251646
2025-02-11 19:31:12 +01:00
Dmitriy Rabotyagov
014a97fe88 Auto-fix yaml[truthy] rules
In order to reduce divergance with ansible-lint rules, we apply
auto-fixing of violations.

In current patch we replace all kind of truthy variables with
`true` or `false` values to align with recommendations.

Change-Id: I7f7dfc3aec4578777e1c886504f5b7e589f17c26
2025-02-11 19:05:15 +01:00
Jonathan Rosser
584d2a0104 Remove support for amqp1
Support is removed in oslo.messaging so we remove support in
openstack-ansible roles.

Change-Id: Ibb9758515e02e318ff58939516cb298ca55725d7
2025-01-06 10:36:50 +00:00
Dmitriy Rabotyagov
ce7c0dae6b Remove cinder v2 references
Cinder v2 has been deprecarted and removed for a while now, so we can
safely cleanup leftovers of it now.

We also clean-up/align variable names which were not used.

Change-Id: Ie49d145775d753323406fd5588a479c60c8807cc
2024-11-13 09:10:35 +01:00
Dmitriy Rabotyagov
eb70c981dd Ensure that first/last host detection is deterministic
With ansible-core 2.16 a breaking changes landed [1] to some filters
making their result returned in arbitrary order. With that, we were
relying on them to always return exactly same ordered lists.

With that we need to ensure that we still have determenistic behaviour
where this is important.

[1] https://github.com/ansible/ansible/issues/82554

Change-Id: I66d23f9e52ef53b1a462878ea2d94cb01faed717
2024-07-22 15:11:16 +02:00
Dmitriy Rabotyagov
7cedd910d0 Disable heartbeat_in_pthread by default
The default value for heartbeat_in_pthread has been reverted in
oslo.messaging to False [1] and backported back to Yoga.

At the moment this setting brings intermittent issues during live
migrations of instances and some other operations. So makes sense
to align it with default value.

[1] https://review.opendev.org/c/openstack/oslo.messaging/+/852251

Change-Id: I06dae436639562c88ef917fd06f1f73e2ce74720
2024-07-16 13:50:17 +02:00
Jonathan Rosser
79f14e4aa4 Start iscsid on cinder-volume hosts using LVM backend
This is needed to create the iscsi initiator ID which may
not be present by default.

Change-Id: I7a1931c51595e2f6ee1c5794230737d68740c049
2024-06-12 15:55:13 +01:00
Dmitriy Rabotyagov
0ce2bc01fd Define unique hostname for QManager
Due to the shortcoming of QManager implementation [1], in case of uWSGI
usage on metal hosts, the flow ends up with having the same
hostname/processname set, making services to fight over same file
under SHM.

In order to avoid this, we prepend the hostname with a service_name.
We can not change processname instead, since it will lead to the fight
between different processes of the same service.

[1] https://bugs.launchpad.net/oslo.messaging/+bug/2065922

Change-Id: I19484946c3049484f27e6e6029afd8152e1b835d
2024-05-25 13:22:46 +02:00
Zuul
af966b7354 Merge "Allow to skip volume type management by role" 2024-05-20 16:11:59 +00:00
Dmitriy Rabotyagov
ddddfde34d Allow to skip volume type management by role
Right now it is not possible to avoid creation of a default volume type
which will have the same name as a backend. In case operator want to
rename volume type or have a different name - it will not be possible
as of today.

While we should migrate to using openstack_resources role for managing
of QoS and volume_types [1], the patch/modules are not ready and will
not be backportable anyway.

[1] https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/906372

Change-Id: Ic1e190fc86bd3b6fd2cc771273e28b5bb095a322
2024-05-20 11:08:02 +02:00
Andrew Bonney
6a649aea8c Add tag to enable targeting of post-install config elements only
<service>-config tags are quite broad and have a long execution
time. Where you only need to modify a service's '.conf' file and
similar it is useful to have a quicker method to do so.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/919816
Change-Id: I6f4d5f3388b71ce874650558b2566f795361a6c7
2024-05-16 15:33:36 +00:00
Zuul
58b9deb05c Merge "reno: Update master for unmaintained/zed" 2024-05-14 14:44:20 +00:00
Dmitriy Rabotyagov
62f3ce0efc Implement variables to address oslo.messaging improvements
During last release cycle oslo.messaging has landed [1] series of extremely
useful changes that are designed to implement modern messaging
techniques for rabbitmq quorum queues.

Since these changes are breaking and require queues being re-created,
it makes total sense to align these with migration to quorum queues by default.

[1] https://review.opendev.org/q/topic:%22bug-2031497%22

Change-Id: I02db2ad91ad036ff24c35882e1746b3d6a8f3cc0
2024-05-13 07:21:38 +00:00
57c13c1154 reno: Update master for unmaintained/zed
Update the zed release notes configuration to build from
unmaintained/zed.

Change-Id: I2caa6b2f34725d9d62f722984df8cb7da7dee86f
2024-05-10 18:54:34 +00:00
Dmitriy Rabotyagov
21f8ede972 Add variable to globally control notifications enablement
In order to be able to globally enable notification reporting for all services,
without an need to have ceilometer deployed or bunch of overrides for each
service, we add `oslomsg_notify_enabled` variable that aims to control
behaviour of enabled notifications.

Presence of ceilometer is still respected by default and being referenced.

Potential usecase are various billing panels that do rely on notifications
but do not require presence of Ceilometer.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/914144
Change-Id: Ifb16270b41eb795dca9b4bd6a1a3e6eabf2f45eb
2024-03-25 18:23:55 +00:00
Dmitriy Rabotyagov
5ad37dd620 Add service policies defenition
In order to allow definition of policies per service, we need to add variables
so service roles, that will be passed to openstack.osa.mq_setup.

Currently this can be handled by leveraging group_vars and overriding `oslomsg_rpc_policies` as a whole, but it's not obvious and
can be non-trivial for some groups which are co-locating multiple services
or in case of metal deployments.

Change-Id: I69fee673ebc1fa454a4af0f9a251e9f50999edd8
2024-03-25 14:08:20 +01:00
Dmitriy Rabotyagov
7e2cf5283f Restart cinder-purge-deleted service only on abnormal exit
Default value of Restart for any service which type is not `oneshot` is
`on-failure`. While this suits most usecases, this leads to unexpected
consequences for cinder-purge-deleted.service.

In case there're some historical inconsistencies in the database which
make impossible to flush deleted volumes from the database
(ie due to prior manual intervention), cinder-manage exists with code 1
which triggers systemd to restart the service and attempt cleanup again.

The troublesome part is the transactional behaviour of the script. With
each run it locks records in it's transaction that is failing and being
reverted in a loops with 2 sec delay, that not only causes unnecessary
load for database itself, but also causes deadlocks during operations
with volumes that are not being re-tryed and fail with 500 return code
in cinder-api.

Changing Restart to `on-abnormal` will leave service in a failed state
and systemd won't attempt to restart it.

Change-Id: Ib091cc11a16fcd31ef351d9ec21d070d25829791
2023-11-08 09:01:19 +00:00
Dmitriy Rabotyagov
b9b06052cb Fix example playbook linters
Change-Id: If0873fd38d40cb2317e644e20f5be5334753d79a
2023-11-08 10:00:57 +01:00
Dmitriy Rabotyagov
79ccde30b4 Split lines to not exceed 160 characters limit
Change-Id: I1e35c59edff17f19916258eee2a8b0a8bc77958f
2023-09-18 16:17:05 +02:00
Zuul
24648387cc Merge "Add quorum queues support for service" 2023-09-04 12:43:30 +00:00
Zuul
ae713f9c8c Merge "Use proper galera port in configuration" 2023-08-14 11:02:01 +00:00
Zuul
5f6e86a7da Merge "Fix linters and metadata" 2023-08-14 10:48:17 +00:00
Dmitriy Rabotyagov
5683c693c1 Use proper galera port in configuration
While <servuce>_galera_port is defined and used for db_setup
role, it's not in fact used in a connection string for oslo.db.

Change-Id: I6b910817ddc6eab68f815f776faeee432e55012e
2023-07-31 14:27:51 +02:00
Dmitriy Rabotyagov
8413941eaa Fix linters and metadata
With update of ansible-lint to version >=6.0.0 a lot of new
linters were added, that enabled by default. In order to comply
with linter rules we're applying changes to the role.

With that we also update metdata to reflect current state.

Depends-On: https://review.opendev.org/c/openstack/ansible-role-systemd_service/+/888223
Change-Id: I671cc35a055b35fb249ad3054c45ec65f2b54ab4
2023-07-14 12:58:19 +00:00
Simon Hensel
3c014fb2b6 Reduce memory consumption in Cinder services
This patch reduces memory usage for Cinder Volume and Backup services by
tuning glibc.

The specific tuning consist on disabling the per thread arenas and
disabling dynamic thresholds.

This is the equivalent of the devstack proposed patch from Change-Id
Ic9030d01468b3189350f83b04a8d1d346c489d3c

Related-bug: #1908805
Change-Id: I066ee76fe0cef9443f9e9f1ed3c8062d6c6f8566
2023-07-10 11:35:06 +02:00
Dmitriy Rabotyagov
af229369fb Add quorum queues support for service
This change implements and enables by default quorum support
for rabbitmq as well as providing default variables to globally tune
it's behaviour.

In order to ensure upgrade path and ability to switch back to HA queues
we change vhost names with removing leading `/`, as enabling quorum
requires to remove exchange which is tricky thing to do with running
services.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/875399
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/873618
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-os_glance/+/873632
Change-Id: I9e1f87fd2c396eb4b48459c3055b43678fae296a
2023-07-06 12:56:08 +00:00
Jonathan Rosser
2820fcc1d6 Use v3 service type in keystone_authtoken config
The service type in this config section must match the service
type in the service catalog, otherwise limited scope application
credentials will not work with the cinder API [1].

[1] https://docs.openstack.org/keystone/2023.1/user/application_credentials.html#access-rules

Change-Id: I711241af8f7520b97f2b1cafd1406ff705fb78a6
2023-06-20 15:11:20 +01:00
Dmitriy Rabotyagov
1af3003e16 Define service_user for cinder services
In order to cover OSSA-2023-003, a requirement to define service_user
section for all cinder services has been added by cinder.

Change-Id: I19c2b03c61f714fedb593da8489e50d3fa08d933
2023-05-22 15:57:41 +02:00
Dmitriy Rabotyagov
f16036e1f1 Add way to periodically trim Cinder DB
We're adding a service that is responsible for executing db purge.
Service will be deployed by default, but left
stopped/disabled. This way we allow deployers to enable/disable
feature by changing value of cinder_purge_deleted.

Otherwise, when variables set to true once, setting them back to false
won't lead to stopping of DB trimming , so timer would need to be
stopped manually.

Change-Id: Ic5ae8c778bff2858fcb31c85d4b910805e452c3f
2023-05-18 08:10:21 +00:00
Zuul
1732bff64b Merge "Add TLS support to cinder backends" 2023-05-03 11:55:46 +00:00
Damian Dabrowski
2de5a8a1a0 Add TLS support to cinder backends
By overriding the variable `cinder_backend_ssl: True` HTTPS will
be enabled, disabling HTTP support on the cinder backend api.

The ansible-role-pki is used to generate the required TLS
certificates if this functionality is enabled.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/879085
Change-Id: Ib682499e900071db38cc2fd7c30822d0c33dba38
2023-04-29 18:36:09 +02:00
Dmitriy Rabotyagov
7e076b39a4 Move online data migrations to post-restart step
It's supposed that online migrations are executed once services are
upgraded and restarted after upgrade. Eventually, you can run
online migrations before the next upgrade according to the doc [1]

So we move that to a separate file that is executed after all services
are upgraded and handlers are flushed. Tasks are delegated to API hosts
and we clean up facts for them as well.

[1] https://docs.openstack.org/cinder/latest/admin/upgrades.html#database-upgrades

Change-Id: Ic3ecdddd7dcc2dd617c8606278590c8e59230fdf
2023-04-12 19:09:30 +02:00
Dmitriy Rabotyagov
92aba932b1 Ensure service is restarted on unit file changes
At the moment we don't restart services if systemd unit file is changed.

We knowingly prevent systemd_service role handlers to execute
by providing `state: started` as otherwise service will be restarted twice.
With that now  we ensure that role handlers will also listen for systemd
unit changes.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/879671
Change-Id: I8140add1a4e4fdacee89bd29bd2e3c87eff0953a
2023-04-07 09:53:38 +00:00
Dmitriy Rabotyagov
369f01589c Remove rsync requirement for cinder
We used rsync to synchronize filters from rootwrap.d. However, with
smart-source that is not needed anymore, since /etc/cinder is simply
a symlink to the source directory of rsync. We still need os-brick
rootwrap linkage though.

Change-Id: Ib1571c5be67155b584c412da8336de49bc80d948
2023-01-10 15:09:56 +01:00
492e519b98 Update master for stable/zed
Add file to the reno documentation build to show release notes for
stable/zed.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/zed.

Sem-Ver: feature
Change-Id: Id4cda2eb6ffdb55a80e555b33b1cae9ee4c5f67c
2022-12-13 13:12:58 +00:00
Zuul
3bc15b4369 Merge "Add coordination support" 2022-12-08 16:17:52 +00:00
Dmitriy Rabotyagov
bfb256d49b Define local facts separately only for distro
We do define local facts locally using python_venv_build role so no need
to do the same as a separate task for source installs. Though these
facts are still needed for distro path.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/862924
Change-Id: I5e4717a780a20914bba345bc0e0e275d6ee7b81f
Needed-By: https://review.opendev.org/c/openstack/openstack-ansible/+/866126
2022-12-06 12:07:51 +00:00
Dmitriy Rabotyagov
789d14de9d Add coordination support
This patch adds configuration for coordination service when cinder
active/active setup is used and coordination hosts exist

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/864750
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/865805
Change-Id: I2dda4c74aa69aea3ecea92954922dbfe1bd56687
2022-11-28 09:05:04 +00:00
Zuul
dd9051ed36 Merge "Replace git-core with git for debian" 2022-10-07 10:12:41 +00:00
Dmitriy Rabotyagov
0ebed2408b Replace git-core with git for debian
With ansible-core 2.13 it tries to substitude package resolution in apt
module.
However git-core is used in Debian as transitional name, but ansible
tries to select it and provide version, which is not correct behaviour.
But since git-core is not really valid anyway, we just replace it
to workaround ansible's imperfectness.

Change-Id: Ib0a75886baffec27c8a7d38d729623c7b41216eb
2022-10-05 10:51:20 +02:00
Dmitriy Rabotyagov
0d4b994049 Add nfs and ceph jobs to templates
With changing cinder code we potentially can break some backends.
In order to detect this in time we are adding ceph and nfs scenarios.

We also fix lvm backend for use on RedHat.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/859339
Change-Id: Ifceb2b816199339ec7725bd95cc890595eed95d9
2022-10-04 12:56:58 +02:00
Erik Berg
13bf575d23 Remove redundant vars line
This line was introduced by I21f84809c44ac4be0165fadfb8da67bbcbc9b05c
for centos-7 support, and should already be covered by the
distribution_major_version line above.

Change-Id: I5d5f84b84de35763024709212e0673607127e264
2022-09-14 13:03:52 +02:00
Jimmy McCrory
75d7ae9093 Remove oslo_policy section from cinder.conf
policy.json was deprecated.
Remove the oslo_policy configuration which was still using it.

Change-Id: I0215bb17219745ab2c838a1999caf0a7baa4242d
2022-08-08 23:21:06 -07:00
Zuul
5fadeff98b Merge "Support service tokens" 2022-06-16 12:28:01 +00:00
Zuul
c07280e339 Merge "Forcefully remove Cinder v2 endpoints" 2022-06-16 09:08:19 +00:00
Dmitriy Rabotyagov
f3b2abb887 Remove mention of haproxy-endpoints role
Role was never migrated to usage of haproxy-endpoints role
and included task was used instead the whole time.
With that to reduce complexity and to have unified approach, all mention
of the role and handler are removed from the code.

Change-Id: I0c055393ccb1c8d61affc2c1bb6d01f0c329afe9
2022-06-14 18:02:35 +00:00
Dmitriy Rabotyagov
f755eadadf Support service tokens
Implement support for service_tokens. For that we convert
role_name to be a list along with renaming corresponding variable.

Additionally service_type is defined now for keystone_authtoken which
enables to validate tokens with restricted access rules

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/845690
Change-Id: I1d0156a2ad829aa730419e1d9dfa1cd49026a6be
Related-Bug: #1948456
2022-06-14 11:30:31 +02:00
Dmitriy Rabotyagov
9a8ea212c0 Forcefully remove Cinder v2 endpoints
Nowadays Cinder does not support v2 api so it makes sense to ensure
that these endpoints or service is not present in catalog.

Change-Id: I62a4ba182cc752a5bc4f6e8c4d2430f7e7aafe54
2022-06-14 11:16:07 +02:00