Commit Graph

288 Commits

Author SHA1 Message Date
Dmitriy Rabotyagov 7e2cf5283f Restart cinder-purge-deleted service only on abnormal exit
Default value of Restart for any service which type is not `oneshot` is
`on-failure`. While this suits most usecases, this leads to unexpected
consequences for cinder-purge-deleted.service.

In case there're some historical inconsistencies in the database which
make impossible to flush deleted volumes from the database
(ie due to prior manual intervention), cinder-manage exists with code 1
which triggers systemd to restart the service and attempt cleanup again.

The troublesome part is the transactional behaviour of the script. With
each run it locks records in it's transaction that is failing and being
reverted in a loops with 2 sec delay, that not only causes unnecessary
load for database itself, but also causes deadlocks during operations
with volumes that are not being re-tryed and fail with 500 return code
in cinder-api.

Changing Restart to `on-abnormal` will leave service in a failed state
and systemd won't attempt to restart it.

Change-Id: Ib091cc11a16fcd31ef351d9ec21d070d25829791
2023-11-08 09:01:19 +00:00
Dmitriy Rabotyagov 8413941eaa Fix linters and metadata
With update of ansible-lint to version >=6.0.0 a lot of new
linters were added, that enabled by default. In order to comply
with linter rules we're applying changes to the role.

With that we also update metdata to reflect current state.

Depends-On: https://review.opendev.org/c/openstack/ansible-role-systemd_service/+/888223
Change-Id: I671cc35a055b35fb249ad3054c45ec65f2b54ab4
2023-07-14 12:58:19 +00:00
Dmitriy Rabotyagov f16036e1f1 Add way to periodically trim Cinder DB
We're adding a service that is responsible for executing db purge.
Service will be deployed by default, but left
stopped/disabled. This way we allow deployers to enable/disable
feature by changing value of cinder_purge_deleted.

Otherwise, when variables set to true once, setting them back to false
won't lead to stopping of DB trimming , so timer would need to be
stopped manually.

Change-Id: Ic5ae8c778bff2858fcb31c85d4b910805e452c3f
2023-05-18 08:10:21 +00:00
Zuul 1732bff64b Merge "Add TLS support to cinder backends" 2023-05-03 11:55:46 +00:00
Damian Dabrowski 2de5a8a1a0 Add TLS support to cinder backends
By overriding the variable `cinder_backend_ssl: True` HTTPS will
be enabled, disabling HTTP support on the cinder backend api.

The ansible-role-pki is used to generate the required TLS
certificates if this functionality is enabled.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/879085
Change-Id: Ib682499e900071db38cc2fd7c30822d0c33dba38
2023-04-29 18:36:09 +02:00
Dmitriy Rabotyagov 7e076b39a4 Move online data migrations to post-restart step
It's supposed that online migrations are executed once services are
upgraded and restarted after upgrade. Eventually, you can run
online migrations before the next upgrade according to the doc [1]

So we move that to a separate file that is executed after all services
are upgraded and handlers are flushed. Tasks are delegated to API hosts
and we clean up facts for them as well.

[1] https://docs.openstack.org/cinder/latest/admin/upgrades.html#database-upgrades

Change-Id: Ic3ecdddd7dcc2dd617c8606278590c8e59230fdf
2023-04-12 19:09:30 +02:00
Dmitriy Rabotyagov 92aba932b1 Ensure service is restarted on unit file changes
At the moment we don't restart services if systemd unit file is changed.

We knowingly prevent systemd_service role handlers to execute
by providing `state: started` as otherwise service will be restarted twice.
With that now  we ensure that role handlers will also listen for systemd
unit changes.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/879671
Change-Id: I8140add1a4e4fdacee89bd29bd2e3c87eff0953a
2023-04-07 09:53:38 +00:00
Dmitriy Rabotyagov 369f01589c Remove rsync requirement for cinder
We used rsync to synchronize filters from rootwrap.d. However, with
smart-source that is not needed anymore, since /etc/cinder is simply
a symlink to the source directory of rsync. We still need os-brick
rootwrap linkage though.

Change-Id: Ib1571c5be67155b584c412da8336de49bc80d948
2023-01-10 15:09:56 +01:00
Dmitriy Rabotyagov bfb256d49b Define local facts separately only for distro
We do define local facts locally using python_venv_build role so no need
to do the same as a separate task for source installs. Though these
facts are still needed for distro path.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/862924
Change-Id: I5e4717a780a20914bba345bc0e0e275d6ee7b81f
Needed-By: https://review.opendev.org/c/openstack/openstack-ansible/+/866126
2022-12-06 12:07:51 +00:00
Dmitriy Rabotyagov 0d4b994049 Add nfs and ceph jobs to templates
With changing cinder code we potentially can break some backends.
In order to detect this in time we are adding ceph and nfs scenarios.

We also fix lvm backend for use on RedHat.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/859339
Change-Id: Ifceb2b816199339ec7725bd95cc890595eed95d9
2022-10-04 12:56:58 +02:00
Erik Berg 13bf575d23 Remove redundant vars line
This line was introduced by I21f84809c44ac4be0165fadfb8da67bbcbc9b05c
for centos-7 support, and should already be covered by the
distribution_major_version line above.

Change-Id: I5d5f84b84de35763024709212e0673607127e264
2022-09-14 13:03:52 +02:00
Zuul 5fadeff98b Merge "Support service tokens" 2022-06-16 12:28:01 +00:00
Dmitriy Rabotyagov f3b2abb887 Remove mention of haproxy-endpoints role
Role was never migrated to usage of haproxy-endpoints role
and included task was used instead the whole time.
With that to reduce complexity and to have unified approach, all mention
of the role and handler are removed from the code.

Change-Id: I0c055393ccb1c8d61affc2c1bb6d01f0c329afe9
2022-06-14 18:02:35 +00:00
Dmitriy Rabotyagov f755eadadf Support service tokens
Implement support for service_tokens. For that we convert
role_name to be a list along with renaming corresponding variable.

Additionally service_type is defined now for keystone_authtoken which
enables to validate tokens with restricted access rules

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/845690
Change-Id: I1d0156a2ad829aa730419e1d9dfa1cd49026a6be
Related-Bug: #1948456
2022-06-14 11:30:31 +02:00
Zuul 9c170cf1f5 Merge "Setup db/mq/service during scheduler run" 2022-04-26 13:09:42 +00:00
Zuul 252010bb8a Merge "Allow to run cinder role in check mode" 2022-04-23 14:52:37 +00:00
Dmitriy Rabotyagov 4aad247b20 Allow to run cinder role in check mode
Always fetch config files as it's treated as safe operation.
On top of that handle case when there's no local config specified.

Change-Id: I5f5f36da96672679d6801c2a52c58d86657ca612
2022-04-14 18:05:55 +02:00
siavash sardari 66eb424de5 Add the ability to disable send_actions option in cinder-volume-usage-audit service.
Change-Id: I351279dab0d90e60573df7db4c80b63320dff787
Closes-Bug: #1968734
2022-04-13 16:06:34 +04:30
Zuul 9f2bf29db8 Merge "Use common service setup tasks from a collection rather than in-role" 2022-01-13 13:01:23 +00:00
Dmitriy Rabotyagov 1d8627b360 Setup db/mq/service during scheduler run
According to our playbook, role runs against api last, after scheduler,
volume and backup services are already setup.

It makes difference only when cinder-scheduler and cinder-api are
deployed to different targets.

Change-Id: I18f68b5cb9dd60d9cf72850e840d0459e1245b76
2022-01-13 14:05:35 +02:00
Jonathan Rosser ef8fa2b358 Use common service setup tasks from a collection rather than in-role
Change-Id: I4ffa85e918e515dde2fcae86ff5cf872e65a82c4
2022-01-12 18:30:02 +00:00
Zuul e9531f6241 Merge "Refactor use of include_vars" 2022-01-12 14:59:00 +00:00
Jonathan Rosser 5c7fa97a00 Refactor use of include_vars
Use a first_found lookup instead of a with_first_found loop so that
the 'paths' parameter can be used.

This ensures that only vars from the role are included, and not vars
from a parent calling role. This can happen when a parent role has
a higher priority vars file available for inclusion than the role
it calls.

Change-Id: I5c50529b5e73bac6094c203d49a32497c7a388c3
2022-01-12 08:38:29 +00:00
Damian Dabrowski b0f7154ffe Enable recursion in combine() filter
Ansible's combine() filter needs recursive=True parameter in order to recursively merge nested hashes.

https: //docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html#combining-hashes-dictionaries
Change-Id: I6d5362afe2c52ae08f60ac877ee3d58da3388da6
2022-01-11 19:23:36 +01:00
OpenStack Proposal Bot 8732c718fd Updated from OpenStack Ansible Tests
Change-Id: Ib6e646678a102b4b86f0ff06304d7d767aea1921
2021-12-04 17:39:44 +00:00
Zuul 53a3fa0672 Merge "Use config_template as a collection" 2021-12-01 19:23:59 +00:00
Dmitriy Rabotyagov 6eb79976a4 Use config_template as a collection
Since we still use ceph-ansible that has their own implementation of
config_template module it's worth to use mentioned module as a collection
explicitly.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/819814



Change-Id: Id31fde6375ab5ebf90e1f13b11f80d43773e4c54
2021-11-30 15:17:14 +02:00
Dmitriy Rabotyagov 3370ad8e03 Refactor definition of lock path
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/819300
Depends-On: https://review.opendev.org/c/openstack/ansible-role-systemd_service/+/819298
Change-Id: I674aa11ecfec2a2bc7bbc84865f9ec5f4a872ebe
2021-11-25 12:13:31 +00:00
OpenStack Proposal Bot 499f6e2862 Updated from OpenStack Ansible Tests
Change-Id: I1243293a0cc9c434365f70342887e29c960db68c
2021-04-19 09:57:12 +00:00
Zuul 67cbe597ac Merge "[goal] Deprecate the JSON formatted policy file" 2021-04-06 16:44:16 +00:00
Dmitriy Rabotyagov 6b5a93edb9 [goal] Deprecate the JSON formatted policy file
As per the community goal of migrating the policy file
the format from JSON to YAML[1], we need to replace policy.json to
policy.yaml and remove deprecated policy.json.

[1]https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html

Change-Id: Iede5a9f0387baaf9c72bd060b59435ffaa76f1bd
2021-03-30 16:43:53 +00:00
Dmitriy Rabotyagov 91fef6b5cc Fix condition when to create backends
Backend creation should happen on the last host in the group, instead
of the first one, since we put LB in MAINT state during deployment, while
other containers are not ready yet. Thus we should be creating backends
at the very end of the deployment, when all containers are set.

Change-Id: I3d5811e059d1b5cb3e87fe7657872e41105c832e
Closes-Bug: #1920964
2021-03-25 09:59:42 +02:00
Jonathan Rosser 13ac3d38ee Remove references to unsupported operating systems
All references to Gentoo, SUSE, Debian stretch and Centos-7  are removed.
Conditional tasks, ternary operators and variables are simplified where possible
OS specific variables files are generalised where possible

Change-Id: I796d9de164a8b79eff8a615442dd46e7de2353e7
2021-03-10 12:16:37 +00:00
Jonathan Rosser a4653722ed Use ansible_facts[] instead of fact variables
See https://github.com/ansible/ansible/issues/73654

Change-Id: I299e71fac6efdb89655bd36de72b95400d7000aa
2021-02-23 18:04:29 +00:00
Dmitriy Rabotyagov 6fd2a1a255 Fix cert verification logic for cinder api
Change-Id: I8cf74ee5d1157357999059227ae9f805fc6fa4de
Closes-Bug: #1914602
2021-02-11 10:55:37 +02:00
Dennis M 39d19059b4 Added tags for policy override task
Policy override task without tags requires a lot of time to just
change policy.json file. cinder-config tag wasn't included either, so
it was added to this task as well. In my system new tag requires 10
times less time than cinder-config, so additional tag is reasonable.

Change-Id: I343dc39f2276e313b4142e6cd494f195e4514ce1
2021-01-12 11:14:38 +02:00
Zuul 9e68a77e69 Merge "Updated from OpenStack Ansible Tests" 2021-01-11 13:37:29 +00:00
Dmitriy Rabotyagov d9bfa963e8 Use global service variables
Instead of overriding each service separatelly it might make
sense for deployers to define some higher level variable that
will be used first or fallback to default variable.

Change-Id: If14b89d4c795ba7e129af6a2f4b4bcbc10208986
2021-01-05 19:09:06 +02:00
OpenStack Proposal Bot 33d4b6464b Updated from OpenStack Ansible Tests
Change-Id: I7cf476e3a7faef1ab66eb84d4117d8c6471e6b95
2021-01-05 08:35:59 +00:00
Dmitriy Rabotyagov 2f005e7c49 Do not symlink ceph libraries for distro path
ceph_client role checks length of the `openstack_service_venv_bin` variable
to determine if libraries symlinking into venv should occur. So for
distro path this should be empty string as no symlinking should be done.

Change-Id: Ie5073c1b178109b4daa95be7bec50f9648260eb2
2020-10-30 17:35:36 +02:00
Zuul d2a1b0eac6 Merge "Trigger uwsgi restart" 2020-10-12 09:10:29 +00:00
Dmitriy Rabotyagov 12224d8681 Trigger uwsgi restart
When we were migrating service to uwsgi usage, we clean forgot to
trigger uwsgi restart on service config change.

Change-Id: Ie1f386f21b955dcc1d2450efa9391c56d05d5695
2020-10-08 17:20:37 +03:00
Dmitriy Rabotyagov b5a4bcafef Remove unecessary apache tasks
These tasks were added for upgrade purposes to uwsgi
and can be safely removed now

Change-Id: I2c17d78933b1c0e72132648623728af39a7f7aa7
2020-10-08 15:15:03 +03:00
Zuul 586c96a17c Merge "Updated from OpenStack Ansible Tests" 2020-10-05 15:10:20 +00:00
Jonathan Rosser ffd9f77d40 Fix linter errors
Change-Id: I44a2c3156db09005608cf15069487eff88dc24c5
2020-10-01 16:59:53 +01:00
OpenStack Proposal Bot 1f4c000182 Updated from OpenStack Ansible Tests
Change-Id: I04e69e86fb8f21d00a4fb60a0b8fab0271f2ed28
2020-10-01 14:29:46 +00:00
Jonathan Rosser 6e7e2fa26c Revert use of _cinder_first_play_host for post install tasks
This task runs against hosts which will fail the group membership
test added in [1] causing the play to fail completely.

[1] https://review.opendev.org/#/c/751260/3/tasks/cinder_post_install.yml

Change-Id: I0919925fe0b518e1370cf087e00d114f0d40d073
2020-09-21 20:22:00 +01:00
Zuul dbb1aaf434 Merge "Use cinder_service_setup_host for qos and types creation" 2020-09-21 08:55:51 +00:00
Zuul 6dd71d2b0e Merge "Define condition for the first play host one time" 2020-09-21 08:49:45 +00:00
Dmitriy Rabotyagov 7f24f8d34a Define condition for the first play host one time
We use the same condition, which defines against what host some "service"
tasks should run against, several times. It's hard to keep it the same
across the role and ansible spending additional resources to evaluate
it each time, so it's simpler and better for the maintenance to set
a boolean variable which will say for all tasks, that we want to run
only against signle host, if they should run or not now.

Change-Id: Ibdb078d39b3189c844aed14aa1ae74cb2ce97600
2020-09-16 11:54:52 +00:00