Commit Graph

75 Commits

Author SHA1 Message Date
Dmitriy Rabotyagov 79ccde30b4 Split lines to not exceed 160 characters limit
Change-Id: I1e35c59edff17f19916258eee2a8b0a8bc77958f
2023-09-18 16:17:05 +02:00
Zuul 24648387cc Merge "Add quorum queues support for service" 2023-09-04 12:43:30 +00:00
Dmitriy Rabotyagov 8413941eaa Fix linters and metadata
With update of ansible-lint to version >=6.0.0 a lot of new
linters were added, that enabled by default. In order to comply
with linter rules we're applying changes to the role.

With that we also update metdata to reflect current state.

Depends-On: https://review.opendev.org/c/openstack/ansible-role-systemd_service/+/888223
Change-Id: I671cc35a055b35fb249ad3054c45ec65f2b54ab4
2023-07-14 12:58:19 +00:00
Dmitriy Rabotyagov af229369fb Add quorum queues support for service
This change implements and enables by default quorum support
for rabbitmq as well as providing default variables to globally tune
it's behaviour.

In order to ensure upgrade path and ability to switch back to HA queues
we change vhost names with removing leading `/`, as enabling quorum
requires to remove exchange which is tricky thing to do with running
services.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/875399
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/873618
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-os_glance/+/873632
Change-Id: I9e1f87fd2c396eb4b48459c3055b43678fae296a
2023-07-06 12:56:08 +00:00
Dmitriy Rabotyagov 92aba932b1 Ensure service is restarted on unit file changes
At the moment we don't restart services if systemd unit file is changed.

We knowingly prevent systemd_service role handlers to execute
by providing `state: started` as otherwise service will be restarted twice.
With that now  we ensure that role handlers will also listen for systemd
unit changes.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/879671
Change-Id: I8140add1a4e4fdacee89bd29bd2e3c87eff0953a
2023-04-07 09:53:38 +00:00
Dmitriy Rabotyagov 369f01589c Remove rsync requirement for cinder
We used rsync to synchronize filters from rootwrap.d. However, with
smart-source that is not needed anymore, since /etc/cinder is simply
a symlink to the source directory of rsync. We still need os-brick
rootwrap linkage though.

Change-Id: Ib1571c5be67155b584c412da8336de49bc80d948
2023-01-10 15:09:56 +01:00
Dmitriy Rabotyagov 789d14de9d Add coordination support
This patch adds configuration for coordination service when cinder
active/active setup is used and coordination hosts exist

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/864750
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/865805
Change-Id: I2dda4c74aa69aea3ecea92954922dbfe1bd56687
2022-11-28 09:05:04 +00:00
Zuul dd9051ed36 Merge "Replace git-core with git for debian" 2022-10-07 10:12:41 +00:00
Dmitriy Rabotyagov 0ebed2408b Replace git-core with git for debian
With ansible-core 2.13 it tries to substitude package resolution in apt
module.
However git-core is used in Debian as transitional name, but ansible
tries to select it and provide version, which is not correct behaviour.
But since git-core is not really valid anyway, we just replace it
to workaround ansible's imperfectness.

Change-Id: Ib0a75886baffec27c8a7d38d729623c7b41216eb
2022-10-05 10:51:20 +02:00
Dmitriy Rabotyagov 0d4b994049 Add nfs and ceph jobs to templates
With changing cinder code we potentially can break some backends.
In order to detect this in time we are adding ceph and nfs scenarios.

We also fix lvm backend for use on RedHat.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/859339
Change-Id: Ifceb2b816199339ec7725bd95cc890595eed95d9
2022-10-04 12:56:58 +02:00
Dmitriy Rabotyagov 9a8ea212c0 Forcefully remove Cinder v2 endpoints
Nowadays Cinder does not support v2 api so it makes sense to ensure
that these endpoints or service is not present in catalog.

Change-Id: I62a4ba182cc752a5bc4f6e8c4d2430f7e7aafe54
2022-06-14 11:16:07 +02:00
Zuul 9c170cf1f5 Merge "Setup db/mq/service during scheduler run" 2022-04-26 13:09:42 +00:00
Andrew Bonney 6efa45e2bd Add configuration option for heartbeat_in_pthread
This configuration option has been observed to result in file
descriptor leaks in certain circumstances. A variable is added
here so that it can be easily overridden.

Related-Bug: #1961603
Change-Id: I8155264b181d6f21728804ef8260979931597427
2022-03-15 10:52:22 +00:00
Dmitriy Rabotyagov 1d8627b360 Setup db/mq/service during scheduler run
According to our playbook, role runs against api last, after scheduler,
volume and backup services are already setup.

It makes difference only when cinder-scheduler and cinder-api are
deployed to different targets.

Change-Id: I18f68b5cb9dd60d9cf72850e840d0459e1245b76
2022-01-13 14:05:35 +02:00
Dmitriy Rabotyagov d120669a7d Remove obsoleted packages
We drop py27 library as it's deprecated and should not be used anywhere

zlibc is also not maintained for a while and has been removed in
bullseye. But it should not be required by other distros/packages anyway

Change-Id: I5835f538634900d210e236b49fb64271d43e9861
2021-06-01 16:59:38 +03:00
Dmitriy Rabotyagov 91fef6b5cc Fix condition when to create backends
Backend creation should happen on the last host in the group, instead
of the first one, since we put LB in MAINT state during deployment, while
other containers are not ready yet. Thus we should be creating backends
at the very end of the deployment, when all containers are set.

Change-Id: I3d5811e059d1b5cb3e87fe7657872e41105c832e
Closes-Bug: #1920964
2021-03-25 09:59:42 +02:00
Jonathan Rosser 13ac3d38ee Remove references to unsupported operating systems
All references to Gentoo, SUSE, Debian stretch and Centos-7  are removed.
Conditional tasks, ternary operators and variables are simplified where possible
OS specific variables files are generalised where possible

Change-Id: I796d9de164a8b79eff8a615442dd46e7de2353e7
2021-03-10 12:16:37 +00:00
Jonathan Rosser a4653722ed Use ansible_facts[] instead of fact variables
See https://github.com/ansible/ansible/issues/73654

Change-Id: I299e71fac6efdb89655bd36de72b95400d7000aa
2021-02-23 18:04:29 +00:00
Jonathan Rosser 4df6d5324d Add rsync to required packages for redhat based OS
This is requried by one of the cinder role ansible tasks so we should
ensure that is it installed

Change-Id: Iee48947d95a8748748dcce6e856e756afbc6d4f2
2020-10-08 08:44:55 +01:00
Zuul dbb1aaf434 Merge "Use cinder_service_setup_host for qos and types creation" 2020-09-21 08:55:51 +00:00
Dmitriy Rabotyagov 7f24f8d34a Define condition for the first play host one time
We use the same condition, which defines against what host some "service"
tasks should run against, several times. It's hard to keep it the same
across the role and ansible spending additional resources to evaluate
it each time, so it's simpler and better for the maintenance to set
a boolean variable which will say for all tasks, that we want to run
only against signle host, if they should run or not now.

Change-Id: Ibdb078d39b3189c844aed14aa1ae74cb2ce97600
2020-09-16 11:54:52 +00:00
Dmitriy Rabotyagov e89b864065 Use cinder_service_setup_host for qos and types creation
Cinder api container may not have required binaries and clients in order
to add qos types and volume types. In the meanwhile
cinder_service_setup_host is supposed to have all necessary things for
the successful creation of the resources.

Change-Id: I97b50cfe3140d0a0f77f6e562cb144cb3d36bd12
2020-09-10 08:58:40 +00:00
Dmitriy Rabotyagov 983ee0aef7 Add Centos-8 support
Change-Id: I2b5ae7b79f9982618a4c3a081f978fc2d42b6a7d
2020-06-16 20:03:14 +03:00
Jonathan Rosser 197e5d77f5 Update ubuntu distro packages for python3
UCA for Train only supports python3

Change-Id: Ie0fe9b1f572daa5b396ca01609f3ba1ad8906424
2019-10-28 16:02:40 +00:00
Ralf Haferkamp da93dc99e8 Fix distro install on openSUSE
With Train openSUSE is switching the openstack packages to use Python 3.
This means that we need to use the python3 variants of python-module
packages.

Depends-On: https://review.opendev.org/683123
Change-Id: Ib51d7c1a119f6d2890d94d1fcb3e7d4b8d76fc91
2019-09-19 18:00:20 +02:00
Zuul 73ad9ab751 Merge "Start using uWSGI role" 2019-09-06 15:40:44 +00:00
Dmitriy Rabotyagov ee018d9083 Start using uWSGI role
Move service to use uWSGI role instead of iternal task for uwsgi
deployment. This aims to ease the maintenance of uWSGI and speedup
metal deployments as the same uwsgi environment will be used
across all services.

Depends-On: https://review.opendev.org/678025/
Change-Id: Ieac6d03a436f6b706d7f12e292ffc98171a43246
2019-09-04 15:34:37 +03:00
Dmitriy Rabotyagov 75d0f298c6 Rename cinder distro packages to match convention
This patch splits distro variables into two:
- cinder_*_distro_packages defines dependant packages
- cinder_*_service_distro_packages defines cinder service packages itself

So that we're installing cinder_*_distro_packages both for source and distro
installs while cinder_*_service_distro_packages are only for distro ones.

Also since only qemu-tools depends on librbd, we leave it alone in
cinder_volume_distro_packages_tools.

Change-Id: I746c800dc75b39f57f7c707aedf35f6814cfdef0
2019-09-04 13:41:28 +03:00
Zuul 58a93bf359 Merge "Install openSUSE distro packages selectively" 2019-07-09 15:14:05 +00:00
Dmitriy Rabotjagov 51f2d13c41 service_setup: refactor service setup to a single file
This patch refactors the openstack user/service/endpoints creation to
service_setup.yml which will eventually be managed by
openstack-ansible-tests.

Change-Id: Idefa676a5851346a83d613aa61cb1d618e015c29
2019-06-28 20:02:27 +01:00
Dirk Mueller b3098ca7bf Install openSUSE distro packages selectively
This patch moves the cinder distro packages into the correct variables
so that each packages is only installed where needed.

Change-Id: I3d2edfefade7e415aa201bccd2fd2be3ed7159d9
2019-06-28 14:56:59 +02:00
Jimmy McCrory d139d40a19 Cleanup debian cinder service distro package list
The debian packages for individual cinder services provide service
configs which automatically start after install, leading to all cinder
services running on each cinder volume host. Instead only install the
python3-cinder library package and rely on the service files OSA manages.

uwsgi packages are only required on hosts running cinder-api, so an
additional variable, 'cinder_api_distro_packages' has been added for
packages specific to those hosts.

Beginning in the Stein release, Ubuntu distro packages are now using
Python3. This requires additionally installing and using the uwsgi
python3 plugin.

Change-Id: Iafcd9a8141fffb2226ec5103960064decc579bd5
2019-06-05 13:51:19 -07:00
Mohammed Naser f9aa4c5a30 [redhat] Update variables to be version agnostic
The RedHat-based operating systems such as CentOS have a pretty
stable list of packages, therefore, we don't need to pin it by
version and we can instead move to a much more generic redhat.yml
which will support a bigger range of systems.

Change-Id: I833e21b1d7bf47121e75ebc07baeb374f5876a42
2019-05-10 00:25:10 -04:00
Mohammed Naser eaadfef829 debian: add support
This patch adds support for this role to be able to deploy on
Debian Stretch.

Change-Id: Id426c570dc18ac2824b5785587bceaa794324eb6
Needed-By: I9a92b73c419a0dc1cca40dacfef75de61a61db94
2019-04-05 22:57:04 -04:00
Jonathan Rosser b982a4e826 Revert "Avoid distro installing unused services"
The distro_install and service_install packages have got all
mixed up together. As a result source installs are dropping
openstack distro packages all over the place.

This reverts commit c0010010c2.

Change-Id: I485ec63d7e3f750734fb07bf97aa4cdfcecae468
2019-02-16 19:55:47 +00:00
Michael Vollman c0010010c2 Avoid distro installing unused services
When performing a distro_install avoid installing services where they
are not needed.  The services are started automatically and are
consuming resources.

Change-Id: I50608e7be402a022c6c488bba38899469dd99a2b
2019-02-14 08:47:20 -05:00
Jonathan Rosser 2e08d866e4 Ensure cinder-volumes tool packages install correct ceph dependancies
The cinder-volumes service requires a set of tools installed from
distro packages, which have dependancies on librados and librbd. This
patch moves the installation of these tools to after the point that
the ceph_client role has run. This ensures that the ceph repo config
is correct and in the case of ubuntu any necessary apt pins have
been installed. This prevents issues where packages in the distro
repo may have sematically higher version numbers that the ceph
repo and then require a package downgrade - this leads to an install
failure.

Change-Id: I9de3a3566df913487b0cbf971c4006e9e52feaac
2019-02-11 12:06:40 +00:00
Kevin Carter f3f956e904 Cleanup files and templates using smart sources
The files and templates we carry are almost always in a state of
maintenance. The upstream services are maintaining these files and
there's really no reason we need to carry duplicate copies of them. This
change removes all of the files we expect to get from the upstream
service. while the focus of this change is to remove configuration file
maintenance burdens it also allows the role to execute faster.

  * Source installs have the configuration files within the venv at
    "<<VENV_PATH>>/etc/<<SERVICE_NAME>>". The role will now link the
    default configuration path to this directory. When the service is
    upgraded the link will move to the new venv path.
  * Distro installs package all of the required configuration files.

To maintain our current capabilities to override configuration the
role will fetch files from the disk whenever an override is provided and
then push the fetched file back to the target using `config_template`.

Change-Id: Ib3447cd5b0bcada4cdf82d9e4a9fe5160299f9c3
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Signed-off-by: Kevin Carter <kevin@cloudnull.com>
2019-01-19 18:02:51 -06:00
Damian Cikowski bb8af3a75b Install network storage related packages only on volume and backup containers
It's unnecessary and even not recommended to install network storage
related packages on all containers including api and scheduler one.

This change makes open-iscsi, nfs-utils and rpcbind required just
for cinder-volume and cinder-backup because it's obligatory when using
various backends.

It's not based on checking used volume drivers because most of them
relies on iSCSI and NFS.

Depends-On: https://review.openstack.org/615869
Change-Id: If289b3e0b00a279afe3c13a628eed2bc4eacc0f7
Closes-Bug: #1704161
2018-11-10 01:43:28 +00:00
Jesse Pretorius 687fb0146a RHEL: Move zlib-devel to cinder_devel_distro_packages
This last devel package was missed in both of these
reviews:
- https://review.openstack.org/614334
- https://review.openstack.org/615443

Change-Id: Ia0c935929bf771aa720d7da7ff2875fde9159156
2018-11-05 15:22:43 +00:00
Jimmy McCrory 02fa53d9de Add libxml2 to cinder_distro_packages lists
libxml2 is required by cinder services.

Package lists were also alphabetized.

Change-Id: I8cc36f5eab6d21bee84f92b8313df6e9e0512f6f
2018-11-04 10:42:15 -08:00
Zuul 32dc5ea488 Merge "Remove unnecessary package install duplication" 2018-11-01 20:31:01 +00:00
Jesse Pretorius 4f6dc1b520 Remove unnecessary package install duplication
Currently the devel packages are installed everywhere,
but they only need to be where the wheels are built.

Also, there is already a task to install the packages
needed on the target hosts when installing - so we do
not need to give the same list to the venv install role
because they will already have been installed.

All -dev/-devel packages are moved into the devel
variable so that they are only installed for source
builds.

The distro install should not need the devel packages
at all, so the list extension is removed.

Finally, we remove the unnecessary installation of the
compiling packages because the python venv build role
already does it.

Depends-On: https://review.openstack.org/613585
Change-Id: I1d8c556c9f71cef9e84f097957682fc7b847f37c
2018-10-31 10:28:01 +00:00
Jonathan Rosser b16f9690dd Add libpython2.7 as a required package.
The cinder service requires libpython at runtime.

Change-Id: I27a838c95f12ceee105af7bf435994e005aa6ff0
2018-10-26 15:52:05 +00:00
Andy Smith 1c938d86e6 Setup oslo.messaging extra packages for optional drivers
Change-Id: I4f8ac2366dabddac78715da03e82080b0bb1fa73
2018-09-18 14:59:21 -04:00
Jesse Pretorius 017f1d5100 Remove python-keystoneclient package install
Now that we no longer use our own keystone module, but
instead make use of the ansible runtime venv's shade
library and upstream ansible modules, we can eliminate
this package/library being installed on the host.

Change-Id: Ib824e6e2b6e8182b2423667f03649b2bbe8da962
2018-09-18 12:32:19 +01:00
Jesse Pretorius 2ff6ab05aa Rename SUSE vars file to cater for any version
To make the transition between versions easier,
we rename the vars file. This also resolves
issues when meta-dependent role inclusions do
not pick up the correct file when using the
include_vars task with multiple search paths.

We also add the 'python-cursive' package to the
distro install package list as cinder requires
it, and the service packages have not yet got
it as a dependency in the spec.

Depends-On: https://review.openstack.org/602924
Change-Id: Ibb712b60757cf085cf794551ca5504e68977cd0a
2018-09-15 21:20:59 -06:00
Jesse Pretorius 7d3a797c2c Use a common python build/install role
In order to radically simplify how we prepare the service
venvs, we use a common role to do the wheel builds and the
venv preparation. This makes the process far simpler to
understand, because the role does its own building and
installing. It also reduces the code maintenance burden,
because instead of duplicating the build processes in the
repo_build role and the service role - we only have it all
done in a single place.

We also change the role venv tag var to use the integrated
build's common venv tag so that we can remove the role's
venv tag in group_vars in the integrated build. This reduces
memory consumption and also reduces the duplication.

This is by no means the final stop in the simplification
process, but it is a step forward. The will be work to follow
which:

1. Replaces 'developer mode' with an equivalent mechanism
   that uses the common role and is simpler to understand.
   We will also simplify the provisioning of pip install
   arguments when doing this.
2. Simplifies the installation of optional pip packages.
   Right now it's more complicated than it needs to be due
   to us needing to keep the py_pkgs plugin working in the
   integrated build.
3. Deduplicates the distro package installs. Right now the
   role installs the distro packages twice - just before
   building the venv, and during the python_venv_build role
   execution.

Depends-On: https://review.openstack.org/598957
Change-Id: Ica126c4ae20d2a7972a9309147cf6db902343bbf
Implements: blueprint python-build-install-simplification
Signed-off-by: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
2018-09-03 11:11:05 +00:00
Kevin Carter (cloudnull) 12ea9635ff
Convert role to use a common systemd service role
This removes the systemd service templates and tasks from this role and
leverages a common systemd service role instead. This change removes a
lot of code duplication across all roles all without sacrificing features
or functionality. The intention of this change is to ensure uniformity and
reduce the maintenance burden on the community when sweeping changes are
needed.

The systemd journal would normally be populated with the standard out of
a service however with the use of uwsgi this is not actually happening
resulting in us only capturing the logs from the uwsgi process instead
of the service itself. This change implements journal logging in the
service config, which is part of OSLO logging.

OSLO logging docs found here: <https://docs.openstack.org/oslo.log/3.28.1/journal.html>

A new variable `cinder_environment_overrides` has been added to ensure
the correct PATH is set for cinder services.

This reverts commit b219b90536.

Change-Id: I1cb10aa591a8262add47e2f5b61cfded9e28241d
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-07-31 15:09:58 -05:00
Jonathan Rosser 537961be52 Use generic vars file for ubuntu
These vars work for both Xenial and Bionic.

Change-Id: I59130799ae9697a762182644737911c774a3932d
2018-07-31 09:23:47 +01:00