Commit Graph

169 Commits

Author SHA1 Message Date
Zuul 24648387cc Merge "Add quorum queues support for service" 2023-09-04 12:43:30 +00:00
Dmitriy Rabotyagov 5683c693c1 Use proper galera port in configuration
While <servuce>_galera_port is defined and used for db_setup
role, it's not in fact used in a connection string for oslo.db.

Change-Id: I6b910817ddc6eab68f815f776faeee432e55012e
2023-07-31 14:27:51 +02:00
Dmitriy Rabotyagov af229369fb Add quorum queues support for service
This change implements and enables by default quorum support
for rabbitmq as well as providing default variables to globally tune
it's behaviour.

In order to ensure upgrade path and ability to switch back to HA queues
we change vhost names with removing leading `/`, as enabling quorum
requires to remove exchange which is tricky thing to do with running
services.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/875399
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/873618
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-os_glance/+/873632
Change-Id: I9e1f87fd2c396eb4b48459c3055b43678fae296a
2023-07-06 12:56:08 +00:00
Jonathan Rosser 2820fcc1d6 Use v3 service type in keystone_authtoken config
The service type in this config section must match the service
type in the service catalog, otherwise limited scope application
credentials will not work with the cinder API [1].

[1] https://docs.openstack.org/keystone/2023.1/user/application_credentials.html#access-rules

Change-Id: I711241af8f7520b97f2b1cafd1406ff705fb78a6
2023-06-20 15:11:20 +01:00
Dmitriy Rabotyagov 1af3003e16 Define service_user for cinder services
In order to cover OSSA-2023-003, a requirement to define service_user
section for all cinder services has been added by cinder.

Change-Id: I19c2b03c61f714fedb593da8489e50d3fa08d933
2023-05-22 15:57:41 +02:00
Dmitriy Rabotyagov 789d14de9d Add coordination support
This patch adds configuration for coordination service when cinder
active/active setup is used and coordination hosts exist

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/864750
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/865805
Change-Id: I2dda4c74aa69aea3ecea92954922dbfe1bd56687
2022-11-28 09:05:04 +00:00
Jimmy McCrory 75d7ae9093 Remove oslo_policy section from cinder.conf
policy.json was deprecated.
Remove the oslo_policy configuration which was still using it.

Change-Id: I0215bb17219745ab2c838a1999caf0a7baa4242d
2022-08-08 23:21:06 -07:00
Dmitriy Rabotyagov f755eadadf Support service tokens
Implement support for service_tokens. For that we convert
role_name to be a list along with renaming corresponding variable.

Additionally service_type is defined now for keystone_authtoken which
enables to validate tokens with restricted access rules

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/845690
Change-Id: I1d0156a2ad829aa730419e1d9dfa1cd49026a6be
Related-Bug: #1948456
2022-06-14 11:30:31 +02:00
Andrew Bonney 6efa45e2bd Add configuration option for heartbeat_in_pthread
This configuration option has been observed to result in file
descriptor leaks in certain circumstances. A variable is added
here so that it can be easily overridden.

Related-Bug: #1961603
Change-Id: I8155264b181d6f21728804ef8260979931597427
2022-03-15 10:52:22 +00:00
Damian Dabrowski 210cfc5f8f Database connection pooling improvements
- Implemented new variable ``connection_recycle_time`` responsible for SQLAlchemy's connection recycling
- Set new default values for db pooling variables which are inherited from the global ones.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/819424
Change-Id: Ib445e0ddd01d52314e50ca6edd2fa20e5f6ef3eb
2021-12-03 11:40:36 +01:00
Dmitriy Rabotyagov 3370ad8e03 Refactor definition of lock path
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/819300
Depends-On: https://review.opendev.org/c/openstack/ansible-role-systemd_service/+/819298
Change-Id: I674aa11ecfec2a2bc7bbc84865f9ec5f4a872ebe
2021-11-25 12:13:31 +00:00
Dmitriy Rabotyagov 9e37558593 Refactor galera_use_ssl behaviour
With PKI role in place in most cases you don't need to explicitly
provide path to the CA file because PKI role ensures that CA is trusted
by the system overall. In the meanwhile in PyMySQL [1] you must either
provide CA file or cert/key or enable verify.

Since current behaviour is to provide path to the custom CA we expect
certificate being trusted overall. Thus we enable cert verification when
galera_use_ssl is True.

[1] 78f0cf99e5/pymysql/connections.py (L267)

Change-Id: Ieab4ab2e36e4953961841be334ff16162f7daeb8
2021-09-20 12:55:31 +03:00
Jonathan Rosser a0d9137b53 Add variables for rabbitmq ssl configuration
Change-Id: I3cafb6197d944337299152273bba7494ff0edcfe
2021-05-13 14:39:17 +00:00
Dmitriy Rabotyagov 3c0106b97c Replace deprecated key_manager
Usage of module path is a deprecated behaviour. All cinder docs
suggest using just `barbican` for [key_manager]/backend value.

We also explicitly specify for barbican to talk over internal
interface instead of public which is default behaviour.

Change-Id: I3d0d401c2024c6051b7fb61a929dd4e6975b0a30
2021-02-23 17:45:39 +02:00
Dmitriy Rabotyagov 00a38c6584 Define credentials for nova interaction
By default cinder will use tenant token for interaction with nova.
However for resize of the in-use volume cinder needs to have admin
credentials set for such kind of interactions

Change-Id: Id32d3a5727fc96e07e09332beb7265610e5c8b10
Closes-Bug: #1902914
2020-11-11 07:05:00 +00:00
Dmitriy Rabotyagov c6b9f011b7 Explicitly enable/disable active/active
As deployers may have usecases with several different backends being
served with the same cinder volume,
we should provide an option to easily override default behaviour

Change-Id: Idc9a71d722b7443cf1437c2d95f75c615b6035a4
2020-06-17 18:49:39 +03:00
Guilherme Steinmüller beb3d002dd Refactor memcached_servers
This patch aims to add a prefix for memcached_server
on each role to give the ability for deployers to
override the location of memcached cluster. I.e users
wants to create a single memcached cluster with k8s
for each service.

We also add pymemcache based on [1]

[1] https://review.opendev.org/711429

Change-Id: Ic83f6371c5d2bbed6a7d6d2f92a69fd3a2afd0d3
2020-03-13 22:16:10 +00:00
Zuul 73ad9ab751 Merge "Start using uWSGI role" 2019-09-06 15:40:44 +00:00
Dmitriy Rabotyagov ee018d9083 Start using uWSGI role
Move service to use uWSGI role instead of iternal task for uwsgi
deployment. This aims to ease the maintenance of uWSGI and speedup
metal deployments as the same uwsgi environment will be used
across all services.

Depends-On: https://review.opendev.org/678025/
Change-Id: Ieac6d03a436f6b706d7f12e292ffc98171a43246
2019-09-04 15:34:37 +03:00
Zuul c4efbadb0f Merge "Fix policy configuration" 2019-08-24 19:33:42 +00:00
Mohammed Naser c148d77e29 rbd: start using active-active
This patch drops the hacky workaround of using backend_host which
is not recommended by the Cinder team and instead uses active-active
RBD which has been implemented since Rocky.

Closes-Bug: #1837403
Change-Id: I0c8aed4d0608c1f117e1baa1f428875956159ffd
2019-07-22 12:10:38 -04:00
Logan V dbaf649e0c Fix policy configuration
Since cinder moved to policy-in-code, oslopolicy-policy-generator
must be used to compile a policy file that can be distributed to
Horizon for policy settings. For cinder, it will only output its
default policy unless cinder.conf contains an oslo_policy
configuration section pointing to the policy config file.

Change-Id: I66a76fdb30ae6afd8a8faf4119bef495600aec50
2019-06-14 19:39:59 -05:00
Jimmy McCrory d139d40a19 Cleanup debian cinder service distro package list
The debian packages for individual cinder services provide service
configs which automatically start after install, leading to all cinder
services running on each cinder volume host. Instead only install the
python3-cinder library package and rely on the service files OSA manages.

uwsgi packages are only required on hosts running cinder-api, so an
additional variable, 'cinder_api_distro_packages' has been added for
packages specific to those hosts.

Beginning in the Stein release, Ubuntu distro packages are now using
Python3. This requires additionally installing and using the uwsgi
python3 plugin.

Change-Id: Iafcd9a8141fffb2226ec5103960064decc579bd5
2019-06-05 13:51:19 -07:00
Chandan Kumar 2b47851430 Set glance_api_version=2 in cinder.conf
While running smoke tests using
https://review.openstack.org/#/c/652060/ in os_tempest jobs, the
tempest volume and compute api tests are failing with
Invalid image identifier or unable to access requested image error.
Adding glance_api_version in cinder.conf fixes the issue.

Change-Id: I8ae1599b96ead1361ee82960fd5fb66c259ea0ff
Signed-off-by: Chandan Kumar <chkumar@redhat.com>
2019-04-17 12:29:09 +05:30
Jakub Jursa 826627c937 Make devices filter in lvm.conf overridable
Change-Id: I2700028965bd89abaa342df91014874c94daf06c
2019-03-01 22:37:24 +00:00
Zuul 34bb90e755 Merge "Fix evaluation of cinder_scheduler_default_filters, remove confusing comment" 2019-02-23 13:19:08 +00:00
Zuul a03cdad6d8 Merge "cinder-volume: rbd driver set backend_host value" 2019-02-22 03:33:01 +00:00
Florian Haas bcc7f1511e Fix evaluation of cinder_scheduler_default_filters, remove confusing comment
Change I12859167d19b9f40e3378ac08fed094a42f40bc7, merged just today,
put the evaluation of cinder_scheduler_default_filters inside an
{% if } block that it wasn't meant to be inside. As a consequence,
that setting would be ignored more often that it would be
honored.

What threw me (sorry about that) was that I had tested this patch
on a Queens box, and stubbornly copied into the template on master.

And changes I02d2bae8712c0ca223cafb5a43304806c4b83125 and
Ib5a128e82e5251077e341b5f428eb097bcc17590 had left the
template in a somewhat confusing state: the template had retained the
"## Cinder API's enabled" comment despite the fact that there were
no settings left to enable or disable any APIs.

So, with this change the evaluation of cinder_scheduler_default_filters
goes to the right place (i.e. outside the {% if } block), *and*
the confusing API comment is removed.

Change-Id: Ic5707615571e62ba2326e2ad436333bac246c8dd
2019-02-20 23:40:23 +01:00
Zuul 75f28d087a Merge "cinder.conf: add [nova] section, override interface defaults" 2019-02-20 16:25:30 +00:00
Florian Haas ffbd0a1aaf Introduce cinder_scheduler_default_filters
Operators may want to define specific default filters. Previously,
they could only do so via cinder_cinder_conf_overrides.

Make the cinder.conf template understand cinder_scheduler_default_filters,
a list variable that, if defined, is folded into a comma-separated
list for scheduler_default_filters in the [DEFAULT] section.

Change-Id: I12859167d19b9f40e3378ac08fed094a42f40bc7
2019-02-15 20:57:57 +01:00
Florian Haas 8c436038e3 cinder.conf: add [nova] section, override interface defaults
To the best of my knowledge, the [nova] section in cinder.conf
is only ever used if the Cinder scheduler is acting as a Nova client
when the operator has enabled the InstanceLocalityFilter.

Per https://docs.openstack.org/cinder/latest/configuration/block-storage/samples/cinder.conf.html,
Cinder defaults to using the public Nova endpoint when using the
Nova API. This is contrary to OSA precedent, where services
normally use internal endpoints for service-to-service API requests.

When enabling the InstanceLocalityFilter in combination with Cinder
talking to the public Nova endpoint, this can create a very confusing
situation, particularly in pre-production clusters: if the public
endpoint has a self-signed SSL certificate, and Cinder is not
explicitly configured not to verify certificates, then this creates
a whole load of connection errors.

Thus, in order to follow POLA, configure the [nova] section to use
the internal endpoint, and (in case the internal endpoint does
use HTTPS) honor the keystone_service_internaluri_insecure setting,
as for other services.

Change-Id: Ie31a7e2917a188027db49ac51e6a77ee39a9abf0
2019-02-15 15:56:29 +01:00
Zuul ada7167002 Merge "Ensure create a volume from image" 2019-02-14 04:35:01 +00:00
Kevin Carter 38807bc2c7 Correct notification driver
The notification driver setup was resulting in the driver and connection string
on the same line. This is caused by the case statement and how jinja formats
the template when a case statement is present. This change modifies how the
driver string is created using a ternary, which will eliminate the case
statement and render the value of the diver correctly.

Change-Id: I94899d14906a0a4e51137dd066f25f8f0e0a2334
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2019-02-07 22:32:43 +00:00
Guilherme Steinmüller 3dd035bbf8 Ensure create a volume from image
This patch aims to add this extra config based on the
openstack docs[1] that allows an user to create a volume
from an image

[1] https://docs.openstack.org/cinder/latest/admin/blockstorage-volume-backed-image.html

Change-Id: Iefdaf569dfdab4e28a8fe41a0846d103b5a7ce23
Closes-Bug: 1812185
2019-01-22 17:28:01 +00:00
Kevin Carter f3f956e904 Cleanup files and templates using smart sources
The files and templates we carry are almost always in a state of
maintenance. The upstream services are maintaining these files and
there's really no reason we need to carry duplicate copies of them. This
change removes all of the files we expect to get from the upstream
service. while the focus of this change is to remove configuration file
maintenance burdens it also allows the role to execute faster.

  * Source installs have the configuration files within the venv at
    "<<VENV_PATH>>/etc/<<SERVICE_NAME>>". The role will now link the
    default configuration path to this directory. When the service is
    upgraded the link will move to the new venv path.
  * Distro installs package all of the required configuration files.

To maintain our current capabilities to override configuration the
role will fetch files from the disk whenever an override is provided and
then push the fetched file back to the target using `config_template`.

Change-Id: Ib3447cd5b0bcada4cdf82d9e4a9fe5160299f9c3
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Signed-off-by: Kevin Carter <kevin@cloudnull.com>
2019-01-19 18:02:51 -06:00
Dmitriy Rabotjagov 5e7e1a7ea9 Adds resource_filters.json distribution
Cinder supposes to see resource_filters.json in
/etc/cinder/resource_filters.json, but role doesn't distribute this file
It implements generalized filters, w/o which non-admins will
experience problems with some operations (i.e in horizon).

Closes-Bug: #1810537
Change-Id: I0f699c9869effc5ccc0d3f79422935975f698134
2019-01-08 16:19:00 +00:00
Justin Alford 8b86eb7a9d cinder-volume: rbd driver set backend_host value
Closes-Bug #1807384

Change-Id: I59d7dd0c3ab9a8ae68dc8438866b158abfbad9a9
2018-12-07 06:36:23 -07:00
Mohammed Naser e5d5116ac5 Fix Cinder backup to use full class path
Cinder no longer supports using a driver name without
specifying the full class path.

This patch fixes all the documentation and adds a
release note for that.

Change-Id: Ia31748e3abb0fca54efd5e8e74bde0440760159e
Related-Change-Id: I3ada2dee1857074746b1893b82dd5f6641c6e579
2018-11-08 16:00:43 +00:00
Andy Smith f1b2eed3eb Update messaging notification configuration
This patch removes the conditional inclusion of the notification
section of the service configuration. This ensures that oslo.messaging
notifications use the correct transport for deployments that have
separate rpc and notify messaging backends. For example, if the
transport_url is not provided in the notification section of the
service configuration, the transport_url specified in the default
section will be used instead.

This patch conditionally selects the notifier driver. The noop
driver will be selected when notification publishing is disabled.
The messagingv2 driver is selected when notification publishing is
enabled.

Change-Id: I65ef897e78b5fb1c9ff6bb3d31cb6aa9e0c8429e
2018-09-22 10:09:23 -04:00
Kevin Carter a9ab3c16c7 Resolve cinder config deprecation warnings
The change removes or updates options that have been changed in the
cinder.conf file and are throwing deprecation warnings in the logs.

Change-Id: Ib5a128e82e5251077e341b5f428eb097bcc17590
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-09-16 03:38:50 +00:00
Mohammed Naser 6c0cb34bb5 Add target_helper to backend_defaults
It seems when using Cinder backends, the value inside backend_defaults
in used which can default to the wrong value.

Change-Id: I771a2655545d0d006c882584657d97814e2fa8e1
2018-08-16 10:04:11 -04:00
Mohammed Naser 47897b0ed7 Drop JSON logging for Cinder
When using debug, JSON logging is enabled due to a bug that
seems to occur when using unicode characters[1].  This patch drops
this in order to make sure that we resolve this in Cinder (or
test that it's already resolved).

[1]: https://bugs.launchpad.net/cinder/+bug/1703493

Change-Id: Ia5eea6b908459366d93d9ceac5d16908e99e051a
2018-08-16 10:04:11 -04:00
Kevin Carter (cloudnull) 12ea9635ff
Convert role to use a common systemd service role
This removes the systemd service templates and tasks from this role and
leverages a common systemd service role instead. This change removes a
lot of code duplication across all roles all without sacrificing features
or functionality. The intention of this change is to ensure uniformity and
reduce the maintenance burden on the community when sweeping changes are
needed.

The systemd journal would normally be populated with the standard out of
a service however with the use of uwsgi this is not actually happening
resulting in us only capturing the logs from the uwsgi process instead
of the service itself. This change implements journal logging in the
service config, which is part of OSLO logging.

OSLO logging docs found here: <https://docs.openstack.org/oslo.log/3.28.1/journal.html>

A new variable `cinder_environment_overrides` has been added to ensure
the correct PATH is set for cinder services.

This reverts commit b219b90536.

Change-Id: I1cb10aa591a8262add47e2f5b61cfded9e28241d
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-07-31 15:09:58 -05:00
Mohammed Naser 6d75b48de0 Change deprecated option iscsi_helper to target_helper
The iscsi_helper configuration option has been deprecated and
replaced by target_helper.  This patch updates it to use the
correct helper.

Change-Id: I3bafacb7d483dd3b1132b3c3b4411e26fc117ad8
2018-07-20 03:21:42 +00:00
Andrew Smith e8eae7d067 Update to use oslo.messaging service for RPC and Notify
This introduces oslo.messaging variables that define the RPC and
Notify transports for the OpenStack services. These parameters
replace the rabbitmq values and are used to generate the messaging
transport_url for the service. The association of the messaging
backend server to the oslo.messaging services will then be
transparent to the cinder service.

This patch:
* Add oslo.messaging variables for RPC and Notify to defaults
* Update transport_url generation
* Add oslo.messaging to inventory
* Add release note

Depends-On: If4326a6848d2d32af284fdbb94798eb0b03734d5
Depends-On: I2b09145b60116c029fc85477399c24f94974b61d
Change-Id: I6b29b89c80889eff34fe78674cd888ea9d398388
2018-06-04 13:39:05 -04:00
Markos Chandras ba7c1e29d9 Add support for using distribution packages for OpenStack services
Distributions provide packages for the OpenStack services so we add
support for using these instead of the pip ones.

Change-Id: I4ff3cbf5e1e1ce04cd4cdc9c1ce97afdeace5159
Depends-On: I5a78e2120e596d36629b4ba978b2b5df76b149b0
Implements: blueprint openstack-distribution-packages
2018-05-04 17:08:45 +01:00
Zuul e055d332ac Merge "Revert "Convert role to use a common systemd service role"" 2018-04-03 13:00:10 +00:00
Jean-Philippe Evrard b219b90536 Revert "Convert role to use a common systemd service role"
This reverts commit 2f78790cc2.

This broke master due to a problem with cinder backup which can't reload (probably an issue with its systemd service file)

See also [1], [2].

[1]: http://logs.openstack.org/periodic/git.openstack.org/openstack/openstack-ansible/master/openstack-ansible-deploy-ceph-ubuntu-xenial/e950b61/job-output.txt.gz#_2018-04-03_07_17_12_755695
[2]: http://logs.openstack.org/periodic/git.openstack.org/openstack/openstack-ansible/master/openstack-ansible-deploy-aio_lxc-ubuntu-xenial/fe598f2/job-output.txt.gz#_2018-04-03_07_17_29_341574

Change-Id: I930f7280bc668ee275d72be07d8a2c088322f653
2018-04-03 08:34:32 +00:00
ZhongShengping 83da6e06cb Deprecate auth_uri option
Option auth_uri from group keystone_authtoken is deprecated[1].
Use option www_authenticate_uri from group keystone_authtoken.

[1]https://review.openstack.org/#/c/508522/

Change-Id: I6f7ba1de5152e62eae1e83f972830d61e37aede3
Implements: blueprint deprecate-auth-uri-option
2018-04-03 14:17:11 +08:00
Kevin Carter 2f78790cc2
Convert role to use a common systemd service role
This removes the systemd service templates and tasks from this role and
leverages a common systemd service role instead. This change removes a
lot of code duplication across all roles all without sacrificing features
or functionality. The intention of this change is to ensure uniformity and
reduce the maintenance burden on the community when sweeping changes are
needed. The exterior role is built to be OSA compatible and may be pulled
into tree should we deem it necessary.

Change-Id: Idf7513a22608a602fe4e53da070c11aac115b6d0
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-03-30 20:20:48 -05:00