In order to allow definition of policies per service, we need to add variables
to service roles, that will be passed to openstack.osa.mq_setup.
Currently this can be handled by leveraging group_vars and overriding `oslomsg_rpc_policies` as a whole, but it's not obvious and
can be non-trivial for some groups which are co-locating multiple services
or in case of metal deployments.
Change-Id: I99cac10abd52d6a3a427615d107c094fd7772caf
This change implements and enables by default quorum support
for rabbitmq as well as providing default variables to globally tune
it's behaviour.
In order to ensure upgrade path and ability to switch back to HA queues
we change vhost names with removing leading `/`, as enabling quorum
requires to remove exchange which is tricky thing to do with running
services.
Depends-On: https://review.opendev.org/c/openstack/ansible-role-zookeeper/+/899386
Change-Id: I8c418906b75edb633948f2c074170454a8f3e2d0
With update of ansible-lint to version >=6.0.0 a lot of new
linters were added, that enabled by default. In order to comply
with linter rules we're applying changes to the role.
With that we also update metdata to reflect current state.
Depends-On: https://review.opendev.org/c/openstack/ansible-role-systemd_service/+/888223
Change-Id: I6ca96cd72600e59c2e6616880d7cd9155a26c5bb
By overriding the variable `designate_backend_ssl: True` HTTPS will
be enabled, disabling HTTP support on the designate backend api.
The ansible-role-pki is used to generate the required TLS
certificates if this functionality is enabled.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/879085
Change-Id: Id5c18a7305c744a2b0252f62debb1b5654e4abd7
Implement support for service_tokens. For that we convert
role_name to be a list along with renaming corresponding variable.
Additionally service_type is defined now for keystone_authtoken which
enables to validate tokens with restricted access rules
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/845690
Change-Id: I7eafa6b989a2fd726369b3959b5e6ba024b82274
- Implemented new variable ``connection_recycle_time`` responsible for SQLAlchemy's connection recycling
- Set new default values for db pooling variables which are inherited from the global ones.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/819424
Change-Id: I28c64b44eadfd726e07cb7159e5d3d94fde106ed
With PKI role in place in most cases you don't need to explicitly
provide path to the CA file because PKI role ensures that CA is trusted
by the system overall. In the meanwhile in PyMySQL [1] you must either
provide CA file or cert/key or enable verify.
Since current behaviour is to provide path to the custom CA we expect
certificate being trusted overall. Thus we enable cert verification when
galera_use_ssl is True.
[1] 78f0cf99e5/pymysql/connections.py (L267)
Change-Id: Ic5b072d983c6d553d996a0a3bd708eec4c2137e5
Instead of overriding each service separatelly it might make
sense for deployers to define some higher level variable that
will be used first or fallback to default variable.
Change-Id: If3627990a028b4c28c85609f5bf1341b836b0ccd
Move it to the service setup host (defaults to utility[0]) instead
of the galera[0] host, and use galera_address (defaults to internal VIP)
as the endpoint instead of a local connection on the db host.
Change-Id: Ia09330b907387f70db5f553f83d5a23a4f7f8a98
This patch aims to add a prefix for memcached_server
on each role to give the ability for deployers to
override the location of memcached cluster. I.e users
wants to create a single memcached cluster with k8s
for each service.
We also add pymemcache based on [1]
[1] https://review.opendev.org/711429
Change-Id: I152220d4c440202de1a61b1aee891bdb659e5577
Install the python3-designate package and rely on the OSA provided
service configs instead of installing each of the individual designate
service packages. The 'Mask any conflicting units' task can also be
removed since the installation of unwanted services will no longer be an
issue.
Change-Id: I40a596803df656f5e6f75996660fe9ace685c2ed
The variables barbican_developer_mode and barbican_venv_download
no longer carry any meaning. This review changes glance to
do the equivalent of what developer_mode was all the time,
meaning that it always builds the venv and never requires
the repo server, but it will use a repo server when available.
As part of this, we move the source build out of its own file
because it's now a single task to include the venv build role.
This is just to make it easier to follow the code.
Change-Id: Ifd7d4d0a314834165141019dbacb82bbc1c4176d
This change broke the admin URL because there is no variable
with the value designate_service_adminurl.
This reverts commit 2950db130d.
Change-Id: I8acb48e8a3fe9721b97be8a87c7be28974642e72
This patch add the conditional inclusion of the notification
section of the service configuration. This ensures that oslo.messaging
notifications use the correct transport for deployments that have
separate rpc and notify messaging backends. For example, if the
transport_url is not provided in the notification section of the
service configuration, the transport_url specified in the default
section will be used instead.
This patch conditionally selects the notifier driver. The noop
driver will be selected when notification publishing is disabled.
The messagingv2 driver is selected when notification publishing is
enabled.
Change-Id: Ie5cc5499980f986f1a7e530adf42f4dcc43fbaca
Closes-Bug: #1794320
This change allow deployer to set project that will be owner of managed
resources like auto-created records and zones.
The owner is specified using project name and defaults to service
tenant.
Depends-On: https://review.openstack.org/628979
Change-Id: I620be82d890aaa547decc59f81f55345f7177900
In order to enable the service setup host python interpreter to
be changed easily, we make it a variable. This will be useful
when someone sets the service setup host to be the utility
container, because we'll be able to set this var by default.
Change-Id: Id70abddc0ba2a4086406d35162d9aca0b526f196
In order to radically simplify how we prepare the service
venvs, we use a common role to do the wheel builds and the
venv preparation. This makes the process far simpler to
understand, because the role does its own building and
installing. It also reduces the code maintenance burden,
because instead of duplicating the build processes in the
repo_build role and the service role - we only have it all
done in a single place.
We also change the role venv tag var to use the integrated
build's common venv tag so that we can remove the role's
venv tag in group_vars in the integrated build. This reduces
memory consumption and also reduces the duplication.
This is by no means the final stop in the simplification
process, but it is a step forward. The will be work to follow
which:
1. Replaces 'developer mode' with an equivalent mechanism
that uses the common role and is simpler to understand.
We will also simplify the provisioning of pip install
arguments when doing this.
2. Simplifies the installation of optional pip packages.
Right now it's more complicated than it needs to be due
to us needing to keep the py_pkgs plugin working in the
integrated build.
3. Deduplicates the distro package installs. Right now the
role installs the distro packages twice - just before
building the venv, and during the python_venv_build role
execution.
Depends-On: https://review.openstack.org/598957
Change-Id: I29833c41eba9b3ff80200b7f567a120f589e170e
Implements: blueprint python-build-install-simplification
Signed-off-by: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
This removes the systemd service templates and tasks from this role and
leverages a common systemd service role instead. This change removes a
lot of code duplication across all roles all without sacrificing features
or functionality. The intention of this change is to ensure uniformity and
reduce the maintenance burden on the community when sweeping changes are
needed.
The systemd journal would normally be populated with the standard out of
a service however with the use of uwsgi this is not actually happening
resulting in us only capturing the logs from the uwsgi process instead
of the service itself. This change implements journal logging in the
service config, which is part of OSLO logging.
OSLO logging docs found here: <https://docs.openstack.org/oslo.log/3.28.1/journal.html>
Change-Id: I9764f557007d97cfcbe02abf7166cce423b39a31
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
When the RPC and Notify service are the same, the credentials
must match - otherwise the tasks to create the user/password
will overwrite with each other.
If the two clusters are different, then the matching credentials
and vhost will not be a problem. However, if the deployer really
wishes to make sure they're different, then the vars can be
overridden.
Also, to ensure that the SSL value is consistently set in the
conf file, we apply the bool filter. We also use the 'notify'
SSL setting as the messaging system for Notifications is more
likely to remain rabbitmq in our default deployment with qrouterd
becoming the default for RPC messaging.
Change-Id: Id94e6337032807ee5baa79945e043c9282e17263
There is no record for why we implement the MQ vhost/user creation
outside of the role in the playbook, when we could do it inside the
role.
Implementing it inside the role allows us to reduce the quantity of
group_vars duplicated from the role, and allows us to better document
the required variables in the role. The delegation can still be done
as it is done in the playbook too.
In this patch we implement two new variables:
- designate_oslomsg_rpc_setup_host
- designate_oslomsg_notify_setup_host
These are used in the role to allow delegation of the MQ vhost/user
setup for each type to any host, but they default to using the first
member of the applicable oslomsg host group.
We also adjust some of the defaults to automatically inherit existing
vars set in group_vars form the integrated build so that we do not
need to do the wiring in the integrated build's group vars. We still
default them in the role too for independent role usage.
Change-Id: I65203372e551d5ab3ef17a823f2089c23a2af8fe
In order to reduce the packages required to pip install on to the hosts,
we allow the service setup to be delegated to a specific host, defaulting
to the deploy host. We also switch as many tasks as possible to using the
built-in Ansible modules which make use of the shade library.
The 'virtualenv' package is now installed appropriately by the openstack_hosts
role, so there's no need to install it any more. The 'httplib2' package is a
legacy Ansible requirement for the get_url/get_uri module which is no longer
needed. The keystone client library is not required any more now that we're
using the upstream modules. As there are no required packages left, the task
to install them is also removed.
With the dependent patches, the openstack_openrc role is now executed once
on the designated host, so it is no longer required as a meta-dependency for
the role.
Depends-On: https://review.openstack.org/579233
Depends-On: https://review.openstack.org/579959
Change-Id: I155b5036dd98f80a0d31d8bb691beae147a97bc3
There is no record for why we implement the database creation outside
of the role in the playbook, when we could do it inside the role.
Implementing it inside the role allows us to reduce the quantity of
group_vars duplicated from the role, and allows us to better document
the required variables in the role. The delegation can still be done
as it is done in the playbook too.
In this patch we implement a new variable called 'designate_db_setup_host'
which is used in the role to allow delegation of the database setup
task to any host, but defaults to the first member of the galera_all
host group. We also document the variable 'designate_galera_address' which
has been used for a long time, but never documented. A bunch of unused
variables have also been removed.
Change-Id: Ie0d09401adaa66142fe12d1b6454641dd76104b7
The following packages are required in-order to run osprofiler.
these packages will provide deployers the ability to profile
a service on demand should they choose to enable the profile
functionality.
Depends-On: I3df2c670beeb78baaa1515bcd27e8f2b0d95b3a9
Change-Id: Ibe97e34f8e65948882f8c93f0852460cc53eb231
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This introduces oslo.messaging variables that define the RPC and Notify
transports for the OpenStack services. These parameters replace the
rabbitmq values and are used to generate the messaging transport_url for
the service. The association of the messaging backend server to the
oslo.messaging service will be transparent to the designate service.
This patch:
* Add oslo.messaging variables for RPC and Notify to defaults
* Update transport_url generation (add for notification)
* Add oslo.messaging to tests inventory
* Update tests
* Add releaes note
* Update README and example playbook
Change-Id: I620a13e1ea3c24c8bd31c02206613d37d769dd30
Distributions provide packages for the OpenStack services so we add
support for using these instead of the pip ones.
Change-Id: Iab155254fa3600e1b6b3e8992f7c1fdc6f710ed6
Implements: blueprint openstack-distribution-packages
In order to clean variables in designate, the uri variable is removed,
and we filter url variable as needed
Change-Id: I537c963722553d798356287de3daefcad99ac62f
virtualenv-tools has a bug which gets triggered in gates: it can't
change the shebang of a virtualenv python bin/ files if they
were generated with a virtualenv script whose shebang ends with
python2 instead of python.
Because we can't modify virtualenv-tools, we use shell scripts
instead.
Change-Id: I7351e9e7e842b6de4b3e841ee9e44b6b6c514522
Partial-Bug: #1741634
When 'designate_galera_use_ssl' is True, use an encrypted connection to
the database using either a self-signed or user-provided CA certificate.
A new non-voting test has been added to verify that the role remains
functional when enabling SSL features.
Change-Id: I0d8e3b685faa8d394fd56f8fbfd9b492d2c2cb60
Partial-Bug: 1667789
Use openstack service proto variable in defaults for designate endpoints
This will align the designate role with other roles method
Partial-Bug: 1717273
Change-Id: I033940c29e91945a96bf9a4a9c9058badac37c5e
The keystonemiddleware library recently switched to using the
cryptography library over pycrypto, which was unmaintained. See
Iced7f5115e49ccf4f7f5bf6813cb5988b95c248b
Change-Id: I7cb31ead9146e8abf02c8ff01188dc69c069607d
Co-Authored-By: Nolan Brubaker <nolan.brubaker@rackspace.com>
This creates a specific slice which all OpenStack services will operate
from. By creating an independent slice these components will be governed
away from the system slice allowing us to better optimise resource
consumption.
See the following for more information on slices:
* https://www.freedesktop.org/software/systemd/man/systemd.slice.html
See for following for more information on resource controls:
* https://www.freedesktop.org/software/systemd/man/systemd.resource-control.html
Tools like ``systemd-cgtop`` and ``systemd-cgls`` will now give us
insight into specific processes, process groups, and resouce consumption
in ways that we've not had access to before. To enable some of this reporting
the accounting options have been added to the [Service] section of the unit
file.
Change-Id: I5885643199db3ef618fc86f0cd80c14f1d7c89c4
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Dmitriy Rabotyagov