This change implements and enables by default quorum support
for rabbitmq as well as providing default variables to globally tune
it's behaviour.
In order to ensure upgrade path and ability to switch back to HA queues
we change vhost names with removing leading `/`, as enabling quorum
requires to remove exchange which is tricky thing to do with running
services.
Change-Id: I78c71cb33f456cfd8d255446d708a1a957e32440
With update of ansible-lint to version >=6.0.0 a lot of new
linters were added, that enabled by default. In order to comply
with linter rules we're applying changes to the role.
With that we also update metdata to reflect current state.
Depends-On: https://review.opendev.org/c/openstack/ansible-role-systemd_service/+/888223
Change-Id: Iaab0ad50fd65287094522f77283a0112859d7730
By overriding the variable `trove_backend_ssl: True` HTTPS will
be enabled, disabling HTTP support on the trove backend api.
The ansible-role-pki is used to generate the required TLS
certificates if this functionality is enabled.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/879085
Change-Id: I15223354a28f0cc6c203f0cb3a19b9af834d4158
Implement support for service_tokens. For that we convert
role_name to be a list along with renaming corresponding variable.
Additionally service_type is defined now for keystone_authtoken which
enables to validate tokens with restricted access rules
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/845690
Change-Id: I7b01de89b459b4992ed9531e6c81259af21ba997
- Implemented new variable ``connection_recycle_time`` responsible for SQLAlchemy's connection recycling
- Set new default values for db pooling variables which are inherited from the global ones.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/819424
Change-Id: Ie9591c96d1e5c8f4dcf04524c04056934ceb00db
With PKI role in place in most cases you don't need to explicitly
provide path to the CA file because PKI role ensures that CA is trusted
by the system overall. In the meanwhile in PyMySQL [1] you must either
provide CA file or cert/key or enable verify.
Since current behaviour is to provide path to the custom CA we expect
certificate being trusted overall. Thus we enable cert verification when
galera_use_ssl is True.
[1] 78f0cf99e5/pymysql/connections.py (L267)
Change-Id: I27bbc7b97a2fec235d43b4ca502a3667450dc1fa
We haven't payed attention to the trove for a while and we need to
align configuration with current state of the service.
trove-conductor and trove-taskmanager configs are not
loaded by trove services anymore (at least by default), so we merge all
options into trove.conf
Also defaults and deprecated options are dropped from the config.
We implement bunch of new variables to cover missed logic and
some usecases.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/784565
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/784113
Change-Id: I2ba758de38ed3fee10fe1485aa25621eddbf9046
This migrate trove-api to usage of the uwsgi from native service to
align with other service way of deployment and since that's how service
is tested in devstack.
Change-Id: I83ad3af282942ff9714757e863f393894ac35f45
To match documentation and general pattern of service networks,
we change default subnets that will be used by trove instances.
Change-Id: I7a0f23b0cdd314206f299a860685e2f96ac3f625
Allow deployers to manage trove guestimages with trove role. This will
allow us to correctly test role with tempest, since we were not able to
upload images with tags otherwise
Change-Id: If2f550ef09cd01ec5eab485f0d81ecfaf32f924f
Instead of overriding each service separatelly it might make
sense for deployers to define some higher level variable that
will be used first or fallback to default variable.
Change-Id: Ic2fa270a3582db3da01b3c449fbdd8c2d5045062
Even the most modest 4C/8T system would run with the maximum 16 processes
due to the calculation being VCPU*2.
We devide amount of CPUs to number of threads for hyperthreaded CPUs
Change-Id: I2ae9039871158f8a94e3f96694ddadea3005537b
We use the same condition, which defines against what host some "service"
tasks should run against, several times. It's hard to keep it the same
across the role and ansible spending additional resources to evaluate
it each time, so it's simpler and better for the maintenance to set
a boolean variable which will say for all tasks, that we want to run
only against signle host, if they should run or not now.
Depends-On: https://review.opendev.org/755484
Change-Id: If7f26c8020d4b02ab9e82dcb69a7ba91a3b5d145
Move it to the service setup host (defaults to utility[0]) instead
of the galera[0] host, and use galera_address (defaults to internal VIP)
as the endpoint instead of a local connection on the db host.
Change-Id: I4f469df19bb371b0d38a6438a05054cd21152abd
This patch aims to add a prefix for memcached_server
on each role to give the ability for deployers to
override the location of memcached cluster. I.e users
wants to create a single memcached cluster with k8s
for each service.
We also add pymemcache based on [1]
[1] https://review.opendev.org/711429
Change-Id: I571a0ac9417767af413937e871ea6cdf156f2d75
As we don't provide wsgi file for mod_wsgi deployment (despite
having task to do that) and trove package don't contain wsgi application
we should drop apache mod_wsgi part of the setup as it's not functional.
Change-Id: I0a12fad27d3f994d79ead52ac90873a1e1362144
This patch aims to migrate service from usage of regular syslog files
to journald. We also disable uwsgi logging, since it dublicates
requests that are logged by service itself.
Change-Id: Ib16362a8becb6666f36e5abdb93c548de2a269da
Trove is now able to properly use the service catalog [1].
We have another[2] patch that this patch depends-on, which
causes us to have a circular dependency. As the change is minimal,
should be ok to squash them together to be able to fix that role.
The paragraph bellow contains the commit message of the patch that is
being squashed.
The 'Get admin tenant id' task was passing invalid arguments to the
os_project_facts module, and the 'Store admin tenant id' task was not
using the correct variable to retrieve the project id.
[1] https://review.opendev.org/#/c/574254/
[2] https://review.opendev.org/#/c/665458/
Change-Id: I779ba715d20d83b1efe4f07226a5eadd7e0a1870
The variables trove_developer_mode and trove_venv_download
no longer carry any meaning. This review changes trove to
do the equivalent of what developer_mode was all the time,
meaning that it always builds the venv and never requires
the repo server, but it will use a repo server when available.
As part of this, we move the source build out of its own file
because it's now a single task to include the venv build role.
This is just to make it easier to follow the code.
Change-Id: I08c864eaef894074326d765bb424fa6b98ec0ed2
In order to enable the service setup host python interpreter to
be changed easily, we make it a variable. This will be useful
when someone sets the service setup host to be the utility
container, because we'll be able to set this var by default.
Change-Id: I2a4b1b82307b3dd195bff8f9a8fb219ad581c2ad
This patch add the conditional inclusion of the notification
section of the service configuration. This ensures that oslo.messaging
notifications use the correct transport for deployments that have
separate rpc and notify messaging backends. For example, if the
transport_url is not provided in the notification section of the
service configuration, the transport_url specified in the default
section will be used instead.
This patch conditionally selects the notifier driver. The noop
driver will be selected when notification publishing is disabled.
The messagingv2 driver is selected when notification publishing is
enabled.
Change-Id: Ie967b281e15127bbf611aca557e99ac6b3d1cd4b
Closes-Bug: #1794320
The SUSE builds are failing due to a change in the galera_client
role causing the trove role's vars file to be found before the
galera_client vars file. We therefore rename the trove vars file
for SUSE. This also makes transitions between versions easier.
The mysql-python package is no longer maintained. We are using
pymysql instead, so this package does not need to be installed.
This resolves current build failures due to the mysql-python
package not being installable due to the new version of MariaDB
being used.
Depends-On: https://review.openstack.org/602924
Change-Id: Iaee84e8cec7c19d306924adf3743e03da6fa6b4f
In order to radically simplify how we prepare the service
venvs, we use a common role to do the wheel builds and the
venv preparation. This makes the process far simpler to
understand, because the role does its own building and
installing. It also reduces the code maintenance burden,
because instead of duplicating the build processes in the
repo_build role and the service role - we only have it all
done in a single place.
We also change the role venv tag var to use the integrated
build's common venv tag so that we can remove the role's
venv tag in group_vars in the integrated build. This reduces
memory consumption and also reduces the duplication.
This is by no means the final stop in the simplification
process, but it is a step forward. The will be work to follow
which:
1. Replaces 'developer mode' with an equivalent mechanism
that uses the common role and is simpler to understand.
We will also simplify the provisioning of pip install
arguments when doing this.
Depends-On: https://review.openstack.org/598957
Change-Id: Iaff49b75b03635fb07260c9a96f6459d270aed83
Implements: blueprint python-build-install-simplification
Signed-off-by: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
When the RPC and Notify service are the same, the credentials
must match - otherwise the tasks to create the user/password
will overwrite with each other.
If the two clusters are different, then the matching credentials
and vhost will not be a problem. However, if the deployer really
wishes to make sure they're different, then the vars can be
overridden.
Also, to ensure that the SSL value is consistently set in the
conf file, we apply the bool filter. We also use the 'notify'
SSL setting as the messaging system for Notifications is more
likely to remain rabbitmq in our default deployment with qrouterd
becoming the default for RPC messaging.
Change-Id: I618e393b9332eac6fd35472c58dfa2935da09a4d
There is no record for why we implement the MQ vhost/user creation
outside of the role in the playbook, when we could do it inside the
role.
Implementing it inside the role allows us to reduce the quantity of
group_vars duplicated from the role, and allows us to better document
the required variables in the role. The delegation can still be done
as it is done in the playbook too.
In this patch we implement two new variables:
- trove_oslomsg_rpc_setup_host
- trove_oslomsg_notify_setup_host
These are used in the role to allow delegation of the MQ vhost/user
setup for each type to any host, but they default to using the first
member of the applicable oslomsg host group.
We also adjust some of the defaults to automatically inherit existing
vars set in group_vars form the integrated build so that we do not
need to do the wiring in the integrated build's group vars. We still
default them in the role too for independent role usage.
Change-Id: I88514f00c4b4c0452b573d765bb895cb562f952e
In order to reduce the packages required to pip install on to the hosts,
we allow the service setup to be delegated to a specific host, defaulting
to the deploy host. We also switch as many tasks as possible to using the
built-in Ansible modules which make use of the shade library.
The 'virtualenv' package is now installed appropriately by the openstack_hosts
role, so there's no need to install it any more. The 'httplib2' package is a
legacy Ansible requirement for the get_url/get_uri module which is no longer
needed. The keystone client library is not required any more now that we're
using the upstream modules. As there are no required packages left, the task
to install them is also removed.
Change-Id: I9ce106569ced891c551b36506d360e4b8718c7e3
This introduces oslo.messaging variables that define the RPC and
Notify transports for the OpenStack services. These parameters replace
the rabbitmq values and are used to generate the messaging
transport_url for the service. The association of the messaging
backend server to the oslo.messaging services will then be transparent
to the trove service.
This patch:
* Add oslo.messaging variables for RPC and Notify to defaults
* Update transport_url generation (add for notification)
* Add oslo.messaging to tests inventory
* Update tests
* Update docs
* Update extras
* Add release note
Change-Id: Ia01317343ae6fbc790d64b5ba282c8c069750d45
The following packages are required in-order to run osprofiler.
these packages will provide deployers the ability to profile
a service on demand should they choose to enable the profile
functionality.
Change-Id: I4df8f1a83b09eba3ce35d88373c392a1715ccad4
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
There is no record for why we implement the database creation outside
of the role in the playbook, when we could do it inside the role.
Implementing it inside the role allows us to reduce the quantity of
group_vars duplicated from the role, and allows us to better document
the required variables in the role. The delegation can still be done
as it is done in the playbook too.
In this patch we implement a new variable called 'trove_db_setup_host'
which is used in the role to allow delegation of the database setup
task to any host, but defaults to the first member of the galera_all
host group. We also document the variable 'trove_galera_address' which
has been used for a long time, but never documented. A bunch of unused
variables have also been removed.
The extras folder is removed given that trove's playbooks have been
merged into the integrated repository.
Change-Id: I0bf801416bd0349534588fc7272dffd4b86b7bb9
This removes the systemd service templates and tasks from this role and
leverages a common systemd service role instead. This change removes a
lot of code duplication across all roles all without sacrificing features
or functionality. The intention of this change is to ensure uniformity and
reduce the maintenance burden on the community when sweeping changes are
needed. The exterior role is built to be OSA compatible and may be pulled
into tree should we deem it necessary.
Change-Id: Id833d4ecf9d17bb74c1c0702c00ace241ac48873
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
virtualenv-tools has a bug which gets triggered in gates: it can't
change the shebang of a virtualenv python bin/ files if they
were generated with a virtualenv script whose shebang ends with
python2 instead of python.
Because we can't modify virtualenv-tools, we use shell scripts
instead.
Change-Id: Iad3a44f35c57d92dcbbfde4acbd86a3a4eb1879d
Partial-Bug: #1741634
When 'trove_galera_use_ssl' is True, use an encrypted connection to
the database using either a self-signed or user-provided CA certificate.
A new non-voting test has been added to verify that the role remains
functional when enabling SSL features.
Change-Id: Ia453e842cabbb46e25f9918e88738340ff08f2ab
Partial-Bug: 1667789
Remove the unnecessary trove_regular_user. The documenation that was
referenced [1] when this was added to the role is intended to configure
Trove for development purposes. The trove_regular_user is not used by the
Trove service and is only being created to give the developer a non-admin
user to use for testing.
[1] https://docs.openstack.org/trove/latest/install/manual_install.html
Change-Id: I7c9289e191255e92530ea62568b212320f705ff8
Dmitriy Rabotyagov