By overriding the variable `aodh_backend_ssl: True` HTTPS will
be enabled, disabling HTTP support on the aodh backend api.
The ansible-role-pki is used to generate the required TLS
certificates if this functionality is enabled.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/879085
Change-Id: Ibb4d7b465f07fff6c172b38aa647fd8d6a4fcd43
Align aodh role with others and use
openstack_service_<type>uri_proto variables as default ones
to define protocol being used for endpoints.
Change-Id: Idbc68f7496fd57f98fc77b9d0e345e576b51d108
- Implemented new variable ``connection_recycle_time`` responsible for SQLAlchemy's connection recycling
- Set new default values for db pooling variables which are inherited from the global ones.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/819424
Change-Id: Id7b0c26409a0d1b33a679201655a4dd08bacc57a
With PKI role in place in most cases you don't need to explicitly
provide path to the CA file because PKI role ensures that CA is trusted
by the system overall. In the meanwhile in PyMySQL [1] you must either
provide CA file or cert/key or enable verify.
Since current behaviour is to provide path to the custom CA we expect
certificate being trusted overall. Thus we enable cert verification when
galera_use_ssl is True.
[1] 78f0cf99e5/pymysql/connections.py (L267)
Change-Id: Ie02010583c9c9fc82211065cedac4fc120a31318
Instead of overriding each service separatelly it might make
sense for deployers to define some higher level variable that
will be used first or fallback to default variable.
Change-Id: I74376cc3c20df54663ffbec5daa7413853391714
Even the most modest 4C/8T system would run with the maximum 16 processes
due to the calculation being VCPU*2.
We devide amount of CPUs to number of threads for hyperthreaded CPUs
Change-Id: Iab8ed570cb6f60afc09f4ae47b97c49c8e61072d
Move it to the service setup host (defaults to utility[0]) instead
of the galera[0] host, and use galera_address (defaults to internal VIP)
as the endpoint instead of a local connection on the db host.
Change-Id: I1171b6a45563e4eccf45e8840e5dd332001ff3b7
This patch aims to add a prefix for memcached_server
on each role to give the ability for deployers to
override the location of memcached cluster. I.e users
wants to create a single memcached cluster with k8s
for each service.
We also add pymemcache based on [1]
[1] https://review.opendev.org/711429
Change-Id: I2d0b500f002e457abcb1d5fe96bf554f96e5700e
This patch moves aodh-api from usage of apache with mod_wsgi
to uWSGI role, which means unification across another roles and
reduced maintenance costs
During migration period tasks that ensures apache won't listen
on panko_service_port are present, but they are supposed to be removed
after train release.
Depends-On: https://review.opendev.org/678025/
Change-Id: I9377d46b4b79f79dbf448b23c67ff21b80714b6c
The variables aodh_developer_mode and aodh_venv_download
no longer carry any meaning. This review changes glance to
do the equivalent of what developer_mode was all the time,
meaning that it always builds the venv and never requires
the repo server, but it will use a repo server when available.
As part of this, we move the source build out of its own file
because it's now a single task to include the venv build role.
This is just to make it easier to follow the code.
Change-Id: I04e119c1404d8681ef7e5d964c95fbeb8970756b
In order to enable the service setup host python interpreter to
be changed easily, we make it a variable. This will be useful
when someone sets the service setup host to be the utility
container, because we'll be able to set this var by default.
Depends-On: https://review.openstack.org/632125
Change-Id: I8ab51d6035d06fea1ee1d7d6427be03115f41924
In order to radically simplify how we prepare the service
venvs, we use a common role to do the wheel builds and the
venv preparation. This makes the process far simpler to
understand, because the role does its own building and
installing. It also reduces the code maintenance burden,
because instead of duplicating the build processes in the
repo_build role and the service role - we only have it all
done in a single place.
We also change the role venv tag var to use the integrated
build's common venv tag so that we can remove the role's
venv tag in group_vars in the integrated build. This reduces
memory consumption and also reduces the duplication.
This is by no means the final stop in the simplification
process, but it is a step forward. The will be work to follow
which:
1. Replaces 'developer mode' with an equivalent mechanism
that uses the common role and is simpler to understand.
We will also simplify the provisioning of pip install
arguments when doing this.
2. Simplifies the installation of optional pip packages.
Right now it's more complicated than it needs to be due
to us needing to keep the py_pkgs plugin working in the
integrated build.
3. Deduplicates the distro package installs. Right now the
role installs the distro packages twice - just before
building the venv, and during the python_venv_build role
execution.
Depends-On: https://review.openstack.org/598957
Change-Id: I4cee1b0b7d5bc3fa53052dabe66e6acdb69afd18
Implements: blueprint python-build-install-simplification
Signed-off-by: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
The systemd journal would normally be populated with the standard out of
a service however with the use of uwsgi this is not actually happening
resulting in us only capturing the logs from the uwsgi process instead
of the service itself. This change implements journal logging in the
service config, which is part of OSLO logging.
OSLO logging docs found here: <https://docs.openstack.org/oslo.log/3.28.1/journal.html>
Change-Id: I7bd5419e7f02593a16614746a974b5f8ab3aa504
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
In order to reduce the packages required to pip install on to the hosts,
we allow the service setup to be delegated to a specific host, defaulting
to the deploy host. We also switch as many tasks as possible to using the
built-in Ansible modules which make use of the shade library.
The 'virtualenv' package is now installed appropriately by the openstack_hosts
role, so there's no need to install it any more. The 'httplib2' package is a
legacy Ansible requirement for the get_url/get_uri module which is no longer
needed. The keystone client library is not required any more now that we're
using the upstream modules. As there are no required packages left, the task
to install them is also removed.
Change-Id: Ia05fc004e11eada426bc81c45bbc3732b9126bbe
When the RPC and Notify service are the same, the credentials
must match - otherwise the tasks to create the user/password
will overwrite with each other.
If the two clusters are different, then the matching credentials
and vhost will not be a problem. However, if the deployer really
wishes to make sure they're different, then the vars can be
overridden.
Also, to ensure that the SSL value is consistently set in the
conf file, we apply the bool filter. We also use the 'notify'
SSL setting as the messaging system for Notifications is more
likely to remain rabbitmq in our default deployment with qrouterd
becoming the default for RPC messaging.
Change-Id: I1d56fc82eebfb2d2ed421fd1c589e1b912328c08
There is no record for why we implement the MQ vhost/user creation
outside of the role in the playbook, when we could do it inside the
role.
Implementing it inside the role allows us to reduce the quantity of
group_vars duplicated from the role, and allows us to better document
the required variables in the role. The delegation can still be done
as it is done in the playbook too.
In this patch we implement two new variables:
- aodh_oslomsg_rpc_setup_host
- aodh_oslomsg_notify_setup_host
These are used in the role to allow delegation of the MQ vhost/user
setup for each type to any host, but they default to using the first
member of the applicable oslomsg host group.
We also adjust some of the defaults to automatically inherit existing
vars set in group_vars form the integrated build so that we do not
need to do the wiring in the integrated build's group vars. We still
default them in the role too for independent role usage.
Change-Id: I9f0daa3c112cc34393357a30551bac71456089ef
The following packages are required in-order to run osprofiler.
these packages will provide deployers the ability to profile
a service on demand should they choose to enable the profile
functionality.
Depends-On: I3df2c670beeb78baaa1515bcd27e8f2b0d95b3a9
Change-Id: I7ac123678d7fcaf2c4f82ffdc1f3cad3ecfc7281
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
There is no record for why we implement the database creation outside
of the role in the playbook, when we could do it inside the role.
Implementing it inside the role allows us to reduce the quantity of
group_vars duplicated from the role, and allows us to better document
the required variables in the role. The delegation can still be done
as it is done in the playbook too.
In this patch we implement a new variable called 'aodh_db_setup_host'
which is used in the role to allow delegation of the database setup
task to any host, but defaults to the first member of the galera_all
host group.
Change-Id: I5e6a2857d14f957ef9167f90d4b2d3a4fec321dc
This introduces oslo.messaging variables that define the RPC and
Notify transports for the OpenStack services. These parameters
replace the rabbitmq values and are used to generate the messaging
transport_url for the service. The association of the messaging
backend server to the oslo.messaging services will then be
transparent to the aodh service.
This patch:
* Add oslo.messaging variables for RPC and Notify to defaults
* Update transport_url generation (add for notifications)
* Add oslo.messaging to tests inventory
* Update tests
* Add release note
* Update README and example
Depends-On: If4326a6848d2d32af284fdbb94798eb0b03734d5
Depends-On: I2b09145b60116c029fc85477399c24f94974b61d
Change-Id: I356e7256f5e8090f35dce8a02fd633638fd659fa
Distributions provide packages for the OpenStack services so we add
support for using these instead of the pip ones.
Change-Id: If6daa1bb784df46e83bbc118981240eb59a1409d
Implements: blueprint openstack-distribution-packages
This removes the systemd service templates and tasks from this role and
leverages a common systemd service role instead. This change removes a
lot of code duplication across all roles all without sacrificing features
or functionality. The intention of this change is to ensure uniformity and
reduce the maintenance burden on the community when sweeping changes are
needed.
Change-Id: I47287ce0deb45538894bd99e57c291c3ae7fa084
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
virtualenv-tools has a bug which gets triggered in gates: it can't
change the shebang of a virtualenv python bin/ files if they
were generated with a virtualenv script whose shebang ends with
python2 instead of python.
Because we can't modify virtualenv-tools, we use shell scripts
instead.
Change-Id: If934acbcfe81578366d1ecc0c5fec7c6de5b2040
Partial-Bug: #1741634
When 'aodh_galera_use_ssl' is True, use an encrypted connection to
the database using either a self-signed or user-provided CA certificate.
A new non-voting test has been added to verify that the role remains
functional when enabling SSL features.
Change-Id: I24f06aeacd92d6eead4100009301e0fb975bc552
Partial-Bug: 1667789
The keystonemiddleware library recently switched to using the
cryptography library over pycrypto, which was unmaintained. See
Iced7f5115e49ccf4f7f5bf6813cb5988b95c248b
Change-Id: I6c811ff6f0647d4c6561b5b698490bd9100de93c
Co-Authored-By: Nolan Brubaker <nolan.brubaker@rackspace.com>
The sqlalchemy upper constraint has recently been removed in
https://review.openstack.org/#/c/460256/4. This commit changes
the constraint in the role default to match the change.
Closes-Bug: #1686483
Change-Id: I22228de6e29c3cf6ad3afa459ae29d3f76c00a28
This is so the repo-build role respects the constraints specified
for extra dependencies in the setup.cfg file.
Change-Id: I2774681cc626f94e6de6b3617e6b92a78edaea33
Closes-Bug:#1682214
This creates a specific slice which all OpenStack services will operate
from. By creating an independent slice these components will be governed
away from the system slice allowing us to better optimise resource
consumption.
See the following for more information on slices:
* https://www.freedesktop.org/software/systemd/man/systemd.slice.html
See for following for more information on resource controls:
* https://www.freedesktop.org/software/systemd/man/systemd.resource-control.html
Tools like ``systemd-cgtop`` and ``systemd-cgls`` will now give us
insight into specific processes, process groups, and resouce consumption
in ways that we've not had access to before. To enable some of this reporting
the accounting options have been added to the [Service] section of the unit
file.
Change-Id: I2b5c6189a6eabbdb7854dcee97edf47ef03f8757
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Users can configure the number of worker processes. However when it's
not specified the calculated number of workers can get too large on
hosts with a large number of CPUs.
Change-Id: Ieb6202e92ce1dcb6f94affd550bd6ce2065ef4bc
Fixes the ability to deploy a venv in cases where:
1) developer_mode is not enabled
2) A cached venv is not downloaded from the repo server
Additional cleanup to the developer_mode venv deployment
logic is implemented by adding a *_venv_download var
which is used to decouple developer_mode from the
cached venv extraction process so that a deployer
can force venv builds in-place (disable cached
venv usage) without enabling developer mode
constraints.
Change-Id: I7ac0d8e732f0e97bf667e4428238acc2dc8bee0b
Dmitriy Rabotyagov