By overriding the variable `gnocchi_backend_ssl: True` HTTPS will
be enabled, disabling HTTP support on the gnocchi backend api.
The ansible-role-pki is used to generate the required TLS
certificates if this functionality is enabled.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/879085
Change-Id: Ie2c824052b0024d440b20febb34b6bde22f4fac2
Implement support for service_tokens. For that we convert
role_name to be a list along with renaming corresponding variable.
Additionally service_type is defined now for keystone_authtoken which
enables to validate tokens with restricted access rules
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/845690
Change-Id: Iaed7f2b4a724aed0f4165e32f3d40aac9d74edd7
As of today we didn't manage amount of gnocchi-metricd that equal to
amount of CPU on host. So things can go off regarding CPU and memory
consumption. For better control of ressources we add a variable to
control the number of workers.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/846349
Change-Id: Idcaec649a4de03f5714c61fda694ada45da41dbc
To provide more convenience in configuring redis as Gnocchi driver,
we add 2 variables that can be used to control
URL to Redis when it's used for storage or incoming data.
Related-Bug: #1955676
Change-Id: Iba5186df3656c116cba48c3be0e39e87ddcb727f
With PKI role in place in most cases you don't need to explicitly
provide path to the CA file because PKI role ensures that CA is trusted
by the system overall. In the meanwhile in PyMySQL [1] you must either
provide CA file or cert/key or enable verify.
Since current behaviour is to provide path to the custom CA we expect
certificate being trusted overall. Thus we enable cert verification when
galera_use_ssl is True.
[1] 78f0cf99e5/pymysql/connections.py (L267)
Change-Id: Id986985e623896698d54496a8f0608e59516eec5
Previously we used different from other services project name to protect
swift from looping. However nowadays ceilometer middleware does
exclude service project by default. So no futher actions needed
unless deployer want to measure service project excluding gnocchi
In this scenario, deployer still can define custom
`gnocchi_service_project_name` in their user_variables.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/791107
Change-Id: Ic0ffa6908bfc55ffdb41ea9b8e7435e4dc88ddca
Related-Bug: #1879192
Instead of overriding each service separatelly it might make
sense for deployers to define some higher level variable that
will be used first or fallback to default variable.
Change-Id: I022d52c39c706fd9f82b9efcdcdc589a2e971d24
Even the most modest 4C/8T system would run with the maximum 16 processes
due to the calculation being VCPU*2.
We devide amount of CPUs to number of threads for hyperthreaded CPUs
Change-Id: I3145692351cadf5c1cd835fe87e1206ee5901164
Move it to the service setup host (defaults to utility[0]) instead
of the galera[0] host, and use galera_address (defaults to internal VIP)
as the endpoint instead of a local connection on the db host.
Depends-On: https://review.opendev.org/754718
Depends-On: https://review.opendev.org/755258
Change-Id: I5098673bdf84d1c2b13ca87e474a5c598260ae94
This patch aims to add a prefix for memcached_server
on each role to give the ability for deployers to
override the location of memcached cluster. I.e users
wants to create a single memcached cluster with k8s
for each service.
We also add pymemcache based on [1]
[1] https://review.opendev.org/711429
Change-Id: Iefac95507d189b86371026e17465adf15b1bf410
Move service to use uWSGI role instead of iternal task for uwsgi
deployment. This aims to ease the maintenance of uWSGI and speedup
metal deployments as the same uwsgi environment will be used
across all services.
Change-Id: Iec03bd79279e694678336880460bcb83f68d9780
This patch moves gnocchi-api from usage of apache with mod_wsgi
to uWSGI, which means unification across another roles and
reduced maintenance costs
During migration period tasks that ensures apache won't listen
on gnocchi_service_port are present, but they are supposed to be removed
after train release.
Depends-On: https://review.opendev.org/671988
Change-Id: I06bbcb2f15108fc517742208ac5291719627ffe2
This patch aims to migrate service from usage of regular syslog files
to journald. We also disable uwsgi logging, since it dublicates
requests that are logged by service itself.
Change-Id: I3f6a1cee0e98372881b015ebf06e405c79495fe1
The variables gnocchi_developer_mode and gnocchi_venv_download
no longer carry any meaning. This review changes gnocchi to
do the equivalent of what developer_mode was all the time,
meaning that it always builds the venv and never requires
the repo server, but it will use a repo server when available.
We also change include_tasks to import_tasks so that the
tags in the python_venv_build role will work.
Change-Id: I6c43b8c9e627b9e10deda4d500da21eb29bb9206
This removes the systemd service templates and tasks from this role and
leverages a common systemd service role instead. This change removes a
lot of code duplication across all roles all without sacrificing
features or functionality. The intention of this change is to ensure
uniformity and reduce the maintenance burden on the community when
sweeping changes are needed. The exterior role is built to be OSA
compatible and may be pulled into tree should we deem it necessary.
Change-Id: I54e3063d6e641a785377f9039641072f8001cf24
In order to enable the service setup host python interpreter to
be changed easily, we make it a variable. This will be useful
when someone sets the service setup host to be the utility
container, because we'll be able to set this var by default.
Change-Id: I04e853fb582d7b39708a2fb6ed854b4e458f06d9
In order to radically simplify how we prepare the service
venvs, we use a common role to do the wheel builds and the
venv preparation. This makes the process far simpler to
understand, because the role does its own building and
installing. It also reduces the code maintenance burden,
because instead of duplicating the build processes in the
repo_build role and the service role - we only have it all
done in a single place.
We also change the role venv tag var to use the integrated
build's common venv tag so that we can remove the role's
venv tag in group_vars in the integrated build. This reduces
memory consumption and also reduces the duplication.
This is by no means the final stop in the simplification
process, but it is a step forward. The will be work to follow
which:
1. Replaces 'developer mode' with an equivalent mechanism
that uses the common role and is simpler to understand.
We will also simplify the provisioning of pip install
arguments when doing this.
2. Simplifies the installation of optional pip packages.
Right now it's more complicated than it needs to be due
to us needing to keep the py_pkgs plugin working in the
integrated build.
Depends-On: https://review.openstack.org/598957
Change-Id: I7a6acaa94265b21fb886a775c3b5b86a4142a905
Implements: blueprint python-build-install-simplification
Signed-off-by: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
Into gnocchi_pip_packages were added kazoo and redis packages.
These services are adviced by Tooz as coordinators backends.
Coordination is required to get metrics processing distribution among
service containers/instances.
Change-Id: Iaa608bf6690f35b1aa4afd08b1d60b0752230c4e
In order to reduce the packages required to pip install on to the hosts,
we allow the service setup to be delegated to a specific host, defaulting
to the deploy host. We also switch as many tasks as possible to using the
built-in Ansible modules which make use of the shade library.
The 'virtualenv' package is now installed appropriately by the openstack_hosts
role, so there's no need to install it any more. The 'httplib2' package is a
legacy Ansible requirement for the get_url/get_uri module which is no longer
needed. The keystone client library is not required any more now that we're
using the upstream modules. As there are no required packages left, the task
to install them is also removed.
With the dependent patches, the openstack_openrc role is now executed once
on the designated host, so it is no longer required as a meta-dependency for
the role.
Depends-On: https://review.openstack.org/579233
Depends-On: https://review.openstack.org/579959
Change-Id: I4131312eea8c743e7803ccc622b7642c6082a4c8
The following packages are required in-order to run osprofiler.
these packages will provide deployers the ability to profile
a service on demand should they choose to enable the profile
functionality.
Change-Id: I3c03d998a2c770398b1be218e8c94e9eee7a527b
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
There is no record for why we implement the database creation outside
of the role in the playbook, when we could do it inside the role.
Implementing it inside the role allows us to reduce the quantity of
group_vars duplicated from the role, and allows us to better document
the required variables in the role. The delegation can still be done
as it is done in the playbook too.
In this patch we implement a new variable called 'gnocchi_db_setup_host'
which is used in the role to allow delegation of the database setup
task to any host, but defaults to the first member of the galera_all
host group. We also document the variable gnocchi_galera_address which
has been used for a long time, but never documented.
Change-Id: Idff20080d825e5afd071cbc03055f96c920aff9d
Instead of manually installing keystonemiddleware, there is an
existing extra when installing which takes care of adding the
tested keystonemiddleware requirement.
Change-Id: I0ac3ae79e30db6d553035a90812d2c7600bfc606
At the moment, all drivers are being installed for Gnocchi even
if some of them are not used. This also means that some libraries
are being built which are not needed (such as for Ceph or Swift
when using file storage).
This patch switches the behaviour to install only the extra for
the driver in use.
Change-Id: I6e9f938885036d2fabc81094e01e220b09e462dd
virtualenv-tools has a bug which gets triggered in gates: it can't
change the shebang of a virtualenv python bin/ files if they
were generated with a virtualenv script whose shebang ends with
python2 instead of python.
Because we can't modify virtualenv-tools, we use shell scripts
instead.
Change-Id: I72dc1c78ac34d405f661d4347d94236b6b82d7da
Partial-Bug: #1741634
When 'gnocchi_galera_use_ssl' is True, use an encrypted connection to
the database using either a self-signed or user-provided CA certificate.
A new non-voting test has been added to verify that the role remains
functional when enabling SSL features.
Change-Id: Ib5a87e9366cd9e5a78ccb774ced46e1791e7691d
Partial-Bug: 1667789
The keystonemiddleware library recently switched to using the
cryptography library over pycrypto, which was unmaintained. See
Iced7f5115e49ccf4f7f5bf6813cb5988b95c248b
Change-Id: Id7aec616e632e53251b4d958a36abd680843b9bd
Co-Authored-By: Nolan Brubaker <nolan.brubaker@rackspace.com>
Gnocchi moved out of the OpenStack namespace, and is now operating from
github.com/gnocchixyz/gnocchi.
Change-Id: I07167d9877192b0d6ac3cff32a3f4e4ea5105402
The current mechanism uses a lookup, causing the
fetch of the default templates to happen via curl
from the deployment node. This causes problems if
the deployment node does not have access to the
repo server web service, which may be the case in
high security environments.
This patch changes the mechanism to only use the
lookup module for the file content that is on the
deployment node, then falls back to using the uri
module to fetch the default content. This ensures
that the deployment node does not have to reach
into the environment for the content.
Change-Id: Ia067a7cd2bc4460462b55f4631eba6e8eb693b0f
This creates a specific slice which all OpenStack services will operate
from. By creating an independent slice these components will be governed
away from the system slice allowing us to better optimise resource
consumption.
See the following for more information on slices:
* https://www.freedesktop.org/software/systemd/man/systemd.slice.html
See for following for more information on resource controls:
* https://www.freedesktop.org/software/systemd/man/systemd.resource-control.html
Tools like ``systemd-cgtop`` and ``systemd-cgls`` will now give us
insight into specific processes, process groups, and resouce consumption
in ways that we've not had access to before. To enable some of this reporting
the accounting options have been added to the [Service] section of the unit
file.
Change-Id: Ife7d6e3c0c11818d80484cf67a887026aaedb92c
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
It is no longer necessary to try to create legacy resources.
Ceilometer is now supposed to be capable of doing this for us.
Change-Id: I213ccbccebb84ac66ebc1f7f5cfd958ba6ee9d1b
Related-Change: Ia7d4ea3d1b9b631ac4c399ae7245e42a531f862a
Users can configure the number of worker processes. However when
it's not specified the calculated number of workers can get too
large on hosts with a large number of CPUs.
Change-Id: Iaa0cd5eba460d50cae18ba25f88a2d2c57a5cc85
Dmitriy Rabotyagov