Commit Graph

42 Commits

Author SHA1 Message Date
Dmitriy Rabotyagov 9b16df267a Add quorum queues support for service
This change implements and enables by default quorum support
for rabbitmq as well as providing default variables to globally tune
it's behaviour.

In order to ensure upgrade path and ability to switch back to HA queues
we change vhost names with removing leading `/`, as enabling quorum
requires to remove exchange which is tricky thing to do with running
services.

Change-Id: I49f9a18430f4912fe3e2fda36da6ad2acf6dde35
2023-10-20 12:32:00 +00:00
Dmitriy Rabotyagov fc0da79db5 Fix linters and metadata
With update of ansible-lint to version >=6.0.0 a lot of new
linters were added, that enabled by default. In order to comply
with linter rules we're applying changes to the role.

With that we also update metdata to reflect current state.

Depends-On: https://review.opendev.org/c/openstack/ansible-role-systemd_service/+/888223
Change-Id: I7163d2e68b1f0f97bd31d7734a99f74ed60b1bb5
2023-07-14 18:33:14 +02:00
Damian Dabrowski 7db1e9724b Add TLS support to masakari backends
By overriding the variable `masakari_backend_ssl: True` HTTPS will
be enabled, disabling HTTP support on the masakari backend api.

The ansible-role-pki is used to generate the required TLS
certificates if this functionality is enabled.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/879085
Change-Id: I2ea927dbfd7c9164b0f4d5fb793164ce4ad17094
2023-04-29 18:42:16 +02:00
Dmitriy Rabotyagov 1ac273041b Support service tokens
Implement support for service_tokens. For that we convert
role_name to be a list along with renaming corresponding variable.

Additionally service_type is defined now for keystone_authtoken which
enables to validate tokens with restricted access rules

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/845690
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/845994
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/845894
Change-Id: I3db09c840b448c665c9f2a16c78fe0e0a38f158a
2022-06-15 18:50:21 +02:00
Damian Dabrowski 859ca95e2c Database connection pooling improvements
- Implemented new variable ``connection_recycle_time`` responsible for SQLAlchemy's connection recycling
- Set new default values for db pooling variables which are inherited from the global ones.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/819424
Change-Id: I02997d0537ac24bdd261b8bce8f4a2e34e1db74a
2021-12-03 11:41:10 +01:00
Dmitriy Rabotyagov 224b2cf396 Refactor galera_use_ssl behaviour
With PKI role in place in most cases you don't need to explicitly
provide path to the CA file because PKI role ensures that CA is trusted
by the system overall. In the meanwhile in PyMySQL [1] you must either
provide CA file or cert/key or enable verify.

Since current behaviour is to provide path to the custom CA we expect
certificate being trusted overall. Thus we enable cert verification when
galera_use_ssl is True.

[1] 78f0cf99e5/pymysql/connections.py (L267)

Change-Id: I933e1edbd4260e2bd61bcceec3ddad8aea85353c
2021-09-21 15:44:37 +03:00
Zuul 1fe2df61d2 Merge "Use ansible_facts[] instead of fact variables" 2021-05-27 23:12:48 +00:00
Zuul 055eb191e5 Merge "Remove references to unsupported operating systems" 2021-05-26 11:08:27 +00:00
Zuul 0f98657941 Merge "Add variables for rabbitmq ssl configuration" 2021-05-18 14:31:00 +00:00
Jonathan Rosser 8884cbad52 Use ansible_facts[] instead of fact variables
See https://github.com/ansible/ansible/issues/73654

Change-Id: Ie827ed8b29b02fa268398be1c38b474a447d5c9d
2021-05-18 12:39:14 +00:00
Jonathan Rosser 9b4ac49fa9 Remove references to unsupported operating systems
All references to Gentoo, SUSE, Debian stretch and Centos-7  are removed.
Conditional tasks, ternary operators and variables are simplified where possible
OS specific variables files are generalised where possible

Change-Id: Ia712d170740c748002aa44d921ade8750997f8f0
2021-05-18 12:39:00 +00:00
Jonathan Rosser db3d249f8b Add variables for rabbitmq ssl configuration
Change-Id: I4f19472f3552a07d97c7cd4219c020bbfbc82137
2021-05-17 11:24:09 +00:00
Dmitriy Rabotyagov 4e26900819 Add masakari-introspectiveinstancemonitor support
Masakari has added introspectiveinstancemonitor for a while, however it
has not bee implemented in OSA.

Change-Id: I3ae31a3518f5580aadcd9935dda9f1bdabb6ae7e
2021-05-03 14:10:44 +03:00
Dmitriy Rabotyagov f8a6ebd134 Allow to configure corosync ports
We add 2 extra variables, to make corosync port configurable along with
enablement of ipmi checks.

Change-Id: I970bdaad0af79599d8a7c8cf95f89d273eb791de
2021-05-03 13:41:58 +03:00
Dmitriy Rabotyagov 08fbfade2a [goal] Deprecate the JSON formatted policy file
As per the community goal of migrating the policy file
the format from JSON to YAML[1], we need to replace policy.json to
policy.yaml and remove deprecated policy.json.

config_template has been choosen instead of the copy, since it can
properly handle content that has been lookuped.

We make a separate task not to restart service when it's not needed.

[1] https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html

Change-Id: I8c89b6d193f221faad4db9e1d0b0152f262b823b
2021-03-24 12:12:48 +00:00
Jonathan Rosser 9dcf91047c Switch default virtualenv to python3
Change-Id: I6d5edd4fd8f3db0839fc1ce3cad8832730ece696
2021-03-10 08:55:58 +00:00
Jonathan Rosser 58a8b65e8c Move masakari pip packages from constraints to requirements
This is necessary to use the new pip resolver

Change-Id: Ibace1c10a3d66ca7f1084ee53f27819a659e1246
2021-01-25 10:11:22 +00:00
Dmitriy Rabotyagov 2454228066 Use global service variables
Instead of overriding each service separatelly it might make
sense for deployers to define some higher level variable that
will be used first or fallback to default variable.

Change-Id: Ie526c3fed0d6a1c706d5ef17e9c1bcb38418e861
2021-01-08 17:39:08 +02:00
Zuul f6cd496e8e Merge "Add masakari-monitors to constraints" 2020-08-24 13:49:17 +00:00
Dmitriy Rabotyagov 8374993aff Use the utility host for db setup tasks
Move it to the service setup host (defaults to utility[0]) instead
of the galera[0] host, and use galera_address (defaults to internal VIP)
as the endpoint instead of a local connection on the db host.

Change-Id: I3347e9a33beff2f0dc57c429acd973f20789912c
2020-08-20 18:34:36 +03:00
Dmitriy Rabotyagov 5935901b8e Add masakari-monitors to constraints
masakari-monitors egg was missing from contraints, which resulted in
failures while installing it dependencies for py2 on stable branches

Depends-On: https://review.opendev.org/745572
Depends-On: https://review.opendev.org/739146
Change-Id: I06f8580cbcf7a2633077bb8429c90474837644f0
2020-08-10 17:13:32 +00:00
Dmitriy Rabotyagov 03fab4c697 Cleanup after repo_build and pip_install retirement
Change-Id: I6da14f03aee7b33cad2818f4cf1affe7504e3348
2020-05-12 22:23:54 +03:00
Dmitriy Rabotyagov 3f5f393d22 Use openstack_venv_python_executable
Make masakari respect set openstack_venv_python_executable
This also will create py3 venvs by default

Change-Id: I9d7abbeff0f68cad3b646cde270437c88554080c
2020-05-06 12:22:36 +03:00
Guilherme Steinmüller 235e1f05a0 Refactor memcached_servers
This patch aims to add a prefix for memcached_server
on each role to give the ability for deployers to
override the location of memcached cluster. I.e users
wants to create a single memcached cluster with k8s
for each service.

We also add pymemcache based on [1]

[1] https://review.opendev.org/711429

Change-Id: I5ce2be9e7a8a648cbb734a2e418d434e7991e298
2020-03-16 14:14:09 +00:00
Dmitriy Rabotyagov 4f8966a231 Use py3 for CentOS
Unfortunatelly CentOS 7 do not have libvirt library
for py3, so instead of symlinking them, we have to install devel package
and build inside venv.

Change-Id: I5a2fc38e86debfb9a73394552b79f83e56dd1231
2020-01-03 17:53:32 +02:00
Dmitriy Rabotyagov f6a45448ad Replace git.openstack.org with opendev.org
This patch replaces git.openstack.org with opendev.org as redirection
from old path was enabled.
Also we change upper constraints url due to [1]

[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-May/006478.html

Change-Id: I87cf6db6e5497ea3c69242b4eaffad7f533448c7
2019-11-14 16:57:58 +02:00
Jonathan Rosser fe7caf3d25 Add global override for service bind address
Change-Id: I7c078883a78c68e3d7c6e68d415eefad4c10d872
2019-10-01 16:24:42 +00:00
Dmitriy Rabotyagov 9d8a95d03a Use systemd-journald instead of log files
This patch aims to migrate service from usage of regular syslog files
to journald. We also disable uwsgi logging, since it dublicates
requests that are logged by service itself.

Change-Id: I4168cd484ef52cc7b47efdbcc26d3d3550d28654
2019-07-17 16:30:20 +03:00
Dmitriy Rabotjagov 40b8f02d61 Fixed variable in example playbook
Fixed variable from glance_galera_address inside example playbook.
Also modified comment for masakari_monitor_corosync_multicast_interface

Change-Id: I8ca88549abedea124522a2290d678bbe65f818a9
2019-03-29 10:39:19 +02:00
Dmitriy Rabotjagov a9c05684e9 Update role for new source build process
The variables masakari_developer_mode and masakari_venv_download
no longer carry any meaning. This review changes masakari to
do the equivalent of what developer_mode was all the time,
meaning that it always builds the venv and never requires
the repo server, but it will use a repo server when available.

We also change include_tasks to import_tasks and include_role
to import_role so that the tags in the python_venv_build role
will work.

Depends-On: https://review.openstack.org/#/c/647571/
Change-Id: I850ba83a8683be45988f2d9eafcc6a2e8ad4d8b2
2019-03-27 15:46:15 +02:00
Jesse Pretorius 0ba806a9b5 Enable overriding the service setup host python interpreter
In order to enable the service setup host python interpreter to
be changed easily, we make it a variable. This will be useful
when someone sets the service setup host to be the utility
container, because we'll be able to set this var by default.

Change-Id: I0f34a17274ec1abcb43efffcba196049b6161edb
2018-11-30 16:23:24 +00:00
Dmitriy Rabotjagov d0e8674f92 Basic implementation of masakari-monitors
With this change masakari-monitors will be installed on hosts
which are in group masakari-monitor (supposed to be nova compute hosts)

hostmonitor depends on pacemaker with corosync, which are implemented by
https://github.com/mit-scripts/ansible-pacemaker-corosync

Masakari api/engine config was simplified and unneccesary defaults
were removed from it.

Role now uses default ``systemd_service`` role for systemctl configuration

Co-Authored-By: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
Implements: blueprint masakari-ansible-plugin
Change-Id: I334877c0111a45d3f3a74e7f56931786f4301713
2018-11-06 19:46:53 +02:00
Jesse Pretorius e5aaaef746 Use a common python build/install role
In order to radically simplify how we prepare the service
venvs, we use a common role to do the wheel builds and the
venv preparation. This makes the process far simpler to
understand, because the role does its own building and
installing. It also reduces the code maintenance burden,
because instead of duplicating the build processes in the
repo_build role and the service role - we only have it all
done in a single place.

We also change the role venv tag var to use the integrated
build's common venv tag so that we can remove the role's
venv tag in group_vars in the integrated build. This reduces
memory consumption and also reduces the duplication.

This is by no means the final stop in the simplification
process, but it is a step forward. The will be work to follow
which:

1. Replaces 'developer mode' with an equivalent mechanism
   that uses the common role and is simpler to understand.
   We will also simplify the provisioning of pip install
   arguments when doing this.
2. Simplifies the installation of optional pip packages.
   Right now it's more complicated than it needs to be due
   to us needing to keep the py_pkgs plugin working in the
   integrated build.

Depends-On: https://review.openstack.org/598957
Change-Id: I676ddbb7028ad203e4ca3f3e1cee7ff1435d23a2
Implements: blueprint python-build-install-simplification
Signed-off-by: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
2018-09-03 12:00:42 +00:00
Jesse Pretorius e517634e1e Add missing masakari_venv_download_url default
There are references to the masakari_venv_download_url var,
but it's not set anywhere.

Change-Id: I28ba50fc0b675dcb9241f97f3930e4fb33ac4c2e
Co-Authored-By: Dmitriy R <dmitriy.r@sitevalley.com>
2018-08-01 18:40:48 +00:00
Jesse Pretorius 8585cb3d3a Move MQ vhost/user creation into role
There is no record for why we implement the MQ vhost/user creation
outside of the role in the playbook, when we could do it inside the
role.

Implementing it inside the role allows us to reduce the quantity of
group_vars duplicated from the role, and allows us to better document
the required variables in the role. The delegation can still be done
as it is done in the playbook too.

In this patch we implement two new variables:
- masakari_oslomsg_rpc_setup_host
- masakari_oslomsg_notify_setup_host

These are used in the role to allow delegation of the MQ vhost/user
setup for each type to any host, but they default to using the first
member of the applicable oslomsg host group.

We also adjust some of the defaults to automatically inherit existing
vars set in group_vars form the integrated build so that we do not
need to do the wiring in the integrated build's group vars. We still
default them in the role too for independent role usage.

Finally, we remove the test mq setup tasks and clean up any unused
or unnecessary variables configured in tests.

We also rename the ubuntu-16.04.yml file to ubuntu.yml to cover both
xenial and bionic. This has become necessary because the
'Gather variables for each operating system' task in the galera_client
role is picking up this role's vars file instead of its own.

Change-Id: Ibf05d527b6dbed8a10ecf0b64cfb161b66295d35
2018-08-01 18:52:35 +01:00
Andy Smith bb88051ec4 Update to use oslo.messaging service for RPC and Notify
This introduces oslo.messaging variables that define the RPC and
Notify transports for the OpenStack services. These parameters replace
the rabbitmq values and are used to generate the messaging
transport_url for the service. The association of the messaging
backend server to the oslo.messaging services will then be transparent
to the masakari service.

This patch:
* Add oslo.messaging variables for RPC and Notify to defaults
* Update transport_url generation in conf
* Add oslo.messaging to tests inventory and update tests
* Install extra packages for optional drivers

Change-Id: I127ce216cfb7b4d5755b8f0a68406bbd251fbdd2
2018-07-26 10:42:42 +00:00
Dmitriy R 3d06e07f9c Added required defaults, env example, Manage LB hook
I've decided to add env and required secrets example for easier stratup.
Also added missing variables to defaults, as playbook was just failiing without them.
Variables regarding rabbitmq were missing.
Also, as masakari_venv_download_url is not defined by default,
masakari_venv_download had been set to false.

Handlers were missing Manage LB task and had wrong variable -
masakari-services instead of masakari_services

Functional test was missing USER system environment variable.
Also test inventory had mistakes.

test-masakari-functions.yml wasn't able to pass test due to error.
Functional tests moved to voting.

Change-Id: I6644d576177f441ca59e9221ce9a2e5b7cc0fc46
2018-07-19 06:42:32 +00:00
Jesse Pretorius faf5f262d5 Execute service setup against a delegated host using Ansible built-in modules
In order to reduce the packages required to pip install on to the hosts,
we allow the service setup to be delegated to a specific host, defaulting
to the deploy host. We also switch as many tasks as possible to using the
built-in Ansible modules which make use of the shade library.

The 'virtualenv' package is now installed appropriately by the openstack_hosts
role, so there's no need to install it any more. The 'httplib2' package is a
legacy Ansible requirement for the get_url/get_uri module which is no longer
needed. The keystone client library is not required any more now that we're
using the upstream modules. The masakari client is not used on the host, so
it serves no purpose. As there are no required packages left, the task to
install them is also removed.

Change-Id: I5f4339b322b967fcfd326c7442d634abf8b6cb05
2018-07-12 17:42:15 +01:00
Jesse Pretorius 056208c580 Move database creation into role
There is no record for why we implement the database creation outside
of the role in the playbook, when we could do it inside the role.

Implementing it inside the role allows us to reduce the quantity of
group_vars duplicated from the role, and allows us to better document
the required variables in the role. The delegation can still be done
as it is done in the playbook too.

In this patch we implement a new variable called 'masakari_db_setup_host'
which is used in the role to allow delegation of the database setup
task to any host, but defaults to the first member of the galera_all
host group. We also document the variable 'masakari_galera_address' which
has been used for a long time, but never documented. A bunch of unused
variables have also been removed.

Change-Id: I6c1e4e32681cbb592f6daa805501031bb84e6e0c
2018-06-28 15:10:47 +01:00
Kevin Carter 4ad997de0a
Add packages required for osprofiler
The following packages are required in-order to run osprofiler.
these packages will provide deployers the ability to profile
a service on demand should they choose to enable the profile
functionality.

Change-Id: Ie9d2909c2e0d3b6951dea14013649a66ce93e7af
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-06-11 22:58:28 -05:00
Jean-Philippe Evrard ab6cfd1ef1 Add Maskari base jobs
This introduces the most basic jobs for masakari.

Change-Id: Ie62ea7ec96b8963b5d3815a9241cce964e8fafc2
2018-05-08 15:39:48 +05:30
nirajsingh 130e4c0e29 First commit 2018-04-16 17:02:09 +05:30