This change implements and enables by default quorum support
for rabbitmq as well as providing default variables to globally tune
it's behaviour.
In order to ensure upgrade path and ability to switch back to HA queues
we change vhost names with removing leading `/`, as enabling quorum
requires to remove exchange which is tricky thing to do with running
services.
Change-Id: Id5f6cabed7ec035845865d6d5facc63590c56d43
With update of ansible-lint to version >=6.0.0 a lot of new
linters were added, that enabled by default. In order to comply
with linter rules we're applying changes to the role.
With that we also update metdata to reflect current state.
Depends-On: https://review.opendev.org/c/openstack/ansible-role-systemd_service/+/888223
Change-Id: I6bbb97cd2f1daac42269a335014eb6cf7f28d24e
By overriding the variable `swift_backend_ssl: True` HTTPS will
be enabled, disabling HTTP support on the swift backend api.
The ansible-role-pki is used to generate the required TLS
certificates if this functionality is enabled.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/879085
Change-Id: Idb7882775a90ada9bb9e1450168916c73bf8ae4b
Implement support for service_tokens. For that we convert
role_name to be a list along with renaming corresponding variable.
Additionally service_type is defined now for keystone_authtoken which
enables to validate tokens with restricted access rules
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/845690
Change-Id: If34e0170ea0e0f7727cfadba982f3c7dae6ae216
Ceilometer middleware now accepts also project names,
so it's not required to gain project UUID [1].
We also switching gnocchi project name to `service` for all scenarios
as it's already whitelisted by default [2]
[1] e2bf485044
[2] 082dabb1b7/ceilometermiddleware/swift.py (L158)
Closes-Bug: #1879192
Change-Id: I40d3178d1b3bd1234ac34c6fa1d0d2bcb7160720
This reverts commit 3e151d97ad.
Reason for revert: Upstream bug has been fixed
Change-Id: I5498030487677523ae71ae2f9a13074ab4552204
Related-Bug: #1872553
Closes-Bug: #1921354
Instead of overriding each service separatelly it might make
sense for deployers to define some higher level variable that
will be used first or fallback to default variable.
Change-Id: Ia08e7c0c1a7398c6edac32dc04cb90e50b502059
Even the most modest 4C/8T system would run with the maximum 16 processes
due to the calculation being VCPU*2.
We devide amount of CPUs to number of threads for hyperthreaded CPUs
Change-Id: Ie3c590d413b001ac9ccdb6522c9654b4372b5e10
Move the percent character to the template from the string to allow
setting either % or %% as needed to work around the bug linked below.
This requires each service 'family' to have two versions of it's config
file.
Bug: https://bugs.launchpad.net/swift/+bug/1872553
Change-Id: I68f276224c51d5682d77123eae697767fadf9b19
Signed-off-by: Matthew Thode <mthode@mthode.org>
Current services were infinitelly spawning replicator process
instead running a server with another config file
Change-Id: I3310c11c0be38ae72b1f0bf94a849587fb8e5cf9
Closes-Bug: #1859159
This patch aims to add a prefix for memcached_server
on each role to give the ability for deployers to
override the location of memcached cluster. I.e users
wants to create a single memcached cluster with k8s
for each service.
We also add pymemcache based on [1]
[1] https://review.opendev.org/711429
Change-Id: If17dd627708f03824939ba062b498675253b11d7
This patch aims to migrate service from usage of regular syslog files
to journald.
By this we mean dropping rsyslog client installation. log_address is set
by default to /dev/log, which is served by journald.
Change-Id: I6dd0d77004394bb1ad674b53538b0679b056bb0f
The swift3 middleware has been deprecated in replacement with the
s3api middleware instead. This removes all the swift3 references
to hopefully enable someone to cleanly add s3api eventually.
Change-Id: I3a8a1ab861ec81b1f4f8dbc02a6a332d4ce495b8
The variables swift_developer_mode and swift_venv_download
no longer carry any meaning. This review changes swift to
do the equivalent of what developer_mode was all the time,
meaning that it always builds the venv and never requires
the repo server, but it will use a repo server when available.
As part of this, we move the source build out of its own file
because it's now a single task to include the venv build role.
This is just to make it easier to follow the code.
Change-Id: I9233e4daff0034339750477fd21d5cfa181afd83
In order to enable the service setup host python interpreter to
be changed easily, we make it a variable. This will be useful
when someone sets the service setup host to be the utility
container, because we'll be able to set this var by default.
Change-Id: Icb3ea9ad782218c357ec0e3c577ecbe6fbf60461
In order to radically simplify how we prepare the service
venvs, we use a common role to do the wheel builds and the
venv preparation. This makes the process far simpler to
understand, because the role does its own building and
installing. It also reduces the code maintenance burden,
because instead of duplicating the build processes in the
repo_build role and the service role - we only have it all
done in a single place.
We also change the role venv tag var to use the integrated
build's common venv tag so that we can remove the role's
venv tag in group_vars in the integrated build. This reduces
memory consumption and also reduces the duplication.
This is by no means the final stop in the simplification
process, but it is a step forward. The will be work to follow
which:
1. Replaces 'developer mode' with an equivalent mechanism
that uses the common role and is simpler to understand.
We will also simplify the provisioning of pip install
arguments when doing this.
2. Simplifies the installation of optional pip packages.
Right now it's more complicated than it needs to be due
to us needing to keep the py_pkgs plugin working in the
integrated build.
3. Deduplicates the distro package installs. Right now the
role installs the distro packages twice - just before
building the venv, and during the python_venv_build role
execution.
Depends-On: https://review.openstack.org/598957
Change-Id: Iecb64d28afe3acfbae7060af55c1a891310e5ef4
Implements: blueprint python-build-install-simplification
Signed-off-by: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
This reverts commit 6ee96f224c.
This variable is no longer referenced anywhere, so we can remove it.
Change-Id: I70a262020918d51b20486cf819dc9131abfce7f1
In order to reduce the packages required to pip install on to the hosts,
we allow the service setup to be delegated to a specific host, defaulting
to the deploy host. We also switch as many tasks as possible to using the
built-in Ansible modules which make use of the shade library.
The 'virtualenv' package is now installed appropriately by the openstack_hosts
role, so there's no need to install it any more. The 'httplib2' package is a
legacy Ansible requirement for the get_url/get_uri module which is no longer
needed. The keystone client library is not required any more now that we're
using the upstream modules. As there are no required packages left, the task
to install them is also removed.
Depends-On: https://review.openstack.org/582359
Depends-On: https://review.openstack.org/587376
Change-Id: I68f3a0bf2b7a3a12cbf40d7d6a853d5b4c6dd0f3
The integrated build has no value for swift_rabbitmq_password
and yet the oslomsg vars reference it. Given that the integrated
gate is quite hard to merge anything into right now - set a
default value here temporarily until we can merge a fix there.
Change-Id: I8c13fe063cd5ec454196bc630cb6362f97c4f146
There is no record for why we implement the MQ vhost/user creation
outside of the role in the playbook, when we could do it inside the
role.
Implementing it inside the role allows us to reduce the quantity of
group_vars duplicated from the role, and allows us to better document
the required variables in the role. The delegation can still be done
as it is done in the playbook too.
In this patch we implement the new variable:
- swift_oslomsg_notify_setup_host
This is used in the role to allow delegation of the MQ vhost/user
setup for each type to any host, but they default to using the first
member of the applicable oslomsg host group.
We also adjust some of the defaults to automatically inherit existing
vars set in group_vars form the integrated build so that we do not
need to do the wiring in the integrated build's group vars. We still
default them in the role too for independent role usage.
Depends-On: https://review.openstack.org/584630
Change-Id: I3d5e00e090d37ea7aa95460965749ef066b63b23
The 'httplib2' package is a legacy Ansible requirement for the
get_url/get_uri module which is no longer needed.
Change-Id: I420dfd74e78963486ed9d5a66b8c3b72372b5ae6
The following packages are required in-order to run osprofiler.
these packages will provide deployers the ability to profile
a service on demand should they choose to enable the profile
functionality.
Change-Id: If60f98fc069c040680f58658aacc63a156c4317f
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This introduces oslo.messaging variables that define the Notify transport
in place of the rabbitmq values.
This patch:
* Add oslo.messaging variable for Notify to defaults
* Update url generation
* Add oslo.messaging to inventory
* Add release note
Change-Id: I1c2e844c4c7a2256087bcc4521f970ca8e8c6b16
The keystoneclient package is being installed on the host by PIP but
that means that a whole bunch of required dependencies are being pulled
in as well.
This brings the host to a rather messed up state when installing
keystone from distro packages, since distribution and
PIP packages are being mixed together. We only need the client to
register the service with keystone so we can simply use the distro
package for that to avoid installing lots of PIP packages on the
host.
Change-Id: Id5d79db00e1a4aa4983aafd92c088ef8f13a7da0
Implements: blueprint openstack-distribution-packages
Distributions provide packages for the OpenStack services so we add
support for using these instead of the pip ones. However, functional
testing is not complete yet since it requires tempest to be in the swift
virtual environment which doesn't exist for distro installs. As such,
for functional testing to cover this method of installation it needs
to be changed quite a bit.
Change-Id: Id75e56440685df407a9991018dc07b5a75429082
Implements: blueprint openstack-distribution-packages
This removes the systemd service templates and tasks from this role and
leverages a common systemd service role instead. This change removes a
lot of code duplication across all roles all without sacrificing features
or functionality. The intention of this change is to ensure uniformity and
reduce the maintenance burden on the community when sweeping changes are
needed. The exterior role is built to be OSA compatible and may be pulled
into tree should we deem it necessary.
Change-Id: Icb7ca523cb19c560de5c84b0d60a06305029192c
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
virtualenv-tools has a bug which gets triggered in gates: it can't
change the shebang of a virtualenv python bin/ files if they
were generated with a virtualenv script whose shebang ends with
python2 instead of python.
Because we can't modify virtualenv-tools, we use shell scripts
instead.
Change-Id: I2dc9a507162fcd2323e19a3a2daa97c6db9cdcdb
Partial-Bug: #1741634
This is the time of the cycle where we update all the static
elements in the roles. PyPy has a static file version, so we
bump it.
Change-Id: I15c21c177c32dcae0a7f2710b50f91829be09586
Beginning with commit 6ffcc29 of swift3, the swift3 and s3token middlewares
must come between authtoken and keystoneauth in the swift proxy pipeline.
When 6ffcc29 was committed, reordering was optional at the expense of an
unneceessary call to keystone. However, the change becomes mandatory when using
keystone v3 tokens. Without this change, authtoken will strip the necessary
headers that s3token just added to the environment.
Change-Id: Ic7d07d869aa617ee00190a9aedf411017539b97b