With efforts to create a resources in same, unified way,
we convert tempest role to use openstack_resources
for creating and managing openstack resources, like projects, flavors,
networks, images, etc. This should reduce maintenance costs
in case of futher collection updates and unify approach.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/911377
Change-Id: I658e6218cf59650d0abfa491ebaa6d3c37db2993
This change implements and enables by default quorum support
for rabbitmq as well as providing default variables to globally tune
it's behaviour.
In order to ensure upgrade path and ability to switch back to HA queues
we change vhost names with removing leading `/`, as enabling quorum
requires to remove exchange which is tricky thing to do with running
services.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/896017
Change-Id: Ic5b425564e494502722106f94e406dc2ed69dcde
With update of ansible-lint to version >=6.0.0 a lot of new
linters were added, that enabled by default. In order to comply
with linter rules we're applying changes to the role.
With that we also update metdata to reflect current state.
Depends-On: https://review.opendev.org/c/openstack/ansible-role-systemd_service/+/888223
Change-Id: I68c3d98de3a0a4444c0c1ea0cc528901613a63bd
By overriding the variable `magnum_backend_ssl: True` HTTPS will
be enabled, disabling HTTP support on the magnum backend api.
The ansible-role-pki is used to generate the required TLS
certificates if this functionality is enabled.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/879085
Change-Id: Id3413eb53ab2948de61b2e67f85c5a19fc670434
As of today we didn't manage amount of magnum-conductors that equal to
amount of CPU on host. So things can go off regarding CPU and memory
consumption. For better control on resources we add variable to control
conductor workers.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/846151
Change-Id: I3eedd74717b3b621b4e0b6ae4a8df4ee6f1eb739
- Implemented new variable ``connection_recycle_time`` responsible for SQLAlchemy's connection recycling
- Set new default values for db pooling variables which are inherited from the global ones.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/819424
Change-Id: I305c6f4fb0b20e6e916fff7c912e8664733a902e
With PKI role in place in most cases you don't need to explicitly
provide path to the CA file because PKI role ensures that CA is trusted
by the system overall. In the meanwhile in PyMySQL [1] you must either
provide CA file or cert/key or enable verify.
Since current behaviour is to provide path to the custom CA we expect
certificate being trusted overall. Thus we enable cert verification when
galera_use_ssl is True.
[1] 78f0cf99e5/pymysql/connections.py (L267)
Change-Id: Ib9d0b810bf5aef475021f886dd19348548a7ec9a
Instead of overriding each service separatelly it might make
sense for deployers to define some higher level variable that
will be used first or fallback to default variable.
Change-Id: I295b6f99422ba81d14a04129939648d0722dd584
Even the most modest 4C/8T system would run with the maximum 16 processes
due to the calculation being VCPU*2.
We devide amount of CPUs to number of threads for hyperthreaded CPUs
Change-Id: I23fa200a017c290b342bfd4594cbffa5efd24566
In case `keystone-auth-enabled` is true in k8s template, magnum requires
keystone_auth_default_policy file to be present.
At this point we suggest creating corresponding roles by deployers
manually, since it's not enabled by default or used widely.
Change-Id: I77bfd3026e3168d7504ef3dc5214cfe706c525dd
Move it to the service setup host (defaults to utility[0]) instead
of the galera[0] host, and use galera_address (defaults to internal VIP)
as the endpoint instead of a local connection on the db host.
Depends-On: https://review.opendev.org/747981
Change-Id: Ia60bdb33889a3d0c2e080dab9badbdd777fcdbeb
This patch aims to add a prefix for memcached_server
on each role to give the ability for deployers to
override the location of memcached cluster. I.e users
wants to create a single memcached cluster with k8s
for each service.
We also add pymemcache based on [1]
[1] https://review.opendev.org/711429
Change-Id: If7bbef32ae1102ff586bd765052d984896bde43d
Creation of cluster templates is pretty routine operation wich
might be easily automated with ansible.
Depends-On: https://review.opendev.org/710245
Change-Id: Ib7f99714502ba94604b8f0bb997c77a768af6d1d
Move service to use uWSGI role instead of iternal task for uwsgi
deployment. This aims to ease the maintenance of uWSGI and speedup
metal deployments as the same uwsgi environment will be used
across all services.
Change-Id: I354ff3e81f4f4586aa2d52e1dcd8359c16a9e39a
This patch aims to migrate service from usage of regular syslog files
to journald. We also disable uwsgi logging, since it dublicates
requests that are logged by service itself.
Change-Id: I8be07495dd84f085de6d4409f2efd67a8359d82e
The variables magum_developer_mode and magnum_venv_download
no longer carry any meaning. This review changes magnum to
do the equivalent of what developer_mode was all the time,
meaning that it always builds the venv and never requires
the repo server, but it will use a repo server when available.
As part of this, we move the source build out of its own file
because it's now a single task to include the venv build role.
This is just to make it easier to follow the code.
Change-Id: I89177a702cd6f31f4b26b4d8fa260fd38a895ba0
Adding the ability to set region_name in keystone_authtoken section of
magnum.conf in the same wqy as for other services. Defaulting to
magnum_service_region.
Change-Id: I7f7e184c5eec6489505a6492ed2786a27bae29ab
Closes-Bug: #1819380
With this variable, users would be able to extend
the list of pip packages in case of needing an
extra pip package.
Currently if we need an extra pip package we need
to override the existing list.
Change-Id: Iaa94479ea780aa0b79b880327b807223f66874d6
In order to enable the service setup host python interpreter to
be changed easily, we make it a variable. This will be useful
when someone sets the service setup host to be the utility
container, because we'll be able to set this var by default.
Change-Id: Ib5245e5f13c8de41a6ca020be53781195903498d
Magnum currently tries to hit the first region that it finds
so it can fail to deploy clusters in multi-region deployments
non-determinsitically.
This patch makes sure that the region can be configured and
it defaults to the same one that Magnum is deployed on.
Change-Id: I136ba0caead3a5afe11152aaed7bae94af8906b4
This patch add the conditional inclusion of the notification
section of the service configuration. This ensures that oslo.messaging
notifications use the correct transport for deployments that have
separate rpc and notify messaging backends. For example, if the
transport_url is not provided in the notification section of the
service configuration, the transport_url specified in the default
section will be used instead.
This patch conditionally selects the notifier driver. The noop
driver will be selected when notification publishing is disabled.
The messagingv2 driver is selected when notification publishing is
enabled.
Change-Id: Iaadc0d852c003e653e00b4736ddc28f16ddfec5a
Closes-Bug: #1794320
The mysql-python package is no longer maintained. We are using
pymysql instead, so this package does not need to be installed.
Depends-On: https://review.openstack.org/602927
Change-Id: I194cea7b270702206b7c3661523804e351798e81
In order to radically simplify how we prepare the service
venvs, we use a common role to do the wheel builds and the
venv preparation. This makes the process far simpler to
understand, because the role does its own building and
installing. It also reduces the code maintenance burden,
because instead of duplicating the build processes in the
repo_build role and the service role - we only have it all
done in a single place.
We also change the role venv tag var to use the integrated
build's common venv tag so that we can remove the role's
venv tag in group_vars in the integrated build. This reduces
memory consumption and also reduces the duplication.
This is by no means the final stop in the simplification
process, but it is a step forward. The will be work to follow
which:
1. Replaces 'developer mode' with an equivalent mechanism
that uses the common role and is simpler to understand.
We will also simplify the provisioning of pip install
arguments when doing this.
2. Simplifies the installation of optional pip packages.
Right now it's more complicated than it needs to be due
to us needing to keep the py_pkgs plugin working in the
integrated build.
3. Deduplicates the distro package installs. Right now the
role installs the distro packages twice - just before
building the venv, and during the python_venv_build role
execution.
Depends-On: https://review.openstack.org/598957
Change-Id: Ib822c0a9c9a06d4505a898c48f1126974c643f2b
Implements: blueprint python-build-install-simplification
Signed-off-by: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
In order to reduce the packages required to pip install on to the hosts,
we allow the service setup to be delegated to a specific host, defaulting
to the deploy host. We also switch as many tasks as possible to using the
built-in Ansible modules which make use of the shade library.
The 'virtualenv' package is now installed appropriately by the openstack_hosts
role, so there's no need to install it any more. The 'httplib2' package is a
legacy Ansible requirement for the get_url/get_uri module which is no longer
needed. The keystone/glance/cinder client libraries are not required any more
now that we're using the upstream modules. As there are no required packages
left, the task to install them is also removed.
Depends-On: https://review.openstack.org/582359
Depends-On: https://review.openstack.org/582579
Depends-On: https://review.openstack.org/582957
Depends-On: https://review.openstack.org/583430
Change-Id: Id3b9d57981006d3f7abbb94af5f72214db3da6cb
When the RPC and Notify service are the same, the credentials
must match - otherwise the tasks to create the user/password
will overwrite with each other.
If the two clusters are different, then the matching credentials
and vhost will not be a problem. However, if the deployer really
wishes to make sure they're different, then the vars can be
overridden.
Also, to ensure that the SSL value is consistently set in the
conf file, we apply the bool filter. We also use the 'notify'
SSL setting as the messaging system for Notifications is more
likely to remain rabbitmq in our default deployment with qrouterd
becoming the default for RPC messaging.
Change-Id: If49da70b8b9768d48c09b8fcfca734c9531f1181
There is no record for why we implement the MQ vhost/user creation
outside of the role in the playbook, when we could do it inside the
role.
Implementing it inside the role allows us to reduce the quantity of
group_vars duplicated from the role, and allows us to better document
the required variables in the role. The delegation can still be done
as it is done in the playbook too.
In this patch we implement two new variables:
- magnum_oslomsg_rpc_setup_host
- magnum_oslomsg_notify_setup_host
These are used in the role to allow delegation of the MQ vhost/user
setup for each type to any host, but they default to using the first
member of the applicable oslomsg host group.
We also adjust some of the defaults to automatically inherit existing
vars set in group_vars form the integrated build so that we do not
need to do the wiring in the integrated build's group vars. We still
default them in the role too for independent role usage.
Depends-On: https://review.openstack.org/583430
Change-Id: I70c2a340d617bdc603d983fecf4d064975aef780
This introduces oslo.messaging variables that define the RPC and
Notify transports for the OpenStack services. These parameters replace
the rabbitmq values and aure used to generate the messaging
transport_url for the service. The association of the messaging
backend server to the oslo.messaging services will then be transparent
to the magnum service.
This patch:
* Add oslo.messaging variables for RPC and Notify to defaults
* Add transport_url generation to conf template
* Add oslo.messaging to tests inventory
* Update tests
* Update examples
* Add release note
Change-Id: Ib44af3b1d153742975351a321d65c8812a994370
Dmitriy Rabotyagov