Due to weird postinst logic of keystone-common package permissions
for SSH private key that is used for fernet rotation are reset to 0640 which
prevents SSH from further functioning.
We add post-package installation task that will ensure private key permissions
for Ubuntu distro installations specifically.
Change-Id: I1ebee33e3cf52cc0a9c474423a4fd5fa7f1cbe81
With update of ansible-lint to version >=6.0.0 a lot of new
linters were added, that enabled by default. In order to comply
with linter rules we're applying changes to the role.
With that we also update metdata to reflect current state.
Depends-On: https://review.opendev.org/c/openstack/ansible-role-systemd_service/+/888223
Change-Id: Id92330b3c709201a74612c8353cefa75778eac0c
Keystone role was never migrated to usage of haproxy-endpoints role
and included task was used instead the whole time.
With that to reduce complexity and to have unified approach, all mention
of the role and handler are removed from the code.
Change-Id: Ib21a5f5caa590daa827e45d26015bf32abe39cf2
There is no reason to support multiple web servers as a proxy for
keystone. Nginx is missing modules to support federation. With it's
removal we simplify code and reduce maintaining efforts needed.
Change-Id: Ib3f90a72dfc8f78cf304b0f130883befdeb09220
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/818674
While shibboleth and mod_auth_openidc can theoretically co installed
now, unfortunately the shibboleth enabled configuation will cause
issues when using mod_auth_openidc.
As we only drop the configuration for one of these apache mods at a time
I have decided that it is best we only support one of these packages
being present at any time to avoid conflicts.
Change-Id: Ib0ebf1711db42dd00b3e14c1e5604fed2632437d
All references to Gentoo, SUSE, Debian stretch and Centos-7 are removed.
Conditional tasks, ternary operators and variables are simplified where possible
OS specific variables files are generalised where possible
Change-Id: I1624730385a7b54cf36a94d313cc298430129736
For a metal deployment the keystone and horizon roles both try to
do this and it should be done in one place. This is moved to the
openstack_hosts role.
Depends-On: https://review.opendev.org/757612
Change-Id: Iba607a89b104ef8a63c9f956e2cd60f6b7ca4466
For Centos, we use by default public repo for nginx and shibboleth. You can
change this behaviour with these role-wide variables:
- keystone_centos_nginx_mirror
- keystone_centos_nginx_key
- keystone_centos_shibboleth_mirror
- keystone_centos_shibboleth_key
Or with these osa-wide variables:
- centos_nginx_mirror
- centos_nginx_key
Change-Id: Icb21c31141d1d78e5a2e23b35378ffb0520c0d1d
When deploying OSA on bare metal (no container tech),
the os keystone role interferes with os horizon as the
package apache2 is automatically uninstalled.
Change-Id: Ifec88251712a99ad7afbb51e386adf15bc4841eb
Closes-Bug: #1830397
The developer mode and venv download modes at the moment
no longer carry any meaning. This review changes this role to
do the equivalent of what developer_mode was all the time,
meaning that it always builds the venv and never requires
the repo server, but it will use a repo server when available.
As part of this, we move the source build out of its own file
because it's now a single task to include the venv build role.
This is just to make it easier to follow the code.
Change-Id: I53d1fb9ace5eb3e66ca8aef08603abf580acc066
===
[DEPRECATION WARNING]: Using tests as filters is deprecated.
Instead of using `result|search` use `result is search`. This
feature will be removed in version 2.9.
===
Change-Id: I8e856ecaeb82446785dbdcd23688bb0db0a30857
With the removal of Keystone V2, the admin and public APIs serve
no difference so we're better off removing those extra services.
Change-Id: I2a8743357934ace7aa2accc53b0df7b3865b866f
In order to radically simplify how we prepare the service
venvs, we use a common role to do the wheel builds and the
venv preparation. This makes the process far simpler to
understand, because the role does its own building and
installing. It also reduces the code maintenance burden,
because instead of duplicating the build processes in the
repo_build role and the service role - we only have it all
done in a single place.
We also change the role venv tag var to use the integrated
build's common venv tag so that we can remove the role's
venv tag in group_vars in the integrated build. This reduces
memory consumption and also reduces the duplication.
This is by no means the final stop in the simplification
process, but it is a step forward. The will be work to follow
which:
1. Replaces 'developer mode' with an equivalent mechanism
that uses the common role and is simpler to understand.
We will also simplify the provisioning of pip install
arguments when doing this.
2. Simplifies the installation of optional pip packages.
Right now it's more complicated than it needs to be due
to us needing to keep the py_pkgs plugin working in the
integrated build.
3. Deduplicates the distro package installs. Right now the
role installs the distro packages twice - just before
building the venv, and during the python_venv_build role
execution.
Depends-On: https://review.openstack.org/598957
Change-Id: Ib7b1cd7ec004de4e3e285c315f63076e7448e50e
Implements: blueprint python-build-install-simplification
Signed-off-by: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
The GPG key for NGINX was added manually therefore YUM would try
to pull it down and fail because the repository cache would not
allow any requests that tried to pull down a GPG key.
This patch adds it via Ansible beforehand that way it doesn't go
through the proxy.
Change-Id: I96ad7f049ab72ce338596ae34362e3a4ca026bb6
Distro installations do not deploy virtual environments but they still
need a local venv_tag deployed. The reason for this is because this fact
is used by the integration repo to determine if the keystone software is
the same across all nodes in the keystone_all group so we can safely run
the DB migration.
Change-Id: I9bab11b3c643f2411d78267aa6114081fe76b9ef
With the more recent versions of ansible, we should now use
"is" instead of the "|" sign for the tests.
This should fix it.
Change-Id: I562dc5430f04c09fdd63b07f0f307a273f232d0a
Distributions provide packages for the OpenStack services so we add
support for using these instead of the pip ones.
Depends-On: I5a78e2120e596d36629b4ba978b2b5df76b149b0
Depends-On: Ib64dcbc960df7d369d202ce8cf7bdc29b3ee0e0a
Depends-On: Id9dd2dea146709414ab9ce8d439f1587e6776fd4
Depends-On: I2ba89e25c0010c9a5b515a3d0c9c731b30876e74
Depends-On: I0442b0aa94c3d0882d1118ad0c824d123bd21c88
Change-Id: I26848678dd07a409ef3e159cffb4ba6f0a228ab4
Implements: blueprint openstack-distribution-packages
We reinitialize the venv to ensure that the right version of
python is in the venv, but we do not want virtualenv to also
replace pip, setuptools and wheel so we tell it not to. If we
don't do this then virtualenv will install the latest available
version, which is not what we want.
Change-Id: I0f0751a11dffbd77eaf2bfd238f2f934f11847c8
Partial-Bug: #1764470
This removes the systemd service templates and tasks from this role and
leverages a common systemd service role instead. This change removes a
lot of code duplication across all roles all without sacrificing features
or functionality. The intention of this change is to ensure uniformity and
reduce the maintenance burden on the community when sweeping changes are
needed.
In converting this role to use the common systemd_service role a
keystone_service dictionary was created in the defaults main.yml file.
This change follows the pattern of other services.
Change-Id: I65902f2483ef2f18ac2d229c5ebd9d090b6ae040
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Until all the roles are ready and have this new role in the
ansible-role-requirements, we should not be merging this. We
should also ensure that the integrated a-r-r includes this
role before merging this.
This reverts commit b42eef0dc4.
Change-Id: I8a944db87948ff783028240d3548016a52ab5af4
This removes the systemd service templates and tasks from this role and
leverages a common systemd service role instead. This change removes a
lot of code duplication across all roles all without sacrificing features
or functionality. The intention of this change is to ensure uniformity and
reduce the maintenance burden on the community when sweeping changes are
needed.
In converting this role to use the common systemd_service role a
keystone_service dictionary was created in the defaults main.yml file.
This change follows the pattern of other services.
Change-Id: I70e1f6007d9f88f05ccdc737b210415274580a46
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
In the previous patch removing the virtualenv-tools, we've updated
paths to the proper binaries, but we don't reset the virtualenv
like what we used to do.
Change-Id: Idf8f634bff5b247639a4673e63aef880a5bfa1d6
Partial-Bug: #1741634
virtualenv-tools has a bug which gets triggered in gates: it can't
change the shebang of a virtualenv python bin/ files if they
were generated with a virtualenv script whose shebang ends with
python2 instead of python.
Because we can't modify virtualenv-tools, we use shell scripts
instead.
Change-Id: I75f189b7e8856f6ec4e844ca8fdf2c8c81fb3faa
Partial-Bug: #1741634
This is similar to I8a27bb268ea2e145ed113c652ab7846f6adc3f82
openSUSE is also suffering from the same virtualenv-tools issue so
we need to remove the existing python-2.7 symlink.
Link: https://github.com/fireteam/virtualenv-tools/issues/5
Related-Bug: #1637509
Partial-Bug: #1644629
Change-Id: I16ea455f8d9336237175e6c43f0b14713cf314d3
Add support for the openSUSE Leap distributions. Additionally,
for openSUSE, we need to load some extra apache2 modules which are
not there by default.
Change-Id: Iac555cefa0a7a6ecf344761d54644fd3fa2443f9
Some deployers make use of the new role without
implementing a change in the software, resulting
in the correct local facts not being deployed.
To hedge against this, we add some more conditionals
to ensure that the facts are deployed.
Change-Id: I2de1a8c2e3bd75f39ac682f48613ff44eb5223ac
This patch implements the use of uWSGI exclusively,
always with a web server acting as a reverse proxy.
It removes the option of using uWSGI with Apache
and mod_wsgi.
In the case of Keystone being used in a Federated
Service Provider configuration, it will use Apache
as the web server but for all other environments
it will use Nginx instead.
Change-Id: If6e95fc0d3f7d34780db1aed2b8cedca87499934
The current rolling upgrade implementation
relies on the role to orchestrate the rolling
upgrade. When the role is executed using
playbook serialisation, the db sync contract
is executed before all hosts are upgraded,
potentially resulting in data corruption.
This patch returns the role to the simpler,
best practice model of expecting that the role
is applied to a single host and that the
playbook handles orchestration. This method
can be used with any form of serialisation.
Depends-On: Ie90cdcbf9e73082a2074c8832b7490d188e178af
Change-Id: I5650f16b9a115bd392012b743788057a94d09226
In order to optimise the distro package installation
process the list of packages to install is prepared
for the host, then installed by passing the package
task the full list instead of using a loop.
Change-Id: Ia5c8f31470f0f917ee556ca1157b9191da9d0d2c
In order to do a developer mode that allows installation of packages
from local file, which take precedence over the developer_mode
constraints, we need to allow the order of the constraints to be
changed.
This patch adds a "pip_install_developer_constraints" var which is used
to set the developer mode constraints. By default this will leave the
same behaviour but will allow additional constraints to be added, or the
developermode constraints file to be overriden altogether.
Change-Id: I2baa5bcaf7886263b9f497c0ad1479d65c561cc2
In order to make it easier to detect the currently deployed
venv for a service, and therefore allow smarter decisions
for things like upgrading, we implement the venv tag as a
local fact.
The file used to store facts will be the same for all
OpenStack services, with each service using its own section.
Example:
"ansible_local": {
"openstack_ansible": {
"keystone": {
"venv_tag": "14.2.1"
}
}
}
Change-Id: Icf443a93424c9eba7b68d3279ebb48ae00602808
We use an SSH bastion host which we do our deployment through. The
deployment host doesn't have direct access to the same network as the
host. As a result the venv local checksum lookup fails.
I have described this here:
https://bugs.launchpad.net/openstack-ansible/+bug/1689283
This is a simple fix for this problem, assuming everything is good it
will need repeating in multiple places in the code base.
Change-Id: Iab5af8a3b1543ade370fa607418c9568e632f333
The update of the apt cache and the package installation
can all be handled in a single task by providing the
package action plugin with the right parameters. This
removes an extra task to optimise execution.
The minimum Ansible version is raised to 2.2 due to a
known bug [1] in Ansible's apt module which does not
update the cache properly if the cache update and the
install are combined in a single task.
[1] https://github.com/ansible/ansible-modules-core/issues/1497
Change-Id: If6b3261bba643759f12a811d849ce9cb1040497f
The policy.json file is currently read continually by the
services and is not only read on service start. We therefore
cannot template directly to the file read by the service
(if the service is already running) because the new policies
may not be valid until the service restarts. This is
particularly important during a major upgrade. We therefore
only put the policy file in place after the service restart.
This patch also tidies up the handlers and some of the install
tasks to simplify them and reduce the tasks/code a little.
Change-Id: Ie913e5eb75f3601107b53bab7bda4a02ab1c1024
If keystone install is done separately from the config (filtering
or interruption), a re-run of the role will not mark the venv
as changed, and the db sync will not trigger.
Here we force a flush on file next to the venv change, and use
this file (local fact) as a way to know if a db sync is required.
When a dbsync is considered done, i.e after the expand and migrate
stages, we mark the db_migrate as done for all the nodes.
Change-Id: I318700582a65cc1ea6571d9460fd58bed949a4e9
Depends-On: Ie4a47e85bf31e1ac3a2bd6f0dec510bb24abdbbd
As EPEL is now deployed in the LXC cache and by the pip_install
role, this is no longer required.
This reverts commit 1897a675c3.
Change-Id: Icf6fb60d46d8f9085a733d505bb70947eef6011f
The recent changes to remove EPEL and install RDO broke keystone
since some packages from EPEL are still required. This patch
ensures that the keystone role can install packages from EPEL if
needed.
Related-bug: 1670012
Change-Id: I2a52c7622a527fd6c14c76547101a3731973a538
During upgrade testing, move Keystone containers behind an HAProxy
load balancer and run a benchmarking tool to make continuous requests
against the keystone API, providing summary output at the end of the
play. This should help ensure the upgrade process between releases
remains without downtime.
To avoid service interruptions because of files being replaced during
installations:
- set the 'keystone_venv_tag' var so that it changes between releases
- perform the 'Create WSGI symlinks' task only after the files being
linked to exist
Benchmarking scripts were provided by:
https://github.com/lbragstad/keystone-performance-upgrade
Co-Authored-By: Lance Bragstad <lbragstad@gmail.com>
Change-Id: I462e6496f125c7e263bbab188e86c45e1f4f7f1e
Dmitriy Rabotyagov