With update of ansible-lint to version >=6.0.0 a lot of new
linters were added, that enabled by default. In order to comply
with linter rules we're applying changes to the role.
With that we also update metdata to reflect current state.
Depends-On: https://review.opendev.org/c/openstack/ansible-role-systemd_service/+/888223
Change-Id: Id92330b3c709201a74612c8353cefa75778eac0c
Keystone role was never migrated to usage of haproxy-endpoints role
and included task was used instead the whole time.
With that to reduce complexity and to have unified approach, all mention
of the role and handler are removed from the code.
Change-Id: Ib21a5f5caa590daa827e45d26015bf32abe39cf2
There is no reason to support multiple web servers as a proxy for
keystone. Nginx is missing modules to support federation. With it's
removal we simplify code and reduce maintaining efforts needed.
Change-Id: Ib3f90a72dfc8f78cf304b0f130883befdeb09220
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/818674
As per the community goal of migrating the policy file
the format from JSON to YAML[1], we need to replace policy.json to
policy.yaml and remove deprecated policy.json.
config_template has been choosen instead of the copy, since it can
properly handle content that has been lookuped.
We make a separate task not to restart service when it's not needed.
[1] https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html
Change-Id: I41ee59a901e54860067edd277fdade394b0b8858
All references to Gentoo, SUSE, Debian stretch and Centos-7 are removed.
Conditional tasks, ternary operators and variables are simplified where possible
OS specific variables files are generalised where possible
Change-Id: I1624730385a7b54cf36a94d313cc298430129736
Seems that ansible_default_ipv4 doesn't always has address key,
so we're setting module default 127.0.0.1 for this case.
Change-Id: I506ba4fc6b134ef0b389a2647fe22ae6b1a17ace
There is no host set for the handler that checks once the service
has gone up. In certain scenario where you may be running all
metal with HAproxy on the same node, you would want Keystone to
bind to the primary IP only to avoid conflicting with HAproxy.
By doing this, you're no longer listening to 0.0.0.0 which includes
127.0.0.1 and this handler fails. By switching the handler to hit
the primary IP, you're able to successfully run this task both in
the existing scenario, as well as cover scenario where you are
listening on the primary IP.
Change-Id: I1f519d74ec25abfe37d2952aac92a346ccbf48f6
The files and templates we carry are almost always in a state of
maintenance. The upstream services are maintaining these files and
there's really no reason we need to carry duplicate copies of them. This
change removes all of the files we expect to get from the upstream
service. while the focus of this change is to remove configuration file
maintenance burdens it also allows the role to execute faster.
* Source installs have the configuration files within the venv at
"<<VENV_PATH>>/etc/<<SERVICE_NAME>>". The role will now link the
default configuration path to this directory. When the service is
upgraded the link will move to the new venv path.
* Distro installs package all of the required configuration files.
To maintain our current capabilities to override configuration the
role will fetch files from the disk whenever an override is provided and
then push the fetched file back to the target using `config_template`.
Change-Id: I93cb6463ca1eb93ab7f4e7a3970a7de829efaf66
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The change removes legacy files which conflict with new services being
deployed during a rolling upgrade. This change adds two tasks to remove
any legacy files should they be found and removes old cleanup handlers
that are no longer in service.
Change-Id: Ie593a80e0e6708c50f7809171fa47c7043a2e136
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The memcache flush implemented in https://review.openstack.org/608066
is actually a workaround. The implementation should really be
implemented more surgically in keystone itself. This has begun with
https://review.openstack.org/612686 but it is not complete. However,
something that's been made clear by the team is that this was only
required for the Newton->Ocata upgrade and has not been required since.
This workaround may be required again in the future, so instead of
removing it, we make the tool opt-in via a toggle which can then easily
be set when doing the appropriate major upgrade.
Related-Bug: 1793389
Change-Id: Ied0ce1e9877697bb627f784a0590f7c7e924479b
Flushing cache when schema changes occur to avoid drift between
what might be cached and what keystone queries expect.
Change-Id: Ibf8f3dd60d6f3c446a14dc8228fa005f12fcc840
Closes-Bug: 1793389
With the removal of Keystone V2, the admin and public APIs serve
no difference so we're better off removing those extra services.
Change-Id: I2a8743357934ace7aa2accc53b0df7b3865b866f
In order to radically simplify how we prepare the service
venvs, we use a common role to do the wheel builds and the
venv preparation. This makes the process far simpler to
understand, because the role does its own building and
installing. It also reduces the code maintenance burden,
because instead of duplicating the build processes in the
repo_build role and the service role - we only have it all
done in a single place.
We also change the role venv tag var to use the integrated
build's common venv tag so that we can remove the role's
venv tag in group_vars in the integrated build. This reduces
memory consumption and also reduces the duplication.
This is by no means the final stop in the simplification
process, but it is a step forward. The will be work to follow
which:
1. Replaces 'developer mode' with an equivalent mechanism
that uses the common role and is simpler to understand.
We will also simplify the provisioning of pip install
arguments when doing this.
2. Simplifies the installation of optional pip packages.
Right now it's more complicated than it needs to be due
to us needing to keep the py_pkgs plugin working in the
integrated build.
3. Deduplicates the distro package installs. Right now the
role installs the distro packages twice - just before
building the venv, and during the python_venv_build role
execution.
Depends-On: https://review.openstack.org/598957
Change-Id: Ib7b1cd7ec004de4e3e285c315f63076e7448e50e
Implements: blueprint python-build-install-simplification
Signed-off-by: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
With the more recent versions of ansible, we should now use
"is" instead of the "|" sign for the tests.
This should fix it.
Change-Id: I562dc5430f04c09fdd63b07f0f307a273f232d0a
When deploying the base templates for api-paste, policy files
and other files which are included in the service git source,
we now use the venv files instead of requiring access to a
git source and a complex set of lookups and variable
implementations.
This is simpler and more cross-series, and works from Queens
due to the related bug's patches.
Change-Id: I6a4e2514e66b15b2ae227e62b6dc9ae1a50a4fbd
Related-Bug: #1718356
This removes the systemd service templates and tasks from this role and
leverages a common systemd service role instead. This change removes a
lot of code duplication across all roles all without sacrificing features
or functionality. The intention of this change is to ensure uniformity and
reduce the maintenance burden on the community when sweeping changes are
needed.
In converting this role to use the common systemd_service role a
keystone_service dictionary was created in the defaults main.yml file.
This change follows the pattern of other services.
Change-Id: I65902f2483ef2f18ac2d229c5ebd9d090b6ae040
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Until all the roles are ready and have this new role in the
ansible-role-requirements, we should not be merging this. We
should also ensure that the integrated a-r-r includes this
role before merging this.
This reverts commit b42eef0dc4.
Change-Id: I8a944db87948ff783028240d3548016a52ab5af4
This removes the systemd service templates and tasks from this role and
leverages a common systemd service role instead. This change removes a
lot of code duplication across all roles all without sacrificing features
or functionality. The intention of this change is to ensure uniformity and
reduce the maintenance burden on the community when sweeping changes are
needed.
In converting this role to use the common systemd_service role a
keystone_service dictionary was created in the defaults main.yml file.
This change follows the pattern of other services.
Change-Id: I70e1f6007d9f88f05ccdc737b210415274580a46
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This patch implements the use of uWSGI exclusively,
always with a web server acting as a reverse proxy.
It removes the option of using uWSGI with Apache
and mod_wsgi.
In the case of Keystone being used in a Federated
Service Provider configuration, it will use Apache
as the web server but for all other environments
it will use Nginx instead.
Change-Id: If6e95fc0d3f7d34780db1aed2b8cedca87499934
The current rolling upgrade implementation
relies on the role to orchestrate the rolling
upgrade. When the role is executed using
playbook serialisation, the db sync contract
is executed before all hosts are upgraded,
potentially resulting in data corruption.
This patch returns the role to the simpler,
best practice model of expecting that the role
is applied to a single host and that the
playbook handles orchestration. This method
can be used with any form of serialisation.
Depends-On: Ie90cdcbf9e73082a2074c8832b7490d188e178af
Change-Id: I5650f16b9a115bd392012b743788057a94d09226
Due to the debug message plugin the handler restart
messages show at the end of the playbook execution
which is a little confusing. Using debug also
requires setting changed_when to true which is a
little extra bit of code which we do not have to
carry.
Instead we use the command module which is simple,
works and less wordy.
Change-Id: I9e3acb966c0d16b61fbc998642e36f4015064155
When the policy file is copied from the templated
file to the active file, it loses its group/mode
settings. This patch ensures that they are properly
replicated during the copy.
Change-Id: I7a6c07cce405c8e7f3dca13cc24dc413895e6bfb
The policy.json file is currently read continually by the
services and is not only read on service start. We therefore
cannot template directly to the file read by the service
(if the service is already running) because the new policies
may not be valid until the service restarts. This is
particularly important during a major upgrade. We therefore
only put the policy file in place after the service restart.
This patch also tidies up the handlers and some of the install
tasks to simplify them and reduce the tasks/code a little.
Change-Id: Ie913e5eb75f3601107b53bab7bda4a02ab1c1024
This change adds the "*-config" tag to all handlers which is being done
so all of them can be skipped or included using the ansible tags syntax.
This is useful when running parts of the role, like when building image
based artifacts from the "*-install" tags.
Change-Id: I92d34b25c51d365e2f8b97b72a1aa08b1537ab79
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This patch implements upgrading keystone with zero downtime as the
default installation process. Handlers have been modified to ensure that
the first keystone node is stopped, facilitates the database migrations,
and that it is started and available before restarting any other keystone
nodes. Migrations also now only occur when there is a change within the
installed keystone venv.
This process is documented at
http://docs.openstack.org/developer/keystone/upgrading.html#upgrading-without-downtime
A new test scenario has been added for testing basic upgradability
between releases.
Implements: blueprint upgrade-testing
Change-Id: I0d3cfcb80b64d005d60f4c8445f991855f844796
This allows deploys to elect to have Keystone run off of Apache
without implying the use of mod_wsgi, such as with uwsgi. A
following patch will introduce the Apache configuration needed to
link Apache to uwsgi so that existing Federation support can be
used.
Release notes summarizing all changes for this blueprint will follow
in the final patch in the series.
Change-Id: Idf9e48b0c93174648982cf27cf922d3801565c74
Related: blueprint keystone-uwsgi
Apply configuration to add request time to the access log.
Creates virtual hosts for each Keystone service.
Enables SSL termination within Nginx.
The Uwsgi sockets are updated to match the Keystone developer docs
to improve consistency of experience for operators.
No Shibboleth integration is included.
Not introducing any additional Nginx restarts based on changes in
Federation configuration yet for this reason.
Change-Id: Iec42810be7ff6d05fa38deb23996e66e0c34da8e
Related: blueprint keystone-uwsgi
This change implements CentOS 7 support within the os_keystone role.
Depends-on: I333fb1887339e8dc9ebf10ff137dda3cff629dc0
Change-Id: Ib339cd0657f7008fa48bf74f8d6ddd4b8add2ea1
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
When an Apache + mod_wsgi configuration is not selected, configure
the two Keystone services with uWSGI service profiles.
Two arbitrary ports are selected for uWSGI to listen on, so that it
may be proxied for by a dedicated web server. This is in preparation
for laying down Nginx in a future patch.
Notify events are updated to restart the Keystone uWSGI services
where Keystone's configuration is modified only. Because federation
concerns will be isolated within the dedicated web server, changes
to federation configuration of Shiboleth do not trigger restarts of
uWSGI. Similarly, SSL certificate changes do not trigger restarts.
Change-Id: I99e16a999c496e68fb25fa2630d9b211c9755ea4
Related: blueprint keystone-uwsgi
In preparation for moving keystone to uwsgi under nginx the Apache
related components are being isolated so a branched install option
(keystone_apache_mod_wsgi_enabled) can be used to accommodate both
deployment models.
Change-Id: Idd9de25d1906dba526b5761ad1a8f75b672af29d
Related: blueprint keystone-uwsgi
This patch adds the ability to configure Keystone as a Service
Provider (SP) for a Federated Identity Provider (IdP).
* New variables to configure Keystone as a service provider are now
supported under a root `keystone_sp` variable. Example configurations
can be seen in Keystone's defaults file. This configuration includes
the list of identity providers and trusted dashboards. (At this time
only one identity provider is supported).
* Identity provider configuration includes the remote-to-local user
mapping and the list of remote attributes the SP can obtain from the
IdP.
* Shibboleth is installed and configured in the Keystone containers when
SP configuration is present.
* Horizon is configured for SSO login
DocImpact
UpgradeImpact
Implements: blueprint keystone-federation
Change-Id: I78b3d740434ea4b3ca0bd9f144e4a07026be23c6
Co-Authored-By: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
This change implements the blueprint to convert all roles and plays into
a more generic setup, following upstream ansible best practices.
Items Changed:
* All tasks have tags.
* All roles use namespaced variables.
* All redundant tasks within a given play and role have been removed.
* All of the repetitive plays have been removed in-favor of a more
simplistic approach. This change duplicates code within the roles but
ensures that the roles only ever run within their own scope.
* All roles have been built using an ansible galaxy syntax.
* The `*requirement.txt` files have been reformatted follow upstream
Openstack practices.
* Dynamically generated inventory is now more organized, this should assist
anyone who may want or need to dive into the JSON blob that is created.
In the inventory a properties field is used for items that customize containers
within the inventory.
* The environment map has been modified to support additional host groups to
enable the seperation of infrastructure pieces. While the old infra_hosts group
will still work this change allows for groups to be divided up into seperate
chunks; eg: deployment of a swift only stack.
* The LXC logic now exists within the plays.
* etc/openstack_deploy/user_variables.yml has all password/token
variables extracted into the separate file
etc/openstack_deploy/user_secrets.yml in order to allow seperate
security settings on that file.
Items Excised:
* All of the roles have had the LXC logic removed from within them which
should allow roles to be consumed outside of the `os-ansible-deployment`
reference architecture.
Note:
* the directory rpc_deployment still exists and is presently pointed at plays
containing a deprecation warning instructing the user to move to the standard
playbooks directory.
* While all of the rackspace specific components and variables have been removed
and or were refactored the repository still relies on an upstream mirror of
Openstack built python files and container images. This upstream mirror is hosted
at rackspace at "http://rpc-repo.rackspace.com" though this is
not locked to and or tied to rackspace specific installations. This repository
contains all of the needed code to create and/or clone your own mirror.
DocImpact
Co-Authored-By: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
Closes-Bug: #1403676
Implements: blueprint galaxy-roles
Change-Id: I03df3328b7655f0cc9e43ba83b02623d038d214e
Dmitriy Rabotyagov