With update of ansible-lint to version >=6.0.0 a lot of new
linters were added, that enabled by default. In order to comply
with linter rules we're applying changes to the role.
With that we also update metdata to reflect current state.
Depends-On: https://review.opendev.org/c/openstack/ansible-role-systemd_service/+/888223
Change-Id: Id92330b3c709201a74612c8353cefa75778eac0c
Keystone role was never migrated to usage of haproxy-endpoints role
and included task was used instead the whole time.
With that to reduce complexity and to have unified approach, all mention
of the role and handler are removed from the code.
Change-Id: Ib21a5f5caa590daa827e45d26015bf32abe39cf2
As per the community goal of migrating the policy file
the format from JSON to YAML[1], we need to replace policy.json to
policy.yaml and remove deprecated policy.json.
config_template has been choosen instead of the copy, since it can
properly handle content that has been lookuped.
We make a separate task not to restart service when it's not needed.
[1] https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html
Change-Id: I41ee59a901e54860067edd277fdade394b0b8858
We use the same condition, which defines against what host some "service"
tasks should run against, several times. It's hard to keep it the same
across the role and ansible spending additional resources to evaluate
it each time, so it's simpler and better for the maintenance to set
a boolean variable which will say for all tasks, that we want to run
only against signle host, if they should run or not now.
Change-Id: Iac06d3f02b1c9ee5e3bfbd28043fbb70d8b1d328
The files and templates we carry are almost always in a state of
maintenance. The upstream services are maintaining these files and
there's really no reason we need to carry duplicate copies of them. This
change removes all of the files we expect to get from the upstream
service. while the focus of this change is to remove configuration file
maintenance burdens it also allows the role to execute faster.
* Source installs have the configuration files within the venv at
"<<VENV_PATH>>/etc/<<SERVICE_NAME>>". The role will now link the
default configuration path to this directory. When the service is
upgraded the link will move to the new venv path.
* Distro installs package all of the required configuration files.
To maintain our current capabilities to override configuration the
role will fetch files from the disk whenever an override is provided and
then push the fetched file back to the target using `config_template`.
Change-Id: I93cb6463ca1eb93ab7f4e7a3970a7de829efaf66
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
In distro installs, the SSO callback template file is created in
/etc/keystone. The 'Retrieve default configuration files from venv' task
copies it from there to a cache folder. If 'keystone_sp' is not provided
the file is being removed, which causes a subsequent run of the role to
fail.
Change-Id: I05df4c2a5f57c7437c008dd73a31b7eda2c3e281
Keystone paste file was recently removed, and the paste mechanism
was completely removed in the past [1].
This removes all references to it.
[1]: d04bb1e7e6
Change-Id: Idc137a255437ae316d2f99f957df7d62bc693bac
We only need to do this task once in the play because it
delegates to all the hosts already. Currently it will do
the sshd enablement 9 times for 3 hosts.
We also add a comment to explain why this is being done
for future reference.
Change-Id: I201101f6d2bf733b375577b6af138272b9d0ce6b
The ssh service on ubuntu based systems is "ssh" which is established by
the service unit path `/lib/systemd/system/ssh.service`. When running
the service will respond to the name "sshd" however this is just an
alias. This change adds a variable to set the service unit name
based on the distro family which will allow the service to start should
it be masked.
The change will now delegate to all nodes within the keystone cluster
ensuring ssh is enabled and started. If SSH is not running everywhere at
the same time keystone key rotation will not be possible later on in the
role.
Change-Id: I552a6bb09b3ab917bfcad140633fe4662c0c5a82
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The keystone containers rely on SSHD for the key synchronisation,
so in this patch we ensure that it is installed, enabled and
running.
Change-Id: I044c080dba1068f79f4018b54b8ad120192b3932
The sso_callback_template file needs to be deployed for a
Keystone SP, not IDP. This patch fixes the conditional.
Closes-Bug: #1772772
Change-Id: I420e291807434382b4d3cd4c8809c7540a419f5a
Distributions provide packages for the OpenStack services so we add
support for using these instead of the pip ones.
Depends-On: I5a78e2120e596d36629b4ba978b2b5df76b149b0
Depends-On: Ib64dcbc960df7d369d202ce8cf7bdc29b3ee0e0a
Depends-On: Id9dd2dea146709414ab9ce8d439f1587e6776fd4
Depends-On: I2ba89e25c0010c9a5b515a3d0c9c731b30876e74
Depends-On: I0442b0aa94c3d0882d1118ad0c824d123bd21c88
Change-Id: I26848678dd07a409ef3e159cffb4ba6f0a228ab4
Implements: blueprint openstack-distribution-packages
When deploying the base templates for api-paste, policy files
and other files which are included in the service git source,
we now use the venv files instead of requiring access to a
git source and a complex set of lookups and variable
implementations.
This is simpler and more cross-series, and works from Queens
due to the related bug's patches.
Change-Id: I6a4e2514e66b15b2ae227e62b6dc9ae1a50a4fbd
Related-Bug: #1718356
In order to allow an install and config split, but not
to have ssh keys left inside an pre-installed container,
the two tasks are split and tagged appropriately.
Change-Id: I468d1178179d70edfe4b19d40a9a32b35ad18258
The current rolling upgrade implementation
relies on the role to orchestrate the rolling
upgrade. When the role is executed using
playbook serialisation, the db sync contract
is executed before all hosts are upgraded,
potentially resulting in data corruption.
This patch returns the role to the simpler,
best practice model of expecting that the role
is applied to a single host and that the
playbook handles orchestration. This method
can be used with any form of serialisation.
Depends-On: Ie90cdcbf9e73082a2074c8832b7490d188e178af
Change-Id: I5650f16b9a115bd392012b743788057a94d09226
The current mechanism uses a lookup, causing the
fetch of the default templates to happen via curl
from the deployment node. This causes problems if
the deployment node does not have access to the
repo server web service, which may be the case in
high security environments.
This patch changes the mechanism to only use the
lookup module for the file content that is on the
deployment node, then falls back to using the uri
module to fetch the default content. This ensures
that the deployment node does not have to reach
into the environment for the content.
Also, in https://review.openstack.org/446235 a
new task was added to template the paste/policy
files from the git source but the owner/group/mode
settings were not implemented. This patch
consolidates the two config_template tasks into
one and ensures that the owner/group/mode settings
are properly implemented for all the files.
Change-Id: I47de2e4b1c864035c8b686e96e65722f312b7280
Instead of sourcing the paste/policy files from the role,
the deploy host will first be checked, then the git source.
This eliminates our need to carry the template files and
keep them up to date. It also ensures that if a custom git
source or alternative SHA is used, the correct source
templates are used.
Related-To: I97476c42172cace5601f777e771ba0aa649b05ca
Change-Id: I910bb1700bafd48185c15d64401c8f8e93c696ea
The policy.json file is currently read continually by the
services and is not only read on service start. We therefore
cannot template directly to the file read by the service
(if the service is already running) because the new policies
may not be valid until the service restarts. This is
particularly important during a major upgrade. We therefore
only put the policy file in place after the service restart.
This patch also tidies up the handlers and some of the install
tasks to simplify them and reduce the tasks/code a little.
Change-Id: Ie913e5eb75f3601107b53bab7bda4a02ab1c1024
The security guide suggests that service config files
should be owned by root and in the service user group with 0640 permissions.
Change-Id: I5dc6e2c44ac5607fc1ff1c9fd2653eb23ef794bf
This patch implements upgrading keystone with zero downtime as the
default installation process. Handlers have been modified to ensure that
the first keystone node is stopped, facilitates the database migrations,
and that it is started and available before restarting any other keystone
nodes. Migrations also now only occur when there is a change within the
installed keystone venv.
This process is documented at
http://docs.openstack.org/developer/keystone/upgrading.html#upgrading-without-downtime
A new test scenario has been added for testing basic upgradability
between releases.
Implements: blueprint upgrade-testing
Change-Id: I0d3cfcb80b64d005d60f4c8445f991855f844796
In https://review.openstack.org/309425 adjustments were
made which resulted in keystone_idp and keystone_sp to
always be defined. Unfortunately the CentOS support for
os_keystone merged after that and reviews did not pick
up the necessary changes.
This patch corrects it so that the repo and packages are
only installed if necessary.
Additionally, the Federation SP callback template was
being unnecessarily copied over. It will now only copy
over when it will be used.
Change-Id: I466f1391893d33be7c83caba70ec93e44dd1d482
This change implements CentOS 7 support within the os_keystone role.
Depends-on: I333fb1887339e8dc9ebf10ff137dda3cff629dc0
Change-Id: Ib339cd0657f7008fa48bf74f8d6ddd4b8add2ea1
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
When an Apache + mod_wsgi configuration is not selected, configure
the two Keystone services with uWSGI service profiles.
Two arbitrary ports are selected for uWSGI to listen on, so that it
may be proxied for by a dedicated web server. This is in preparation
for laying down Nginx in a future patch.
Notify events are updated to restart the Keystone uWSGI services
where Keystone's configuration is modified only. Because federation
concerns will be isolated within the dedicated web server, changes
to federation configuration of Shiboleth do not trigger restarts of
uWSGI. Similarly, SSL certificate changes do not trigger restarts.
Change-Id: I99e16a999c496e68fb25fa2630d9b211c9755ea4
Related: blueprint keystone-uwsgi
This change will update the file paths of the keystone WSGI
applications so that keystone will run properly and not just
produce errors in the logs. The virtualenvs were the primary reason
for this bug, and this solution allows for the deployment of
keystone without requiring virtualenvs provided that they override
keystone_bin.
Closes-bug: 1586161
Change-Id: Iedf4c2651d304d3034be64c781a6daf7ab0b400f
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Remove all tasks and variables related to toggling between installation
of keystone inside or outside of a Python virtual environment.
Installing within a venv is now the only supported deployment.
Additionally, a few changes have been made to make the creation of the
venv more resistant to interruptions during a run of the role.
* unarchiving a pre-built venv will now also occur when the venv
directory is created, not only after being downloaded
* virtualenv-tools is run against both pre-built and non pre-built venvs
to account for interruptions during or prior to unarchiving
Change-Id: Ic0a0dac84a26aba2ef0ce5410dc7c722570cd410
Implements: blueprint only-install-venvs
The numerous tags within the role have been condensed
to two tags: keystone-install and keystone-config
These tags have been chosen as they are namespaced
and cover the two major functions of the role.
Documentation has been updated to inform how each tag
influences the function of the role.
Change-Id: Iea4bff944ce0a35a4b1bc044171472ea44eda323
This patch changes the keystone_ldap configuration to allow multiple
LDAP identity back-end configurations to be implemented.
Example implementation in user_variables.yml:
keystone_ldap:
Users:
url: "ldap://10.10.10.10"
user: "root"
password: "secrete"
...
Admins:
url: "ldap://20.20.20.20"
user: "root"
password: "secrete"
...
This will place two configuration files into /etc/keystone/domains/,
both of which will be configured to use the LDAP driver.
- keystone.Users.conf
- keystone.Admins.conf
Each first level key entry is a domain name. Each entry below that
are key-value pairs for the 'ldap' section in the configuration
file.
Note that the reason why only LDAP is catered for is due to the fact
that LDAP is the only supported driver in OpenStack for
Domain-specific configuration files.
Also the reason that only the identity back-end is catered for is that
the LDAP driver for the role, resource and assignment back-ends have
been deprecated and are scheduled for removal in Mitaka.
UpgradeImpact:
- keystone_ldap's first key tier is now the domain name.
An existing keystone_ldap configuration entry can be converted by
renaming the 'ldap' key to the domain name 'Default'.
**Note** that the domain name entry is case-sensitive.
- keystone_ldap_identity_driver has been removed, as the driver
for ldap is now simply 'ldap' and there are no other back-end
options for Keystone at this time.
Change-Id: Ifa4c42f7dbcc40a256a3156f74f0150384f9ab87
This commit conditionally allows the os_keystone role to
install build and deploy within a venv. This is the new
default behavior of the role however the functionality
can be disabled.
Change-Id: Ie9e51926c96125a543e05eaa1912684fb01fecda
Implements: blueprint enable-venv-support-within-the-roles
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The change modifies the keystone template tasks such that it's now
using the config_template action plugin. This change will make so that
config files can be dynamically updated, by a deployer, at run time,
without requiring the need to modify the in tree templates or defaults.
Partially implements: blueprint tunable-openstack-configuration
Change-Id: I957d55df81c7edd4e2a95597a62a75c6bd0f46fe
This PR replaces the copy_update module with a proper Ansible action
plugin. This change allows for dynamic updates to configuration files
that are ini, json, and yaml.
All of the policy files have been moved to the role templates directories
and the task syntax has been updated to facilitate the new action plugin.
An entry has been added to the ansible.cfg file to inform Ansible to look
into the new directory. In order for the action plugin to work as a
"module" a virtual module was added to the library directory.
Change-Id: I80331628b2c3d426a95c89d9c1b766e2e3f70e6d
Partially implements: blueprint tunable-openstack-configuration
This patch adds the ability to configure Keystone as a Service
Provider (SP) for a Federated Identity Provider (IdP).
* New variables to configure Keystone as a service provider are now
supported under a root `keystone_sp` variable. Example configurations
can be seen in Keystone's defaults file. This configuration includes
the list of identity providers and trusted dashboards. (At this time
only one identity provider is supported).
* Identity provider configuration includes the remote-to-local user
mapping and the list of remote attributes the SP can obtain from the
IdP.
* Shibboleth is installed and configured in the Keystone containers when
SP configuration is present.
* Horizon is configured for SSO login
DocImpact
UpgradeImpact
Implements: blueprint keystone-federation
Change-Id: I78b3d740434ea4b3ca0bd9f144e4a07026be23c6
Co-Authored-By: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
This change adds the bits necessary to configure Keystone as an
identity provider (IdP) for an external service provider (SP).
* New variables to configure Keystone as an identity provider are now
supported under a root `keystone_idp` variable. Example configurations
can be seen in Keystone's defaults file. This configuration includes
the location of the signing certificate, authentication endpoints and
list of allowed service providers.
* xmlsec1 is installed in the Keystone containers when IdP configuration
is enabled.
* The IdP metadata and signing certiciate are generated and installed.
Implements: blueprint keystone-federation
Change-Id: I81455e593e3059633a55f7e341511d5ad9eba76f
Enables default domain support using ldap. This change moves the
ldap config to the default domain and enables domain specific
drivers.
Change-Id: I85f6610a25617fdea1fc216b53df0ab30260fed9
Cloes-Bug: 1447768
Currently, the default policy.json files for all projects are just
dropped into place. There is no flexibility to customize, edit or update
certain key-values in the policy.json. This change adds a custom module
called 'copy_updates' that allows any updates to be applied to the
default file.
Implements: blueprint dynamically-manage-policy.json
Change-Id: Iadaaeb935f4602835b1cec858b53d4db8c18b1b0
This change implements the blueprint to convert all roles and plays into
a more generic setup, following upstream ansible best practices.
Items Changed:
* All tasks have tags.
* All roles use namespaced variables.
* All redundant tasks within a given play and role have been removed.
* All of the repetitive plays have been removed in-favor of a more
simplistic approach. This change duplicates code within the roles but
ensures that the roles only ever run within their own scope.
* All roles have been built using an ansible galaxy syntax.
* The `*requirement.txt` files have been reformatted follow upstream
Openstack practices.
* Dynamically generated inventory is now more organized, this should assist
anyone who may want or need to dive into the JSON blob that is created.
In the inventory a properties field is used for items that customize containers
within the inventory.
* The environment map has been modified to support additional host groups to
enable the seperation of infrastructure pieces. While the old infra_hosts group
will still work this change allows for groups to be divided up into seperate
chunks; eg: deployment of a swift only stack.
* The LXC logic now exists within the plays.
* etc/openstack_deploy/user_variables.yml has all password/token
variables extracted into the separate file
etc/openstack_deploy/user_secrets.yml in order to allow seperate
security settings on that file.
Items Excised:
* All of the roles have had the LXC logic removed from within them which
should allow roles to be consumed outside of the `os-ansible-deployment`
reference architecture.
Note:
* the directory rpc_deployment still exists and is presently pointed at plays
containing a deprecation warning instructing the user to move to the standard
playbooks directory.
* While all of the rackspace specific components and variables have been removed
and or were refactored the repository still relies on an upstream mirror of
Openstack built python files and container images. This upstream mirror is hosted
at rackspace at "http://rpc-repo.rackspace.com" though this is
not locked to and or tied to rackspace specific installations. This repository
contains all of the needed code to create and/or clone your own mirror.
DocImpact
Co-Authored-By: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
Closes-Bug: #1403676
Implements: blueprint galaxy-roles
Change-Id: I03df3328b7655f0cc9e43ba83b02623d038d214e