Commit Graph

40 Commits

Author SHA1 Message Date
Dmitriy Rabotyagov eea1a4853f Fix linters and metadata
With update of ansible-lint to version >=6.0.0 a lot of new
linters were added, that enabled by default. In order to comply
with linter rules we're applying changes to the role.

With that we also update metdata to reflect current state.

Depends-On: https://review.opendev.org/c/openstack/ansible-role-systemd_service/+/888223
Change-Id: Id92330b3c709201a74612c8353cefa75778eac0c
2023-07-14 20:44:53 +02:00
Dmitriy Rabotyagov e26aabe440 Remove mention of haproxy-endpoints role
Keystone role was never migrated to usage of haproxy-endpoints role
and included task was used instead the whole time.
With that to reduce complexity and to have unified approach, all mention
of the role and handler are removed from the code.

Change-Id: Ib21a5f5caa590daa827e45d26015bf32abe39cf2
2022-06-14 14:42:36 +02:00
Jonathan Rosser 19af9dabc8 Use ssh_keypairs role to generate fernet sync ssh keys
This uses ssh signed certificates so there is no longer the need
to distribute the keystone public key from each keystone host to all
other keystone hosts.

The legacy scripts and authorized key files are removed as a
migration step.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/836377
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/825292
Change-Id: If39df0cc80860576abac1830d5cfc66ca50fc655
2022-04-04 15:59:10 +00:00
Dmitriy Rabotyagov 59e879d28a Use config_template as a collection
Since we still use ceph-ansible that has their own implementation of
config_template module it's worth to use mentioned module as a collection
explicitly.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/819814
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-tests/+/819897


Change-Id: I9c4caf10192e2a25b1819d47065163ec78356a91
2021-11-30 15:56:09 +00:00
Dmitriy Rabotyagov d89c6a153a [goal] Deprecate the JSON formatted policy file
As per the community goal of migrating the policy file
the format from JSON to YAML[1], we need to replace policy.json to
policy.yaml and remove deprecated policy.json.

config_template has been choosen instead of the copy, since it can
properly handle content that has been lookuped.

We make a separate task not to restart service when it's not needed.

[1] https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html

Change-Id: I41ee59a901e54860067edd277fdade394b0b8858
2021-04-06 11:13:10 +00:00
Dmitriy Rabotyagov dcc16da7e2 Define condition for the first play host one time
We use the same condition, which defines against what host some "service"
tasks should run against, several times. It's hard to keep it the same
across the role and ansible spending additional resources to evaluate
it each time, so it's simpler and better for the maintenance to set
a boolean variable which will say for all tasks, that we want to run
only against signle host, if they should run or not now.

Change-Id: Iac06d3f02b1c9ee5e3bfbd28043fbb70d8b1d328
2020-09-08 18:20:43 +03:00
Kevin Carter 62d9f9c10d
Cleanup files and templates using smart sources
The files and templates we carry are almost always in a state of
maintenance. The upstream services are maintaining these files and
there's really no reason we need to carry duplicate copies of them. This
change removes all of the files we expect to get from the upstream
service. while the focus of this change is to remove configuration file
maintenance burdens it also allows the role to execute faster.

  * Source installs have the configuration files within the venv at
    "<<VENV_PATH>>/etc/<<SERVICE_NAME>>". The role will now link the
    default configuration path to this directory. When the service is
    upgraded the link will move to the new venv path.
  * Distro installs package all of the required configuration files.

To maintain our current capabilities to override configuration the
role will fetch files from the disk whenever an override is provided and
then push the fetched file back to the target using `config_template`.

Change-Id: I93cb6463ca1eb93ab7f4e7a3970a7de829efaf66
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2019-01-09 18:30:07 -06:00
Jimmy McCrory f66eca27d3 Do not remove SSO callback template file
In distro installs, the SSO callback template file is created in
/etc/keystone. The 'Retrieve default configuration files from venv' task
copies it from there to a cache folder. If 'keystone_sp' is not provided
the file is being removed, which causes a subsequent run of the role to
fail.

Change-Id: I05df4c2a5f57c7437c008dd73a31b7eda2c3e281
2018-10-30 10:26:06 -07:00
Jean-Philippe Evrard 76d39a6c2b Remove paste references in Keystone
Keystone paste file was recently removed, and the paste mechanism
was completely removed in the past [1].

This removes all references to it.

[1]: d04bb1e7e6

Change-Id: Idc137a255437ae316d2f99f957df7d62bc693bac
2018-10-24 20:57:57 +01:00
Jesse Pretorius aefb580803 Only do sshd enablement task on the first play host
We only need to do this task once in the play because it
delegates to all the hosts already. Currently it will do
the sshd enablement 9 times for 3 hosts.

We also add a comment to explain why this is being done
for future reference.

Change-Id: I201101f6d2bf733b375577b6af138272b9d0ce6b
2018-10-03 10:54:00 +01:00
Kevin Carter 2589ce6db0 Add variable for the ssh service
The ssh service on ubuntu based systems is "ssh" which is established by
the service unit path `/lib/systemd/system/ssh.service`. When running
the service will respond to the name "sshd" however this is just an
alias. This change adds a variable to set the service unit name
based on the distro family which will allow the service to start should
it be masked.

The change will now delegate to all nodes within the keystone cluster
ensuring ssh is enabled and started. If SSH is not running everywhere at
the same time keystone key rotation will not be possible later on in the
role.

Change-Id: I552a6bb09b3ab917bfcad140633fe4662c0c5a82
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-10-01 19:59:13 -05:00
Jesse Pretorius 1fe56e8129 Ensure that SSHD is installed, enabled and running
The keystone containers rely on SSHD for the key synchronisation,
so in this patch we ensure that it is installed, enabled and
running.

Change-Id: I044c080dba1068f79f4018b54b8ad120192b3932
2018-09-15 20:38:39 -06:00
Jesse Pretorius f88841f37f Use correct variable for sso_callback_template deployment
The sso_callback_template file needs to be deployed for a
Keystone SP, not IDP. This patch fixes the conditional.

Closes-Bug: #1772772
Change-Id: I420e291807434382b4d3cd4c8809c7540a419f5a
2018-06-05 18:50:14 +01:00
Markos Chandras 1460a23423 Add support for using distribution packages for OpenStack services
Distributions provide packages for the OpenStack services so we add
support for using these instead of the pip ones.

Depends-On: I5a78e2120e596d36629b4ba978b2b5df76b149b0
Depends-On: Ib64dcbc960df7d369d202ce8cf7bdc29b3ee0e0a
Depends-On: Id9dd2dea146709414ab9ce8d439f1587e6776fd4
Depends-On: I2ba89e25c0010c9a5b515a3d0c9c731b30876e74
Depends-On: I0442b0aa94c3d0882d1118ad0c824d123bd21c88
Change-Id: I26848678dd07a409ef3e159cffb4ba6f0a228ab4
Implements: blueprint openstack-distribution-packages
2018-05-14 13:13:41 +01:00
Jesse Pretorius 8367e44230 Use the venv contents instead of a git source for templates
When deploying the base templates for api-paste, policy files
and other files which are included in the service git source,
we now use the venv files instead of requiring access to a
git source and a complex set of lookups and variable
implementations.

This is simpler and more cross-series, and works from Queens
due to the related bug's patches.

Change-Id: I6a4e2514e66b15b2ae227e62b6dc9ae1a50a4fbd
Related-Bug: #1718356
2018-04-03 09:53:34 +01:00
Logan V 5e23c765b3 Add external LB management handler hook interface
Based on conversation on an ansible issue[1], I implemented
a LB orchestration role[2] similar to the POC here[3].

This will allow external loadbalancer management roles to hook
into a universal notify listener "Manage LB" to perform before/
after endpoint management actions when the service is being
restarted.

[1]: https://github.com/ansible/ansible/issues/27813
[2]: https://github.com/Logan2211/ansible-haproxy-endpoints
[3]: https://github.com/Logan2211/tmp-ansible-27813

Change-Id: Ide9efbc79e4fd2c761a3ee4f463f501181da1df2
2017-09-16 13:41:28 -05:00
Jenkins ce95b2669e Merge "Split user create and ssh key generation" 2017-06-22 21:34:10 +00:00
Jesse Pretorius 43d03a03cc Split user create and ssh key generation
In order to allow an install and config split, but not
to have ssh keys left inside an pre-installed container,
the two tasks are split and tagged appropriately.

Change-Id: I468d1178179d70edfe4b19d40a9a32b35ad18258
2017-06-21 06:23:09 +00:00
Jesse Pretorius aad8144bfb Implement serialisable rolling upgrade
The current rolling upgrade implementation
relies on the role to orchestrate the rolling
upgrade. When the role is executed using
playbook serialisation, the db sync contract
is executed before all hosts are upgraded,
potentially resulting in data corruption.

This patch returns the role to the simpler,
best practice model of expecting that the role
is applied to a single host and that the
playbook handles orchestration. This method
can be used with any form of serialisation.

Depends-On: Ie90cdcbf9e73082a2074c8832b7490d188e178af
Change-Id: I5650f16b9a115bd392012b743788057a94d09226
2017-06-19 19:32:45 +00:00
Jesse Pretorius c59387340f Use uri module for git sourced configs
The current mechanism uses a lookup, causing the
fetch of the default templates to happen via curl
from the deployment node. This causes problems if
the deployment node does not have access to the
repo server web service, which may be the case in
high security environments.

This patch changes the mechanism to only use the
lookup module for the file content that is on the
deployment node, then falls back to using the uri
module to fetch the default content. This ensures
that the deployment node does not have to reach
into the environment for the content.

Also, in https://review.openstack.org/446235 a
new task was added to template the paste/policy
files from the git source but the owner/group/mode
settings were not implemented. This patch
consolidates the two config_template tasks into
one and ensures that the owner/group/mode settings
are properly implemented for all the files.

Change-Id: I47de2e4b1c864035c8b686e96e65722f312b7280
2017-05-19 16:12:21 +00:00
Jesse Pretorius ffcdaf0c92 Source template files from git or deploy host
Instead of sourcing the paste/policy files from the role,
the deploy host will first be checked, then the git source.

This eliminates our need to carry the template files and
keep them up to date. It also ensures that if a custom git
source or alternative SHA is used, the correct source
templates are used.

Related-To: I97476c42172cace5601f777e771ba0aa649b05ca
Change-Id: I910bb1700bafd48185c15d64401c8f8e93c696ea
2017-04-04 09:51:09 +00:00
Jesse Pretorius 94293c86c2 Perform an atomic policy file change
The policy.json file is currently read continually by the
services and is not only read on service start. We therefore
cannot template directly to the file read by the service
(if the service is already running) because the new policies
may not be valid until the service restarts. This is
particularly important during a major upgrade. We therefore
only put the policy file in place after the service restart.

This patch also tidies up the handlers and some of the install
tasks to simplify them and reduce the tasks/code a little.

Change-Id: Ie913e5eb75f3601107b53bab7bda4a02ab1c1024
2017-04-04 10:49:52 +01:00
Travis Truman a4c5811465 Implementing stricter permissions on config files
The security guide suggests that service config files
should be owned by root and in the service user group with 0640 permissions.

Change-Id: I5dc6e2c44ac5607fc1ff1c9fd2653eb23ef794bf
2017-02-08 09:18:34 -05:00
Jimmy McCrory 04737f5dbd Implement zero downtime upgrades
This patch implements upgrading keystone with zero downtime as the
default installation process. Handlers have been modified to ensure that
the first keystone node is stopped, facilitates the database migrations,
and that it is started and available before restarting any other keystone
nodes. Migrations also now only occur when there is a change within the
installed keystone venv.

This process is documented at
http://docs.openstack.org/developer/keystone/upgrading.html#upgrading-without-downtime

A new test scenario has been added for testing basic upgradability
between releases.

Implements: blueprint upgrade-testing
Change-Id: I0d3cfcb80b64d005d60f4c8445f991855f844796
2016-11-17 08:10:57 -08:00
Jesse Pretorius 56b547eb2d CentOS: Only install Federation IDP/SP Packages when necessary
In https://review.openstack.org/309425 adjustments were
made which resulted in keystone_idp and keystone_sp to
always be defined. Unfortunately the CentOS support for
os_keystone merged after that and reviews did not pick
up the necessary changes.

This patch corrects it so that the repo and packages are
only installed if necessary.

Additionally, the Federation SP callback template was
being unnecessarily copied over. It will now only copy
over when it will be used.

Change-Id: I466f1391893d33be7c83caba70ec93e44dd1d482
2016-11-15 13:02:45 +00:00
Kevin Carter 0de819e92a Implement CentOS 7 support in os_keystone
This change implements CentOS 7 support within the os_keystone role.

Depends-on: I333fb1887339e8dc9ebf10ff137dda3cff629dc0
Change-Id: Ib339cd0657f7008fa48bf74f8d6ddd4b8add2ea1
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2016-08-29 14:20:48 +00:00
Steve Lewis 9082c793cc Install and configure uWSGI
When an Apache + mod_wsgi configuration is not selected, configure
the two Keystone services with uWSGI service profiles.

Two arbitrary ports are selected for uWSGI to listen on, so that it
may be proxied for by a dedicated web server. This is in preparation
for laying down Nginx in a future patch.

Notify events are updated to restart the Keystone uWSGI services
where Keystone's configuration is modified only. Because federation
concerns will be isolated within the dedicated web server, changes
to federation configuration of Shiboleth do not trigger restarts of
uWSGI. Similarly, SSL certificate changes do not trigger restarts.

Change-Id: I99e16a999c496e68fb25fa2630d9b211c9755ea4
Related: blueprint keystone-uwsgi
2016-08-22 11:04:55 -07:00
cloader89 8d046aa91c Update the keystone WSGI application locations
This change will update the file paths of the keystone WSGI
applications so that keystone will run properly and not just
produce errors in the logs. The virtualenvs were the primary reason
for this bug, and this solution allows for the deployment of
keystone without requiring virtualenvs provided that they override
keystone_bin.

Closes-bug: 1586161

Change-Id: Iedf4c2651d304d3034be64c781a6daf7ab0b400f
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2016-08-16 19:50:48 +00:00
Jimmy McCrory 44d053c888 Only install to virtual environment
Remove all tasks and variables related to toggling between installation
of keystone inside or outside of a Python virtual environment.
Installing within a venv is now the only supported deployment.

Additionally, a few changes have been made to make the creation of the
venv more resistant to interruptions during a run of the role.
* unarchiving a pre-built venv will now also occur when the venv
  directory is created, not only after being downloaded
* virtualenv-tools is run against both pre-built and non pre-built venvs
  to account for interruptions during or prior to unarchiving

Change-Id: Ic0a0dac84a26aba2ef0ce5410dc7c722570cd410
Implements: blueprint only-install-venvs
2016-07-06 18:42:09 -07:00
Travis Truman 8797fc707a Cleanup/standardize usage of tags
The numerous tags within the role have been condensed
to two tags: keystone-install and keystone-config

These tags have been chosen as they are namespaced
and cover the two major functions of the role.

Documentation has been updated to inform how each tag
influences the function of the role.

Change-Id: Iea4bff944ce0a35a4b1bc044171472ea44eda323
2016-05-24 15:51:03 -04:00
Travis Truman c35ba5fbce Removing unneeded with_items usage for clarity
Change-Id: Ia893b793b308da076124f7b97d1e558feaa39277
2016-03-06 16:31:19 -05:00
Jesse Pretorius 9dc6e6f7b7 Implement multi-domain LDAP configuration for Keystone
This patch changes the keystone_ldap configuration to allow multiple
LDAP identity back-end configurations to be implemented.

Example implementation in user_variables.yml:

keystone_ldap:
  Users:
    url: "ldap://10.10.10.10"
    user: "root"
    password: "secrete"
    ...
  Admins:
    url: "ldap://20.20.20.20"
    user: "root"
    password: "secrete"
    ...

This will place two configuration files into /etc/keystone/domains/,
both of which will be configured to use the LDAP driver.

 - keystone.Users.conf
 - keystone.Admins.conf

Each first level key entry is a domain name. Each entry below that
are key-value pairs for the 'ldap' section in the configuration
file.

Note that the reason why only LDAP is catered for is due to the fact
that LDAP is the only supported driver in OpenStack for
Domain-specific configuration files.

Also the reason that only the identity back-end is catered for is that
the LDAP driver for the role, resource and assignment back-ends have
been deprecated and are scheduled for removal in Mitaka.

UpgradeImpact:

- keystone_ldap's first key tier is now the domain name.
  An existing keystone_ldap configuration entry can be converted by
  renaming the 'ldap' key to the domain name 'Default'.
  **Note** that the domain name entry is case-sensitive.

- keystone_ldap_identity_driver has been removed, as the driver
  for ldap is now simply 'ldap' and there are no other back-end
  options for Keystone at this time.

Change-Id: Ifa4c42f7dbcc40a256a3156f74f0150384f9ab87
2016-01-26 13:08:57 +00:00
Kevin Carter 423d0cfa7d Implement keystone venv support
This commit conditionally allows the os_keystone role to
install build and deploy within a venv. This is the new
default behavior of the role however the functionality
can be disabled.

Change-Id: Ie9e51926c96125a543e05eaa1912684fb01fecda
Implements: blueprint enable-venv-support-within-the-roles
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2015-10-14 13:59:47 -05:00
Kevin Carter ae74a32d6c Adds the config_template to keystone
The change modifies the keystone template tasks such that it's now
using the config_template action plugin. This change will make so that
config files can be dynamically updated, by a deployer, at run time,
without requiring the need to modify the in tree templates or defaults.

Partially implements: blueprint tunable-openstack-configuration

Change-Id: I957d55df81c7edd4e2a95597a62a75c6bd0f46fe
2015-09-21 11:14:54 +00:00
kevin 5826d683e2 Replaced the copy_update module
This PR replaces the copy_update module with a proper Ansible action
plugin. This change allows for dynamic updates to configuration files
that are ini, json, and yaml.

All of the policy files have been moved to the role templates directories
and the task syntax has been updated to facilitate the new action plugin.

An entry has been added to the ansible.cfg file to inform Ansible to look
into the new directory. In order for the action plugin to work as a
"module" a virtual module was added to the library directory.

Change-Id: I80331628b2c3d426a95c89d9c1b766e2e3f70e6d
Partially implements: blueprint tunable-openstack-configuration
2015-09-10 17:14:03 +00:00
Miguel Grinberg 17ac571e7a Keystone Federation Service Provider Configuration
This patch adds the ability to configure Keystone as a Service
Provider (SP) for a Federated Identity Provider (IdP).

* New variables to configure Keystone as a service provider are now
  supported under a root `keystone_sp` variable. Example configurations
  can be seen in Keystone's defaults file. This configuration includes
  the list of identity providers and trusted dashboards. (At this time
  only one identity provider is supported).

* Identity provider configuration includes the remote-to-local user
  mapping and the list of remote attributes the SP can obtain from the
  IdP.

* Shibboleth is installed and configured in the Keystone containers when
  SP configuration is present.

* Horizon is configured for SSO login

DocImpact
UpgradeImpact
Implements: blueprint keystone-federation
Change-Id: I78b3d740434ea4b3ca0bd9f144e4a07026be23c6
Co-Authored-By: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
2015-08-07 08:44:51 +00:00
Miguel Grinberg 0f3dcf6e0e Keystone Federation Identity Provider Configuration
This change adds the bits necessary to configure Keystone as an
identity provider (IdP) for an external service provider (SP).

* New variables to configure Keystone as an identity provider are now
  supported under a root `keystone_idp` variable. Example configurations
  can be seen in Keystone's defaults file. This configuration includes
  the location of the signing certificate, authentication endpoints and
  list of allowed service providers.

* xmlsec1 is installed in the Keystone containers when IdP configuration
  is enabled.

* The IdP metadata and signing certiciate are generated and installed.

Implements: blueprint keystone-federation
Change-Id: I81455e593e3059633a55f7e341511d5ad9eba76f
2015-07-23 20:46:52 +01:00
Kevin Carter d2bff403bd Updated ldap config to support multi domain
Enables default domain support using ldap. This change moves the
ldap config to the default domain and enables domain specific
drivers.

Change-Id: I85f6610a25617fdea1fc216b53df0ab30260fed9
Cloes-Bug: 1447768
2015-05-08 18:18:46 +00:00
Sudarshan Acharya 8fa62d0829 Managing policy file with default file and user variables.
Currently, the default policy.json files for all projects are just
dropped into place. There is no flexibility to customize, edit or update
certain key-values in the policy.json. This change adds a custom module
called 'copy_updates' that allows any updates to be applied to the
default file.

Implements: blueprint dynamically-manage-policy.json
Change-Id: Iadaaeb935f4602835b1cec858b53d4db8c18b1b0
2015-04-16 17:00:05 +00:00
Kevin Carter 9f68cf8781 Convert existing roles into galaxy roles
This change implements the blueprint to convert all roles and plays into
a more generic setup, following upstream ansible best practices.

Items Changed:
* All tasks have tags.
* All roles use namespaced variables.
* All redundant tasks within a given play and role have been removed.
* All of the repetitive plays have been removed in-favor of a more
  simplistic approach. This change duplicates code within the roles but
  ensures that the roles only ever run within their own scope.
* All roles have been built using an ansible galaxy syntax.
* The `*requirement.txt` files have been reformatted follow upstream
  Openstack practices.
* Dynamically generated inventory is now more organized, this should assist
  anyone who may want or need to dive into the JSON blob that is created.
  In the inventory a properties field is used for items that customize containers
  within the inventory.
* The environment map has been modified to support additional host groups to
  enable the seperation of infrastructure pieces. While the old infra_hosts group
  will still work this change allows for groups to be divided up into seperate
  chunks; eg: deployment of a swift only stack.
* The LXC logic now exists within the plays.
* etc/openstack_deploy/user_variables.yml has all password/token
  variables extracted into the separate file
  etc/openstack_deploy/user_secrets.yml in order to allow seperate
  security settings on that file.

Items Excised:
* All of the roles have had the LXC logic removed from within them which
  should allow roles to be consumed outside of the `os-ansible-deployment`
  reference architecture.

Note:
* the directory rpc_deployment still exists and is presently pointed at plays
  containing a deprecation warning instructing the user to move to the standard
  playbooks directory.
* While all of the rackspace specific components and variables have been removed
  and or were refactored the repository still relies on an upstream mirror of
  Openstack built python files and container images. This upstream mirror is hosted
  at rackspace at "http://rpc-repo.rackspace.com" though this is
  not locked to and or tied to rackspace specific installations. This repository
  contains all of the needed code to create and/or clone your own mirror.

DocImpact
Co-Authored-By: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
Closes-Bug: #1403676
Implements: blueprint galaxy-roles
Change-Id: I03df3328b7655f0cc9e43ba83b02623d038d214e
2015-02-18 10:56:25 +00:00