Recursive task against /etc/cinder/rootwrap.d set directory permissions
to 0640 as well, which is not really valid. Also it was not idempotent
with dir creation from pre-install step.
Change-Id: Idbd09c78943d954f42f89acce40096c3754024be
At the moment glance_cinder_store.filters is distributed through
glance_store [1] package.
Moreover, for quite some time glance has migrated to using privsep [2]
so internally maintained filters are not up to date anymore.
[1] f3f5bdb45b/etc/glance/rootwrap.d/glance_cinder_store.filters
[2] c369ba013f
Related-Bug: #900930
Change-Id: Ie097a019f058bac123acf14f51253c272c56aca5
With update of ansible-lint to version >=6.0.0 a lot of new
linters were added, that enabled by default. In order to comply
with linter rules we're applying changes to the role.
With that we also update metdata to reflect current state.
Depends-On: https://review.opendev.org/c/openstack/ansible-role-systemd_service/+/888223
Change-Id: Ifb3711157e77d5c917d05e4a384dead2abe72a7c
Role was never migrated to usage of haproxy-endpoints role
and included task was used instead the whole time.
With that to reduce complexity and to have unified approach, all mention
of the role and handler are removed from the code.
Change-Id: I01225e479e22f3867e811055c8e5e87f644bf46e
When default value has any concatenation, it tries to resolve variables
and always contcatenate even when it's not needed.
With that we need to set defaults for the variables that are inside
defaults, otherwise even if default not needed - task would fail on
default being undefined.
Change-Id: I4f445f280a71173f1b72a3b37bd9d54ea5694ac2
Currently we have bunch of limitations related to the format
of ``glance_nfs_client``. While systemd_mount role is flexible enough
to allow mount cephfs or s3fs, variable format has weird assumptions
that we want to change for better flexability.
Since keys of variable are changing, new name for it was picked to
reflect purpose of the variable better.
Change-Id: Ic0d91a3a873b4253255beac79becf01b4a304695
When we were migrating service to uwsgi usage, we clean forgot to
trigger uwsgi restart on service config change.
Change-Id: I4f230347a9c464244a6bac42e4234b627ca460ed
run_once can't be replaced with such condition since config files should
be put against all groups, and not only api one.
Change-Id: I345fcfbab0b7735ae671ff592c8549eb4dd53b02
We use the same condition, which defines against what host some "service"
tasks should run against, several times. It's hard to keep it the same
across the role and ansible spending additional resources to evaluate
it each time, so it's simpler and better for the maintenance to set
a boolean variable which will say for all tasks, that we want to run
only against signle host, if they should run or not now.
Change-Id: Ida1d88be3ae8f52b6c467563a9b99a7f7d44c4c0
Glance-registry service has been removed in V cycle with [1]
We do all necessary cleanup to fully remove service deployment.
[1] https://review.opendev.org/738671/
Change-Id: I0b2e2e39040fd0daef04724f94a39f2d11e4d105
Glance has dropped default policy.json [1] which
was used by "smart sources". We are fixing this by setting content
to empty dict, that way the only content deployed will be the one
provided by overrides, so that won't change current behaviour.
Additionally `glance_policy_content` has been introduced, which
eventually is going to replace `glance_policy_overrides` in the future.
[1] dd1975bd3e
Change-Id: I3f365684542b390ea02c08ab56f76a447f65a814
There are a number of missing dependencies in the role when using cinder
store with glance. Specifically rootwrap is required for elevating access
when using os-brick to connect to cinder iscsi/fc volume back end storage.
This patch addresses the following:
- olso.rootwrap is not included in glance_pip_packages
- files/rootwrap.d/glance_cinder_store.filters is missing
- glance user is not added to sudoers
glance_pip_packages updated, missing rootwrap.d and sudoer files now dropped in to
Their required locations by glance_post_install.yml task
Change-Id: I55162bc2bf3cbb8858950e4abcf60a3de9929008
Closes-Bug: #1833725
New config file glance-image-import.conf was introduced for a while
and it provides ability to customize image upload process.
As it's optional file, we don't distribute it by default, but only when
it's path to config file is set.
Change-Id: I437647aff026a460e6e5f8b73827e31631fd8d42
The private option on include role was never implemented and
will no longer be developed. This change removes the option
so ansible no longer raises a deprecation warning.
Change-Id: I011fba2674dc4c6888ba93c0ea90f4d60e4b7657
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The files and templates we carry are almost always in a state of
maintenance. The upstream services are maintaining these files and
there's really no reason we need to carry duplicate copies of them. This
change removes all of the files we expect to get from the upstream
service. while the focus of this change is to remove configuration file
maintenance burdens it also allows the role to execute faster.
* Source installs have the configuration files within the venv at
"<<VENV_PATH>>/etc/<<SERVICE_NAME>>". The role will now link the
default configuration path to this directory. When the service is
upgraded the link will move to the new venv path.
* Distro installs package all of the required configuration files.
To maintain our current capabilities to override configuration the
role will fetch files from the disk whenever an override is provided and
then push the fetched file back to the target using `config_template`.
Change-Id: I3e7283bf778a9d686f3ae500b289c1fb43b42b92
Signed-off-by: cloudnull <kevin@cloudnull.com>
===
[DEPRECATION WARNING]: Using tests as filters is deprecated.
Instead of using `result|search` use `result is search`. This
feature will be removed in version 2.9.
===
Change-Id: I158a42b82d1818fa4e9296343b1b4ab787e8d89d
Use a seed in the glance cache management cron task's random
filter so that each host's results are idempotent when re-running.
This prevents the constant changing of the crontab whenever the
playbook is run.
Change-Id: Ia14e867347aeadc6c55cef49f9f2ae0e6f86412f
The systemd unit files are being converted to use common roles to reduce
code sprawl throughout the playbooks. This change allows us to use a
common systemd_mount role as an include which will give us a consistent
experience when deploying services and setting up their resournces on
OS's that uses systemd.
Closes-Bug: #1774037
Change-Id: I11d083788cd388dab0695878193ab18af1b5038b
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
When using glance + NFS the role deploys everything perfectly the first
time however if the role is executed again it will result in failure due
to some base directories being a mount. This change adds a new variable
which will create a list of all NFS mount points. This list is then used
in the required tasks to ensure we're not attempting to recreate
directories that should already exist and are being used as
mount-points.
Change-Id: Id28176833c0b783c20ee1d2ce71fa0654ccf683e
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Systemd has the ability to manage mounts and ensure functionality
/ resource management. Using a systemd mount has the benifit of not
requiring writes to the legacy fstab file which can impact OS
functionality especially when deploying on baremetal. This change
moves the glance NFS mount to a systemd unit file allowing systemd
to manage it independently with no potentially breaking impact to
the underlying operating system.
Changes:
- This PR corrects a long standing issue when using Glance+NFS where
initial deployment would work but if the playbooks were run again
it would fail due to the glance images location being an NFS mount
point with a potentially different UID/GID. To correct this we stat
the directory and if it does NOT exist it is created.
- Following the nova pattern options have been provided to set the UID
and GID of the glance user.
- To ensure out NFS backend solution works with the installation of
glance a test has been added to deploy glance using an NFS backend.
- An upgrade task has been added to this commit to clean up legacy
mounts, This task should be removed in R.
Change-Id: I716c9fe35391629532e67e212d45ea27a5422d1b
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This patch removes the jinja2 delimiters from the when clause
and cleans up the name of the configuration file deployment
task.
Closes-Bug: 1735782
Change-Id: I7e1ead5d3dfc9d36b182c73631801cefefb2fbad
The glance v1 API is deprecated and intended to be removed
from the glance code within the Queens or Rocky cycles.
When using the glance v2 API the glance-registry service is
optional, and the intention is to remove the glance-registry
service in the S cycle. The glance-registry service is required
when using the v1 API though.
Furthermore, when using the glance-registry service it is not
possible to execute a rolling upgrade without losing API
transactions.
Given the above information, this patch enables the deployment
of glance with only the v2 API enabled, and without the
glance-registry service. It adds a per-commit test to validate
that this configuration works.
This patch also corrects a previous misconfiguration which
enabled the v2 registry service, but did not set the data_api
correctly for the API service to inform it that the registry
was operating.
The glance_enable_v1_registry variable is also removed as it
is meaningless. The v1 API *requires* the registry to be
enabled, so we just enable it if glance_enable_v1_api is
enabled.
Change-Id: Ie95daed286798d139f0a35ffdd2a4dd1cdda6ff9
The policy.json file is currently read continually by the
services and is not only read on service start. We therefore
cannot template directly to the file read by the service
(if the service is already running) because the new policies
may not be valid until the service restarts. This is
particularly important during a major upgrade. We therefore
only put the policy file in place after the service restart.
This patch also tidies up the handlers and some of the install
tasks to simplify them and reduce the tasks/code a little.
Change-Id: I81de53d8ddc4b462b878b412e53a6de219b71f86
The security guide suggests that all OpenStack service config files
should be owned by root and in the service user group with 0640 permissions.
Change-Id: I60f8aa7cfb61b15be847fb3d380b0926fb1009dd
Change the 'glance_service_names' from a list to a dictionary mapping
of services, groups that install those services. This brings the
method into line with that used in the os_neutron role in order to
implement a more standardised method.
The init tasks have been updated to run once and loop through this
mapping rather than being included multiple times and re-run against
each host. This may potentially reduce role run times.
Currently the reload of upstart/systemd scripts may not happen if
only one script changes as the task uses a loop with only one result
register. This patch implements handlers to reload upstart/systemd
scripts to ensure that this happens when any one of the scripts
change.
The handler to reload the services now only tries to restart the
service if the host is in the group for the service according to the
service group mapping. This allows us to ensure that handler
failures are no longer ignored and that no execution time is wasted
trying to restart services which do not exist on the host.
Finally:
- Common variables shared by each service's template files have
been updated to use the service namespaced variables.
- Unused handlers have been removed.
- Unused variables have been removed.
Change-Id: Ia74bbcac35c27928f7e96056b9449932253b75de
The numerous tags within the role have been condensed
to two tags: glance-install and glance-config
These tags have been chosen as they are namespaced
and cover the two major functions of the role.
Documentation has been updated to inform how each tag
influences the function of the role.
Change-Id: Iafd4aeed4c76404605c77abae2e6ea57a7ae0355
Remove all tasks and variables related to toggling between installation
of glance inside or outside of a Python virtual environment.
Installing within a venv is now the only supported deployment.
Additionally, a few changes have been made to make the creation of the
venv more resistant to interruptions during a run of the role.
* unarchiving a pre-built venv will now also occur when the venv
directory is created, not only after being downloaded
* virtualenv-tools is run against both pre-built and non pre-built venvs
to account for interruptions during or prior to unarchiving
Change-Id: Icfdfe3a6d47413875a49af3c38537fbd342bc16a
Implements: blueprint only-install-venvs
As per [1] The way of configuring a Swift back-end for Glance
has changed. This change has been done in such a way that when
using Keystone v3 for authentication, the current configuraiton
method breaks.
This patch adjusts the way the configuration is done in order
to achieve the same result as before - a single working Swift
backend for Glance if the deployment opts to use Swift as a
backend.
[1] http://docs.openstack.org/developer/glance/configuring.html#configuring-swift-configuration-file
Change-Id: I02b69cf5985e8788513db58f7e8015f8135b9d58
Closes-Bug: #1561947
This commit conditionally allows the os_glance role to
install build and deploy within a venv. This is the new
default behavior of the role however the functionality
can be disabled.
Change-Id: If58b1f2ebe15672602effe5c06c85524385562ba
Implements: blueprint enable-venv-support-within-the-roles
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The change modifies the glance template tasks such that it's now
using the config_template action plugin. This change will make so that
config files can be dynamically updated, by a deployer, at run time,
without requiring the need to modify the in tree templates or defaults.
Partially implements: blueprint tunable-openstack-configuration
Change-Id: I13fc6f38679267a84104cae948517d185e9f4ece
This PR replaces the copy_update module with a proper Ansible action
plugin. This change allows for dynamic updates to configuration files
that are ini, json, and yaml.
All of the policy files have been moved to the role templates directories
and the task syntax has been updated to facilitate the new action plugin.
An entry has been added to the ansible.cfg file to inform Ansible to look
into the new directory. In order for the action plugin to work as a
"module" a virtual module was added to the library directory.
Change-Id: I80331628b2c3d426a95c89d9c1b766e2e3f70e6d
Partially implements: blueprint tunable-openstack-configuration
glance_nfs_client adds 2 fstab entries, with then handler entry being
incorrectly ordered (src and name are the wrong way around).
This patch removes the redundant handler and moves the existing fstab
task to use the "mount" module which will add the entry to fstab and
ensure the filesystems are mounted.
Additionally this fixes a documentation bug where an incorrect variable
is referenced (glance_nfs_mounts).
Change-Id: I6e0f964d4279800d31119f380a239e2c4ae61cb5
Fixes-Bug: #1477081
Currently, running every 24 hours is long enough to allow the cache to
completely fill up the container's disk. By running the jobs every 6
hours instead (4 times per day), we hope to eliminate the problem where
the container will fill up.
Closes-bug: 1439892
Change-Id: Ie7311f70656447382c2403e79453fe88401efca1
Adding support for dynamically updating the policy files for
nova, glance, neutron, cinder and heat. Uses the copy_update
plugin to detect any updates and applies the changes to the default
policy.json
Implements: blueprint dynamically-manage-policy.json
Change-Id: I573229d6f18a5fe32460b2373ab8b2c36ac722b4
To support a future release target deploying OpenStack Kilo release this
change applies the changes required to provide an updated role for
new deploys of Glance configured to run with the Kilo release.
Highlights:
- added template for glance-manage.conf
- changed api and registry paste.ini files to be templated
- added osprofiler filter to piplines in api and registry paste files
New user secrets:
- glance_profiler_hmac_key added for osprofiler use
New Tunables:
- glance_profiler_enabled for osprofiler use, default 'False'
- glance_http_keepalive for api and registry services, default 'True'
- glance_digest_algorithm for digital signatures, default 'SHA1'
- glance_task_executor for task execution, default 'taskflow'
- glance_policy_dirs & glance_policy_file for alternate policy config
sources, default 'policy.d' and 'policy.json' respectively
- glance_policy_default_rule for policy enforcement, default 'default'
Change-Id: I611a0ce3145861233c81c81084b1648b2b4b4423
Partially implements: blueprint master-kilofication
* Updated Keystone wsgi and paste files from upstream.
* Updated all clients in the openstack_client.yml file.
* Kilo services are tracking the head of master.
* Removed pinned middleware because they're pinned else where.
* Added additional service references for neutron vpnaas, fwaas, and
lbaas which have now been moved into their own repos and no longer
exist within the core neutron repository.
* The neutron vpnaas, fwaas, and lbaas have been removed from the
basic plugins being loaded and a comment has been added to describe
how one might add them back in.
* Updated rootwrap filters for neutron dhcp and l3.
* Updated heat policy.json
* Added the `python-libguestfs` to the nova-compute installation
packages.
* Updates all services to point to the latest kilo tag
Services updated due to deprecated configs:
* Keystone
* Glance
* Nova
* Neutron (is still using the deprecated nova auth plugin)
* Heat
* Tempest
Items for future work post initial release:
* roles/os_neutron/files/post-up-checksum-rules:25:
TODO(cloudnull) remove this script once the bug is fixed.
* roles/rabbitmq_server/tasks/rabbitmq_cluster_join.yml:17:
TODO(someone): implement a more robust way of checking
Implements: blueprint minimal-kilo
Closes-Bug: 1428421
Closes-Bug: 1428431
Closes-Bug: 1428437
Closes-Bug: 1428445
Closes-Bug: 1428451
Closes-Bug: 1428469
Closes-Bug: 1428639
Change-Id: I28a305d9e40a9cf70148ef7d7b00d467a65ca076
This change implements the blueprint to convert all roles and plays into
a more generic setup, following upstream ansible best practices.
Items Changed:
* All tasks have tags.
* All roles use namespaced variables.
* All redundant tasks within a given play and role have been removed.
* All of the repetitive plays have been removed in-favor of a more
simplistic approach. This change duplicates code within the roles but
ensures that the roles only ever run within their own scope.
* All roles have been built using an ansible galaxy syntax.
* The `*requirement.txt` files have been reformatted follow upstream
Openstack practices.
* Dynamically generated inventory is now more organized, this should assist
anyone who may want or need to dive into the JSON blob that is created.
In the inventory a properties field is used for items that customize containers
within the inventory.
* The environment map has been modified to support additional host groups to
enable the seperation of infrastructure pieces. While the old infra_hosts group
will still work this change allows for groups to be divided up into seperate
chunks; eg: deployment of a swift only stack.
* The LXC logic now exists within the plays.
* etc/openstack_deploy/user_variables.yml has all password/token
variables extracted into the separate file
etc/openstack_deploy/user_secrets.yml in order to allow seperate
security settings on that file.
Items Excised:
* All of the roles have had the LXC logic removed from within them which
should allow roles to be consumed outside of the `os-ansible-deployment`
reference architecture.
Note:
* the directory rpc_deployment still exists and is presently pointed at plays
containing a deprecation warning instructing the user to move to the standard
playbooks directory.
* While all of the rackspace specific components and variables have been removed
and or were refactored the repository still relies on an upstream mirror of
Openstack built python files and container images. This upstream mirror is hosted
at rackspace at "http://rpc-repo.rackspace.com" though this is
not locked to and or tied to rackspace specific installations. This repository
contains all of the needed code to create and/or clone your own mirror.
DocImpact
Co-Authored-By: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
Closes-Bug: #1403676
Implements: blueprint galaxy-roles
Change-Id: I03df3328b7655f0cc9e43ba83b02623d038d214e
Dmitriy Rabotyagov