Commit Graph

122 Commits

Author SHA1 Message Date
Jonathan Rosser bf25a670ff Add worker_self_reference_url to glance configuration
This is a new requirement for glance in 2024.1

Change-Id: I6b696d4ccba91bcf84d92f29de49f3c6def784ac
2024-03-26 19:59:44 +00:00
Aleksandr Chudinov e8de2a5f0b Add property protection configuration
Allows to configure property protection according to
https://docs.openstack.org/glance/latest/admin/property-protections.html

Change-Id: I9b4c0985969a4945ce8a1058206376b0347fbb27
2024-02-28 11:30:50 +02:00
Dmitriy Rabotyagov 2f4344550b Fix iteration over backends config
Backend config is a dictionary and it's attempted to be iterated as dict,
however it was missing items() call.

Issue has been introduced with the patch [1]

[1] https://review.opendev.org/c/openstack/openstack-ansible-os_glance/+/901034

Change-Id: I0a6a96a70f05815871a499935a61af9993186edb
2024-01-18 16:50:11 +01:00
Dmitriy Rabotyagov e4acad2bb6 Fix cinder store configuration
According to configuration guide of using cinder as a storage [1], some
auth data should be provided in storage section.
It also needs show_multiple_locations to be enabled.

[1] https://docs.openstack.org/cinder/latest/admin/volume-backed-image.html

Change-Id: Iacd5b74cbda1fdf48a073dc17b42caa37c2359e5
2023-11-27 11:36:42 +00:00
Dmitriy Rabotyagov aa69a6fe5c Make glance_additional_stores work nicely with multi-store
While we assume that glance_additional_stores can be list of mappings
for multistore glance support, bunch of other logic in role still treats
it as simple list and make verifications against it. So in case one
dares to override variable according to our suggestion, they also need
to override bunch of other things.

We change defaults for `glance_available_stores` variable and always
define it as a multistore list of mappings.

Then we introduce a variable `glance_available_store_types` that is a
list of types for each of configured storage.

Logic of how storages are defined in glance config is also changed now.
Storages won't be defined if there's no "default" record for them in
glance_available_stores.

For each new store that deployer wants to provision, they now can pass
`config` key for glance stores, rather then use config overrides.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/901041
Change-Id: I1416e0f6e3ed79abd10f468b52fc712d35a61bd2
2023-11-27 11:36:33 +00:00
Zuul 4e0ca28c67 Merge "Add quorum support for glance" 2023-09-04 08:19:43 +00:00
Dmitriy Rabotyagov eeab8bf6c8 Use proper galera port in configuration
While <service>_galera_port is defined and used for db_setup
role, it's not in fact used in a connection string for oslo.db.

Change-Id: I6967d3832396f375580039c73510ea4f02002b3b
2023-08-07 07:04:49 +00:00
Dmitriy Rabotyagov a5d922e4ed Add quorum support for glance
This change implements and enables by default quorum support
for rabbitmq as well as providing default variables to globally tune
it's behaviour.

Depends-On: https://review.opendev.org/c/openstack/oslo.messaging/+/873630
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/873618
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/875399
Change-Id: I87a5a1bb5028706e1768f53f53c4567cbdff64c9
2023-07-06 14:04:59 +02:00
Damian Dabrowski fafd49959e Move _glance_available_stores to defaults
We need to define _glance_available_stores outside glance role to
use it in haproxy service definition.

It's a good idea to make `_glance_available_stores` public by moving it
out of role variables to role defaults beforehand.

Change-Id: Ieb10a0e5c9faf72c6bea4c45f7e216469971a1f3
2023-04-20 20:51:58 +02:00
Dmitriy Rabotyagov fc6f342194 Support service tokens
Implement support for service_tokens. For that we convert
role_name to be a list along with renaming corresponding variable.

Additionally service_type is defined now for keystone_authtoken which
enables to validate tokens with restricted access rules

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/845690
Change-Id: Ib7fd1a80affe0fa8c6b030fdbfdd60693f104cd6
Related-Bug: #1948456
2022-06-14 14:36:52 +00:00
Dmitriy Rabotyagov 887a59256e Use systemd_mount native syntax for mounts
Currently we have bunch of limitations related to the format
of ``glance_nfs_client``. While systemd_mount role is flexible enough
to allow mount cephfs or s3fs, variable format has weird assumptions
that we want to change for better flexability.

Since keys of variable are changing, new name for it was picked to
reflect purpose of the variable better.

Change-Id: Ic0d91a3a873b4253255beac79becf01b4a304695
2022-04-12 12:56:38 +02:00
Dmitriy Rabotyagov 091f33c8da Define _glance_available_stores in variables
Glance available stores is usefull outsie of the template as well, so we
move defining variable from Jinja template only to ansible variables.

Depends-On: https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/822998
Change-Id: I312fdede3c3bffa0c897588d33fcc5d5d6988ccd
2021-12-27 16:53:47 +00:00
Damian Dabrowski 5f599df051 Database connection pooling improvements
- Implemented new variable ``connection_recycle_time`` responsible for SQLAlchemy's connection recycling
- Set new default values for db pooling variables which are inherited from the global ones.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/819424
Change-Id: I4fd6de7ca38d561306e8c868c063b68edeafc68a
2021-12-03 11:40:51 +01:00
Dmitriy Rabotyagov 7b719c7ffb Refactor galera_use_ssl behaviour
With PKI role in place in most cases you don't need to explicitly
provide path to the CA file because PKI role ensures that CA is trusted
by the system overall. In the meanwhile in PyMySQL [1] you must either
provide CA file or cert/key or enable verify.

Since current behaviour is to provide path to the custom CA we expect
certificate being trusted overall. Thus we enable cert verification when
galera_use_ssl is True.

[1] 78f0cf99e5/pymysql/connections.py (L267)

Change-Id: I71ebc2fc4e386f3a1599fe73d49fae185ec9d2ff
2021-09-17 13:58:42 +00:00
Jonathan Rosser ffdcbe7ff8 Add variables for rabbitmq ssl configuration
Change-Id: Ie87b33a672168e9ab54c8d7869a09010a25458ad
2021-05-13 14:37:02 +00:00
Dmitriy Rabotyagov b69110a300 Cleanup glance variables
Remove deprecated config options that no longer have effect.
We also set cinder_catalog_info to valid default.

Change-Id: Ic24f9a912fc0e7ef73e4e8de4a8440fbf5ddac17
2021-01-28 19:10:32 +02:00
Dmitriy Rabotyagov b7673143de Add glance cache variables into glance-api
In glance caching doc is stated[1], that some of the variables
should be defined in both glance-api and glance-cache config and should
be exactly the same, otherwise issues might raise.

We also introduce glance_image_cache_stall_time variable to control
cache time reliable across config files

[1] https://docs.openstack.org/glance/train/admin/cache.html#configuration-options-for-the-image-cache

Change-Id: Ic229e71978961546cec5f58a9c963c71e05ffba4
2021-01-13 13:58:22 +02:00
Dmitriy Rabotyagov ed736fd783 Cleanup glance-registry service
Glance-registry service has been removed in V cycle with [1]

We do all necessary cleanup to fully remove service deployment.

[1] https://review.opendev.org/738671/

Change-Id: I0b2e2e39040fd0daef04724f94a39f2d11e4d105
2020-07-07 07:30:17 +03:00
Dmitriy Rabotyagov fbe7177e7d Make possible to run glance-api not with uwsgi
While running as uwsgi glance has malfunctioning interoperable image
import feature. So we add new variable `glance_use_uwsgi` based on which
glance will be either started via uwsgi or as a regular service.
Also once glance_use_uwsgi is true, enable_image_import will be disabled

Change-Id: Icf572c656c24b646110ce3fd90727205c22eff15
2020-04-27 10:54:15 +00:00
Dmitriy Rabotyagov 6495419a9f Fix glance stores
Patch [1] brought a bug to glance configuration due to which glance-api
wasn't brought up. The issue was in the multiple backends issue which
require each backend to be specified in their own section.[2]

[1] https://review.opendev.org/#/c/718513/2
[2] https://docs.openstack.org/glance/train/admin/multistores.html

Change-Id: If0c06a5b727c4f0c3e5abf36bd6d2434f09b5857
Related-Bug: #1846052
2020-04-24 18:30:25 +03:00
Dmitriy Rabotyagov 22c9bc06a9 Update glance_store variables
Some variables were deprecated in rocky and marked for removal in Ussuri
We do replace them not to have things broken afterwards.

Change-Id: I75d2e3631b0dfebb72efd946fd61252bb9b766b0
Related-Bug: #1846052
2020-04-15 18:00:18 +03:00
Guilherme Steinmüller 0b9bcc3e5a Refactor memcached_servers
This patch aims to add a prefix for memcached_server
on each role to give the ability for deployers to
override the location of memcached cluster. I.e users
wants to create a single memcached cluster with k8s
for each service.

We also add pymemcache based on [1]

[1] https://review.opendev.org/711429

Change-Id: I19b74c3bc5119953256d3d8f2a98cb5f23787755
2020-03-13 22:14:53 +00:00
Panagiotis Karalis c27d0123ab Turn the filesystem directory to be configurable
Update the ownership of the directory about NFS mount point(s).

This patch could be also stand as an improvement for future use.

Making the filesystem directory configurable, we are able to store
the image in the different directory (or in a new path) under
glance_system_user_home repo, which is able to be configured
dynamically, for instance, via deployment of a scenario.

Change-Id: I7403ac9bd85ea3ed149e13cb57c51039602f6ba1
Signed-off-by: Panagiotis Karalis <pkaralis@intracom-telecom.com>
2019-09-19 19:45:04 +03:00
Dmitriy Rabotyagov 4b2724b44b Start using uWSGI role
Move service to use uWSGI role instead of iternal task for uwsgi
deployment. This aims to ease the maintenance of uWSGI and speedup
metal deployments as the same uwsgi environment will be used
across all services.

Depends-On: https://review.opendev.org/678025/
Change-Id: I6f129940e55130c289d94138171cee54dbd28fc1
2019-09-04 15:32:08 +03:00
Zuul cce497d5d6 Merge "Add missing pre-reqs for glance cinder store" 2019-07-30 20:32:56 +00:00
Craig McIntyre 2636d7eaea Add missing pre-reqs for glance cinder store
There are a number of missing dependencies in the role when using cinder
store with glance. Specifically rootwrap is required for elevating access
when using os-brick to connect to cinder iscsi/fc volume back end storage.
This patch addresses the following:

 - olso.rootwrap is not included in glance_pip_packages

 - files/rootwrap.d/glance_cinder_store.filters is missing

 - glance user is not added to sudoers

glance_pip_packages updated, missing rootwrap.d and sudoer files now dropped in to
Their required locations by glance_post_install.yml task

Change-Id: I55162bc2bf3cbb8858950e4abcf60a3de9929008
Closes-Bug: #1833725
2019-07-30 15:52:38 +01:00
Dmitriy Rabotyagov 002d2a91ea Use systemd-journald instead of log files
This patch aims to migrate service from usage of regular syslog files
to journald. We also disable uwsgi logging, since it dublicates
requests that are logged by service itself.

Change-Id: I12c5a117d9ca508f24a36a477d2d71c36e6c8c96
2019-07-16 16:09:28 +03:00
Jimmy McCrory e45c741f5b Fix distro installs on Ubuntu
Beginning in the Stein release, Ubuntu distro packages are now using
Python3. This requires additionally installing and using the uwsgi
python3 plugin.

Install the 'python3-glance' package instead of 'glance-api'. glance-api
provides a service config file that conflicts with the one OSA provides.

Change-Id: I24e7a05372b6b1831529c620d3346889d5505f09
2019-06-04 10:20:04 -07:00
Kevin Carter 9748e6b154
Cleanup files and templates using smart sources
The files and templates we carry are almost always in a state of
maintenance. The upstream services are maintaining these files and
there's really no reason we need to carry duplicate copies of them. This
change removes all of the files we expect to get from the upstream
service. while the focus of this change is to remove configuration file
maintenance burdens it also allows the role to execute faster.

  * Source installs have the configuration files within the venv at
    "<<VENV_PATH>>/etc/<<SERVICE_NAME>>". The role will now link the
    default configuration path to this directory. When the service is
    upgraded the link will move to the new venv path.
  * Distro installs package all of the required configuration files.

To maintain our current capabilities to override configuration the
role will fetch files from the disk whenever an override is provided and
then push the fetched file back to the target using `config_template`.

Change-Id: I3e7283bf778a9d686f3ae500b289c1fb43b42b92
Signed-off-by: cloudnull <kevin@cloudnull.com>
2019-02-18 09:06:26 -06:00
Kevin Carter 2edb1b1a4d Correct notification driver
The notification driver setup was resulting in the driver and connection string
on the same line. This is caused by the case statement and how jinja formats
the template when a case statement is present. This change modifies how the
driver string is created using a ternary, which will eliminate the case
statement and render the value of the diver correctly.

Change-Id: If361de5d4112a9e7235972dc7bc5e857c68fef06
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2019-02-07 22:32:47 +00:00
Andy Smith 209e2ba35c Update messaging notification configuration
This patch removes the conditional inclusion of the notification
section of the service configuration. This ensures that oslo.messaging
notifications use the correct transport for deployments that have
separate rpc and notify messaging backends.

This patch conditionally selects the notifier driver for when
ceilometer is enabled.

Change-Id: Ie73bf32a62d0e959e4905de31517b20b83b5c583
2018-09-24 09:55:33 -04:00
Zuul b3e2bcda20 Merge "Add ability to configure cross-origin requests" 2018-09-04 12:21:27 +00:00
Mohammed Naser 788273c6b5 Add ability to configure cross-origin requests
The patch introduces a variable `glance_cors_allowed_origin` which
allows a user to configure a specific origin that can do cross
domain requests.

Change-Id: I45f30d2ea7070e62d5d14ad87c872e98af1d7890
2018-08-31 10:41:05 +00:00
Zuul 5586769730 Merge "Convert role to use a common systemd mount role" 2018-07-31 15:13:33 +00:00
Zuul ae48cd3f18 Merge "Default MQ RPC/Notify credentials/vhosts to match" 2018-07-31 14:48:33 +00:00
Kevin Carter 659e3278d0 Convert role to use a common systemd mount role
The systemd unit files are being converted to use common roles to reduce
code sprawl throughout the playbooks. This change allows us to use a
common systemd_mount role as an include which will give us a consistent
experience when deploying services and setting up their resournces on
OS's that uses systemd.

Closes-Bug: #1774037
Change-Id: I11d083788cd388dab0695878193ab18af1b5038b
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-07-30 18:39:12 +00:00
Jonathan Rosser 33b0e35e06 Remove glance v1 api and tests
The glance v1 endpoints were removed in [1].

[1] 3dde3204d5

Change-Id: I90f0b7c6963c433aba6d91e7f4df4401e410439e
2018-07-30 17:37:23 +01:00
Jesse Pretorius 1bff76c624 Default MQ RPC/Notify credentials/vhosts to match
When the RPC and Notify service are the same, the credentials
must match - otherwise the tasks to create the user/password
will overwrite with each other.

If the two clusters are different, then the matching credentials
and vhost will not be a problem. However, if the deployer really
wishes to make sure they're different, then the vars can be
overridden.

Also, to ensure that the SSL value is consistently set in the
conf file, we apply the bool filter. We also use the 'notify'
SSL setting as the messaging system for Notifications is more
likely to remain rabbitmq in our default deployment with qrouterd
becoming the default for RPC messaging.

Change-Id: If95706a85c68767936e7e9d3618e95f5658f5200
2018-07-26 08:44:12 +00:00
Andrew Smith 6b12020eae Restore rabbit driver ssl config option
The driver option is necessary as the transport_url query param
override requires a value. Default will be to use the rpc setting.

Change-Id: Ifc3414a7de6343ae4e7784ed9f7822c18211bb6c
2018-06-04 12:21:09 -04:00
Andrew Smith 87d070f4be Update to use oslo.messaging services for RPC and Notify
This introduces oslo.messaging variables that define the RPC and
Notify transports for the OpenStack services. These parameters
replace the rabbitmq values and are used to generate the messaging
transport_url for the service. The association of the messaging
backend server to the oslo.messaging services will then be
transparent to the glance service.

This patch:
* Add oslo.messaging variables for RPC and Notify to defaults
* Update transport_url generation
* Add oslo.messaging to inventory
* Add release note

Depends-On: If4326a6848d2d32af284fdbb94798eb0b03734d5
Change-Id: Ib647d87df040c77ee3906b1bf58764ca5f3d765d
2018-05-30 14:33:02 -04:00
Markos Chandras ba64ce3083 Add support for using distribution packages for OpenStack services
Distributions provide packages for the OpenStack services so we add
support for using these instead of the pip ones.

Change-Id: I026a440b6a0fda43b613e30f359b2a23c3c1151f
Depends-On: I5a78e2120e596d36629b4ba978b2b5df76b149b0
Implements: blueprint openstack-distribution-packages
2018-05-04 15:42:26 +01:00
ZhongShengping 4968cfd8d6 Deprecate auth_uri option
Option auth_uri from group keystone_authtoken is deprecated[1].
Use option www_authenticate_uri from group keystone_authtoken.

[1]https://review.openstack.org/#/c/508522/

Change-Id: I5f439e371853921394698bf385b7f1fa012d476e
Implements: blueprint deprecate-auth-uri-option
2018-04-03 14:16:52 +08:00
Kevin Carter 9eb319c22b
Convert role to use a common systemd service role
This removes the systemd service templates and tasks from this role and
leverages a common systemd service role instead. This change removes a
lot of code duplication across all roles all without sacrificing features
or functionality. The intention of this change is to ensure uniformity and
reduce the maintenance burden on the community when sweeping changes are
needed. The exterior role is built to be OSA compatible and may be pulled
into tree should we deem it necessary.

Change-Id: Ie558875fcfbcd92c38d55e2d24087fce90889eaf
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-03-27 14:27:13 -05:00
Major Hayden 1302539f11
Skip logging for haproxy health checks
The health check requests from haproxy cause uwsgi to write a
lot of useless log lines. This can make it more difficult to find
a problem with a particular service.

This patch adds a route to look for the `osa-haproxy-healthcheck`
user agent string, which haproxy uses when performing health checks.
Any requests with that user agent are not logged.

Closes-Bug: 1742718
Change-Id: Id06e939f25299d48f4054eec927505ed2ecdb554
2018-02-23 08:39:37 -06:00
Jimmy McCrory 45cb804dc2 Remove registry options from scrubber config
A recent change in glance removed the glance-scrubber utility's
dependency on the glance registry client. The scrubber now connects
directly to the database and these options are no longer needed.

Change-Id: I7389b7e090882eed93a56a6d8bdb297a7a3f9442
Related-Change: https://review.openstack.org/#/c/510449/
2018-02-14 22:08:41 +00:00
Jimmy McCrory e5ac2f07e3 Add MySQL connection SSL support
When 'glance_galera_use_ssl' is True, use an encrypted connection to
the database using either a self-signed or user-provided CA certificate.

A new non-voting test has been added to verify that the role remains
functional when enabling SSL features.

Partial-Bug: 1667789

Change-Id: I18e9d47e88e61ff287e5120dead49b02cdf1f8ac
Depends-On: I95cc994df5118fce7ce588fc0bff979bc283a6f3
2017-12-13 17:12:14 -08:00
Kevin Carter 477d44cab9
Update glance NFS for systemd
Systemd has the ability to manage mounts and ensure functionality
/ resource management. Using a systemd mount has the benifit of not
requiring writes to the legacy fstab file which can impact OS
functionality especially when deploying on baremetal. This change
moves the glance NFS mount to a systemd unit file allowing systemd
to manage it independently with no potentially breaking impact to
the underlying operating system.

Changes:
 - This PR corrects a long standing issue when using Glance+NFS where
   initial deployment would work but if the playbooks were run again
   it would fail due to the glance images location being an NFS mount
   point with a potentially different UID/GID. To correct this we stat
   the directory and if it does NOT exist it is created.
 - Following the nova pattern options have been provided to set the UID
   and GID of the glance user.
 - To ensure out NFS backend solution works with the installation of
   glance a test has been added to deploy glance using an NFS backend.
 - An upgrade task has been added to this commit to clean up legacy
   mounts, This task should be removed in R.

Change-Id: I716c9fe35391629532e67e212d45ea27a5422d1b
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-12-12 17:49:55 -06:00
Jimmy McCrory 43aa00424f Fix systemd init template for program override
The systemd-init template was not looking at the program_override
variable within each service's dictionary.

This also fixes glance-api so that it's running under uWSGI when the v1
API is disabled. Creating images from a remote URL is exclusive to the
v1 API and does not work when glance-api is run under uWSGI.

The libxml2-dev package is required by uWSGI and has been added to the
distro package list.

Additional options have been added to the uWSGI configuration to better
support requests containing chunked data (image uploads).

Change-Id: I14f3162a4666d770beec9746469021466fa4d449
2017-12-06 08:56:48 -08:00
Jesse Pretorius 4e0c72cfe9 Template api/registry enablement settings properly
If a deployer sets the boolean vars to 'yes', 'no' or
other equivalent booleans valid for ansible, the glance
services will currently use those in the .conf files
and it will break.

This patch ensures that we cast the output into the
.conf files properly so that regardless of the input,
the output is correct.

The tests are adjusted to deliberately use one of the
Ansible boolean values instead of a python boolean to
validate that this works.

Change-Id: I267b97da618bbda05140d2a0332798fc77db06a5
2017-11-21 09:41:42 +00:00
Jesse Pretorius 1416013cd4 Allow deployment without glance-registry
The glance v1 API is deprecated and intended to be removed
from the glance code within the Queens or Rocky cycles.

When using the glance v2 API the glance-registry service is
optional, and the intention is to remove the glance-registry
service in the S cycle. The glance-registry service is required
when using the v1 API though.

Furthermore, when using the glance-registry service it is not
possible to execute a rolling upgrade without losing API
transactions.

Given the above information, this patch enables the deployment
of glance with only the v2 API enabled, and without the
glance-registry service. It adds a per-commit test to validate
that this configuration works.

This patch also corrects a previous misconfiguration which
enabled the v2 registry service, but did not set the data_api
correctly for the API service to inform it that the registry
was operating.

The glance_enable_v1_registry variable is also removed as it
is meaningless. The v1 API *requires* the registry to be
enabled, so we just enable it if glance_enable_v1_api is
enabled.

Change-Id: Ie95daed286798d139f0a35ffdd2a4dd1cdda6ff9
2017-11-20 12:48:02 +00:00