Commit Graph

234 Commits

Author SHA1 Message Date
Alex Kavanagh 39462317b6 Improve platform mocking
Patch out charmhelpers.osplatform.get_platform() and
charmhelpers.core.host.lsb_release() globally in the unit tests to
insulate the unit tests from the platform that the unit tests are being
run on.

Change-Id: I6807128492cb184de3ef5f454d0ea745246b242e
2023-10-24 12:03:17 +01:00
Gabriel Cocenza 51e32a4f7e Add support for HAProxy L7 checks
This change add several configuration options to enable HTTP checks
to the HAProxy configuration, instead of the default TCP connection
checks (which continue to be the default). It also enables /healthcheck
endpoint for cinder-api on openstack releases >= ocata.

Closes-Bug: #1880610
Change-Id: I9d118f70fc1390be7b800ad20ae20e77818adac7
2023-03-29 09:48:30 -03:00
Trent Lloyd ceafd9f67f Fix missing assert_ prefix on called_with
test_clean_storage_unmount incorrectly used 'called_with' instead of
'assert_called_with' which just made a mock function call and didn't
check or assert that the function was actually called.

As a result this function didn't actually test anything, the expected
parameters were not correct so they have been updated.

An error to catch this accidental mistake was added in the recently
released mock 5.0.1 (https://github.com/python/cpython/issues/100690)
  AttributeError: 'called_with' is not a valid assertion.
  Use a spec for the mock if 'called_with' is meant to be an attribute.

The mock library ships in python as unittest.mock but also publishes the
latest code as the 'mock' python package so that older python versions
can get the latest code. The stable branches currently install mock
but the master branch does not, so this error does not actually appear
on master but only the <=yoga stable branches currently. The test is
however wrong either way.

Change-Id: I3076778e8fc62c086651d29abb2c5a3d9921be97
2023-01-25 13:23:37 +08:00
Trent Lloyd ba8d8fc3e1 Add identity-credentials relation support
Implement support for the identity-credentials relation as an
alternative way to get keystone credentials when we are not registering
a service endpoint via the identity-service relation.

This solves an issue where the image volume cache does not work when the
cinder volume service is deployed as a second cinder application
('cinder-volume') having enabled-services=volume set.

Previously the following items were missing from cinder.conf:
cinder_internal_tenant_project_id
cinder_internal_tenant_user_id

Resulting in the image cache not functioning with the following warnings:
Unable to get internal tenant context: Missing required config
parameters.
Unable to get Cinder internal context, will not use image-volume cache.

As there are now two possible interfaces to keystone ('identity-service'
and 'identity-credentials') any existing bundles that don't specify the
interface 'identity-service' when relating to keystone will fail to
deploy and will need to be updated.

Closes-Bug: #1978452
Change-Id: Ieef500c9c55eb3968b3e2e231a8ff6e2a5ec148d
2023-01-20 14:53:39 +08:00
Felipe Reyes ca61c8e7fe Retry online_data_migrations until exit code 0 or 2.
The command 'cinder-manage db online_data_migrations' has three possible
exit codes:

- 0: The migrations have been completed successfully.
- 1: Some migrations have been completed, but more objects need to be
  migrated.
- 2: Some migrations are generating errors and manual intervention is
  needed.

This change will run in a loop the command online_data_migrations in
batches of 5000 objects, when the exit code is 1 runs again, when the
exit code is 0 the loop is stopped, when the exit code is 2 a
CalledProcessError exception is raised.

Change-Id: Iee99e9ab9777392f05c1a0db55951d97773bb0ba
2022-10-07 17:09:24 -03:00
Billy Olsen d33a2d0567 Check for online_data_migrations when syncing db
Due to bug #1908037, the database sync can fail if the volume types
have not been migrated to the __DEFAULT__ volume type. When this fails,
a message will be sent telling the user to run online_data_migrations.
Check for this error, and when it occurs, run the online_data_migrations
and then retry the db sync.

Closes-Bug: #1908037

Change-Id: I0cd9a9db256b0adffbb244222a66c85d055840c4
2022-10-06 21:27:26 -03:00
Corey Bryant cdc2fc738f Add Kinetic and Zed support
* sync charm-helpers to classic charms
* change openstack-origin/source default to zed
* align testing with zed
* add new zed bundles
* add zed bundles to tests.yaml
* add zed tests to osci.yaml and .zuul.yaml
* update build-on and run-on bases
* add bindep.txt for py310
* sync tox.ini and requirements.txt for ruamel
* use charmcraft_channel 2.0/stable
* drop reactive plugin overrides
* move interface/layer env vars to charmcraft.yaml

Change-Id: Ie133bc41bf789e07d4d777e128617b21d822de30
2022-08-26 18:40:29 +00:00
Alex Kavanagh a50281db83 Stop endless cycles on openstack upgrade
If an HA cinder deployment is upgraded twice and the leadership has
changed, there is a potential race that can be triggered where because
the CINDER_DB_INIT_RKEY is different on the units that have been the
leader.  This race is that when the unit tries to decide whether it has
reflected the 'new' RKEY it sees a different one from each unit.  By
clearing the rkey, it ensures that only the relation changed event from
the leader actually causes a restart.

Change-Id: I712502a666f23421cc58925f10b7643e3482d861
Closes-Bug: #1928383
2022-05-12 12:01:40 +01:00
dongdong tao 93dfc624d4 Check if the newhost exists instead of currenthost
Closes-Bug: #1956470

Signed-off-by: Dongdong Tao <dongdong.tao@canonical.com>
Change-Id: I2b9eec030a5b0f6d9a405cecea29bef04eb72f5d
2022-04-25 16:22:53 +08:00
Alex Kavanagh 7c551e4dfa Migrate charm to charmhub latest/edge track
Change-Id: Ida0ad9ac927de7ed8f9206521014404b38187cd6
Co-authored-by: Aurelien Lourot <aurelien.lourot@canonical.com>
2022-02-01 12:52:13 +01:00
Samuel Walladge 7f97a6d30e Add support to configure scheduler_default_filters
Add new config option: `scheduler-default-filters`.
This is unset by default, so cinder retains the default value for
scheduler_default_filters.

Closes-Bug: #1956727
Change-Id: I9777bf8fe5ddbb69689db60c2790e8a4be57e1ab
2022-01-25 11:04:40 +10:30
Hervé Beraud 77fdcf3e27 Use unittest.mock instead of mock
The mock third party library was needed for mock support in py2
runtimes. Since we now only support py36 and later, we can use the
standard lib unittest.mock module instead.

Note that https://github.com/openstack/charms.openstack is used during tests
and he need `mock`, unfortunatelly it doesn't declare `mock` in its
requirements so it retrieve mock from other charm project (cross dependency).
So we depend on charms.openstack first and when
Ib1ed5b598a52375e29e247db9ab4786df5b6d142 will be merged then CI
will pass without errors.

Depends-On: Ib1ed5b598a52375e29e247db9ab4786df5b6d142
Change-Id: I538b5c2a9df4de66c9874e65247496e54c0522e8
2021-12-15 10:56:43 +00:00
Nobuto Murata bde329d973 Make sure iscsid has a unique InitiatorName
os_brick may require InitiatorName in /etc/iscsi/initiatorname.iscsi
before iscsid is invoked via iscsid.socket with iscsiadm. Cloud images
including MAAS ones have "GenerateName=yes" instead of "InitiatorName="
on purpose not to clone the initiator name. Let's initialize it so
Cinder units can be fully ready to accept iSCSI based subordinate and
storage backend charms.

Closes-Bug: 1825809
Change-Id: I413bbb29dd609e0c86ac3691556f37a9fcc13439
2021-12-01 12:17:49 +09:00
Corey Bryant bc48c60f1c Drop creation of v2 endpoint from OpenStack Xena+
In this patch we drop creation of the v2 volume endpoint for
OpenStack Xena onward. Upstream cinder removed the block storage
v2 API in commit e05b261a "Remove Block Storage API v2".

func-test-pr: https://github.com/openstack-charmers/zaza-openstack-tests/pull/670
Depends-On: https://review.opendev.org/c/openstack/charm-cinder/+/816007
Closes-Bug: #1947387
Change-Id: Ie553a53f794f2cdd262bf77364e6374244844b92
2021-11-22 09:18:02 -05:00
Billy Olsen c5896a884a Revert "Group cinder-volume services using cluster config"
This reverts commit 2603a774b1, which
converted cinder to use cluster configuration for stateless services.
While well-intentioned, this causes problems due to the lack of a
proper migration for those backends which are not able to support
ACTIVE-ACTIVE configuration.

Configurations that do not support ACTIVE_ACTIVE driver will have the
volume service fail to start, which will cause a service outage.

Change-Id: I7bf4baaf80e5bb58b5c1cf55a2065f3bc50dbced
Related-Bug: #1945239
2021-10-21 18:55:47 -07:00
Zuul 6bc43bd756 Merge "Group cinder-volume services using cluster config" 2021-10-01 09:09:30 +00:00
Rodrigo Barbieri 2603a774b1 Group cinder-volume services using cluster config
Usage of "host" config for grouping cinder-volume
services for the purpose of stateless config and/or
active-active HA is not recommended.

The usage of "cluster" config is recommended for
achieving that behavior. This change replaces the
usage of "host" with "cluster".

Change-Id: I249ce179cfdaf2394ee4e8481a8c9644d667ea99
Closes-bug: #1945239
2021-09-30 17:23:18 -03:00
Aurelien Lourot 3d043ae144 Process subordinate releases packages map
For principal - subordinate plugin type relations where the
principal Python payload imports code from packages managed by a
subordinate, upgrades can be problematic.

This change will allow a subordinate charm that have opted into the
feature to inform its principal about all implemented release -
packages combinations ahead of time. With this information in place
the principal can do the upgrade in one operation without risk of
charm relation RPC type processing at a critical moment.

Related-Bug: #1806111
Change-Id: Ic8ea4fe6109081814045adea7ce6688b3564c9e5
Co-authored-by: Aurelien Lourot <aurelien.lourot@canonical.com>
2021-09-28 10:44:26 +02:00
eric-chen dd5137c2e7 Add support for configuring image volume cache
Support new configuration to turn on image volume cache in Cinder
service. User can control the maximum size and count from
configuration too.

Closes-Bug: #1869903
Change-Id: If96dbb9c0974bfa1f6d67405bb430a7cd251f821
2021-09-16 15:25:13 +08:00
Alex Kavanagh 14e29e8d89 Updates for testing period for 20.01 release
Includes updates to charmhelpers/charms.openstack for cert_utils
and unit-get for the install hook error on Juju 2.9

* charm-helpers sync for classic charms
* rebuild for reactive charms
* ensure tox.ini is from release-tools
* ensure requirements.txt files are from release-tools
* On reactive charms:
  - ensure master branch for charms.openstack
  - ensure master branch for charm-helpers
* Replace unit-get mock with local_address due to c-h sync changes.

Change-Id: Iec48d0ee3f8bc8f26939c8562fd93a98db06670a
2021-01-18 19:59:02 +00:00
Liam Young 5cd4045b84 Fix mocking
Mocked call to is_device_mounted which was causing a subprocess
call to lsblk.

Change-Id: I375ced9e61ff93c88feda4000f9e3a8b91bf8687
Closes-Bug: #1480840
2021-01-08 11:06:08 +00:00
Alex Kavanagh 5a3bea35c4 Sync libraries & common files prior to freeze
* charm-helpers sync for classic charms
* charms.ceph sync for ceph charms
* rebuild for reactive charms
* sync tox.ini files as needed
* sync requirements.txt files to sync to standard

Change-Id: I6027b4bb12dce36a7b10df2e14cafd5dc6ee963b
2020-09-28 13:07:05 +01:00
Nobuto Murata 12193d2bc2 Allow specifying default_volume_type
It's useful when multiple storage backends to be connected to Cinder.
The corresponding volume type must be created after a deployment via API
to take effect, e.g., `openstack volume type create VOLUME_TYPE
--property volume_backend_name=BACKEND_NAME`

Please note that there is a regression in upstream as LP: #1879578, so
it doesn't work for Train or later releases until the issue gets fixed.

The other way to have the similar effect is to edit the definition of
__DEFAULT__ volume type via API (available for Train or later releases).
Howevers it's not as flexible as the option in cinder.conf since it
doesn't allow any modification unless all of the volumes with the
__DEFAULT__ type get deleted.

Change-Id: I031a6bf2a066bb9d3157e545bb9df782a76551f3
Closes-Bug: #1884548
2020-08-19 17:51:35 +00:00
Alvaro Uria 71f968faf1 Ensure API not configured on volume-only units
COMMON_PACKAGES included haproxy and apache2. However, enabled-services
can request a unit to only run cinder-volume. Cinder API related
services should not run or get monitored.

func-test-pr: https://github.com/openstack-charmers/zaza-openstack-tests/pull/337

Change-Id: I69f15c3fd164f7114f5498d100b2832caf93fb00
Closes-Bug: #1779310
2020-07-01 10:50:03 +02:00
Liam Young 0585d9a96b Do not access DB when it is in maintenance mode.
If the database is in maintenace mode do not attempt to access
it.

Depends-On: I5d8ed7d3935db5568c50f8d585e37a4d0cc6914f
Change-Id: I131079ea390d11c1208f75b8b97ab4cff7fb47ff
2020-01-30 12:35:18 +00:00
Liam Young 605352cfe0 When resuming, exclude haproxy
When resuming services exclude those managed by hacluster, in
this case haproxy. If pacemaker lacks quorum it may shut haproxy
down which will cause this charm to error.

Charmhelper sync included to bring in required
get_managed_services_and_ports method.

Change-Id: I00fa695edb7651098d9c7afdd4df823d66a01aef
2020-01-25 07:45:19 +00:00
Stamatis Katsaounis 0858083637 Fix unit tests
This patch mocks list_thin_logical_volume_pools method for every unit
test running configure_lvm_storage method. Without this mocking these
unit tests are trying to call configure_lvm_storage which in turn calls
list_thin_logical_volume_pools which in turn calls list_logical_volumes
which finally fails to execute lvs command if it is not found in the
running environment.

Change-Id: I9b21e0a16f46071aaae5dfba1699fa98b3281191
Signed-off-by: Stamatis Katsaounis <skatsaounis@admin.grnet.gr>
2019-12-18 11:07:52 +00:00
Alex Kavanagh 6ee32006e5 Policyd override implementation
This patchset implements policy overrides for cinder.  It uses the
code in charmhelpers.

It also fixes several bugs in the bundles where the actual version of
cinder that was being installed was the distro default rather than the
one that the bundle described.

Change-Id: Ic979dcb96ddb931fadb1fa4a4b36108244ddf306
Closed-Bug: #1741723
2019-10-14 15:11:13 +01:00
James Page 8d977e8152 Ensure identity-joined execute post upgrade.
Re-execute the identity-joined relation to ensure that cinder v3
API endpoints are registered for OpenStack Pike or later.

Change-Id: Ia259696b787188218003f63ab510eb143c0ea563
Closes-Bug: 1809634
2019-03-04 11:56:27 +00:00
Alex Kavanagh 834cde35ec Migrate to python3 only charm
Change-Id: Ia4bcf570d40083625358fdd4fea14202ff3d89af
2019-01-30 12:36:21 +00:00
James Page a070870967 upgrade-charm: filter previously installed packages
The upgrade-charm hook installs any new packages required for the
new charm version however this needs to be filtered against packages
that are already installed to ensure that any pending package
updates are not installed as a side effect of a charm upgrade.

Change-Id: Ie23b65fd97014134ba6f3d9004f18a4600cd2d5a
Closes-Bug: 1812982
2019-01-23 17:05:00 +00:00
Liam Young 7873b20126 Use chelper generate_ha_relation_data for ha rel
Use the generate_ha_relation_data helper from charmhelpers to
generate the data to send down the relation to the hacluster
charm.

This results in a few changes in behaviour:

1) The charm will no longer specify a nic name to bind the vip. This
   is because Pacemaker VIP resources are able to automatically
   detect and configure correct iface and netmask parameters based
   on local configuration of the unit.
2) The original iface named VIP resource will be stopped and deleted
   prior to the creation of the new short hash named VIP resource.

Change-Id: I50bddc990cb0182c8ba3a3d473fba2d60186322a
2018-12-03 13:04:34 +00:00
Syed Mohammad Adnan Karim e6a150c0d2 Update hook to remove old ceph.conf alternatives
When adding ceph-mon relation to cinder, the charm installs ceph.conf
with the update-alternatives via cinder_utils.resource_map().
However when the relation is removed, the alternative isn't cleaned up.
This can cause issues if installing a cinder-ceph subordinate charm.
The cinder-ceph charm also installs a ceph.conf alternative that will
point to the leftover ceph.conf installed by the ceph-mon charm.

Added remove_alternative() in ceph-relation-broken hook to ensure
that leftover ceph.conf alternatives is removed upon relation removal.

Change-Id: If9a8d460ee8209ef917fa55ec970379e9c741ec6
Related-Bug: 1778084
2018-11-26 18:51:36 +00:00
Liam Young d3ca911e24 Purge old packages on upgrade-charm
On charm upgrade the charm may switch to py3 packages. If so, ensure
the old py2 packages are purged. If the purge occurs then restart
services.

Change-Id: I3e67d35324888522f6c138ebd74a2ea7f1a4ce05
Closes-Bug: 1803451
2018-11-15 11:33:32 +00:00
Ryan Beisner 8a1009137b
Fix lint in unit tests re: py3-first and py2 compat
Use six for now due to the transitional state of this charm
from py2 to py3.

Change-Id: I24d3cbd66964000600dbcb7849564bb53956fe5c
2018-11-01 21:34:52 -05:00
Ryan Beisner 30166020fb
Fix lint in unit test
Change-Id: I7d43891ab69c881d40a0774bf095de7ac88cade0
2018-11-01 14:58:34 -05:00
James Page e31d5eb0a4 py3: Switch to using Python 3 for rocky or later
Switch package install to Python 3 for OpenStack Rocky or later.

When upgrading, remove any python-* packages that where explicitly
installated and then autoremove --purge any dependencies that are
no longer required.

This change also switches to using the cinder-manage binary in
preference to using the internal API of Cinder to query and manage
services in cinder, avoiding the need to continue to have python-cinder
installed for charm usage.

Change-Id: Ie8f7d2d7e1ef7b3065d6d9ed244e5fd05e2f613b
2018-09-18 12:24:27 +02:00
Craige McWhirter 8369ea875c Only enable the default backend for ocata+
When deploying on icehouse with Ceph, the RBD backend is
configured globally, but cinder sets enabled_backends to "LVM",
which exists (and is wrong to boot).

This commit moves the check for empty "backends" under the ocata
check.  Per comment #5 on LP:1719742: "The cinder-volume service
fails to start in ocata+ if enable-backends is not set."

Closes-Bug: #1753611

Change-Id: I07a9f9c739064bc93870a1f8f68a7624b7ce3033
Signed-off-by: Craige McWhirter <craige@mcwhirter.io>
2018-05-30 09:18:10 +02:00
Liam Young db0a66a4cf Add support for tls-certificates relation
Add support for the charm to request and receive certificates from
the tls-certificates relation.

Change-Id: I25c770272eff7be93deea5c42b4b4b19aeef9bb9
2018-05-15 14:26:03 +00:00
James Page 3451c1c498 Tidy ceph backend configuration
Drop generation of upstart override file and /etc/environment and
scrub any existing charm configuration in these locations from
an existing install.

These where required way back in the dawn of time when ceph support
was alpha/beta in cinder.

Provide backend specific configuration file path, allowing multiple
ceph clusters to be used with a single cinder application.

Change-Id: I7adba0d35fb7406afa40f047b79a9ab51a6a333d
Closes-Bug: 1769196
2018-05-10 11:51:04 +01:00
James Page 4ea9d6a427 Remove deploy from source support
Drop support for deployment from Git repositories, as deprecated
in the 17.02 charm release.  This feature is unmaintained and has
no known users.

Change-Id: Icd464e950c6f53470311e3c110b530a69bff6e2f
2018-01-12 13:59:33 +00:00
James Page 46f0b43dd3 Drop postgresql support
Remove postgresql DB support; This feature is untested as part
of the charms, is not in use and was deprecated as part of
the 1708 charms release.

Change-Id: I456125442dd721c1b7b48b000fe4b4e4322bb476
2017-12-21 15:11:39 +00:00
Liam Young f9937c9007 Extend the cinder lvm thin pool with new devices
If a new device is added to the block-device configuration option
then add the new device to the cinder thin pool if it exists.

From pike+ if the lvm backend is being used then it defaults to
using a thin provisioning pool. This patch makes sure any new
devices are used to extend that pool.

Change-Id: I851e4eb58dbdc122086d53fe346ea045c6044f50
Partial-Bug: #1735931
2017-12-11 10:11:16 +00:00
Liam Young d8d180aaaf Fix _parse_block_device to return (str, int)
_parse_block_device was returning inconsistent types. This seems
to have been working by accident under py2 but fails for py3.

This change fixes _parse_block_device to always returns (str, int)

Closes-Bug: #1732413
Change-Id: If0bfb4030ab9acfdf599ef8e70fed14816df3e26
2017-11-15 14:12:14 +00:00
Edward Hope-Morley 5a5e542f9a Part 2 of upgrades fix in commit c1653c2
In the first patch I forgot to ensure that we also
reload the configs that are globally declared in
cinder_hooks and therefore loaded on import. So here
it is.

Change-Id: I613959fa91912b388aea8bd6926d085956c1d67e
Closes-Bug: #1726527
2017-10-24 18:22:43 +01:00
Edward Hope-Morley c1653c2113 Ensure upgrade installs correct configs
When we do an openstack upgrade we must ensure that
release version dependant decisions are made at the
right time. In this patch we ensure that we choose
configs after the target release packages have been
installed so as to be sure that e.g. when upgrading
from Newton to Ocata the apache2 wsgi conf is
installed for the cinder-api.

Change-Id: I9414940535406c7c7491636839a328df1ac9235b
Closes-Bug: 1726527
2017-10-23 23:24:14 +01:00
Zuul 6c731ad9fe Merge "Add LVM as default backend" 2017-10-17 10:25:49 +00:00
Liam Young f9654dce7d Add LVM as default backend
This change set the enable-backends cinder config option to the
package default of 'lvm'. The reason is to stop the service flapping
and fix the bug which is causing cinder-volume service to be down
in some deployments.

Change-Id: I29b9bccf75019b73f60c1160b8b6c7cb11d3ad28
Closes-Bug: 1719742
2017-10-16 07:56:52 +00:00
Liam Young 1975c41271 Block endpoint reg if cluster partially formed
When an existing cluster of the service is scaled out the new unit
will join with keystone before it is fully clustered. In identity
joined hook the charmhelpers function canonical_url is called which
in turn uses another charmhelpers function, resolve_address.
resolve_address will only return the vip if the vip is set in config
AND the unit is clustered. This means that the units local address
is returned and that is then registered with keystone.

This change gates registering an endpoint if the cluster is
partially formed.

Change-Id: If483147e17dab8de2883058ee0f2718a3b7f8ca6
Partial-Bug: #1544959
2017-10-06 13:15:53 +00:00
melissaml 44ea2653ae Change assert(Not)Equals to assert(Not)Equal
According to http://docs.python.org/2/library/unittest.html
assert(Not)Equals is a deprecated alias of assert(Not)Equal.

Change-Id: I242db7583487d735faba46ac76798cc06967adb6
Closes-Bug: #1329757
2017-09-13 16:54:19 +08:00