Commit Graph

141 Commits

Author SHA1 Message Date
Nobuto Murata d9873fd22c Add support of a common volume_backend_name
Previously, we didn't have a control over volume_backend_name other than
the default app name in the Juju model. A common backend name to
multiple backends with the same character is useful because those can be
treated as a single virtual backend associated with a single volume
type.

Change-Id: I4b57f7979837d21a1b116007f3da707ee154792b
Closes-Bug: #1884511
2021-03-30 15:10:14 +09:00
Ionut Balutoiu b8efc0e698 Fixes for ceph-replication-device relation
* Re-trigger `ceph_access_joined` from `ceph_replication_device_changed`.
  Without this, we could end up with incomplete `ceph-access` relation
  if `ceph_access_joined` is executed before `ceph_replication_device_changed`.

* Seed `replication-device-secret-uuid` early in `config-changed`, to make
  sure that it's set if `storage_backend` is executed before `ceph_access_joined`.

* Set `secret_uuid` as part of the `replication_device` config in the backend
  config. Without this, Nova won't be able to access the proper secret for
  the Cinder Ceph volumes, after a failover.

Change-Id: Ic023d05d5d17a663e1719de393bdd15f18a40484
2021-01-14 13:25:12 +00:00
Marius Oprin 3ff94b3581 Implement new relation: 'ceph-replication-device'
Add CephReplicationDeviceContext Class
Add CinderCephContext Class

Co-authored-by: Ionut Balutoiu <ibalutoiu@cloudbasesolutions.com>
Change-Id: I5f9139294c63f1adec3bbbe3bb9c99b2e7f5413b
2020-12-17 21:46:49 +02:00
Ionut Balutoiu 026233bd7c Add 'rbd-mirroring-mode' to replicated 'create-pool' broker req op
The new config option is applied only to the broker request to
create the charm replicated pool.

Co-authored-by: Marius Oprin <moprin@cloudbasesolutions.com>
Change-Id: I6bf9544af02d0622b8f714da97b5dbcf49d1d1af
2020-12-15 15:38:16 +02:00
Liam Young 9e2db44075 Waiting state if broker request is incomplete
When assessing the charms status check that the current
ceph broker request has been complete. If it has not
put the charm in a 'waiting' state and update the status
message.

Change-Id: Iaaa2021a86b7e360f3255a52b27a49ef859beecd
Closes-Bug: #1899918
2020-11-04 13:43:55 +00:00
Liam Young a55a6e22c2 Send application name to ceph-mon
Send application name to ceph-mon as ceph-mon cannot derive it from
CMR relations.

Change-Id: Idffa90cb6e46cf016eac5ecff0094e4d145d67ac
2020-10-01 10:34:51 +00:00
Frode Nordahl 86986d13ce
Add BlueStore Compression support
Unpin flake8

Func-Test-Pr: https://github.com/openstack-charmers/zaza-openstack-tests/pull/398
Change-Id: I19de6b53a8f6c8f6492f5acccc3ea8d8a14cc4d9
2020-09-17 12:34:26 +02:00
Pedro Guimaraes 9f02453256 Add support for Erasure Coded pools
Enable support for use of Erasure Coded (EC) pools for
Cinder volumes.

Add the standard set of EC based configuration options to the
charm.

Update Ceph broker request to create a replicated pool, an erasure
coding profile and an erasure coded pool (using the profile) when
pool-type == erasure-coded is specified.

Resync charm-helpers to pick changes to the standard ceph.conf
template and associated contexts for rbd default data pool mangle
due to lack for explicit support in OpenStack Services.

Update context to use metadata pool name in cinder configuration
when erasure-coding is enabled.

Change-Id: Iae0b9ba2e57a0dcc4ba1074ebeba4c644f1d830c
Co-Authored-By: James Page <james.page@ubuntu.com>
Depends-On: Iec4de19f7b39f0b08158d96c5cc1561b40aefa10
2020-08-07 14:52:55 +01:00
Nobuto Murata 271e2b0f1b Allow specifying AZ for the volume backend
We cannot rely on JUJU_AVAILABILITY_ZONE so let admin(s) set AZ for the
storage backend explicitly through a charm config. Nova-compute charm,
for example, can use JUJU_AVAILABILITY_ZONE because AZ can be set per
unit / compute node basis. However, when it comes to Cinder backends, we
cannot use JUJU_AVAILABILITY_ZONE of cinder-{api,scheduler,volume} units
since those are not related to where storage backends reside. Ceph-mon
units are not suitable either since it consist of three units usually
and the common JUJU_AVAILABILITY_ZONE across those units to represent
the volume backend AZ is not assured.

Change-Id: I38f6926b859de46efde9219f4be7dde83e0a7985
Closes-Bug: #1884014
2020-06-21 07:01:57 +09:00
James Page f200d8dff5 Require relation to nova-compute application
Without a relation to at least one nova-compute application a
cinder-ceph backend will not be functional as the libvirt
secrets will not have been created to allow access to the
ceph cluster from libvirt/qemu.

Add a simple context to check that the 'ceph-access' relation
is present.  This will result in a blocked status if the
relation is not detected - for example:

  Missing relations: nova-compute

Change-Id: Iedbf4aafc2348cbf6f14257417e86aa9aeb48a81
Closes-Bug: 1718051
2020-05-28 06:46:39 +01:00
Zuul cd6249252e Merge "Fix log update-status error" 2019-08-14 14:10:15 +00:00
Alex Kavanagh 18ac7a93cc Fix log update-status error
This patch adds a dummy update_status function so that the update-status
hook 'has' a function to run and thus silence the log error.

Change-Id: I6a73943af9609810f1e40789c8d351278843dece
Closes-bug: #1837639
2019-07-24 11:31:13 +01:00
Edward Hope-Morley 276a567e4f Fix amulet test that should have been resolved in commit 616ba36
Also resolve merge collision between commits 2f8b158b and
616ba364 that accidentally enabled rbd_flatten_volume_from_snapshot
for >= Ocata rather than >= Queens (which is also causing the
amulet test to fail).

Change-Id: I8a8b95d34f498cc3a7a52aaf90a8684ab80399b3
2019-06-18 17:06:44 +01:00
Zuul 43ce391086 Merge "Add rbd_exclusive_pool support for Ocata and Pike" 2019-05-13 16:05:11 +00:00
Liam Young 2f8b158b15 Allow users to choose to flattern volumes
Expose rbd_flatten_volume_from_snapshot to the user which allows
then to flatten volumes created from snapshots to remove
dependency from volume to snapshot.

Change-Id: I22a3c82535efac5334dd5deaadbba0dd1eae83ab
Closes-Bug: #1824582
2019-04-17 15:32:11 +00:00
Edward Hope-Morley 616ba36429 Add rbd_exclusive_pool support for Ocata and Pike
The fix has been backported so we can now use it.

Change-Id: I5dfad42def68634313af05d21ebe61bc229ebc35
Closes-Bug: #1704106
2019-04-11 12:18:16 +01:00
Frode Nordahl 00b069df3f Request Ceph application name ``rbd`` for pools we create.
Use cases for the Ceph pool application name tagging is emerging
and thus far the protocol appears to be ``rbd`` or ``rgw``.  Others
might emerge too.

We make use of this to provide "it just works" behaviour to the
ongoing ``rbd-mirror`` feature work in the Ceph charms.

Sync charm-helpers.

Change-Id: Id8e59abdf5aaf578e9f11a223a79209fa971f51c
2019-02-19 19:42:26 +01:00
Ryan Beisner d3cfc3ff13
Update pre-install hooks to fail on error
The pre-install operations may fail, yet that failure is not
elevated to the user. This masks the failure and makes early
package install issues difficult to troubleshoot.

If the basic pre-install script fails, the charm should not
proceed to later hooks as the requirements may not be met.

Hashbangs for bash should specify -e (errexit) on all of the
pre-install bash scripts.

Change-Id: I67b3a965b11844a2768800225396927f17864116
Closes-bug: #1815243
Partial-bug: #1815231
2019-02-08 15:48:23 -06:00
Chris MacNaughton 511f2f0a19 Ensure we use the correct pool name
Change-Id: Ieaee887b03ac4e9ae880ff27e0660f9269c16c2e
Closes-Bug: #1814603
2019-02-05 09:52:24 +01:00
Alex Kavanagh 9802731e58 Migrate charm to python3 only runtime
Change-Id: I921f3e2fc8250ca7250ca7dfc47e12644ce4ef98
2019-01-31 22:05:20 +00:00
Liam Young df6b239526 Expose rbd-pool-name
In some situations an existing rbd pool may already be populated
with images that are in use. This is the case when migrating
from the old topology where cinder had a direct relation to
ceph-mon.

Change-Id: I93eb801ca4a166f862d5d86711d9476c61851344
2019-01-23 17:51:46 +00:00
Pete Vander Giessen 4e3bf3e7f3 charmhelpers-sync
Pull in fix for call to ceph-authtool when updating keys.

Change-Id: I8e676cc3cfecf6cfb832197aebe0570d879266fd
Closes-Bug: #1810917
2019-01-16 11:08:14 -05:00
Corey Bryant bd98873277 Sync charm-helpers
Change-Id: Icad06b681d34e4a4fc04c8bd77d2aab3b85aaa1d
2019-01-11 14:36:16 +00:00
Zuul 3e4dece9ba Merge "fix typos in the docstring" 2019-01-09 09:33:07 +00:00
Corey Bryant e7dcb3fc5b Sync charm-helpers
Change-Id: Icad296a5c3142424931d4fb9b549f674f8996850
2019-01-08 15:16:35 +00:00
melissaml 3852a8586e fix typos in the docstring
Change-Id: Iad78f32b3a76feb332ef2d31117a871291acc0d7
2018-12-22 03:52:32 +08:00
Syed Mohammad Adnan Karim 18cbf7f6e1 Update hook to remove old ceph.conf alternatives
When adding ceph-mon relation to cinder, the charm installs ceph.conf
with the update-alternatives via cinder_utils.resource_map().
However when the relation is removed, the alternative isn't cleaned up.
This can cause issues if installing a cinder-ceph subordinate charm.
The cinder-ceph charm also installs a ceph.conf alternative that will
point to the leftover ceph.conf installed by the ceph-mon charm.

Added remove_alternative() in ceph-relation-broken hook to ensure
that leftover ceph.conf alternatives is removed upon relation removal.

Change-Id: I308e62a626f31eb8ef690a09035fe3908920ccc9
Closes-Bug: 1778084
2018-11-26 19:04:14 +00:00
Ryan Beisner 50eec67129
Sync charm-helpers
Change-Id: Ic222ca663b5ba18bcdb2d1ee1f3a7e1de9bb8a1d
2018-11-07 15:34:07 -06:00
Edward Hope-Morley 9444b4402c Enable discard
By default nova/libvirt will not enable trim for
attached volumes so to allow users to use this
feature we now enable it by default.

Also removed < Icehouse unit test.

Change-Id: I58ffaa43e2836068aeed7795df670d279d5e28f8
Closes-Bug: #1781382
2018-09-28 10:29:30 +01:00
Zuul ac53d3994d Merge "Series Upgrade" 2018-09-25 16:54:16 +00:00
David Ames ea09a8bee6 Series Upgrade
Implement the series-upgrade feature allowing to move between Ubuntu
series.

Change-Id: Ib101bcc6055ebec230d202383e9aa82535dc0533
2018-09-22 20:20:19 +00:00
Zuul 11d87b8d5a Merge "py3: Enable Python 3 at Rocky" 2018-09-21 10:19:52 +00:00
Ryan Beisner c0d4b93ebc Add cosmic
Add a tactical change which is already merged into charm-helpers.

This needs to go into all charms to solve the chicken:egg issue
where cosmic is untestable until this change exists.

Reference:

4835c6c167

Change-Id: Ic979610078651e4479f2c251c809e7ff3f542e73
2018-09-19 13:25:48 +02:00
James Page 3f3eb45777 py3: Enable Python 3 at Rocky
Misc updates for rocky:

  - Switch default smoke test to bionic-rocky
  - Resync charm helpers

The change for this charm is minimal as it directly uses
ceph-common.  The cinder charm actually deals with installation
of the required python-rados/rbd libraries for the ceph
integration.

Change-Id: Ic2ee4b845ab604d80b7e27492f522d57f9463af1
2018-09-18 15:08:48 +02:00
Edward Hope-Morley b04b5f3b5f Add support for rbd_exclusive_cinder_pool
As of the of the queens release cinder supports this config
option which, if enabled, stops cinder from query all
volumes in a pool every time it does a delete in order to
get accurate pool usage stats. The problem is that this
causes tons of non-fatal race conditions and slows down deletes
to the point where the rpc thread pool fills up blocking
further requests. Our charms do not configure pool by default
and we are not aware of anyone doing this in the field so
this patch enables this option by default.

Change-Id: I5377e2886a6e206d30bd7dc38a7e43a085aa524c
Closes-Bug: 1789828
2018-09-11 14:02:43 -06:00
Ryan Beisner f2d17851a4 Update functional test definitions
Change-Id: Id439ece547dc1922211e8aebba6e8c872928a3c0
2018-08-27 18:14:03 -05:00
Chris MacNaughton a4c6c088f7 Sync charm-helpers to ensure Rocky support
Change-Id: Ibb458a0eae4c4cc25dc5eb511e91aae0ed53106b
2018-07-13 15:53:47 +02:00
Corey Bryant 9dfc73e772 Sync charm-helpers for Rocky series support
Change-Id: Ide82e8b03af359146325bfccbadf61f558b027fd
2018-06-21 19:07:29 +00:00
Chris MacNaughton 930d7c3608 Update tests to use Juju storage
Due to changes to the ceph-osd charm, it is
suggested to use Juju storage for testing.

Change-Id: I14ab9533a53105f8edc2c4af1d98b336a898df00
Related-Bug: #1698154
2018-06-04 17:31:46 +02:00
David Ames 65a188172f Enable Bionic as a gate test
Change bionic test from dev to gate for 18.05.

Change-Id: I808c6451583d66c8b8a17ed1bffc8a37660f6022
2018-05-16 13:28:02 -07:00
James Page b1829e983a Tidy ceph backend configuration
Drop generation of upstart override file and /etc/environment
and scrub any existing charm configuration in these locations
from an existing install.

These where required way back in the dawn of time when ceph
support was alpha/beta in cinder.

Provide backend specific configuration file path, allowing
multiple ceph clusters to be used with a single cinder
application.

Change-Id: I8a097e4de1c5c980f118a587a1a64792fad2fa05
Closes-Bug: 1769196
2018-05-10 17:14:17 +01:00
Liam Young d6ef6b317d Ensure cinder-ceph installs its prerequisites
The charm has been assuming that the principle charm will install
the packages this charm needs to run. This is not always the case
so the change forces the charm to install what it needs.

Change-Id: I1a394bd9f0a008a403d36ba5d7332b7fb5659006
Closes-Bug: #1754007
2018-03-20 09:04:23 +00:00
Ryan Beisner 1a956ebf73 Sync charm-helpers
Change-Id: Iac5794aec967ffc85a42659d469769a63b2cf428
2018-02-21 14:08:15 -06:00
Ryan Beisner b86be52fd7 Sync charm-helpers
Notable issues resolved:

openstack_upgrade_available() broken for swift
https://bugs.launchpad.net/charm-swift-proxy/+bug/1743847

haproxy context doesn't consider bindings
https://bugs.launchpad.net/charm-helpers/+bug/1735421

regression in haproxy check
https://bugs.launchpad.net/charm-helpers/+bug/1743287

Change-Id: Ieb05f48ad52a1dc8d79bcffec1c467495bbf0001
2018-01-19 17:55:38 +00:00
Liam Young 194b9274a6 Request class-read object_prefix rbd_children perm
When using ceph as a backend request the additional privilege
class-read on rbd_children. This fixes bug 1696073.

Change-Id: I023781e01c1e314cb2755e7867cdf588432791fc
Closes-Bug: #1696073
Depends-On: Icf844ec7d33f2e558dee7935fe5fa3d7f08e0d59
2017-12-15 11:06:10 +00:00
Ryan Beisner 34f39329c0 Update amulet test definitions
Also sync charm-helpers

This change requires the following charm-helpers change
to land first:

 - https://github.com/juju/charm-helpers/pull/32

Change-Id: I2c33e25e14ad49d65f5fc7eb000830086cf829c1
2017-12-06 15:39:54 -08:00
Ryan Beisner b770073b55 Sync charm-helpers
Change-Id: Ic7e246ea945ac785967d7f9e1eee2b20f861a1a1
2017-11-28 14:37:14 +00:00
Ryan Beisner 4e4c949561 Sync charm-helpers
Change-Id: Id6d32b2e0fb951617970eb1c19f67c8d02916e3f
2017-08-24 16:51:57 -05:00
Chris MacNaughton b928e65019 Ensure we setup the ceph-access relation
Change-Id: I0d95523fd84338ca55e141c12231f82bf99056df
Closes-bug: #1711642
2017-08-22 15:30:15 +02:00
Chris MacNaughton 6cc19d8f36 Setup Ceph default RBD features
Nova-lxd requires that our Ceph images only contain the features
supported by the kernel RBD driver, and a discussion on the dev mailing
list suggests that 1 should work fine as the driver level

This commit contains a charmhelpers sync to bring in the new
flag to support configuration sent from the ceph charms.

Change-Id: I860584810dc3b8923635d7d45cc468ea96e4ce07
2017-08-01 09:09:40 +02:00