Commit Graph

51 Commits

Author SHA1 Message Date
Alex Kavanagh f8ebb908d7 Improve platform mocking
Patch out charmhelpers.osplatform.get_platform() and
charmhelpers.core.host.lsb_release() globally in the unit tests to
insulate the unit tests from the platform that the unit tests are being
run on.

Change-Id: Ifda8aa2675b4a756109254f62b9a600c42679cac
2023-10-25 14:40:56 +01:00
Nobuto Murata d9873fd22c Add support of a common volume_backend_name
Previously, we didn't have a control over volume_backend_name other than
the default app name in the Juju model. A common backend name to
multiple backends with the same character is useful because those can be
treated as a single virtual backend associated with a single volume
type.

Change-Id: I4b57f7979837d21a1b116007f3da707ee154792b
Closes-Bug: #1884511
2021-03-30 15:10:14 +09:00
Ionut Balutoiu b8efc0e698 Fixes for ceph-replication-device relation
* Re-trigger `ceph_access_joined` from `ceph_replication_device_changed`.
  Without this, we could end up with incomplete `ceph-access` relation
  if `ceph_access_joined` is executed before `ceph_replication_device_changed`.

* Seed `replication-device-secret-uuid` early in `config-changed`, to make
  sure that it's set if `storage_backend` is executed before `ceph_access_joined`.

* Set `secret_uuid` as part of the `replication_device` config in the backend
  config. Without this, Nova won't be able to access the proper secret for
  the Cinder Ceph volumes, after a failover.

Change-Id: Ic023d05d5d17a663e1719de393bdd15f18a40484
2021-01-14 13:25:12 +00:00
Marius Oprin 3ff94b3581 Implement new relation: 'ceph-replication-device'
Add CephReplicationDeviceContext Class
Add CinderCephContext Class

Co-authored-by: Ionut Balutoiu <ibalutoiu@cloudbasesolutions.com>
Change-Id: I5f9139294c63f1adec3bbbe3bb9c99b2e7f5413b
2020-12-17 21:46:49 +02:00
Ionut Balutoiu 026233bd7c Add 'rbd-mirroring-mode' to replicated 'create-pool' broker req op
The new config option is applied only to the broker request to
create the charm replicated pool.

Co-authored-by: Marius Oprin <moprin@cloudbasesolutions.com>
Change-Id: I6bf9544af02d0622b8f714da97b5dbcf49d1d1af
2020-12-15 15:38:16 +02:00
Liam Young 9e2db44075 Waiting state if broker request is incomplete
When assessing the charms status check that the current
ceph broker request has been complete. If it has not
put the charm in a 'waiting' state and update the status
message.

Change-Id: Iaaa2021a86b7e360f3255a52b27a49ef859beecd
Closes-Bug: #1899918
2020-11-04 13:43:55 +00:00
Liam Young a55a6e22c2 Send application name to ceph-mon
Send application name to ceph-mon as ceph-mon cannot derive it from
CMR relations.

Change-Id: Idffa90cb6e46cf016eac5ecff0094e4d145d67ac
2020-10-01 10:34:51 +00:00
Frode Nordahl 86986d13ce
Add BlueStore Compression support
Unpin flake8

Func-Test-Pr: https://github.com/openstack-charmers/zaza-openstack-tests/pull/398
Change-Id: I19de6b53a8f6c8f6492f5acccc3ea8d8a14cc4d9
2020-09-17 12:34:26 +02:00
Pedro Guimaraes 9f02453256 Add support for Erasure Coded pools
Enable support for use of Erasure Coded (EC) pools for
Cinder volumes.

Add the standard set of EC based configuration options to the
charm.

Update Ceph broker request to create a replicated pool, an erasure
coding profile and an erasure coded pool (using the profile) when
pool-type == erasure-coded is specified.

Resync charm-helpers to pick changes to the standard ceph.conf
template and associated contexts for rbd default data pool mangle
due to lack for explicit support in OpenStack Services.

Update context to use metadata pool name in cinder configuration
when erasure-coding is enabled.

Change-Id: Iae0b9ba2e57a0dcc4ba1074ebeba4c644f1d830c
Co-Authored-By: James Page <james.page@ubuntu.com>
Depends-On: Iec4de19f7b39f0b08158d96c5cc1561b40aefa10
2020-08-07 14:52:55 +01:00
Zuul 6ff4fe370d Merge "Use unittest.mock instead of mock" 2020-07-13 05:55:52 +00:00
Nobuto Murata 271e2b0f1b Allow specifying AZ for the volume backend
We cannot rely on JUJU_AVAILABILITY_ZONE so let admin(s) set AZ for the
storage backend explicitly through a charm config. Nova-compute charm,
for example, can use JUJU_AVAILABILITY_ZONE because AZ can be set per
unit / compute node basis. However, when it comes to Cinder backends, we
cannot use JUJU_AVAILABILITY_ZONE of cinder-{api,scheduler,volume} units
since those are not related to where storage backends reside. Ceph-mon
units are not suitable either since it consist of three units usually
and the common JUJU_AVAILABILITY_ZONE across those units to represent
the volume backend AZ is not assured.

Change-Id: I38f6926b859de46efde9219f4be7dde83e0a7985
Closes-Bug: #1884014
2020-06-21 07:01:57 +09:00
Hervé Beraud 455cc94d0f Use unittest.mock instead of mock
The mock third party library was needed for mock support in py2
runtimes. Since we now only support py36 and later, we can use the
standard lib unittest.mock module instead.

Change-Id: I7e645c6fcc302ed152c57e997c32ebe658e4d0bd
2020-06-08 22:37:47 +02:00
James Page f200d8dff5 Require relation to nova-compute application
Without a relation to at least one nova-compute application a
cinder-ceph backend will not be functional as the libvirt
secrets will not have been created to allow access to the
ceph cluster from libvirt/qemu.

Add a simple context to check that the 'ceph-access' relation
is present.  This will result in a blocked status if the
relation is not detected - for example:

  Missing relations: nova-compute

Change-Id: Iedbf4aafc2348cbf6f14257417e86aa9aeb48a81
Closes-Bug: 1718051
2020-05-28 06:46:39 +01:00
Liam Young 2f8b158b15 Allow users to choose to flattern volumes
Expose rbd_flatten_volume_from_snapshot to the user which allows
then to flatten volumes created from snapshots to remove
dependency from volume to snapshot.

Change-Id: I22a3c82535efac5334dd5deaadbba0dd1eae83ab
Closes-Bug: #1824582
2019-04-17 15:32:11 +00:00
Frode Nordahl 00b069df3f Request Ceph application name ``rbd`` for pools we create.
Use cases for the Ceph pool application name tagging is emerging
and thus far the protocol appears to be ``rbd`` or ``rgw``.  Others
might emerge too.

We make use of this to provide "it just works" behaviour to the
ongoing ``rbd-mirror`` feature work in the Ceph charms.

Sync charm-helpers.

Change-Id: Id8e59abdf5aaf578e9f11a223a79209fa971f51c
2019-02-19 19:42:26 +01:00
Chris MacNaughton 511f2f0a19 Ensure we use the correct pool name
Change-Id: Ieaee887b03ac4e9ae880ff27e0660f9269c16c2e
Closes-Bug: #1814603
2019-02-05 09:52:24 +01:00
Alex Kavanagh 9802731e58 Migrate charm to python3 only runtime
Change-Id: I921f3e2fc8250ca7250ca7dfc47e12644ce4ef98
2019-01-31 22:05:20 +00:00
Liam Young df6b239526 Expose rbd-pool-name
In some situations an existing rbd pool may already be populated
with images that are in use. This is the case when migrating
from the old topology where cinder had a direct relation to
ceph-mon.

Change-Id: I93eb801ca4a166f862d5d86711d9476c61851344
2019-01-23 17:51:46 +00:00
Syed Mohammad Adnan Karim 18cbf7f6e1 Update hook to remove old ceph.conf alternatives
When adding ceph-mon relation to cinder, the charm installs ceph.conf
with the update-alternatives via cinder_utils.resource_map().
However when the relation is removed, the alternative isn't cleaned up.
This can cause issues if installing a cinder-ceph subordinate charm.
The cinder-ceph charm also installs a ceph.conf alternative that will
point to the leftover ceph.conf installed by the ceph-mon charm.

Added remove_alternative() in ceph-relation-broken hook to ensure
that leftover ceph.conf alternatives is removed upon relation removal.

Change-Id: I308e62a626f31eb8ef690a09035fe3908920ccc9
Closes-Bug: 1778084
2018-11-26 19:04:14 +00:00
Ryan Beisner 5549903533
Fix lint in unit test
Change-Id: I4f380039f31231f8d78acb83d65abb32ba80263c
2018-11-01 14:58:01 -05:00
Edward Hope-Morley 9444b4402c Enable discard
By default nova/libvirt will not enable trim for
attached volumes so to allow users to use this
feature we now enable it by default.

Also removed < Icehouse unit test.

Change-Id: I58ffaa43e2836068aeed7795df670d279d5e28f8
Closes-Bug: #1781382
2018-09-28 10:29:30 +01:00
Edward Hope-Morley b04b5f3b5f Add support for rbd_exclusive_cinder_pool
As of the of the queens release cinder supports this config
option which, if enabled, stops cinder from query all
volumes in a pool every time it does a delete in order to
get accurate pool usage stats. The problem is that this
causes tons of non-fatal race conditions and slows down deletes
to the point where the rpc thread pool fills up blocking
further requests. Our charms do not configure pool by default
and we are not aware of anyone doing this in the field so
this patch enables this option by default.

Change-Id: I5377e2886a6e206d30bd7dc38a7e43a085aa524c
Closes-Bug: 1789828
2018-09-11 14:02:43 -06:00
James Page b1829e983a Tidy ceph backend configuration
Drop generation of upstart override file and /etc/environment
and scrub any existing charm configuration in these locations
from an existing install.

These where required way back in the dawn of time when ceph
support was alpha/beta in cinder.

Provide backend specific configuration file path, allowing
multiple ceph clusters to be used with a single cinder
application.

Change-Id: I8a097e4de1c5c980f118a587a1a64792fad2fa05
Closes-Bug: 1769196
2018-05-10 17:14:17 +01:00
Liam Young 194b9274a6 Request class-read object_prefix rbd_children perm
When using ceph as a backend request the additional privilege
class-read on rbd_children. This fixes bug 1696073.

Change-Id: I023781e01c1e314cb2755e7867cdf588432791fc
Closes-Bug: #1696073
Depends-On: Icf844ec7d33f2e558dee7935fe5fa3d7f08e0d59
2017-12-15 11:06:10 +00:00
zhangyangyang 195deb4355 change assert(Not)Equals to assert(Not)Equal
According to http://docs.python.org/2/library/unittest.html
assert(Not)Equals is a deprecated alias of assert(Not)Equal.

Change-Id: I17bc8fef4090098f3d7e8670cfc34b327ec0210e
Closes-Bug: #1329757
2017-09-11 21:37:36 +08:00
James Page 62613456e7 Fix support for cinder ceph rbd on Ocata
As of Ocata, the ceph key used to access a specific Cinder
Ceph backend must match the name of the key used by cinder,
with an appropriate secret configured for libvirt use with
the cephx key used by the cinder-ceph charm.

Add support for the new ceph-access relation to allow
nova-compute units to communicate with multiple ceph
backends using different cephx keys and user names.

The lead cinder-ceph unit will generate a UUID for use in
the cinder configuration file, and for use by the remote
nova-compute units when configuring libvirt secrets,
ensuring that both ends of the integration match up.

The side effect of this change is that nova-compute will
have a key for use with its own ephemeral backend ceph
access, and a key for each cinder ceph backend configured
in the deployment.

Change-Id: I974ecb39132feddfffabd6dcef401e91b5548d05
Closes-Bug: 1671422
2017-03-13 13:43:07 +00:00
James Page cee77d9414 Add support for cephx pool grouping and permissions
Sync charmhelpers and add configuration option to allow access
to ceph pools to be limited based on grouping.

Cinder requires rwx access to pools associated with volumes,
images and vms (to support rbd snapshots).

Change-Id: If09137f5e36d78ab35d27f88624de5533c34ce53
Partial-Bug: 1424771
2017-02-14 11:19:34 +00:00
James Page cc7256b305 Add support for application version
Juju 2.0 provides support for display of the version of
an application deployed by a charm in juju status.

Insert the os_application_version_set function into the
existing assess_status function - this gets called after
all hook executions, and periodically after that, so any
changes in package versions due to normal system updates
will also be reflected in the status output.

This review also includes a resync of charm-helpers to
pickup hookenv and contrib.openstack support for this
feature.

Change-Id: Ie8c969718b8e34f4b27e7d75f2b58167bf04c14d
2016-09-20 12:06:15 +01:00
Billy Olsen 2e5c583a93 Add ceph-pool-weight option for calculating pgs
Provide the weight option to the Ceph broker request API for requesting
the creation of a new Ceph storage pool. The weight is used to indicate
the percentage of the data that the pool is expected to consume. Each
environment may have slightly different needs based on the type of
workload so a config option labelled ceph-pool-weight is provided to
allow the operator to tune this value.

Closes-Bug: #1492742

Change-Id: I844353dc8b354751de1af5d30b6d512712d40a62
2016-07-14 08:02:03 -07:00
James Page 1012eedc3f Re-license charm as Apache-2.0
All contributions to this charm where made under Canonical
copyright; switch to Apache-2.0 license as agreed so we
can move forward with official project status.

Change-Id: Ibaa7c300be9d7de7d227f56b3ea85f2bfbd0470b
2016-07-01 17:22:51 +01:00
James Page 2fb135098d Add tox support 2015-11-03 14:16:36 +00:00
Liam Young dee39ad6e7 Add workload status support 2015-10-08 15:46:16 +00:00
Edward Hope-Morley 5f62672af8 [gnuoy,r=hopem]
Update cinder-ceph broker code to scale-out bug.

Closes-Bug: #1453940
2015-09-16 03:07:56 +02:00
James Page 227abac9b5 Switch to stateless rather than stateful for flag 2015-09-10 10:34:26 +01:00
James Page ecba1f60e1 Fixup tests 2015-09-10 10:23:56 +01:00
Liam Young 88650e76aa Fix unit tests 2015-09-07 09:36:20 +01:00
Liam Young 292ea3f9a3 Update unit test 2015-08-27 15:43:08 +01:00
Corey Bryant ea788a1b84 [corey.bryant,trivial] Patch out service_restart() in unit test. 2015-07-02 11:31:25 -04:00
Liang Chen liang.chen@canonical.com 26a67b7013 Clean up stale ceph keyring
[cbjchen,r=]
Clean up the keyring after ceph relation is broken. So when next
time ceph relation is joined, ensure_ceph_keyring will not ignore
the new key because of the existance of the old one.
2015-02-04 14:24:05 -05:00
Edward Hope-Morley f933f81dee synced ch and fixed tests 2014-12-15 10:40:21 +00:00
Edward Hope-Morley 68e425023b synced /next 2014-12-15 10:19:03 +00:00
Corey Bryant be8adf17ff Mock out config(). 2014-12-11 02:37:26 +00:00
Edward Hope-Morley 80a19dc6a1 Added ceph broker support 2014-11-20 10:29:40 -06:00
Edward Hope-Morley 42a0253172 [hopem] fixed unit test error introduced by previous commit 2014-10-08 11:55:35 +01:00
James Page 5dde575dc6 Resync helpers, tidy lint 2014-07-15 16:41:42 +01:00
James Page 0f977121e3 Add some units tests for the storage-backend relation 2014-04-09 14:08:54 +01:00
James Page fb23fca9f7 Add some more tests 2014-04-02 09:31:50 +01:00
James Page a44090317d Fixup icehouse driver handling, add test case 2014-04-02 09:13:47 +01:00
James Page 16eda8c99f Fixup hook use of CEPH_CONF 2014-02-12 10:48:29 +00:00
James Page 2eefa60332 Update unit tests to cope with alternatives use 2014-02-12 10:40:54 +00:00