When fetching ceph keys, they are upgraded if it's deemed necessary
to fix bug #1968369. This worked for internal ceph entities such
as RBD or MDS, but fails for external ceph clients. This is because
for those clients, key capabilities can be set via broker requests.
As such, when the ceph-mon charm is upgraded, the key capabilities
for those clients are reset, and since the broker request has been
processed already, no further changes are done, leaving those keys
with incorrect caps.
Change-Id: I878091f9387f22059d460d28c2d0072507e66c6f
Closes-Bug: #2028387
This patchset modifies the 'get_running_osds' function so that
it also considers crimson-osds as valid OSD's. In addition, it
also modifies the 'get_local_osd_ids' to _not_ include Crimson
OSDs so that unsupported functionality doesn't make some hooks
fail in the ceph-osd charm.
Change-Id: If58bde4d5445ed5de420abc007db6bf8b8e43269
Update rbd-mirror key caps to use the rbd-mirror-peer mon profile.
This adds required config get/set permissions for the rbd-mirror
daemon.
Also add permission to use the service {dump,status} mon commands;
these are not required by the rbd-mirror daemon but are needed by
the charm to check pool mirror status.
Change-Id: Ia5efa090f5b7fa23d7261da53592d6a527dcbfe1
Closes-Bug: 1879749
Starting with Pacific, the 'osd blacklist' command was renamed to
'osd blocklist'. This patchset changes the allowed commands to
reflect this change.
Change-Id: If2169734f67d21c1c7c1b75677f14ebd0ea054ae
Closes-Bug: #1968369
In order to fully support OSD replacement, we need a way to recycle
previously used OSD ids. This is achieved by passing the parameter
'--osd-id' to 'ceph-volume lvm create'. As such, this patchset adds
the needed support to specify that parameter.
Change-Id: I43d0a0bc11664c37532c0117711affc93c9d1ad1
Add helper functions to parse information from the ceph tree and
fix creating CrushLocation object in get_osd_tree function.
Closes-Bug: #1918721
Change-Id: I59c742e594042ad527e71c88404999459f0373c2
The methods for checking modules are taken from the ceph-mon
charm and charmhelpers.contrib.storage.linux.ceph module. When
this lands I'll update the ch methods to emit a deprecation warning
and update the ceph-mon charm to use these methods instead.
Change-Id: I1b7f08c3bd94828a5c6468df7aab6b38517cf3fa
Unfortunately, the original patch didn't work [1]. Then a fix was
applied also without a test [2]. Unfortunately, this wasn't correct
either.
This patch simplifies the function, which makes it a little easier to
reason about. Also a test is added to stop regressions in the future.
[1] Ie3c9c5899c1d46edd21c32868938d3290db321e7
[2] Ifef76eacd8bd837d2181ec75e406aa35f88b8b5b
Closes-Bug: #1929054
Change-Id: I93f53274bc8c7e62ebb8cb7bf81ff816f4bfd98b
Use the Pool classes ability to initialize themselves from op.
Some pool properties should be allowed adjusted after a pool is
created.
The Pool quota property is already handled ad-hoc in the
`charms.ceph` broker handling code, let's bring it over to the
pool objects along with the handling of the compression properties.
Move some missing unit tests here from ceph-mon charm.
Merge after https://github.com/juju/charm-helpers/pull/497
Change-Id: Ibec4e3221387199adbc1a920e130975d7b25343c
The mock third party library was needed for mock support in py2
runtimes. Since we now only support py36 and later, we can use the
standard lib unittest.mock module instead.
Change-Id: Id257b597368aff252ce7120b9dbc02f21a41e629
Adding mon allow r for rbd mirror clients to fix failure to query
mirror pool status.
Change-Id: I63d9f137556b8aef379ee54ea977094722dbf032
Partial-Bug: #1879464
Querying mirror pool status now requires read perms on mgr. Without
this perm a warning is thrown which corrupts the json output.
Closes-Bug: #1851869
Change-Id: I59266da1bf393468f8e0c69dd7212e4cc6de96cf
The new python3-ceph-common deb package (introduced in ceph octopus)
adds a new ceph directory (a parent package in python terms) in
/usr/lib/python3/dist-packages/ceph/. This results in a conflict with
charm-ceph-xyz/lib/ceph/ on imports. For example, with the current
import of ceph.utils in ceph-osd/hooks/ceph_hooks.py, Python finds no
utils.py in /usr/lib/python3/dist-packages/ceph/ and then stops
searching. Therefore, rename ceph to charms_ceph to avoid the
conflict.
Change-Id: I6eaf03e291f65125269b1836dd2636806f9945e1
The charms.ceph utils.py library defines a constant based
on the running OS version. This change updates where the
version is read, allowing it to be nicely mocked.
Change-Id: Ibd65526cad0812f93963bd82619fcd27e055f6f3
Closes-Bug: #1861659
This bug handles OSError when trying to read from CDrom devices with no
disc in the is_pristine_disk function which makes the run-action
list-disks to fail.
Change-Id: I951897a699305604821f2c910ee9ea91582c4e40
Closes-Bug: #1833857
Co-authored-by: Stamatis Katsaounis <skatsaounis@admin.grnet.gr>
Signed-off-by: Alexandros Soumplis <soumplis@admin.grnet.gr>
This method should be call after the quorum has been done. The right
pattern would be:
ceph.wait_quorum()
ceph.create_keyrings()
Change-Id: I6c754cb2d1de57a59416e96aa6ab5cfcaec5df44
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@canonical.com>
For Nautilus the Ceph monitors automatically create the admin and
bootstrap keyrings as part of cluster formation so the charm just
needs to retrieve the admin keyring after the monitors have been
bootstrapped.
As the bootstrap process is two quite distinct steps, this change
refactors the bootstrap_monitor_cluster function into two discrete
functions - one to create and start the ceph monitor, and a second
to create the ceph keyrings on the local filesystem.
As the osd-bootstrap keyring is not actually created on disk from
Nautilus onwards, the 'is_bootstrapped' function now looks for the
client.admin keyring across all Ceph releases - this should be
backwards compatible.
This change also adds the standing data for upgrade paths to
Nautilus, however this is as yet untested.
Change-Id: I2dfab7404b72e314625ea554ee64595c5e26f3c6
To support automation of pool creation on a secondary Ceph cluster
in a RBD Mirror setup the actor will require additional information
about pools on the source cluster.
Change-Id: Id88d268d42da6e453879f850ae76575ac3b0adbf
tox: Remove cleanup of Python object files as this breaks
in the upstream gate.
tox: While at it migrate from os-testr to stestr.
Change-Id: I1bad5311ed034188a78dc67b493c22bff7ce4f7d
If an older version of ceph-osd is deployed and then
upgraded to a version that keeps track of bootstrapped
OSDs, then the list of osd-devices never gets updated
with the pre-existing devices.
This change allows us to add existing, mounted Ceph
OSDs to the osd-devices entry in the local KV storage.
Change-Id: I940b108d914b39b55013a4617c3d17ff7122df60
Closes-Bug: #1814597
When creating a new encrypted disk, ensure that
we zero the first region to ensure that pvcreate
won't stumble on random artifacts
Closes-Bug: #1783427
Change-Id: Iac45e9c00d093befb893c61a3c3aee70ac4b851a
Tracking directory backed OSDs in the kv store allows
us to bootstrap further relations based on bootstrapped
OSD counts.
Change-Id: I1a28a28dbd325dd7ffd74940a07be9908d74689c
Closes-Bug: #1802134
Fully support multi-tier hierarchy within the Ceph OSD tree; this
change refactors the get_osd_tree function to simply filter for
nodes of type 'host' which was in effect what the original code
did, but in a more efficient manner.
This change includes a set of test data taken directly from
the bug report.
Change-Id: I4734a5f6279c02f13bc4eafec15851968881dd5a
Closes-Bug: 1788722
Provide UCA mappings and upgrade path for Ceph Mimic release,
enabling upgrades on Bionic from Luminous to Mimic.
Change-Id: I73a283b86dec6a84c2a5bdfb6be5a7d263aaf070
This patch looks for multiple nodes in the OSD tree with type root and
iterates through all root parent node children to allow for upgrading ceph-osd
cluster/devices when running both a "default" and an "ssd" pool of OSD hosts,
for instance.
Closes-Bug: #1788722
Change-Id: I69d653f9f3ea4ee8469f3d7323ee68435ba22099
Ensure that local configuration files are used for all
settings, instead of mon set configuration which requires
access to the mon cluster which may not be running at the
point in time that configuration is required.
Change-Id: I1fa80c78d6e6e501848d14098fe88d0df4f9e8bc
This is motivated by the fact that:
* the ntp/chrony package is already a dependency of the ceph-mon/osd
package
* NTP will be managed by operators through the NTP charm
It is thus useless to keep that package mentioned here.
Change-Id: Iddb7ffcc7ab7a74700855b950f619208511c2fab
Closes-Bug: #1780690
With BlueStore there are no file system mounts so having a simple check
for a LUKS header presence is good enough to avoid osdizing a used dev
which does not have an LVM super block indicating that a device is in
use.
Partial-Bug: #1781453
Change-Id: Idb98690f0bf11e6932fbe8c2930bb314b0dd3c48
Ensure that the default permissions for clients include the
'osd blacklist' command; This ensures that in the event of
a client crashing (due to power outage or segfault), the
client and re-connect and write to any devices on reboot.
Change-Id: I0b43dece4e1c56fb838b0147bfb75fb9906e6657
Closes-Bug: 1773449
As a part of the mitigation of LP Bug #1698154 remove the reformat
option and initialization of journal devices entirely. Ceph charms
can rely on actions to have operator administratively execute
zap_disk.
There should not be a code path that could accidentally execute
zap_disk without administrative intervention.
Make py27 test noop
Flip pep8 test to py3
Partial-Bug: #1698154
Change-Id: I90a866aa138d18e4242783c42d4c7c587f696d7d
The previous commit to drop creation of partitions on devices
for use with ceph-volume missed some pre-flight checks that
checked the first partition on a disk, rather than the underlying
block device.
Tidyup missed changes and update unit tests.
Change-Id: I0fb4bb782ec76cd6c70425b07303a079b2900cd3
If a block device is due to be used for a LVM PV, don't bother
creating a single partition as its not required and just adds
a layer of complexity.
This also allows partitions or bcache devices to be consumed
as block devices for ceph OSD's.
Change-Id: I6fac0c781e269c229fb4781dd6683a7236d8e960
Peter Sabaini