Commit Graph

161 Commits

Author SHA1 Message Date
Corey Bryant c5a7e8f1c8 Rename ceph to charms_ceph
The new python3-ceph-common deb package (introduced in ceph octopus)
adds a new ceph directory (a parent package in python terms) in
/usr/lib/python3/dist-packages/ceph/. This results in a conflict with
charm-ceph-xyz/lib/ceph/ on imports. For example, with the current
import of ceph.utils in ceph-osd/hooks/ceph_hooks.py, Python finds no
utils.py in /usr/lib/python3/dist-packages/ceph/ and then stops
searching. Therefore, rename ceph to charms_ceph to avoid the
conflict.

Change-Id: I6eaf03e291f65125269b1836dd2636806f9945e1
2020-02-24 15:19:34 +00:00
Chris MacNaughton 0434024b2b Fix charms.ceph mocks to not depend on system version
The charms.ceph utils.py library defines a constant based
on the running OS version. This change updates where the
version is read, allowing it to be nicely mocked.

Change-Id: Ibd65526cad0812f93963bd82619fcd27e055f6f3
Closes-Bug: #1861659
2020-02-03 10:18:08 +01:00
Zuul 8f9e80d26e Merge "Let unmounted_disks not skipping device "/dev/sdm"" 2019-12-19 11:58:48 +00:00
Alexandros Soumplis d393585cfb Ignore devices with OSError
This bug handles OSError when trying to read from CDrom devices with no
disc in the is_pristine_disk function which makes the run-action
list-disks to fail.

Change-Id: I951897a699305604821f2c910ee9ea91582c4e40
Closes-Bug: #1833857
Co-authored-by: Stamatis Katsaounis <skatsaounis@admin.grnet.gr>
Signed-off-by: Alexandros Soumplis <soumplis@admin.grnet.gr>
2019-12-17 14:49:27 +02:00
Alex Kavanagh 1512de6472 Fix Traceback issue when ceph-osd upgrade fails
Bug/1770740 surfaced an issue where get_upgrade_position() returns None
but the calling function expects and exception to the thrown if the
"None" condition exists.  This just fixes the code so that the Traceback
is stopped and the appropriate error/message is logged for the
condition.

Change-Id: I16539b2bc35104eed54033bebb1154cad8a5cf0f
Partial-Bug: #1770740
2019-11-20 10:22:23 +00:00
Zuul 3b6a74bea0 Merge "Add new permission request to broker" 2019-11-07 13:04:22 +00:00
Chris MacNaughton bb708ad4c4 Add new permission request to broker
Change-Id: Ifd341bd80833d4a7fd62e89e3c0e2b7fd64fafba
2019-11-07 20:07:58 +08:00
Andrew McLeod 6469bba74f Install btrfs-progs instead of btrfs-tools for eoan and later
Change-Id: I3d609b92adb8d08cdf04eaec7146a898a419cf01
2019-10-31 00:08:36 +01:00
taodd 740eadcde5 Let unmounted_disks not skipping device "/dev/sdm"
Change-Id: I7bf90531fe5bb01ff47cdeff879b5f72fd227855
Closes-Bug: #1848000
2019-10-23 22:00:13 +08:00
Sahid Orentino Ferdjaoui 7839ca39bf put create_keyrings as a public method
This method should be call after the quorum has been done. The right
pattern would be:

  ceph.wait_quorum()
  ceph.create_keyrings()

Change-Id: I6c754cb2d1de57a59416e96aa6ab5cfcaec5df44
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@canonical.com>
2019-10-22 15:40:44 +00:00
Alex Kavanagh 727a9dd156 Add manager_available() and wait_for_manager() methods
These methods combine to provide a mechanism to wait for the mgr
daemon to be available before performing commands that rely on the
daemon being available; e.g. the adding a mgr module like pg-autotune.

Change-Id: I6255a2e947053a6f8d545aa3f6be1a8330d8bd57
2019-10-18 13:49:13 +01:00
Zuul ee5de74d0a Merge "Nautilus no longer supports creating directory backed OSDs" 2019-09-12 14:06:18 +00:00
Chris MacNaughton cea0280fff Nautilus no longer supports creating directory backed OSDs
Additionally, update our bootstrap check to look for the monitor's
"done" file

Change-Id: Ic168392290867c431fd8cb21a75944768ff92fa9
2019-09-12 12:58:41 +00:00
Zuul 21eb2b91d0 Merge "Update default CRUSH algorithm to straw2" 2019-09-11 11:30:58 +00:00
dongdong tao 6d21a5d0fd Add protection for dir based osd, to avoid it being re-initialized
Change-Id: Id1f3a78bb3108a77f7e2e1d1866657d50ee9dff1
Closes-Bug: #1841021
2019-08-27 17:25:06 +08:00
Chris MacNaughton cf0131d0bf Refactor OSD bootstrap code into ceph-osd charm directly
This has been added to charm-ceph-osd in https://review.opendev.org/#/c/677693/3

Change-Id: I7d1ab2b1b6ca2c8ff4f9b52ab4304261ac647239
2019-08-23 13:47:41 +02:00
Chris MacNaughton (icey) bcfa45b762 Revert "Messanger v2 should be enabled in Nautilus"
This reverts commit 64c8854382.

Change-Id: Ifc066c9548b7915beb14cd132bf8a248b18d4ac1
2019-08-19 20:07:58 +00:00
Chris MacNaughton 64c8854382
Messanger v2 should be enabled in Nautilus
Change-Id: Id1eef4676ac7bdae76c0b8975c3e9ae57e0c9e2d
Closes-Bug: #1840701
2019-08-19 15:18:48 -04:00
Chris MacNaughton 5f095b59cd Update default CRUSH algorithm to straw2
straw2 has been the default CRUSH algorithm since hammer
and Ubuntu Xenial ships with Jewel, meaning that straw2
is the default in all current LTS releases.

Change-Id: I6fa4072eb33d763aca5721ee9fdd30b60d468a44
Closes-Bug: #1836092
2019-07-11 08:24:36 +02:00
James Page f29aba7672 Updates to support Ceph Nautilus release
For Nautilus the Ceph monitors automatically create the admin and
bootstrap keyrings as part of cluster formation so the charm just
needs to retrieve the admin keyring after the monitors have been
bootstrapped.

As the bootstrap process is two quite distinct steps, this change
refactors the bootstrap_monitor_cluster function into two discrete
functions - one to create and start the ceph monitor, and a second
to create the ceph keyrings on the local filesystem.

As the osd-bootstrap keyring is not actually created on disk from
Nautilus onwards, the 'is_bootstrapped' function now looks for the
client.admin keyring across all Ceph releases - this should be
backwards compatible.

This change also adds the standing data for upgrade paths to
Nautilus, however this is as yet untested.

Change-Id: I2dfab7404b72e314625ea554ee64595c5e26f3c6
2019-05-31 10:43:34 +01:00
Zuul bc018ee858 Merge "Validate output of list_logical_volumes" 2019-05-16 14:58:46 +00:00
Zuul e4e0100441 Merge "don't change ownership of empty OSD directories" 2019-05-16 11:07:45 +00:00
Zuul b32c23a48b Merge "move use_bluestore to sharable location" 2019-05-16 08:51:28 +00:00
Chris MacNaughton ed05968141 move use_bluestore to sharable location
Change-Id: Ie19e5318ea35c38e5d02963260b85fec0f233df6
2019-05-16 10:11:25 +02:00
David Ames cd5360736d Validate output of list_logical_volumes
The charm was checking for the zeroth value of the return value of
list_logical_volumes. However, if no logical volumes are found it
returns an empty list.

This change validates that the list has an entry.

Change-Id: I75a6b1dda15dd7c2cece8cfe97b28317b3d5162b
Partial-Bug: #1819382
2019-05-15 10:59:06 -07:00
Trent Lloyd 84d384b4c5 don't change ownership of empty OSD directories
dirs_need_ownership_update detects when OSD directories require an
ownership change from root to ceph. However if an OSD directory exists,
but the OSD is not yet mounted, then the directory is expected to be
owned by root.

This may happen if an OSD is stopped or the OSD is no longer in use,
including on reboot when the OSD may not yet be started before hooks
run.

Change-Id: If9d7e5bcfc0b1f152643a14432e76572ac14a812
Partial-Bug: #1779828
2019-04-10 09:15:30 +08:00
Frode Nordahl 989bd67694
Add ``smartmontools`` to list of packages
Change-Id: I0b5a5e8c2817fe400b68062b0b8568861747e792
Needed-By: I7b83605c827eb4058bc4b46c92eb114c11108c93
Closes-Bug: #1823631
2019-04-08 10:20:18 +02:00
Chris MacNaughton 98183011bb Update bluestore checks to match what we use in ceph-osd hooks
Change-Id: I2ea119f5a1b2a36ccd36df4db094f208a1db100e
Closes-Bug: #1820271
2019-03-15 14:04:02 +01:00
Frode Nordahl 203840bd69
Add setting ``max_objects`` quota and retrieving erasure code profile
Also fix a few missing redirects of STDERR

Change-Id: I243328af5db202f092c53b4d48e0d293378d2ca0
2019-02-27 11:48:29 +01:00
Frode Nordahl 03685ca4a6
Add helper for listing detailed pool information
To support automation of pool creation on a secondary Ceph cluster
in a RBD Mirror setup the actor will require additional information
about pools on the source cluster.

Change-Id: Id88d268d42da6e453879f850ae76575ac3b0adbf
2019-02-19 14:05:32 +01:00
James Page c0d3f4d246 Add support for crush_rule pool key
Later ceph versions allow the crush_rule to be changed on a pool;
add this key with type str to the list of supported keys for
the pool broker.

Change-Id: I2c405f9a4956ff1ccf1edb7372a9bb11e1e591a4
2019-02-18 10:58:19 +00:00
Frode Nordahl 151a242dba
Add helper for creating rbd-mirror key
tox: Remove cleanup of Python object files as this breaks
in the upstream gate.

tox: While at it migrate from os-testr to stestr.

Change-Id: I1bad5311ed034188a78dc67b493c22bff7ce4f7d
2019-02-13 07:08:22 +01:00
Chris MacNaughton 886ef0b224 Ensure we populate osd-devices with existing devices
If an older version of ceph-osd is deployed and then
upgraded to a version that keeps track of bootstrapped
OSDs, then the list of osd-devices never gets updated
with the pre-existing devices.

This change allows us to add existing, mounted Ceph
OSDs to the osd-devices entry in the local KV storage.

Change-Id: I940b108d914b39b55013a4617c3d17ff7122df60
Closes-Bug: #1814597
2019-02-05 11:35:53 +00:00
Felipe Reyes 33b4082685 Add Stein to UCA mappings
Change-Id: I298bd966a253a24197394e0af37f76ba9b8d9120
Related-Bug: 1778823
2019-01-31 19:55:41 -03:00
Liam Young 76709c3a98 Add standalone udevadm_settle function
Add a standalone udevadm_settle that can be called alone. This is
to enable it to be called by the ceph-osd charm. I think the
subprocess calls should be switched to check_call in the longer
term but that may be a disruptive change if udevadm does not
always return 0 so I propose to leave them as they are for the
moment.

Change-Id: Iec5932a4d819ad87e54c2af391abe1befe84f164
Partial-Bug: #1812925
2019-01-23 14:44:01 +00:00
James Page 414c4b9923 Support named radosgw keys
Allow an optional name to be passed when generating radosgw keys;
this supports use of per-unit cephx keys in the ceph-radosgw charm.

Change-Id: I7ac4d23a91fa654b62afb62c389a8f9823ef2e05
Closes-Bug: 1808140
2019-01-08 11:01:24 +00:00
Chris MacNaughton 31ffa13124 Ensure that newly initialized disks are zero'd
When creating a new encrypted disk, ensure that
we zero the first region to ensure that pvcreate
won't stumble on random artifacts

Closes-Bug: #1783427
Change-Id: Iac45e9c00d093befb893c61a3c3aee70ac4b851a
2018-12-11 16:24:44 +01:00
James Page 0c2f77ff60 Drop explicit install of python modules
python-ceph does not exist under Py3; any dependency on underlying
Python modules is expressed as package dependencies from ceph-*
packages.

python-pyudev is used by ceph.utils and needs to be installed
prior to using pyudev (which is typically done using a bash based
install hook - see ceph-osd for examples).

Change-Id: Id653345fd951a2eb044a6f88d03409a4d4b69b83
2018-12-05 14:25:29 +00:00
Chris MacNaughton 646c45dcf4 Add directories to osd-devices as well
Tracking directory backed OSDs in the kv store allows
us to bootstrap further relations based on bootstrapped
OSD counts.

Change-Id: I1a28a28dbd325dd7ffd74940a07be9908d74689c
Closes-Bug: #1802134
2018-11-07 17:13:45 +01:00
James Page 7ff0534471 Support multi-tier hierarchy in Ceph OSD tree
Fully support multi-tier hierarchy within the Ceph OSD tree; this
change refactors the get_osd_tree function to simply filter for
nodes of type 'host' which was in effect what the original code
did, but in a more efficient manner.

This change includes a set of test data taken directly from
the bug report.

Change-Id: I4734a5f6279c02f13bc4eafec15851968881dd5a
Closes-Bug: 1788722
2018-10-11 17:18:36 +01:00
James Page 298d3e6869 Support upgrade: luminous -> mimic
Provide UCA mappings and upgrade path for Ceph Mimic release,
enabling upgrades on Bionic from Luminous to Mimic.

Change-Id: I73a283b86dec6a84c2a5bdfb6be5a7d263aaf070
2018-10-11 09:34:13 +01:00
Gabor Meszaros 412612a286 Provide the real path of device for ceph to prepare
This is a followup patch for the previous commits that are
addressing issues with bcache based configuration.

Providing dname path for ceph-disk is unsafe, the prepare command
uses gdisk which breaks the dname symlinks.

Change-Id: Id3f360c90d40b77720d7bc7c1bf94b79f71b6827
Closes-Bug: 1790186
Signed-off-by: Gabor Meszaros <gabor.meszaros@canonical.com>
2018-09-10 15:37:01 +02:00
Chris MacNaughton a97120757c Ensure that we get all OSDs in get_osd_tree
This patch looks for multiple nodes in the OSD tree with type root and
iterates through all root parent node children to allow for upgrading ceph-osd
cluster/devices when running both a "default" and an "ssd" pool of OSD hosts,
for instance.

Closes-Bug: #1788722
Change-Id: I69d653f9f3ea4ee8469f3d7323ee68435ba22099
2018-08-24 09:34:09 -05:00
dongdong tao 661984bec7 Fix ceph-mon can not print log due to incorrect permission on log file
Change-Id: Iea0bc531446b79dfbc9ecd9d8b432b1efef2aaaf
Signed-off-by: dongdong tao <dongdong.tao@canonical.com>
Closes-Bug: #1783526
2018-07-30 09:42:48 +08:00
Zuul 8e8e5f706a Merge "Resolve symlinks to underlying block devices" 2018-07-27 12:47:52 +00:00
James Page 0e1be87d68 Resolve symlinks to underlying block devices
When checking for holders on a block device, ensure that any
by-uuid or by-dname provided paths are resolved to the underlying
block device when searching for holders.

Change-Id: If0e1fbc62bfe7d0f9e21db9bfdeee761060de846
Closes-Bug: 1782439
2018-07-27 13:13:45 +01:00
James Page 427e6d38f8 mimic: pass --no-mon-config for osd config
Ensure that local configuration files are used for all
settings, instead of mon set configuration which requires
access to the mon cluster which may not be running at the
point in time that configuration is required.

Change-Id: I1fa80c78d6e6e501848d14098fe88d0df4f9e8bc
2018-07-24 06:45:33 +01:00
Nicolas Pochet afdd9a974d Remove reference to ntp package
This is motivated by the fact that:
* the ntp/chrony package is already a dependency of the ceph-mon/osd
package
* NTP will be managed by operators through the NTP charm
It is thus useless to keep that package mentioned here.

Change-Id: Iddb7ffcc7ab7a74700855b950f619208511c2fab
Closes-Bug: #1780690
2018-07-20 16:05:52 +02:00
Dmitrii Shcherbakov 6ee0f95a2b skip mapped LUKS devices in osdize_dev
With BlueStore there are no file system mounts so having a simple check
for a LUKS header presence is good enough to avoid osdizing a used dev
which does not have an LVM super block indicating that a device is in
use.

Partial-Bug: #1781453

Change-Id: Idb98690f0bf11e6932fbe8c2930bb314b0dd3c48
2018-07-17 14:32:41 +03:00
James Page 4d8f31d0ea Add 'osd blacklist' to default mon perms
Ensure that the default permissions for clients include the
'osd blacklist' command;  This ensures that in the event of
a client crashing (due to power outage or segfault), the
client and re-connect and write to any devices on reboot.

Change-Id: I0b43dece4e1c56fb838b0147bfb75fb9906e6657
Closes-Bug: 1773449
2018-06-11 09:11:08 +01:00