The new python3-ceph-common deb package (introduced in ceph octopus)
adds a new ceph directory (a parent package in python terms) in
/usr/lib/python3/dist-packages/ceph/. This results in a conflict with
charm-ceph-osd/lib/ceph/. For example, with the current import of
ceph.utils in hooks/ceph_hooks.py, Python finds no utils.py in
/usr/lib/python3/dist-packages/ceph/ and then stops searching.
Therefore, rename lib/ceph to lib/charms_ceph to avoid the conflict.
Depends-On: https://review.opendev.org/#/c/709226
Change-Id: I13ae7c048d8f1eef2ea64b13ae14b51dbfaaf3cd
This bug handles OSError when trying to read from CDrom devices with no
disc in the is_pristine_disk function which makes the run-action
list-disks to fail.
Change-Id: I5e86895d1adff3f95c7feb5ed0f78b998c28ed1f
Depends-On: I951897a699305604821f2c910ee9ea91582c4e40
Closes-Bug: #1833857
Signed-off-by: Alexandros Soumplis <soumplis@admin.grnet.gr>
This was originally fixed in Ib7d1fdc8f91bc992ccf618ef6f57e99bb90c2dbc
but unfortunately wasn't also added to the charms.ceph library. Thus,
this is a re-application of that fix; the charms to ceph fix is in [1].
Bug/1770740 surfaced an issue where get_upgrade_position() returns None
but the calling function expects and exception to the thrown if the
"None" condition exists. This just fixes the code so that the Traceback
is stopped and the appropriate error/message is logged for the
condition.
[1] https://review.opendev.org/#/c/695163/
I16539b2bc35104eed54033bebb1154cad8a5cf0f
Change-Id: Ieee8d13f25027ad540a23a6428c2226b6c20999a
Partial-Bug: #1770740
use btrfs-progs instead of btrfs-tools for eoan and later
update bionic-train zaza test bundle to include placement
Closes-Bug: #1850181
Change-Id: I6c414cb7db45f5a1a2cf9b65f6fd93c40f631a46
7839ca3 put create_keyrings as a public method
Change-Id: Ia1ab1860df33960519d0fac70b8315bc3e8e25a3
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@canonical.com>
This change includes a charms.ceph sync.
The _upgrade_keyring function was removed from charms.ceph
so this charm needs to use the already-existing, charm-
internal version.
Change-Id: Ia7cf352a2456dc85aca6f61d2e88327cd0c15f7e
Closes-Bug: #1845975
The charm was checking for the zeroth value of the return value of
list_logical_volumes. However, if no logical volumes are found it
returns an empty list.
This change validates that the list has an entry.
Depends-On: I75a6b1dda15dd7c2cece8cfe97b28317b3d5162b
Change-Id: I2d371dae94dca328cf4782a79e85c1c6fd77f547
Closes-Bug: #1819382
Run udevadm settle before processing osd-devices as there may by
udev events that have been queued for processing on server startup.
Includes charms.ceph sync
Corresponding charms.ceph change is:
Iec5932a4d819ad87e54c2af391abe1befe84f164
Change-Id: Ia4210d6a5c2c8a0b0e2038b25f24d2d82600e10a
Closes-Bug: #1812925
This change implements a new option called 'bdev-enable-discard' to control
behaviour of issuing discards to SSDs on ceph bluestore. The new code tries
to autodetect cases where it should be enabled by default but will allow
forcing if desired.
Change-Id: I7b83605c827eb4058bc4b46c92eb114c11108c93
Closes-Bug: #1788433
If an older version of ceph-osd is deployed and then
upgraded to a version that keeps track of bootstrapped
OSDs, then the list of osd-devices never gets updated
with the pre-existing devices.
This change allows us to add existing, mounted Ceph OSDs
to the osd-devices entry in the local KV storage.
Change-Id: I17fab658511275f1dde15683ef296d4c72e7980e
Closes-Bug: #1814597
Depends-On: I940b108d914b39b55013a4617c3d17ff7122df60
ceph-osd charm only upgrades when the ceph version changes, for the
case of upgrading from Pike to Queens the charm is skipping any
upgrades, because the Cloud Archive has Luminous for those 2 releases.
This patch checks if the requested ceph version is luminous and if the
'source' changed from pike to queens to then upgrade
/etc/apt/sources.list.d/cloud-archive.list via add_source()
Change-Id: I27c2d7648a1add6528924724a03682060d2d6007
Closes-Bug: 1778823
When adding directories to osd-devices (e.g. juju config ceph-osd
osd-devices="/srv/ceph1 /srv/ceph2"), the charm was recursively changing
owner and group for all files below /var/lib/ceph, for each added
directory. This was taking time and was not useful.
The fix is to remove unnecessary recursive chown on /var/lib/ceph, since
ceph already manages the ownership of the files itself.
Change-Id: I39a00591bc86ec49c4ced53eadce75ddb21e2431
Closes-Bug: #1795383
Resync with latest updates to charms.ceph to avoid explicit
installation of python-ceph which is not required and breaks
under the laters Ceph packaging which no longer ships Python 2
support.
Change-Id: I4ce2b91dd476f90c30d1379dac5b00b8aaa9c73a
Tracking directory backed OSDs in the kv store allows
us to bootstrap further relations based on bootstrapped
OSD counts.
Change-Id: I1abd767d15c204845d9909d9c7ee9414dbe87a5c
Closes-Bug: #1802134
Depends-On: https://review.openstack.org/#/c/616230/
Resync ceph helpers, picking up fixes for:
- Upgrades from Luminous to Mimic.
- Correct build of OSD list in more complex CRUSH
configurations, resolving upgrade issues.
Closes-Bug: 1788722
Change-Id: I7d8fca74ec6eadae21a6e669e8b2522d9e4c9367
This patch looks for multiple nodes in the OSD tree with type root and
iterates through all root parent node children to allow for upgrading ceph-osd
cluster/devices when running both a default and an ssd pool of OSD hosts,
for instance.
Change-Id: Iea9812ee7ac67f9b45a6b38c43c130353e68ad8f
Closes-Bug: #1788722
Depends-On: I69d653f9f3ea4ee8469f3d7323ee68435ba22099
Ensure that device paths provided by end users are used for OSD's,
rather than the link target device as this may change between
reboots. The specific use case is bcache, where:
/dev/bcacheX:
changes between reboots
/dev/disk/by-dname/bcacheX:
udev managed and consistent
This change also ensures that any unit data is updated to switch
back to using the provided block device path, rather than the
actual target which may have been used in prior charm revisions.
Change-Id: If5e88d93b9323052ea762d3a4b66f2442d4a19be
Depends-On: If0e1fbc62bfe7d0f9e21db9bfdeee761060de846
Closes-Bug: 1782439
Ensures mon cluster is not queried early in unit lifecycle
when the cluster is not bootstrapped or accessible by the
ceph-osd unit.
Block device sizes are set in local config so querying the
monitor is OK to skip.
Change-Id: Iea37455b0946560056ee665e819ee0a2a4a7832c
This is motivated by the fact that:
* the ntp/chrony package is already a dependency of the ceph-mon/osd
package
* NTP will be managed by operators through the NTP charm
It is thus useless to keep that package mentioned here.
Change-Id: I5834ff22d4306707529e958cd26f14bbb752c796
Closes-Bug: #1780690
Depends-on: Iddb7ffcc7ab7a74700855b950f619208511c2fab
Similar to how osdize in charms.ceph checks for already processed
devices we need to avoid checking if they are pristine or not.
Additionally, mapped LUKS devices need to be filtered from being zapped
as they may hold valuable data. They are only used as underlying devices
for device mapper and dmcrypt to provide a decrypted block device
abstration so if they really need to be zapped a mapping needs to be
removed first.
This change also pulls charms.ceph modifications.
Change-Id: I96b3d40b3f9e56681be142377e454b15f9e22be3
Co-Authored-By: Dmitrii Shcherbakov <dmitrii.shcherbakov@canonical.com>
Co-Authored-By: Chris Procter <chris.procter@canonical.com>
Closes-Bug: 1781453
Add `non-pristine` key to `list-disks` action.
No longer attempt to do initializtion of `osd-journal` devices.
Make py27 test noop
Flip pep8 test to py3
Partial-Bug: #1698154
Change-Id: I0ca574fa7f0683b4e8a693b9f62fbf6b39689789
Depends-On: I90a866aa138d18e4242783c42d4c7c587f696d7d
Do not reformat devices. A subsequent change will be necessary
to account for conditions where a reformat is still desired,
such as a set of blocking states and user-driven actions.
Partial-bug: #1698154
Depends-On: I90a866aa138d18e4242783c42d4c7c587f696d7d
Change-Id: I3a41ab38e7a1679cf4f5380a7cc56556da3aaf2b
At present this action does not work. While looking to repair the
functionality I found a number of issues with the current
implementation.
For now I suggest we remove this functionality, and at some point we may
consider replacing it with a `remove-disk` action.
Sync in relevant changes from charms.ceph
Depends-On: Id61b87927c43d807aacc93cf05ec8f88d91b7a39
Change-Id: Ic71d304ff65a05ab7249f4dd07adc45429a323e9
Bug/1770740 surfaced an issue where get_upgrade_position() returns None
but the calling function expects and exception to the thrown if the
"None" condition exists. This just fixes the code so that the Traceback
is stopped and the appropriate error/message is logged for the
condition.
Change-Id: Ib7d1fdc8f91bc992ccf618ef6f57e99bb90c2dbc
Partial-Bug: #1770740
vaultlocker provides support for storage of encryption keys
for LUKS based dm-crypt device in Hashicorp Vault.
Add support for this key management approach for Ceph
Luminous or later. Applications will block until vault
has been initialized and unsealed at which point OSD devices
will be prepared and booted into the Ceph cluster.
The dm-crypt layer is placed between the block device
parition and the top level LVM PV used to create VG's
and LV's to support OSD operation.
Vaultlocker enables a systemd unit for each encrypted
block device to perform unlocking during reboots of the
unit; ceph-volume will then detect the new VG/LV's and
boot the ceph-osd processes as required.
Note that vault/vaultlocker usage is only supported with
ceph-volume, which was introduced into the Ubuntu packages
as of the 12.2.4 point release for Luminous. If vault is
configured as the key manager in deployments using older
versions, a hook error will be thrown with a blocked
status message to this effect.
Change-Id: I713492d1fd8d371439e96f9eae824b4fe7260e47
Depends-On: If73e7bd518a7bc60c2db08e2aa3a93dcfe79c0dd
Depends-On: https://github.com/juju/charm-helpers/pull/159
Resync ceph helpers to support use of partitions and bcache devices
for OSD filestore, blockstore, journal, db and wal devices.
Change-Id: Iac5eefc4c3f53b1ec2f5cfc6e98a9a2168c1c0b6
Closes-Bug: 1769678
Resync charms.ceph to pickup improvements in recording
of block devices that have been processed as OSD devices
to support better idempotency of block device processing
codepaths.
This fixes a particularly nasty issue with osd-reformat
is set to True where the charm can wipe and re-prepare
an OSD device prior to the systemd unit actually booting
and mounting the OSD's associated filesystem.
This change also makes the osd-reformat option a boolean
option which is more accessible to users of the charm
via the CLI and the Juju GUI.
Change-Id: I578203aeebf6da2efc21a10d2e157324186e2a66
Depends-On: I2c6e9d5670c8d1d70584ae19b34eaf16be5dea19
Switch to using ceph-volume + LVM for managing block devices
for Luminous and later; this is the upstream preferred approach
to managing OSD devices, allowing for more flexibility in terms
of use of crypto and logical volumes.
Change-Id: I30c4d29e6f568ac2e30a45b1a7bc0e68685c3707
Depends-On: I1675b67d364ae6042129a8a717d4bdffff5bde92
Adds bluestore-specific options related to the metadata-only journal.
The options allow a user to control:
1. path to a bluestore wal (block special file or regular file)
2. path to a bluestore db (block special file or regular file)
3. size of both
Their configuration works similarly to the FileStore journal. If paths
are not specified both WAL and DB will be collocated on the same block
device as data.
Other options can be configured via an existing config-flags option if needed.
http://docs.ceph.com/docs/master/rados/configuration/bluestore-config-ref/
Closes-Bug: #1710474
Change-Id: Ia85092230d4dcb0435354deb276012f923547393
Depends-On: I483ee9dae4ce69c71ae06359d0fb96aaa1c56cbc
Depends-On: Idbbb69acec92b2f2efca80691ca73a2030bcf633
* Synced version of charm-helpers
* Synced version of charms.ceph to bring in Py3 compatible library
methods.
Change-Id: I5ac45740f48a71d9cb0c5943472fc8590a723514
- ensure filestore is used by default
- allow upgrades to luminous
Change-Id: I00aa4fcba53d9bd28592b9ba1b89f74d869148c2
Depends-on: I8205f7c7c63ec30900c4afdc76df174d3d9a8466
Corey Bryant