Commit Graph

81 Commits

Author SHA1 Message Date
Peter Sabaini 1bac66ee50 Remove FileStore support
Remove support for creating FileStore OSDs. Also prevent upgrade
attempts to Reef if a FileStore OSD is detected

Change-Id: I9609bc0222365cb1f4059312b466a12ef4e0397f
2023-10-06 09:03:51 +02:00
Luciano Lo Giudice 7821a38e25 Sync libraries again (Reef mapping was missing)
Change-Id: I41b5abba9c29a99c16c67ee0d4a8c10e760c76fa
2023-09-13 14:11:36 -03:00
Luciano Lo Giudice e97890043c Sync charm libraries
Change-Id: Ie6ae6fa1369db537fa606fa04df538f9a5587a0c
2023-09-12 19:23:43 -03:00
Corey Bryant 986981c6f4 Add 2023.2 Bobcat support
* sync charm-helpers to classic charms
* change openstack-origin/source default to quincy
* add mantic to metadata series
* align testing with bobcat
* add new bobcat bundles
* add bobcat bundles to tests.yaml
* add bobcat tests to osci.yaml
* update build-on and run-on bases
* drop kinetic

Change-Id: I7449eba63107b43525359fb92ae1a0ad9e648bab
2023-07-25 17:26:56 -04:00
Samuel Walladge 97be046f9b Save the crash module auth key
Read the key set on the mon relation,
and use ceph-authtool to save it to a keyring,
for use by the crash module for crash reporting.

When this auth key is set, the crash module (enabled by default)
will update ceph-mon with a report.
It also results in a neat summary of recent crashes
that can be viewed by `ceph health detail`.
For example:

```
$ juju ssh ceph-mon/leader -- sudo ceph health detail

HEALTH_WARN 1 daemons have recently crashed
[WRN] RECENT_CRASH: 1 daemons have recently crashed
    osd.1 crashed on host node-3 at 2023-01-04T05:25:18.218628Z
```

ref. https://docs.ceph.com/en/latest/mgr/crash/

See also https://review.opendev.org/c/openstack/charm-ceph-mon/+/869138
for where the client_crash_key relation data set is implemented.

Depends-On: https://review.opendev.org/c/openstack/charm-ceph-mon/+/869138

Closes-Bug: #2000630
Change-Id: I77c84c368e6665e4988ebe9a735f000f99d0b78e
2023-01-20 15:13:13 +10:30
Billy Olsen d4f3dd2ee6 Updates for jammy quincy and finalise charmcraft builds
- Update charmcraft.yaml for arch builds
- Remove impish-xena tests
- Update osci.yaml to include py3.10 default job
- Resync charmhelpers and charms.ceph

Change-Id: I41abace773d8926eaa00076dd83f52849c96150d
2022-04-07 16:47:55 +02:00
Luciano Lo Giudice 55720fa087 Implement the 'remove-disk' action
This new action allows users to either purge an OSD, or remove it,
opening up the possibility of recycling the previous OSD id. In
addition, this action will clean up any bcache devices that were
created in previous steps.

Change-Id: If3566031ba3f02dac0bc86938dcf9e85a66a66f0
Depends-On: Ib959e81833eb2094d02c7bdd507b1c8b7fbcd3db
func-test-pr: https://github.com/openstack-charmers/zaza-openstack-tests/pull/683
2022-03-31 18:50:22 +00:00
Luciano Lo Giudice 65207967cf Enhance the 'add-disk' action for disk replacement
As part of the task to improve disk replacement, the 'add-disk' action
needs some changes. This includes:
- Creating 'bcache' devices to accelerate disk access.
- Creating caching partitions of a specified size.
- Recycling previously deactivated OSD ids.

Change-Id: Id5027f30d51c23d2be4c34f82867d65a50b35137
Depends-On: I43d0a0bc11664c37532c0117711affc93c9d1ad1
func-test-pr: https://github.com/openstack-charmers/zaza-openstack-tests/pull/675
2022-02-18 12:43:46 -03:00
James Troup 73fe60b3df Spelling fixes from codespell with additional consistency cleanups.
Includes a resync of charms.ceph.

Change-Id: I281fe17acaea826b79bfa902a8d9e7a8b59482a9
2021-12-20 13:21:33 +00:00
Corey Bryant f2d704a14d Add yoga bundles and release-tool syncs
* charm-helpers sync for classic charms
* sync from release-tools
* switch to release-specific zosci functional tests
* run focal-ussuri as smoke tests
* remove trusty, xenial, and groovy metadata/tests
* drop py35 and add py39
* charms.ceph sync

Change-Id: I4d517c40a4450ef1395dd0240513eeebc069384c
2021-11-17 13:46:25 -05:00
Luciano Lo Giudice 93e9885aa7 Only consider mounted OSD directories
When gathering the list of local OSD ids, the charm would consider
the entries under '/var/lib/ceph/osd/ceph-XXX' where 'XXX" was the
OSD id. However, if an entry under that directory isn't mounted,
then the OSD that would represent that entry should be discarded,
as it's no longer active. This patchset thus filters those entries
by looking for them in the mount points.

Closes-Bug: #1934938
Change-Id: I69c6356e450cc0c96de4afe571b438d4a2ea5177
2021-09-02 18:55:18 -03:00
Alex Kavanagh cf2948f8f0 21.04 libraries freeze for charms on master branch
* charm-helpers sync for classic charms
* build.lock file for reactive charms
* ensure tox.ini is from release-tools
* ensure requirements.txt files are from release-tools
* On reactive charms:
  - ensure stable/21.04 branch for charms.openstack
  - ensure stable/21.04 branch for charm-helpers

Change-Id: I773d5f9f699af1a2ea7be543c4e58e0f7bc4433a
2021-04-09 08:42:26 -07:00
Aurelien Lourot 27d035f89b Add Groovy to the test gate
Change-Id: I273eeb41f9750ddb32ae1ca5933278ac8af5c4b7
2020-11-05 12:44:12 +01:00
Alex Kavanagh 1adefd359b Sync libraries & common files prior to freeze
* charm-helpers sync for classic charms
* charms.ceph sync for ceph charms
* rebuild for reactive charms
* sync tox.ini files as needed
* sync requirements.txt files to sync to standard

Required additional fix:

* Also sync section-ceph-bluestore-compression template

Change-Id: I621c3a0e4e3594808b4c6ec298ed79aada44bae0
2020-09-28 18:48:15 +01:00
Frode Nordahl b49468fc10
Add BlueStore Compression support
Sync in updates from charm-helpers and charms.ceph.

Depends-On: I153c22efb952fc38c5e3d36eed5d85c953e695f7
Depends-On: Ibec4e3221387199adbc1a920e130975d7b25343c
Change-Id: I028440002cdd36be13aaee4a0f50c6a0bca7abda
2020-08-26 16:30:24 +02:00
Alex Kavanagh 6b0a11b404 Add a progress watchdog for OSD upgrades
This patch (in charms.ceph [1], copied here) add the concept of a
watchdog to the upgrade_monitor so that the charm can achieve two
objectives of 1. Waiting for much longer, but 2. detecting whether the
previous node has died / gone away.  This is needed for 'large' OSDs
where the time to upgrade a node may exceed the current limit of 10
minutes, but also not to wait for 30 minutes on a dead previous node.
The watchdog implements two timeouts and an addition 'alive' key from
the previous node to indicate that it is still running.  Otherwise,
functionality is identical.

[1] See depends on below
Depends-On: Ia450e936c2096f092af3be5a369b7abaf5023b16
Closes-Bug: #1762852

Change-Id: I6204a5ade684f0564c4be2d30df467c75baa6dba
2020-07-15 15:49:28 +01:00
Corey Bryant 189e0c2b98 Sync charm-helpers for Victoria/Groovy updates
This sync picks up the release and version details for Victoria/Groovy.

Change-Id: I459b37cb22ac00d8d4fd9983d9a995b475a0207d
2020-07-13 18:59:17 +00:00
Alex Kavanagh 8b0b24d6c6 Updates for 20.08 cycle start for groovy and libs
- Adds groovy to the series in the metadata
- Classic charms: sync charm-helpers.
- Classic ceph based charms:  also sync charms.ceph
- Reactive charms: trigger a rebuild

Change-Id: I56a59d0c4e72a35b7c4ac5d989e0d005fae20946
2020-06-02 14:28:22 +01:00
Aurelien Lourot c64f4eb36a Sync helpers for 20.05
Change-Id: Ibebef746614a9de92c4e3c9ca109f313562784fc
2020-05-18 14:39:04 +02:00
James Page b1aab5d0e1 Trigger udev rescan if pv_dev disappears
Workaround for kernel by in Ubuntu 20.04 LTS.

When using by-dname device paths with MAAS and bcache, the pvcreate
operation results in the by-dname entry for the block device being
deleted.  The subsequent vgcreate then fails as the path cannot
be found.

Trigger a rescan of block devices if the pv_dev path does not
exists after the pvcreate operation.

Change-Id: If7e11f6bd1effd2d5fc2dc5abbaba6865104006f
Depends-On: Ifb16c47ae5ff316cbcfc3798de3446a3774fa012
Related-Bug: 1878752
2020-05-18 08:41:17 +01:00
Liam Young 3e795f6a62 Apply OSD settings from mons.
Apply OSD settings requested by the mons via the juju relation.
Add the OSD settings to config too. Before applying the settings
config-flags is checked to ensure there is no overlap.

Change-Id: Id69222217a1c99d0269831913abdf488791cb572
2020-05-04 08:23:00 +00:00
Corey Bryant cb0f757f18 Maintain OSD state on upgrade
Sync charms.ceph

Ensure each OSD reaches its pre-restart state before proceeding
after restart. This prevents the charm from finalizing the upgrade
prior to OSDs recovering after upgrade. For example, if the state
is 'active' prior to restart, then it must reach 'active' after
restart, at which point the upgrade will be allowed to complete.

Change-Id: I1067a8cdd1e2b706db07f194eca6fb2efeccb817
Depends-On: https://review.opendev.org/#/c/713743/
Closes-Bug: #1821028
2020-03-19 02:07:45 +00:00
Ryan Beisner 61ffd77184
Sync charm-helpers for py38, distro, and other updates
Change-Id: I9e8848cc6dff2addd4db087740ba5bbc967577cf
2020-03-05 13:30:42 +01:00
Corey Bryant fcfa499f11 Rename lib/ceph to lib/charms_ceph
The new python3-ceph-common deb package (introduced in ceph octopus)
adds a new ceph directory (a parent package in python terms) in
/usr/lib/python3/dist-packages/ceph/. This results in a conflict with
charm-ceph-osd/lib/ceph/. For example, with the current import of
ceph.utils in hooks/ceph_hooks.py, Python finds no utils.py in
/usr/lib/python3/dist-packages/ceph/ and then stops searching.
Therefore, rename lib/ceph to lib/charms_ceph to avoid the conflict.

Depends-On: https://review.opendev.org/#/c/709226
Change-Id: I13ae7c048d8f1eef2ea64b13ae14b51dbfaaf3cd
2020-02-24 15:21:33 +00:00
Liam Young 620ff1c20a Charmhelper sync for 20.02
Change-Id: I696c0f3303f038a66cf2298d10f081b580e0a9fe
2020-02-04 16:38:23 +00:00
Corey Bryant d4a6142a03 Sync charm-helpers for Ussuri/Focal release and version details
Change-Id: I5ac45332a126143ce0299c2c534523bd890f5f57
2020-01-17 14:21:26 -05:00
Alexandros Soumplis df8f477cd7 Ignore devices with OSError
This bug handles OSError when trying to read from CDrom devices with no
disc in the is_pristine_disk function which makes the run-action
list-disks to fail.

Change-Id: I5e86895d1adff3f95c7feb5ed0f78b998c28ed1f
Depends-On: I951897a699305604821f2c910ee9ea91582c4e40
Closes-Bug: #1833857
Signed-off-by: Alexandros Soumplis <soumplis@admin.grnet.gr>
2019-12-17 14:43:18 +02:00
Alex Kavanagh b15a23ef6a Re-fix Traceback issue when ceph-osd upgrade fails
This was originally fixed in Ib7d1fdc8f91bc992ccf618ef6f57e99bb90c2dbc
but unfortunately wasn't also added to the charms.ceph library.  Thus,
this is a re-application of that fix; the charms to ceph fix is in [1].

Bug/1770740 surfaced an issue where get_upgrade_position() returns None
but the calling function expects and exception to the thrown if the
"None" condition exists.  This just fixes the code so that the Traceback
is stopped and the appropriate error/message is logged for the
condition.

[1] https://review.opendev.org/#/c/695163/
    I16539b2bc35104eed54033bebb1154cad8a5cf0f

Change-Id: Ieee8d13f25027ad540a23a6428c2226b6c20999a
Partial-Bug: #1770740
2019-11-20 10:29:04 +00:00
Andrew McLeod 67254e334f Sync charms.ceph
use btrfs-progs instead of btrfs-tools for eoan and later
update bionic-train zaza test bundle to include placement

Closes-Bug: #1850181
Change-Id: I6c414cb7db45f5a1a2cf9b65f6fd93c40f631a46
2019-10-31 19:37:18 +01:00
Sahid Orentino Ferdjaoui 019e5ba45a sync with last charms.ceph
7839ca3 put create_keyrings as a public method

Change-Id: Ia1ab1860df33960519d0fac70b8315bc3e8e25a3
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@canonical.com>
2019-10-24 10:00:25 +00:00
Chris MacNaughton 23103d2958 Use charm-internal _upgrade_keyring
This change includes a charms.ceph sync.

The _upgrade_keyring function was removed from charms.ceph
so this charm needs to use the already-existing, charm-
internal version.

Change-Id: Ia7cf352a2456dc85aca6f61d2e88327cd0c15f7e
Closes-Bug: #1845975
2019-10-02 15:18:21 +00:00
taodd 7772560510 Sync charms.ceph to get fix
Change-Id: Ib3d4b79690eb5931b4f0680b937590b317a91427
Closes-Bug: #1841021
2019-09-24 20:07:42 +08:00
David Ames e6086328f4 Validate output of list_logical_volumes
The charm was checking for the zeroth value of the return value of
list_logical_volumes. However, if no logical volumes are found it
returns an empty list.

This change validates that the list has an entry.

Depends-On: I75a6b1dda15dd7c2cece8cfe97b28317b3d5162b
Change-Id: I2d371dae94dca328cf4782a79e85c1c6fd77f547
Closes-Bug: #1819382
2019-05-15 11:00:34 -07:00
Zuul 4a3a981e9d Merge "udevadm settle before processing osd-devices" 2019-03-19 07:55:02 +00:00
Liam Young 8655dbd9e3 udevadm settle before processing osd-devices
Run udevadm settle before processing osd-devices as there may by
udev events that have been queued for processing on server startup.

Includes charms.ceph sync

Corresponding charms.ceph change is:
Iec5932a4d819ad87e54c2af391abe1befe84f164

Change-Id: Ia4210d6a5c2c8a0b0e2038b25f24d2d82600e10a
Closes-Bug: #1812925
2019-03-05 07:59:46 +00:00
Andre Ruiz 56495eecba Implement new option to enable discard on SSDs
This change implements a new option called 'bdev-enable-discard' to control
behaviour of issuing discards to SSDs on ceph bluestore. The new code tries
to autodetect cases where it should be enabled by default but will allow
forcing if desired.

Change-Id: I7b83605c827eb4058bc4b46c92eb114c11108c93
Closes-Bug: #1788433
2019-03-01 15:26:56 +01:00
Zuul dc3e5c80f7 Merge "Update cloud-archive.list when upgrading from Pike to Queens" 2019-02-06 16:20:04 +00:00
Chris MacNaughton 9bc5abab55 Ensure we populate osd-devices with existing devices
If an older version of ceph-osd is deployed and then
upgraded to a version that keeps track of bootstrapped
OSDs, then the list of osd-devices never gets updated
with the pre-existing devices.

This change allows us to add existing, mounted Ceph OSDs
to the osd-devices entry in the local KV storage.

Change-Id: I17fab658511275f1dde15683ef296d4c72e7980e
Closes-Bug: #1814597
Depends-On: I940b108d914b39b55013a4617c3d17ff7122df60
2019-02-05 12:36:28 +01:00
Felipe Reyes 336581efeb Update cloud-archive.list when upgrading from Pike to Queens
ceph-osd charm only upgrades when the ceph version changes, for the
case of upgrading from Pike to Queens the charm is skipping any
upgrades, because the Cloud Archive has Luminous for those 2 releases.

This patch checks if the requested ceph version is luminous and if the
'source' changed from pike to queens to then upgrade
/etc/apt/sources.list.d/cloud-archive.list via add_source()

Change-Id: I27c2d7648a1add6528924724a03682060d2d6007
Closes-Bug: 1778823
2019-02-01 11:47:05 -03:00
Zuul eec8464e93 Merge "Resync charms.ceph and charmhelpers" 2018-12-11 19:07:23 +00:00
Przemysław Hausman 87e43ce27a Remove chown'ing on adding dirs to osd-devices
When adding directories to osd-devices (e.g. juju config ceph-osd
osd-devices="/srv/ceph1 /srv/ceph2"), the charm was recursively changing
owner and group for all files below /var/lib/ceph, for each added
directory. This was taking time and was not useful.

The fix is to remove unnecessary recursive chown on /var/lib/ceph, since
ceph already manages the ownership of the files itself.

Change-Id: I39a00591bc86ec49c4ced53eadce75ddb21e2431
Closes-Bug: #1795383
2018-12-07 09:05:06 +01:00
James Page cfc7b05332 Resync charms.ceph and charmhelpers
Resync with latest updates to charms.ceph to avoid explicit
installation of python-ceph which is not required and breaks
under the laters Ceph packaging which no longer ships Python 2
support.

Change-Id: I4ce2b91dd476f90c30d1379dac5b00b8aaa9c73a
2018-12-05 15:24:45 +00:00
Chris MacNaughton c158d7432b Add directories to osd-devices as well
Tracking directory backed OSDs in the kv store allows
us to bootstrap further relations based on bootstrapped
OSD counts.

Change-Id: I1abd767d15c204845d9909d9c7ee9414dbe87a5c
Closes-Bug: #1802134
Depends-On: https://review.openstack.org/#/c/616230/
2018-11-07 17:17:44 +01:00
James Page ce97b7a479 Resync ceph helpers
Resync ceph helpers, picking up fixes for:

 - Upgrades from Luminous to Mimic.
 - Correct build of OSD list in more complex CRUSH
   configurations, resolving upgrade issues.

Closes-Bug: 1788722

Change-Id: I7d8fca74ec6eadae21a6e669e8b2522d9e4c9367
2018-10-12 09:55:03 +01:00
Chris MacNaughton 0cb16d8c66 Sync in charms.ceph
This patch looks for multiple nodes in the OSD tree with type root and
iterates through all root parent node children to allow for upgrading ceph-osd
cluster/devices when running both a default and an ssd pool of OSD hosts,
for instance.

Change-Id: Iea9812ee7ac67f9b45a6b38c43c130353e68ad8f
Closes-Bug: #1788722
Depends-On: I69d653f9f3ea4ee8469f3d7323ee68435ba22099
2018-08-24 16:38:06 +02:00
James Page 488f7dffef Use provided device paths
Ensure that device paths provided by end users are used for OSD's,
rather than the link target device as this may change between
reboots. The specific use case is bcache, where:

   /dev/bcacheX:
        changes between reboots
   /dev/disk/by-dname/bcacheX:
        udev managed and consistent

This change also ensures that any unit data is updated to switch
back to using the provided block device path, rather than the
actual target which may have been used in prior charm revisions.

Change-Id: If5e88d93b9323052ea762d3a4b66f2442d4a19be
Depends-On: If0e1fbc62bfe7d0f9e21db9bfdeee761060de846
Closes-Bug: 1782439
2018-07-27 13:51:55 +01:00
James Page a26b5e4551 mimic: Sync charms.ceph for misc fixes
Ensures mon cluster is not queried early in unit lifecycle
when the cluster is not bootstrapped or accessible by the
ceph-osd unit.

Block device sizes are set in local config so querying the
monitor is OK to skip.

Change-Id: Iea37455b0946560056ee665e819ee0a2a4a7832c
2018-07-24 06:54:48 +01:00
Nicolas Pochet bb3a7f198c Remove reference to ntp package
This is motivated by the fact that:
* the ntp/chrony package is already a dependency of the ceph-mon/osd
package
* NTP will be managed by operators through the NTP charm
It is thus useless to keep that package mentioned here.

Change-Id: I5834ff22d4306707529e958cd26f14bbb752c796
Closes-Bug: #1780690
Depends-on: Iddb7ffcc7ab7a74700855b950f619208511c2fab
2018-07-20 16:07:12 +02:00
Dmitrii Shcherbakov e340cc851c ignore devices that have already been processed
Similar to how osdize in charms.ceph checks for already processed
devices we need to avoid checking if they are pristine or not.

Additionally, mapped LUKS devices need to be filtered from being zapped
as they may hold valuable data. They are only used as underlying devices
for device mapper and dmcrypt to provide a decrypted block device
abstration so if they really need to be zapped a mapping needs to be
removed first.

This change also pulls charms.ceph modifications.

Change-Id: I96b3d40b3f9e56681be142377e454b15f9e22be3
Co-Authored-By: Dmitrii Shcherbakov <dmitrii.shcherbakov@canonical.com>
Co-Authored-By: Chris Procter <chris.procter@canonical.com>
Closes-Bug: 1781453
2018-07-17 14:30:51 +03:00
Frode Nordahl 352d699387
Add pre-flight check for device pristinity
Add `non-pristine` key to `list-disks` action.

No longer attempt to do initializtion of `osd-journal` devices.

Make py27 test noop

Flip pep8 test to py3

Partial-Bug: #1698154
Change-Id: I0ca574fa7f0683b4e8a693b9f62fbf6b39689789
Depends-On: I90a866aa138d18e4242783c42d4c7c587f696d7d
2018-06-04 17:02:11 +02:00