Commit Graph

128 Commits

Author SHA1 Message Date
Luciano Lo Giudice 77f73ae97c Add the mapping for Reef
Change-Id: Id29d9a5ebc55b7fef716406527648fbd98278ffd
2023-09-13 13:03:11 -03:00
Luciano Lo Giudice 44520d2330 Only upgrade keys for internal clients
When fetching ceph keys, they are upgraded if it's deemed necessary
to fix bug #1968369. This worked for internal ceph entities such
as RBD or MDS, but fails for external ceph clients. This is because
for those clients, key capabilities can be set via broker requests.
As such, when the ceph-mon charm is upgraded, the key capabilities
for those clients are reset, and since the broker request has been
processed already, no further changes are done, leaving those keys
with incorrect caps.

Change-Id: I878091f9387f22059d460d28c2d0072507e66c6f
Closes-Bug: #2028387
2023-07-26 16:01:31 -03:00
Samuel Walladge 07671ae83d Fix crash in extracting local osd ids
The function `get_local_osd_ids` crashes when the /var/lib/ceph/osd/
directory contains a file whose name doesn't contain a `-`.

unit-ceph-osd-0: 03:34:14 WARNING unit.ceph-osd/0.mon-relation-changed Traceback (most recent call last):
unit-ceph-osd-0: 03:34:14 WARNING unit.ceph-osd/0.mon-relation-changed   File "/var/lib/juju/agents/unit-ceph-osd-0/charm/hooks/mon-relation-changed", line 904, in <module>
unit-ceph-osd-0: 03:34:14 WARNING unit.ceph-osd/0.mon-relation-changed     hooks.execute(sys.argv)
unit-ceph-osd-0: 03:34:14 WARNING unit.ceph-osd/0.mon-relation-changed   File "/var/lib/juju/agents/unit-ceph-osd-0/charm/hooks/charmhelpers/core/hookenv.py", line 963, in execute
unit-ceph-osd-0: 03:34:14 WARNING unit.ceph-osd/0.mon-relation-changed     self._hooks[hook_name]()
unit-ceph-osd-0: 03:34:14 WARNING unit.ceph-osd/0.mon-relation-changed   File "/var/lib/juju/agents/unit-ceph-osd-0/charm/hooks/mon-relation-changed", line 668, in mon_relation
unit-ceph-osd-0: 03:34:14 WARNING unit.ceph-osd/0.mon-relation-changed     ceph.apply_osd_settings(settings)
unit-ceph-osd-0: 03:34:14 WARNING unit.ceph-osd/0.mon-relation-changed   File "/var/lib/juju/agents/unit-ceph-osd-0/charm/lib/charms_ceph/utils.py", line 3420, in apply_osd_settings
unit-ceph-osd-0: 03:34:14 WARNING unit.ceph-osd/0.mon-relation-changed     for osd_id in get_local_osd_ids():
unit-ceph-osd-0: 03:34:14 WARNING unit.ceph-osd/0.mon-relation-changed   File "/var/lib/juju/agents/unit-ceph-osd-0/charm/lib/charms_ceph/utils.py", line 698, in get_local_osd_ids
unit-ceph-osd-0: 03:34:14 WARNING unit.ceph-osd/0.mon-relation-changed     osd_id = osd_dir.split('-')[1]
unit-ceph-osd-0: 03:34:14 WARNING unit.ceph-osd/0.mon-relation-changed IndexError: list index out of range

Fix the issue by setting a fallback in this case.

Change-Id: I0fec65c9b88e6484905a76140d0921cf3dd843c0
2023-01-06 12:22:22 +10:30
Zuul c2cf2fcf18 Merge "Consider crimson-osds as well" 2022-11-08 14:48:05 +00:00
Luciano Lo Giudice 84adacf5c8 Consider crimson-osds as well
This patchset modifies the 'get_running_osds' function so that
it also considers crimson-osds as valid OSD's. In addition, it
also modifies the 'get_local_osd_ids' to _not_ include Crimson
OSDs so that unsupported functionality doesn't make some hooks
fail in the ceph-osd charm.

Change-Id: If58bde4d5445ed5de420abc007db6bf8b8e43269
2022-10-19 17:09:01 -03:00
Chris MacNaughton becd07a584 Cache calls to ceph get_named_key
Change-Id: Iccc1d4a3ef116e9edb4d3047f40993fa6107718e
2022-09-28 20:24:26 +00:00
Luciano Lo Giudice 78bd3fd21a Fix linting errors
Change-Id: I16d01e15438e3d61033863b4c03087d1a0b7a008
2022-08-01 17:44:29 -03:00
James Page 89f5ee0d04 Amend rbd-mirror capabilities
Update rbd-mirror key caps to use the rbd-mirror-peer mon profile.

This adds required config get/set permissions for the rbd-mirror
daemon.

Also add permission to use the service {dump,status} mon commands;
these are not required by the rbd-mirror daemon but are needed by
the charm to check pool mirror status.

Change-Id: Ia5efa090f5b7fa23d7261da53592d6a527dcbfe1
Closes-Bug: 1879749
2022-04-20 17:37:16 +01:00
Luciano Lo Giudice 5745ed3ba8 Add permission in mon key for newly named command
Starting with Pacific, the 'osd blacklist' command was renamed to
'osd blocklist'. This patchset changes the allowed commands to
reflect this change.

Change-Id: If2169734f67d21c1c7c1b75677f14ebd0ea054ae
Closes-Bug: #1968369
2022-04-13 10:27:39 -03:00
Chris MacNaughton 476ae63f3b Resolve type change in Quincy for enabled_manager_modules
Closes-Bug: #1968715
Change-Id: I193f787fe8c7d271028c0b404a6a745952308697
2022-04-12 12:55:03 +02:00
Chris MacNaughton 18fc2c6748 Ensure we restart the correct systemd unit on upgrade.
Closes-Bug: #1962296
Change-Id: I7acabfcb28b5faee3d4a6018595d24914db552b2
2022-02-25 11:09:47 +01:00
Zuul 0d8bfe8e10 Merge "Add argument 'osd_id' to 'osdize'" 2022-02-10 10:32:36 +00:00
Luciano Lo Giudice 136f6d8586 Add argument 'osd_id' to 'osdize'
In order to fully support OSD replacement, we need a way to recycle
previously used OSD ids. This is achieved by passing the parameter
'--osd-id' to 'ceph-volume lvm create'. As such, this patchset adds
the needed support to specify that parameter.

Change-Id: I43d0a0bc11664c37532c0117711affc93c9d1ad1
2022-01-18 18:31:41 -03:00
Zuul bfa3691822 Merge "Explicitly enable msgr2 for Nautilus and later" 2022-01-11 09:54:44 +00:00
James Troup 0dec5c0188 Spelling fixes from codespell with additional consistency cleanups.
Change-Id: I1d409043eec980982fc6fc472524c562f908d402
2021-12-20 13:54:16 +00:00
James Vaughn ff752de0f0 Explicitly enable msgr2 for Nautilus and later
When updating to Ceph Nautilus (OpenStack Train) or later releases from
a pre-Nautilus release, msgr2 will not be enabled without manual
intervention (e.g. running `juju ssh ceph-mon/leader 'sudo ceph mon
enable-msgr2'`).

Leaving msgr2 disabled (i.e. using msgr1 only) isn't an issue for normal
operations; Ceph will simply show a warning in `sudo ceph status` that
this is the case. However, Ceph Nautilus and later releases default to
msgr2, meaning that new ceph-mon units will attempt to communicate with
the existing cluster using msgr2.

This commit enables msgr2 on package upgrades on all Ceph releases
including and after Nautilus.

Closes-Bug: #1840701
Change-Id: Ib90cd03b16f2062fa6c6a43d242d51306ae8ca95
2021-12-14 14:51:06 +01:00
Aurelien Lourot 328aa0f051 Add missing pacific and quincy bits
Change-Id: I5a057463e79e8b954cd2f8bf71207df0e0276689
2021-11-16 13:07:00 +01:00
Luciano Lo Giudice 16bcf75712 Only consider mounted OSD directories
When gathering the list of local OSD ids, the charm would consider
the entries under '/var/lib/ceph/osd/ceph-XXX' where 'XXX" was the
OSD id. However, if an entry under that directory isn't mounted,
then the OSD that would represent that entry should be discarded,
as it's no longer active. This patchset thus filters those entries
by looking for them in the mount points.

Closes-Bug: #1934938
Change-Id: I7ca84ef0b73620fe69d309546c746933e95ceb03
2021-08-30 20:10:30 -03:00
Robert Gildein 5a6fbb88fd Fix parsing information from ceph tree output
Add helper functions to parse information from the ceph tree and
fix creating CrushLocation object in get_osd_tree function.

Closes-Bug: #1918721
Change-Id: I59c742e594042ad527e71c88404999459f0373c2
2021-07-28 07:55:34 +02:00
Liam Young e8240a764e Utils for managing modules and setting config
The methods for checking modules are taken from the ceph-mon
charm and charmhelpers.contrib.storage.linux.ceph module. When
this lands I'll update the ch methods to emit a deprecation warning
and  update the ceph-mon charm to use these methods instead.

Change-Id: I1b7f08c3bd94828a5c6468df7aab6b38517cf3fa
2021-06-21 13:52:18 +00:00
Alex Kavanagh 626a5ce3ca Fix "Remove chrony in a container" patch with test
Unfortunately, the original patch didn't work [1].  Then a fix was
applied also without a test [2].  Unfortunately, this wasn't correct
either.

This patch simplifies the function, which makes it a little easier to
reason about.  Also a test is added to stop regressions in the future.

[1] Ie3c9c5899c1d46edd21c32868938d3290db321e7
[2] Ifef76eacd8bd837d2181ec75e406aa35f88b8b5b
Closes-Bug: #1929054

Change-Id: I93f53274bc8c7e62ebb8cb7bf81ff816f4bfd98b
2021-05-24 16:02:12 +01:00
Edward Hope-Morley a14800daa1 Upgrade mon packages immediately
Instead of deferring the upgrading of packages
when doing a release upgrade, do them immediately
and only defer daemon restarts. This is to avoid
any api incompatibilities between newer mons and
older clients.

Change-Id: Idf77871561fa14ad61e0c7901eeb333254e37012
Closes-Bug: #1897594
2020-09-29 11:44:17 +01:00
Alex Kavanagh 272abf6e55 Fix breaking in code if extra_pools is passed as None
The code currently assumes that a empty list will be passed rather than
None, so this change defensively codes against a None being passed
(deserialised from json) so that it is correctly parsed as "no extra
pools".  Also modified the test to push None to the function to verify
that it handles the situation

Change-Id: I27220696f64b65280f24cab2328e1366fcbbdf4f
Closes-Bug: #1897574
2020-09-28 16:44:57 +01:00
Liam Young 648cac3ba8 Handle additional pools for CephFS
If ceph-fs requests additional pools are added to the ceph fs
share to be created then add them once the share is created.

This allows ceph-fs to specify replicated pools for metadata and
default datapool but to then extend the fs with EC pools.

Change-Id: I80b7a5cc87d7d53bb55d4d65999a0f9b3cdcb77d
2020-09-10 14:34:58 +00:00
Frode Nordahl af0eac506d
Use common pool create/update handling code
Use the Pool classes ability to initialize themselves from op.

Some pool properties should be allowed adjusted after a pool is
created.

The Pool quota property is already handled ad-hoc in the
`charms.ceph` broker handling code, let's bring it over to the
pool objects along with the handling of the compression properties.

Move some missing unit tests here from ceph-mon charm.

Merge after https://github.com/juju/charm-helpers/pull/497

Change-Id: Ibec4e3221387199adbc1a920e130975d7b25343c
2020-08-26 15:27:03 +02:00
Pedro Guimaraes 89b3b8d1eb Update support for Erasure Coded pools
Add support to allow 'allow_ec_overwrites' to be set on a pool,
supporting RBD and CephFS uses cases for Erasure Coding.

Add support to allow the EC technique to be provided when
creating EC profiles.

Add support for extended configuration of EC plugins.

Depends-On: https://github.com/juju/charm-helpers/pull/498
Change-Id: I2547933964849f7af1c623b2fbc014fb332839ef
2020-08-07 09:10:11 +01:00
Alex Kavanagh b688f04889 Add a progress watchdog for OSD upgrades
This patch add the concept of a watchdog to the upgrade_monitor so that
the charm can achieve two objectives of 1. Waiting for much longer, but
2. detecting whether the previous node has died / gone away.  This is
needed for 'large' OSDs where the time to upgrade a node may exceed the
current limit of 10 minutes, but also not to wait for 30 minutes on a
dead previous node.  The watchdog implements two timeouts and an
addition 'alive' key from the previous node to indicate that it is still
running.  Otherwise, functionality is identical.

Change-Id: Ia450e936c2096f092af3be5a369b7abaf5023b16
Related-Bug: #1762852
2020-07-14 12:53:40 +01:00
Zuul 5c06f74ab3 Merge "Use unittest.mock instead of mock" 2020-07-13 05:54:38 +00:00
Alex Kavanagh 05c8a8a5b8 Fix upgrade bug (luminous) where mgr is not restarted
From luminous to mimic, the ceph-mgr isn't restarted, leaving it at
luminous, rather than mimic.  This change ensures that the ceph-mgr is
restarted when the version is luminous or later.

Change-Id: I2844736af2de27ce071db859311ac99b9b3057ad
Related-Bug: #1849874
2020-07-08 15:39:43 +01:00
Hervé Beraud 4f135d9e97 Use unittest.mock instead of mock
The mock third party library was needed for mock support in py2
runtimes. Since we now only support py36 and later, we can use the
standard lib unittest.mock module instead.

Change-Id: Id257b597368aff252ce7120b9dbc02f21a41e629
2020-06-09 00:03:48 +02:00
Liam Young 7686d4b584 An extra permission for rbd mirror on focal
Adding mon allow r for rbd mirror clients to fix failure to query
mirror pool status.

Change-Id: I63d9f137556b8aef379ee54ea977094722dbf032
Partial-Bug: #1879464
2020-05-19 09:40:36 +00:00
Liam Young 3261c5aa5d Allow rbd mirror client to query mgr
Querying mirror pool status now requires read perms on mgr. Without
this perm a warning is thrown which corrupts the json output.

Closes-Bug: #1851869
Change-Id: I59266da1bf393468f8e0c69dd7212e4cc6de96cf
2020-05-18 18:56:27 +00:00
James Page 2d61a56544 Trigger udev rescan if pv_dev disappears
Workaround for kernel by in Ubuntu 20.04 LTS.

When using by-dname device paths with MAAS and bcache, the pvcreate
operation results in the by-dname entry for the block device being
deleted.  The subsequent vgcreate then fails as the path cannot
be found.

Trigger a rescan of block devices if the pv_dev path does not
exists after the pvcreate operation.

Change-Id: Ifb16c47ae5ff316cbcfc3798de3446a3774fa012
Related-Bug: 1878752
2020-05-18 08:40:41 +01:00
Zuul cc06e5010f Merge "Add apply_osd_settings" 2020-04-17 12:50:32 +00:00
Liam Young 4fc32cc401 Add apply_osd_settings
Add apply_osd_settings for applying osd settings to running osd
daemons.

Change-Id: If8fa9020d9cbe2c8fcf981abd737653173926341
2020-04-17 12:10:52 +00:00
James Page d0d175aec3 upgrades: add upgrade paths for octopus
Detail octopus upgrade path and link with Ussuri UCA.

Change-Id: I7dbdc5c265641d4eb561b69f57b8bab41781b698
2020-04-09 10:58:29 +01:00
Corey Bryant 26114b2a54 Maintain OSD state on upgrade
Ensure each OSD reaches its pre-restart state before proceeding
after restart. This prevents the charm from finalizing the upgrade
prior to OSDs recovering after upgrade. For example, if the state
is 'active' prior to restart, then it must reach 'active' after
restart, at which point the upgrade will be allowed to complete.

Change-Id: I66359963b5e8e792d23b7989f07b7ac67afcd9b0
Closes-Bug: #1821028
2020-03-18 21:31:39 +00:00
Zuul 35608d6bf7 Merge "Unpin flake8, fix lint" 2020-03-16 17:08:06 +00:00
Andrew McLeod 425465d473 Fast exit if device_node for block device is None
Also update corresponding unit test

Change-Id: I6e61d13617f2e7ec500a0b010516a07af2a1e917
Closes-Bug: #1866956
2020-03-13 14:48:14 +01:00
Frode Nordahl fead2944a8
Unpin flake8, fix lint
We need to unpin flake8 to allow linting on Python 3.8 systems.

Change-Id: Idf2fea27b19cec47ffed9891b518ac7b5b75e405
2020-03-12 11:12:37 +01:00
Frode Nordahl 6a00cf92e1
Add newline at end of file when writing keyring
Change-Id: If79977c2ec7588b09ca2e118141c7f3d9cf61767
Closes-Bug: #1864706
2020-02-26 11:27:35 +01:00
Corey Bryant c5a7e8f1c8 Rename ceph to charms_ceph
The new python3-ceph-common deb package (introduced in ceph octopus)
adds a new ceph directory (a parent package in python terms) in
/usr/lib/python3/dist-packages/ceph/. This results in a conflict with
charm-ceph-xyz/lib/ceph/ on imports. For example, with the current
import of ceph.utils in ceph-osd/hooks/ceph_hooks.py, Python finds no
utils.py in /usr/lib/python3/dist-packages/ceph/ and then stops
searching. Therefore, rename ceph to charms_ceph to avoid the
conflict.

Change-Id: I6eaf03e291f65125269b1836dd2636806f9945e1
2020-02-24 15:19:34 +00:00
Chris MacNaughton 0434024b2b Fix charms.ceph mocks to not depend on system version
The charms.ceph utils.py library defines a constant based
on the running OS version. This change updates where the
version is read, allowing it to be nicely mocked.

Change-Id: Ibd65526cad0812f93963bd82619fcd27e055f6f3
Closes-Bug: #1861659
2020-02-03 10:18:08 +01:00
Zuul 8f9e80d26e Merge "Let unmounted_disks not skipping device "/dev/sdm"" 2019-12-19 11:58:48 +00:00
Alexandros Soumplis d393585cfb Ignore devices with OSError
This bug handles OSError when trying to read from CDrom devices with no
disc in the is_pristine_disk function which makes the run-action
list-disks to fail.

Change-Id: I951897a699305604821f2c910ee9ea91582c4e40
Closes-Bug: #1833857
Co-authored-by: Stamatis Katsaounis <skatsaounis@admin.grnet.gr>
Signed-off-by: Alexandros Soumplis <soumplis@admin.grnet.gr>
2019-12-17 14:49:27 +02:00
Zuul 3b6a74bea0 Merge "Add new permission request to broker" 2019-11-07 13:04:22 +00:00
Chris MacNaughton bb708ad4c4 Add new permission request to broker
Change-Id: Ifd341bd80833d4a7fd62e89e3c0e2b7fd64fafba
2019-11-07 20:07:58 +08:00
taodd 740eadcde5 Let unmounted_disks not skipping device "/dev/sdm"
Change-Id: I7bf90531fe5bb01ff47cdeff879b5f72fd227855
Closes-Bug: #1848000
2019-10-23 22:00:13 +08:00
Sahid Orentino Ferdjaoui 7839ca39bf put create_keyrings as a public method
This method should be call after the quorum has been done. The right
pattern would be:

  ceph.wait_quorum()
  ceph.create_keyrings()

Change-Id: I6c754cb2d1de57a59416e96aa6ab5cfcaec5df44
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@canonical.com>
2019-10-22 15:40:44 +00:00
Zuul ee5de74d0a Merge "Nautilus no longer supports creating directory backed OSDs" 2019-09-12 14:06:18 +00:00