Commit Graph

256 Commits

Author SHA1 Message Date
Luciano Lo Giudice 5d62e13ae3 Add support for bytes-like requests
Under some circumstances, some requests may be a bytes-like object.
This patchset adds support for them.

Change-Id: I274ddd586604f93c1335e8511f5dd1773736ed4d
2023-10-09 17:56:16 -03:00
Luciano Lo Giudice 77f73ae97c Add the mapping for Reef
Change-Id: Id29d9a5ebc55b7fef716406527648fbd98278ffd
2023-09-13 13:03:11 -03:00
Zuul 571659f4a3 Merge "Only upgrade keys for internal clients" 2023-07-26 19:25:50 +00:00
Luciano Lo Giudice 44520d2330 Only upgrade keys for internal clients
When fetching ceph keys, they are upgraded if it's deemed necessary
to fix bug #1968369. This worked for internal ceph entities such
as RBD or MDS, but fails for external ceph clients. This is because
for those clients, key capabilities can be set via broker requests.
As such, when the ceph-mon charm is upgraded, the key capabilities
for those clients are reset, and since the broker request has been
processed already, no further changes are done, leaving those keys
with incorrect caps.

Change-Id: I878091f9387f22059d460d28c2d0072507e66c6f
Closes-Bug: #2028387
2023-07-26 16:01:31 -03:00
Corey Bryant 3afdbc586c Add bobcat:quincy release mapping
Change-Id: I9041be6b9aa3cb1650cf507ee90a2fea11378bc3
2023-07-19 14:13:13 -04:00
Zuul fb13f31d3e Merge "Key capabilities should be kept up to date" 2023-06-28 17:26:39 +00:00
Peter Sabaini 7dff501405 Add antelope:quincy release mapping
Change-Id: I2b9ccc1aa7f198f086588c320c647661ef7bbdd5
2023-06-20 18:01:50 +02:00
Chris MacNaughton 7cffb0b7a7 Key capabilities should be kept up to date
When capabilities are only ever set on key-creation,
they can get out of date with what the services need
or want over time as new Ceph versions are released,
or as clients update their requirements.

Closes-Bug: #1968369
Change-Id: I475bd55117e3dc630514c92f6a76260e4f37ae0e
2023-05-26 14:06:08 -04:00
Samuel Walladge 07671ae83d Fix crash in extracting local osd ids
The function `get_local_osd_ids` crashes when the /var/lib/ceph/osd/
directory contains a file whose name doesn't contain a `-`.

unit-ceph-osd-0: 03:34:14 WARNING unit.ceph-osd/0.mon-relation-changed Traceback (most recent call last):
unit-ceph-osd-0: 03:34:14 WARNING unit.ceph-osd/0.mon-relation-changed   File "/var/lib/juju/agents/unit-ceph-osd-0/charm/hooks/mon-relation-changed", line 904, in <module>
unit-ceph-osd-0: 03:34:14 WARNING unit.ceph-osd/0.mon-relation-changed     hooks.execute(sys.argv)
unit-ceph-osd-0: 03:34:14 WARNING unit.ceph-osd/0.mon-relation-changed   File "/var/lib/juju/agents/unit-ceph-osd-0/charm/hooks/charmhelpers/core/hookenv.py", line 963, in execute
unit-ceph-osd-0: 03:34:14 WARNING unit.ceph-osd/0.mon-relation-changed     self._hooks[hook_name]()
unit-ceph-osd-0: 03:34:14 WARNING unit.ceph-osd/0.mon-relation-changed   File "/var/lib/juju/agents/unit-ceph-osd-0/charm/hooks/mon-relation-changed", line 668, in mon_relation
unit-ceph-osd-0: 03:34:14 WARNING unit.ceph-osd/0.mon-relation-changed     ceph.apply_osd_settings(settings)
unit-ceph-osd-0: 03:34:14 WARNING unit.ceph-osd/0.mon-relation-changed   File "/var/lib/juju/agents/unit-ceph-osd-0/charm/lib/charms_ceph/utils.py", line 3420, in apply_osd_settings
unit-ceph-osd-0: 03:34:14 WARNING unit.ceph-osd/0.mon-relation-changed     for osd_id in get_local_osd_ids():
unit-ceph-osd-0: 03:34:14 WARNING unit.ceph-osd/0.mon-relation-changed   File "/var/lib/juju/agents/unit-ceph-osd-0/charm/lib/charms_ceph/utils.py", line 698, in get_local_osd_ids
unit-ceph-osd-0: 03:34:14 WARNING unit.ceph-osd/0.mon-relation-changed     osd_id = osd_dir.split('-')[1]
unit-ceph-osd-0: 03:34:14 WARNING unit.ceph-osd/0.mon-relation-changed IndexError: list index out of range

Fix the issue by setting a fallback in this case.

Change-Id: I0fec65c9b88e6484905a76140d0921cf3dd843c0
2023-01-06 12:22:22 +10:30
Zuul c2cf2fcf18 Merge "Consider crimson-osds as well" 2022-11-08 14:48:05 +00:00
Luciano Lo Giudice 84adacf5c8 Consider crimson-osds as well
This patchset modifies the 'get_running_osds' function so that
it also considers crimson-osds as valid OSD's. In addition, it
also modifies the 'get_local_osd_ids' to _not_ include Crimson
OSDs so that unsupported functionality doesn't make some hooks
fail in the ceph-osd charm.

Change-Id: If58bde4d5445ed5de420abc007db6bf8b8e43269
2022-10-19 17:09:01 -03:00
Chris MacNaughton 381b04ffde add zed to release mapping
Change-Id: I75afd4eb312f4394e8911e4b4a5eb131c7b55655
2022-10-03 11:22:43 -04:00
Chris MacNaughton becd07a584 Cache calls to ceph get_named_key
Change-Id: Iccc1d4a3ef116e9edb4d3047f40993fa6107718e
2022-09-28 20:24:26 +00:00
Chris MacNaughton a7f16a76f8 bump CI jobs to Yoga
Change-Id: I350bd2ff0748279f4b46d83a37dd6caf145da725
2022-09-28 16:22:07 -04:00
Luciano Lo Giudice 45fe9ad44e Properly account for charms.ceph dependencies
The setup.py file needs to be specify the dependencies of this
library, to avoid depending on them being manually installed.

Change-Id: I3ff2338f967aff5613bde777b224cd96ea81c1fd
Closes-Bug: #1987547
2022-08-24 20:19:33 +00:00
Luciano Lo Giudice 78bd3fd21a Fix linting errors
Change-Id: I16d01e15438e3d61033863b4c03087d1a0b7a008
2022-08-01 17:44:29 -03:00
James Page 89f5ee0d04 Amend rbd-mirror capabilities
Update rbd-mirror key caps to use the rbd-mirror-peer mon profile.

This adds required config get/set permissions for the rbd-mirror
daemon.

Also add permission to use the service {dump,status} mon commands;
these are not required by the rbd-mirror daemon but are needed by
the charm to check pool mirror status.

Change-Id: Ia5efa090f5b7fa23d7261da53592d6a527dcbfe1
Closes-Bug: 1879749
2022-04-20 17:37:16 +01:00
Luciano Lo Giudice 5745ed3ba8 Add permission in mon key for newly named command
Starting with Pacific, the 'osd blacklist' command was renamed to
'osd blocklist'. This patchset changes the allowed commands to
reflect this change.

Change-Id: If2169734f67d21c1c7c1b75677f14ebd0ea054ae
Closes-Bug: #1968369
2022-04-13 10:27:39 -03:00
Chris MacNaughton 476ae63f3b Resolve type change in Quincy for enabled_manager_modules
Closes-Bug: #1968715
Change-Id: I193f787fe8c7d271028c0b404a6a745952308697
2022-04-12 12:55:03 +02:00
Chris MacNaughton 18fc2c6748 Ensure we restart the correct systemd unit on upgrade.
Closes-Bug: #1962296
Change-Id: I7acabfcb28b5faee3d4a6018595d24914db552b2
2022-02-25 11:09:47 +01:00
Zuul 0d8bfe8e10 Merge "Add argument 'osd_id' to 'osdize'" 2022-02-10 10:32:36 +00:00
Luciano Lo Giudice 136f6d8586 Add argument 'osd_id' to 'osdize'
In order to fully support OSD replacement, we need a way to recycle
previously used OSD ids. This is achieved by passing the parameter
'--osd-id' to 'ceph-volume lvm create'. As such, this patchset adds
the needed support to specify that parameter.

Change-Id: I43d0a0bc11664c37532c0117711affc93c9d1ad1
2022-01-18 18:31:41 -03:00
Zuul bfa3691822 Merge "Explicitly enable msgr2 for Nautilus and later" 2022-01-11 09:54:44 +00:00
James Troup 0dec5c0188 Spelling fixes from codespell with additional consistency cleanups.
Change-Id: I1d409043eec980982fc6fc472524c562f908d402
2021-12-20 13:54:16 +00:00
James Vaughn ff752de0f0 Explicitly enable msgr2 for Nautilus and later
When updating to Ceph Nautilus (OpenStack Train) or later releases from
a pre-Nautilus release, msgr2 will not be enabled without manual
intervention (e.g. running `juju ssh ceph-mon/leader 'sudo ceph mon
enable-msgr2'`).

Leaving msgr2 disabled (i.e. using msgr1 only) isn't an issue for normal
operations; Ceph will simply show a warning in `sudo ceph status` that
this is the case. However, Ceph Nautilus and later releases default to
msgr2, meaning that new ceph-mon units will attempt to communicate with
the existing cluster using msgr2.

This commit enables msgr2 on package upgrades on all Ceph releases
including and after Nautilus.

Closes-Bug: #1840701
Change-Id: Ib90cd03b16f2062fa6c6a43d242d51306ae8ca95
2021-12-14 14:51:06 +01:00
Chris MacNaughton d364605a49 Stop setting default capabilities in get_named_key.
When get_named_key is called on a keyring that
has customised capabilities, it overwrites the
capabilities to the default capabilities.

Change-Id: I777650ae2194609a95e5cfb101e343824b3d0a34
2021-12-10 15:01:45 +01:00
Aurelien Lourot 328aa0f051 Add missing pacific and quincy bits
Change-Id: I5a057463e79e8b954cd2f8bf71207df0e0276689
2021-11-16 13:07:00 +01:00
Luciano Lo Giudice 16bcf75712 Only consider mounted OSD directories
When gathering the list of local OSD ids, the charm would consider
the entries under '/var/lib/ceph/osd/ceph-XXX' where 'XXX" was the
OSD id. However, if an entry under that directory isn't mounted,
then the OSD that would represent that entry should be discarded,
as it's no longer active. This patchset thus filters those entries
by looking for them in the mount points.

Closes-Bug: #1934938
Change-Id: I7ca84ef0b73620fe69d309546c746933e95ceb03
2021-08-30 20:10:30 -03:00
Robert Gildein 5a6fbb88fd Fix parsing information from ceph tree output
Add helper functions to parse information from the ceph tree and
fix creating CrushLocation object in get_osd_tree function.

Closes-Bug: #1918721
Change-Id: I59c742e594042ad527e71c88404999459f0373c2
2021-07-28 07:55:34 +02:00
Liam Young e8240a764e Utils for managing modules and setting config
The methods for checking modules are taken from the ceph-mon
charm and charmhelpers.contrib.storage.linux.ceph module. When
this lands I'll update the ch methods to emit a deprecation warning
and  update the ceph-mon charm to use these methods instead.

Change-Id: I1b7f08c3bd94828a5c6468df7aab6b38517cf3fa
2021-06-21 13:52:18 +00:00
Zuul fed8b29000 Merge "Fixed raise text value instead of Exception" 2021-06-08 06:32:42 +00:00
Alex Kavanagh 626a5ce3ca Fix "Remove chrony in a container" patch with test
Unfortunately, the original patch didn't work [1].  Then a fix was
applied also without a test [2].  Unfortunately, this wasn't correct
either.

This patch simplifies the function, which makes it a little easier to
reason about.  Also a test is added to stop regressions in the future.

[1] Ie3c9c5899c1d46edd21c32868938d3290db321e7
[2] Ifef76eacd8bd837d2181ec75e406aa35f88b8b5b
Closes-Bug: #1929054

Change-Id: I93f53274bc8c7e62ebb8cb7bf81ff816f4bfd98b
2021-05-24 16:02:12 +01:00
Trent Lloyd 53a183ee59 filter_missing_packages expects a list, not string
filter_missing_packages expects a list and not a single package name as
a string. Pass the chrony package as a single item list to avoid it
checking installation candiates for 'c', 'h', 'r', 'o', 'n' and 'y'

WARNING juju-log Package c has no installation candidate.
WARNING juju-log Package h has no installation candidate.
WARNING juju-log Package r has no installation candidate.
WARNING juju-log Package o has no installation candidate.
WARNING juju-log Package n has no installation candidate.
WARNING juju-log Package y has no installation candidate.
INFO juju-log Purging ['chrony']

Change-Id: Ifef76eacd8bd837d2181ec75e406aa35f88b8b5b
2021-05-20 11:47:43 +08:00
Robert Gildein 1a7e8df4ef Fixed raise text value instead of Exception
I used ValueError for both exception.
Also removed unnecessary flake8 ignore option.

Closes-Bug: #1917351
Change-Id: I5ee58aa0a34d7c266b3811f9c6b9b3076bfdc048
2021-03-31 21:26:21 +02:00
Zuul 9bfe43ee65 Merge "modify get_version to use charmhelpers get_installed_version" 2021-03-24 10:31:27 +00:00
Alex Kavanagh d0d002cacf Replace ostestr with stestr in comment in setup.py
charms.ceph actually uses stestr to perform unit testing.

Change-Id: I07c53fdea4e79ba92d0a3c2535020f9cf723d86d
2021-02-10 11:33:42 +00:00
Hemanth Nakkina 9cdabd1b61 modify get_version to use charmhelpers get_installed_version
get_version() uses charmhelpers apt_cache to retreive the
current version of ceph package which takes longer time to
execute. This has been improved in charm-helpers PR [1] by
introducing new function get_installed_version.

This patch updates get_version() to use new charm-helpers
get_installed_version function.
ceph_user() is updated to return always 'ceph' as ceph is
not supported for versions < 1.0

Note: This patch depends on PR [1] and so need to wait until
charm-helpers PR is merged and new version of charmhelpers
is released.

[1] https://github.com/juju/charm-helpers/pull/567

Partial-Bug: #1913992
Change-Id: Ibdb36cb08950344ee55fdb8b54b47b0381f72e2c
2021-02-05 12:42:56 +05:30
Edward Hope-Morley a14800daa1 Upgrade mon packages immediately
Instead of deferring the upgrading of packages
when doing a release upgrade, do them immediately
and only defer daemon restarts. This is to avoid
any api incompatibilities between newer mons and
older clients.

Change-Id: Idf77871561fa14ad61e0c7901eeb333254e37012
Closes-Bug: #1897594
2020-09-29 11:44:17 +01:00
Alex Kavanagh 272abf6e55 Fix breaking in code if extra_pools is passed as None
The code currently assumes that a empty list will be passed rather than
None, so this change defensively codes against a None being passed
(deserialised from json) so that it is correctly parsed as "no extra
pools".  Also modified the test to push None to the function to verify
that it handles the situation

Change-Id: I27220696f64b65280f24cab2328e1366fcbbdf4f
Closes-Bug: #1897574
2020-09-28 16:44:57 +01:00
Zuul 771981e23f Merge "Handle additional pools for CephFS" 2020-09-18 09:02:23 +00:00
Zuul a98681d2d1 Merge "Remove chrony if inside a container" 2020-09-17 08:06:35 +00:00
Zuul c22511dab0 Merge "Fix a couple of typos" 2020-09-15 16:51:01 +00:00
Ponnuvel Palaniyappan 8648c73dd5 Remove chrony if inside a container
When running ceph-mon in containers, best practice is
to have chrony/ntp configured and installed on the bare
metal and then have the container trust the system
clock, as the container should not manage the system
clock.

The chrony package get installed automatically as
part of the dependencies of other packages, which
gets removed in this change.

ceph-mon charm will use this change here to remove
the chrony package.

Change-Id: Ie3c9c5899c1d46edd21c32868938d3290db321e7
Closes-Bug: #1852441
2020-09-15 09:13:39 +01:00
Liam Young 648cac3ba8 Handle additional pools for CephFS
If ceph-fs requests additional pools are added to the ceph fs
share to be created then add them once the share is created.

This allows ceph-fs to specify replicated pools for metadata and
default datapool but to then extend the fs with EC pools.

Change-Id: I80b7a5cc87d7d53bb55d4d65999a0f9b3cdcb77d
2020-09-10 14:34:58 +00:00
Frode Nordahl af0eac506d
Use common pool create/update handling code
Use the Pool classes ability to initialize themselves from op.

Some pool properties should be allowed adjusted after a pool is
created.

The Pool quota property is already handled ad-hoc in the
`charms.ceph` broker handling code, let's bring it over to the
pool objects along with the handling of the compression properties.

Move some missing unit tests here from ceph-mon charm.

Merge after https://github.com/juju/charm-helpers/pull/497

Change-Id: Ibec4e3221387199adbc1a920e130975d7b25343c
2020-08-26 15:27:03 +02:00
Pedro Guimaraes 89b3b8d1eb Update support for Erasure Coded pools
Add support to allow 'allow_ec_overwrites' to be set on a pool,
supporting RBD and CephFS uses cases for Erasure Coding.

Add support to allow the EC technique to be provided when
creating EC profiles.

Add support for extended configuration of EC plugins.

Depends-On: https://github.com/juju/charm-helpers/pull/498
Change-Id: I2547933964849f7af1c623b2fbc014fb332839ef
2020-08-07 09:10:11 +01:00
Alex Kavanagh b688f04889 Add a progress watchdog for OSD upgrades
This patch add the concept of a watchdog to the upgrade_monitor so that
the charm can achieve two objectives of 1. Waiting for much longer, but
2. detecting whether the previous node has died / gone away.  This is
needed for 'large' OSDs where the time to upgrade a node may exceed the
current limit of 10 minutes, but also not to wait for 30 minutes on a
dead previous node.  The watchdog implements two timeouts and an
addition 'alive' key from the previous node to indicate that it is still
running.  Otherwise, functionality is identical.

Change-Id: Ia450e936c2096f092af3be5a369b7abaf5023b16
Related-Bug: #1762852
2020-07-14 12:53:40 +01:00
Zuul 5c06f74ab3 Merge "Use unittest.mock instead of mock" 2020-07-13 05:54:38 +00:00
Zuul c96595e8d4 Merge "Stop to use the __future__ module." 2020-07-13 05:39:29 +00:00
Zuul 85beb2fae5 Merge "Enable coercion of values (if requested) for validation" 2020-07-10 12:23:30 +00:00