Commit Graph

53 Commits

Author SHA1 Message Date
Corey Bryant c5a7e8f1c8 Rename ceph to charms_ceph
The new python3-ceph-common deb package (introduced in ceph octopus)
adds a new ceph directory (a parent package in python terms) in
/usr/lib/python3/dist-packages/ceph/. This results in a conflict with
charm-ceph-xyz/lib/ceph/ on imports. For example, with the current
import of ceph.utils in ceph-osd/hooks/ceph_hooks.py, Python finds no
utils.py in /usr/lib/python3/dist-packages/ceph/ and then stops
searching. Therefore, rename ceph to charms_ceph to avoid the
conflict.

Change-Id: I6eaf03e291f65125269b1836dd2636806f9945e1
2020-02-24 15:19:34 +00:00
Edward Hope-Morley dd978b7bd0 Code cleanup
* import ordering
* doctrings
* unit tests cleanup
* move code out of __init__.py
* rename files

Change-Id: Id56d4ea1a0cca33df03f40222bbf51a6bef58b20
2017-08-11 12:42:29 +01:00
Corey Bryant 55a89963d6 Add ceph-mgr bootstrap function
As of the Luminous release, Ceph requires ceph-mgr daemons be run
to have a fully functional cluster. [0]

[0] http://docs.ceph.com/docs/master/mgr/administrator/

Change-Id: I31197dcd3ad05c0db3811096c6d88308e7c10658
2017-08-10 20:32:09 +00:00
James Page f2b0fcf9e8 Add bluestore support for OSD's
Add support for use of experimental bluestore storage format
for Ceph Jewel or later.

This will be exposed in the charms via a new configuration
option.

Change-Id: I36f7aa9d7b96ec5c9eaa7a3a970593f9ca14cb34
2017-07-07 09:25:44 +01:00
James Page 43e7a3c691 Call ceph-create-keys for >= Luminos
Ceph Luminos removes the automatic call to ceph-create-keys on
ceph-mon startup; deployment tools are required to call this
after the monitors have been started to create the admin keyring
across the cluster.

ceph-create-keys still blocks until the cluster is in quorum, at
which points keys are created and written out to the normal location
in /etc/ceph.

Change-Id: Ifc9b8aaf3dbce71cb62567ba03455d2196eaf29f
2017-07-05 11:05:22 +01:00
Jenkins 7ed769c05c Merge "Fix typos in ceph health method" 2017-05-19 01:39:59 +00:00
David Ames e60c420377 Do not install NTP when installed in a container
Check if the charm is installed inside a container and do not install
NTP if that is the case.

Change-Id: I9a9a50e724e24aa0d5b04d544a82b2d8202cd181
Partial-Bug: #169051
2017-05-16 10:49:20 -07:00
Xav Paice 1add343574 Fix typos in ceph health method
Fix the errors in the log message.

Change ceph health to be ceph status - which has more info, and is
compatible in current usage.

Change-Id: I3bed0a3e87bb3fdf98266a91757ad39bfc58337e
2017-05-04 20:45:03 +12:00
Xav Paice 59f113d912 Add ceph methods for reweight
Adds the following methods:
- get_ceph_pg_stat
- get_ceph_health
- reweight_osd

Change-Id: I23a1563875cfa53cf4341f17ce14c5322cc955ca
2017-04-28 23:19:45 +12:00
Jenkins 1fec99b09c Merge "Fix alphanumeric comparisons for ubuntu releases" 2017-04-28 09:18:10 +00:00
Chris Holcombe 573bad86dc Zap Journals
The is_osd_disk code assumes that the first partition
is the osd journal.  That might not always be the case.
This patch makes that function more general by searching
all partitions on a disk.

Change-Id: I4239ae498b9b6d2d97551d61d1a4f2981f72b88a
2017-04-25 08:29:26 -07:00
Alex Kavanagh 080c1f76b0 Fix alphanumeric comparisons for ubuntu releases
- fix up code where the alpha comparisons are done

Change-Id: I992d8a18a6a573eabe2f5675f21eb64a5e99895f
Related-Bug: #1659575
2017-04-05 13:23:01 +01:00
Edward Hope-Morley b483fa2668 Fix pylint errors
Change-Id: I3cbb839a75ec3be91c74d968f528901f82e10129
2017-03-29 11:47:58 +01:00
Billy Olsen 80c8256fb6 Add unit tests and fix related bugs
This change adds more unit tests to the rolling OSD upgrade
scenario to ensure more complete coverage.

Change-Id: Ibd7c8d6c46520957a3298446efc6b5fff210a51a
Related-Bug: #1662591
2017-03-23 23:28:09 -07:00
Billy Olsen cd75b59b33 Fix typo when listing dirs
The directory listing incorrectly uses os.path.listdir but the
correct call is os.listdir.

Change-Id: I2fe87c8a007cfdb0d54395e78597c98647cb661c
Related-Bug: #1662591
2017-03-22 12:27:08 -07:00
James Page 8e58c99777 Fixup python 3 compatibility for subprocess calls
check_output needs to be decoded in py3 environments; provide
a helper wrapper function todo this + update unit tests to deal
with this slightly different approach to UTF-8 decoding.

Change-Id: I95c4fce0d5cc997ba19f22b611cf99fbd5b17dd1
2017-03-10 16:02:48 +00:00
Billy Olsen 0e0e3933e3 Fix rendering of hdparm.conf file
Use the charmhelpers.core.templating template rendering logic
instead of the charmhelpers.core.services template rendering logic
when generating the hdparm.conf file.

Change-Id: I4b9718e49336d72501585279c0fc0cfcd465abb2
Closes-Bug: #1660240
2017-03-09 15:31:45 -07:00
Billy Olsen 857a00c65b Revert "Use libblkid instead of CLI"
This reverts commit a7f29e52ef
which switched to using libblkid instead of invoking the blkid
command on the commandline.

The replacement code introduced as part of that commit does not
actually fix the bug that it was intended to fix and only adds
unnecessary complexity to the code. The bug is being tagged
by this commit message to help identify the full set of fixes
for the actual bug.

Change-Id: I21b32197c8db4a3c667c986180cdd213fced2c26
Related-Bug: #1660240
2017-03-09 09:11:06 -07:00
Jenkins 0142d5045a Merge "Roll osd ownership changes through node" 2017-03-07 16:42:47 +00:00
Billy Olsen c421aa7429 Roll osd ownership changes through node
Change the OSD upgrade path so that the file ownership change
for the OSD directories are run one OSD at a time rather than
all of the OSDs at once.

Partial-Bug: #1662591

Change-Id: I3a1cf05207c070a8699e7ba749a0587b619d4679
2017-03-03 15:24:54 -07:00
Chris Holcombe e422e65fc9 Fix parse_key
If the returned cephx key from ceph contains [caps] the
parse_key function returns the wrong output.

Partial-Bug: 1665149
Change-Id: Icca4c174fa15b9637732a919ab5149f34260ae0f
2017-02-16 11:00:52 -08:00
Chris MacNaughton 9881001be3 Fix get_named_key
this change allows the get_named_key to check if
a key with the given name already exists (with any
permissions) and returns the named key if so.
Otherwise, it returns a new key created with the
requested name and permissions

Closes-Bug: 1664546
Change-Id: Ifd1ff35e5e55e2f4d480609643174d3bf4afd0c0
2017-02-14 10:09:29 -06:00
Jenkins 30141278c2 Merge "Bail autotune if uuid is None" 2017-02-02 14:39:41 +00:00
Jenkins d55f79f4e6 Merge "Use libblkid instead of CLI" 2017-02-01 09:32:53 +00:00
Chris Holcombe a7f29e52ef Use libblkid instead of CLI
This patch changes the get_block_uuid function around
to use libblkid instead of scraping the cli.

Closes-Bug: 1660240
Change-Id: Ieebf5714bceb9d41c70fc1dd0cfd17c903c3a107
2017-01-30 13:15:30 -08:00
Chris Holcombe 69b691edaf Bail autotune if uuid is None
tune_dev logs a debug message saying it cannot
continue and then tries to continue.

Closes-Bug: 1660240
Change-Id: I62317d80f9249030dcf1d9ccd80e4340ad8594a1
2017-01-30 08:25:00 -08:00
Billy Olsen c835714905 chownr /var/lib/ceph only when needed
Infernalis release of Ceph switched to running the ceph daemons
under the ceph:ceph user instead of root:root. The upgrade OSD
logic does not take this into conideration and blindly runs the
recursive chown for whatever the current user is expected to be
when the upgrade is run.

This is problematic in production clusters with large numbers of
objects stored across 10 or more OSDs as this will cause a walk
of all the inodes under /var/lib/ceph unnecessarily. This change
ensures that the chown is only done when the upgrade target is
Jewel (Ubuntu Cloud Archives do not contain Infernalis release).

Change-Id: Id4bd48ebb90928b989df718404263e40f926c201
Partial-Bug: #1657527
2017-01-18 11:31:38 -07:00
James Page 9c707f166a Ensure that links are followed during upgrade
When upgrading mon and osd units, ensure that symlinks are followed
to ensure that directory based OSD's are upgraded correctly (which
are typically a symlink from /var/lib/ceph/osd/X -> actual_dir).

Change-Id: I7656bbc633b9706bd406dbca0f11fe6108916a02
Partial-Bug: 1611082
2017-01-07 16:35:34 +00:00
Chris MacNaughton 2ecb4f641a Add broker request to move Disk
This allows a remote charm, ideally an OSD unit,
to request that the monitor move a disk to a given
bucket in the Crushmap

Change-Id: I77d8159d588f3b31d6eb1c25cd5ca16e53c6f01d
2016-12-06 14:34:42 -05:00
gengchc2 c04b57a79f Correct reraising of exception
When an exception was caught and rethrown, it should
call 'raise' without any arguments because it shows
the place where an exception occured initially instead
of place where the exception re-raised

Change-Id: I8742aeeecb64bbd0ab5e89b1cc3d057cea5edf35
2016-12-02 10:37:56 +08:00
Chris MacNaughton c79af13c15 Add support for listing unmounted devices
Change-Id: Ia30b3104086a1b1bd45e14f7d328ce383d1e4ee0
2016-11-15 09:45:36 -05:00
James Page f5b53c6abe Tidy up status set/log messages during upgrades
Misc updates to use of status_set, and ensure that
the new version of ceph is correctly logged by passing down
to upgrade_{mon,osd}.

Change-Id: If094eacbc8cab598a0913bf1f93e5a6de88203d7
2016-10-20 15:24:47 +01:00
James Page 316a1f5d1f Add helper for UCA -> ceph version resolution
resolve_ceph_version will map the provided source configuration
option to an associated UCA/OpenStack release, and the version
of Ceph that the UCA shipped with.

This will be used for generalization of upgrade workflows for all
ceph charms.

Change-Id: I3981347f3e4a0bd1b763d3da0afb65d1b7084a05
Partial-Bug: 1611082
2016-10-20 14:18:12 +01:00
Jenkins 73acc9acd6 Merge "List Ceph Pools" 2016-10-19 15:15:06 +00:00
James Page 8ab6c49c86 Downgrade default mon caps
Prior to the introduction of the ceph broker for managing pools,
consuming charms would create their own pools, requiring rw
permissions for on mons.

Drop the 'w' capability as all charms now use the broker for
creation of pools

Change-Id: Iefffe047214555a15c4201fca605f07ac39c8f5c
Partial-Bug: 1424771
2016-10-17 08:58:41 +01:00
Chris Holcombe f61addc947 List Ceph Pools
Adding a function in the ceph library to allow listing
of the currently available pools in ceph.

Change-Id: I28d684cc3041ed9f8661fb45f096b70a1095f64f
2016-10-07 16:15:40 -07:00
Chris Holcombe 4189681edb CephFS Helpers
Added a helper function to test if ceph fs new has already
been called.  If it has been than the ceph_broker will
skip calling it again.

Change-Id: Ib39542a55db5f03f08af72054c14bd6abb3d6a2a
2016-10-07 11:08:51 -07:00
James Page af4e3458af Add helper to generate mds bootstrap key
Bootstrapping MDS units into a Ceph cluster needs a special kind
of key using the bootstrap-mds profile; provide a helper to allow
the ceph-mon charm todo this easily.

Change-Id: I981aa7fa52c43eee1c6bce7f04d281d762e9b59a
2016-09-29 21:31:00 +01:00
Edward Hope-Morley 6a2df56e35 Fix radosgw relation code broken by commit b360537
Change-Id: I6b3c502316b2696dd9df0ee18eb33f34eb1d220d
Partially-Closes-Bug: 1627718
2016-09-26 14:21:36 +01:00
James Page 7766bef0b9 Fixup py3 compatibility 2016-09-20 14:49:19 +01:00
Chris Holcombe d79596ad5f Add function to create mds key 2016-09-19 15:29:22 -07:00
Chris Holcombe bc36d208d9 Install the Radosgw tools 2016-09-08 15:44:10 -07:00
Chris Holcombe 33d1d35e7a Move unit tests to shared library and remove dependence on utils code 2016-09-07 13:31:23 -07:00
Chris Holcombe 6e2431dd83 PEP8 fix 2016-09-06 15:40:11 -07:00
Chris Holcombe 7bcc9cf286 Add function to prevent osds from upgrading before the entire mon cluster is upgraded 2016-09-06 14:45:37 -07:00
Chris Holcombe 9f2d290881 Ensure we are using the correct upgrade cephx key 2016-09-06 10:13:20 -07:00
Chris Holcombe 94c4449311 merge 2016-09-02 15:53:02 -07:00
Chris Holcombe ee37e1004c Move upgrade mon and osd cluster code to here 2016-09-02 15:39:01 -07:00
Chris Holcombe 6b087f4be2 Move upgrade mon and osd cluster code to here 2016-09-02 15:34:36 -07:00
Chris Holcombe 9201440716 Adjust the admin_caps slightly to allow * for mds. The caps should now match what ceph expects 2016-08-25 09:38:01 -07:00