Patch out charmhelpers.osplatform.get_platform() and
charmhelpers.core.host.lsb_release() globally in the unit tests to
insulate the unit tests from the platform that the unit tests are being
run on.
Also, add mock for `charmhelpers.contrib.openstack.context.is_ipv6_disabled`
in the `HAProxyContextTests.test_ctxt` unit test.
The charmhelpers function `is_ipv6_disabled` calls `sysctl` and this doesn't
exist on Debian container images. This fixes the following unit test failure:
```
FileNotFoundError: [Errno 2] No such file or directory: 'sysctl'
```
Signed-off-by: Ionut Balutoiu <ibalutoiu@cloudbasesolutions.com>
Co-authored-by: Alex Kavanagh <alex.kavanagh@canonical.com>
Change-Id: I60bc9550a83eb342e78d3c1916d98bfaa8035572
The latest Ceph versions forbid pool names that start with a dot.
Since the RadosGW charm uses pools named so extensively, this
patchset fixes that issue.
In addition, the Ceph libraries are synced as well, since they
were outdated.
Change-Id: I50112480bb3669de08ee85a9bf9a594b379e9ec3
* sync charm-helpers to classic charms
* change openstack-origin/source default to quincy
* add mantic to metadata series
* align testing with bobcat
* add new bobcat bundles
* add bobcat bundles to tests.yaml
* add bobcat tests to osci.yaml
* update build-on and run-on bases
* drop kinetic
* add additional unit test https mocks needed since
charm-helpers commit 6064a34627882d1c8acf74644c48d05db67ee3b4
* update charmcraft_channel to 2.x/stable
Change-Id: I2d9c41c294668c3bb7fcba253adb8bc0c939d150
This option is required for server-side encryption to be allowed
if radosgw is behind a reverse proxy,
such as here when certificates are configured and apache2 is running.
ref. https://docs.ceph.com/en/latest/radosgw/encryption/
It is safe to always enable when https is configured in the charm,
because it will be securely behind the reverse proxy in the unit.
This option must not be enabled when https is not configured in the charm,
because this would allow clients to spoof headers.
Closes-Bug: #2021560
Change-Id: I940f9b2f424a3d98936b5f185bf8f87b71091317
A new relation with primary/secondary nomenclature is added and the
old master/slave relation is marked as *Deprecated*. In future,
master/slave relation would be completely removed.
Change-Id: I9cda48b74a20aaa9a41baedc79332bfaf13951d3
func-test-pr: https://github.com/openstack-charmers/zaza-openstack-tests/pull/926
Adds implementation for relation-departed hooks to cleanly remove
participant sites from the multisite system. The replication
between zones is stopped and both zones split up to continue as
separate master zones.
Change-Id: I420f7933db55f3004f752949b5c09b1b79774f64
func-test-pr: https://github.com/openstack-charmers/zaza-openstack-tests/pull/863
Multisite config values (realm, zonegroup, zone) are written
to ceph.conf as the defaults without verifying their existence, this
causes failure for commands which use the default values.
Closes-Bug: #1987127
Change-Id: I0ab4df34f0000339227e5d5b80352355ea7bd36e
1.) Currently multi-site can only be configured when system is being
deployed from scratch, migration works by renaming the existing
Zone/Zonegroups (Z/ZG) to Juju config values on primary site before
secondary site pulls the realm data and then rename and configure
secondary Zone accordingly.
During migration:
2.) If multiple Z/ZG not matching the config values are present at
primary site, the leader unit will block and prompt use of
'force-enable-multisite' which renames and configures selected Z/ZG
according to multisite config values.
3.) If the site being added as a secondary already contain Buckets,
the unit will block and prompt the operator to purge all such Buckets
before proceeding.
Closes-Bug: #1959837
Change-Id: I01a4c1c4551c797f0a32951dfbde8a1a4126c2d6
func-test-pr: https://github.com/openstack-charmers/zaza-openstack-tests/pull/840
This patchset adds these 2 additional keys to the ceph.conf file,
which are used in multisite configurations when present.
Change-Id: I51ca46bbb3479cb73ec4d9966208ed794f0ed774
Closes-Bug: #1975857
Ceph radosgw supports [0] the swift health check endpoint
"/swift/healthcheck". This change adds the haproxy
configuration [1] necessary to take the response of "GET
/swift/healthcheck" into account when determining the health
of a radosgw service.
For testing, I verified that:
- HAProxy starts and responds to requests normally with this
configuration.
- Servers with status != 2xx or 3xx are removed from the
backend.
- Servers that take too long to respond are also removed
from the backend. The default timeout value is 2s.
[0] https://tracker.ceph.com/issues/11682
[1] https://www.haproxy.com/documentation/hapee/2-0r1/onepage/#4.2-option%20httpchk
Closes-Bug: 1946280
Change-Id: I82634255ca3423fec3fc15c1e714dcb31db5da7a
The mock third party library was needed for mock support in py2
runtimes. Since we now only support py36 and later, we can use the
standard lib unittest.mock module instead.
Note that https://github.com/openstack/charms.openstack is used during tests
and he need `mock`, unfortunatelly it doesn't declare `mock` in its
requirements so it retrieve mock from other charm project (cross dependency).
So we depend on charms.openstack first and when
Ib1ed5b598a52375e29e247db9ab4786df5b6d142 will be merged then CI
will pass without errors.
Depends-On: Ib1ed5b598a52375e29e247db9ab4786df5b6d142
Change-Id: If352ea32d18cd3d1d8bc5577a32c0397e1cb7e93
Fix the create_system_user method so it returns the access_key
and secret when a user is created.
This patch also includes the following changes:
* Improve logging of multisite methods to help with debugging issues.
* Fix multisite relations in bundles.
Func-Test-Pr: https://github.com/openstack-charmers/zaza-openstack-tests/pull/667
Closes-Bug: #1950329
Change-Id: I0528fe7f4a89c69f2790a0e472f6f43e23c2de19
Add a radosgw-user relation to allow charms to request a user. The
requesting charm should supply the 'system-role' key in the app
relation data bag to indicate whether the requested user should
be a system user. This charm creates the user if it does not exist
or looks up the users credentials if it does. The username and
credentials are then passed back to the requestor via the
app relation data bag. The units radosgw url and daemon id
are also passed back this time using the unit relation data
bag.
Change-Id: Ieff1943b02f490559ccd245f60b744fb76a5d832
When the MonContext becomes incomplete during regular operation from,
for example, the replacement of an existing mon unit due to failure,
Ceph Radosgw shoud be able to continue while the new mon
bootstraps itself into the cluster. By ensuring that the
context can complete with one of the mons not reporting an
FSID, the remaining members of the monitor cluster can
support the continuing functioning of RadosGW.
Closes-Bug: #1938919
Change-Id: I293224f46d06cc427b2d3c8f4ae65366ed06909e
When radosgw packages are upgraded, the radosgw service needs to
be restarted by the charm. Check to see that packages were installed
on the upgrade path and if so, restart the radosgw service.
Change-Id: I61055ea4605a9a7c490c18f611d0eb583c617ce3
Closes-Bug: #1906707
Introduce support for the beast web frontend for the Ceph
RADOS Gateway which brings improvements to speed and scalability.
Default behaviour is changed in that for Octopus and later
(aside from some unsupported architectures) beast is enabled by
default; for older releases civetweb is still used.
This may be overridden using the 'http-frontend' configuration
option which accepts either 'beast' or 'civetweb' as valid
values. 'beast' is only supported with Ceph Mimic or later.
Closes-Bug: 1865396
Change-Id: Ib73e58e21219eca611cd4293da69bf80040f5803
Ceph RGW checks revocation list for every 600 seconds. This is not
required for non PKI tokens and PKI tokens are removed in OpenStack
Pike release. This results in unnecessary logs in ceph and keystone.
Set the rgw keystone revocation interval to 0 in ceph conf. Also
this parameter is removed in upstream from Ceph Octopus. So ensure
not to add this parameter from ceph release Octopus.
Closes-Bug: #1758982
Change-Id: Iaeb10dc25bb52df9dd3746ecf4fe5859d4efd459
Ceph RADOS gateway >= Mimic has an additional metadata pool (otp).
Add this to the broker request to ensure that its created correctly
by the ceph-mon application rather than being auto-created by the
radosgw application
Change-Id: I5e9b4e449bd1bc300225d223329bb62f3a381705
Closes-Bug: 1921453
When the charm config option `port` is changed,
the previously opened port is not closed.
This leads to leaks of open ports (potential security
issue), and long ports field on status after tests:
Test:
$ juju config ceph-radosgw port=1111
$ juju config ceph-radosgw port=2222
$ juju config ceph-radosgw port=3333
$ juju status ceph-radosgw
...
Unit Workload Agent Machine Public address Ports Message
ceph-radosgw/1* blocked idle 3 10.5.2.210
80/tcp,1111/tcp,2222/tcp,3333/tcp Missing relations: mon
...
$ juju run --unit ceph-radosgw/1 'opened-ports'
80/tcp
1111/tcp
2222/tcp
3333/tcp
Patched:
$ juju run --unit ceph-radosgw/1 'opened-ports'
80/tcp
1111/tcp
1234/tcp
2222/tcp
3333/tcp
33331/tcp
33332/tcp
33334/tcp
$ juju config ceph-radosgw port=33335
$ juju run --unit ceph-radosgw/1 'opened-ports'
33335/tcp
$ juju status ceph-radosgw
...
Unit Workload Agent Machine Public address Ports
Message
ceph-radosgw/1* blocked idle 3 10.5.2.210 33335/tcp
Missing relations: mon
@ unit log
2021-03-24 13:20:51 INFO juju-log Closed port 80 in favor of port 33335
2021-03-24 13:20:51 INFO juju-log Closed port 1111 in favor of port 33335
2021-03-24 13:20:51 INFO juju-log Closed port 1234 in favor of port 33335
2021-03-24 13:20:51 INFO juju-log Closed port 2222 in favor of port 33335
2021-03-24 13:20:52 INFO juju-log Closed port 3333 in favor of port 33335
2021-03-24 13:20:52 INFO juju-log Closed port 33331 in favor of port 33335
2021-03-24 13:20:52 INFO juju-log Closed port 33332 in favor of port 33335
2021-03-24 13:20:52 INFO juju-log Closed port 33334 in favor of port 33335
Signed-off-by: Mauricio Faria de Oliveira <mfo@canonical.com>
Closes-bug: #1921131
Change-Id: I5ac4b66137faffee82ae0f1e13718f21274f1f56
As a part of the 20.05 charms release, the boostrap process for
Keystone led to changing the case of the member role. This work
was performed in https://review.opendev.org/#/c/712040/ so this
change is aligning ceph-radosgw with that new default.
Change-Id: I116bb1def1b6bc8c111f30018598673da4dfdb5d
Closes-Bug: #1904183
Includes updates to charmhelpers/charms.openstack for cert_utils
and unit-get for the install hook error on Juju 2.9
* charm-helpers sync for classic charms
* rebuild for reactive charms
* ensure tox.ini is from release-tools
* ensure requirements.txt files are from release-tools
* On reactive charms:
- ensure master branch for charms.openstack
- ensure master branch for charm-helpers
* Remove mocked out unit_get as it's no longer present in charm-helpers
sync.
Change-Id: I72fc602ca3f8546da39e0da52b3144ab372b8d90
This patch adds the config option rgw-swift-versioning-enabled boolean that enables swift versioning for the ceph-backed storage solution. This uses X-Versions-Location as it is the only header that radosgw interprets.
closes-bug: #1910679
Change-Id: I5b42c34882b46e96f4cc92d91ec441a4bdfd76f6
Ensure the right key is selected on pre-systemd deploys. Whether
to request unit specific keys is already gated on the
request_per_unit_key *1 this patch applies the same logic to
_key_name for selecting the key.
*1 https://github.com/openstack/charm-ceph-radosgw/blob/master/hooks/hooks.py#L258
Also update testing to use cephx auth.
Change-Id: I92fe75fb7f483cc70b35e48587cf376a16d856a5
Closes-Bug: #1899676
Implements the swift-proxy interface. This is needed in order for
the glance (or any other) charm to be able to consume RadosGW the
same way they would consume Swift.
Change-Id: Ia59e1286ca25a71bcdf74be38c9dffb07c5be64f
Add support for use of Erasure Coded pools with the Ceph RADOS Gateway.
Only the data pool is actually Erasure Coded - all other pools continue
to be replicated but have much smaller data footprints.
Depends-On: Iec4de19f7b39f0b08158d96c5cc1561b40aefa10
Change-Id: I661639e67853ff471a7d7ddea0e3fc2fcb30fed1
Fix intermittent deployment failure with TLS.
Default to TLS in the functional test.
The call to ``configure_https`` in identity_changed is remains
from the time when Keystone provided certificates, remove it.
Hold service down until keys are rendered.
Change-Id: Ia16e6200520972c503102d80cda35e36daea82a2
Closes-Bug: #1868387
At present the charm configures the Ceph RADOS GW with the
admin_token as credentials when connecting to a deployment with
Keystone V2.0 API.
We want to move away from that and as such we need to update the
charm to configure username, password and project name instead.
Change-Id: Idab6a5740a541b922f9dbd65165d0328d747e78e
Because laeder_get only deals with stringy types, it is
unsound to evaluate them as booleans to toggle something
on or off.
Change-Id: I18c3763dce53d1d652185f9fba73523a5c5a65a6
Closes-Bug: #1847769
This change enabled automatic tenant namespacing,
which also allows enabling global read permissions
on buckets.
Change-Id: Ic37c7161b7dddad49e3c2ab075d7e8b72f436b35
Closes-Bug: #1833072
Additionally, this has unit test fixes for a CephContext
update and a Keystone V3 update that came with this sync.
Change-Id: I8ad78dbebf94ac0e6d0bcee6af2e24552c7175a3
RADOS Gateway supports setting keystone operator and admin
roles. RADOS Gateway requires admin roles for keystone users
to change their user quota. Regular operator/member roles
are not allowed to do so.
The lack of this config option prevents swift users with admin
roles from being able to set their quotas. Therefore, a config
option 'admin-roles' is now added to the charm to map to
'rgw keystone accepted admin roles' RADOS Gateway config.
Please note that this is only effective from Luminous
Ceph Release.
Change-Id: Ic0b9aa39eef9fbc6c43eb4e66ab72d90787c2017
Closes-Bug: #1831577
This change adds the fsid to the mon context
to ensure that the charm knows when the relation
is complete. Without an explicit key that only
comes from the relation, the ceph-radosgw charm
will happily go "ready" before that relation
has completed, leading to a confusing state
where the charm goes "blocked" because the
radosgw service is not running.
Change-Id: I4bfdabbd36400701debfb7a39a9c40701fc8b5ee
Currently, when the charm tears down the default radosgw daemon in
order to make way for per host daemons, it does not remove the nrpe
check for the daemon. This PR fixes the issue.
It also closes a gap where alerts for the per host daemons are not
setup until a hook that happens to call update_nrpe_checks as a
side-effect is run.
Change-Id: I7621b9671b010a77bb3e94bdd1e80f45274c73e5
Closes-Bug: #1825843
Shunde Zhang