Patch out charmhelpers.osplatform.get_platform() and
charmhelpers.core.host.lsb_release() globally in the unit tests to
insulate the unit tests from the platform that the unit tests are being
run on.
Change-Id: Ic558fa7bab80b0dabc3e5a06d7494e2688c027b3
The mock third party library was needed for mock support in py2
runtimes. Since we now only support py36 and later, we can use the
standard lib unittest.mock module instead.
Note that https://github.com/openstack/charms.openstack is used during tests
and he need `mock`, unfortunatelly it doesn't declare `mock` in its
requirements so it retrieve mock from other charm project (cross dependency).
So we depend on charms.openstack first and when
Ib1ed5b598a52375e29e247db9ab4786df5b6d142 will be merged then CI
will pass without errors.
Depends-On: Ib1ed5b598a52375e29e247db9ab4786df5b6d142
Change-Id: I8757d430ab233f4a21d1d599f1a7a9a0dc68d7ef
Port replication configs are not being set into the ring files.
When replication port configs
(account|container|object)-server-port-rep, are changed in the
swift-storage charm, swift-storage changes the related configs in
the config files, but that does not update the rings.
This patch adds a function that runs in every config-change
triggered from the swift-storage nodes and make sure any replication
config is written to the ring and distributed to all nodes.
Partial-bug: #1903762
Change-Id: I87eb23de94e3f2f5b06d44df1f8bd9d2324456a0
* charm-helpers sync for classic charms
* charms.ceph sync for ceph charms
* rebuild for reactive charms
* sync tox.ini files as needed
* sync requirements.txt files to sync to standard
Change-Id: I452af2fa5a833c435b913571fd059282bbe0a1a6
When running set-weight and remove-devices actions, they must be run
on the leader of the swift-proxy cluster. Lacking this validation,
the ring builder file on a non-leader would get updated, but the
balance_rings action would fail, and the changes to the ring files
would not propogate to the remainder of the swift cluster.
Change-Id: I9134eb88b16e5ffe18f16ce30e03e469027b6670
Closes-Bug: 1882250
Upstream removed the api_version configuration option and started
to expect the version to be encoded in the Keystone URL at Rocky.
Add S3 API functional test.
Remove invalid configuration items in the s3token section.
Remove configuration templates for OpenStack versions prior to
Mitaka.
Change-Id: Ia3306441222f7c078d460c659701a57a5944c928
Func-Test-Pr: https://github.com/openstack-charmers/zaza-openstack-tests/pull/194
Closes-Bug: #1867373
When resuming services exclude those managed by hacluster, in
this case haproxy. If pacemaker lacks quorum it may shut haproxy
down which will cause this charm to error.
Charmhelper sync included to bring in required
get_managed_services_and_ports method.
Change-Id: I85c380a2cffcd18031a32b6e3eb422aa5ff14994
swauth is no longer maintained as of OpenStack Train [1] so we
need to stop supporting it for OpenStack Train and later.
[1] https://opendev.org/x/swauth
Closes-Bug: #1851394
Change-Id: Ic2345427951a311477d6808077fb18b248036b82
This patchset adds a support for Swift Global Cluster feature as
described at:
https://docs.openstack.org/swift/latest/overview_global_cluster.html
It allows specifying affinity settings as parrt of the deployment.
Moreover, the master - slave relation is introduced for the purpose of
rings distribution across proxy nodes participating in the Swift Global
Cluster.
Change-Id: I406445493e2226aa5ae40a09c9053ac8633a46e9
Closes-Bug: 1815879
Depends-On: I11b6c7802e5bfbd61b06e4d11c65804a165781b6
Switch package install to Python 3 for OpenStack Train and later.
When upgrading, remove any python-* packages that were explicitly
installated and then autoremove --purge any dependencies that are
no longer required.
This patch also includes the following related changes:
* Use the common files package, swift, rather than python-swift
as the package name when determining releases.
* Drop the python2 shebang from manager.py in favor of specifying
the interpreter on the subprocess call. The python interpreter
version must match the python version of the OpenStack payload
due to the swift library imports.
* Enable the cPickle import in manager.py for Python 3
(C-optimized module name is _pickle), and fix 'result'
variable that is a set type but should be a dict type.
Change-Id: Ia3fdbf2020137bcf39039478ee3606717d3d6e20
Closes-Bug: #1841184
This patchset adds a template for the 'dispersion.conf' file which is
used for swift cluster health monitoring.
Change-Id: I348ded9f94f2bcb7a680b2c2280ff163cde65c46
Closes-Bug: 1328064
This patchset adds the 'set-weight' action which can be used to set
weight of device(s) based on the search pattern.
Change-Id: I6bbc71f66d948e97d2dd6a5174dedb3212503a3f
Closes-Bug: 1821697
This patchset implements 'replicas-account' and 'replicas-container'
charm options which can be used to set the desired number of replicas
per ring.
Change-Id: Ie7fa1b8c5619d0a5a278cd0eddaf1051de11f2a3
Closes-Bug: 1823696
This patchset adds the 'remove-devices' action which can be used remove
devices from rings based on search patterns.
Change-Id: I1380193b27cc4f385f156af8d63c5594dfea9cfa
Co-Authored-By: Tytus Kurek <tytus.kurek@canonical.com>
Closes-Bug: 1822058
Python 2 support has been removed from the keystone package in
Disco (Stein), therefore there is no longer a python-keystone
binary package. python-keystone doesn't appear to be used by the
charm. python-keystonemiddleware, however, is used by the charm.
This change can safely be applied from Mitaka onwards.
Change-Id: I9aa6d66761ffa4f64a8573f3b0ab1e20b9c4f774
Swift support is in-tree for Swift since OpenStack Rocky, and
the swift-plugin-s3 package has been removed from the archive at
Cosmic so drop installation at Rocky.
Add new template for Rocky to use the in-tree s3api and s3token
middleware.
Enable cosmic test to validate changes.
Change-Id: Ie5447dc44203c1ea2ad27e6d71248ba59d7549d0
Closes-Bug: 1805597
If local unit is no longer leader, clear rings_url on
storage relations to avoid storage units getting
rings from wrong proxy unit.
Also send broker-timestamp to storage units when providing
rings_url so that we have a means of knowing which is the most
recent. Broker timestamp is the same for peer and storage
sync so this enables identifying most recent.
Change-Id: I2c7e9028f345791bad0a736cb89979284b144e33
Closes-Bug: #1765203
This results in a few changes in behaviour:
1) The charm will no longer specify a nic name to bind the vip. This
is because Pacemaker VIP resources are able to automatically
detect and configure correct iface and netmask parameters based
on local configuration of the unit.
2) The original iface named VIP resource will be stopped and deleted
prior to the creation of the new short hash named VIP resource.
Change-Id: Iad743ebdb6283d72126142122ce393092cd60f18
Due to a change to PY3, the charmhelpers directory is not in the
CHARMDIR rather than in CHARMDIR/hooks. This broke the
copy_nrpe_checks() function in c-h. c-h has been updated, to look in
both CHARMDIR and CHARMDIR/hooks for the charmhelpers directory, and
this patchset includes the new function definition.
Note: the change in unit_tests/test_actions.py is to cope with a
difference between how subprocess.CalledProcessError() formats strings
when using str() between py35 and py36. Python3.6 adds a period/dot to
the end of the sentence that Python3.5 didn't.
Change-Id: I8ebb1d6d78f0804007a9f79f31b3b0c4e0c441bd
Closes-Bug: #1796830
Switch to using automatic keystone v2/v3 version detection
using new amulet charmhelpers in functional tests.
Enable xenial/queens amulet test as part of extended tests.
Change-Id: I4de220d3561bf065a3f1f64455cda22a7a807c1a
Depends-On: Ie6e2733f34de10a4d34b18dbf1fd9ba623af0e18
Depends-On: Ibcc36ca22d72d310921f840e6081608be1fbc7e1
* Needed to add a swift_manager/manager.py file which uses the payload
software python modules to perform certain functions on behalf of the
charm. These were part of the main charm, which couldn't be retained
in the charm due to the charm changing to Py3.
* Changed to absolute imports using the charm root as the root for all
charm modules.
* The py2 target in tox.ini is used to test the swift_manager/manager.py
file only.
* The .testr.conf file has been migrated to .stestr.conf
Change-Id: If37a393aa6ed27651b04810aa0bbf69eda37d7b4
By default, statsd metrics can be sent by swift-proxy
for diagnostic and monitoring purposes, but are disabled
by default. This change exposes charm config settings
that allow it to be enabled by setting 'statsd_host'
to a non-empty value. 'statsd_port' and 'statsd_sample_rate'
are also supported for changing the destination port
and rate at which metrics are collected.
Closes-Bug: #1729771
Change-Id: I2d5cca233d48022073b5975c06c3da2b1896d8d9
Ensure that any network space binding provided by the end user
is used as units join the swift-storage relation.
This allows backend communication to swift-storage units to be
network isolated from frontend public access to the swift
deployment.
Change-Id: If29ba3dfb1379f0cda20d9685e654d911d67df1d
Closes-Bug: 1697491
This reverts commit 1cf5ea71d0 which
is not required as the should_balance function already inspects
the configuration option that enforces ring balancing via actions.
Change-Id: I79c1be12f680c446d556dc16e9dfbd60a7d9db38
The Swift rings may have "holes" for devices which no longer
exist, but the code does not handle the holes in general. Holes
appear in the Swift rings as None objects in the devices list.
This change adds checks to the places in the code which are loading
the devices from the Swift ring to account for None. Generally,
this is just checking the truthy value of a device.
However, this change also removes the next device id calculation
from the add_to_ring function, deferring the device id selection
to the Swift RingBuilder. Upon examining the Swift RingBuilder
code, it was seen that the RingBuilder will automatically calculate
a device id for a new device when no id is specified. The Swift
algorithm considers factors other than a hole in the ring (e.g.
out of order devices) which were missing from the charm's code.
Change-Id: Ibb0908338ac958ebf1adf17c12f9484f6963c695
Closes-Bug: #1708310
Co-Authored-By: Drew Freiberger <drew.freiberger@canonical.com>"
Use the get_relation_ip function for selecting addresses for the
cluster relationship. Including overrides for the admin, internal,
and public config settings or extra bindings.
Change-Id: I33de35055ec11be01988c36e69f5d48b10bf7390
Partial-Bug: #1687439
Some cluster relation settings are dependant on
config so ensure that if confg changes, those
changes are reflected on the cluster relation.
Change-Id: I0bf8601bc7d3c769a59c4eafd89643811b46dbe6
Closes-Bug: 1641870
When adding new storage capacity, its desirable to disable ring
rebalancing until all new storage has been added, allowing the
end-user to determing when all new capacity has been added and its
OK to rebalance the rings and re-distribute.
Ensure that storage hook events from swift-storage observe the
'disable-ring-rebalance' configuration option, enabling end users
to perform this type of orchestration storage expansion.
Change-Id: I95727e663b369d5feb28147b19edcc6cab36b905
Closes-Bug: 1638981
No need for refresh of proxy-server.conf template for Mitaka. Update
template for Kilo and later to make use of domain_name and project_name
parameters instead of domain_id and project_id parameters.
The current template sets up auth to user in default domain
but project in service domain. This does not work with service
domain layout.
Do not request configured operator_roles roles from Keystone. From
which roles swift-proxy should accept requests are still configured
in proxy-server.conf, but requesting and setting up these roles for
the s3_swift user in Keystone is incorrect behaviour.
Register required relation data for identity-service immediatelly when
relation to 'identity-service' exists. Do not postpone registration
until context is complete which may cause the swift-proxy unit marking
itself ready while still being in a unconfigured state.
Add tests to verify configuration and operation of swift-proxy when
using Keystone v3 auth.
Change-Id: I8bf182a9256f96af50e4cc37505d9c0ca3d62e47
Closes-Bug: 1646765
This change implements the alternative authentication system,
swauth in addition to adding an action to add users to swauth
Change-Id: Ib752cd3a2a58f6c8cb06119c6be595cfc07ddc9f
Juju 2.0 provides support for display of the version of
an application deployed by a charm in juju status.
Insert the os_application_version_set function into the
existing assess_status function - this gets called after
all hook executions, and periodically after that, so any
changes in package versions due to normal system updates
will also be reflected in the status output.
This review also includes a resync of charm-helpers to
pickup hookenv and contrib.openstack support for this
feature.
Change-Id: I81013c7ced18a5d21dc151b023c0656f7300f58f
Swift proxy is currently rejecting valid v3 tokens because it is
failing to validate them due to its credentials being in the v2
format and missing domain information. This change examines the
version of the API keystone has advertised down the identity-service
relation and configures the proxy conf appropriately
Change-Id: Id2215168ffbad1caf0e7203ded26c41913181306
Closes-Bug: 1624304
This change registers the configured S3 proxy with Keystone, so that
your cloud knows about the S3 endpoints.
Also includes an update to ensure that the s3token middleware
authenticates against the correct keystone endpoint.
Change-Id: I07d25df6332028a99e0bf79b39f998f84613a4fc
All contributions to this charm where made under Canonical
copyright; switch to Apache-2.0 license as agreed so we
can move forward with official project status.
Change-Id: I78e8d98d51c7ff28baf1c8421d093fbefd65ae7d
Implement DNS high availability. Pass the correct information to
hacluster to register a DNS entry with MAAS 2.0 or greater rather
than using a virtual IP.
Charm-helpers sync to bring in DNS HA helpers
Change-Id: I3e356a85c0893171ac8db26300e0530054fc93bd
Juju 2.0 renames the environment variable JUJU_ENV_UUID
to JUJU_MODEL_UUID; use this environment variable as a
fallback if JUJU_ENV_UUID is not set to support Juju 2.0,
whilst continuing to provide support for Juju < 2.0.
Change-Id: I26ae366c84d3cd9a9c2471385effd2a6202e9314
Closes-Bug: 1572575
The existing pause/resume has been enhanced by adding more robust service
checks and hooking into the pause/resume functionality in the charmhelpers
library.
Change-Id: Ia487499ec4d8e4f41ec985eac02d97e085a06e2c
Make the ring sync code clearer and improve logic around leader
switching during or after a sync. Also add more debug logs to
make it easier to debug when things go wrong.
Closes-Bug: 1448884
Change-Id: I10d51c74001710b6b7a2502e14370996b15ffb40
Add charmhelpers.contrib.hardening and calls to install,
config-changed, upgrade-charm and update-status hooks.
Also add new config option to allow one or more hardening
modules to be applied at runtime.
Change-Id: I7e6abb16d16d08575cad4f3a52b6fc54b3ac01c2