Commit Graph

92 Commits

Author SHA1 Message Date
Alex Kavanagh 89d7e78998 Improve platform mocking
Patch out charmhelpers.osplatform.get_platform() and
charmhelpers.core.host.lsb_release() globally in the unit tests to
insulate the unit tests from the platform that the unit tests are being
run on.

Change-Id: Ic558fa7bab80b0dabc3e5a06d7494e2688c027b3
2023-10-24 16:15:39 +01:00
jneo8 daef40a837 Add new version template: yoga
Closes-Bug: #1960866
Change-Id: I90769c16ca05c725c212255da5cb04a5e36bdcda
2022-04-21 15:17:08 +08:00
Hervé Beraud 50417cdb32 Use unittest.mock instead of mock
The mock third party library was needed for mock support in py2
runtimes. Since we now only support py36 and later, we can use the
standard lib unittest.mock module instead.

Note that https://github.com/openstack/charms.openstack is used during tests
and he need `mock`, unfortunatelly it doesn't declare `mock` in its
requirements so it retrieve mock from other charm project (cross dependency).
So we depend on charms.openstack first and when
Ib1ed5b598a52375e29e247db9ab4786df5b6d142 will be merged then CI
will pass without errors.

Depends-On: Ib1ed5b598a52375e29e247db9ab4786df5b6d142
Change-Id: I8757d430ab233f4a21d1d599f1a7a9a0dc68d7ef
2021-12-15 11:45:01 +00:00
Bartlomiej Poniecki-Klotz d92cf5756e Fixed removing multiple devices
Added a confirmation flag for swift-ring-builder command when multiple devices are found to be removed.

func-test-pr: https://github.com/openstack-charmers/zaza-openstack-tests/pull/656
Closes-Bug: #1937289
Change-Id: Ieeb326e7d9af21a77954e95e909d6fcf20a3092c
2021-11-22 14:13:07 +01:00
Erlon R. Cruz 498d3b0f1e Fix swift port replication configuration
Port replication configs are not being set into the ring files.
When replication port configs
(account|container|object)-server-port-rep, are changed in the
swift-storage charm, swift-storage changes the related configs in
the config files, but that does not update the rings.

This patch adds a function that runs in every config-change
triggered from the swift-storage nodes and make sure any replication
config is written to the ring and distributed to all nodes.

Partial-bug: #1903762
Change-Id: I87eb23de94e3f2f5b06d44df1f8bd9d2324456a0
2021-01-26 15:53:08 -03:00
Alex Kavanagh e869f021cb Sync libraries & common files prior to freeze
* charm-helpers sync for classic charms
* charms.ceph sync for ceph charms
* rebuild for reactive charms
* sync tox.ini files as needed
* sync requirements.txt files to sync to standard

Change-Id: I452af2fa5a833c435b913571fd059282bbe0a1a6
2020-09-27 19:26:04 +01:00
Drew Freiberger 0ce1ee67f8 set-weight/remove-devices actions are leader-only
When running set-weight and remove-devices actions, they must be run
on the leader of the swift-proxy cluster.  Lacking this validation,
the ring builder file on a non-leader would get updated, but the
balance_rings action would fail, and the changes to the ring files
would not propogate to the remainder of the swift cluster.

Change-Id: I9134eb88b16e5ffe18f16ce30e03e469027b6670
Closes-Bug: 1882250
2020-06-10 08:03:37 -05:00
Frode Nordahl ed9051a9f3
s3token: Fix Keystone API configuration
Upstream removed the api_version configuration option and started
to expect the version to be encoded in the Keystone URL at Rocky.

Add S3 API functional test.

Remove invalid configuration items in the s3token section.

Remove configuration templates for OpenStack versions prior to
Mitaka.

Change-Id: Ia3306441222f7c078d460c659701a57a5944c928
Func-Test-Pr: https://github.com/openstack-charmers/zaza-openstack-tests/pull/194
Closes-Bug: #1867373
2020-03-16 12:26:52 +01:00
Liam Young ab8b60a21c When resuming, exclude haproxy
When resuming services exclude those managed by hacluster, in
this case haproxy. If pacemaker lacks quorum it may shut haproxy
down which will cause this charm to error.

Charmhelper sync included to bring in required
get_managed_services_and_ports method.

Change-Id: I85c380a2cffcd18031a32b6e3eb422aa5ff14994
2020-01-27 19:19:07 +00:00
Corey Bryant 1b47612f87 Drop support of swauth for Train and later
swauth is no longer maintained as of OpenStack Train [1] so we
need to stop supporting it for OpenStack Train and later.

[1] https://opendev.org/x/swauth

Closes-Bug: #1851394
Change-Id: Ic2345427951a311477d6808077fb18b248036b82
2020-01-07 15:03:08 -05:00
Tytus Kurek 44df5db97d Swift Global Cluster
This patchset adds a support for Swift Global Cluster feature as
described at:

https://docs.openstack.org/swift/latest/overview_global_cluster.html

It allows specifying affinity settings as parrt of the deployment.
Moreover, the master - slave relation is introduced for the purpose of
rings distribution across proxy nodes participating in the Swift Global
Cluster.

Change-Id: I406445493e2226aa5ae40a09c9053ac8633a46e9
Closes-Bug: 1815879
Depends-On: I11b6c7802e5bfbd61b06e4d11c65804a165781b6
2019-12-16 14:51:56 +00:00
Corey Bryant 5f3735f884 py3: Switch packages to Python 3 for train and later
Switch package install to Python 3 for OpenStack Train and later.

When upgrading, remove any python-* packages that were explicitly
installated and then autoremove --purge any dependencies that are
no longer required.

This patch also includes the following related changes:
  * Use the common files package, swift, rather than python-swift
    as the package name when determining releases.
  * Drop the python2 shebang from manager.py in favor of specifying
    the interpreter on the subprocess call. The python interpreter
    version must match the python version of the OpenStack payload
    due to the swift library imports.
  * Enable the cPickle import in manager.py for Python 3
    (C-optimized module name is _pickle), and fix 'result'
    variable that is a set type but should be a dict type.

Change-Id: Ia3fdbf2020137bcf39039478ee3606717d3d6e20
Closes-Bug: #1841184
2019-10-10 06:51:20 +00:00
Tytus Kurek f9a02a3f58 Dispersion report
This patchset adds a template for the 'dispersion.conf' file which is
used for swift cluster health monitoring.

Change-Id: I348ded9f94f2bcb7a680b2c2280ff163cde65c46
Closes-Bug: 1328064
2019-07-01 09:24:10 +00:00
Tytus Kurek 3809df73fb Add 'set-weight' action.
This patchset adds the 'set-weight' action which can be used to set
weight of device(s) based on the search pattern.

Change-Id: I6bbc71f66d948e97d2dd6a5174dedb3212503a3f
Closes-Bug: 1821697
2019-06-18 10:43:44 +00:00
Zuul f2ba19bce3 Merge "Add 'remove-devices' action." 2019-05-22 10:49:49 +00:00
Edward Hope-Morley cc280d9dcb Add S3 SigV4 location support
Change-Id: I2ee8cb968ae2cf34558546b1be6be5273c2f52d3
Closes-Bug: #1822322
2019-05-03 11:16:25 +01:00
Tytus Kurek 1160493f3f Replicas per ring
This patchset implements 'replicas-account' and 'replicas-container'
charm options which can be used to set the desired number of replicas
per ring.

Change-Id: Ie7fa1b8c5619d0a5a278cd0eddaf1051de11f2a3
Closes-Bug: 1823696
2019-04-29 07:16:05 +00:00
Levente Tamas 20b420ac69 Add 'remove-devices' action.
This patchset adds the 'remove-devices' action which can be used remove
devices from rings based on search patterns.

Change-Id: I1380193b27cc4f385f156af8d63c5594dfea9cfa
Co-Authored-By: Tytus Kurek <tytus.kurek@canonical.com>
Closes-Bug: 1822058
2019-04-12 13:04:58 +02:00
Corey Bryant 354030638d Drop python-keystone and add python-keystonemiddleware
Python 2 support has been removed from the keystone package in
Disco (Stein), therefore there is no longer a python-keystone
binary package. python-keystone doesn't appear to be used by the
charm. python-keystonemiddleware, however, is used by the charm.

This change can safely be applied from Mitaka onwards.

Change-Id: I9aa6d66761ffa4f64a8573f3b0ab1e20b9c4f774
2019-02-26 15:43:55 +01:00
James Page 8cf9dd4e1a Support deploy of Swift with internal S3 support
Swift support is in-tree for Swift since OpenStack Rocky, and
the swift-plugin-s3 package has been removed from the archive at
Cosmic so drop installation at Rocky.

Add new template for Rocky to use the in-tree s3api and s3token
middleware.

Enable cosmic test to validate changes.

Change-Id: Ie5447dc44203c1ea2ad27e6d71248ba59d7549d0
Closes-Bug: 1805597
2018-12-18 10:12:19 +00:00
Edward Hope-Morley f25d2c2d7f Cleanup ring manager storage relation settings
If local unit is no longer leader, clear rings_url on
storage relations to avoid storage units getting
rings from wrong proxy unit.

Also send broker-timestamp to storage units when providing
rings_url so that we have a means of knowing which is the most
recent. Broker timestamp is the same for peer and storage
sync so this enables identifying most recent.

Change-Id: I2c7e9028f345791bad0a736cb89979284b144e33
Closes-Bug: #1765203
2018-12-07 11:42:09 +00:00
Liam Young cff08e7b98 Use chelper generate_ha_relation_data for ha rel
This results in a few changes in behaviour:

1) The charm will no longer specify a nic name to bind the vip. This
   is because Pacemaker VIP resources are able to automatically
   detect and configure correct iface and netmask parameters based
   on local configuration of the unit.
2) The original iface named VIP resource will be stopped and deleted
   prior to the creation of the new short hash named VIP resource.

Change-Id: Iad743ebdb6283d72126142122ce393092cd60f18
2018-12-05 15:34:37 +00:00
Alex Kavanagh 96741629b4 Sync c-h nrpe.py file to fix nrpe files copy issue
Due to a change to PY3, the charmhelpers directory is not in the
CHARMDIR rather than in CHARMDIR/hooks. This broke the
copy_nrpe_checks() function in c-h. c-h has been updated, to look in
both CHARMDIR and CHARMDIR/hooks for the charmhelpers directory, and
this patchset includes the new function definition.

Note: the change in unit_tests/test_actions.py is to cope with a
difference between how subprocess.CalledProcessError() formats strings
when using str() between py35 and py36.  Python3.6 adds a period/dot to
the end of the sentence that Python3.5 didn't.

Change-Id: I8ebb1d6d78f0804007a9f79f31b3b0c4e0c441bd
Closes-Bug: #1796830
2018-11-26 11:15:56 +00:00
James Page 59971e1c7d Enable xenial/queens amulet tests
Switch to using automatic keystone v2/v3 version detection
using new amulet charmhelpers in functional tests.

Enable xenial/queens amulet test as part of extended tests.

Change-Id: I4de220d3561bf065a3f1f64455cda22a7a807c1a
Depends-On: Ie6e2733f34de10a4d34b18dbf1fd9ba623af0e18
Depends-On: Ibcc36ca22d72d310921f840e6081608be1fbc7e1
2018-03-20 09:54:11 +00:00
Alex Kavanagh 4336b8d644 Convert charm to Python 3 only
* Needed to add a swift_manager/manager.py file which uses the payload
  software python modules to perform certain functions on behalf of the
  charm.  These were part of the main charm, which couldn't be retained
  in the charm due to the charm changing to Py3.
* Changed to absolute imports using the charm root as the root for all
  charm modules.
* The py2 target in tox.ini is used to test the swift_manager/manager.py
  file only.
* The .testr.conf file has been migrated to .stestr.conf

Change-Id: If37a393aa6ed27651b04810aa0bbf69eda37d7b4
2017-12-06 14:20:25 +00:00
Zuul bf8ebc3fa0 Merge "Add an action to report utilization" 2017-11-08 05:56:36 +00:00
James Hebden a11ff79fc7 Add statsd logging support to the swift-proxy charm
By default, statsd metrics can be sent by swift-proxy
for diagnostic and monitoring purposes, but are disabled
by default. This change exposes charm config settings
that allow it to be enabled by setting 'statsd_host'
to a non-empty value. 'statsd_port' and 'statsd_sample_rate'
are also supported for changing the destination port
and rate at which metrics are collected.

Closes-Bug: #1729771

Change-Id: I2d5cca233d48022073b5975c06c3da2b1896d8d9
2017-11-05 18:03:54 +11:00
Jill Rouleau dcec924d47 Add an action to report utilization
Using swift-recon -d, report disk-utilization in GB.

Change-Id: I02f9b85106046fc038a66c7059c23c8a89fd32b1
Closes-Bug: 1720258
2017-11-03 15:15:36 -07:00
James Page 27c669a657 Enable network-spaces for swift-storage relation
Ensure that any network space binding provided by the end user
is used as units join the swift-storage relation.

This allows backend communication to swift-storage units to be
network isolated from frontend public access to the swift
deployment.

Change-Id: If29ba3dfb1379f0cda20d9685e654d911d67df1d
Closes-Bug: 1697491
2017-11-02 09:55:47 +00:00
Jenkins 727aff428b Merge "Handle holes in swift rings" 2017-09-18 13:35:03 +00:00
James Page 655d5be16a Revert "Ensure storage hooks observe disable-ring-rebalance"
This reverts commit 1cf5ea71d0 which
is not required as the should_balance function already inspects
the configuration option that enforces ring balancing via actions.

Change-Id: I79c1be12f680c446d556dc16e9dfbd60a7d9db38
2017-09-12 16:09:02 -06:00
Billy Olsen ac553419f4 Handle holes in swift rings
The Swift rings may have "holes" for devices which no longer
exist, but the code does not handle the holes in general. Holes
appear in the Swift rings as None objects in the devices list.

This change adds checks to the places in the code which are loading
the devices from the Swift ring to account for None. Generally,
this is just checking the truthy value of a device.

However, this change also removes the next device id calculation
from the add_to_ring function, deferring the device id selection
to the Swift RingBuilder. Upon examining the Swift RingBuilder
code, it was seen that the RingBuilder will automatically calculate
a device id for a new device when no id is specified. The Swift
algorithm considers factors other than a hole in the ring (e.g.
out of order devices) which were missing from the charm's code.

Change-Id: Ibb0908338ac958ebf1adf17c12f9484f6963c695
Closes-Bug: #1708310
Co-Authored-By: Drew Freiberger <drew.freiberger@canonical.com>"
2017-08-25 16:00:53 -07:00
David Ames 92c0ec5ff3 Network space aware address for cluster relation
Use the get_relation_ip function for selecting addresses for the
cluster relationship. Including overrides for the admin, internal,
and public config settings or extra bindings.

Change-Id: I33de35055ec11be01988c36e69f5d48b10bf7390
Partial-Bug: #1687439
2017-05-05 10:44:30 -07:00
Edward Hope-Morley f001baf944 Ensure cluster relation updated on config-changed
Some cluster relation settings are dependant on
config so ensure that if confg changes, those
changes are reflected on the cluster relation.

Change-Id: I0bf8601bc7d3c769a59c4eafd89643811b46dbe6
Closes-Bug: 1641870
2017-04-12 12:39:15 +01:00
James Page 1cf5ea71d0 Ensure storage hooks observe disable-ring-rebalance
When adding new storage capacity, its desirable to disable ring
rebalancing until all new storage has been added, allowing the
end-user to determing when all new capacity has been added and its
OK to rebalance the rings and re-distribute.

Ensure that storage hook events from swift-storage observe the
'disable-ring-rebalance' configuration option, enabling end users
to perform this type of orchestration storage expansion.

Change-Id: I95727e663b369d5feb28147b19edcc6cab36b905
Closes-Bug: 1638981
2017-02-02 11:01:13 +00:00
Frode Nordahl 7c24ae8128 Fix Keystone v3 auth for swift-proxy
No need for refresh of proxy-server.conf template for Mitaka. Update
template for Kilo and later to make use of domain_name and project_name
parameters instead of domain_id and project_id parameters.

The current template sets up auth to user in default domain
but project in service domain. This does not work with service
domain layout.

Do not request configured operator_roles roles from Keystone. From
which roles swift-proxy should accept requests are still configured
in proxy-server.conf, but requesting and setting up these roles for
the s3_swift user in Keystone is incorrect behaviour.

Register required relation data for identity-service immediatelly when
relation to 'identity-service' exists. Do not postpone registration
until context is complete which may cause the swift-proxy unit marking
itself ready while still being in a unconfigured state.

Add tests to verify configuration and operation of swift-proxy when
using Keystone v3 auth.

Change-Id: I8bf182a9256f96af50e4cc37505d9c0ca3d62e47
Closes-Bug: 1646765
2016-12-08 07:17:26 +01:00
Chris MacNaughton ce3f15310e Implement swauth
This change implements the alternative authentication system,
swauth in addition to adding an action to add users to swauth

Change-Id: Ib752cd3a2a58f6c8cb06119c6be595cfc07ddc9f
2016-09-23 09:12:43 -04:00
James Page 2a3b0857b6 Add support for application version
Juju 2.0 provides support for display of the version of
an application deployed by a charm in juju status.

Insert the os_application_version_set function into the
existing assess_status function - this gets called after
all hook executions, and periodically after that, so any
changes in package versions due to normal system updates
will also be reflected in the status output.

This review also includes a resync of charm-helpers to
pickup hookenv and contrib.openstack support for this
feature.

Change-Id: I81013c7ced18a5d21dc151b023c0656f7300f58f
2016-09-20 12:45:17 +01:00
Liam Young 5b8b04a213 Fix support for Keystone v3 API
Swift proxy is currently rejecting valid v3 tokens because it is
failing to validate them due to its credentials being in the v2
format and missing domain information. This change examines the
version of the API keystone has advertised down the identity-service
relation and configures the proxy conf appropriately

Change-Id: Id2215168ffbad1caf0e7203ded26c41913181306
Closes-Bug: 1624304
2016-09-16 13:58:26 +00:00
Chris MacNaughton 25a2ea7fdd Fix S3 support, register endpoint in keystone
This change registers the configured S3 proxy with Keystone, so that
your cloud knows about the S3 endpoints.

Also includes an update to ensure that the s3token middleware
authenticates against the correct keystone endpoint.

Change-Id: I07d25df6332028a99e0bf79b39f998f84613a4fc
2016-07-14 15:44:02 +01:00
James Page f48ecdbe0d Re-license charm as Apache-2.0
All contributions to this charm where made under Canonical
copyright; switch to Apache-2.0 license as agreed so we
can move forward with official project status.

Change-Id: I78e8d98d51c7ff28baf1c8421d093fbefd65ae7d
2016-07-01 18:12:11 +01:00
David Ames 570ef17e3d DNS HA
Implement DNS high availability. Pass the correct information to
hacluster to register a DNS entry with MAAS 2.0 or greater rather
than using a virtual IP.

Charm-helpers sync to bring in DNS HA helpers

Change-Id: I3e356a85c0893171ac8db26300e0530054fc93bd
2016-06-15 12:03:18 -07:00
James Page 0fa129873b Use JUJU_MODEL_UUID for Juju 2.0
Juju 2.0 renames the environment variable JUJU_ENV_UUID
to JUJU_MODEL_UUID; use this environment variable as a
fallback if JUJU_ENV_UUID is not set to support Juju 2.0,
whilst continuing to provide support for Juju < 2.0.

Change-Id: I26ae366c84d3cd9a9c2471385effd2a6202e9314
Closes-Bug: 1572575
2016-05-23 10:08:57 +01:00
Alex Kavanagh d009a254f1 Enhanced pause/resume for maintenance mode
The existing pause/resume has been enhanced by adding more robust service
checks and hooking into the pause/resume functionality in the charmhelpers
library.

Change-Id: Ia487499ec4d8e4f41ec985eac02d97e085a06e2c
2016-04-11 16:28:51 +00:00
Edward Hope-Morley ccaeae4706 Refactor of ring management code
Make the ring sync code clearer and improve logic around leader
switching during or after a sync. Also add more debug logs to
make it easier to debug when things go wrong.

Closes-Bug: 1448884
Change-Id: I10d51c74001710b6b7a2502e14370996b15ffb40
2016-03-30 10:40:43 +01:00
Edward Hope-Morley 74b951b517 Add hardening support
Add charmhelpers.contrib.hardening and calls to install,
config-changed, upgrade-charm and update-status hooks.
Also add new config option to allow one or more hardening
modules to be applied at runtime.

Change-Id: I7e6abb16d16d08575cad4f3a52b6fc54b3ac01c2
2016-03-24 11:36:26 +00:00
Edward Hope-Morley f8dd5395c3 remove unit from relation_set 2015-11-27 09:48:45 +00:00
Edward Hope-Morley dd4a7ac0fb [hopem,r=]
Add support for settting swift-storage unit rsyncd acls.
Partially-Closes-Bug: 1427361
2015-11-24 13:35:26 +00:00
James Page dbdfbb0964 Add tox support, tidy unit tests 2015-11-03 13:52:37 +00:00
Corey Bryant 0cb10f1247 [thedac,r=corey.bryant] Enable multiple devices per node 2015-10-13 21:29:27 +00:00