Commit Graph

199 Commits

Author SHA1 Message Date
Brett Milford c6b0d798b7 Add vm.watermark_scale_factor tuning
Depends: https://github.com/juju/charm-helpers/pull/609
Implements: spec memory-fragmentation-tuning
Change-Id: I0195d5f65f36442abf1355dd5150d48a37184e97
2022-09-02 08:53:06 +10:00
Edin Sarajlic 9feb5e5596 Monitor Swift Object/Container/Account API availability
As per the bug report, it's not enough to simply monitor that the
appropriate process is alive; there have been instances of the process
being alive but the port/API being unavailable.

This patch adds monitoring for Object/Container/Account API
availability.

I've tested the fix in my small test environment and I can confirm
it's working.

For reference, the following branch/commit was used as a functional
test (later rejected with the aim of moving the checks to Mojo):
https://github.com/openstack-charmers/zaza-openstack-tests/pull/395

Change-Id: I60c5b74279f71ca8f8bc769c93af2eab1f59e002
Closes-Bug: #1854299
2020-08-28 10:13:01 +10:00
Drew Freiberger 74daa465d6 Add monitoring for object-replicator logs disappearing
As noted in lp#1691570, there is an issue with storage I/O during
coro-thread cleanup that affects the swift-object-replicator's ability
to complete replication successfully.  This is most easily witnessed by
the lack of the every 5 minute replicated percentage complete messages
that should come from the swift-object-replicator daemon to syslog.
This patch monitors for and alerts on the condition of the "replicated"
line missing from syslog within the past 15 minutes.

Change-Id: Ieb15da3f3f67fa9bcad03151e36c70faae4c36c9
Closes-Bug: 1691570
2020-07-21 15:31:06 -05:00
Alex Kavanagh 91394ea9d6 Ensure python3-hvac is installed for charms with encypt option
The referenced bug is essentially: make vault:secrets relation to vault
but keep the 'encrypt' option as False. In this case, the Context
handling code in charm-helpers is expecting python3-hvac to be
available, but it is only installed if the encrypt option is set to
True.  Hence the charm crashes.  This resolves that crash.

Note the related charm-helpers fix [1].

[1]: https://github.com/juju/charm-helpers/pull/431

Change-Id: I92773b7c1f48d456091062751e69581fabe4c5f3
Closes-bug: #1862085
2020-02-17 12:18:24 +00:00
Tytus Kurek fc261a5adc Swift Global Cluster
This patchset adds a support for Swift Global Cluster feature as
described at:

https://docs.openstack.org/swift/latest/overview_global_cluster.html

It allows specifying 'region' config option as part of the deployment.
Theregion is reflected later on in the Swift rings.

Change-Id: I11b6c7802e5bfbd61b06e4d11c65804a165781b6
Partial-Bug: 1815879
Needed-By: I406445493e2226aa5ae40a09c9053ac8633a46e9
2019-12-17 10:08:13 +00:00
Edward Hope-Morley 60dd2f0189 Charmhelpers sync to get vaultlocker fixes
Also gate checking vault context completing on whether
dependencies are installed.

Change-Id: Ib424abe608081da21207db262fb82362f23fe6ca
Closes-Bug: #1849323
2019-12-11 14:30:27 +00:00
Corey Bryant fb6e21b51f py3: Switch packages to Python 3 for train and later
Switch package install to Python 3 for OpenStack Train and later.

When upgrading, remove any python-* packages that were explicitly
installated and then autoremove --purge any dependencies that are
no longer required.

This patch also includes the following related changes:
  * Use the common files package, swift, rather than python-swift
    when using a package name to determine release.

Also add OS_* to tox.ini to allow functional tests to execute.

Change-Id: I121af845bf11c22052479a497b196a4670021256
2019-10-14 08:29:59 +00:00
Ryan Beisner 30466da23d
Update pre-install hooks to fail on error
The pre-install operations may fail, yet that failure is not
elevated to the user. This masks the failure and makes early
package install issues difficult to troubleshoot.

If the basic pre-install script fails, the charm should not
proceed to later hooks as the requirements may not be met.

Hashbangs for bash should specify -e (errexit) on all of the
pre-install bash scripts.

Change-Id: Ie2a9e473e35eb8067db2890c62b4af9adce3ed43
Closes-bug: #1815243
Partial-bug: #1815231
2019-02-08 15:41:29 -06:00
Zuul c4897eff53 Merge "Catch exception of ring sync fails" 2018-12-12 09:43:26 +00:00
Zuul fc6f6a93c0 Merge "Replace deprecated "decodestring()" by "decodebytes()"" 2018-12-11 14:10:46 +00:00
Edward Hope-Morley ae6826734f Catch exception of ring sync fails
It is possible for swift-storage units to attempt
to request rings from a proxy unit that is no longer
serving them so instead of raising an exception we
catch it and move on since there will likely be a
another proxy notification waiting to be consumed.

Change-Id: Ib2e634d2ed3509bfe2aa9b792cc17c2ed89029f1
Closes-Bug: #1765203
2018-12-06 14:35:00 +00:00
Zuul 29b030f2ac Merge "Drop use of vaultlocker PPA" 2018-12-03 12:48:32 +00:00
Liam Young 3066151e77 Install prerequisite packages on charm upgrade.
When upgrading the charm some package dependancies are missing
such as python3-distutils. This patch uses the same approach as the
install hook to install packages before calling the the main python
hook code.

Change-Id: If3bfa6efd5a3924f3ca6d1309f82c3867cfd4c67
Closes-Bug: #1803418
2018-11-14 19:27:57 +00:00
James Page 9dfe75d82e Drop use of vaultlocker PPA
vaultlocker is now provided in the UCA and in backports for bionic.

Change-Id: Ia2b4cfe75a152d4d8e1de7cda076b692ebd9d398
Closes-Bug: 1802905
2018-11-13 12:02:32 +00:00
Vu Cong Tuan c5b72675b0 Replace deprecated "decodestring()" by "decodebytes()"
decodestring() is deprecated alias of decodebytes()
https://docs.python.org/3/library/base64.html#base64.decodestring

The same has been done for nova:
https://review.openstack.org/#/c/610401/

Change-Id: I81d33a166602fbcf927ea62787db019e380b52aa
2018-11-02 16:10:40 +07:00
Alex Kavanagh fc81200315 Upgrade the charm to py3 runtime
Change-Id: I98f4e6664080407a045ca5e76db59d46ffa9c38a
2018-10-24 14:57:05 +01:00
David Ames 9f33b8135e Series Upgrade
Implement the series-upgrade feature allowing to move between Ubuntu
series.

Change-Id: Ib5f234c219437c14b54ebe248c9598492542fd23
2018-10-13 17:29:48 +00:00
James Page 591c084532 Add support for block device encryption
Add new secrets-storage relation to vault, supporting the
use of block device encryption using dm-crypt/vaultlocker.

Prepared devices are now recorded in the local unit kv
store; this information is used to provide a list of
configured block devices to the swift-proxy charm,
rather than the previous best guess provided by
determine_block_devices.  This allows us to use the
dm-crypt device name, rather than the underlying
block device.

Encrypted block devices are unlocked on boot using
vaultlocker-decrypt systemd units (enabled by vaultlocker);
/etc/fstab entries for such devices make use of a
x-systemd.requires option to ensure that the block device
is unlocked prior to attempting to mount it.

Add new storage binding to allow charm to be used with
Juju storage.

Add new ephemeral-unmount configuration option to allow
cloud ephemeral storage to be used for testing purposes;
update functional testing to use this option.  The behaviour
of 'overwrite' was changed to accomodate the use of
encrypted block devices.

Change-Id: I9b3f8cd2de412ee96e0139dba4d4abdf998ecaf2
2018-05-08 12:52:37 +01:00
David Ames 18d0a891db Allow GRE traffic in converged architecture
In a converged architecture with storage and compute on the same
host, UFW can get in the way of tunneled traffic interpreting it as
INVALID. UFW makes solving this more difficult than it needs to be.
See http://northernmost.org/blog/gre-tunnels-and-ufw/index.html for
context.

This change updates /etc/ufw/before.rules to add GRE as an allowed
input.

Also, guarantee ufw is installed for LP #1763716

Please review and merge charm-helpers first:
https://github.com/juju/charm-helpers/pull/170

Change-Id: I789854c33e3af12f7412633dbf7c921beb0ed2b5
Closes-Bug: #1757564
Closes-Bug: #1763716
2018-05-03 21:42:46 +02:00
David Ames 6c549d8018 Set ufw default policy for routed and outgoing
The charm neglected to set default policy for routed which cases
applications deployed to containers to fail to deploy.

This change explicitly sets default policy allow for routed and
outgoing.

Closes-Bug: #1747032

Change-Id: I440e5e040ad433bf0cf6c4ce99439da456476914
2018-02-02 08:57:30 -08:00
David Ames 5368af6302 Swift storage ACLs
Ensure that only the swift-proxy units and swift-storage peers have
access to direct communication with swift storage daemons.

Charm-helpers sync to include ufw module and the ingress_address and
iter_units_for_relation_name functions.

Please review and merge first:
https://github.com/juju/charm-helpers/pull/35

Closes-Bug: #1727463

Change-Id: Id5677edbc40b0b891cbe66867d39d076a94c5436
2017-11-07 10:24:53 -08:00
James Page b32f68af0f Add network-space support for swift-storage relation
Resolve correct private-address for use on the swift-storage
relation, supporting 'prefer-ipv6' as well as Juju 2.0 network
spaces.

Change-Id: I3ee111c6abdd028c2c29e80dceb99178443da45a
Closes-Bug: 1697491
2017-09-29 11:33:48 +01:00
Chris MacNaughton 58c92e15b1 Add config for sysctl
Allows the administrator to set custom sysctls on the storage
nodes. This is especially useful for setting kernel.pid_max.

Change-Id: Id902f799796a121578eba10de22e0eb9cc627b9d
Closes-bug: 1396613
2017-02-02 07:56:53 -05:00
Chris MacNaughton 47a4ac6e48 Remove /srv/node from updatedb
This stops updatedb from indexing the storage locations

Change-Id: I6ca6b8667fb06d3b52cedd151531fc0033cf2526
Closes-bug: 1520226
2017-01-25 09:21:31 -05:00
James Page cae0a2c4f5 Add support for application version
Juju 2.0 provides support for display of the version of
an application deployed by a charm in juju status.

Insert the os_application_version_set function into the
existing assess_status function - this gets called after
all hook executions, and periodically after that, so any
changes in package versions due to normal system updates
will also be reflected in the status output.

This review also includes a resync of charm-helpers to
pickup hookenv and contrib.openstack support for this
feature.

Change-Id: I75009a66ce9c9d43e234f9c5acbb185ac4a66ba5
2016-09-20 13:32:28 +01:00
David Ames c9e60b2799 Install python dependencies early for CH ip.py
When using charmhelpers.contrib.network.ip apt install was quietly
failing. Install python dependencies in the early install hook
(before install.real).

Charm-helpers sync to bring in apt_install with fatal=True

Change-Id: I03b43c5bb75a831fa55fd09c43c5ba2f79ae123b
Partial-Bug: 1601972
2016-07-18 10:04:21 -07:00
James Page ab35249f38 Re-license charm as Apache-2.0
All contributions to this charm where made under Canonical
copyright; switch to Apache-2.0 license as agreed so we
can move forward with official project status.

Change-Id: I97206ee8be76220cb0937a09be3230432e04535a
2016-07-01 18:15:00 +01:00
Edward Hope-Morley 6ab28b3639 Add hardening support
Add charmhelpers.contrib.hardening and calls to install,
config-changed, upgrade-charm and update-status hooks.
Also add new config option to allow one or more hardening
modules to be applied at runtime.

Change-Id: If0d1e10b58ed506e0aca659f30120b8d5c96c04f
2016-03-24 11:11:58 +00:00
Edward Hope-Morley e7bf2f910e Allow devices to be added post-install
Track devices that have been added to the ring and allow
devices to be added to the ring post-install (currently
only allowed within the install hook). Devices added to
the ring prior to this patch existing will be migrated
to the tracking store of devices to avoid conflicts.

Change-Id: Id268dc6369041a4d2db6f30a997dfa0c0d73b93f
Closes-Bug: 1383390
2016-03-07 22:25:29 +00:00
Edward Hope-Morley 4e109f4602 [hopem,r=]
Add support for setting swift-storage unit rsyncd acls.
Partially-Closes-Bug: 1427361
2015-11-24 13:51:05 +00:00
Corey Bryant acd500f953 [corey.bryant,trivial] Revert "Enable multiple devices per node when config changed" 2015-10-15 15:05:42 -04:00
Corey Bryant d0906aa167 [corey.bryant,trivial] Revert "Setup storage for new block devices in config changed." 2015-10-15 15:04:38 -04:00
David Ames cd36735391 Also setup_storage() on config-changed in case block-device has changed 2015-10-14 15:04:42 -07:00
David Ames 0802f55959 Update swift-storage relation when block-device changes. Associated with fix for LP Bug#1479938 2015-10-13 12:05:02 -07:00
Corey Bryant cd57171aa4 Add assess_status() callback 2015-10-13 09:18:52 +00:00
Corey Bryant 1dbd144445 Revert "Don't overwrite paused status in main()" 2015-10-13 08:52:10 +00:00
Corey Bryant 35b2ab8948 Don't overwrite paused status in main() 2015-10-12 16:37:08 +00:00
Corey Bryant 1a02c96e7e Drop assess_status() as it's duplicating work done by set_os_workload_status(). 2015-10-09 12:00:26 +00:00
Corey Bryant e57d8793e5 Move to using REQUIRED_INTERFACES and check for swift-hash. 2015-10-09 03:47:26 +00:00
Corey Bryant 3c3064f723 Initial workload status support. 2015-10-08 02:15:44 +00:00
James Page 15a99db8a6 [thedac,r=james-page] Add action to manage upgrades 2015-09-23 14:52:50 +01:00
David Ames 72b2507f6c Action Managed Upgrades 2015-09-22 15:58:18 -07:00
James Page b0f3e5f618 Ensure python2 is installed before hook execution 2015-09-22 14:57:37 +01:00
Adam Collard f3532fe560 Move helpers to utils 2015-08-28 11:21:17 +01:00
Adam Collard 34bbc335b0 Don't need with_message=True anymore 2015-08-27 09:42:17 +01:00
Adam Collard 346252a6f0 Initial WIP for guarding restarts 2015-08-27 09:22:33 +01:00
Adam Collard c171188b10 Add misssing __init__.py 2015-07-17 16:54:12 +01:00
Adam Collard fbe3a71f36 Add missed move of misc_utils 2015-07-17 11:51:25 +01:00
Adam Collard 6ecc72dabd Move charmhelpers to top-level, symlink into hooks 2015-07-17 10:58:24 +01:00
Adam Collard a3ab496153 Add lib, move library code there. Symlink into hooks directory. 2015-07-17 10:57:19 +01:00