Commit Graph

24 Commits

Author SHA1 Message Date
Erlon R. Cruz 4337b753f5 Fix swift replication errors
There are several errors related to the swift replication service.
The swift storage charm is not properly restarting the services
after configuration changes, the correct object_lockup_timeout
value (that per the behaviour observerd in our environments must
be greater than object_rsync_timeout) and we also needed to fix
the object replicator config file to honor the
object-handoffs-first configuration.

This patch along with the swift proxy-change should fix the
currently known replication problems.

Closes-bug: #1903762
Depends-on: I87eb23de94e3f2f5b06d44df1f8bd9d2324456a0
Change-Id: I87eb23de94e3f2f5b06d44df1f8bd9d2324c8470
2021-01-05 11:19:13 -03:00
Andrea Ieri 66c14b2aff Ensure lockup_timeout is never < 2*rsync_timeout
Change-Id: Id0afc210b7be8ac7c740753ef06b51db0bae410c
Closes-Bug: 1888725
2020-07-23 15:57:46 -04:00
Drew Freiberger d31095d482 Add config option for fallocate_reserve
In order to prevent disks from filling up, add the option to configure
fallocate_reserve parameter in account-server.conf,
container-server.conf, and object-server.conf, as described in
https://docs.openstack.org/swift/latest/admin_guide.html#preventing-disk-full-scenarios

This change adds the config option file-allocation-reserve with a
default of 0.

Change-Id: Ib396a151250bb8d1733f9b5b9cab7eb506c9f6c6
Closes-Bug: 1872069
2020-07-22 17:30:18 -04:00
Edward Hope-Morley 2c7acd2dfc Allow configuring node_timeout
Swift default is used if no config provided.
As per swift docs, if node_timeout is set,
http_timeout ensured to be greater than
node_timeout.

Change-Id: Idf5945a6c2f64fcad8f2ab7cb137f1ce1f1dc424
Closes-Bug: #1878657
2020-06-08 19:59:06 +01:00
Drew Freiberger 9809a3b347 Add support for object replication handoffs_first
In extreme cases of object rebalance or node failure, swift environments
with millions of objects may have a hard time with objects landing in
handoff or misplaced partitions and the operator of the cloud may need
to prioritize migrating data in handoff partitions back to primary
partitions over the 3 primary partitions replicating amongst each other.

To allow for this, the object-server.conf [object-replicator]
configuration can have handoffs_first set to True which will force
the object-replicators to focus on handoff partitions before syncing
primary partitions for any given object partition.

Change-Id: I8b44c287567a0e6d634def0b13baf0fe4ad4aa7b
Closes-Bug: 1878087
2020-05-11 15:01:22 -05:00
Tytus Kurek fc261a5adc Swift Global Cluster
This patchset adds a support for Swift Global Cluster feature as
described at:

https://docs.openstack.org/swift/latest/overview_global_cluster.html

It allows specifying 'region' config option as part of the deployment.
Theregion is reflected later on in the Swift rings.

Change-Id: I11b6c7802e5bfbd61b06e4d11c65804a165781b6
Partial-Bug: 1815879
Needed-By: I406445493e2226aa5ae40a09c9053ac8633a46e9
2019-12-17 10:08:13 +00:00
Chris MacNaughton 6724011ad8 Fix gid key in swift conf
Change-Id: Ib3dec9e6aedb57e7a7970d45ebeb80b56cf94c41
Closes-bug: #1746402
2018-08-21 15:00:18 +02:00
James Page 591c084532 Add support for block device encryption
Add new secrets-storage relation to vault, supporting the
use of block device encryption using dm-crypt/vaultlocker.

Prepared devices are now recorded in the local unit kv
store; this information is used to provide a list of
configured block devices to the swift-proxy charm,
rather than the previous best guess provided by
determine_block_devices.  This allows us to use the
dm-crypt device name, rather than the underlying
block device.

Encrypted block devices are unlocked on boot using
vaultlocker-decrypt systemd units (enabled by vaultlocker);
/etc/fstab entries for such devices make use of a
x-systemd.requires option to ensure that the block device
is unlocked prior to attempting to mount it.

Add new storage binding to allow charm to be used with
Juju storage.

Add new ephemeral-unmount configuration option to allow
cloud ephemeral storage to be used for testing purposes;
update functional testing to use this option.  The behaviour
of 'overwrite' was changed to accomodate the use of
encrypted block devices.

Change-Id: I9b3f8cd2de412ee96e0139dba4d4abdf998ecaf2
2018-05-08 12:52:37 +01:00
James Hebden 0876b4fb73 Add statsd metrics support to the swift-storage charm
By default, statsd metrics can be sent by the swift account,
container and object storage services for diagnostic and
monitoring purposes, but are disabled by default. This change exposes
charm config settings that allow it to be enabled by setting
'statsd_host' to a non-empty value. 'statsd_port' and
'statsd_sample_rate' are also supported for changing the destination
port and rate at which metrics are collected.

Closes-Bug: #1729770

Change-Id: If1bf3ced8a9ed07af81f352eb0263659d147e3aa
2017-11-05 15:23:38 +11:00
Xav Paice 18419a87c5 Add object-rsync-timeout option
Adds the rsync_timeout option to to object-server.conf, so we can adjust
it away from the default of 900s.

If there are a number of large partitions needing replication,
occasionally one needs to adjust the timeout in order to allow the rsync
to complete rather than timeout and retry.

Change-Id: I2d895741cb0528836a675deb6399005a5bf59ab5
Closes-bug: 1702039
2017-07-04 07:57:22 +12:00
Edward Hope-Morley c1f3358a66 fix minor nit in template 2015-11-27 09:28:53 +00:00
Edward Hope-Morley 4e109f4602 [hopem,r=]
Add support for setting swift-storage unit rsyncd acls.
Partially-Closes-Bug: 1427361
2015-11-24 13:51:05 +00:00
Brad Marshall d1e2c14274 [bradm] Fixed rsync to use /etc/rsync-juju.d, check if host_context is defined before using it 2014-11-06 17:38:43 +10:00
Brad Marshall 512ff04af3 [bradm] Removed local_ip binding from base rsync fragments 2014-10-30 17:18:15 +10:00
root 5727c68515 [bradm] Adding nrpe checks, handle rsyncd config fragments 2014-10-30 00:52:15 -05:00
James Page a04de8cd91 [mthaddon,r=james-page] Add configuration options for tuning worker thread counts. 2014-10-06 22:45:39 +01:00
Hui Xiang 20b6a8994c Use general BindHostContext(). 2014-09-25 14:36:13 +08:00
Tom Haddon 6391ad523b Add config options for max connections in rsync stanzas and concurrency of object replicator 2014-09-12 10:17:27 +01:00
Hui Xiang 00493eeeb1 Need to bind host ip to ipv6 addr. 2014-08-15 14:54:48 +08:00
Tom Haddon 89cac18e1f Add in a tunable option for object server threads_per_disk and tune default to recommendations from http://docs.openstack.org/developer/swift/deployment_guide.html#object-server-configuration 2014-06-18 11:48:24 +01:00
Tom Haddon ee219bdd38 Make the number of workers for account, container and object servers a multiplier of the number of CPUs, and default to a sensible setting per recommendations in http://docs.openstack.org/developer/swift/deployment_guide.html#general-service-tuning 2014-06-18 11:18:02 +01:00
James Page 5fa90096c7 Support versioning of objects 2014-04-02 13:49:56 +01:00
James Page 464bc4bc65 bind to all network interfaces by default 2013-09-30 14:29:32 +01:00
Adam Gandelman b99f2cf010 Fix up port handling, checkin Makefile+setup.cfg+.coveragerc, rename tests/ -> unit_tests/ 2013-07-19 12:52:45 -07:00