Commit Graph

114 Commits

Author SHA1 Message Date
Takashi Kajinami 453b1a9731 Use ini provider to manage storage config files
With this change now config files for storage services(account,
container and object) are managed by ini provider instead of concat
resources. This allows us to use the default file provided by packages
instead of replacing the whole file content by our templates, and use
the $::os_service_default value instead of hard-coding own defaults.

Related-Bug: #1921319
Change-Id: I375634214e88acc5075f64090287f934ce8451d1
2022-08-16 11:27:03 +09:00
Zuul 0b2ab8214f Merge "Remove deprecated swift::test_file" 2022-08-01 11:08:19 +00:00
Takashi Kajinami e004e46273 Remove deprecated allow_versions
... because the parameters were deprecated during Yoga cycle[1].

[1] a1088139ec

Change-Id: I237aa3196d24a287fd891c0a818dfd20bfd64d04
2022-06-28 22:40:26 +09:00
Takashi Kajinami 1414bd7d85 Remove deprecated swift::test_file
... because it was deprecated during Yoga cycle[1].

[1] dd3a5bbf0d

Change-Id: Ia990bffee54f6c0344060f16cf418a6c39ee81fc
2022-06-28 22:35:27 +09:00
Rajesh Tailor 4d937a4a02 Fix typos in parameter descriptions and tests
Change-Id: Ic91d910b973303b0d4db5164fdf9b5fd807a4f70
2022-06-27 13:39:37 +00:00
Zuul 625f0c44c0 Merge "Avoid hard-coding OS user/group in each manifest" 2022-02-22 17:33:28 +00:00
Takashi Kajinami f781eb9e34 Avoid hard-coding OS user/group in each manifest
and replace hard-codes by definition in params.pp .

Change-Id: I7bb32a93bbd0299917f6e7e13b84e808e179e5cc
2022-02-22 01:09:15 +09:00
Thomas Goirand bb483f0e28 Add support for container sharder
This patch ensure that the swift-container-sharder service is started,
and provides all the facility (ie: parameters) to configure the sharder.

Change-Id: I0db7479e8634038871ce65990fd0104841d2010c
2022-02-04 08:21:21 +00:00
Rocky 54c75b9f1f Add rsync parameters for object server
Change-Id: I752dc30755febcfa94cb266c2faf3b7aedc35ca9
2021-12-17 16:48:32 +11:00
Takashi Kajinami a1088139ec Deprecate support for the allow_versions parameter
... which was deprecated since swift 2.16.0[1] in favor of
the versioned_writes middleware.

[1] 5e673a542401a2d95249c3c03f26175214f08c79

Change-Id: I6cf3e63839a2044ded0e5c4af2fe6f1312b42fdf
2021-11-10 16:36:24 +09:00
Thomas Goirand fa2b39dd65 Add client_timeout parameter for object-server.conf
Sometimes, the default of 60s is just not enough, so we make this
configurable for the object-server.conf.

Change-Id: I965d5178f1f367e8b7b1f6137ff5290a75ffb030
2020-05-19 16:50:11 +02:00
Thomas Goirand 3b4e562faf Add object_server_mb_per_sync
The default mb_per_sync value of Swift leads to adding a lot of
latencies when transfering to an object store. This patch adds
the possibility to override this value for object store (it is
less relevant for other backend which rarely will reach the
cache size anyway). According to our tests, a value of 16 MB
is very good in production.

Change-Id: Id379d4221bc7e2a0c16125b97140e39b7726fd02
2020-03-09 21:35:11 +01:00
Zuul 12b62e289c Merge "Allow users to change container-replicator interval value." 2020-01-20 16:03:46 +00:00
David Hill 7d46d12b41 Allow users to change container-replicator interval value.
The default interval value is 30s and in some case, this is way
too agressive and we need an automated way of tweaking this value.

Change-Id: I271c218b45d10f15544077e93ded9d74d7842f65
Closes-bug: #1850856
2020-01-18 08:18:05 -05:00
Simeon Gourlin 14d1bda0b3
This add chunk_size related parameters for proxy-server and object-server.
New parameters for proxy class are object_chunk_size and client_chunk_size,
and for storage class: network_chunk_size, disk_chunk_size and
auditor_disk_chunk_size.

Change-Id: If2bbc9cea1f6e0959fd67d2e004b607a8e40beee
2019-11-20 09:55:29 +01:00
Thomas Goirand 31dc95fd15 Add servers_per_port parameter in object-server
According to the doc at this URL:
https://docs.openstack.org/swift/latest/deployment_guide.html

for performance reasons, it is desirable to use forks of the
object-server, each on a different port, one per device. This
is what I'm doing in openstack-cluster-install in Debian. For
this, I've been using this patch. It'd be nice to have it
upstream as well.

Change-Id: I9224eb7022871e92cc18f82e9985d91d5e0c2823
2018-09-18 15:48:54 +02:00
Thiago da Silva 287f2113c9 Enable use of splice in object server
Use splice() for zero-copy object GETs. This requires Linux kernel
version 3.0 or greater

Change-Id: I0c25b0f99c7fabe5def4ab253c0164a88c250796
2018-07-09 08:49:04 -04:00
Christian Schwede a4e1c84091 Add section for container-sharder service
The container sharder needs at least a default section to run.

Change-Id: Ifd59d69ec0bf85f8fbf5374f0547271eec972506
2018-06-11 10:02:43 +02:00
Christian Schwede 5f84cddfb9 Remove unused template for rsyncd.conf
Looks like this template was never used at all; the actual used template
is from puppet-rsync itself.

Change-Id: I19a82bc5b013486bd3eac1368db0583144fd16fb
2017-11-27 15:40:50 +01:00
ZhongShengping 86f435a2a5 Update URLs in documents according to document migration
Change-Id: I1b8e63b37be6599a4263afdf12d8f617163fde20
2017-07-14 13:05:37 +08:00
Christian Schwede af6fb24845 Add missing Swift object reconstructor service
Adding the object-reconstructor service and enabling it by default.
This is required for erasure coding.

Change-Id: If388bcc1532d7db2132d4b206e13252a30fa9041
2017-06-16 14:39:17 +02:00
Monty Taylor fb9e6dcecc
Replace references to swift.openstack.org
The policy of giving projects vanity domains stopped about 5 years ago.
swift.openstack.org is a redirect to the canonical location -
docs.openstack.org/developer/swift. While we are not aiming to remove
the redirect any time in the forseeable future due to existing published
links pointing to it, we should at the very least stop adding more of
those links to the world.

Change-Id: Ia56ce22c65c1fac29a2d8fb367797a4d7929a9b4
2017-03-08 09:50:24 -06:00
Adam Vinsh 60ed2ee79c Replace proxy concat/template with ini provider
Maintains full backward compatibility.
Use swift_proxy_config ini provider to mange proxy-server.conf.
Remove all erb templates. Move all template logic into proxy
middleware classes. To purge proxy.conf of settings that are
not specified set the new paramater "purge_config" to "true".

Change-Id: I0a143cf812043ea0f9a008a6e5c60ec87f9a4e9a
2016-10-06 10:29:04 -04:00
Christian Schwede da01dd7bd0 Fix middleware order of proxy pipeline and add missing modules
This patch fixes the order of the middlewares defined in the Swift proxy
server pipeline.

Sources for the order:
https://github.com/openstack/swift/blob/master/etc/proxy-server.conf-sample#L91-L99
http://docs.openstack.org/developer/swift/middleware.html#
https://github.com/openstack/swift3/blob/master/etc/proxy-server.conf-sample#L9
https://github.com/openstack/ceilometermiddleware/blob/master/ceilometermiddleware/swift.py#L21-L22

These are all values in order:

 10 catch_errors
 20 gatekeeper
 30 healthcheck
 40 (reserved for proxy_logging, see below)
 50 cache
 60 container_sync
 70 swift3
 80 s3token
 90 ratelimit
100 crossdomain
110 bulk
120 tempurl
130 formpost
140 copy
150 tempauth
160 swauth
170 authtoken
180 keystone
190 staticweb
200 copy
210 account_quotas
220 container_quotas
230 slo
240 dlo
250 versioned_writes
260 ceilometer
270 proxy-logging
280 proxy-server

The copy and versioned_writes middlewares have been added too.

The proxy_logging should be added a second time at position 40, but the
actual value is 270 to have the final proxy-logging just before the
proxy-server.

Also renamed the suffix 'swauth' in the 'tempauth' manifest to be
'tempauth'; 'swauth' is used in a different manifest.

Closes-Bug: 1618514
Change-Id: I99433720e32dc3557b809c8d42ce3d5981c199c3
2016-09-01 14:51:28 +02:00
Jenkins 0917dd0ed9 Merge "Change swift authtoken" 2016-08-25 22:07:05 +00:00
Jenkins f5344c16aa Merge "Trim useless blank lines in generated config file" 2016-08-25 22:01:55 +00:00
Martin André 2d970a2edb Trim useless blank lines in generated config file
The proxy-server.conf file had many blank lines due to optional
parameters and bad usage of erb templating.

Change-Id: I3c8e7cdb284e052c82e9a339c12eb7f60bb4ee12
2016-08-25 10:24:52 +02:00
Iury Gregory Melo Ferreira a266ab91ed Change swift authtoken
Update authtoken class with parameters from
documentation [1]

We cant use the resource because swift does not
use keystone_authtoken section.

deprecations:
- admin_password, use password instead.
- admin_user, use username instead.
- admin_tenant_name, use project_name instead.
- identity_uri, use auth_url instead.
- admin_token, no usage.

removals:
- Old parameters (already deprecated)

[1]
http://docs.openstack.org/mitaka/config-reference/object-storage/proxy-server.html

Change-Id: I1eb4b0712214c059c713001ad991cbe92cd01711
2016-08-25 01:22:56 -03:00
Alex Schultz 658f6589de Add statsd configuration options
This change adds the ability to configure the statsd configuration
options for the account, container, object and proxy services.

Change-Id: I9b06c86b44162cbd35b0aa3a5d237dcc5eea318b
Closes-Bug: #1616512
2016-08-24 12:27:19 -06:00
Martin André 623dffa2f7 Add CORS settings
Allow configuring CORS settings.

Change-Id: I60df82ce6272a4c1ad22b6842f22713c7d7c7c63
2016-08-23 18:52:31 +02:00
Christian Schwede 0bb520569b Deprecate is_admin option from Keystone mw config in Swift proxy
This option has been deprecated some time ago, and finally been removed
in Mitaka. Let's deprecate this setting in puppet-swift too; there is no
longer a reason to use it from Mitaka on. This patch makes it a noop
parameter.

See Id6bc10c3e84262c0a9e6160a76af03c0ad363e9c in Swift for reference.

Change-Id: I525f5010d56ccdd205a0f9d55db99b0f359f35c3
Co-Authored-By: Pete Zaitcev <zaitcev@redhat.com>
2016-07-21 16:59:08 +02:00
Denis Egorenko ec4db849c8 Allow to configure container_sync middleware
Add possibility to configure container_sync middleware. Also
allow to configure swift container-sync-realms, which specifying
the allowable clusters and their information, which is required
for container_sync.

Change-Id: I183841cca3daf72a30db160b88c9e3a9335ad7dc
2016-05-30 16:03:44 +03:00
Kyrylo Galanov 379d9fc5f5 Add rmq credentials to each host in proxy
Otherwise it will try to connect with default guest/guest credentials

Change-Id: I097e768afd8783c78caee7aba4ac6600c7e55e47
Closes-bug: #1560055
2016-03-28 17:02:10 +02:00
Emilien Macchi 318b900706 Set mount_check to True by default
In a previous release, we sent a deprecation warning for mount_check
parameter which was set to False by default, while upstream uses True.

This patch aims to use upstream default, True, by default in our
interface.

Depends-On: I0bb1ccf4c0adc4e217e2989bd50d96741fa156f7
Change-Id: I453e24509a3f05d1fea69d47577cf81535a1ee13
2016-01-05 09:57:24 -05:00
Denis Egorenko e8ad981eff Update ceilometer proxy template
Now for using Ceilometer notifications from Swift we should
use Ceilometer middleware. Hence we need update template for Swift
to account for incoming and outgoing traffic notifications.

[1] I686e5d94fb72fd2a00973c91673edc417142ee2f
[2] http://docs.openstack.org/developer/oslo.messaging/transport.html#oslo_messaging.TransportURL

Change-Id: Ib6efd4a1ab4b76c74cffc546383f5382c24a8c43
Related-bug: #1510064
2015-12-02 18:20:59 +03:00
Emilien Macchi f1f9a24998 drop useless comment in authtoken.conf.erb
Drop useless comment in authtoken.conf.erb.
Change-Id: I7d027d0647e0fb10aa32313e9550815b74ec7bac
2015-09-29 18:41:19 +00:00
Jenkins 79b596b817 Merge "Add support for DLO configuration." 2015-09-25 13:54:35 +00:00
Aleksandr Didenko 4ee7b74aed Add support for DLO configuration.
New class swift::proxy::dlo for DLO configuration.
Add acceptance tests
Add docs to README

Change-Id: Ie209e494d7affd72c3fea3f2cc3459b91d7c6765
2015-09-24 16:36:21 -05:00
Jenkins df70d51f16 Merge "Warn that object storage parameter mount_check changes next release" 2015-09-24 20:17:56 +00:00
Ben Kero b3ff1ac9d4 Warn that object storage parameter mount_check changes next release
The 'mount_check' parameter is different in this Puppet module than
in Swift. This patch adds a warning() to tell the user that starting
in the next version, the default will change to match the upstream
default from False to True.

Closes-Bug: 1265243

Change-Id: I7f98e523be2f8bbf39b70c544a1965c975c690dd
Signed-off-by: Ben Kero <ben.kero@gmail.com>
2015-09-23 22:13:42 -07:00
Benedikt von St. Vieth c2bb2fb31e provide means to disable log_requests in config templates
it is hard coded into account, object and container config templates to log
every request. Provide a means to disable log_requests via
swift::storage::server and swift::storage::all, this reduces the log output.

Change-Id: I2fd97c1406710bb0305955f8ff9b47f91c09e13d
Closes-Bug: #1484614
2015-09-12 14:30:16 +02:00
Guilherme Maluf 4c5f53d0d3 Add a blank line to the beginning of each filter
Filters fragments leads to section headers and previous options with no
blank line between them

  foo = bar
  [section]

Add a blank line to the beginning of each filter so that when applied
the resulting file is more human readable

Change-Id: I411eba5da5fc4e1a7217df197edbaf5ef2d5e4fa
2015-08-28 16:50:38 -03:00
Jenkins 8a09d4c33d Merge "Add tempurl middleware options" 2015-08-20 14:28:29 +00:00
Guilherme Maluf 62ba90a475 Add tempauth middleware options
Tempauth middleware template included only example entries

This commit provides most options to configure tempauth middleware,
especialy an array of accounts/users hash which defines the authenticated
list available by tempauth

It's add by default the admin account/user
  user_admin_admin = admin .admin .reseller_admin

Change-Id: Ib67d7deeeb2f98a464d18813ae4569c28a04472a
2015-08-03 07:18:55 -03:00
Guilherme Maluf 25a96920f6 Add tempurl middleware options
Tempurl middleware is set with default options only

This commits make it possible to manage all tempurl configs defined in
proxy-server.conf-sample[1]:

  methods
  incoming_remove_headers
  incoming_allow_headers
  outgoing_remove_headers
  outgoing_allow_headers

Parameters are optional and can be set with array or string

[1]  https://github.com/openstack/swift/blob/master/etc/proxy-server.conf-sample

Change-Id: I1fab905a46aa58c43b8b3fd35ecdaf310bffc121
2015-07-29 18:40:42 -03:00
Adam Vinsh c9809f8c92 Added rsyslog logging support to object-server
Also change both proxy-server and object-server
.conf templates to test if log_udp_server and
log_udp_port are nil, instead of an empty string.
Testing for an empty string would have retruned
true if set to 'undef' - for futer parser compat.

Change-Id: Idac805f628dbec70a4adb6a1c021d85a743fdb72
2015-05-03 21:22:12 -06:00
vinsh e04486fe46 read_affinity requires affinity sorting_method
proxy-server.conf read_affinity setting has no
effect unless sorting_method is set to 'affinity'

Change-Id: Ife0a30158289ea1dd44c1e93f058538c45c3a9c8
2015-04-08 17:08:47 -06:00
David Moreau Simard 7a0c615db8 Add node_timeout parameter for proxy-server.conf
This commit allows to configure the node_timeout parameter
of proxy-server.conf proxy-server middleware in the pipeline.

Change-Id: Idbac35336a5b266f6de288a28243f91ee4d67afd
2015-03-25 13:55:39 -04:00
Dan Prince 119ff9c3e0 Add support for identity_uri.
This patch adds the ability to set a new identity_uri parameter.
It also deprecates the old auth_host, auth_port, auth_protocol,
and auth_admin_prefix parameters. Logic is in place so that
users of the deprecated settings should have a smooth upgrade
process and get deprecation warnings until they adopt the
new settings.

Change-Id: Ideefb4d824cbd5b4b83f9eb773a75e536e3458fb
2015-03-17 14:31:19 +01:00
Jamie Lennox 128ce97061 Use keystonemiddleware instead of client
The auth_token middleware that lives in
keystoneclient.middleware.auth_token has been deprecated in favour of
keystonemiddleware.auth_token.

This prints a deprecation warning and means we are not getting new
features.

Closes-Bug: #1428376
Change-Id: If5352801b8105533600965a25f02a577a858f1a2
2015-03-05 00:54:31 +00:00