Add swift-sync role and environment
Add the swift-remote host group and environment file. Add an os_swift_sync role which will sync the swift ring and ssh keys for swift hosts (remote and not-remote). Which has the following: * Moves the key and ring tasks out of os_swift role to os_swift_sync. * This adds the use of the "-r" flag that was added to the swift_rings.py and swift_rings_check.py. * Adds a ring.builder vs contents file consistency check. * Adjusts the rsync process to use the built-in synchronize module * Ensure services have started post ring/ssh key sync. Adds environment file and sample configuration file for swift-remote hosts (conf.d). Move appropriate default vars to the os_swift_sync role, and remove them from the os_swift role. Rename the "os-swift-install.yml" playbook to "os-swift-setup.yml" as this handles only the setup, and add a playbook to for both os-swift-sync.yml and an overarching playbook (os-swift-install.yml) that will call both the os-swift-sync.yml and os-swift-setup.yml playbooks. This means the funcitonality of "os-swift-install.yml" remains unchanged. Adjust the run-playbooks.sh so that it calls the new overarching swift playbook. Change-Id: Ie2d8041b4bc46f092a96882fe3ca430be92195ed Partially-Implements: blueprint multi-region-swift
This commit is contained in:
commit
6f8d14aa19
|
@ -0,0 +1,85 @@
|
|||
OpenStack swift key & ring sync
|
||||
###############################
|
||||
:tags: openstack, swift, cloud, ansible
|
||||
:category: \*nix
|
||||
|
||||
contributor guidelines
|
||||
^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Filing Bugs
|
||||
-----------
|
||||
|
||||
Bugs should be filed on Launchpad, not GitHub: "https://bugs.launchpad.net/openstack-ansible"
|
||||
|
||||
|
||||
When submitting a bug, or working on a bug, please ensure the following criteria are met:
|
||||
* The description clearly states or describes the original problem or root cause of the problem.
|
||||
* Include historical information on how the problem was identified.
|
||||
* Any relevant logs are included.
|
||||
* The provided information should be totally self-contained. External access to web services/sites should not be needed.
|
||||
* Steps to reproduce the problem if possible.
|
||||
|
||||
|
||||
Submitting Code
|
||||
---------------
|
||||
|
||||
Changes to the project should be submitted for review via the Gerrit tool, following
|
||||
the workflow documented at: "http://docs.openstack.org/infra/manual/developers.html#development-workflow"
|
||||
|
||||
Pull requests submitted through GitHub will be ignored and closed without regard.
|
||||
|
||||
|
||||
Extra
|
||||
-----
|
||||
|
||||
Tags:
|
||||
If it's a bug that needs fixing in a branch in addition to Master, add a '\<release\>-backport-potential' tag (eg ``juno-backport-potential``). There are predefined tags that will autocomplete.
|
||||
|
||||
Status:
|
||||
Please leave this alone, it should be New till someone triages the issue.
|
||||
|
||||
Importance:
|
||||
Should only be touched if it is a Blocker/Gating issue. If it is, please set to High, and only use Critical if you have found a bug that can take down whole infrastructures.
|
||||
|
||||
|
||||
Style guide
|
||||
-----------
|
||||
|
||||
When creating tasks and other roles for use in Ansible please create then using the YAML dictionary format.
|
||||
|
||||
Example YAML dictionary format:
|
||||
.. code-block:: yaml
|
||||
|
||||
- name: The name of the tasks
|
||||
module_name:
|
||||
thing1: "some-stuff"
|
||||
thing2: "some-other-stuff"
|
||||
tags:
|
||||
- some-tag
|
||||
- some-other-tag
|
||||
|
||||
|
||||
Example **NOT** in YAML dictionary format:
|
||||
.. code-block:: yaml
|
||||
|
||||
- name: The name of the tasks
|
||||
module_name: thing1="some-stuff" thing2="some-other-stuff"
|
||||
tags:
|
||||
- some-tag
|
||||
- some-other-tag
|
||||
|
||||
|
||||
Usage of the ">" and "|" operators should be limited to Ansible conditionals and command modules such as the ansible ``shell`` module.
|
||||
|
||||
|
||||
Issues
|
||||
------
|
||||
|
||||
When submitting an issue, or working on an issue please ensure the following criteria are met:
|
||||
* The description clearly states or describes the original problem or root cause of the problem.
|
||||
* Include historical information on how the problem was identified.
|
||||
* Any relevant logs are included.
|
||||
* If the issue is a bug that needs fixing in a branch other than Master, add the ‘backport potential’ tag TO THE ISSUE (not the PR).
|
||||
* The provided information should be totally self-contained. External access to web services/sites should not be needed.
|
||||
* If the issue is needed for a hotfix release, add the 'expedite' label.
|
||||
* Steps to reproduce the problem if possible.
|
|
@ -0,0 +1,202 @@
|
|||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
APPENDIX: How to apply the Apache License to your work.
|
||||
|
||||
To apply the Apache License to your work, attach the following
|
||||
boilerplate notice, with the fields enclosed by brackets "{}"
|
||||
replaced with your own identifying information. (Don't include
|
||||
the brackets!) The text should be enclosed in the appropriate
|
||||
comment syntax for the file format. We also recommend that a
|
||||
file or class name and description of purpose be included on the
|
||||
same "printed page" as the copyright notice for easier
|
||||
identification within third-party archives.
|
||||
|
||||
Copyright {yyyy} {name of copyright owner}
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
|
@ -0,0 +1,18 @@
|
|||
OpenStack swift key/ring sync
|
||||
#############################
|
||||
:tags: openstack, swift, cloud, ansible
|
||||
:category: \*nix
|
||||
|
||||
Role to synchronise keys and the ring for swift hosts
|
||||
|
||||
This role will synchronise the following:
|
||||
* ring
|
||||
* ssh keys
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
- name: Sync swift rings and keys
|
||||
hosts: swift_all:swift_remote_all
|
||||
user: root
|
||||
roles:
|
||||
- { role: "os_swift_sync", tags: [ "os-swift-sync" ] }
|
|
@ -0,0 +1,59 @@
|
|||
---
|
||||
# Copyright 2014, Rackspace US, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
# Set the managed regions as a list of swift regions to manage
|
||||
# Use for global clusters, default when not set is all regions.
|
||||
# swift_managed_regions:
|
||||
# - 1
|
||||
# - 2
|
||||
|
||||
swift_proxy_port: "8080"
|
||||
swift_object_port: "6000"
|
||||
swift_container_port: "6001"
|
||||
swift_account_port: "6002"
|
||||
|
||||
swift_default_replication_number: 3
|
||||
swift_default_min_part_hours: 1
|
||||
swift_default_host_zone: 0
|
||||
swift_default_host_region: 1
|
||||
swift_default_drive_weight: 100
|
||||
|
||||
swift_account_program_names:
|
||||
- swift-account-server
|
||||
- swift-account-auditor
|
||||
- swift-account-replicator
|
||||
- swift-account-reaper
|
||||
- swift-account-replicator-server
|
||||
|
||||
swift_container_program_names:
|
||||
- swift-container-server
|
||||
- swift-container-auditor
|
||||
- swift-container-replicator
|
||||
- swift-container-sync
|
||||
- swift-container-updater
|
||||
- swift-container-replicator-server
|
||||
|
||||
swift_object_program_names:
|
||||
- swift-object-server
|
||||
- swift-object-auditor
|
||||
- swift-object-replicator
|
||||
- swift-object-updater
|
||||
- swift-object-replicator-server
|
||||
|
||||
swift_proxy_program_names:
|
||||
- swift-proxy-server
|
||||
|
||||
swift_system_user_name: swift
|
||||
swift_system_home_folder: "/var/lib/{{ swift_system_user_name }}"
|
|
@ -0,0 +1,255 @@
|
|||
#!/usr/bin/env python
|
||||
# Copyright 2014, Rackspace US, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
from __future__ import print_function
|
||||
from optparse import OptionParser
|
||||
from os.path import exists
|
||||
from swift.cli.ringbuilder import main as rb_main
|
||||
|
||||
import json
|
||||
import pickle
|
||||
import sys
|
||||
import threading
|
||||
|
||||
|
||||
USAGE = "usage: %prog -f <swift_ring.contents> -r <managed_region>"
|
||||
|
||||
DEVICE_KEY = "%(ip)s:%(port)d/%(device)s"
|
||||
|
||||
|
||||
class RingValidationError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def create_buildfile(build_file, part_power, repl, min_part_hours,
|
||||
update=False, data=None, validate=False):
|
||||
if update:
|
||||
# build file exists, so lets just update the existing build file
|
||||
if not data:
|
||||
data = get_build_file_data(build_file)
|
||||
if data is None:
|
||||
data = {}
|
||||
|
||||
if repl != data.get('replicas') and not validate:
|
||||
run_and_wait(rb_main, ["swift-ring-builder", build_file,
|
||||
"set_replicas", repl])
|
||||
if min_part_hours != data.get('min_part_hours') and not validate:
|
||||
run_and_wait(rb_main, ["swift-ring-builder", build_file,
|
||||
"set_min_part_hours", min_part_hours])
|
||||
if part_power != data.get('part_power'):
|
||||
raise RingValidationError('Part power cannot be changed! '
|
||||
'you must rebuild the ring if you need '
|
||||
'to change it.\nRing part power: %s '
|
||||
'Inventory part power: %s'
|
||||
% (data.get('part_power'), part_power))
|
||||
|
||||
elif not validate:
|
||||
run_and_wait(rb_main, ["swift-ring-builder", build_file, "create",
|
||||
part_power, repl, min_part_hours])
|
||||
|
||||
|
||||
def change_host_weight(build_file, host_search_str, weight):
|
||||
run_and_wait(rb_main, ["swift-ring-builder", build_file, "set_weight",
|
||||
host_search_str, weight])
|
||||
|
||||
|
||||
def remove_host_from_ring(build_file, host):
|
||||
run_and_wait(rb_main, ["swift-ring-builder", build_file, "remove",
|
||||
host])
|
||||
|
||||
|
||||
def update_host_in_ring(build_file, new_host, old_host, validate=False):
|
||||
if new_host.get('zone', 0) != old_host['zone']:
|
||||
devstr = DEVICE_KEY % new_host
|
||||
raise RingValidationError('Cannot update zone on %s, this can only be '
|
||||
'done when the drive is added' % devstr)
|
||||
if new_host.get('region', 1) != old_host['region']:
|
||||
devstr = DEVICE_KEY % new_host
|
||||
raise RingValidationError('Cannot update region on %s, this can only '
|
||||
'be done when the drive is added' % devstr)
|
||||
|
||||
try:
|
||||
r_ip = new_host.get('repl_ip', new_host['ip'])
|
||||
r_port = new_host.get('repl_port', new_host['port'])
|
||||
weight = new_host.get('weight')
|
||||
|
||||
old_r_ip = old_host['replication_ip']
|
||||
old_r_port = old_host['replication_port']
|
||||
|
||||
if r_ip != old_r_ip or r_port != old_r_port:
|
||||
host_d = {'r_ip': r_ip, 'r_port': r_port}
|
||||
host_d.update(new_host)
|
||||
host_str = (
|
||||
"%(ip)s:%(port)dR%(r_ip)s:%(r_port)d/%(device)s" % host_d
|
||||
)
|
||||
if not validate:
|
||||
run_and_wait(rb_main, ["swift-ring-builder", build_file,
|
||||
"set_info", DEVICE_KEY % new_host,
|
||||
host_str])
|
||||
except Exception as ex:
|
||||
raise RingValidationError(ex)
|
||||
|
||||
if weight != old_host['weight'] and not validate:
|
||||
change_host_weight(build_file, DEVICE_KEY % new_host, weight)
|
||||
|
||||
|
||||
def add_host_to_ring(build_file, host, validate=False):
|
||||
host_str = ""
|
||||
try:
|
||||
if host.get('region') is not None:
|
||||
host_str += 'r%(region)d' % host
|
||||
host_str += "z%d" % (host.get('zone'))
|
||||
host_str += "-%(ip)s:%(port)d" % host
|
||||
if host.get('repl_ip'):
|
||||
r_ip = host['repl_ip']
|
||||
r_port = host.get('repl_port', host['port'])
|
||||
host_str += "R%s:%d" % (r_ip, r_port)
|
||||
elif host.get('repl_port'):
|
||||
r_ip = host.get('repl_ip', host['ip'])
|
||||
r_port = host['repl_port']
|
||||
host_str += "R%s:%d" % (r_ip, r_port)
|
||||
host_str += "/%(device)s" % host
|
||||
weight = host.get('weight')
|
||||
except Exception as ex:
|
||||
raise RingValidationError(ex)
|
||||
if not validate:
|
||||
run_and_wait(rb_main, ["swift-ring-builder", build_file, 'add',
|
||||
host_str, str(weight)])
|
||||
|
||||
|
||||
def run_and_wait(func, *args):
|
||||
t = threading.Thread(target=func, args=args)
|
||||
t.start()
|
||||
return t.join()
|
||||
|
||||
|
||||
def get_build_file_data(build_file):
|
||||
build_file_data = None
|
||||
if exists(build_file):
|
||||
try:
|
||||
with open(build_file) as bf_stream:
|
||||
build_file_data = pickle.load(bf_stream)
|
||||
except Exception as ex:
|
||||
print("Error: failed to load build file '%s': %s" % (build_file,
|
||||
ex))
|
||||
build_file_data = None
|
||||
return build_file_data
|
||||
|
||||
|
||||
def build_ring(build_name, repl, min_part_hours, part_power, hosts,
|
||||
region=None, validate=False):
|
||||
# Create the build file
|
||||
build_file = "%s.builder" % build_name
|
||||
build_file_data = get_build_file_data(build_file)
|
||||
|
||||
update = build_file_data is not None
|
||||
create_buildfile(
|
||||
build_file,
|
||||
part_power,
|
||||
repl,
|
||||
min_part_hours,
|
||||
update,
|
||||
data=build_file_data,
|
||||
validate=validate
|
||||
)
|
||||
|
||||
old_hosts = {}
|
||||
if update:
|
||||
for i, dev in enumerate(build_file_data['devs']):
|
||||
if dev is not None:
|
||||
if region is None or int(region) == int(dev['region']):
|
||||
old_hosts[DEVICE_KEY % dev] = i
|
||||
for host in hosts:
|
||||
host_key = DEVICE_KEY % host
|
||||
if region is None or int(region) == int(host['region']):
|
||||
if host_key in old_hosts:
|
||||
old_host = build_file_data['devs'][old_hosts[host_key]]
|
||||
update_host_in_ring(build_file, host, old_host,
|
||||
validate=validate)
|
||||
old_hosts.pop(host_key)
|
||||
else:
|
||||
add_host_to_ring(build_file, host, validate=validate)
|
||||
|
||||
if old_hosts and not validate:
|
||||
# There are still old hosts, these hosts must've been removed
|
||||
for host in old_hosts:
|
||||
remove_host_from_ring(build_file, host)
|
||||
|
||||
# Rebalance ring
|
||||
if not validate:
|
||||
if not hosts:
|
||||
run_and_wait(
|
||||
rb_main, ["swift-ring-builder", build_file, "write_ring"]
|
||||
)
|
||||
else:
|
||||
run_and_wait(
|
||||
rb_main, ["swift-ring-builder", build_file, "rebalance"]
|
||||
)
|
||||
|
||||
|
||||
def main(setup, region):
|
||||
# load the json file
|
||||
try:
|
||||
with open(setup) as json_stream:
|
||||
_contents_file = json.load(json_stream)
|
||||
except Exception as ex:
|
||||
print("Failed to load json string %s" % ex)
|
||||
return 1
|
||||
|
||||
hosts = _contents_file['drives']
|
||||
kargs = {'validate': True, 'hosts': hosts, 'region': region}
|
||||
ring_call = [
|
||||
_contents_file['builder_file'],
|
||||
_contents_file['repl_number'],
|
||||
_contents_file['min_part_hours'],
|
||||
_contents_file['part_power']
|
||||
]
|
||||
|
||||
try:
|
||||
build_ring(*ring_call, **kargs)
|
||||
except RingValidationError as ex:
|
||||
print(ex)
|
||||
return 2
|
||||
|
||||
# If the validation passes lets go ahead and build the rings.
|
||||
kargs.pop('validate')
|
||||
build_ring(*ring_call, **kargs)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = OptionParser(USAGE)
|
||||
parser.add_option(
|
||||
"-f",
|
||||
"--file",
|
||||
dest="setup",
|
||||
help="Specify the swift ring contents file.",
|
||||
metavar="FILE"
|
||||
)
|
||||
parser.add_option(
|
||||
"-r",
|
||||
"--region",
|
||||
help="Specify the region to manage for the ring file.",
|
||||
dest="region",
|
||||
type='int',
|
||||
metavar="REGION"
|
||||
)
|
||||
|
||||
options, _args = parser.parse_args(sys.argv[1:])
|
||||
if options.setup and not exists(options.setup):
|
||||
print("Swift ring contents file not found or doesn't exist")
|
||||
parser.print_help()
|
||||
sys.exit(1)
|
||||
|
||||
sys.exit(main(options.setup, options.region))
|
|
@ -0,0 +1,174 @@
|
|||
#!/usr/bin/env python
|
||||
# Copyright 2014, Rackspace US, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
from __future__ import print_function
|
||||
from optparse import OptionParser
|
||||
from os.path import exists
|
||||
|
||||
import json
|
||||
import pickle
|
||||
import sys
|
||||
|
||||
USAGE = "usage: %prog -f <swift_ring.contentsa> -r <managed_region>"
|
||||
|
||||
DEVICE_KEY = "%(ip)s:%(port)d/%(device)s"
|
||||
|
||||
|
||||
class RingComparisonError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def get_build_file_data(build_file):
|
||||
build_file_data = None
|
||||
if exists(build_file):
|
||||
try:
|
||||
with open(build_file) as bf_stream:
|
||||
build_file_data = pickle.load(bf_stream)
|
||||
except Exception as ex:
|
||||
print("Error: failed to load build file '%s': %s" % (build_file,
|
||||
ex))
|
||||
build_file_data = None
|
||||
return build_file_data
|
||||
|
||||
|
||||
def check_ring_settings(build_file, part_power, repl, min_part_hours,
|
||||
data=None):
|
||||
# Check if the build file is emptuy
|
||||
if data is None:
|
||||
raise RingComparisonError('Build file %s is empty or does '
|
||||
'not exist.' % build_file)
|
||||
# Check if replica count matches for contents and ring file
|
||||
if repl != data.get('replicas'):
|
||||
raise RingComparisonError('Replica count does not match')
|
||||
# Check min_part_hours matches for contents and ring file
|
||||
if min_part_hours != data.get('min_part_hours'):
|
||||
raise RingComparisonError('min_part_hours does not match')
|
||||
# Check part_power matches for contents and ring file
|
||||
if part_power != data.get('part_power'):
|
||||
raise RingComparisonError('part_power does not match')
|
||||
|
||||
|
||||
def check_host_settings(content_host, ring_host):
|
||||
devstr = DEVICE_KEY % content_host
|
||||
if content_host.get('zone', 0) != ring_host['zone']:
|
||||
raise RingComparisonError('Zone on device %s differs to the ring.'
|
||||
% devstr)
|
||||
if content_host.get('region', 1) != ring_host['region']:
|
||||
raise RingComparisonError('Region on device %s differs to the ring.'
|
||||
% devstr)
|
||||
|
||||
content_repl_ip = content_host.get('repl_ip', content_host['ip'])
|
||||
content_repl_port = content_host.get('repl_port', content_host['port'])
|
||||
content_weight = content_host.get('weight')
|
||||
ring_repl_ip = ring_host['replication_ip']
|
||||
ring_repl_port = ring_host['replication_port']
|
||||
ring_weight = ring_host['weight']
|
||||
if content_repl_ip != ring_repl_ip:
|
||||
raise RingComparisonError('Replication IP for device %s differs '
|
||||
'to the ring.' % devstr)
|
||||
if content_repl_port != ring_repl_port:
|
||||
raise RingComparisonError('Replication Port for device %s differs '
|
||||
'to the ring.' % devstr)
|
||||
if content_weight != ring_weight:
|
||||
raise RingComparisonError('Device weight for device %s differs to the '
|
||||
'ring.' % devstr)
|
||||
|
||||
|
||||
def check_ring(build_name, repl, min_part_hours, part_power, content_hosts,
|
||||
region=None):
|
||||
build_file = "%s.builder" % build_name
|
||||
build_file_data = get_build_file_data(build_file)
|
||||
check_ring_settings(
|
||||
build_file,
|
||||
part_power,
|
||||
repl,
|
||||
min_part_hours,
|
||||
data=build_file_data
|
||||
)
|
||||
|
||||
ring_hosts = {}
|
||||
for i, dev in enumerate(build_file_data['devs']):
|
||||
if dev is not None:
|
||||
if region is None or int(region) == int(dev['region']):
|
||||
ring_hosts[DEVICE_KEY % dev] = i
|
||||
for content_host in content_hosts:
|
||||
host_key = DEVICE_KEY % content_host
|
||||
if region is None or int(region) == int(content_host['region']):
|
||||
if host_key in ring_hosts:
|
||||
ring_host = build_file_data['devs'][ring_hosts[host_key]]
|
||||
check_host_settings(content_host, ring_host)
|
||||
ring_hosts.pop(host_key)
|
||||
else:
|
||||
raise RingComparisonError('Device %s is not in the ring.'
|
||||
% host_key)
|
||||
|
||||
if ring_hosts:
|
||||
for ring_host in ring_hosts:
|
||||
if build_file_data['devs'][ring_hosts[ring_host]]['weight'] != 0:
|
||||
raise RingComparisonError('There are devices in the ring that'
|
||||
'are not in the inventory/contents'
|
||||
'file.')
|
||||
|
||||
|
||||
def main(setup, region):
|
||||
# load the json file
|
||||
try:
|
||||
with open(setup) as json_stream:
|
||||
_contents_file = json.load(json_stream)
|
||||
except Exception as ex:
|
||||
print("Failed to load json string %s" % ex)
|
||||
return 1
|
||||
|
||||
content_hosts = _contents_file['drives']
|
||||
kargs = {'content_hosts': content_hosts, 'region': region}
|
||||
ring_call = [
|
||||
_contents_file['builder_file'],
|
||||
_contents_file['repl_number'],
|
||||
_contents_file['min_part_hours'],
|
||||
_contents_file['part_power']
|
||||
]
|
||||
|
||||
try:
|
||||
check_ring(*ring_call, **kargs)
|
||||
print('SUCCESS: Ring is consistent with contents file')
|
||||
except RingComparisonError as ex:
|
||||
print(ex)
|
||||
return 2
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = OptionParser(USAGE)
|
||||
parser.add_option(
|
||||
"-f",
|
||||
"--file",
|
||||
dest="setup",
|
||||
help="Specify the swift ring contents file.",
|
||||
metavar="FILE"
|
||||
)
|
||||
parser.add_option(
|
||||
"-r",
|
||||
"--region",
|
||||
help="Specify the region to manage for the ring file.",
|
||||
dest="region",
|
||||
type='int',
|
||||
metavar="REGION"
|
||||
)
|
||||
|
||||
options, _args = parser.parse_args(sys.argv[1:])
|
||||
if options.setup and not exists(options.setup):
|
||||
print("Swift ring contents file not found or doesn't exist")
|
||||
parser.print_help()
|
||||
sys.exit(1)
|
||||
|
||||
sys.exit(main(options.setup, options.region))
|
|
@ -0,0 +1,31 @@
|
|||
---
|
||||
# Copyright 2014, Rackspace US, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
galaxy_info:
|
||||
author: rcbops
|
||||
description: Synchronisation of swift ring and ssh keys
|
||||
company: Rackspace
|
||||
license: Apache2
|
||||
min_ansible_version: 1.6.6
|
||||
platforms:
|
||||
- name: Ubuntu
|
||||
versions:
|
||||
- trusty
|
||||
categories:
|
||||
- cloud
|
||||
- python
|
||||
- swift
|
||||
- development
|
||||
- openstack
|
|
@ -0,0 +1,27 @@
|
|||
---
|
||||
# Copyright 2014, Rackspace US, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
- include: swift_key_setup.yml
|
||||
tags:
|
||||
- swift-key
|
||||
- swift-key-distribute
|
||||
|
||||
- include: swift_rings.yml
|
||||
tags:
|
||||
- swift-rings
|
||||
|
||||
- include: swift_sync_post_install.yml
|
||||
tags:
|
||||
- swift-sync-post-install
|
|
@ -0,0 +1,24 @@
|
|||
---
|
||||
# Copyright 2014, Rackspace US, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
- name: Create authorized keys file from host vars
|
||||
authorized_key:
|
||||
user: "{{ swift_system_user_name }}"
|
||||
key: "{{ hostvars[item]['swift_pubkey'] }}"
|
||||
with_items: groups['swift_all'] + groups['swift_remote_all']
|
||||
tags:
|
||||
- swift-key
|
||||
- swift-key-create
|
||||
- swift-key-distribute
|
|
@ -0,0 +1,30 @@
|
|||
---
|
||||
# Copyright 2014, Rackspace US, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
- name: Get public key contents and store as var
|
||||
command: |
|
||||
cat {{ swift_system_home_folder }}/.ssh/id_rsa.pub
|
||||
register: swift_pub
|
||||
changed_when: false
|
||||
tags:
|
||||
- swift-key
|
||||
- swift-key-create
|
||||
|
||||
- name: Register a fact for the swift pub key
|
||||
set_fact:
|
||||
swift_pubkey: "{{ swift_pub.stdout }}"
|
||||
tags:
|
||||
- swift-key
|
||||
- swift-key-create
|
|
@ -0,0 +1,25 @@
|
|||
---
|
||||
# Copyright 2014, Rackspace US, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
- include: swift_key_populate.yml
|
||||
tags:
|
||||
- swift-key
|
||||
- swift-key-create
|
||||
|
||||
- include: swift_key_distribute.yml
|
||||
tags:
|
||||
- swift-key
|
||||
- swift-key-create
|
||||
- swift-key-distribute
|
|
@ -0,0 +1,32 @@
|
|||
---
|
||||
# Copyright 2014, Rackspace US, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
- include: swift_rings_md5sum.yml
|
||||
|
||||
- include: swift_rings_check.yml
|
||||
when: >
|
||||
inventory_hostname == groups['swift_hosts'][0]
|
||||
|
||||
- include: swift_rings_build.yml
|
||||
when: >
|
||||
inventory_hostname == groups['swift_hosts'][0]
|
||||
|
||||
- include: swift_rings_distribute.yml
|
||||
|
||||
- include: swift_rings_md5sum.yml
|
||||
|
||||
- include: swift_rings_post_distribution_check.yml
|
||||
when: >
|
||||
inventory_hostname == groups['swift_hosts'][0]
|
|
@ -0,0 +1,106 @@
|
|||
---
|
||||
# Copyright 2014, Rackspace US, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
- name: "Copy the swift_rings.py file"
|
||||
copy:
|
||||
src: swift_rings.py
|
||||
dest: "/etc/swift/scripts/swift_rings.py"
|
||||
owner: "{{ swift_system_user_name }}"
|
||||
group: "{{ swift_system_group_name }}"
|
||||
mode: "0700"
|
||||
tags:
|
||||
- swift-rings
|
||||
- swift-rings-scripts
|
||||
|
||||
- name: "Build ring-contents files"
|
||||
template:
|
||||
src: ring.contents.j2
|
||||
dest: "/etc/swift/scripts/{{ item.type }}.contents"
|
||||
owner: "{{ swift_system_user_name }}"
|
||||
group: "{{ swift_system_group_name }}"
|
||||
with_items:
|
||||
- { item: "{{ swift.account }}", port: "{{ swift_account_port }}", type: "account" }
|
||||
- { item: "{{ swift.container }}", port: "{{ swift_container_port}}", type: "container" }
|
||||
tags:
|
||||
- swift-rings
|
||||
- swift-rings-contents
|
||||
|
||||
- name: "Build ring-contents files for storage policies"
|
||||
template:
|
||||
src: ring.contents.j2
|
||||
dest: "/etc/swift/scripts/object-{{ item[0].policy.index }}.contents"
|
||||
owner: "{{ swift_system_user_name }}"
|
||||
group: "{{ swift_system_group_name }}"
|
||||
with_nested:
|
||||
- "{{ swift.storage_policies }}"
|
||||
- [{ type: 'object', port: "{{ swift_object_port }}" }]
|
||||
tags:
|
||||
- swift-rings
|
||||
- swift-rings-contents
|
||||
|
||||
- name: "build rings for account/container from contents files"
|
||||
command: "/usr/bin/python /etc/swift/scripts/swift_rings.py -f /etc/swift/scripts/{{ item }}.contents"
|
||||
with_items:
|
||||
- account
|
||||
- container
|
||||
when: swift_managed_regions is not defined
|
||||
sudo: yes
|
||||
sudo_user: "{{ swift_system_user_name }}"
|
||||
args:
|
||||
chdir: /etc/swift/ring_build_files/
|
||||
tags:
|
||||
- swift-rings
|
||||
- swift-rings-build
|
||||
|
||||
- name: "build rings for account/container from contents files"
|
||||
command: "/usr/bin/python /etc/swift/scripts/swift_rings.py -f /etc/swift/scripts/{{ item[0] }}.contents -r {{ item[1] }}"
|
||||
with_nested:
|
||||
- [ 'account', 'container' ]
|
||||
- "{{ swift_managed_regions }}"
|
||||
when: swift_managed_regions is defined
|
||||
sudo: yes
|
||||
sudo_user: "{{ swift_system_user_name }}"
|
||||
args:
|
||||
chdir: /etc/swift/ring_build_files/
|
||||
tags:
|
||||
- swift-rings
|
||||
- swift-rings-build
|
||||
|
||||
- name: "build rings for storage policies from contents files"
|
||||
command: "/usr/bin/python /etc/swift/scripts/swift_rings.py -f /etc/swift/scripts/object-{{ item.policy.index }}.contents"
|
||||
with_items:
|
||||
- "{{ swift.storage_policies }}"
|
||||
when: swift_managed_regions is not defined
|
||||
sudo: yes
|
||||
sudo_user: "{{ swift_system_user_name }}"
|
||||
args:
|
||||
chdir: /etc/swift/ring_build_files/
|
||||
tags:
|
||||
- swift-rings
|
||||
- swift-rings-build
|
||||
|
||||
- name: "build rings for storage policies from contents files"
|
||||
command: "/usr/bin/python /etc/swift/scripts/swift_rings.py -f /etc/swift/scripts/object-{{ item[0].policy.index }}.contents -r {{ item[1] }}"
|
||||
with_nested:
|
||||
- "{{ swift.storage_policies }}"
|
||||
- "{{ swift_managed_regions }}"
|
||||
when: swift_managed_regions is defined
|
||||
sudo: yes
|
||||
sudo_user: "{{ swift_system_user_name }}"
|
||||
args:
|
||||
chdir: /etc/swift/ring_build_files/
|
||||
tags:
|
||||
- swift-rings
|
||||
- swift-rings-build
|
|
@ -0,0 +1,43 @@
|
|||
---
|
||||
# Copyright 2014, Rackspace US, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
- name: Get md5sum of local builder files
|
||||
shell: |
|
||||
cat /etc/swift/ring_build_files/*.builder 2>/dev/null | md5sum | cut -d " " -f1
|
||||
register: md5sum
|
||||
tags:
|
||||
- swift-ring-check
|
||||
|
||||
- name: Get empty md5sum
|
||||
shell: |
|
||||
echo -n | md5sum | cut -d " " -f1
|
||||
register: empty_md5sum
|
||||
tags:
|
||||
- swift-ring-check
|
||||
|
||||
# Fail if the remote hosts builder files is not empty AND
|
||||
# does not match the md5sum of the local host.
|
||||
- name: Verify md5sum of builder files
|
||||
fail:
|
||||
msg: >
|
||||
"The builder files on the remote host {{ item }}:{{ hostvars[item]['builder_md5sum'] }}
|
||||
do not match {{ inventory_hostname }}:{{ md5sum.stdout }}
|
||||
and are not empty on the remote host"
|
||||
when: >
|
||||
("{{ hostvars[item]['builder_md5sum'] }}" != "{{ empty_md5sum.stdout }}") and
|
||||
("{{ hostvars[item]['builder_md5sum'] }}" != "{{ md5sum.stdout }}")
|
||||
with_items: groups['swift_all'] + groups['swift_remote_all']
|
||||
tags:
|
||||
- swift-ring-check
|
|
@ -0,0 +1,38 @@
|
|||
---
|
||||
# Copyright 2014, Rackspace US, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
- name: "Get swift rings"
|
||||
synchronize:
|
||||
src: "/etc/swift/ring_build_files/"
|
||||
mode: push
|
||||
dest: "/etc/swift/"
|
||||
set_remote_user: False
|
||||
delegate_to: "{{ groups['swift_hosts'][0] }}"
|
||||
sudo: yes
|
||||
sudo_user: "{{ swift_system_user_name }}"
|
||||
tags:
|
||||
- swift-get-rings
|
||||
|
||||
- name: "Get swift rings for backup in ring_build_files directory"
|
||||
synchronize:
|
||||
src: "/etc/swift/ring_build_files/"
|
||||
mode: push
|
||||
dest: "/etc/swift/ring_build_files/"
|
||||
set_remote_user: False
|
||||
delegate_to: "{{ groups['swift_hosts'][0] }}"
|
||||
sudo: yes
|
||||
sudo_user: "{{ swift_system_user_name }}"
|
||||
tags:
|
||||
- swift-get-rings
|
|
@ -0,0 +1,27 @@
|
|||
---
|
||||
# Copyright 2014, Rackspace US, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
- name: Get md5sum of builder files
|
||||
shell: |
|
||||
cat /etc/swift/*.builder 2>/dev/null | md5sum | cut -d " " -f1
|
||||
register: md5sum
|
||||
tags:
|
||||
- swift-md5sum
|
||||
|
||||
- name: Register a fact for the md5sum
|
||||
set_fact:
|
||||
builder_md5sum: "{{ md5sum.stdout }}"
|
||||
tags:
|
||||
- swift-md5sum
|
|
@ -0,0 +1,99 @@
|
|||
---
|
||||
# Copyright 2014, Rackspace US, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
- name: Get md5sum of local builder files
|
||||
shell: |
|
||||
cat /etc/swift/ring_build_files/*.builder 2>/dev/null | md5sum | cut -d " " -f1
|
||||
register: md5sum
|
||||
tags:
|
||||
- swift-ring-check
|
||||
|
||||
# Fail if the remote hosts builder files don't match
|
||||
# As this is post sync all should match and not be empty
|
||||
- name: Verify md5sum of builder files
|
||||
fail:
|
||||
msg: >
|
||||
"The builder files on the remote host {{ item }}:{{ hostvars[item]['builder_md5sum'] }}
|
||||
do not match {{ inventory_hostname }}:{{ md5sum.stdout }}"
|
||||
when: hostvars[item]['builder_md5sum'] != md5sum.stdout
|
||||
with_items: groups['swift_all'] + groups['swift_remote_all']
|
||||
tags:
|
||||
- swift-ring-check
|
||||
|
||||
- name: "Copy the swift_rings_check.py file"
|
||||
copy:
|
||||
src: swift_rings_check.py
|
||||
dest: "/etc/swift/scripts/swift_rings_check.py"
|
||||
owner: "{{ swift_system_user_name }}"
|
||||
group: "{{ swift_system_group_name }}"
|
||||
mode: "0700"
|
||||
tags:
|
||||
- swift-rings
|
||||
- swift-rings-scripts
|
||||
|
||||
- name: "Ensure contents file matches ring after ring sync for account/container"
|
||||
command: "/usr/bin/python /etc/swift/scripts/swift_rings_check.py -f /etc/swift/scripts/{{ item }}.contents"
|
||||
with_items:
|
||||
- account
|
||||
- container
|
||||
when: swift_managed_regions is not defined
|
||||
sudo: yes
|
||||
sudo_user: "{{ swift_system_user_name }}"
|
||||
args:
|
||||
chdir: /etc/swift/ring_build_files/
|
||||
tags:
|
||||
- swift-rings
|
||||
- swift-rings-check
|
||||
|
||||
- name: "Ensure contents file matches ring after ring sync for account/container"
|
||||
command: "/usr/bin/python /etc/swift/scripts/swift_rings_check.py -f /etc/swift/scripts/{{ item[0] }}.contents -r {{ item[1] }}"
|
||||
with_nested:
|
||||
- [ 'account', 'container' ]
|
||||
- "{{ swift_managed_regions }}"
|
||||
when: swift_managed_regions is defined
|
||||
sudo: yes
|
||||
sudo_user: "{{ swift_system_user_name }}"
|
||||
args:
|
||||
chdir: /etc/swift/ring_build_files/
|
||||
tags:
|
||||
- swift-rings
|
||||
- swift-rings-check
|
||||
|
||||
- name: "Ensure contents file matches ring after ring sync for storage policies"
|
||||
command: "/usr/bin/python /etc/swift/scripts/swift_rings_check.py -f /etc/swift/scripts/object-{{ item.policy.index }}.contents"
|
||||
with_items:
|
||||
- "{{ swift.storage_policies }}"
|
||||
when: swift_managed_regions is not defined
|
||||
sudo: yes
|
||||
sudo_user: "{{ swift_system_user_name }}"
|
||||
args:
|
||||
chdir: /etc/swift/ring_build_files/
|
||||
tags:
|
||||
- swift-rings
|
||||
- swift-rings-check
|
||||
|
||||
- name: "Ensure contents file matches ring after ring sync for storage policies"
|
||||
command: "/usr/bin/python /etc/swift/scripts/swift_rings_check.py -f /etc/swift/scripts/object-{{ item[0].policy.index }}.contents -r {{ item[1] }}"
|
||||
with_nested:
|
||||
- "{{ swift.storage_policies }}"
|
||||
- "{{ swift_managed_regions }}"
|
||||
when: swift_managed_regions is defined
|
||||
sudo: yes
|
||||
sudo_user: "{{ swift_system_user_name }}"
|
||||
args:
|
||||
chdir: /etc/swift/ring_build_files/
|
||||
tags:
|
||||
- swift-rings
|
||||
- swift-rings-check
|
|
@ -0,0 +1,30 @@
|
|||
---
|
||||
# Copyright 2014, Rackspace US, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
- name: "Ensure services are started"
|
||||
service:
|
||||
name: "{{ item }}"
|
||||
state: "started"
|
||||
pattern: "{{ item }}"
|
||||
with_items: swift_account_program_names + swift_container_program_names + swift_object_program_names
|
||||
when: inventory_hostname in groups['swift_hosts']
|
||||
|
||||
- name: "Ensure services are started"
|
||||
service:
|
||||
name: "{{ item }}"
|
||||
state: "started"
|
||||
pattern: "{{ item }}"
|
||||
with_items: swift_proxy_program_names
|
||||
when: inventory_hostname in groups['swift_proxy']
|
|
@ -0,0 +1,100 @@
|
|||
{### Check if this is an object storage policy #}
|
||||
{% if item[1] is defined %}
|
||||
{% set port = item[1]['port'] %}
|
||||
{% set type = item[1]['type'] %}
|
||||
{% set item = item[0]['policy'] %}
|
||||
{### If the index is 0 then it needs to be object without index #}
|
||||
{% if item.index == 0 %}
|
||||
{% set builder_file = type %}
|
||||
{% else %}
|
||||
{% set builder_file = type + '-' + item.index|string %}
|
||||
{% endif %}
|
||||
{% set name = item.name %}
|
||||
{### Otherwise this should be account or container rings #}
|
||||
{### Make the port/type/item/builder_file/name vals uniform across rings #}
|
||||
{% elif item.port is defined %}
|
||||
{% set port = item.port %}
|
||||
{% set type = item.type %}
|
||||
{% set item = item.item %}
|
||||
{% set builder_file = type %}
|
||||
{% set name = type %}
|
||||
{% endif %}
|
||||
{### Lets get the min_part_hours, part_power and repl_number vals #}
|
||||
{% set min_part_hours = item.min_part_hours | default(swift.min_part_hours | default(swift_default_min_part_hours)) %}
|
||||
{% set part_power = item.part_power | default(swift.part_power) %}
|
||||
{% if (item.policy_type is defined) and (item.policy_type == "erasure_coding") %}
|
||||
{% set repl_number = item.ec_num_data_fragments + item.ec_num_parity_fragments %}
|
||||
{% else %}
|
||||
{% set repl_number = item.repl_number | default(swift.repl_number | default(swift_default_replication_number)) %}
|
||||
{% endif %}
|
||||
{### Create the builder dict #}
|
||||
{% set builder = {} %}
|
||||
{### This is a hacky way of updating the builder dict #}
|
||||
{% set _update = builder.update({'min_part_hours':min_part_hours}) %}
|
||||
{% set _update = builder.update({'repl_number':repl_number}) %}
|
||||
{% set _update = builder.update({'part_power':part_power}) %}
|
||||
{% set _update = builder.update({'builder_file':builder_file}) %}
|
||||
{### Now we need to add the drives #}
|
||||
{### Create an update the builder dict to have drives as an empty list #}
|
||||
{% set _update = builder.update({'drives':[]}) %}
|
||||
{### Lets get the default groups for drives and find the default storage_policy #}
|
||||
{% set def_groups = [ 'account', 'container' ] %}
|
||||
{% for policy in swift.storage_policies %}
|
||||
{% if policy.policy.default is defined and policy.policy.default == True %}
|
||||
{% set _update = def_groups.append(policy.policy.name) %}
|
||||
{% endif %}
|
||||
{% endfor %}
|
||||
{### Loop through the swift_hosts #}
|
||||
{% for host in groups['swift_hosts'] %}
|
||||
{### Set the default storage_ip #}
|
||||
{% set def_storage_ip = hostvars[host]['ansible_ssh_host'] %}
|
||||
{% if swift.storage_network is defined %}
|
||||
{% set storage_bridge = 'ansible_' + swift.storage_network|replace('-', '_') %}
|
||||
{% set def_storage_ip = hostvars[host][storage_bridge]['ipv4']['address'] | default(hostvars[host]['ansible_ssh_host']) %}
|
||||
{% endif %}
|
||||
{### Set the default replication_ip #}
|
||||
{% set def_repl_ip = def_storage_ip %}
|
||||
{% if swift.replication_network is defined %}
|
||||
{% set repl_bridge = 'ansible_' + swift.replication_network|replace('-', '_') %}
|
||||
{% set def_repl_ip = hostvars[host][repl_bridge]['ipv4']['address'] | default(def_storage_ip) %}
|
||||
{% endif %}
|
||||
{### Default swift_vars to an empty dict if not defined #}
|
||||
{### swift_vars needs to exist for swift_vars[setting] checks to wrok #}
|
||||
{% if hostvars[host]['swift_vars'] is defined %}
|
||||
{% set swift_vars = hostvars[host]['swift_vars'] %}
|
||||
{% else %}
|
||||
{% set swift_vars = {} %}
|
||||
{% endif %}
|
||||
{### Get the drives use swift global as default #}
|
||||
{% set drives = swift_vars.drives | default(swift.drives | default([])) %}
|
||||
{### Loop through the drives #}
|
||||
{% for drive in drives %}
|
||||
{### Check if groups is defined per host or drive #}
|
||||
{% set groups = drive.groups | default(swift_vars.groups | default(swift.groups | default(def_groups))) %}
|
||||
{### Only build the device if it is part of the group we're building the ring for #}
|
||||
{% if name in groups %}
|
||||
{### Build an empty device which we'll update with the appropriate details #}
|
||||
{% set device = {} %}
|
||||
{% set weight = drive.weight | default(swift_vars.weight | default(swift.weight | default(swift_default_drive_weight))) %}
|
||||
{% set region = drive.region | default(swift_vars.region | default(swift.region | default(swift_default_host_region))) %}
|
||||
{% set zone = drive.zone | default(swift_vars.zone | default(swift.zone | default(swift_default_host_zone))) %}
|
||||
{% set repl_ip = drive.repl_ip | default(swift_vars.repl_ip | default(def_repl_ip)) %}
|
||||
{% set repl_port = drive.repl_port | default(swift_vars.repl_port | default(port)) %}
|
||||
{% set storage_ip = drive.storage_ip | default(swift_vars.storage_ip | default(def_storage_ip)) %}
|
||||
{% set storage_port = drive.storage_port | default(swift_vars.storage_port | default(port)) %}
|
||||
{### Update the device with the appropriate values #}
|
||||
{% set _update = device.update({'device':drive.name}) %}
|
||||
{% set _update = device.update({'weight': weight}) %}
|
||||
{% set _update = device.update({'region': region}) %}
|
||||
{% set _update = device.update({'zone': zone}) %}
|
||||
{% set _update = device.update({'repl_ip': repl_ip}) %}
|
||||
{% set _update = device.update({'repl_port': repl_port|int}) %}
|
||||
{% set _update = device.update({'ip': storage_ip}) %}
|
||||
{% set _update = device.update({'port': storage_port|int}) %}
|
||||
{### Append the device to the drives list of the builder dict #}
|
||||
{% set _update = builder.drives.append(device) %}
|
||||
{% endif %}
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
{### Output the builder file #}
|
||||
{{ builder | to_nice_json }}
|
Loading…
Reference in New Issue