Retire Project
Removed everything except README.md Change-Id: I4980005d4a75cf7bfb031979746dd0dc79e68b04 Signed-off-by: Bilal Baqar <mbilalbaqar@gmail.com>
This commit is contained in:
parent
f7946e5254
commit
f878d765fe
202
LICENSE
202
LICENSE
|
@ -1,202 +0,0 @@
|
|||
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
APPENDIX: How to apply the Apache License to your work.
|
||||
|
||||
To apply the Apache License to your work, attach the following
|
||||
boilerplate notice, with the fields enclosed by brackets "[]"
|
||||
replaced with your own identifying information. (Don't include
|
||||
the brackets!) The text should be enclosed in the appropriate
|
||||
comment syntax for the file format. We also recommend that a
|
||||
file or class name and description of purpose be included on the
|
||||
same "printed page" as the copyright notice for easier
|
||||
identification within third-party archives.
|
||||
|
||||
Copyright [yyyy] [name of copyright owner]
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
31
Makefile
31
Makefile
|
@ -1,31 +0,0 @@
|
|||
#!/usr/bin/make
|
||||
PYTHON := /usr/bin/env python
|
||||
|
||||
virtualenv:
|
||||
virtualenv .venv
|
||||
.venv/bin/pip install flake8 nose coverage mock pyyaml netifaces \
|
||||
netaddr jinja2 pyflakes pep8 six pbr funcsigs psutil
|
||||
|
||||
lint: virtualenv
|
||||
.venv/bin/flake8 --exclude hooks/charmhelpers hooks unit_tests tests --ignore E402
|
||||
@charm proof
|
||||
|
||||
unit_test: virtualenv
|
||||
@echo Starting tests...
|
||||
@.venv/bin/nosetests --nologcapture --with-coverage unit_tests
|
||||
|
||||
bin/charm_helpers_sync.py:
|
||||
@mkdir -p bin
|
||||
@bzr cat lp:charm-helpers/tools/charm_helpers_sync/charm_helpers_sync.py \
|
||||
> bin/charm_helpers_sync.py
|
||||
|
||||
sync: bin/charm_helpers_sync.py
|
||||
@$(PYTHON) bin/charm_helpers_sync.py -c charm-helpers-sync.yaml
|
||||
|
||||
publish: lint unit_test
|
||||
bzr push lp:charms/plumgrid-director
|
||||
bzr push lp:charms/trusty/plumgrid-director
|
||||
|
||||
test:
|
||||
@echo Starting Amulet tests...
|
||||
@juju test -v -p AMULET_HTTP_PROXY,AMULET_OS_VIP --timeout 2700
|
66
README.md
66
README.md
|
@ -1,62 +1,6 @@
|
|||
# About the PLUMgrid Platform
|
||||
This project is no longer maintained.
|
||||
|
||||
The [PLUMgrid Platform](http://www.plumgrid.com/technology/plumgrid-platform/) is a software-only solution that provides a rich set of distributed network functions such as routers, switches, NAT, IPAM, DHCP, and it also supports security policies, end-to-end encryption, and third party Layer 4-7 service insertion.
|
||||
|
||||
## About Plumgrid Director
|
||||
|
||||
The Director is the brain of the PLUMgrid Platform. It is responsible for coordinating and managing all the other platform components. Based on PLUMgrid's distributed system architecture, it provides built-in high availability and scaling. The Director allows you to create Virtual Domains on a per tenant or application basis.
|
||||
|
||||
|
||||
# Overview
|
||||
|
||||
Once deployed this charm performs the configurations required for a PLUMgrid Director and starts the essential services on the node.
|
||||
|
||||
|
||||
# Usage
|
||||
|
||||
Instructions on using the charm:
|
||||
|
||||
juju deploy neutron-api
|
||||
juju deploy neutron-api-plumgrid
|
||||
juju deploy plumgrid-director
|
||||
|
||||
juju add-relation neutron-api neutron-api-plumgrid
|
||||
|
||||
For plumgrid-director to work make the configuration in the neutron-api and neutron-api-plumgrid charms as specified in the configuration section below.
|
||||
|
||||
# Known Limitations and Issues
|
||||
|
||||
This charm currently doesn't support Ubuntu 16.04.
|
||||
|
||||
# Configuration
|
||||
|
||||
Example Config
|
||||
|
||||
plumgrid-director:
|
||||
plumgrid-virtual-ip: "192.168.100.250"
|
||||
install_sources: 'ppa:plumgrid-team/stable'
|
||||
install_keys: 'null'
|
||||
neutron-api-plumgrid:
|
||||
install_sources: 'ppa:plumgrid-team/stable'
|
||||
install_keys: 'null'
|
||||
enable-metadata: True
|
||||
neutron-api:
|
||||
neutron-plugin: "plumgrid"
|
||||
plumgrid-virtual-ip: "192.168.100.250"
|
||||
|
||||
Provide the virtual IP you want PLUMgrid GUI to be accessible.
|
||||
Make sure that it is the same IP specified in the neutron-api charm configuration for PLUMgrid.
|
||||
The virtual IP passed on in the neutron-api charm has to be same as the one passed in the plumgrid-director charm.
|
||||
Provide the source repo path for PLUMgrid Debs in 'install_sources' and the corresponding keys in 'install_keys'.
|
||||
|
||||
You can access the PG Console at https://192.168.100.250
|
||||
|
||||
In order to configure networking, PLUMgrid License needs to be posted.
|
||||
|
||||
juju set plumgrid-director plumgrid-license-key="$LICENSE_KEY"
|
||||
|
||||
# Contact Information
|
||||
|
||||
Bilal Baqar <bbaqar@plumgrid.com>
|
||||
Javeria Khan <javeriak@plumgrid.com>
|
||||
Junaid Ali <junaidali@plumgrid.com>
|
||||
The contents of this repository are still available in the Git
|
||||
source code management system. To see the contents of this
|
||||
repository before it reached its end of life, please check out the
|
||||
previous commit with "git checkout HEAD^1".
|
||||
|
|
|
@ -1,8 +0,0 @@
|
|||
restart-pg:
|
||||
description: Restart the plumgrid-director unit's service.
|
||||
sapi-post-ips:
|
||||
description: Post PLUMgrid nodes IPs to Solutions API server.
|
||||
sapi-post-zone-info:
|
||||
description: Post PLUMgrid Zone info to Solutions API server.
|
||||
sapi-post-license:
|
||||
description: Post PLUMgrid License to Solutions API server.
|
|
@ -1,65 +0,0 @@
|
|||
#!/usr/bin/python
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.append('hooks/')
|
||||
|
||||
from charmhelpers.core.hookenv import action_fail
|
||||
from pg_dir_utils import (
|
||||
restart_pg,
|
||||
sapi_post_zone_info,
|
||||
sapi_post_license,
|
||||
sapi_post_ips
|
||||
)
|
||||
|
||||
|
||||
def restart_pg(args):
|
||||
"""
|
||||
Restart PLUMgrid services.
|
||||
"""
|
||||
restart_pg()
|
||||
|
||||
|
||||
def post_ips(args):
|
||||
"""
|
||||
POST PLUMgrid nodes IPs to solutions api server.
|
||||
"""
|
||||
sapi_post_ips()
|
||||
|
||||
|
||||
def post_zone_info(args):
|
||||
"""
|
||||
POST PLUMgrid zone information to solutions api server
|
||||
"""
|
||||
sapi_post_zone_info()
|
||||
|
||||
|
||||
def post_license(args):
|
||||
"""
|
||||
POST PLUMgrid License key to solutions api server
|
||||
"""
|
||||
sapi_post_license()
|
||||
|
||||
|
||||
# A dictionary of all the defined actions to callables (which take
|
||||
# parsed arguments).
|
||||
ACTIONS = {"restart-pg": restart_pg, "sapi-post-ips": post_ips, "sapi-post-zone-info": post_zone_info,
|
||||
"sapi-post-license": post_license}
|
||||
|
||||
|
||||
def main(args):
|
||||
action_name = os.path.basename(args[0])
|
||||
try:
|
||||
action = ACTIONS[action_name]
|
||||
except KeyError:
|
||||
return "Action %s undefined" % action_name
|
||||
else:
|
||||
try:
|
||||
action(args)
|
||||
except Exception as e:
|
||||
action_fail(str(e))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv))
|
|
@ -1 +0,0 @@
|
|||
actions.py
|
|
@ -1 +0,0 @@
|
|||
actions.py
|
|
@ -1 +0,0 @@
|
|||
actions.py
|
|
@ -1 +0,0 @@
|
|||
actions.py
|
|
@ -1,253 +0,0 @@
|
|||
#!/usr/bin/python
|
||||
|
||||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
# Authors:
|
||||
# Adam Gandelman <adamg@ubuntu.com>
|
||||
|
||||
import logging
|
||||
import optparse
|
||||
import os
|
||||
import subprocess
|
||||
import shutil
|
||||
import sys
|
||||
import tempfile
|
||||
import yaml
|
||||
from fnmatch import fnmatch
|
||||
|
||||
import six
|
||||
|
||||
CHARM_HELPERS_BRANCH = 'lp:charm-helpers'
|
||||
|
||||
|
||||
def parse_config(conf_file):
|
||||
if not os.path.isfile(conf_file):
|
||||
logging.error('Invalid config file: %s.' % conf_file)
|
||||
return False
|
||||
return yaml.load(open(conf_file).read())
|
||||
|
||||
|
||||
def clone_helpers(work_dir, branch):
|
||||
dest = os.path.join(work_dir, 'charm-helpers')
|
||||
logging.info('Checking out %s to %s.' % (branch, dest))
|
||||
cmd = ['bzr', 'checkout', '--lightweight', branch, dest]
|
||||
subprocess.check_call(cmd)
|
||||
return dest
|
||||
|
||||
|
||||
def _module_path(module):
|
||||
return os.path.join(*module.split('.'))
|
||||
|
||||
|
||||
def _src_path(src, module):
|
||||
return os.path.join(src, 'charmhelpers', _module_path(module))
|
||||
|
||||
|
||||
def _dest_path(dest, module):
|
||||
return os.path.join(dest, _module_path(module))
|
||||
|
||||
|
||||
def _is_pyfile(path):
|
||||
return os.path.isfile(path + '.py')
|
||||
|
||||
|
||||
def ensure_init(path):
|
||||
'''
|
||||
ensure directories leading up to path are importable, omitting
|
||||
parent directory, eg path='/hooks/helpers/foo'/:
|
||||
hooks/
|
||||
hooks/helpers/__init__.py
|
||||
hooks/helpers/foo/__init__.py
|
||||
'''
|
||||
for d, dirs, files in os.walk(os.path.join(*path.split('/')[:2])):
|
||||
_i = os.path.join(d, '__init__.py')
|
||||
if not os.path.exists(_i):
|
||||
logging.info('Adding missing __init__.py: %s' % _i)
|
||||
open(_i, 'wb').close()
|
||||
|
||||
|
||||
def sync_pyfile(src, dest):
|
||||
src = src + '.py'
|
||||
src_dir = os.path.dirname(src)
|
||||
logging.info('Syncing pyfile: %s -> %s.' % (src, dest))
|
||||
if not os.path.exists(dest):
|
||||
os.makedirs(dest)
|
||||
shutil.copy(src, dest)
|
||||
if os.path.isfile(os.path.join(src_dir, '__init__.py')):
|
||||
shutil.copy(os.path.join(src_dir, '__init__.py'),
|
||||
dest)
|
||||
ensure_init(dest)
|
||||
|
||||
|
||||
def get_filter(opts=None):
|
||||
opts = opts or []
|
||||
if 'inc=*' in opts:
|
||||
# do not filter any files, include everything
|
||||
return None
|
||||
|
||||
def _filter(dir, ls):
|
||||
incs = [opt.split('=').pop() for opt in opts if 'inc=' in opt]
|
||||
_filter = []
|
||||
for f in ls:
|
||||
_f = os.path.join(dir, f)
|
||||
|
||||
if not os.path.isdir(_f) and not _f.endswith('.py') and incs:
|
||||
if True not in [fnmatch(_f, inc) for inc in incs]:
|
||||
logging.debug('Not syncing %s, does not match include '
|
||||
'filters (%s)' % (_f, incs))
|
||||
_filter.append(f)
|
||||
else:
|
||||
logging.debug('Including file, which matches include '
|
||||
'filters (%s): %s' % (incs, _f))
|
||||
elif (os.path.isfile(_f) and not _f.endswith('.py')):
|
||||
logging.debug('Not syncing file: %s' % f)
|
||||
_filter.append(f)
|
||||
elif (os.path.isdir(_f) and not
|
||||
os.path.isfile(os.path.join(_f, '__init__.py'))):
|
||||
logging.debug('Not syncing directory: %s' % f)
|
||||
_filter.append(f)
|
||||
return _filter
|
||||
return _filter
|
||||
|
||||
|
||||
def sync_directory(src, dest, opts=None):
|
||||
if os.path.exists(dest):
|
||||
logging.debug('Removing existing directory: %s' % dest)
|
||||
shutil.rmtree(dest)
|
||||
logging.info('Syncing directory: %s -> %s.' % (src, dest))
|
||||
|
||||
shutil.copytree(src, dest, ignore=get_filter(opts))
|
||||
ensure_init(dest)
|
||||
|
||||
|
||||
def sync(src, dest, module, opts=None):
|
||||
|
||||
# Sync charmhelpers/__init__.py for bootstrap code.
|
||||
sync_pyfile(_src_path(src, '__init__'), dest)
|
||||
|
||||
# Sync other __init__.py files in the path leading to module.
|
||||
m = []
|
||||
steps = module.split('.')[:-1]
|
||||
while steps:
|
||||
m.append(steps.pop(0))
|
||||
init = '.'.join(m + ['__init__'])
|
||||
sync_pyfile(_src_path(src, init),
|
||||
os.path.dirname(_dest_path(dest, init)))
|
||||
|
||||
# Sync the module, or maybe a .py file.
|
||||
if os.path.isdir(_src_path(src, module)):
|
||||
sync_directory(_src_path(src, module), _dest_path(dest, module), opts)
|
||||
elif _is_pyfile(_src_path(src, module)):
|
||||
sync_pyfile(_src_path(src, module),
|
||||
os.path.dirname(_dest_path(dest, module)))
|
||||
else:
|
||||
logging.warn('Could not sync: %s. Neither a pyfile or directory, '
|
||||
'does it even exist?' % module)
|
||||
|
||||
|
||||
def parse_sync_options(options):
|
||||
if not options:
|
||||
return []
|
||||
return options.split(',')
|
||||
|
||||
|
||||
def extract_options(inc, global_options=None):
|
||||
global_options = global_options or []
|
||||
if global_options and isinstance(global_options, six.string_types):
|
||||
global_options = [global_options]
|
||||
if '|' not in inc:
|
||||
return (inc, global_options)
|
||||
inc, opts = inc.split('|')
|
||||
return (inc, parse_sync_options(opts) + global_options)
|
||||
|
||||
|
||||
def sync_helpers(include, src, dest, options=None):
|
||||
if not os.path.isdir(dest):
|
||||
os.makedirs(dest)
|
||||
|
||||
global_options = parse_sync_options(options)
|
||||
|
||||
for inc in include:
|
||||
if isinstance(inc, str):
|
||||
inc, opts = extract_options(inc, global_options)
|
||||
sync(src, dest, inc, opts)
|
||||
elif isinstance(inc, dict):
|
||||
# could also do nested dicts here.
|
||||
for k, v in six.iteritems(inc):
|
||||
if isinstance(v, list):
|
||||
for m in v:
|
||||
inc, opts = extract_options(m, global_options)
|
||||
sync(src, dest, '%s.%s' % (k, inc), opts)
|
||||
|
||||
if __name__ == '__main__':
|
||||
parser = optparse.OptionParser()
|
||||
parser.add_option('-c', '--config', action='store', dest='config',
|
||||
default=None, help='helper config file')
|
||||
parser.add_option('-D', '--debug', action='store_true', dest='debug',
|
||||
default=False, help='debug')
|
||||
parser.add_option('-b', '--branch', action='store', dest='branch',
|
||||
help='charm-helpers bzr branch (overrides config)')
|
||||
parser.add_option('-d', '--destination', action='store', dest='dest_dir',
|
||||
help='sync destination dir (overrides config)')
|
||||
(opts, args) = parser.parse_args()
|
||||
|
||||
if opts.debug:
|
||||
logging.basicConfig(level=logging.DEBUG)
|
||||
else:
|
||||
logging.basicConfig(level=logging.INFO)
|
||||
|
||||
if opts.config:
|
||||
logging.info('Loading charm helper config from %s.' % opts.config)
|
||||
config = parse_config(opts.config)
|
||||
if not config:
|
||||
logging.error('Could not parse config from %s.' % opts.config)
|
||||
sys.exit(1)
|
||||
else:
|
||||
config = {}
|
||||
|
||||
if 'branch' not in config:
|
||||
config['branch'] = CHARM_HELPERS_BRANCH
|
||||
if opts.branch:
|
||||
config['branch'] = opts.branch
|
||||
if opts.dest_dir:
|
||||
config['destination'] = opts.dest_dir
|
||||
|
||||
if 'destination' not in config:
|
||||
logging.error('No destination dir. specified as option or config.')
|
||||
sys.exit(1)
|
||||
|
||||
if 'include' not in config:
|
||||
if not args:
|
||||
logging.error('No modules to sync specified as option or config.')
|
||||
sys.exit(1)
|
||||
config['include'] = []
|
||||
[config['include'].append(a) for a in args]
|
||||
|
||||
sync_options = None
|
||||
if 'options' in config:
|
||||
sync_options = config['options']
|
||||
tmpd = tempfile.mkdtemp()
|
||||
try:
|
||||
checkout = clone_helpers(tmpd, config['branch'])
|
||||
sync_helpers(config['include'], checkout, config['destination'],
|
||||
options=sync_options)
|
||||
except Exception as e:
|
||||
logging.error("Could not sync: %s" % e)
|
||||
raise e
|
||||
finally:
|
||||
logging.debug('Cleaning up %s' % tmpd)
|
||||
shutil.rmtree(tmpd)
|
|
@ -1,12 +0,0 @@
|
|||
branch: lp:charm-helpers
|
||||
destination: hooks/charmhelpers
|
||||
include:
|
||||
- core
|
||||
- fetch
|
||||
- contrib.amulet
|
||||
- contrib.hahelpers
|
||||
- contrib.network
|
||||
- contrib.openstack
|
||||
- contrib.python
|
||||
- contrib.storage
|
||||
- payload
|
75
config.yaml
75
config.yaml
|
@ -1,75 +0,0 @@
|
|||
options:
|
||||
plumgrid-virtual-ip:
|
||||
default: 192.168.100.250
|
||||
type: string
|
||||
description: IP address of the Director's Management interface. Same IP can be used to access PG Console.
|
||||
plumgrid-username:
|
||||
default: plumgrid
|
||||
type: string
|
||||
description: Username to access PLUMgrid Director
|
||||
plumgrid-password:
|
||||
default: plumgrid
|
||||
type: string
|
||||
description: Password to access PLUMgrid Director
|
||||
lcm-ssh-key:
|
||||
default: 'null'
|
||||
type: string
|
||||
description: Public SSH key of PLUMgrid LCM which is running PG-Tools.
|
||||
mgmt-interface:
|
||||
type: string
|
||||
description: The interface connected to PLUMgrid Managment network.
|
||||
fabric-interfaces:
|
||||
default: 'MANAGEMENT'
|
||||
type: string
|
||||
description: |
|
||||
Interfaces that will provide fabric connectivity on the director nodes.
|
||||
Provided in form of json in a string. Default value is MANAGEMENT which
|
||||
will configure the management interface as the fabric interface on each
|
||||
director.
|
||||
network-device-mtu:
|
||||
type: string
|
||||
default: '1580'
|
||||
description: The MTU size for interfaces managed by director.
|
||||
install_sources:
|
||||
default: 'ppa:plumgrid-team/stable'
|
||||
type: string
|
||||
description: Provide the install source from where to install the PLUMgrid debs.
|
||||
install_keys:
|
||||
default: null
|
||||
type: string
|
||||
description: Provide the respective keys of the install sources.
|
||||
plumgrid-build:
|
||||
default: 'latest'
|
||||
type: string
|
||||
description: |
|
||||
Provide the build version of PLUMgrid packages that needs to be installed
|
||||
iovisor-build:
|
||||
default: 'latest'
|
||||
type: string
|
||||
description: |
|
||||
Provide the build version of iovisor package that needs to be installed
|
||||
plumgrid-license-key:
|
||||
default: null
|
||||
type: string
|
||||
description: Provide the PLUMgrid ONS License key.
|
||||
opsvm-ip:
|
||||
default: 127.0.0.1
|
||||
type: string
|
||||
description: IP address of the PLUMgrid Operations VM Management interface.
|
||||
lcm-ip:
|
||||
type: string
|
||||
description: IP used by Solutions API to get/post cloud information.
|
||||
sapi-port:
|
||||
default: 8099
|
||||
type: int
|
||||
description: Port used by Solutions API to get/post cloud information.
|
||||
sapi-zone:
|
||||
default: pgzone
|
||||
type: string
|
||||
description: Zone name used by Solutions API to get/post cloud information.
|
||||
openstack-release:
|
||||
default: kilo
|
||||
type: string
|
||||
description: |
|
||||
OpenStack release to determine solution version that will be posted to
|
||||
Solutions API server.
|
16
copyright
16
copyright
|
@ -1,16 +0,0 @@
|
|||
Format: http://www.debian.org/doc/packaging-manuals/copyright-format/1.0
|
||||
|
||||
Files: *
|
||||
Copyright: 2016, PLUMgrid Inc., All Rights Reserved.
|
||||
License: Apache-2.0
|
||||
Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
not use this file except in compliance with the License. You may obtain
|
||||
a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
License for the specific language governing permissions and limitations
|
||||
under the License.
|
|
@ -1,38 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
# Bootstrap charm-helpers, installing its dependencies if necessary using
|
||||
# only standard libraries.
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
try:
|
||||
import six # flake8: noqa
|
||||
except ImportError:
|
||||
if sys.version_info.major == 2:
|
||||
subprocess.check_call(['apt-get', 'install', '-y', 'python-six'])
|
||||
else:
|
||||
subprocess.check_call(['apt-get', 'install', '-y', 'python3-six'])
|
||||
import six # flake8: noqa
|
||||
|
||||
try:
|
||||
import yaml # flake8: noqa
|
||||
except ImportError:
|
||||
if sys.version_info.major == 2:
|
||||
subprocess.check_call(['apt-get', 'install', '-y', 'python-yaml'])
|
||||
else:
|
||||
subprocess.check_call(['apt-get', 'install', '-y', 'python3-yaml'])
|
||||
import yaml # flake8: noqa
|
|
@ -1,15 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
|
@ -1,15 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
|
@ -1,95 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
import amulet
|
||||
import os
|
||||
import six
|
||||
|
||||
|
||||
class AmuletDeployment(object):
|
||||
"""Amulet deployment.
|
||||
|
||||
This class provides generic Amulet deployment and test runner
|
||||
methods.
|
||||
"""
|
||||
|
||||
def __init__(self, series=None):
|
||||
"""Initialize the deployment environment."""
|
||||
self.series = None
|
||||
|
||||
if series:
|
||||
self.series = series
|
||||
self.d = amulet.Deployment(series=self.series)
|
||||
else:
|
||||
self.d = amulet.Deployment()
|
||||
|
||||
def _add_services(self, this_service, other_services):
|
||||
"""Add services.
|
||||
|
||||
Add services to the deployment where this_service is the local charm
|
||||
that we're testing and other_services are the other services that
|
||||
are being used in the local amulet tests.
|
||||
"""
|
||||
if this_service['name'] != os.path.basename(os.getcwd()):
|
||||
s = this_service['name']
|
||||
msg = "The charm's root directory name needs to be {}".format(s)
|
||||
amulet.raise_status(amulet.FAIL, msg=msg)
|
||||
|
||||
if 'units' not in this_service:
|
||||
this_service['units'] = 1
|
||||
|
||||
self.d.add(this_service['name'], units=this_service['units'],
|
||||
constraints=this_service.get('constraints'))
|
||||
|
||||
for svc in other_services:
|
||||
if 'location' in svc:
|
||||
branch_location = svc['location']
|
||||
elif self.series:
|
||||
branch_location = 'cs:{}/{}'.format(self.series, svc['name']),
|
||||
else:
|
||||
branch_location = None
|
||||
|
||||
if 'units' not in svc:
|
||||
svc['units'] = 1
|
||||
|
||||
self.d.add(svc['name'], charm=branch_location, units=svc['units'],
|
||||
constraints=svc.get('constraints'))
|
||||
|
||||
def _add_relations(self, relations):
|
||||
"""Add all of the relations for the services."""
|
||||
for k, v in six.iteritems(relations):
|
||||
self.d.relate(k, v)
|
||||
|
||||
def _configure_services(self, configs):
|
||||
"""Configure all of the services."""
|
||||
for service, config in six.iteritems(configs):
|
||||
self.d.configure(service, config)
|
||||
|
||||
def _deploy(self):
|
||||
"""Deploy environment and wait for all hooks to finish executing."""
|
||||
try:
|
||||
self.d.setup(timeout=900)
|
||||
self.d.sentry.wait(timeout=900)
|
||||
except amulet.helpers.TimeoutError:
|
||||
amulet.raise_status(amulet.FAIL, msg="Deployment timed out")
|
||||
except Exception:
|
||||
raise
|
||||
|
||||
def run_tests(self):
|
||||
"""Run all of the methods that are prefixed with 'test_'."""
|
||||
for test in dir(self):
|
||||
if test.startswith('test_'):
|
||||
getattr(self, test)()
|
|
@ -1,829 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
import io
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import socket
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
import uuid
|
||||
|
||||
import amulet
|
||||
import distro_info
|
||||
import six
|
||||
from six.moves import configparser
|
||||
if six.PY3:
|
||||
from urllib import parse as urlparse
|
||||
else:
|
||||
import urlparse
|
||||
|
||||
|
||||
class AmuletUtils(object):
|
||||
"""Amulet utilities.
|
||||
|
||||
This class provides common utility functions that are used by Amulet
|
||||
tests.
|
||||
"""
|
||||
|
||||
def __init__(self, log_level=logging.ERROR):
|
||||
self.log = self.get_logger(level=log_level)
|
||||
self.ubuntu_releases = self.get_ubuntu_releases()
|
||||
|
||||
def get_logger(self, name="amulet-logger", level=logging.DEBUG):
|
||||
"""Get a logger object that will log to stdout."""
|
||||
log = logging
|
||||
logger = log.getLogger(name)
|
||||
fmt = log.Formatter("%(asctime)s %(funcName)s "
|
||||
"%(levelname)s: %(message)s")
|
||||
|
||||
handler = log.StreamHandler(stream=sys.stdout)
|
||||
handler.setLevel(level)
|
||||
handler.setFormatter(fmt)
|
||||
|
||||
logger.addHandler(handler)
|
||||
logger.setLevel(level)
|
||||
|
||||
return logger
|
||||
|
||||
def valid_ip(self, ip):
|
||||
if re.match(r"^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$", ip):
|
||||
return True
|
||||
else:
|
||||
return False
|
||||
|
||||
def valid_url(self, url):
|
||||
p = re.compile(
|
||||
r'^(?:http|ftp)s?://'
|
||||
r'(?:(?:[A-Z0-9](?:[A-Z0-9-]{0,61}[A-Z0-9])?\.)+(?:[A-Z]{2,6}\.?|[A-Z0-9-]{2,}\.?)|' # noqa
|
||||
r'localhost|'
|
||||
r'\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})'
|
||||
r'(?::\d+)?'
|
||||
r'(?:/?|[/?]\S+)$',
|
||||
re.IGNORECASE)
|
||||
if p.match(url):
|
||||
return True
|
||||
else:
|
||||
return False
|
||||
|
||||
def get_ubuntu_release_from_sentry(self, sentry_unit):
|
||||
"""Get Ubuntu release codename from sentry unit.
|
||||
|
||||
:param sentry_unit: amulet sentry/service unit pointer
|
||||
:returns: list of strings - release codename, failure message
|
||||
"""
|
||||
msg = None
|
||||
cmd = 'lsb_release -cs'
|
||||
release, code = sentry_unit.run(cmd)
|
||||
if code == 0:
|
||||
self.log.debug('{} lsb_release: {}'.format(
|
||||
sentry_unit.info['unit_name'], release))
|
||||
else:
|
||||
msg = ('{} `{}` returned {} '
|
||||
'{}'.format(sentry_unit.info['unit_name'],
|
||||
cmd, release, code))
|
||||
if release not in self.ubuntu_releases:
|
||||
msg = ("Release ({}) not found in Ubuntu releases "
|
||||
"({})".format(release, self.ubuntu_releases))
|
||||
return release, msg
|
||||
|
||||
def validate_services(self, commands):
|
||||
"""Validate that lists of commands succeed on service units. Can be
|
||||
used to verify system services are running on the corresponding
|
||||
service units.
|
||||
|
||||
:param commands: dict with sentry keys and arbitrary command list vals
|
||||
:returns: None if successful, Failure string message otherwise
|
||||
"""
|
||||
self.log.debug('Checking status of system services...')
|
||||
|
||||
# /!\ DEPRECATION WARNING (beisner):
|
||||
# New and existing tests should be rewritten to use
|
||||
# validate_services_by_name() as it is aware of init systems.
|
||||
self.log.warn('DEPRECATION WARNING: use '
|
||||
'validate_services_by_name instead of validate_services '
|
||||
'due to init system differences.')
|
||||
|
||||
for k, v in six.iteritems(commands):
|
||||
for cmd in v:
|
||||
output, code = k.run(cmd)
|
||||
self.log.debug('{} `{}` returned '
|
||||
'{}'.format(k.info['unit_name'],
|
||||
cmd, code))
|
||||
if code != 0:
|
||||
return "command `{}` returned {}".format(cmd, str(code))
|
||||
return None
|
||||
|
||||
def validate_services_by_name(self, sentry_services):
|
||||
"""Validate system service status by service name, automatically
|
||||
detecting init system based on Ubuntu release codename.
|
||||
|
||||
:param sentry_services: dict with sentry keys and svc list values
|
||||
:returns: None if successful, Failure string message otherwise
|
||||
"""
|
||||
self.log.debug('Checking status of system services...')
|
||||
|
||||
# Point at which systemd became a thing
|
||||
systemd_switch = self.ubuntu_releases.index('vivid')
|
||||
|
||||
for sentry_unit, services_list in six.iteritems(sentry_services):
|
||||
# Get lsb_release codename from unit
|
||||
release, ret = self.get_ubuntu_release_from_sentry(sentry_unit)
|
||||
if ret:
|
||||
return ret
|
||||
|
||||
for service_name in services_list:
|
||||
if (self.ubuntu_releases.index(release) >= systemd_switch or
|
||||
service_name in ['rabbitmq-server', 'apache2']):
|
||||
# init is systemd (or regular sysv)
|
||||
cmd = 'sudo service {} status'.format(service_name)
|
||||
output, code = sentry_unit.run(cmd)
|
||||
service_running = code == 0
|
||||
elif self.ubuntu_releases.index(release) < systemd_switch:
|
||||
# init is upstart
|
||||
cmd = 'sudo status {}'.format(service_name)
|
||||
output, code = sentry_unit.run(cmd)
|
||||
service_running = code == 0 and "start/running" in output
|
||||
|
||||
self.log.debug('{} `{}` returned '
|
||||
'{}'.format(sentry_unit.info['unit_name'],
|
||||
cmd, code))
|
||||
if not service_running:
|
||||
return u"command `{}` returned {} {}".format(
|
||||
cmd, output, str(code))
|
||||
return None
|
||||
|
||||
def _get_config(self, unit, filename):
|
||||
"""Get a ConfigParser object for parsing a unit's config file."""
|
||||
file_contents = unit.file_contents(filename)
|
||||
|
||||
# NOTE(beisner): by default, ConfigParser does not handle options
|
||||
# with no value, such as the flags used in the mysql my.cnf file.
|
||||
# https://bugs.python.org/issue7005
|
||||
config = configparser.ConfigParser(allow_no_value=True)
|
||||
config.readfp(io.StringIO(file_contents))
|
||||
return config
|
||||
|
||||
def validate_config_data(self, sentry_unit, config_file, section,
|
||||
expected):
|
||||
"""Validate config file data.
|
||||
|
||||
Verify that the specified section of the config file contains
|
||||
the expected option key:value pairs.
|
||||
|
||||
Compare expected dictionary data vs actual dictionary data.
|
||||
The values in the 'expected' dictionary can be strings, bools, ints,
|
||||
longs, or can be a function that evaluates a variable and returns a
|
||||
bool.
|
||||
"""
|
||||
self.log.debug('Validating config file data ({} in {} on {})'
|
||||
'...'.format(section, config_file,
|
||||
sentry_unit.info['unit_name']))
|
||||
config = self._get_config(sentry_unit, config_file)
|
||||
|
||||
if section != 'DEFAULT' and not config.has_section(section):
|
||||
return "section [{}] does not exist".format(section)
|
||||
|
||||
for k in expected.keys():
|
||||
if not config.has_option(section, k):
|
||||
return "section [{}] is missing option {}".format(section, k)
|
||||
|
||||
actual = config.get(section, k)
|
||||
v = expected[k]
|
||||
if (isinstance(v, six.string_types) or
|
||||
isinstance(v, bool) or
|
||||
isinstance(v, six.integer_types)):
|
||||
# handle explicit values
|
||||
if actual != v:
|
||||
return "section [{}] {}:{} != expected {}:{}".format(
|
||||
section, k, actual, k, expected[k])
|
||||
# handle function pointers, such as not_null or valid_ip
|
||||
elif not v(actual):
|
||||
return "section [{}] {}:{} != expected {}:{}".format(
|
||||
section, k, actual, k, expected[k])
|
||||
return None
|
||||
|
||||
def _validate_dict_data(self, expected, actual):
|
||||
"""Validate dictionary data.
|
||||
|
||||
Compare expected dictionary data vs actual dictionary data.
|
||||
The values in the 'expected' dictionary can be strings, bools, ints,
|
||||
longs, or can be a function that evaluates a variable and returns a
|
||||
bool.
|
||||
"""
|
||||
self.log.debug('actual: {}'.format(repr(actual)))
|
||||
self.log.debug('expected: {}'.format(repr(expected)))
|
||||
|
||||
for k, v in six.iteritems(expected):
|
||||
if k in actual:
|
||||
if (isinstance(v, six.string_types) or
|
||||
isinstance(v, bool) or
|
||||
isinstance(v, six.integer_types)):
|
||||
# handle explicit values
|
||||
if v != actual[k]:
|
||||
return "{}:{}".format(k, actual[k])
|
||||
# handle function pointers, such as not_null or valid_ip
|
||||
elif not v(actual[k]):
|
||||
return "{}:{}".format(k, actual[k])
|
||||
else:
|
||||
return "key '{}' does not exist".format(k)
|
||||
return None
|
||||
|
||||
def validate_relation_data(self, sentry_unit, relation, expected):
|
||||
"""Validate actual relation data based on expected relation data."""
|
||||
actual = sentry_unit.relation(relation[0], relation[1])
|
||||
return self._validate_dict_data(expected, actual)
|
||||
|
||||
def _validate_list_data(self, expected, actual):
|
||||
"""Compare expected list vs actual list data."""
|
||||
for e in expected:
|
||||
if e not in actual:
|
||||
return "expected item {} not found in actual list".format(e)
|
||||
return None
|
||||
|
||||
def not_null(self, string):
|
||||
if string is not None:
|
||||
return True
|
||||
else:
|
||||
return False
|
||||
|
||||
def _get_file_mtime(self, sentry_unit, filename):
|
||||
"""Get last modification time of file."""
|
||||
return sentry_unit.file_stat(filename)['mtime']
|
||||
|
||||
def _get_dir_mtime(self, sentry_unit, directory):
|
||||
"""Get last modification time of directory."""
|
||||
return sentry_unit.directory_stat(directory)['mtime']
|
||||
|
||||
def _get_proc_start_time(self, sentry_unit, service, pgrep_full=None):
|
||||
"""Get start time of a process based on the last modification time
|
||||
of the /proc/pid directory.
|
||||
|
||||
:sentry_unit: The sentry unit to check for the service on
|
||||
:service: service name to look for in process table
|
||||
:pgrep_full: [Deprecated] Use full command line search mode with pgrep
|
||||
:returns: epoch time of service process start
|
||||
:param commands: list of bash commands
|
||||
:param sentry_units: list of sentry unit pointers
|
||||
:returns: None if successful; Failure message otherwise
|
||||
"""
|
||||
if pgrep_full is not None:
|
||||
# /!\ DEPRECATION WARNING (beisner):
|
||||
# No longer implemented, as pidof is now used instead of pgrep.
|
||||
# https://bugs.launchpad.net/charm-helpers/+bug/1474030
|
||||
self.log.warn('DEPRECATION WARNING: pgrep_full bool is no '
|
||||
'longer implemented re: lp 1474030.')
|
||||
|
||||
pid_list = self.get_process_id_list(sentry_unit, service)
|
||||
pid = pid_list[0]
|
||||
proc_dir = '/proc/{}'.format(pid)
|
||||
self.log.debug('Pid for {} on {}: {}'.format(
|
||||
service, sentry_unit.info['unit_name'], pid))
|
||||
|
||||
return self._get_dir_mtime(sentry_unit, proc_dir)
|
||||
|
||||
def service_restarted(self, sentry_unit, service, filename,
|
||||
pgrep_full=None, sleep_time=20):
|
||||
"""Check if service was restarted.
|
||||
|
||||
Compare a service's start time vs a file's last modification time
|
||||
(such as a config file for that service) to determine if the service
|
||||
has been restarted.
|
||||
"""
|
||||
# /!\ DEPRECATION WARNING (beisner):
|
||||
# This method is prone to races in that no before-time is known.
|
||||
# Use validate_service_config_changed instead.
|
||||
|
||||
# NOTE(beisner) pgrep_full is no longer implemented, as pidof is now
|
||||
# used instead of pgrep. pgrep_full is still passed through to ensure
|
||||
# deprecation WARNS. lp1474030
|
||||
self.log.warn('DEPRECATION WARNING: use '
|
||||
'validate_service_config_changed instead of '
|
||||
'service_restarted due to known races.')
|
||||
|
||||
time.sleep(sleep_time)
|
||||
if (self._get_proc_start_time(sentry_unit, service, pgrep_full) >=
|
||||
self._get_file_mtime(sentry_unit, filename)):
|
||||
return True
|
||||
else:
|
||||
return False
|
||||
|
||||
def service_restarted_since(self, sentry_unit, mtime, service,
|
||||
pgrep_full=None, sleep_time=20,
|
||||
retry_count=30, retry_sleep_time=10):
|
||||
"""Check if service was been started after a given time.
|
||||
|
||||
Args:
|
||||
sentry_unit (sentry): The sentry unit to check for the service on
|
||||
mtime (float): The epoch time to check against
|
||||
service (string): service name to look for in process table
|
||||
pgrep_full: [Deprecated] Use full command line search mode with pgrep
|
||||
sleep_time (int): Initial sleep time (s) before looking for file
|
||||
retry_sleep_time (int): Time (s) to sleep between retries
|
||||
retry_count (int): If file is not found, how many times to retry
|
||||
|
||||
Returns:
|
||||
bool: True if service found and its start time it newer than mtime,
|
||||
False if service is older than mtime or if service was
|
||||
not found.
|
||||
"""
|
||||
# NOTE(beisner) pgrep_full is no longer implemented, as pidof is now
|
||||
# used instead of pgrep. pgrep_full is still passed through to ensure
|
||||
# deprecation WARNS. lp1474030
|
||||
|
||||
unit_name = sentry_unit.info['unit_name']
|
||||
self.log.debug('Checking that %s service restarted since %s on '
|
||||
'%s' % (service, mtime, unit_name))
|
||||
time.sleep(sleep_time)
|
||||
proc_start_time = None
|
||||
tries = 0
|
||||
while tries <= retry_count and not proc_start_time:
|
||||
try:
|
||||
proc_start_time = self._get_proc_start_time(sentry_unit,
|
||||
service,
|
||||
pgrep_full)
|
||||
self.log.debug('Attempt {} to get {} proc start time on {} '
|
||||
'OK'.format(tries, service, unit_name))
|
||||
except IOError as e:
|
||||
# NOTE(beisner) - race avoidance, proc may not exist yet.
|
||||
# https://bugs.launchpad.net/charm-helpers/+bug/1474030
|
||||
self.log.debug('Attempt {} to get {} proc start time on {} '
|
||||
'failed\n{}'.format(tries, service,
|
||||
unit_name, e))
|
||||
time.sleep(retry_sleep_time)
|
||||
tries += 1
|
||||
|
||||
if not proc_start_time:
|
||||
self.log.warn('No proc start time found, assuming service did '
|
||||
'not start')
|
||||
return False
|
||||
if proc_start_time >= mtime:
|
||||
self.log.debug('Proc start time is newer than provided mtime'
|
||||
'(%s >= %s) on %s (OK)' % (proc_start_time,
|
||||
mtime, unit_name))
|
||||
return True
|
||||
else:
|
||||
self.log.warn('Proc start time (%s) is older than provided mtime '
|
||||
'(%s) on %s, service did not '
|
||||
'restart' % (proc_start_time, mtime, unit_name))
|
||||
return False
|
||||
|
||||
def config_updated_since(self, sentry_unit, filename, mtime,
|
||||
sleep_time=20, retry_count=30,
|
||||
retry_sleep_time=10):
|
||||
"""Check if file was modified after a given time.
|
||||
|
||||
Args:
|
||||
sentry_unit (sentry): The sentry unit to check the file mtime on
|
||||
filename (string): The file to check mtime of
|
||||
mtime (float): The epoch time to check against
|
||||
sleep_time (int): Initial sleep time (s) before looking for file
|
||||
retry_sleep_time (int): Time (s) to sleep between retries
|
||||
retry_count (int): If file is not found, how many times to retry
|
||||
|
||||
Returns:
|
||||
bool: True if file was modified more recently than mtime, False if
|
||||
file was modified before mtime, or if file not found.
|
||||
"""
|
||||
unit_name = sentry_unit.info['unit_name']
|
||||
self.log.debug('Checking that %s updated since %s on '
|
||||
'%s' % (filename, mtime, unit_name))
|
||||
time.sleep(sleep_time)
|
||||
file_mtime = None
|
||||
tries = 0
|
||||
while tries <= retry_count and not file_mtime:
|
||||
try:
|
||||
file_mtime = self._get_file_mtime(sentry_unit, filename)
|
||||
self.log.debug('Attempt {} to get {} file mtime on {} '
|
||||
'OK'.format(tries, filename, unit_name))
|
||||
except IOError as e:
|
||||
# NOTE(beisner) - race avoidance, file may not exist yet.
|
||||
# https://bugs.launchpad.net/charm-helpers/+bug/1474030
|
||||
self.log.debug('Attempt {} to get {} file mtime on {} '
|
||||
'failed\n{}'.format(tries, filename,
|
||||
unit_name, e))
|
||||
time.sleep(retry_sleep_time)
|
||||
tries += 1
|
||||
|
||||
if not file_mtime:
|
||||
self.log.warn('Could not determine file mtime, assuming '
|
||||
'file does not exist')
|
||||
return False
|
||||
|
||||
if file_mtime >= mtime:
|
||||
self.log.debug('File mtime is newer than provided mtime '
|
||||
'(%s >= %s) on %s (OK)' % (file_mtime,
|
||||
mtime, unit_name))
|
||||
return True
|
||||
else:
|
||||
self.log.warn('File mtime is older than provided mtime'
|
||||
'(%s < on %s) on %s' % (file_mtime,
|
||||
mtime, unit_name))
|
||||
return False
|
||||
|
||||
def validate_service_config_changed(self, sentry_unit, mtime, service,
|
||||
filename, pgrep_full=None,
|
||||
sleep_time=20, retry_count=30,
|
||||
retry_sleep_time=10):
|
||||
"""Check service and file were updated after mtime
|
||||
|
||||
Args:
|
||||
sentry_unit (sentry): The sentry unit to check for the service on
|
||||
mtime (float): The epoch time to check against
|
||||
service (string): service name to look for in process table
|
||||
filename (string): The file to check mtime of
|
||||
pgrep_full: [Deprecated] Use full command line search mode with pgrep
|
||||
sleep_time (int): Initial sleep in seconds to pass to test helpers
|
||||
retry_count (int): If service is not found, how many times to retry
|
||||
retry_sleep_time (int): Time in seconds to wait between retries
|
||||
|
||||
Typical Usage:
|
||||
u = OpenStackAmuletUtils(ERROR)
|
||||
...
|
||||
mtime = u.get_sentry_time(self.cinder_sentry)
|
||||
self.d.configure('cinder', {'verbose': 'True', 'debug': 'True'})
|
||||
if not u.validate_service_config_changed(self.cinder_sentry,
|
||||
mtime,
|
||||
'cinder-api',
|
||||
'/etc/cinder/cinder.conf')
|
||||
amulet.raise_status(amulet.FAIL, msg='update failed')
|
||||
Returns:
|
||||
bool: True if both service and file where updated/restarted after
|
||||
mtime, False if service is older than mtime or if service was
|
||||
not found or if filename was modified before mtime.
|
||||
"""
|
||||
|
||||
# NOTE(beisner) pgrep_full is no longer implemented, as pidof is now
|
||||
# used instead of pgrep. pgrep_full is still passed through to ensure
|
||||
# deprecation WARNS. lp1474030
|
||||
|
||||
service_restart = self.service_restarted_since(
|
||||
sentry_unit, mtime,
|
||||
service,
|
||||
pgrep_full=pgrep_full,
|
||||
sleep_time=sleep_time,
|
||||
retry_count=retry_count,
|
||||
retry_sleep_time=retry_sleep_time)
|
||||
|
||||
config_update = self.config_updated_since(
|
||||
sentry_unit,
|
||||
filename,
|
||||
mtime,
|
||||
sleep_time=sleep_time,
|
||||
retry_count=retry_count,
|
||||
retry_sleep_time=retry_sleep_time)
|
||||
|
||||
return service_restart and config_update
|
||||
|
||||
def get_sentry_time(self, sentry_unit):
|
||||
"""Return current epoch time on a sentry"""
|
||||
cmd = "date +'%s'"
|
||||
return float(sentry_unit.run(cmd)[0])
|
||||
|
||||
def relation_error(self, name, data):
|
||||
return 'unexpected relation data in {} - {}'.format(name, data)
|
||||
|
||||
def endpoint_error(self, name, data):
|
||||
return 'unexpected endpoint data in {} - {}'.format(name, data)
|
||||
|
||||
def get_ubuntu_releases(self):
|
||||
"""Return a list of all Ubuntu releases in order of release."""
|
||||
_d = distro_info.UbuntuDistroInfo()
|
||||
_release_list = _d.all
|
||||
return _release_list
|
||||
|
||||
def file_to_url(self, file_rel_path):
|
||||
"""Convert a relative file path to a file URL."""
|
||||
_abs_path = os.path.abspath(file_rel_path)
|
||||
return urlparse.urlparse(_abs_path, scheme='file').geturl()
|
||||
|
||||
def check_commands_on_units(self, commands, sentry_units):
|
||||
"""Check that all commands in a list exit zero on all
|
||||
sentry units in a list.
|
||||
|
||||
:param commands: list of bash commands
|
||||
:param sentry_units: list of sentry unit pointers
|
||||
:returns: None if successful; Failure message otherwise
|
||||
"""
|
||||
self.log.debug('Checking exit codes for {} commands on {} '
|
||||
'sentry units...'.format(len(commands),
|
||||
len(sentry_units)))
|
||||
for sentry_unit in sentry_units:
|
||||
for cmd in commands:
|
||||
output, code = sentry_unit.run(cmd)
|
||||
if code == 0:
|
||||
self.log.debug('{} `{}` returned {} '
|
||||
'(OK)'.format(sentry_unit.info['unit_name'],
|
||||
cmd, code))
|
||||
else:
|
||||
return ('{} `{}` returned {} '
|
||||
'{}'.format(sentry_unit.info['unit_name'],
|
||||
cmd, code, output))
|
||||
return None
|
||||
|
||||
def get_process_id_list(self, sentry_unit, process_name,
|
||||
expect_success=True):
|
||||
"""Get a list of process ID(s) from a single sentry juju unit
|
||||
for a single process name.
|
||||
|
||||
:param sentry_unit: Amulet sentry instance (juju unit)
|
||||
:param process_name: Process name
|
||||
:param expect_success: If False, expect the PID to be missing,
|
||||
raise if it is present.
|
||||
:returns: List of process IDs
|
||||
"""
|
||||
cmd = 'pidof -x {}'.format(process_name)
|
||||
if not expect_success:
|
||||
cmd += " || exit 0 && exit 1"
|
||||
output, code = sentry_unit.run(cmd)
|
||||
if code != 0:
|
||||
msg = ('{} `{}` returned {} '
|
||||
'{}'.format(sentry_unit.info['unit_name'],
|
||||
cmd, code, output))
|
||||
amulet.raise_status(amulet.FAIL, msg=msg)
|
||||
return str(output).split()
|
||||
|
||||
def get_unit_process_ids(self, unit_processes, expect_success=True):
|
||||
"""Construct a dict containing unit sentries, process names, and
|
||||
process IDs.
|
||||
|
||||
:param unit_processes: A dictionary of Amulet sentry instance
|
||||
to list of process names.
|
||||
:param expect_success: if False expect the processes to not be
|
||||
running, raise if they are.
|
||||
:returns: Dictionary of Amulet sentry instance to dictionary
|
||||
of process names to PIDs.
|
||||
"""
|
||||
pid_dict = {}
|
||||
for sentry_unit, process_list in six.iteritems(unit_processes):
|
||||
pid_dict[sentry_unit] = {}
|
||||
for process in process_list:
|
||||
pids = self.get_process_id_list(
|
||||
sentry_unit, process, expect_success=expect_success)
|
||||
pid_dict[sentry_unit].update({process: pids})
|
||||
return pid_dict
|
||||
|
||||
def validate_unit_process_ids(self, expected, actual):
|
||||
"""Validate process id quantities for services on units."""
|
||||
self.log.debug('Checking units for running processes...')
|
||||
self.log.debug('Expected PIDs: {}'.format(expected))
|
||||
self.log.debug('Actual PIDs: {}'.format(actual))
|
||||
|
||||
if len(actual) != len(expected):
|
||||
return ('Unit count mismatch. expected, actual: {}, '
|
||||
'{} '.format(len(expected), len(actual)))
|
||||
|
||||
for (e_sentry, e_proc_names) in six.iteritems(expected):
|
||||
e_sentry_name = e_sentry.info['unit_name']
|
||||
if e_sentry in actual.keys():
|
||||
a_proc_names = actual[e_sentry]
|
||||
else:
|
||||
return ('Expected sentry ({}) not found in actual dict data.'
|
||||
'{}'.format(e_sentry_name, e_sentry))
|
||||
|
||||
if len(e_proc_names.keys()) != len(a_proc_names.keys()):
|
||||
return ('Process name count mismatch. expected, actual: {}, '
|
||||
'{}'.format(len(expected), len(actual)))
|
||||
|
||||
for (e_proc_name, e_pids), (a_proc_name, a_pids) in \
|
||||
zip(e_proc_names.items(), a_proc_names.items()):
|
||||
if e_proc_name != a_proc_name:
|
||||
return ('Process name mismatch. expected, actual: {}, '
|
||||
'{}'.format(e_proc_name, a_proc_name))
|
||||
|
||||
a_pids_length = len(a_pids)
|
||||
fail_msg = ('PID count mismatch. {} ({}) expected, actual: '
|
||||
'{}, {} ({})'.format(e_sentry_name, e_proc_name,
|
||||
e_pids, a_pids_length,
|
||||
a_pids))
|
||||
|
||||
# If expected is a list, ensure at least one PID quantity match
|
||||
if isinstance(e_pids, list) and \
|
||||
a_pids_length not in e_pids:
|
||||
return fail_msg
|
||||
# If expected is not bool and not list,
|
||||
# ensure PID quantities match
|
||||
elif not isinstance(e_pids, bool) and \
|
||||
not isinstance(e_pids, list) and \
|
||||
a_pids_length != e_pids:
|
||||
return fail_msg
|
||||
# If expected is bool True, ensure 1 or more PIDs exist
|
||||
elif isinstance(e_pids, bool) and \
|
||||
e_pids is True and a_pids_length < 1:
|
||||
return fail_msg
|
||||
# If expected is bool False, ensure 0 PIDs exist
|
||||
elif isinstance(e_pids, bool) and \
|
||||
e_pids is False and a_pids_length != 0:
|
||||
return fail_msg
|
||||
else:
|
||||
self.log.debug('PID check OK: {} {} {}: '
|
||||
'{}'.format(e_sentry_name, e_proc_name,
|
||||
e_pids, a_pids))
|
||||
return None
|
||||
|
||||
def validate_list_of_identical_dicts(self, list_of_dicts):
|
||||
"""Check that all dicts within a list are identical."""
|
||||
hashes = []
|
||||
for _dict in list_of_dicts:
|
||||
hashes.append(hash(frozenset(_dict.items())))
|
||||
|
||||
self.log.debug('Hashes: {}'.format(hashes))
|
||||
if len(set(hashes)) == 1:
|
||||
self.log.debug('Dicts within list are identical')
|
||||
else:
|
||||
return 'Dicts within list are not identical'
|
||||
|
||||
return None
|
||||
|
||||
def validate_sectionless_conf(self, file_contents, expected):
|
||||
"""A crude conf parser. Useful to inspect configuration files which
|
||||
do not have section headers (as would be necessary in order to use
|
||||
the configparser). Such as openstack-dashboard or rabbitmq confs."""
|
||||
for line in file_contents.split('\n'):
|
||||
if '=' in line:
|
||||
args = line.split('=')
|
||||
if len(args) <= 1:
|
||||
continue
|
||||
key = args[0].strip()
|
||||
value = args[1].strip()
|
||||
if key in expected.keys():
|
||||
if expected[key] != value:
|
||||
msg = ('Config mismatch. Expected, actual: {}, '
|
||||
'{}'.format(expected[key], value))
|
||||
amulet.raise_status(amulet.FAIL, msg=msg)
|
||||
|
||||
def get_unit_hostnames(self, units):
|
||||
"""Return a dict of juju unit names to hostnames."""
|
||||
host_names = {}
|
||||
for unit in units:
|
||||
host_names[unit.info['unit_name']] = \
|
||||
str(unit.file_contents('/etc/hostname').strip())
|
||||
self.log.debug('Unit host names: {}'.format(host_names))
|
||||
return host_names
|
||||
|
||||
def run_cmd_unit(self, sentry_unit, cmd):
|
||||
"""Run a command on a unit, return the output and exit code."""
|
||||
output, code = sentry_unit.run(cmd)
|
||||
if code == 0:
|
||||
self.log.debug('{} `{}` command returned {} '
|
||||
'(OK)'.format(sentry_unit.info['unit_name'],
|
||||
cmd, code))
|
||||
else:
|
||||
msg = ('{} `{}` command returned {} '
|
||||
'{}'.format(sentry_unit.info['unit_name'],
|
||||
cmd, code, output))
|
||||
amulet.raise_status(amulet.FAIL, msg=msg)
|
||||
return str(output), code
|
||||
|
||||
def file_exists_on_unit(self, sentry_unit, file_name):
|
||||
"""Check if a file exists on a unit."""
|
||||
try:
|
||||
sentry_unit.file_stat(file_name)
|
||||
return True
|
||||
except IOError:
|
||||
return False
|
||||
except Exception as e:
|
||||
msg = 'Error checking file {}: {}'.format(file_name, e)
|
||||
amulet.raise_status(amulet.FAIL, msg=msg)
|
||||
|
||||
def file_contents_safe(self, sentry_unit, file_name,
|
||||
max_wait=60, fatal=False):
|
||||
"""Get file contents from a sentry unit. Wrap amulet file_contents
|
||||
with retry logic to address races where a file checks as existing,
|
||||
but no longer exists by the time file_contents is called.
|
||||
Return None if file not found. Optionally raise if fatal is True."""
|
||||
unit_name = sentry_unit.info['unit_name']
|
||||
file_contents = False
|
||||
tries = 0
|
||||
while not file_contents and tries < (max_wait / 4):
|
||||
try:
|
||||
file_contents = sentry_unit.file_contents(file_name)
|
||||
except IOError:
|
||||
self.log.debug('Attempt {} to open file {} from {} '
|
||||
'failed'.format(tries, file_name,
|
||||
unit_name))
|
||||
time.sleep(4)
|
||||
tries += 1
|
||||
|
||||
if file_contents:
|
||||
return file_contents
|
||||
elif not fatal:
|
||||
return None
|
||||
elif fatal:
|
||||
msg = 'Failed to get file contents from unit.'
|
||||
amulet.raise_status(amulet.FAIL, msg)
|
||||
|
||||
def port_knock_tcp(self, host="localhost", port=22, timeout=15):
|
||||
"""Open a TCP socket to check for a listening sevice on a host.
|
||||
|
||||
:param host: host name or IP address, default to localhost
|
||||
:param port: TCP port number, default to 22
|
||||
:param timeout: Connect timeout, default to 15 seconds
|
||||
:returns: True if successful, False if connect failed
|
||||
"""
|
||||
|
||||
# Resolve host name if possible
|
||||
try:
|
||||
connect_host = socket.gethostbyname(host)
|
||||
host_human = "{} ({})".format(connect_host, host)
|
||||
except socket.error as e:
|
||||
self.log.warn('Unable to resolve address: '
|
||||
'{} ({}) Trying anyway!'.format(host, e))
|
||||
connect_host = host
|
||||
host_human = connect_host
|
||||
|
||||
# Attempt socket connection
|
||||
try:
|
||||
knock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
knock.settimeout(timeout)
|
||||
knock.connect((connect_host, port))
|
||||
knock.close()
|
||||
self.log.debug('Socket connect OK for host '
|
||||
'{} on port {}.'.format(host_human, port))
|
||||
return True
|
||||
except socket.error as e:
|
||||
self.log.debug('Socket connect FAIL for'
|
||||
' {} port {} ({})'.format(host_human, port, e))
|
||||
return False
|
||||
|
||||
def port_knock_units(self, sentry_units, port=22,
|
||||
timeout=15, expect_success=True):
|
||||
"""Open a TCP socket to check for a listening sevice on each
|
||||
listed juju unit.
|
||||
|
||||
:param sentry_units: list of sentry unit pointers
|
||||
:param port: TCP port number, default to 22
|
||||
:param timeout: Connect timeout, default to 15 seconds
|
||||
:expect_success: True by default, set False to invert logic
|
||||
:returns: None if successful, Failure message otherwise
|
||||
"""
|
||||
for unit in sentry_units:
|
||||
host = unit.info['public-address']
|
||||
connected = self.port_knock_tcp(host, port, timeout)
|
||||
if not connected and expect_success:
|
||||
return 'Socket connect failed.'
|
||||
elif connected and not expect_success:
|
||||
return 'Socket connected unexpectedly.'
|
||||
|
||||
def get_uuid_epoch_stamp(self):
|
||||
"""Returns a stamp string based on uuid4 and epoch time. Useful in
|
||||
generating test messages which need to be unique-ish."""
|
||||
return '[{}-{}]'.format(uuid.uuid4(), time.time())
|
||||
|
||||
# amulet juju action helpers:
|
||||
def run_action(self, unit_sentry, action,
|
||||
_check_output=subprocess.check_output,
|
||||
params=None):
|
||||
"""Run the named action on a given unit sentry.
|
||||
|
||||
params a dict of parameters to use
|
||||
_check_output parameter is used for dependency injection.
|
||||
|
||||
@return action_id.
|
||||
"""
|
||||
unit_id = unit_sentry.info["unit_name"]
|
||||
command = ["juju", "action", "do", "--format=json", unit_id, action]
|
||||
if params is not None:
|
||||
for key, value in params.iteritems():
|
||||
command.append("{}={}".format(key, value))
|
||||
self.log.info("Running command: %s\n" % " ".join(command))
|
||||
output = _check_output(command, universal_newlines=True)
|
||||
data = json.loads(output)
|
||||
action_id = data[u'Action queued with id']
|
||||
return action_id
|
||||
|
||||
def wait_on_action(self, action_id, _check_output=subprocess.check_output):
|
||||
"""Wait for a given action, returning if it completed or not.
|
||||
|
||||
_check_output parameter is used for dependency injection.
|
||||
"""
|
||||
command = ["juju", "action", "fetch", "--format=json", "--wait=0",
|
||||
action_id]
|
||||
output = _check_output(command, universal_newlines=True)
|
||||
data = json.loads(output)
|
||||
return data.get(u"status") == "completed"
|
||||
|
||||
def status_get(self, unit):
|
||||
"""Return the current service status of this unit."""
|
||||
raw_status, return_code = unit.run(
|
||||
"status-get --format=json --include-data")
|
||||
if return_code != 0:
|
||||
return ("unknown", "")
|
||||
status = json.loads(raw_status)
|
||||
return (status["status"], status["message"])
|
|
@ -1,15 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
|
@ -1,82 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
#
|
||||
# Copyright 2012 Canonical Ltd.
|
||||
#
|
||||
# This file is sourced from lp:openstack-charm-helpers
|
||||
#
|
||||
# Authors:
|
||||
# James Page <james.page@ubuntu.com>
|
||||
# Adam Gandelman <adamg@ubuntu.com>
|
||||
#
|
||||
|
||||
import subprocess
|
||||
|
||||
from charmhelpers.core.hookenv import (
|
||||
config as config_get,
|
||||
relation_get,
|
||||
relation_ids,
|
||||
related_units as relation_list,
|
||||
log,
|
||||
INFO,
|
||||
)
|
||||
|
||||
|
||||
def get_cert(cn=None):
|
||||
# TODO: deal with multiple https endpoints via charm config
|
||||
cert = config_get('ssl_cert')
|
||||
key = config_get('ssl_key')
|
||||
if not (cert and key):
|
||||
log("Inspecting identity-service relations for SSL certificate.",
|
||||
level=INFO)
|
||||
cert = key = None
|
||||
if cn:
|
||||
ssl_cert_attr = 'ssl_cert_{}'.format(cn)
|
||||
ssl_key_attr = 'ssl_key_{}'.format(cn)
|
||||
else:
|
||||
ssl_cert_attr = 'ssl_cert'
|
||||
ssl_key_attr = 'ssl_key'
|
||||
for r_id in relation_ids('identity-service'):
|
||||
for unit in relation_list(r_id):
|
||||
if not cert:
|
||||
cert = relation_get(ssl_cert_attr,
|
||||
rid=r_id, unit=unit)
|
||||
if not key:
|
||||
key = relation_get(ssl_key_attr,
|
||||
rid=r_id, unit=unit)
|
||||
return (cert, key)
|
||||
|
||||
|
||||
def get_ca_cert():
|
||||
ca_cert = config_get('ssl_ca')
|
||||
if ca_cert is None:
|
||||
log("Inspecting identity-service relations for CA SSL certificate.",
|
||||
level=INFO)
|
||||
for r_id in relation_ids('identity-service'):
|
||||
for unit in relation_list(r_id):
|
||||
if ca_cert is None:
|
||||
ca_cert = relation_get('ca_cert',
|
||||
rid=r_id, unit=unit)
|
||||
return ca_cert
|
||||
|
||||
|
||||
def install_ca_cert(ca_cert):
|
||||
if ca_cert:
|
||||
with open('/usr/local/share/ca-certificates/keystone_juju_ca_cert.crt',
|
||||
'w') as crt:
|
||||
crt.write(ca_cert)
|
||||
subprocess.check_call(['update-ca-certificates', '--fresh'])
|
|
@ -1,316 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
#
|
||||
# Copyright 2012 Canonical Ltd.
|
||||
#
|
||||
# Authors:
|
||||
# James Page <james.page@ubuntu.com>
|
||||
# Adam Gandelman <adamg@ubuntu.com>
|
||||
#
|
||||
|
||||
"""
|
||||
Helpers for clustering and determining "cluster leadership" and other
|
||||
clustering-related helpers.
|
||||
"""
|
||||
|
||||
import subprocess
|
||||
import os
|
||||
|
||||
from socket import gethostname as get_unit_hostname
|
||||
|
||||
import six
|
||||
|
||||
from charmhelpers.core.hookenv import (
|
||||
log,
|
||||
relation_ids,
|
||||
related_units as relation_list,
|
||||
relation_get,
|
||||
config as config_get,
|
||||
INFO,
|
||||
ERROR,
|
||||
WARNING,
|
||||
unit_get,
|
||||
is_leader as juju_is_leader
|
||||
)
|
||||
from charmhelpers.core.decorators import (
|
||||
retry_on_exception,
|
||||
)
|
||||
from charmhelpers.core.strutils import (
|
||||
bool_from_string,
|
||||
)
|
||||
|
||||
DC_RESOURCE_NAME = 'DC'
|
||||
|
||||
|
||||
class HAIncompleteConfig(Exception):
|
||||
pass
|
||||
|
||||
|
||||
class CRMResourceNotFound(Exception):
|
||||
pass
|
||||
|
||||
|
||||
class CRMDCNotFound(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def is_elected_leader(resource):
|
||||
"""
|
||||
Returns True if the charm executing this is the elected cluster leader.
|
||||
|
||||
It relies on two mechanisms to determine leadership:
|
||||
1. If juju is sufficiently new and leadership election is supported,
|
||||
the is_leader command will be used.
|
||||
2. If the charm is part of a corosync cluster, call corosync to
|
||||
determine leadership.
|
||||
3. If the charm is not part of a corosync cluster, the leader is
|
||||
determined as being "the alive unit with the lowest unit numer". In
|
||||
other words, the oldest surviving unit.
|
||||
"""
|
||||
try:
|
||||
return juju_is_leader()
|
||||
except NotImplementedError:
|
||||
log('Juju leadership election feature not enabled'
|
||||
', using fallback support',
|
||||
level=WARNING)
|
||||
|
||||
if is_clustered():
|
||||
if not is_crm_leader(resource):
|
||||
log('Deferring action to CRM leader.', level=INFO)
|
||||
return False
|
||||
else:
|
||||
peers = peer_units()
|
||||
if peers and not oldest_peer(peers):
|
||||
log('Deferring action to oldest service unit.', level=INFO)
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def is_clustered():
|
||||
for r_id in (relation_ids('ha') or []):
|
||||
for unit in (relation_list(r_id) or []):
|
||||
clustered = relation_get('clustered',
|
||||
rid=r_id,
|
||||
unit=unit)
|
||||
if clustered:
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def is_crm_dc():
|
||||
"""
|
||||
Determine leadership by querying the pacemaker Designated Controller
|
||||
"""
|
||||
cmd = ['crm', 'status']
|
||||
try:
|
||||
status = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
|
||||
if not isinstance(status, six.text_type):
|
||||
status = six.text_type(status, "utf-8")
|
||||
except subprocess.CalledProcessError as ex:
|
||||
raise CRMDCNotFound(str(ex))
|
||||
|
||||
current_dc = ''
|
||||
for line in status.split('\n'):
|
||||
if line.startswith('Current DC'):
|
||||
# Current DC: juju-lytrusty-machine-2 (168108163) - partition with quorum
|
||||
current_dc = line.split(':')[1].split()[0]
|
||||
if current_dc == get_unit_hostname():
|
||||
return True
|
||||
elif current_dc == 'NONE':
|
||||
raise CRMDCNotFound('Current DC: NONE')
|
||||
|
||||
return False
|
||||
|
||||
|
||||
@retry_on_exception(5, base_delay=2,
|
||||
exc_type=(CRMResourceNotFound, CRMDCNotFound))
|
||||
def is_crm_leader(resource, retry=False):
|
||||
"""
|
||||
Returns True if the charm calling this is the elected corosync leader,
|
||||
as returned by calling the external "crm" command.
|
||||
|
||||
We allow this operation to be retried to avoid the possibility of getting a
|
||||
false negative. See LP #1396246 for more info.
|
||||
"""
|
||||
if resource == DC_RESOURCE_NAME:
|
||||
return is_crm_dc()
|
||||
cmd = ['crm', 'resource', 'show', resource]
|
||||
try:
|
||||
status = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
|
||||
if not isinstance(status, six.text_type):
|
||||
status = six.text_type(status, "utf-8")
|
||||
except subprocess.CalledProcessError:
|
||||
status = None
|
||||
|
||||
if status and get_unit_hostname() in status:
|
||||
return True
|
||||
|
||||
if status and "resource %s is NOT running" % (resource) in status:
|
||||
raise CRMResourceNotFound("CRM resource %s not found" % (resource))
|
||||
|
||||
return False
|
||||
|
||||
|
||||
def is_leader(resource):
|
||||
log("is_leader is deprecated. Please consider using is_crm_leader "
|
||||
"instead.", level=WARNING)
|
||||
return is_crm_leader(resource)
|
||||
|
||||
|
||||
def peer_units(peer_relation="cluster"):
|
||||
peers = []
|
||||
for r_id in (relation_ids(peer_relation) or []):
|
||||
for unit in (relation_list(r_id) or []):
|
||||
peers.append(unit)
|
||||
return peers
|
||||
|
||||
|
||||
def peer_ips(peer_relation='cluster', addr_key='private-address'):
|
||||
'''Return a dict of peers and their private-address'''
|
||||
peers = {}
|
||||
for r_id in relation_ids(peer_relation):
|
||||
for unit in relation_list(r_id):
|
||||
peers[unit] = relation_get(addr_key, rid=r_id, unit=unit)
|
||||
return peers
|
||||
|
||||
|
||||
def oldest_peer(peers):
|
||||
"""Determines who the oldest peer is by comparing unit numbers."""
|
||||
local_unit_no = int(os.getenv('JUJU_UNIT_NAME').split('/')[1])
|
||||
for peer in peers:
|
||||
remote_unit_no = int(peer.split('/')[1])
|
||||
if remote_unit_no < local_unit_no:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def eligible_leader(resource):
|
||||
log("eligible_leader is deprecated. Please consider using "
|
||||
"is_elected_leader instead.", level=WARNING)
|
||||
return is_elected_leader(resource)
|
||||
|
||||
|
||||
def https():
|
||||
'''
|
||||
Determines whether enough data has been provided in configuration
|
||||
or relation data to configure HTTPS
|
||||
.
|
||||
returns: boolean
|
||||
'''
|
||||
use_https = config_get('use-https')
|
||||
if use_https and bool_from_string(use_https):
|
||||
return True
|
||||
if config_get('ssl_cert') and config_get('ssl_key'):
|
||||
return True
|
||||
for r_id in relation_ids('identity-service'):
|
||||
for unit in relation_list(r_id):
|
||||
# TODO - needs fixing for new helper as ssl_cert/key suffixes with CN
|
||||
rel_state = [
|
||||
relation_get('https_keystone', rid=r_id, unit=unit),
|
||||
relation_get('ca_cert', rid=r_id, unit=unit),
|
||||
]
|
||||
# NOTE: works around (LP: #1203241)
|
||||
if (None not in rel_state) and ('' not in rel_state):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def determine_api_port(public_port, singlenode_mode=False):
|
||||
'''
|
||||
Determine correct API server listening port based on
|
||||
existence of HTTPS reverse proxy and/or haproxy.
|
||||
|
||||
public_port: int: standard public port for given service
|
||||
|
||||
singlenode_mode: boolean: Shuffle ports when only a single unit is present
|
||||
|
||||
returns: int: the correct listening port for the API service
|
||||
'''
|
||||
i = 0
|
||||
if singlenode_mode:
|
||||
i += 1
|
||||
elif len(peer_units()) > 0 or is_clustered():
|
||||
i += 1
|
||||
if https():
|
||||
i += 1
|
||||
return public_port - (i * 10)
|
||||
|
||||
|
||||
def determine_apache_port(public_port, singlenode_mode=False):
|
||||
'''
|
||||
Description: Determine correct apache listening port based on public IP +
|
||||
state of the cluster.
|
||||
|
||||
public_port: int: standard public port for given service
|
||||
|
||||
singlenode_mode: boolean: Shuffle ports when only a single unit is present
|
||||
|
||||
returns: int: the correct listening port for the HAProxy service
|
||||
'''
|
||||
i = 0
|
||||
if singlenode_mode:
|
||||
i += 1
|
||||
elif len(peer_units()) > 0 or is_clustered():
|
||||
i += 1
|
||||
return public_port - (i * 10)
|
||||
|
||||
|
||||
def get_hacluster_config(exclude_keys=None):
|
||||
'''
|
||||
Obtains all relevant configuration from charm configuration required
|
||||
for initiating a relation to hacluster:
|
||||
|
||||
ha-bindiface, ha-mcastport, vip
|
||||
|
||||
param: exclude_keys: list of setting key(s) to be excluded.
|
||||
returns: dict: A dict containing settings keyed by setting name.
|
||||
raises: HAIncompleteConfig if settings are missing.
|
||||
'''
|
||||
settings = ['ha-bindiface', 'ha-mcastport', 'vip']
|
||||
conf = {}
|
||||
for setting in settings:
|
||||
if exclude_keys and setting in exclude_keys:
|
||||
continue
|
||||
|
||||
conf[setting] = config_get(setting)
|
||||
missing = []
|
||||
[missing.append(s) for s, v in six.iteritems(conf) if v is None]
|
||||
if missing:
|
||||
log('Insufficient config data to configure hacluster.', level=ERROR)
|
||||
raise HAIncompleteConfig
|
||||
return conf
|
||||
|
||||
|
||||
def canonical_url(configs, vip_setting='vip'):
|
||||
'''
|
||||
Returns the correct HTTP URL to this host given the state of HTTPS
|
||||
configuration and hacluster.
|
||||
|
||||
:configs : OSTemplateRenderer: A config tempating object to inspect for
|
||||
a complete https context.
|
||||
|
||||
:vip_setting: str: Setting in charm config that specifies
|
||||
VIP address.
|
||||
'''
|
||||
scheme = 'http'
|
||||
if 'https' in configs.complete_contexts():
|
||||
scheme = 'https'
|
||||
if is_clustered():
|
||||
addr = config_get(vip_setting)
|
||||
else:
|
||||
addr = unit_get('private-address')
|
||||
return '%s://%s' % (scheme, addr)
|
|
@ -1,15 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
|
@ -1,482 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
import glob
|
||||
import re
|
||||
import subprocess
|
||||
import six
|
||||
import socket
|
||||
|
||||
from functools import partial
|
||||
|
||||
from charmhelpers.core.hookenv import unit_get
|
||||
from charmhelpers.fetch import apt_install, apt_update
|
||||
from charmhelpers.core.hookenv import (
|
||||
log,
|
||||
WARNING,
|
||||
)
|
||||
|
||||
try:
|
||||
import netifaces
|
||||
except ImportError:
|
||||
apt_update(fatal=True)
|
||||
apt_install('python-netifaces', fatal=True)
|
||||
import netifaces
|
||||
|
||||
try:
|
||||
import netaddr
|
||||
except ImportError:
|
||||
apt_update(fatal=True)
|
||||
apt_install('python-netaddr', fatal=True)
|
||||
import netaddr
|
||||
|
||||
|
||||
def _validate_cidr(network):
|
||||
try:
|
||||
netaddr.IPNetwork(network)
|
||||
except (netaddr.core.AddrFormatError, ValueError):
|
||||
raise ValueError("Network (%s) is not in CIDR presentation format" %
|
||||
network)
|
||||
|
||||
|
||||
def no_ip_found_error_out(network):
|
||||
errmsg = ("No IP address found in network(s): %s" % network)
|
||||
raise ValueError(errmsg)
|
||||
|
||||
|
||||
def get_address_in_network(network, fallback=None, fatal=False):
|
||||
"""Get an IPv4 or IPv6 address within the network from the host.
|
||||
|
||||
:param network (str): CIDR presentation format. For example,
|
||||
'192.168.1.0/24'. Supports multiple networks as a space-delimited list.
|
||||
:param fallback (str): If no address is found, return fallback.
|
||||
:param fatal (boolean): If no address is found, fallback is not
|
||||
set and fatal is True then exit(1).
|
||||
"""
|
||||
if network is None:
|
||||
if fallback is not None:
|
||||
return fallback
|
||||
|
||||
if fatal:
|
||||
no_ip_found_error_out(network)
|
||||
else:
|
||||
return None
|
||||
|
||||
networks = network.split() or [network]
|
||||
for network in networks:
|
||||
_validate_cidr(network)
|
||||
network = netaddr.IPNetwork(network)
|
||||
for iface in netifaces.interfaces():
|
||||
addresses = netifaces.ifaddresses(iface)
|
||||
if network.version == 4 and netifaces.AF_INET in addresses:
|
||||
addr = addresses[netifaces.AF_INET][0]['addr']
|
||||
netmask = addresses[netifaces.AF_INET][0]['netmask']
|
||||
cidr = netaddr.IPNetwork("%s/%s" % (addr, netmask))
|
||||
if cidr in network:
|
||||
return str(cidr.ip)
|
||||
|
||||
if network.version == 6 and netifaces.AF_INET6 in addresses:
|
||||
for addr in addresses[netifaces.AF_INET6]:
|
||||
if not addr['addr'].startswith('fe80'):
|
||||
cidr = netaddr.IPNetwork("%s/%s" % (addr['addr'],
|
||||
addr['netmask']))
|
||||
if cidr in network:
|
||||
return str(cidr.ip)
|
||||
|
||||
if fallback is not None:
|
||||
return fallback
|
||||
|
||||
if fatal:
|
||||
no_ip_found_error_out(network)
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def is_ipv6(address):
|
||||
"""Determine whether provided address is IPv6 or not."""
|
||||
try:
|
||||
address = netaddr.IPAddress(address)
|
||||
except netaddr.AddrFormatError:
|
||||
# probably a hostname - so not an address at all!
|
||||
return False
|
||||
|
||||
return address.version == 6
|
||||
|
||||
|
||||
def is_address_in_network(network, address):
|
||||
"""
|
||||
Determine whether the provided address is within a network range.
|
||||
|
||||
:param network (str): CIDR presentation format. For example,
|
||||
'192.168.1.0/24'.
|
||||
:param address: An individual IPv4 or IPv6 address without a net
|
||||
mask or subnet prefix. For example, '192.168.1.1'.
|
||||
:returns boolean: Flag indicating whether address is in network.
|
||||
"""
|
||||
try:
|
||||
network = netaddr.IPNetwork(network)
|
||||
except (netaddr.core.AddrFormatError, ValueError):
|
||||
raise ValueError("Network (%s) is not in CIDR presentation format" %
|
||||
network)
|
||||
|
||||
try:
|
||||
address = netaddr.IPAddress(address)
|
||||
except (netaddr.core.AddrFormatError, ValueError):
|
||||
raise ValueError("Address (%s) is not in correct presentation format" %
|
||||
address)
|
||||
|
||||
if address in network:
|
||||
return True
|
||||
else:
|
||||
return False
|
||||
|
||||
|
||||
def _get_for_address(address, key):
|
||||
"""Retrieve an attribute of or the physical interface that
|
||||
the IP address provided could be bound to.
|
||||
|
||||
:param address (str): An individual IPv4 or IPv6 address without a net
|
||||
mask or subnet prefix. For example, '192.168.1.1'.
|
||||
:param key: 'iface' for the physical interface name or an attribute
|
||||
of the configured interface, for example 'netmask'.
|
||||
:returns str: Requested attribute or None if address is not bindable.
|
||||
"""
|
||||
address = netaddr.IPAddress(address)
|
||||
for iface in netifaces.interfaces():
|
||||
addresses = netifaces.ifaddresses(iface)
|
||||
if address.version == 4 and netifaces.AF_INET in addresses:
|
||||
addr = addresses[netifaces.AF_INET][0]['addr']
|
||||
netmask = addresses[netifaces.AF_INET][0]['netmask']
|
||||
network = netaddr.IPNetwork("%s/%s" % (addr, netmask))
|
||||
cidr = network.cidr
|
||||
if address in cidr:
|
||||
if key == 'iface':
|
||||
return iface
|
||||
else:
|
||||
return addresses[netifaces.AF_INET][0][key]
|
||||
|
||||
if address.version == 6 and netifaces.AF_INET6 in addresses:
|
||||
for addr in addresses[netifaces.AF_INET6]:
|
||||
if not addr['addr'].startswith('fe80'):
|
||||
network = netaddr.IPNetwork("%s/%s" % (addr['addr'],
|
||||
addr['netmask']))
|
||||
cidr = network.cidr
|
||||
if address in cidr:
|
||||
if key == 'iface':
|
||||
return iface
|
||||
elif key == 'netmask' and cidr:
|
||||
return str(cidr).split('/')[1]
|
||||
else:
|
||||
return addr[key]
|
||||
|
||||
return None
|
||||
|
||||
|
||||
get_iface_for_address = partial(_get_for_address, key='iface')
|
||||
|
||||
|
||||
get_netmask_for_address = partial(_get_for_address, key='netmask')
|
||||
|
||||
|
||||
def resolve_network_cidr(ip_address):
|
||||
'''
|
||||
Resolves the full address cidr of an ip_address based on
|
||||
configured network interfaces
|
||||
'''
|
||||
netmask = get_netmask_for_address(ip_address)
|
||||
return str(netaddr.IPNetwork("%s/%s" % (ip_address, netmask)).cidr)
|
||||
|
||||
|
||||
def format_ipv6_addr(address):
|
||||
"""If address is IPv6, wrap it in '[]' otherwise return None.
|
||||
|
||||
This is required by most configuration files when specifying IPv6
|
||||
addresses.
|
||||
"""
|
||||
if is_ipv6(address):
|
||||
return "[%s]" % address
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def get_iface_addr(iface='eth0', inet_type='AF_INET', inc_aliases=False,
|
||||
fatal=True, exc_list=None):
|
||||
"""Return the assigned IP address for a given interface, if any."""
|
||||
# Extract nic if passed /dev/ethX
|
||||
if '/' in iface:
|
||||
iface = iface.split('/')[-1]
|
||||
|
||||
if not exc_list:
|
||||
exc_list = []
|
||||
|
||||
try:
|
||||
inet_num = getattr(netifaces, inet_type)
|
||||
except AttributeError:
|
||||
raise Exception("Unknown inet type '%s'" % str(inet_type))
|
||||
|
||||
interfaces = netifaces.interfaces()
|
||||
if inc_aliases:
|
||||
ifaces = []
|
||||
for _iface in interfaces:
|
||||
if iface == _iface or _iface.split(':')[0] == iface:
|
||||
ifaces.append(_iface)
|
||||
|
||||
if fatal and not ifaces:
|
||||
raise Exception("Invalid interface '%s'" % iface)
|
||||
|
||||
ifaces.sort()
|
||||
else:
|
||||
if iface not in interfaces:
|
||||
if fatal:
|
||||
raise Exception("Interface '%s' not found " % (iface))
|
||||
else:
|
||||
return []
|
||||
|
||||
else:
|
||||
ifaces = [iface]
|
||||
|
||||
addresses = []
|
||||
for netiface in ifaces:
|
||||
net_info = netifaces.ifaddresses(netiface)
|
||||
if inet_num in net_info:
|
||||
for entry in net_info[inet_num]:
|
||||
if 'addr' in entry and entry['addr'] not in exc_list:
|
||||
addresses.append(entry['addr'])
|
||||
|
||||
if fatal and not addresses:
|
||||
raise Exception("Interface '%s' doesn't have any %s addresses." %
|
||||
(iface, inet_type))
|
||||
|
||||
return sorted(addresses)
|
||||
|
||||
|
||||
get_ipv4_addr = partial(get_iface_addr, inet_type='AF_INET')
|
||||
|
||||
|
||||
def get_iface_from_addr(addr):
|
||||
"""Work out on which interface the provided address is configured."""
|
||||
for iface in netifaces.interfaces():
|
||||
addresses = netifaces.ifaddresses(iface)
|
||||
for inet_type in addresses:
|
||||
for _addr in addresses[inet_type]:
|
||||
_addr = _addr['addr']
|
||||
# link local
|
||||
ll_key = re.compile("(.+)%.*")
|
||||
raw = re.match(ll_key, _addr)
|
||||
if raw:
|
||||
_addr = raw.group(1)
|
||||
|
||||
if _addr == addr:
|
||||
log("Address '%s' is configured on iface '%s'" %
|
||||
(addr, iface))
|
||||
return iface
|
||||
|
||||
msg = "Unable to infer net iface on which '%s' is configured" % (addr)
|
||||
raise Exception(msg)
|
||||
|
||||
|
||||
def sniff_iface(f):
|
||||
"""Ensure decorated function is called with a value for iface.
|
||||
|
||||
If no iface provided, inject net iface inferred from unit private address.
|
||||
"""
|
||||
def iface_sniffer(*args, **kwargs):
|
||||
if not kwargs.get('iface', None):
|
||||
kwargs['iface'] = get_iface_from_addr(unit_get('private-address'))
|
||||
|
||||
return f(*args, **kwargs)
|
||||
|
||||
return iface_sniffer
|
||||
|
||||
|
||||
@sniff_iface
|
||||
def get_ipv6_addr(iface=None, inc_aliases=False, fatal=True, exc_list=None,
|
||||
dynamic_only=True):
|
||||
"""Get assigned IPv6 address for a given interface.
|
||||
|
||||
Returns list of addresses found. If no address found, returns empty list.
|
||||
|
||||
If iface is None, we infer the current primary interface by doing a reverse
|
||||
lookup on the unit private-address.
|
||||
|
||||
We currently only support scope global IPv6 addresses i.e. non-temporary
|
||||
addresses. If no global IPv6 address is found, return the first one found
|
||||
in the ipv6 address list.
|
||||
"""
|
||||
addresses = get_iface_addr(iface=iface, inet_type='AF_INET6',
|
||||
inc_aliases=inc_aliases, fatal=fatal,
|
||||
exc_list=exc_list)
|
||||
|
||||
if addresses:
|
||||
global_addrs = []
|
||||
for addr in addresses:
|
||||
key_scope_link_local = re.compile("^fe80::..(.+)%(.+)")
|
||||
m = re.match(key_scope_link_local, addr)
|
||||
if m:
|
||||
eui_64_mac = m.group(1)
|
||||
iface = m.group(2)
|
||||
else:
|
||||
global_addrs.append(addr)
|
||||
|
||||
if global_addrs:
|
||||
# Make sure any found global addresses are not temporary
|
||||
cmd = ['ip', 'addr', 'show', iface]
|
||||
out = subprocess.check_output(cmd).decode('UTF-8')
|
||||
if dynamic_only:
|
||||
key = re.compile("inet6 (.+)/[0-9]+ scope global dynamic.*")
|
||||
else:
|
||||
key = re.compile("inet6 (.+)/[0-9]+ scope global.*")
|
||||
|
||||
addrs = []
|
||||
for line in out.split('\n'):
|
||||
line = line.strip()
|
||||
m = re.match(key, line)
|
||||
if m and 'temporary' not in line:
|
||||
# Return the first valid address we find
|
||||
for addr in global_addrs:
|
||||
if m.group(1) == addr:
|
||||
if not dynamic_only or \
|
||||
m.group(1).endswith(eui_64_mac):
|
||||
addrs.append(addr)
|
||||
|
||||
if addrs:
|
||||
return addrs
|
||||
|
||||
if fatal:
|
||||
raise Exception("Interface '%s' does not have a scope global "
|
||||
"non-temporary ipv6 address." % iface)
|
||||
|
||||
return []
|
||||
|
||||
|
||||
def get_bridges(vnic_dir='/sys/devices/virtual/net'):
|
||||
"""Return a list of bridges on the system."""
|
||||
b_regex = "%s/*/bridge" % vnic_dir
|
||||
return [x.replace(vnic_dir, '').split('/')[1] for x in glob.glob(b_regex)]
|
||||
|
||||
|
||||
def get_bridge_nics(bridge, vnic_dir='/sys/devices/virtual/net'):
|
||||
"""Return a list of nics comprising a given bridge on the system."""
|
||||
brif_regex = "%s/%s/brif/*" % (vnic_dir, bridge)
|
||||
return [x.split('/')[-1] for x in glob.glob(brif_regex)]
|
||||
|
||||
|
||||
def is_bridge_member(nic):
|
||||
"""Check if a given nic is a member of a bridge."""
|
||||
for bridge in get_bridges():
|
||||
if nic in get_bridge_nics(bridge):
|
||||
return True
|
||||
|
||||
return False
|
||||
|
||||
|
||||
def is_ip(address):
|
||||
"""
|
||||
Returns True if address is a valid IP address.
|
||||
"""
|
||||
try:
|
||||
# Test to see if already an IPv4 address
|
||||
socket.inet_aton(address)
|
||||
return True
|
||||
except socket.error:
|
||||
return False
|
||||
|
||||
|
||||
def ns_query(address):
|
||||
try:
|
||||
import dns.resolver
|
||||
except ImportError:
|
||||
apt_install('python-dnspython')
|
||||
import dns.resolver
|
||||
|
||||
if isinstance(address, dns.name.Name):
|
||||
rtype = 'PTR'
|
||||
elif isinstance(address, six.string_types):
|
||||
rtype = 'A'
|
||||
else:
|
||||
return None
|
||||
|
||||
answers = dns.resolver.query(address, rtype)
|
||||
if answers:
|
||||
return str(answers[0])
|
||||
return None
|
||||
|
||||
|
||||
def get_host_ip(hostname, fallback=None):
|
||||
"""
|
||||
Resolves the IP for a given hostname, or returns
|
||||
the input if it is already an IP.
|
||||
"""
|
||||
if is_ip(hostname):
|
||||
return hostname
|
||||
|
||||
ip_addr = ns_query(hostname)
|
||||
if not ip_addr:
|
||||
try:
|
||||
ip_addr = socket.gethostbyname(hostname)
|
||||
except:
|
||||
log("Failed to resolve hostname '%s'" % (hostname),
|
||||
level=WARNING)
|
||||
return fallback
|
||||
return ip_addr
|
||||
|
||||
|
||||
def get_hostname(address, fqdn=True):
|
||||
"""
|
||||
Resolves hostname for given IP, or returns the input
|
||||
if it is already a hostname.
|
||||
"""
|
||||
if is_ip(address):
|
||||
try:
|
||||
import dns.reversename
|
||||
except ImportError:
|
||||
apt_install("python-dnspython")
|
||||
import dns.reversename
|
||||
|
||||
rev = dns.reversename.from_address(address)
|
||||
result = ns_query(rev)
|
||||
|
||||
if not result:
|
||||
try:
|
||||
result = socket.gethostbyaddr(address)[0]
|
||||
except:
|
||||
return None
|
||||
else:
|
||||
result = address
|
||||
|
||||
if fqdn:
|
||||
# strip trailing .
|
||||
if result.endswith('.'):
|
||||
return result[:-1]
|
||||
else:
|
||||
return result
|
||||
else:
|
||||
return result.split('.')[0]
|
||||
|
||||
|
||||
def port_has_listener(address, port):
|
||||
"""
|
||||
Returns True if the address:port is open and being listened to,
|
||||
else False.
|
||||
|
||||
@param address: an IP address or hostname
|
||||
@param port: integer port
|
||||
|
||||
Note calls 'zc' via a subprocess shell
|
||||
"""
|
||||
cmd = ['nc', '-z', address, str(port)]
|
||||
result = subprocess.call(cmd)
|
||||
return not(bool(result))
|
|
@ -1,100 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
''' Helpers for interacting with OpenvSwitch '''
|
||||
import subprocess
|
||||
import os
|
||||
from charmhelpers.core.hookenv import (
|
||||
log, WARNING
|
||||
)
|
||||
from charmhelpers.core.host import (
|
||||
service
|
||||
)
|
||||
|
||||
|
||||
def add_bridge(name, datapath_type=None):
|
||||
''' Add the named bridge to openvswitch '''
|
||||
log('Creating bridge {}'.format(name))
|
||||
cmd = ["ovs-vsctl", "--", "--may-exist", "add-br", name]
|
||||
if datapath_type is not None:
|
||||
cmd += ['--', 'set', 'bridge', name,
|
||||
'datapath_type={}'.format(datapath_type)]
|
||||
subprocess.check_call(cmd)
|
||||
|
||||
|
||||
def del_bridge(name):
|
||||
''' Delete the named bridge from openvswitch '''
|
||||
log('Deleting bridge {}'.format(name))
|
||||
subprocess.check_call(["ovs-vsctl", "--", "--if-exists", "del-br", name])
|
||||
|
||||
|
||||
def add_bridge_port(name, port, promisc=False):
|
||||
''' Add a port to the named openvswitch bridge '''
|
||||
log('Adding port {} to bridge {}'.format(port, name))
|
||||
subprocess.check_call(["ovs-vsctl", "--", "--may-exist", "add-port",
|
||||
name, port])
|
||||
subprocess.check_call(["ip", "link", "set", port, "up"])
|
||||
if promisc:
|
||||
subprocess.check_call(["ip", "link", "set", port, "promisc", "on"])
|
||||
else:
|
||||
subprocess.check_call(["ip", "link", "set", port, "promisc", "off"])
|
||||
|
||||
|
||||
def del_bridge_port(name, port):
|
||||
''' Delete a port from the named openvswitch bridge '''
|
||||
log('Deleting port {} from bridge {}'.format(port, name))
|
||||
subprocess.check_call(["ovs-vsctl", "--", "--if-exists", "del-port",
|
||||
name, port])
|
||||
subprocess.check_call(["ip", "link", "set", port, "down"])
|
||||
subprocess.check_call(["ip", "link", "set", port, "promisc", "off"])
|
||||
|
||||
|
||||
def set_manager(manager):
|
||||
''' Set the controller for the local openvswitch '''
|
||||
log('Setting manager for local ovs to {}'.format(manager))
|
||||
subprocess.check_call(['ovs-vsctl', 'set-manager',
|
||||
'ssl:{}'.format(manager)])
|
||||
|
||||
|
||||
CERT_PATH = '/etc/openvswitch/ovsclient-cert.pem'
|
||||
|
||||
|
||||
def get_certificate():
|
||||
''' Read openvswitch certificate from disk '''
|
||||
if os.path.exists(CERT_PATH):
|
||||
log('Reading ovs certificate from {}'.format(CERT_PATH))
|
||||
with open(CERT_PATH, 'r') as cert:
|
||||
full_cert = cert.read()
|
||||
begin_marker = "-----BEGIN CERTIFICATE-----"
|
||||
end_marker = "-----END CERTIFICATE-----"
|
||||
begin_index = full_cert.find(begin_marker)
|
||||
end_index = full_cert.rfind(end_marker)
|
||||
if end_index == -1 or begin_index == -1:
|
||||
raise RuntimeError("Certificate does not contain valid begin"
|
||||
" and end markers.")
|
||||
full_cert = full_cert[begin_index:(end_index + len(end_marker))]
|
||||
return full_cert
|
||||
else:
|
||||
log('Certificate not found', level=WARNING)
|
||||
return None
|
||||
|
||||
|
||||
def full_restart():
|
||||
''' Full restart and reload of openvswitch '''
|
||||
if os.path.exists('/etc/init/openvswitch-force-reload-kmod.conf'):
|
||||
service('start', 'openvswitch-force-reload-kmod')
|
||||
else:
|
||||
service('force-reload-kmod', 'openvswitch-switch')
|
|
@ -1,318 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
"""
|
||||
This module contains helpers to add and remove ufw rules.
|
||||
|
||||
Examples:
|
||||
|
||||
- open SSH port for subnet 10.0.3.0/24:
|
||||
|
||||
>>> from charmhelpers.contrib.network import ufw
|
||||
>>> ufw.enable()
|
||||
>>> ufw.grant_access(src='10.0.3.0/24', dst='any', port='22', proto='tcp')
|
||||
|
||||
- open service by name as defined in /etc/services:
|
||||
|
||||
>>> from charmhelpers.contrib.network import ufw
|
||||
>>> ufw.enable()
|
||||
>>> ufw.service('ssh', 'open')
|
||||
|
||||
- close service by port number:
|
||||
|
||||
>>> from charmhelpers.contrib.network import ufw
|
||||
>>> ufw.enable()
|
||||
>>> ufw.service('4949', 'close') # munin
|
||||
"""
|
||||
import re
|
||||
import os
|
||||
import subprocess
|
||||
|
||||
from charmhelpers.core import hookenv
|
||||
from charmhelpers.core.kernel import modprobe, is_module_loaded
|
||||
|
||||
__author__ = "Felipe Reyes <felipe.reyes@canonical.com>"
|
||||
|
||||
|
||||
class UFWError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
class UFWIPv6Error(UFWError):
|
||||
pass
|
||||
|
||||
|
||||
def is_enabled():
|
||||
"""
|
||||
Check if `ufw` is enabled
|
||||
|
||||
:returns: True if ufw is enabled
|
||||
"""
|
||||
output = subprocess.check_output(['ufw', 'status'],
|
||||
universal_newlines=True,
|
||||
env={'LANG': 'en_US',
|
||||
'PATH': os.environ['PATH']})
|
||||
|
||||
m = re.findall(r'^Status: active\n', output, re.M)
|
||||
|
||||
return len(m) >= 1
|
||||
|
||||
|
||||
def is_ipv6_ok(soft_fail=False):
|
||||
"""
|
||||
Check if IPv6 support is present and ip6tables functional
|
||||
|
||||
:param soft_fail: If set to True and IPv6 support is broken, then reports
|
||||
that the host doesn't have IPv6 support, otherwise a
|
||||
UFWIPv6Error exception is raised.
|
||||
:returns: True if IPv6 is working, False otherwise
|
||||
"""
|
||||
|
||||
# do we have IPv6 in the machine?
|
||||
if os.path.isdir('/proc/sys/net/ipv6'):
|
||||
# is ip6tables kernel module loaded?
|
||||
if not is_module_loaded('ip6_tables'):
|
||||
# ip6tables support isn't complete, let's try to load it
|
||||
try:
|
||||
modprobe('ip6_tables')
|
||||
# great, we can load the module
|
||||
return True
|
||||
except subprocess.CalledProcessError as ex:
|
||||
hookenv.log("Couldn't load ip6_tables module: %s" % ex.output,
|
||||
level="WARN")
|
||||
# we are in a world where ip6tables isn't working
|
||||
if soft_fail:
|
||||
# so we inform that the machine doesn't have IPv6
|
||||
return False
|
||||
else:
|
||||
raise UFWIPv6Error("IPv6 firewall support broken")
|
||||
else:
|
||||
# the module is present :)
|
||||
return True
|
||||
|
||||
else:
|
||||
# the system doesn't have IPv6
|
||||
return False
|
||||
|
||||
|
||||
def disable_ipv6():
|
||||
"""
|
||||
Disable ufw IPv6 support in /etc/default/ufw
|
||||
"""
|
||||
exit_code = subprocess.call(['sed', '-i', 's/IPV6=.*/IPV6=no/g',
|
||||
'/etc/default/ufw'])
|
||||
if exit_code == 0:
|
||||
hookenv.log('IPv6 support in ufw disabled', level='INFO')
|
||||
else:
|
||||
hookenv.log("Couldn't disable IPv6 support in ufw", level="ERROR")
|
||||
raise UFWError("Couldn't disable IPv6 support in ufw")
|
||||
|
||||
|
||||
def enable(soft_fail=False):
|
||||
"""
|
||||
Enable ufw
|
||||
|
||||
:param soft_fail: If set to True silently disables IPv6 support in ufw,
|
||||
otherwise a UFWIPv6Error exception is raised when IP6
|
||||
support is broken.
|
||||
:returns: True if ufw is successfully enabled
|
||||
"""
|
||||
if is_enabled():
|
||||
return True
|
||||
|
||||
if not is_ipv6_ok(soft_fail):
|
||||
disable_ipv6()
|
||||
|
||||
output = subprocess.check_output(['ufw', 'enable'],
|
||||
universal_newlines=True,
|
||||
env={'LANG': 'en_US',
|
||||
'PATH': os.environ['PATH']})
|
||||
|
||||
m = re.findall('^Firewall is active and enabled on system startup\n',
|
||||
output, re.M)
|
||||
hookenv.log(output, level='DEBUG')
|
||||
|
||||
if len(m) == 0:
|
||||
hookenv.log("ufw couldn't be enabled", level='WARN')
|
||||
return False
|
||||
else:
|
||||
hookenv.log("ufw enabled", level='INFO')
|
||||
return True
|
||||
|
||||
|
||||
def disable():
|
||||
"""
|
||||
Disable ufw
|
||||
|
||||
:returns: True if ufw is successfully disabled
|
||||
"""
|
||||
if not is_enabled():
|
||||
return True
|
||||
|
||||
output = subprocess.check_output(['ufw', 'disable'],
|
||||
universal_newlines=True,
|
||||
env={'LANG': 'en_US',
|
||||
'PATH': os.environ['PATH']})
|
||||
|
||||
m = re.findall(r'^Firewall stopped and disabled on system startup\n',
|
||||
output, re.M)
|
||||
hookenv.log(output, level='DEBUG')
|
||||
|
||||
if len(m) == 0:
|
||||
hookenv.log("ufw couldn't be disabled", level='WARN')
|
||||
return False
|
||||
else:
|
||||
hookenv.log("ufw disabled", level='INFO')
|
||||
return True
|
||||
|
||||
|
||||
def default_policy(policy='deny', direction='incoming'):
|
||||
"""
|
||||
Changes the default policy for traffic `direction`
|
||||
|
||||
:param policy: allow, deny or reject
|
||||
:param direction: traffic direction, possible values: incoming, outgoing,
|
||||
routed
|
||||
"""
|
||||
if policy not in ['allow', 'deny', 'reject']:
|
||||
raise UFWError(('Unknown policy %s, valid values: '
|
||||
'allow, deny, reject') % policy)
|
||||
|
||||
if direction not in ['incoming', 'outgoing', 'routed']:
|
||||
raise UFWError(('Unknown direction %s, valid values: '
|
||||
'incoming, outgoing, routed') % direction)
|
||||
|
||||
output = subprocess.check_output(['ufw', 'default', policy, direction],
|
||||
universal_newlines=True,
|
||||
env={'LANG': 'en_US',
|
||||
'PATH': os.environ['PATH']})
|
||||
hookenv.log(output, level='DEBUG')
|
||||
|
||||
m = re.findall("^Default %s policy changed to '%s'\n" % (direction,
|
||||
policy),
|
||||
output, re.M)
|
||||
if len(m) == 0:
|
||||
hookenv.log("ufw couldn't change the default policy to %s for %s"
|
||||
% (policy, direction), level='WARN')
|
||||
return False
|
||||
else:
|
||||
hookenv.log("ufw default policy for %s changed to %s"
|
||||
% (direction, policy), level='INFO')
|
||||
return True
|
||||
|
||||
|
||||
def modify_access(src, dst='any', port=None, proto=None, action='allow',
|
||||
index=None):
|
||||
"""
|
||||
Grant access to an address or subnet
|
||||
|
||||
:param src: address (e.g. 192.168.1.234) or subnet
|
||||
(e.g. 192.168.1.0/24).
|
||||
:param dst: destiny of the connection, if the machine has multiple IPs and
|
||||
connections to only one of those have to accepted this is the
|
||||
field has to be set.
|
||||
:param port: destiny port
|
||||
:param proto: protocol (tcp or udp)
|
||||
:param action: `allow` or `delete`
|
||||
:param index: if different from None the rule is inserted at the given
|
||||
`index`.
|
||||
"""
|
||||
if not is_enabled():
|
||||
hookenv.log('ufw is disabled, skipping modify_access()', level='WARN')
|
||||
return
|
||||
|
||||
if action == 'delete':
|
||||
cmd = ['ufw', 'delete', 'allow']
|
||||
elif index is not None:
|
||||
cmd = ['ufw', 'insert', str(index), action]
|
||||
else:
|
||||
cmd = ['ufw', action]
|
||||
|
||||
if src is not None:
|
||||
cmd += ['from', src]
|
||||
|
||||
if dst is not None:
|
||||
cmd += ['to', dst]
|
||||
|
||||
if port is not None:
|
||||
cmd += ['port', str(port)]
|
||||
|
||||
if proto is not None:
|
||||
cmd += ['proto', proto]
|
||||
|
||||
hookenv.log('ufw {}: {}'.format(action, ' '.join(cmd)), level='DEBUG')
|
||||
p = subprocess.Popen(cmd, stdout=subprocess.PIPE)
|
||||
(stdout, stderr) = p.communicate()
|
||||
|
||||
hookenv.log(stdout, level='INFO')
|
||||
|
||||
if p.returncode != 0:
|
||||
hookenv.log(stderr, level='ERROR')
|
||||
hookenv.log('Error running: {}, exit code: {}'.format(' '.join(cmd),
|
||||
p.returncode),
|
||||
level='ERROR')
|
||||
|
||||
|
||||
def grant_access(src, dst='any', port=None, proto=None, index=None):
|
||||
"""
|
||||
Grant access to an address or subnet
|
||||
|
||||
:param src: address (e.g. 192.168.1.234) or subnet
|
||||
(e.g. 192.168.1.0/24).
|
||||
:param dst: destiny of the connection, if the machine has multiple IPs and
|
||||
connections to only one of those have to accepted this is the
|
||||
field has to be set.
|
||||
:param port: destiny port
|
||||
:param proto: protocol (tcp or udp)
|
||||
:param index: if different from None the rule is inserted at the given
|
||||
`index`.
|
||||
"""
|
||||
return modify_access(src, dst=dst, port=port, proto=proto, action='allow',
|
||||
index=index)
|
||||
|
||||
|
||||
def revoke_access(src, dst='any', port=None, proto=None):
|
||||
"""
|
||||
Revoke access to an address or subnet
|
||||
|
||||
:param src: address (e.g. 192.168.1.234) or subnet
|
||||
(e.g. 192.168.1.0/24).
|
||||
:param dst: destiny of the connection, if the machine has multiple IPs and
|
||||
connections to only one of those have to accepted this is the
|
||||
field has to be set.
|
||||
:param port: destiny port
|
||||
:param proto: protocol (tcp or udp)
|
||||
"""
|
||||
return modify_access(src, dst=dst, port=port, proto=proto, action='delete')
|
||||
|
||||
|
||||
def service(name, action):
|
||||
"""
|
||||
Open/close access to a service
|
||||
|
||||
:param name: could be a service name defined in `/etc/services` or a port
|
||||
number.
|
||||
:param action: `open` or `close`
|
||||
"""
|
||||
if action == 'open':
|
||||
subprocess.check_output(['ufw', 'allow', str(name)],
|
||||
universal_newlines=True)
|
||||
elif action == 'close':
|
||||
subprocess.check_output(['ufw', 'delete', 'allow', str(name)],
|
||||
universal_newlines=True)
|
||||
else:
|
||||
raise UFWError(("'{}' not supported, use 'allow' "
|
||||
"or 'delete'").format(action))
|
|
@ -1,15 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
|
@ -1,33 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
''' Helper for managing alternatives for file conflict resolution '''
|
||||
|
||||
import subprocess
|
||||
import shutil
|
||||
import os
|
||||
|
||||
|
||||
def install_alternative(name, target, source, priority=50):
|
||||
''' Install alternative configuration '''
|
||||
if (os.path.exists(target) and not os.path.islink(target)):
|
||||
# Move existing file/directory away before installing
|
||||
shutil.move(target, '{}.bak'.format(target))
|
||||
cmd = [
|
||||
'update-alternatives', '--force', '--install',
|
||||
target, name, source, str(priority)
|
||||
]
|
||||
subprocess.check_call(cmd)
|
|
@ -1,15 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
|
@ -1,304 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
import logging
|
||||
import re
|
||||
import sys
|
||||
import six
|
||||
from collections import OrderedDict
|
||||
from charmhelpers.contrib.amulet.deployment import (
|
||||
AmuletDeployment
|
||||
)
|
||||
|
||||
DEBUG = logging.DEBUG
|
||||
ERROR = logging.ERROR
|
||||
|
||||
|
||||
class OpenStackAmuletDeployment(AmuletDeployment):
|
||||
"""OpenStack amulet deployment.
|
||||
|
||||
This class inherits from AmuletDeployment and has additional support
|
||||
that is specifically for use by OpenStack charms.
|
||||
"""
|
||||
|
||||
def __init__(self, series=None, openstack=None, source=None,
|
||||
stable=True, log_level=DEBUG):
|
||||
"""Initialize the deployment environment."""
|
||||
super(OpenStackAmuletDeployment, self).__init__(series)
|
||||
self.log = self.get_logger(level=log_level)
|
||||
self.log.info('OpenStackAmuletDeployment: init')
|
||||
self.openstack = openstack
|
||||
self.source = source
|
||||
self.stable = stable
|
||||
# Note(coreycb): this needs to be changed when new next branches come
|
||||
# out.
|
||||
self.current_next = "trusty"
|
||||
|
||||
def get_logger(self, name="deployment-logger", level=logging.DEBUG):
|
||||
"""Get a logger object that will log to stdout."""
|
||||
log = logging
|
||||
logger = log.getLogger(name)
|
||||
fmt = log.Formatter("%(asctime)s %(funcName)s "
|
||||
"%(levelname)s: %(message)s")
|
||||
|
||||
handler = log.StreamHandler(stream=sys.stdout)
|
||||
handler.setLevel(level)
|
||||
handler.setFormatter(fmt)
|
||||
|
||||
logger.addHandler(handler)
|
||||
logger.setLevel(level)
|
||||
|
||||
return logger
|
||||
|
||||
def _determine_branch_locations(self, other_services):
|
||||
"""Determine the branch locations for the other services.
|
||||
|
||||
Determine if the local branch being tested is derived from its
|
||||
stable or next (dev) branch, and based on this, use the corresonding
|
||||
stable or next branches for the other_services."""
|
||||
|
||||
self.log.info('OpenStackAmuletDeployment: determine branch locations')
|
||||
|
||||
# Charms outside the lp:~openstack-charmers namespace
|
||||
base_charms = ['mysql', 'mongodb', 'nrpe']
|
||||
|
||||
# Force these charms to current series even when using an older series.
|
||||
# ie. Use trusty/nrpe even when series is precise, as the P charm
|
||||
# does not possess the necessary external master config and hooks.
|
||||
force_series_current = ['nrpe']
|
||||
|
||||
if self.series in ['precise', 'trusty']:
|
||||
base_series = self.series
|
||||
else:
|
||||
base_series = self.current_next
|
||||
|
||||
for svc in other_services:
|
||||
if svc['name'] in force_series_current:
|
||||
base_series = self.current_next
|
||||
# If a location has been explicitly set, use it
|
||||
if svc.get('location'):
|
||||
continue
|
||||
if self.stable:
|
||||
temp = 'lp:charms/{}/{}'
|
||||
svc['location'] = temp.format(base_series,
|
||||
svc['name'])
|
||||
else:
|
||||
if svc['name'] in base_charms:
|
||||
temp = 'lp:charms/{}/{}'
|
||||
svc['location'] = temp.format(base_series,
|
||||
svc['name'])
|
||||
else:
|
||||
temp = 'lp:~openstack-charmers/charms/{}/{}/next'
|
||||
svc['location'] = temp.format(self.current_next,
|
||||
svc['name'])
|
||||
|
||||
return other_services
|
||||
|
||||
def _add_services(self, this_service, other_services):
|
||||
"""Add services to the deployment and set openstack-origin/source."""
|
||||
self.log.info('OpenStackAmuletDeployment: adding services')
|
||||
|
||||
other_services = self._determine_branch_locations(other_services)
|
||||
|
||||
super(OpenStackAmuletDeployment, self)._add_services(this_service,
|
||||
other_services)
|
||||
|
||||
services = other_services
|
||||
services.append(this_service)
|
||||
|
||||
# Charms which should use the source config option
|
||||
use_source = ['mysql', 'mongodb', 'rabbitmq-server', 'ceph',
|
||||
'ceph-osd', 'ceph-radosgw', 'ceph-mon']
|
||||
|
||||
# Charms which can not use openstack-origin, ie. many subordinates
|
||||
no_origin = ['cinder-ceph', 'hacluster', 'neutron-openvswitch', 'nrpe',
|
||||
'openvswitch-odl', 'neutron-api-odl', 'odl-controller',
|
||||
'cinder-backup', 'nexentaedge-data',
|
||||
'nexentaedge-iscsi-gw', 'nexentaedge-swift-gw',
|
||||
'cinder-nexentaedge', 'nexentaedge-mgmt']
|
||||
|
||||
if self.openstack:
|
||||
for svc in services:
|
||||
if svc['name'] not in use_source + no_origin:
|
||||
config = {'openstack-origin': self.openstack}
|
||||
self.d.configure(svc['name'], config)
|
||||
|
||||
if self.source:
|
||||
for svc in services:
|
||||
if svc['name'] in use_source and svc['name'] not in no_origin:
|
||||
config = {'source': self.source}
|
||||
self.d.configure(svc['name'], config)
|
||||
|
||||
def _configure_services(self, configs):
|
||||
"""Configure all of the services."""
|
||||
self.log.info('OpenStackAmuletDeployment: configure services')
|
||||
for service, config in six.iteritems(configs):
|
||||
self.d.configure(service, config)
|
||||
|
||||
def _auto_wait_for_status(self, message=None, exclude_services=None,
|
||||
include_only=None, timeout=1800):
|
||||
"""Wait for all units to have a specific extended status, except
|
||||
for any defined as excluded. Unless specified via message, any
|
||||
status containing any case of 'ready' will be considered a match.
|
||||
|
||||
Examples of message usage:
|
||||
|
||||
Wait for all unit status to CONTAIN any case of 'ready' or 'ok':
|
||||
message = re.compile('.*ready.*|.*ok.*', re.IGNORECASE)
|
||||
|
||||
Wait for all units to reach this status (exact match):
|
||||
message = re.compile('^Unit is ready and clustered$')
|
||||
|
||||
Wait for all units to reach any one of these (exact match):
|
||||
message = re.compile('Unit is ready|OK|Ready')
|
||||
|
||||
Wait for at least one unit to reach this status (exact match):
|
||||
message = {'ready'}
|
||||
|
||||
See Amulet's sentry.wait_for_messages() for message usage detail.
|
||||
https://github.com/juju/amulet/blob/master/amulet/sentry.py
|
||||
|
||||
:param message: Expected status match
|
||||
:param exclude_services: List of juju service names to ignore,
|
||||
not to be used in conjuction with include_only.
|
||||
:param include_only: List of juju service names to exclusively check,
|
||||
not to be used in conjuction with exclude_services.
|
||||
:param timeout: Maximum time in seconds to wait for status match
|
||||
:returns: None. Raises if timeout is hit.
|
||||
"""
|
||||
self.log.info('Waiting for extended status on units...')
|
||||
|
||||
all_services = self.d.services.keys()
|
||||
|
||||
if exclude_services and include_only:
|
||||
raise ValueError('exclude_services can not be used '
|
||||
'with include_only')
|
||||
|
||||
if message:
|
||||
if isinstance(message, re._pattern_type):
|
||||
match = message.pattern
|
||||
else:
|
||||
match = message
|
||||
|
||||
self.log.debug('Custom extended status wait match: '
|
||||
'{}'.format(match))
|
||||
else:
|
||||
self.log.debug('Default extended status wait match: contains '
|
||||
'READY (case-insensitive)')
|
||||
message = re.compile('.*ready.*', re.IGNORECASE)
|
||||
|
||||
if exclude_services:
|
||||
self.log.debug('Excluding services from extended status match: '
|
||||
'{}'.format(exclude_services))
|
||||
else:
|
||||
exclude_services = []
|
||||
|
||||
if include_only:
|
||||
services = include_only
|
||||
else:
|
||||
services = list(set(all_services) - set(exclude_services))
|
||||
|
||||
self.log.debug('Waiting up to {}s for extended status on services: '
|
||||
'{}'.format(timeout, services))
|
||||
service_messages = {service: message for service in services}
|
||||
self.d.sentry.wait_for_messages(service_messages, timeout=timeout)
|
||||
self.log.info('OK')
|
||||
|
||||
def _get_openstack_release(self):
|
||||
"""Get openstack release.
|
||||
|
||||
Return an integer representing the enum value of the openstack
|
||||
release.
|
||||
"""
|
||||
# Must be ordered by OpenStack release (not by Ubuntu release):
|
||||
(self.precise_essex, self.precise_folsom, self.precise_grizzly,
|
||||
self.precise_havana, self.precise_icehouse,
|
||||
self.trusty_icehouse, self.trusty_juno, self.utopic_juno,
|
||||
self.trusty_kilo, self.vivid_kilo, self.trusty_liberty,
|
||||
self.wily_liberty, self.trusty_mitaka,
|
||||
self.xenial_mitaka) = range(14)
|
||||
|
||||
releases = {
|
||||
('precise', None): self.precise_essex,
|
||||
('precise', 'cloud:precise-folsom'): self.precise_folsom,
|
||||
('precise', 'cloud:precise-grizzly'): self.precise_grizzly,
|
||||
('precise', 'cloud:precise-havana'): self.precise_havana,
|
||||
('precise', 'cloud:precise-icehouse'): self.precise_icehouse,
|
||||
('trusty', None): self.trusty_icehouse,
|
||||
('trusty', 'cloud:trusty-juno'): self.trusty_juno,
|
||||
('trusty', 'cloud:trusty-kilo'): self.trusty_kilo,
|
||||
('trusty', 'cloud:trusty-liberty'): self.trusty_liberty,
|
||||
('trusty', 'cloud:trusty-mitaka'): self.trusty_mitaka,
|
||||
('utopic', None): self.utopic_juno,
|
||||
('vivid', None): self.vivid_kilo,
|
||||
('wily', None): self.wily_liberty,
|
||||
('xenial', None): self.xenial_mitaka}
|
||||
return releases[(self.series, self.openstack)]
|
||||
|
||||
def _get_openstack_release_string(self):
|
||||
"""Get openstack release string.
|
||||
|
||||
Return a string representing the openstack release.
|
||||
"""
|
||||
releases = OrderedDict([
|
||||
('precise', 'essex'),
|
||||
('quantal', 'folsom'),
|
||||
('raring', 'grizzly'),
|
||||
('saucy', 'havana'),
|
||||
('trusty', 'icehouse'),
|
||||
('utopic', 'juno'),
|
||||
('vivid', 'kilo'),
|
||||
('wily', 'liberty'),
|
||||
('xenial', 'mitaka'),
|
||||
])
|
||||
if self.openstack:
|
||||
os_origin = self.openstack.split(':')[1]
|
||||
return os_origin.split('%s-' % self.series)[1].split('/')[0]
|
||||
else:
|
||||
return releases[self.series]
|
||||
|
||||
def get_ceph_expected_pools(self, radosgw=False):
|
||||
"""Return a list of expected ceph pools in a ceph + cinder + glance
|
||||
test scenario, based on OpenStack release and whether ceph radosgw
|
||||
is flagged as present or not."""
|
||||
|
||||
if self._get_openstack_release() >= self.trusty_kilo:
|
||||
# Kilo or later
|
||||
pools = [
|
||||
'rbd',
|
||||
'cinder',
|
||||
'glance'
|
||||
]
|
||||
else:
|
||||
# Juno or earlier
|
||||
pools = [
|
||||
'data',
|
||||
'metadata',
|
||||
'rbd',
|
||||
'cinder',
|
||||
'glance'
|
||||
]
|
||||
|
||||
if radosgw:
|
||||
pools.extend([
|
||||
'.rgw.root',
|
||||
'.rgw.control',
|
||||
'.rgw',
|
||||
'.rgw.gc',
|
||||
'.users.uid'
|
||||
])
|
||||
|
||||
return pools
|
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
|
@ -1,18 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
# dummy __init__.py to fool syncer into thinking this is a syncable python
|
||||
# module
|
|
@ -1,179 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
|
||||
from charmhelpers.core.hookenv import (
|
||||
config,
|
||||
unit_get,
|
||||
service_name,
|
||||
network_get_primary_address,
|
||||
)
|
||||
from charmhelpers.contrib.network.ip import (
|
||||
get_address_in_network,
|
||||
is_address_in_network,
|
||||
is_ipv6,
|
||||
get_ipv6_addr,
|
||||
resolve_network_cidr,
|
||||
)
|
||||
from charmhelpers.contrib.hahelpers.cluster import is_clustered
|
||||
|
||||
PUBLIC = 'public'
|
||||
INTERNAL = 'int'
|
||||
ADMIN = 'admin'
|
||||
|
||||
ADDRESS_MAP = {
|
||||
PUBLIC: {
|
||||
'binding': 'public',
|
||||
'config': 'os-public-network',
|
||||
'fallback': 'public-address',
|
||||
'override': 'os-public-hostname',
|
||||
},
|
||||
INTERNAL: {
|
||||
'binding': 'internal',
|
||||
'config': 'os-internal-network',
|
||||
'fallback': 'private-address',
|
||||
'override': 'os-internal-hostname',
|
||||
},
|
||||
ADMIN: {
|
||||
'binding': 'admin',
|
||||
'config': 'os-admin-network',
|
||||
'fallback': 'private-address',
|
||||
'override': 'os-admin-hostname',
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
def canonical_url(configs, endpoint_type=PUBLIC):
|
||||
"""Returns the correct HTTP URL to this host given the state of HTTPS
|
||||
configuration, hacluster and charm configuration.
|
||||
|
||||
:param configs: OSTemplateRenderer config templating object to inspect
|
||||
for a complete https context.
|
||||
:param endpoint_type: str endpoint type to resolve.
|
||||
:param returns: str base URL for services on the current service unit.
|
||||
"""
|
||||
scheme = _get_scheme(configs)
|
||||
|
||||
address = resolve_address(endpoint_type)
|
||||
if is_ipv6(address):
|
||||
address = "[{}]".format(address)
|
||||
|
||||
return '%s://%s' % (scheme, address)
|
||||
|
||||
|
||||
def _get_scheme(configs):
|
||||
"""Returns the scheme to use for the url (either http or https)
|
||||
depending upon whether https is in the configs value.
|
||||
|
||||
:param configs: OSTemplateRenderer config templating object to inspect
|
||||
for a complete https context.
|
||||
:returns: either 'http' or 'https' depending on whether https is
|
||||
configured within the configs context.
|
||||
"""
|
||||
scheme = 'http'
|
||||
if configs and 'https' in configs.complete_contexts():
|
||||
scheme = 'https'
|
||||
return scheme
|
||||
|
||||
|
||||
def _get_address_override(endpoint_type=PUBLIC):
|
||||
"""Returns any address overrides that the user has defined based on the
|
||||
endpoint type.
|
||||
|
||||
Note: this function allows for the service name to be inserted into the
|
||||
address if the user specifies {service_name}.somehost.org.
|
||||
|
||||
:param endpoint_type: the type of endpoint to retrieve the override
|
||||
value for.
|
||||
:returns: any endpoint address or hostname that the user has overridden
|
||||
or None if an override is not present.
|
||||
"""
|
||||
override_key = ADDRESS_MAP[endpoint_type]['override']
|
||||
addr_override = config(override_key)
|
||||
if not addr_override:
|
||||
return None
|
||||
else:
|
||||
return addr_override.format(service_name=service_name())
|
||||
|
||||
|
||||
def resolve_address(endpoint_type=PUBLIC):
|
||||
"""Return unit address depending on net config.
|
||||
|
||||
If unit is clustered with vip(s) and has net splits defined, return vip on
|
||||
correct network. If clustered with no nets defined, return primary vip.
|
||||
|
||||
If not clustered, return unit address ensuring address is on configured net
|
||||
split if one is configured, or a Juju 2.0 extra-binding has been used.
|
||||
|
||||
:param endpoint_type: Network endpoing type
|
||||
"""
|
||||
resolved_address = _get_address_override(endpoint_type)
|
||||
if resolved_address:
|
||||
return resolved_address
|
||||
|
||||
vips = config('vip')
|
||||
if vips:
|
||||
vips = vips.split()
|
||||
|
||||
net_type = ADDRESS_MAP[endpoint_type]['config']
|
||||
net_addr = config(net_type)
|
||||
net_fallback = ADDRESS_MAP[endpoint_type]['fallback']
|
||||
binding = ADDRESS_MAP[endpoint_type]['binding']
|
||||
clustered = is_clustered()
|
||||
|
||||
if clustered and vips:
|
||||
if net_addr:
|
||||
for vip in vips:
|
||||
if is_address_in_network(net_addr, vip):
|
||||
resolved_address = vip
|
||||
break
|
||||
else:
|
||||
# NOTE: endeavour to check vips against network space
|
||||
# bindings
|
||||
try:
|
||||
bound_cidr = resolve_network_cidr(
|
||||
network_get_primary_address(binding)
|
||||
)
|
||||
for vip in vips:
|
||||
if is_address_in_network(bound_cidr, vip):
|
||||
resolved_address = vip
|
||||
break
|
||||
except NotImplementedError:
|
||||
# If no net-splits configured and no support for extra
|
||||
# bindings/network spaces so we expect a single vip
|
||||
resolved_address = vips[0]
|
||||
else:
|
||||
if config('prefer-ipv6'):
|
||||
fallback_addr = get_ipv6_addr(exc_list=vips)[0]
|
||||
else:
|
||||
fallback_addr = unit_get(net_fallback)
|
||||
|
||||
if net_addr:
|
||||
resolved_address = get_address_in_network(net_addr, fallback_addr)
|
||||
else:
|
||||
# NOTE: only try to use extra bindings if legacy network
|
||||
# configuration is not in use
|
||||
try:
|
||||
resolved_address = network_get_primary_address(binding)
|
||||
except NotImplementedError:
|
||||
resolved_address = fallback_addr
|
||||
|
||||
if resolved_address is None:
|
||||
raise ValueError("Unable to resolve a suitable IP address based on "
|
||||
"charm state and configuration. (net_type=%s, "
|
||||
"clustered=%s)" % (net_type, clustered))
|
||||
|
||||
return resolved_address
|
|
@ -1,384 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
# Various utilies for dealing with Neutron and the renaming from Quantum.
|
||||
|
||||
import six
|
||||
from subprocess import check_output
|
||||
|
||||
from charmhelpers.core.hookenv import (
|
||||
config,
|
||||
log,
|
||||
ERROR,
|
||||
)
|
||||
|
||||
from charmhelpers.contrib.openstack.utils import os_release
|
||||
|
||||
|
||||
def headers_package():
|
||||
"""Ensures correct linux-headers for running kernel are installed,
|
||||
for building DKMS package"""
|
||||
kver = check_output(['uname', '-r']).decode('UTF-8').strip()
|
||||
return 'linux-headers-%s' % kver
|
||||
|
||||
QUANTUM_CONF_DIR = '/etc/quantum'
|
||||
|
||||
|
||||
def kernel_version():
|
||||
""" Retrieve the current major kernel version as a tuple e.g. (3, 13) """
|
||||
kver = check_output(['uname', '-r']).decode('UTF-8').strip()
|
||||
kver = kver.split('.')
|
||||
return (int(kver[0]), int(kver[1]))
|
||||
|
||||
|
||||
def determine_dkms_package():
|
||||
""" Determine which DKMS package should be used based on kernel version """
|
||||
# NOTE: 3.13 kernels have support for GRE and VXLAN native
|
||||
if kernel_version() >= (3, 13):
|
||||
return []
|
||||
else:
|
||||
return [headers_package(), 'openvswitch-datapath-dkms']
|
||||
|
||||
|
||||
# legacy
|
||||
|
||||
|
||||
def quantum_plugins():
|
||||
from charmhelpers.contrib.openstack import context
|
||||
return {
|
||||
'ovs': {
|
||||
'config': '/etc/quantum/plugins/openvswitch/'
|
||||
'ovs_quantum_plugin.ini',
|
||||
'driver': 'quantum.plugins.openvswitch.ovs_quantum_plugin.'
|
||||
'OVSQuantumPluginV2',
|
||||
'contexts': [
|
||||
context.SharedDBContext(user=config('neutron-database-user'),
|
||||
database=config('neutron-database'),
|
||||
relation_prefix='neutron',
|
||||
ssl_dir=QUANTUM_CONF_DIR)],
|
||||
'services': ['quantum-plugin-openvswitch-agent'],
|
||||
'packages': [determine_dkms_package(),
|
||||
['quantum-plugin-openvswitch-agent']],
|
||||
'server_packages': ['quantum-server',
|
||||
'quantum-plugin-openvswitch'],
|
||||
'server_services': ['quantum-server']
|
||||
},
|
||||
'nvp': {
|
||||
'config': '/etc/quantum/plugins/nicira/nvp.ini',
|
||||
'driver': 'quantum.plugins.nicira.nicira_nvp_plugin.'
|
||||
'QuantumPlugin.NvpPluginV2',
|
||||
'contexts': [
|
||||
context.SharedDBContext(user=config('neutron-database-user'),
|
||||
database=config('neutron-database'),
|
||||
relation_prefix='neutron',
|
||||
ssl_dir=QUANTUM_CONF_DIR)],
|
||||
'services': [],
|
||||
'packages': [],
|
||||
'server_packages': ['quantum-server',
|
||||
'quantum-plugin-nicira'],
|
||||
'server_services': ['quantum-server']
|
||||
}
|
||||
}
|
||||
|
||||
NEUTRON_CONF_DIR = '/etc/neutron'
|
||||
|
||||
|
||||
def neutron_plugins():
|
||||
from charmhelpers.contrib.openstack import context
|
||||
release = os_release('nova-common')
|
||||
plugins = {
|
||||
'ovs': {
|
||||
'config': '/etc/neutron/plugins/openvswitch/'
|
||||
'ovs_neutron_plugin.ini',
|
||||
'driver': 'neutron.plugins.openvswitch.ovs_neutron_plugin.'
|
||||
'OVSNeutronPluginV2',
|
||||
'contexts': [
|
||||
context.SharedDBContext(user=config('neutron-database-user'),
|
||||
database=config('neutron-database'),
|
||||
relation_prefix='neutron',
|
||||
ssl_dir=NEUTRON_CONF_DIR)],
|
||||
'services': ['neutron-plugin-openvswitch-agent'],
|
||||
'packages': [determine_dkms_package(),
|
||||
['neutron-plugin-openvswitch-agent']],
|
||||
'server_packages': ['neutron-server',
|
||||
'neutron-plugin-openvswitch'],
|
||||
'server_services': ['neutron-server']
|
||||
},
|
||||
'nvp': {
|
||||
'config': '/etc/neutron/plugins/nicira/nvp.ini',
|
||||
'driver': 'neutron.plugins.nicira.nicira_nvp_plugin.'
|
||||
'NeutronPlugin.NvpPluginV2',
|
||||
'contexts': [
|
||||
context.SharedDBContext(user=config('neutron-database-user'),
|
||||
database=config('neutron-database'),
|
||||
relation_prefix='neutron',
|
||||
ssl_dir=NEUTRON_CONF_DIR)],
|
||||
'services': [],
|
||||
'packages': [],
|
||||
'server_packages': ['neutron-server',
|
||||
'neutron-plugin-nicira'],
|
||||
'server_services': ['neutron-server']
|
||||
},
|
||||
'nsx': {
|
||||
'config': '/etc/neutron/plugins/vmware/nsx.ini',
|
||||
'driver': 'vmware',
|
||||
'contexts': [
|
||||
context.SharedDBContext(user=config('neutron-database-user'),
|
||||
database=config('neutron-database'),
|
||||
relation_prefix='neutron',
|
||||
ssl_dir=NEUTRON_CONF_DIR)],
|
||||
'services': [],
|
||||
'packages': [],
|
||||
'server_packages': ['neutron-server',
|
||||
'neutron-plugin-vmware'],
|
||||
'server_services': ['neutron-server']
|
||||
},
|
||||
'n1kv': {
|
||||
'config': '/etc/neutron/plugins/cisco/cisco_plugins.ini',
|
||||
'driver': 'neutron.plugins.cisco.network_plugin.PluginV2',
|
||||
'contexts': [
|
||||
context.SharedDBContext(user=config('neutron-database-user'),
|
||||
database=config('neutron-database'),
|
||||
relation_prefix='neutron',
|
||||
ssl_dir=NEUTRON_CONF_DIR)],
|
||||
'services': [],
|
||||
'packages': [determine_dkms_package(),
|
||||
['neutron-plugin-cisco']],
|
||||
'server_packages': ['neutron-server',
|
||||
'neutron-plugin-cisco'],
|
||||
'server_services': ['neutron-server']
|
||||
},
|
||||
'Calico': {
|
||||
'config': '/etc/neutron/plugins/ml2/ml2_conf.ini',
|
||||
'driver': 'neutron.plugins.ml2.plugin.Ml2Plugin',
|
||||
'contexts': [
|
||||
context.SharedDBContext(user=config('neutron-database-user'),
|
||||
database=config('neutron-database'),
|
||||
relation_prefix='neutron',
|
||||
ssl_dir=NEUTRON_CONF_DIR)],
|
||||
'services': ['calico-felix',
|
||||
'bird',
|
||||
'neutron-dhcp-agent',
|
||||
'nova-api-metadata',
|
||||
'etcd'],
|
||||
'packages': [determine_dkms_package(),
|
||||
['calico-compute',
|
||||
'bird',
|
||||
'neutron-dhcp-agent',
|
||||
'nova-api-metadata',
|
||||
'etcd']],
|
||||
'server_packages': ['neutron-server', 'calico-control', 'etcd'],
|
||||
'server_services': ['neutron-server', 'etcd']
|
||||
},
|
||||
'vsp': {
|
||||
'config': '/etc/neutron/plugins/nuage/nuage_plugin.ini',
|
||||
'driver': 'neutron.plugins.nuage.plugin.NuagePlugin',
|
||||
'contexts': [
|
||||
context.SharedDBContext(user=config('neutron-database-user'),
|
||||
database=config('neutron-database'),
|
||||
relation_prefix='neutron',
|
||||
ssl_dir=NEUTRON_CONF_DIR)],
|
||||
'services': [],
|
||||
'packages': [],
|
||||
'server_packages': ['neutron-server', 'neutron-plugin-nuage'],
|
||||
'server_services': ['neutron-server']
|
||||
},
|
||||
'plumgrid': {
|
||||
'config': '/etc/neutron/plugins/plumgrid/plumgrid.ini',
|
||||
'driver': 'neutron.plugins.plumgrid.plumgrid_plugin.plumgrid_plugin.NeutronPluginPLUMgridV2',
|
||||
'contexts': [
|
||||
context.SharedDBContext(user=config('database-user'),
|
||||
database=config('database'),
|
||||
ssl_dir=NEUTRON_CONF_DIR)],
|
||||
'services': [],
|
||||
'packages': ['plumgrid-lxc',
|
||||
'iovisor-dkms'],
|
||||
'server_packages': ['neutron-server',
|
||||
'neutron-plugin-plumgrid'],
|
||||
'server_services': ['neutron-server']
|
||||
},
|
||||
'midonet': {
|
||||
'config': '/etc/neutron/plugins/midonet/midonet.ini',
|
||||
'driver': 'midonet.neutron.plugin.MidonetPluginV2',
|
||||
'contexts': [
|
||||
context.SharedDBContext(user=config('neutron-database-user'),
|
||||
database=config('neutron-database'),
|
||||
relation_prefix='neutron',
|
||||
ssl_dir=NEUTRON_CONF_DIR)],
|
||||
'services': [],
|
||||
'packages': [determine_dkms_package()],
|
||||
'server_packages': ['neutron-server',
|
||||
'python-neutron-plugin-midonet'],
|
||||
'server_services': ['neutron-server']
|
||||
}
|
||||
}
|
||||
if release >= 'icehouse':
|
||||
# NOTE: patch in ml2 plugin for icehouse onwards
|
||||
plugins['ovs']['config'] = '/etc/neutron/plugins/ml2/ml2_conf.ini'
|
||||
plugins['ovs']['driver'] = 'neutron.plugins.ml2.plugin.Ml2Plugin'
|
||||
plugins['ovs']['server_packages'] = ['neutron-server',
|
||||
'neutron-plugin-ml2']
|
||||
# NOTE: patch in vmware renames nvp->nsx for icehouse onwards
|
||||
plugins['nvp'] = plugins['nsx']
|
||||
if release >= 'kilo':
|
||||
plugins['midonet']['driver'] = (
|
||||
'neutron.plugins.midonet.plugin.MidonetPluginV2')
|
||||
if release >= 'liberty':
|
||||
plugins['midonet']['driver'] = (
|
||||
'midonet.neutron.plugin_v1.MidonetPluginV2')
|
||||
plugins['midonet']['server_packages'].remove(
|
||||
'python-neutron-plugin-midonet')
|
||||
plugins['midonet']['server_packages'].append(
|
||||
'python-networking-midonet')
|
||||
plugins['plumgrid']['driver'] = (
|
||||
'networking_plumgrid.neutron.plugins.plugin.NeutronPluginPLUMgridV2')
|
||||
plugins['plumgrid']['server_packages'].remove(
|
||||
'neutron-plugin-plumgrid')
|
||||
return plugins
|
||||
|
||||
|
||||
def neutron_plugin_attribute(plugin, attr, net_manager=None):
|
||||
manager = net_manager or network_manager()
|
||||
if manager == 'quantum':
|
||||
plugins = quantum_plugins()
|
||||
elif manager == 'neutron':
|
||||
plugins = neutron_plugins()
|
||||
else:
|
||||
log("Network manager '%s' does not support plugins." % (manager),
|
||||
level=ERROR)
|
||||
raise Exception
|
||||
|
||||
try:
|
||||
_plugin = plugins[plugin]
|
||||
except KeyError:
|
||||
log('Unrecognised plugin for %s: %s' % (manager, plugin), level=ERROR)
|
||||
raise Exception
|
||||
|
||||
try:
|
||||
return _plugin[attr]
|
||||
except KeyError:
|
||||
return None
|
||||
|
||||
|
||||
def network_manager():
|
||||
'''
|
||||
Deals with the renaming of Quantum to Neutron in H and any situations
|
||||
that require compatability (eg, deploying H with network-manager=quantum,
|
||||
upgrading from G).
|
||||
'''
|
||||
release = os_release('nova-common')
|
||||
manager = config('network-manager').lower()
|
||||
|
||||
if manager not in ['quantum', 'neutron']:
|
||||
return manager
|
||||
|
||||
if release in ['essex']:
|
||||
# E does not support neutron
|
||||
log('Neutron networking not supported in Essex.', level=ERROR)
|
||||
raise Exception
|
||||
elif release in ['folsom', 'grizzly']:
|
||||
# neutron is named quantum in F and G
|
||||
return 'quantum'
|
||||
else:
|
||||
# ensure accurate naming for all releases post-H
|
||||
return 'neutron'
|
||||
|
||||
|
||||
def parse_mappings(mappings, key_rvalue=False):
|
||||
"""By default mappings are lvalue keyed.
|
||||
|
||||
If key_rvalue is True, the mapping will be reversed to allow multiple
|
||||
configs for the same lvalue.
|
||||
"""
|
||||
parsed = {}
|
||||
if mappings:
|
||||
mappings = mappings.split()
|
||||
for m in mappings:
|
||||
p = m.partition(':')
|
||||
|
||||
if key_rvalue:
|
||||
key_index = 2
|
||||
val_index = 0
|
||||
# if there is no rvalue skip to next
|
||||
if not p[1]:
|
||||
continue
|
||||
else:
|
||||
key_index = 0
|
||||
val_index = 2
|
||||
|
||||
key = p[key_index].strip()
|
||||
parsed[key] = p[val_index].strip()
|
||||
|
||||
return parsed
|
||||
|
||||
|
||||
def parse_bridge_mappings(mappings):
|
||||
"""Parse bridge mappings.
|
||||
|
||||
Mappings must be a space-delimited list of provider:bridge mappings.
|
||||
|
||||
Returns dict of the form {provider:bridge}.
|
||||
"""
|
||||
return parse_mappings(mappings)
|
||||
|
||||
|
||||
def parse_data_port_mappings(mappings, default_bridge='br-data'):
|
||||
"""Parse data port mappings.
|
||||
|
||||
Mappings must be a space-delimited list of bridge:port.
|
||||
|
||||
Returns dict of the form {port:bridge} where ports may be mac addresses or
|
||||
interface names.
|
||||
"""
|
||||
|
||||
# NOTE(dosaboy): we use rvalue for key to allow multiple values to be
|
||||
# proposed for <port> since it may be a mac address which will differ
|
||||
# across units this allowing first-known-good to be chosen.
|
||||
_mappings = parse_mappings(mappings, key_rvalue=True)
|
||||
if not _mappings or list(_mappings.values()) == ['']:
|
||||
if not mappings:
|
||||
return {}
|
||||
|
||||
# For backwards-compatibility we need to support port-only provided in
|
||||
# config.
|
||||
_mappings = {mappings.split()[0]: default_bridge}
|
||||
|
||||
ports = _mappings.keys()
|
||||
if len(set(ports)) != len(ports):
|
||||
raise Exception("It is not allowed to have the same port configured "
|
||||
"on more than one bridge")
|
||||
|
||||
return _mappings
|
||||
|
||||
|
||||
def parse_vlan_range_mappings(mappings):
|
||||
"""Parse vlan range mappings.
|
||||
|
||||
Mappings must be a space-delimited list of provider:start:end mappings.
|
||||
|
||||
The start:end range is optional and may be omitted.
|
||||
|
||||
Returns dict of the form {provider: (start, end)}.
|
||||
"""
|
||||
_mappings = parse_mappings(mappings)
|
||||
if not _mappings:
|
||||
return {}
|
||||
|
||||
mappings = {}
|
||||
for p, r in six.iteritems(_mappings):
|
||||
mappings[p] = tuple(r.split(':'))
|
||||
|
||||
return mappings
|
|
@ -1,18 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
# dummy __init__.py to fool syncer into thinking this is a syncable python
|
||||
# module
|
|
@ -1,323 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
import os
|
||||
|
||||
import six
|
||||
|
||||
from charmhelpers.fetch import apt_install, apt_update
|
||||
from charmhelpers.core.hookenv import (
|
||||
log,
|
||||
ERROR,
|
||||
INFO
|
||||
)
|
||||
from charmhelpers.contrib.openstack.utils import OPENSTACK_CODENAMES
|
||||
|
||||
try:
|
||||
from jinja2 import FileSystemLoader, ChoiceLoader, Environment, exceptions
|
||||
except ImportError:
|
||||
apt_update(fatal=True)
|
||||
apt_install('python-jinja2', fatal=True)
|
||||
from jinja2 import FileSystemLoader, ChoiceLoader, Environment, exceptions
|
||||
|
||||
|
||||
class OSConfigException(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def get_loader(templates_dir, os_release):
|
||||
"""
|
||||
Create a jinja2.ChoiceLoader containing template dirs up to
|
||||
and including os_release. If directory template directory
|
||||
is missing at templates_dir, it will be omitted from the loader.
|
||||
templates_dir is added to the bottom of the search list as a base
|
||||
loading dir.
|
||||
|
||||
A charm may also ship a templates dir with this module
|
||||
and it will be appended to the bottom of the search list, eg::
|
||||
|
||||
hooks/charmhelpers/contrib/openstack/templates
|
||||
|
||||
:param templates_dir (str): Base template directory containing release
|
||||
sub-directories.
|
||||
:param os_release (str): OpenStack release codename to construct template
|
||||
loader.
|
||||
:returns: jinja2.ChoiceLoader constructed with a list of
|
||||
jinja2.FilesystemLoaders, ordered in descending
|
||||
order by OpenStack release.
|
||||
"""
|
||||
tmpl_dirs = [(rel, os.path.join(templates_dir, rel))
|
||||
for rel in six.itervalues(OPENSTACK_CODENAMES)]
|
||||
|
||||
if not os.path.isdir(templates_dir):
|
||||
log('Templates directory not found @ %s.' % templates_dir,
|
||||
level=ERROR)
|
||||
raise OSConfigException
|
||||
|
||||
# the bottom contains tempaltes_dir and possibly a common templates dir
|
||||
# shipped with the helper.
|
||||
loaders = [FileSystemLoader(templates_dir)]
|
||||
helper_templates = os.path.join(os.path.dirname(__file__), 'templates')
|
||||
if os.path.isdir(helper_templates):
|
||||
loaders.append(FileSystemLoader(helper_templates))
|
||||
|
||||
for rel, tmpl_dir in tmpl_dirs:
|
||||
if os.path.isdir(tmpl_dir):
|
||||
loaders.insert(0, FileSystemLoader(tmpl_dir))
|
||||
if rel == os_release:
|
||||
break
|
||||
log('Creating choice loader with dirs: %s' %
|
||||
[l.searchpath for l in loaders], level=INFO)
|
||||
return ChoiceLoader(loaders)
|
||||
|
||||
|
||||
class OSConfigTemplate(object):
|
||||
"""
|
||||
Associates a config file template with a list of context generators.
|
||||
Responsible for constructing a template context based on those generators.
|
||||
"""
|
||||
def __init__(self, config_file, contexts):
|
||||
self.config_file = config_file
|
||||
|
||||
if hasattr(contexts, '__call__'):
|
||||
self.contexts = [contexts]
|
||||
else:
|
||||
self.contexts = contexts
|
||||
|
||||
self._complete_contexts = []
|
||||
|
||||
def context(self):
|
||||
ctxt = {}
|
||||
for context in self.contexts:
|
||||
_ctxt = context()
|
||||
if _ctxt:
|
||||
ctxt.update(_ctxt)
|
||||
# track interfaces for every complete context.
|
||||
[self._complete_contexts.append(interface)
|
||||
for interface in context.interfaces
|
||||
if interface not in self._complete_contexts]
|
||||
return ctxt
|
||||
|
||||
def complete_contexts(self):
|
||||
'''
|
||||
Return a list of interfaces that have satisfied contexts.
|
||||
'''
|
||||
if self._complete_contexts:
|
||||
return self._complete_contexts
|
||||
self.context()
|
||||
return self._complete_contexts
|
||||
|
||||
|
||||
class OSConfigRenderer(object):
|
||||
"""
|
||||
This class provides a common templating system to be used by OpenStack
|
||||
charms. It is intended to help charms share common code and templates,
|
||||
and ease the burden of managing config templates across multiple OpenStack
|
||||
releases.
|
||||
|
||||
Basic usage::
|
||||
|
||||
# import some common context generates from charmhelpers
|
||||
from charmhelpers.contrib.openstack import context
|
||||
|
||||
# Create a renderer object for a specific OS release.
|
||||
configs = OSConfigRenderer(templates_dir='/tmp/templates',
|
||||
openstack_release='folsom')
|
||||
# register some config files with context generators.
|
||||
configs.register(config_file='/etc/nova/nova.conf',
|
||||
contexts=[context.SharedDBContext(),
|
||||
context.AMQPContext()])
|
||||
configs.register(config_file='/etc/nova/api-paste.ini',
|
||||
contexts=[context.IdentityServiceContext()])
|
||||
configs.register(config_file='/etc/haproxy/haproxy.conf',
|
||||
contexts=[context.HAProxyContext()])
|
||||
# write out a single config
|
||||
configs.write('/etc/nova/nova.conf')
|
||||
# write out all registered configs
|
||||
configs.write_all()
|
||||
|
||||
**OpenStack Releases and template loading**
|
||||
|
||||
When the object is instantiated, it is associated with a specific OS
|
||||
release. This dictates how the template loader will be constructed.
|
||||
|
||||
The constructed loader attempts to load the template from several places
|
||||
in the following order:
|
||||
- from the most recent OS release-specific template dir (if one exists)
|
||||
- the base templates_dir
|
||||
- a template directory shipped in the charm with this helper file.
|
||||
|
||||
For the example above, '/tmp/templates' contains the following structure::
|
||||
|
||||
/tmp/templates/nova.conf
|
||||
/tmp/templates/api-paste.ini
|
||||
/tmp/templates/grizzly/api-paste.ini
|
||||
/tmp/templates/havana/api-paste.ini
|
||||
|
||||
Since it was registered with the grizzly release, it first seraches
|
||||
the grizzly directory for nova.conf, then the templates dir.
|
||||
|
||||
When writing api-paste.ini, it will find the template in the grizzly
|
||||
directory.
|
||||
|
||||
If the object were created with folsom, it would fall back to the
|
||||
base templates dir for its api-paste.ini template.
|
||||
|
||||
This system should help manage changes in config files through
|
||||
openstack releases, allowing charms to fall back to the most recently
|
||||
updated config template for a given release
|
||||
|
||||
The haproxy.conf, since it is not shipped in the templates dir, will
|
||||
be loaded from the module directory's template directory, eg
|
||||
$CHARM/hooks/charmhelpers/contrib/openstack/templates. This allows
|
||||
us to ship common templates (haproxy, apache) with the helpers.
|
||||
|
||||
**Context generators**
|
||||
|
||||
Context generators are used to generate template contexts during hook
|
||||
execution. Doing so may require inspecting service relations, charm
|
||||
config, etc. When registered, a config file is associated with a list
|
||||
of generators. When a template is rendered and written, all context
|
||||
generates are called in a chain to generate the context dictionary
|
||||
passed to the jinja2 template. See context.py for more info.
|
||||
"""
|
||||
def __init__(self, templates_dir, openstack_release):
|
||||
if not os.path.isdir(templates_dir):
|
||||
log('Could not locate templates dir %s' % templates_dir,
|
||||
level=ERROR)
|
||||
raise OSConfigException
|
||||
|
||||
self.templates_dir = templates_dir
|
||||
self.openstack_release = openstack_release
|
||||
self.templates = {}
|
||||
self._tmpl_env = None
|
||||
|
||||
if None in [Environment, ChoiceLoader, FileSystemLoader]:
|
||||
# if this code is running, the object is created pre-install hook.
|
||||
# jinja2 shouldn't get touched until the module is reloaded on next
|
||||
# hook execution, with proper jinja2 bits successfully imported.
|
||||
apt_install('python-jinja2')
|
||||
|
||||
def register(self, config_file, contexts):
|
||||
"""
|
||||
Register a config file with a list of context generators to be called
|
||||
during rendering.
|
||||
"""
|
||||
self.templates[config_file] = OSConfigTemplate(config_file=config_file,
|
||||
contexts=contexts)
|
||||
log('Registered config file: %s' % config_file, level=INFO)
|
||||
|
||||
def _get_tmpl_env(self):
|
||||
if not self._tmpl_env:
|
||||
loader = get_loader(self.templates_dir, self.openstack_release)
|
||||
self._tmpl_env = Environment(loader=loader)
|
||||
|
||||
def _get_template(self, template):
|
||||
self._get_tmpl_env()
|
||||
template = self._tmpl_env.get_template(template)
|
||||
log('Loaded template from %s' % template.filename, level=INFO)
|
||||
return template
|
||||
|
||||
def render(self, config_file):
|
||||
if config_file not in self.templates:
|
||||
log('Config not registered: %s' % config_file, level=ERROR)
|
||||
raise OSConfigException
|
||||
ctxt = self.templates[config_file].context()
|
||||
|
||||
_tmpl = os.path.basename(config_file)
|
||||
try:
|
||||
template = self._get_template(_tmpl)
|
||||
except exceptions.TemplateNotFound:
|
||||
# if no template is found with basename, try looking for it
|
||||
# using a munged full path, eg:
|
||||
# /etc/apache2/apache2.conf -> etc_apache2_apache2.conf
|
||||
_tmpl = '_'.join(config_file.split('/')[1:])
|
||||
try:
|
||||
template = self._get_template(_tmpl)
|
||||
except exceptions.TemplateNotFound as e:
|
||||
log('Could not load template from %s by %s or %s.' %
|
||||
(self.templates_dir, os.path.basename(config_file), _tmpl),
|
||||
level=ERROR)
|
||||
raise e
|
||||
|
||||
log('Rendering from template: %s' % _tmpl, level=INFO)
|
||||
return template.render(ctxt)
|
||||
|
||||
def write(self, config_file):
|
||||
"""
|
||||
Write a single config file, raises if config file is not registered.
|
||||
"""
|
||||
if config_file not in self.templates:
|
||||
log('Config not registered: %s' % config_file, level=ERROR)
|
||||
raise OSConfigException
|
||||
|
||||
_out = self.render(config_file)
|
||||
|
||||
with open(config_file, 'wb') as out:
|
||||
out.write(_out)
|
||||
|
||||
log('Wrote template %s.' % config_file, level=INFO)
|
||||
|
||||
def write_all(self):
|
||||
"""
|
||||
Write out all registered config files.
|
||||
"""
|
||||
[self.write(k) for k in six.iterkeys(self.templates)]
|
||||
|
||||
def set_release(self, openstack_release):
|
||||
"""
|
||||
Resets the template environment and generates a new template loader
|
||||
based on a the new openstack release.
|
||||
"""
|
||||
self._tmpl_env = None
|
||||
self.openstack_release = openstack_release
|
||||
self._get_tmpl_env()
|
||||
|
||||
def complete_contexts(self):
|
||||
'''
|
||||
Returns a list of context interfaces that yield a complete context.
|
||||
'''
|
||||
interfaces = []
|
||||
[interfaces.extend(i.complete_contexts())
|
||||
for i in six.itervalues(self.templates)]
|
||||
return interfaces
|
||||
|
||||
def get_incomplete_context_data(self, interfaces):
|
||||
'''
|
||||
Return dictionary of relation status of interfaces and any missing
|
||||
required context data. Example:
|
||||
{'amqp': {'missing_data': ['rabbitmq_password'], 'related': True},
|
||||
'zeromq-configuration': {'related': False}}
|
||||
'''
|
||||
incomplete_context_data = {}
|
||||
|
||||
for i in six.itervalues(self.templates):
|
||||
for context in i.contexts:
|
||||
for interface in interfaces:
|
||||
related = False
|
||||
if interface in context.interfaces:
|
||||
related = context.get_related()
|
||||
missing_data = context.missing_data
|
||||
if missing_data:
|
||||
incomplete_context_data[interface] = {'missing_data': missing_data}
|
||||
if related:
|
||||
if incomplete_context_data.get(interface):
|
||||
incomplete_context_data[interface].update({'related': True})
|
||||
else:
|
||||
incomplete_context_data[interface] = {'related': True}
|
||||
else:
|
||||
incomplete_context_data[interface] = {'related': False}
|
||||
return incomplete_context_data
|
File diff suppressed because it is too large
Load Diff
|
@ -1,15 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
|
@ -1,56 +0,0 @@
|
|||
#!/usr/bin/env python
|
||||
# coding: utf-8
|
||||
|
||||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
from __future__ import print_function
|
||||
|
||||
import atexit
|
||||
import sys
|
||||
|
||||
from charmhelpers.contrib.python.rpdb import Rpdb
|
||||
from charmhelpers.core.hookenv import (
|
||||
open_port,
|
||||
close_port,
|
||||
ERROR,
|
||||
log
|
||||
)
|
||||
|
||||
__author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>"
|
||||
|
||||
DEFAULT_ADDR = "0.0.0.0"
|
||||
DEFAULT_PORT = 4444
|
||||
|
||||
|
||||
def _error(message):
|
||||
log(message, level=ERROR)
|
||||
|
||||
|
||||
def set_trace(addr=DEFAULT_ADDR, port=DEFAULT_PORT):
|
||||
"""
|
||||
Set a trace point using the remote debugger
|
||||
"""
|
||||
atexit.register(close_port, port)
|
||||
try:
|
||||
log("Starting a remote python debugger session on %s:%s" % (addr,
|
||||
port))
|
||||
open_port(port)
|
||||
debugger = Rpdb(addr=addr, port=port)
|
||||
debugger.set_trace(sys._getframe().f_back)
|
||||
except:
|
||||
_error("Cannot start a remote debug session on %s:%s" % (addr,
|
||||
port))
|
|
@ -1,145 +0,0 @@
|
|||
#!/usr/bin/env python
|
||||
# coding: utf-8
|
||||
|
||||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
from charmhelpers.fetch import apt_install, apt_update
|
||||
from charmhelpers.core.hookenv import charm_dir, log
|
||||
|
||||
__author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>"
|
||||
|
||||
|
||||
def pip_execute(*args, **kwargs):
|
||||
"""Overriden pip_execute() to stop sys.path being changed.
|
||||
|
||||
The act of importing main from the pip module seems to cause add wheels
|
||||
from the /usr/share/python-wheels which are installed by various tools.
|
||||
This function ensures that sys.path remains the same after the call is
|
||||
executed.
|
||||
"""
|
||||
try:
|
||||
_path = sys.path
|
||||
try:
|
||||
from pip import main as _pip_execute
|
||||
except ImportError:
|
||||
apt_update()
|
||||
apt_install('python-pip')
|
||||
from pip import main as _pip_execute
|
||||
_pip_execute(*args, **kwargs)
|
||||
finally:
|
||||
sys.path = _path
|
||||
|
||||
|
||||
def parse_options(given, available):
|
||||
"""Given a set of options, check if available"""
|
||||
for key, value in sorted(given.items()):
|
||||
if not value:
|
||||
continue
|
||||
if key in available:
|
||||
yield "--{0}={1}".format(key, value)
|
||||
|
||||
|
||||
def pip_install_requirements(requirements, constraints=None, **options):
|
||||
"""Install a requirements file.
|
||||
|
||||
:param constraints: Path to pip constraints file.
|
||||
http://pip.readthedocs.org/en/stable/user_guide/#constraints-files
|
||||
"""
|
||||
command = ["install"]
|
||||
|
||||
available_options = ('proxy', 'src', 'log', )
|
||||
for option in parse_options(options, available_options):
|
||||
command.append(option)
|
||||
|
||||
command.append("-r {0}".format(requirements))
|
||||
if constraints:
|
||||
command.append("-c {0}".format(constraints))
|
||||
log("Installing from file: {} with constraints {} "
|
||||
"and options: {}".format(requirements, constraints, command))
|
||||
else:
|
||||
log("Installing from file: {} with options: {}".format(requirements,
|
||||
command))
|
||||
pip_execute(command)
|
||||
|
||||
|
||||
def pip_install(package, fatal=False, upgrade=False, venv=None, **options):
|
||||
"""Install a python package"""
|
||||
if venv:
|
||||
venv_python = os.path.join(venv, 'bin/pip')
|
||||
command = [venv_python, "install"]
|
||||
else:
|
||||
command = ["install"]
|
||||
|
||||
available_options = ('proxy', 'src', 'log', 'index-url', )
|
||||
for option in parse_options(options, available_options):
|
||||
command.append(option)
|
||||
|
||||
if upgrade:
|
||||
command.append('--upgrade')
|
||||
|
||||
if isinstance(package, list):
|
||||
command.extend(package)
|
||||
else:
|
||||
command.append(package)
|
||||
|
||||
log("Installing {} package with options: {}".format(package,
|
||||
command))
|
||||
if venv:
|
||||
subprocess.check_call(command)
|
||||
else:
|
||||
pip_execute(command)
|
||||
|
||||
|
||||
def pip_uninstall(package, **options):
|
||||
"""Uninstall a python package"""
|
||||
command = ["uninstall", "-q", "-y"]
|
||||
|
||||
available_options = ('proxy', 'log', )
|
||||
for option in parse_options(options, available_options):
|
||||
command.append(option)
|
||||
|
||||
if isinstance(package, list):
|
||||
command.extend(package)
|
||||
else:
|
||||
command.append(package)
|
||||
|
||||
log("Uninstalling {} package with options: {}".format(package,
|
||||
command))
|
||||
pip_execute(command)
|
||||
|
||||
|
||||
def pip_list():
|
||||
"""Returns the list of current python installed packages
|
||||
"""
|
||||
return pip_execute(["list"])
|
||||
|
||||
|
||||
def pip_create_virtualenv(path=None):
|
||||
"""Create an isolated Python environment."""
|
||||
apt_install('python-virtualenv')
|
||||
|
||||
if path:
|
||||
venv_path = path
|
||||
else:
|
||||
venv_path = os.path.join(charm_dir(), 'venv')
|
||||
|
||||
if not os.path.exists(venv_path):
|
||||
subprocess.check_call(['virtualenv', venv_path])
|
|
@ -1,58 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
"""Remote Python Debugger (pdb wrapper)."""
|
||||
|
||||
import pdb
|
||||
import socket
|
||||
import sys
|
||||
|
||||
__author__ = "Bertrand Janin <b@janin.com>"
|
||||
__version__ = "0.1.3"
|
||||
|
||||
|
||||
class Rpdb(pdb.Pdb):
|
||||
|
||||
def __init__(self, addr="127.0.0.1", port=4444):
|
||||
"""Initialize the socket and initialize pdb."""
|
||||
|
||||
# Backup stdin and stdout before replacing them by the socket handle
|
||||
self.old_stdout = sys.stdout
|
||||
self.old_stdin = sys.stdin
|
||||
|
||||
# Open a 'reusable' socket to let the webapp reload on the same port
|
||||
self.skt = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
self.skt.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, True)
|
||||
self.skt.bind((addr, port))
|
||||
self.skt.listen(1)
|
||||
(clientsocket, address) = self.skt.accept()
|
||||
handle = clientsocket.makefile('rw')
|
||||
pdb.Pdb.__init__(self, completekey='tab', stdin=handle, stdout=handle)
|
||||
sys.stdout = sys.stdin = handle
|
||||
|
||||
def shutdown(self):
|
||||
"""Revert stdin and stdout, close the socket."""
|
||||
sys.stdout = self.old_stdout
|
||||
sys.stdin = self.old_stdin
|
||||
self.skt.close()
|
||||
self.set_continue()
|
||||
|
||||
def do_continue(self, arg):
|
||||
"""Stop all operation on ``continue``."""
|
||||
self.shutdown()
|
||||
return 1
|
||||
|
||||
do_EOF = do_quit = do_exit = do_c = do_cont = do_continue
|
|
@ -1,34 +0,0 @@
|
|||
#!/usr/bin/env python
|
||||
# coding: utf-8
|
||||
|
||||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
import sys
|
||||
|
||||
__author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>"
|
||||
|
||||
|
||||
def current_version():
|
||||
"""Current system python version"""
|
||||
return sys.version_info
|
||||
|
||||
|
||||
def current_version_string():
|
||||
"""Current system python version as string major.minor.micro"""
|
||||
return "{0}.{1}.{2}".format(sys.version_info.major,
|
||||
sys.version_info.minor,
|
||||
sys.version_info.micro)
|
|
@ -1,15 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
|
@ -1,15 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
File diff suppressed because it is too large
Load Diff
|
@ -1,88 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
import os
|
||||
import re
|
||||
from subprocess import (
|
||||
check_call,
|
||||
check_output,
|
||||
)
|
||||
|
||||
import six
|
||||
|
||||
|
||||
##################################################
|
||||
# loopback device helpers.
|
||||
##################################################
|
||||
def loopback_devices():
|
||||
'''
|
||||
Parse through 'losetup -a' output to determine currently mapped
|
||||
loopback devices. Output is expected to look like:
|
||||
|
||||
/dev/loop0: [0807]:961814 (/tmp/my.img)
|
||||
|
||||
:returns: dict: a dict mapping {loopback_dev: backing_file}
|
||||
'''
|
||||
loopbacks = {}
|
||||
cmd = ['losetup', '-a']
|
||||
devs = [d.strip().split(' ') for d in
|
||||
check_output(cmd).splitlines() if d != '']
|
||||
for dev, _, f in devs:
|
||||
loopbacks[dev.replace(':', '')] = re.search('\((\S+)\)', f).groups()[0]
|
||||
return loopbacks
|
||||
|
||||
|
||||
def create_loopback(file_path):
|
||||
'''
|
||||
Create a loopback device for a given backing file.
|
||||
|
||||
:returns: str: Full path to new loopback device (eg, /dev/loop0)
|
||||
'''
|
||||
file_path = os.path.abspath(file_path)
|
||||
check_call(['losetup', '--find', file_path])
|
||||
for d, f in six.iteritems(loopback_devices()):
|
||||
if f == file_path:
|
||||
return d
|
||||
|
||||
|
||||
def ensure_loopback_device(path, size):
|
||||
'''
|
||||
Ensure a loopback device exists for a given backing file path and size.
|
||||
If it a loopback device is not mapped to file, a new one will be created.
|
||||
|
||||
TODO: Confirm size of found loopback device.
|
||||
|
||||
:returns: str: Full path to the ensured loopback device (eg, /dev/loop0)
|
||||
'''
|
||||
for d, f in six.iteritems(loopback_devices()):
|
||||
if f == path:
|
||||
return d
|
||||
|
||||
if not os.path.exists(path):
|
||||
cmd = ['truncate', '--size', size, path]
|
||||
check_call(cmd)
|
||||
|
||||
return create_loopback(path)
|
||||
|
||||
|
||||
def is_mapped_loopback_device(device):
|
||||
"""
|
||||
Checks if a given device name is an existing/mapped loopback device.
|
||||
:param device: str: Full path to the device (eg, /dev/loop1).
|
||||
:returns: str: Path to the backing file if is a loopback device
|
||||
empty string otherwise
|
||||
"""
|
||||
return loopback_devices().get(device, "")
|
|
@ -1,105 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
from subprocess import (
|
||||
CalledProcessError,
|
||||
check_call,
|
||||
check_output,
|
||||
Popen,
|
||||
PIPE,
|
||||
)
|
||||
|
||||
|
||||
##################################################
|
||||
# LVM helpers.
|
||||
##################################################
|
||||
def deactivate_lvm_volume_group(block_device):
|
||||
'''
|
||||
Deactivate any volume gruop associated with an LVM physical volume.
|
||||
|
||||
:param block_device: str: Full path to LVM physical volume
|
||||
'''
|
||||
vg = list_lvm_volume_group(block_device)
|
||||
if vg:
|
||||
cmd = ['vgchange', '-an', vg]
|
||||
check_call(cmd)
|
||||
|
||||
|
||||
def is_lvm_physical_volume(block_device):
|
||||
'''
|
||||
Determine whether a block device is initialized as an LVM PV.
|
||||
|
||||
:param block_device: str: Full path of block device to inspect.
|
||||
|
||||
:returns: boolean: True if block device is a PV, False if not.
|
||||
'''
|
||||
try:
|
||||
check_output(['pvdisplay', block_device])
|
||||
return True
|
||||
except CalledProcessError:
|
||||
return False
|
||||
|
||||
|
||||
def remove_lvm_physical_volume(block_device):
|
||||
'''
|
||||
Remove LVM PV signatures from a given block device.
|
||||
|
||||
:param block_device: str: Full path of block device to scrub.
|
||||
'''
|
||||
p = Popen(['pvremove', '-ff', block_device],
|
||||
stdin=PIPE)
|
||||
p.communicate(input='y\n')
|
||||
|
||||
|
||||
def list_lvm_volume_group(block_device):
|
||||
'''
|
||||
List LVM volume group associated with a given block device.
|
||||
|
||||
Assumes block device is a valid LVM PV.
|
||||
|
||||
:param block_device: str: Full path of block device to inspect.
|
||||
|
||||
:returns: str: Name of volume group associated with block device or None
|
||||
'''
|
||||
vg = None
|
||||
pvd = check_output(['pvdisplay', block_device]).splitlines()
|
||||
for l in pvd:
|
||||
l = l.decode('UTF-8')
|
||||
if l.strip().startswith('VG Name'):
|
||||
vg = ' '.join(l.strip().split()[2:])
|
||||
return vg
|
||||
|
||||
|
||||
def create_lvm_physical_volume(block_device):
|
||||
'''
|
||||
Initialize a block device as an LVM physical volume.
|
||||
|
||||
:param block_device: str: Full path of block device to initialize.
|
||||
|
||||
'''
|
||||
check_call(['pvcreate', block_device])
|
||||
|
||||
|
||||
def create_lvm_volume_group(volume_group, block_device):
|
||||
'''
|
||||
Create an LVM volume group backed by a given block device.
|
||||
|
||||
Assumes block device has already been initialized as an LVM PV.
|
||||
|
||||
:param volume_group: str: Name of volume group to create.
|
||||
:block_device: str: Full path of PV-initialized block device.
|
||||
'''
|
||||
check_call(['vgcreate', volume_group, block_device])
|
|
@ -1,71 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
import os
|
||||
import re
|
||||
from stat import S_ISBLK
|
||||
|
||||
from subprocess import (
|
||||
check_call,
|
||||
check_output,
|
||||
call
|
||||
)
|
||||
|
||||
|
||||
def is_block_device(path):
|
||||
'''
|
||||
Confirm device at path is a valid block device node.
|
||||
|
||||
:returns: boolean: True if path is a block device, False if not.
|
||||
'''
|
||||
if not os.path.exists(path):
|
||||
return False
|
||||
return S_ISBLK(os.stat(path).st_mode)
|
||||
|
||||
|
||||
def zap_disk(block_device):
|
||||
'''
|
||||
Clear a block device of partition table. Relies on sgdisk, which is
|
||||
installed as pat of the 'gdisk' package in Ubuntu.
|
||||
|
||||
:param block_device: str: Full path of block device to clean.
|
||||
'''
|
||||
# https://github.com/ceph/ceph/commit/fdd7f8d83afa25c4e09aaedd90ab93f3b64a677b
|
||||
# sometimes sgdisk exits non-zero; this is OK, dd will clean up
|
||||
call(['sgdisk', '--zap-all', '--', block_device])
|
||||
call(['sgdisk', '--clear', '--mbrtogpt', '--', block_device])
|
||||
dev_end = check_output(['blockdev', '--getsz',
|
||||
block_device]).decode('UTF-8')
|
||||
gpt_end = int(dev_end.split()[0]) - 100
|
||||
check_call(['dd', 'if=/dev/zero', 'of=%s' % (block_device),
|
||||
'bs=1M', 'count=1'])
|
||||
check_call(['dd', 'if=/dev/zero', 'of=%s' % (block_device),
|
||||
'bs=512', 'count=100', 'seek=%s' % (gpt_end)])
|
||||
|
||||
|
||||
def is_device_mounted(device):
|
||||
'''Given a device path, return True if that device is mounted, and False
|
||||
if it isn't.
|
||||
|
||||
:param device: str: Full path of the device to check.
|
||||
:returns: boolean: True if the path represents a mounted device, False if
|
||||
it doesn't.
|
||||
'''
|
||||
try:
|
||||
out = check_output(['lsblk', '-P', device]).decode('UTF-8')
|
||||
except:
|
||||
return False
|
||||
return bool(re.search(r'MOUNTPOINT=".+"', out))
|
|
@ -1,15 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
|
@ -1,57 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
#
|
||||
# Copyright 2014 Canonical Ltd.
|
||||
#
|
||||
# Authors:
|
||||
# Edward Hope-Morley <opentastic@gmail.com>
|
||||
#
|
||||
|
||||
import time
|
||||
|
||||
from charmhelpers.core.hookenv import (
|
||||
log,
|
||||
INFO,
|
||||
)
|
||||
|
||||
|
||||
def retry_on_exception(num_retries, base_delay=0, exc_type=Exception):
|
||||
"""If the decorated function raises exception exc_type, allow num_retries
|
||||
retry attempts before raise the exception.
|
||||
"""
|
||||
def _retry_on_exception_inner_1(f):
|
||||
def _retry_on_exception_inner_2(*args, **kwargs):
|
||||
retries = num_retries
|
||||
multiplier = 1
|
||||
while True:
|
||||
try:
|
||||
return f(*args, **kwargs)
|
||||
except exc_type:
|
||||
if not retries:
|
||||
raise
|
||||
|
||||
delay = base_delay * multiplier
|
||||
multiplier += 1
|
||||
log("Retrying '%s' %d more times (delay=%s)" %
|
||||
(f.__name__, retries, delay), level=INFO)
|
||||
retries -= 1
|
||||
if delay:
|
||||
time.sleep(delay)
|
||||
|
||||
return _retry_on_exception_inner_2
|
||||
|
||||
return _retry_on_exception_inner_1
|
|
@ -1,45 +0,0 @@
|
|||
#!/usr/bin/env python
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
__author__ = 'Jorge Niedbalski <niedbalski@ubuntu.com>'
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
|
||||
|
||||
def sed(filename, before, after, flags='g'):
|
||||
"""
|
||||
Search and replaces the given pattern on filename.
|
||||
|
||||
:param filename: relative or absolute file path.
|
||||
:param before: expression to be replaced (see 'man sed')
|
||||
:param after: expression to replace with (see 'man sed')
|
||||
:param flags: sed-compatible regex flags in example, to make
|
||||
the search and replace case insensitive, specify ``flags="i"``.
|
||||
The ``g`` flag is always specified regardless, so you do not
|
||||
need to remember to include it when overriding this parameter.
|
||||
:returns: If the sed command exit code was zero then return,
|
||||
otherwise raise CalledProcessError.
|
||||
"""
|
||||
expression = r's/{0}/{1}/{2}'.format(before,
|
||||
after, flags)
|
||||
|
||||
return subprocess.check_call(["sed", "-i", "-r", "-e",
|
||||
expression,
|
||||
os.path.expanduser(filename)])
|
|
@ -1,134 +0,0 @@
|
|||
#!/usr/bin/env python
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
import io
|
||||
import os
|
||||
|
||||
__author__ = 'Jorge Niedbalski R. <jorge.niedbalski@canonical.com>'
|
||||
|
||||
|
||||
class Fstab(io.FileIO):
|
||||
"""This class extends file in order to implement a file reader/writer
|
||||
for file `/etc/fstab`
|
||||
"""
|
||||
|
||||
class Entry(object):
|
||||
"""Entry class represents a non-comment line on the `/etc/fstab` file
|
||||
"""
|
||||
def __init__(self, device, mountpoint, filesystem,
|
||||
options, d=0, p=0):
|
||||
self.device = device
|
||||
self.mountpoint = mountpoint
|
||||
self.filesystem = filesystem
|
||||
|
||||
if not options:
|
||||
options = "defaults"
|
||||
|
||||
self.options = options
|
||||
self.d = int(d)
|
||||
self.p = int(p)
|
||||
|
||||
def __eq__(self, o):
|
||||
return str(self) == str(o)
|
||||
|
||||
def __str__(self):
|
||||
return "{} {} {} {} {} {}".format(self.device,
|
||||
self.mountpoint,
|
||||
self.filesystem,
|
||||
self.options,
|
||||
self.d,
|
||||
self.p)
|
||||
|
||||
DEFAULT_PATH = os.path.join(os.path.sep, 'etc', 'fstab')
|
||||
|
||||
def __init__(self, path=None):
|
||||
if path:
|
||||
self._path = path
|
||||
else:
|
||||
self._path = self.DEFAULT_PATH
|
||||
super(Fstab, self).__init__(self._path, 'rb+')
|
||||
|
||||
def _hydrate_entry(self, line):
|
||||
# NOTE: use split with no arguments to split on any
|
||||
# whitespace including tabs
|
||||
return Fstab.Entry(*filter(
|
||||
lambda x: x not in ('', None),
|
||||
line.strip("\n").split()))
|
||||
|
||||
@property
|
||||
def entries(self):
|
||||
self.seek(0)
|
||||
for line in self.readlines():
|
||||
line = line.decode('us-ascii')
|
||||
try:
|
||||
if line.strip() and not line.strip().startswith("#"):
|
||||
yield self._hydrate_entry(line)
|
||||
except ValueError:
|
||||
pass
|
||||
|
||||
def get_entry_by_attr(self, attr, value):
|
||||
for entry in self.entries:
|
||||
e_attr = getattr(entry, attr)
|
||||
if e_attr == value:
|
||||
return entry
|
||||
return None
|
||||
|
||||
def add_entry(self, entry):
|
||||
if self.get_entry_by_attr('device', entry.device):
|
||||
return False
|
||||
|
||||
self.write((str(entry) + '\n').encode('us-ascii'))
|
||||
self.truncate()
|
||||
return entry
|
||||
|
||||
def remove_entry(self, entry):
|
||||
self.seek(0)
|
||||
|
||||
lines = [l.decode('us-ascii') for l in self.readlines()]
|
||||
|
||||
found = False
|
||||
for index, line in enumerate(lines):
|
||||
if line.strip() and not line.strip().startswith("#"):
|
||||
if self._hydrate_entry(line) == entry:
|
||||
found = True
|
||||
break
|
||||
|
||||
if not found:
|
||||
return False
|
||||
|
||||
lines.remove(line)
|
||||
|
||||
self.seek(0)
|
||||
self.write(''.join(lines).encode('us-ascii'))
|
||||
self.truncate()
|
||||
return True
|
||||
|
||||
@classmethod
|
||||
def remove_by_mountpoint(cls, mountpoint, path=None):
|
||||
fstab = cls(path=path)
|
||||
entry = fstab.get_entry_by_attr('mountpoint', mountpoint)
|
||||
if entry:
|
||||
return fstab.remove_entry(entry)
|
||||
return False
|
||||
|
||||
@classmethod
|
||||
def add(cls, device, mountpoint, filesystem, options=None, path=None):
|
||||
return cls(path=path).add_entry(Fstab.Entry(device,
|
||||
mountpoint, filesystem,
|
||||
options=options))
|
File diff suppressed because it is too large
Load Diff
|
@ -1,717 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
"""Tools for working with the host system"""
|
||||
# Copyright 2012 Canonical Ltd.
|
||||
#
|
||||
# Authors:
|
||||
# Nick Moffitt <nick.moffitt@canonical.com>
|
||||
# Matthew Wedgwood <matthew.wedgwood@canonical.com>
|
||||
|
||||
import os
|
||||
import re
|
||||
import pwd
|
||||
import glob
|
||||
import grp
|
||||
import random
|
||||
import string
|
||||
import subprocess
|
||||
import hashlib
|
||||
import functools
|
||||
import itertools
|
||||
from contextlib import contextmanager
|
||||
from collections import OrderedDict
|
||||
|
||||
import six
|
||||
|
||||
from .hookenv import log
|
||||
from .fstab import Fstab
|
||||
|
||||
|
||||
def service_start(service_name):
|
||||
"""Start a system service"""
|
||||
return service('start', service_name)
|
||||
|
||||
|
||||
def service_stop(service_name):
|
||||
"""Stop a system service"""
|
||||
return service('stop', service_name)
|
||||
|
||||
|
||||
def service_restart(service_name):
|
||||
"""Restart a system service"""
|
||||
return service('restart', service_name)
|
||||
|
||||
|
||||
def service_reload(service_name, restart_on_failure=False):
|
||||
"""Reload a system service, optionally falling back to restart if
|
||||
reload fails"""
|
||||
service_result = service('reload', service_name)
|
||||
if not service_result and restart_on_failure:
|
||||
service_result = service('restart', service_name)
|
||||
return service_result
|
||||
|
||||
|
||||
def service_pause(service_name, init_dir="/etc/init", initd_dir="/etc/init.d"):
|
||||
"""Pause a system service.
|
||||
|
||||
Stop it, and prevent it from starting again at boot."""
|
||||
stopped = True
|
||||
if service_running(service_name):
|
||||
stopped = service_stop(service_name)
|
||||
upstart_file = os.path.join(init_dir, "{}.conf".format(service_name))
|
||||
sysv_file = os.path.join(initd_dir, service_name)
|
||||
if init_is_systemd():
|
||||
service('disable', service_name)
|
||||
elif os.path.exists(upstart_file):
|
||||
override_path = os.path.join(
|
||||
init_dir, '{}.override'.format(service_name))
|
||||
with open(override_path, 'w') as fh:
|
||||
fh.write("manual\n")
|
||||
elif os.path.exists(sysv_file):
|
||||
subprocess.check_call(["update-rc.d", service_name, "disable"])
|
||||
else:
|
||||
raise ValueError(
|
||||
"Unable to detect {0} as SystemD, Upstart {1} or"
|
||||
" SysV {2}".format(
|
||||
service_name, upstart_file, sysv_file))
|
||||
return stopped
|
||||
|
||||
|
||||
def service_resume(service_name, init_dir="/etc/init",
|
||||
initd_dir="/etc/init.d"):
|
||||
"""Resume a system service.
|
||||
|
||||
Reenable starting again at boot. Start the service"""
|
||||
upstart_file = os.path.join(init_dir, "{}.conf".format(service_name))
|
||||
sysv_file = os.path.join(initd_dir, service_name)
|
||||
if init_is_systemd():
|
||||
service('enable', service_name)
|
||||
elif os.path.exists(upstart_file):
|
||||
override_path = os.path.join(
|
||||
init_dir, '{}.override'.format(service_name))
|
||||
if os.path.exists(override_path):
|
||||
os.unlink(override_path)
|
||||
elif os.path.exists(sysv_file):
|
||||
subprocess.check_call(["update-rc.d", service_name, "enable"])
|
||||
else:
|
||||
raise ValueError(
|
||||
"Unable to detect {0} as SystemD, Upstart {1} or"
|
||||
" SysV {2}".format(
|
||||
service_name, upstart_file, sysv_file))
|
||||
|
||||
started = service_running(service_name)
|
||||
if not started:
|
||||
started = service_start(service_name)
|
||||
return started
|
||||
|
||||
|
||||
def service(action, service_name):
|
||||
"""Control a system service"""
|
||||
if init_is_systemd():
|
||||
cmd = ['systemctl', action, service_name]
|
||||
else:
|
||||
cmd = ['service', service_name, action]
|
||||
return subprocess.call(cmd) == 0
|
||||
|
||||
|
||||
def systemv_services_running():
|
||||
output = subprocess.check_output(
|
||||
['service', '--status-all'],
|
||||
stderr=subprocess.STDOUT).decode('UTF-8')
|
||||
return [row.split()[-1] for row in output.split('\n') if '[ + ]' in row]
|
||||
|
||||
|
||||
def service_running(service_name):
|
||||
"""Determine whether a system service is running"""
|
||||
if init_is_systemd():
|
||||
return service('is-active', service_name)
|
||||
else:
|
||||
try:
|
||||
output = subprocess.check_output(
|
||||
['service', service_name, 'status'],
|
||||
stderr=subprocess.STDOUT).decode('UTF-8')
|
||||
except subprocess.CalledProcessError:
|
||||
return False
|
||||
else:
|
||||
# This works for upstart scripts where the 'service' command
|
||||
# returns a consistent string to represent running 'start/running'
|
||||
if ("start/running" in output or "is running" in output or
|
||||
"up and running" in output):
|
||||
return True
|
||||
# Check System V scripts init script return codes
|
||||
if service_name in systemv_services_running():
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def service_available(service_name):
|
||||
"""Determine whether a system service is available"""
|
||||
try:
|
||||
subprocess.check_output(
|
||||
['service', service_name, 'status'],
|
||||
stderr=subprocess.STDOUT).decode('UTF-8')
|
||||
except subprocess.CalledProcessError as e:
|
||||
return b'unrecognized service' not in e.output
|
||||
else:
|
||||
return True
|
||||
|
||||
|
||||
SYSTEMD_SYSTEM = '/run/systemd/system'
|
||||
|
||||
|
||||
def init_is_systemd():
|
||||
"""Return True if the host system uses systemd, False otherwise."""
|
||||
return os.path.isdir(SYSTEMD_SYSTEM)
|
||||
|
||||
|
||||
def adduser(username, password=None, shell='/bin/bash', system_user=False,
|
||||
primary_group=None, secondary_groups=None):
|
||||
"""Add a user to the system.
|
||||
|
||||
Will log but otherwise succeed if the user already exists.
|
||||
|
||||
:param str username: Username to create
|
||||
:param str password: Password for user; if ``None``, create a system user
|
||||
:param str shell: The default shell for the user
|
||||
:param bool system_user: Whether to create a login or system user
|
||||
:param str primary_group: Primary group for user; defaults to username
|
||||
:param list secondary_groups: Optional list of additional groups
|
||||
|
||||
:returns: The password database entry struct, as returned by `pwd.getpwnam`
|
||||
"""
|
||||
try:
|
||||
user_info = pwd.getpwnam(username)
|
||||
log('user {0} already exists!'.format(username))
|
||||
except KeyError:
|
||||
log('creating user {0}'.format(username))
|
||||
cmd = ['useradd']
|
||||
if system_user or password is None:
|
||||
cmd.append('--system')
|
||||
else:
|
||||
cmd.extend([
|
||||
'--create-home',
|
||||
'--shell', shell,
|
||||
'--password', password,
|
||||
])
|
||||
if not primary_group:
|
||||
try:
|
||||
grp.getgrnam(username)
|
||||
primary_group = username # avoid "group exists" error
|
||||
except KeyError:
|
||||
pass
|
||||
if primary_group:
|
||||
cmd.extend(['-g', primary_group])
|
||||
if secondary_groups:
|
||||
cmd.extend(['-G', ','.join(secondary_groups)])
|
||||
cmd.append(username)
|
||||
subprocess.check_call(cmd)
|
||||
user_info = pwd.getpwnam(username)
|
||||
return user_info
|
||||
|
||||
|
||||
def user_exists(username):
|
||||
"""Check if a user exists"""
|
||||
try:
|
||||
pwd.getpwnam(username)
|
||||
user_exists = True
|
||||
except KeyError:
|
||||
user_exists = False
|
||||
return user_exists
|
||||
|
||||
|
||||
def add_group(group_name, system_group=False):
|
||||
"""Add a group to the system"""
|
||||
try:
|
||||
group_info = grp.getgrnam(group_name)
|
||||
log('group {0} already exists!'.format(group_name))
|
||||
except KeyError:
|
||||
log('creating group {0}'.format(group_name))
|
||||
cmd = ['addgroup']
|
||||
if system_group:
|
||||
cmd.append('--system')
|
||||
else:
|
||||
cmd.extend([
|
||||
'--group',
|
||||
])
|
||||
cmd.append(group_name)
|
||||
subprocess.check_call(cmd)
|
||||
group_info = grp.getgrnam(group_name)
|
||||
return group_info
|
||||
|
||||
|
||||
def add_user_to_group(username, group):
|
||||
"""Add a user to a group"""
|
||||
cmd = ['gpasswd', '-a', username, group]
|
||||
log("Adding user {} to group {}".format(username, group))
|
||||
subprocess.check_call(cmd)
|
||||
|
||||
|
||||
def rsync(from_path, to_path, flags='-r', options=None):
|
||||
"""Replicate the contents of a path"""
|
||||
options = options or ['--delete', '--executability']
|
||||
cmd = ['/usr/bin/rsync', flags]
|
||||
cmd.extend(options)
|
||||
cmd.append(from_path)
|
||||
cmd.append(to_path)
|
||||
log(" ".join(cmd))
|
||||
return subprocess.check_output(cmd).decode('UTF-8').strip()
|
||||
|
||||
|
||||
def symlink(source, destination):
|
||||
"""Create a symbolic link"""
|
||||
log("Symlinking {} as {}".format(source, destination))
|
||||
cmd = [
|
||||
'ln',
|
||||
'-sf',
|
||||
source,
|
||||
destination,
|
||||
]
|
||||
subprocess.check_call(cmd)
|
||||
|
||||
|
||||
def mkdir(path, owner='root', group='root', perms=0o555, force=False):
|
||||
"""Create a directory"""
|
||||
log("Making dir {} {}:{} {:o}".format(path, owner, group,
|
||||
perms))
|
||||
uid = pwd.getpwnam(owner).pw_uid
|
||||
gid = grp.getgrnam(group).gr_gid
|
||||
realpath = os.path.abspath(path)
|
||||
path_exists = os.path.exists(realpath)
|
||||
if path_exists and force:
|
||||
if not os.path.isdir(realpath):
|
||||
log("Removing non-directory file {} prior to mkdir()".format(path))
|
||||
os.unlink(realpath)
|
||||
os.makedirs(realpath, perms)
|
||||
elif not path_exists:
|
||||
os.makedirs(realpath, perms)
|
||||
os.chown(realpath, uid, gid)
|
||||
os.chmod(realpath, perms)
|
||||
|
||||
|
||||
def write_file(path, content, owner='root', group='root', perms=0o444):
|
||||
"""Create or overwrite a file with the contents of a byte string."""
|
||||
log("Writing file {} {}:{} {:o}".format(path, owner, group, perms))
|
||||
uid = pwd.getpwnam(owner).pw_uid
|
||||
gid = grp.getgrnam(group).gr_gid
|
||||
with open(path, 'wb') as target:
|
||||
os.fchown(target.fileno(), uid, gid)
|
||||
os.fchmod(target.fileno(), perms)
|
||||
target.write(content)
|
||||
|
||||
|
||||
def fstab_remove(mp):
|
||||
"""Remove the given mountpoint entry from /etc/fstab"""
|
||||
return Fstab.remove_by_mountpoint(mp)
|
||||
|
||||
|
||||
def fstab_add(dev, mp, fs, options=None):
|
||||
"""Adds the given device entry to the /etc/fstab file"""
|
||||
return Fstab.add(dev, mp, fs, options=options)
|
||||
|
||||
|
||||
def mount(device, mountpoint, options=None, persist=False, filesystem="ext3"):
|
||||
"""Mount a filesystem at a particular mountpoint"""
|
||||
cmd_args = ['mount']
|
||||
if options is not None:
|
||||
cmd_args.extend(['-o', options])
|
||||
cmd_args.extend([device, mountpoint])
|
||||
try:
|
||||
subprocess.check_output(cmd_args)
|
||||
except subprocess.CalledProcessError as e:
|
||||
log('Error mounting {} at {}\n{}'.format(device, mountpoint, e.output))
|
||||
return False
|
||||
|
||||
if persist:
|
||||
return fstab_add(device, mountpoint, filesystem, options=options)
|
||||
return True
|
||||
|
||||
|
||||
def umount(mountpoint, persist=False):
|
||||
"""Unmount a filesystem"""
|
||||
cmd_args = ['umount', mountpoint]
|
||||
try:
|
||||
subprocess.check_output(cmd_args)
|
||||
except subprocess.CalledProcessError as e:
|
||||
log('Error unmounting {}\n{}'.format(mountpoint, e.output))
|
||||
return False
|
||||
|
||||
if persist:
|
||||
return fstab_remove(mountpoint)
|
||||
return True
|
||||
|
||||
|
||||
def mounts():
|
||||
"""Get a list of all mounted volumes as [[mountpoint,device],[...]]"""
|
||||
with open('/proc/mounts') as f:
|
||||
# [['/mount/point','/dev/path'],[...]]
|
||||
system_mounts = [m[1::-1] for m in [l.strip().split()
|
||||
for l in f.readlines()]]
|
||||
return system_mounts
|
||||
|
||||
|
||||
def fstab_mount(mountpoint):
|
||||
"""Mount filesystem using fstab"""
|
||||
cmd_args = ['mount', mountpoint]
|
||||
try:
|
||||
subprocess.check_output(cmd_args)
|
||||
except subprocess.CalledProcessError as e:
|
||||
log('Error unmounting {}\n{}'.format(mountpoint, e.output))
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def file_hash(path, hash_type='md5'):
|
||||
"""Generate a hash checksum of the contents of 'path' or None if not found.
|
||||
|
||||
:param str hash_type: Any hash alrgorithm supported by :mod:`hashlib`,
|
||||
such as md5, sha1, sha256, sha512, etc.
|
||||
"""
|
||||
if os.path.exists(path):
|
||||
h = getattr(hashlib, hash_type)()
|
||||
with open(path, 'rb') as source:
|
||||
h.update(source.read())
|
||||
return h.hexdigest()
|
||||
else:
|
||||
return None
|
||||
|
||||
|
||||
def path_hash(path):
|
||||
"""Generate a hash checksum of all files matching 'path'. Standard
|
||||
wildcards like '*' and '?' are supported, see documentation for the 'glob'
|
||||
module for more information.
|
||||
|
||||
:return: dict: A { filename: hash } dictionary for all matched files.
|
||||
Empty if none found.
|
||||
"""
|
||||
return {
|
||||
filename: file_hash(filename)
|
||||
for filename in glob.iglob(path)
|
||||
}
|
||||
|
||||
|
||||
def check_hash(path, checksum, hash_type='md5'):
|
||||
"""Validate a file using a cryptographic checksum.
|
||||
|
||||
:param str checksum: Value of the checksum used to validate the file.
|
||||
:param str hash_type: Hash algorithm used to generate `checksum`.
|
||||
Can be any hash alrgorithm supported by :mod:`hashlib`,
|
||||
such as md5, sha1, sha256, sha512, etc.
|
||||
:raises ChecksumError: If the file fails the checksum
|
||||
|
||||
"""
|
||||
actual_checksum = file_hash(path, hash_type)
|
||||
if checksum != actual_checksum:
|
||||
raise ChecksumError("'%s' != '%s'" % (checksum, actual_checksum))
|
||||
|
||||
|
||||
class ChecksumError(ValueError):
|
||||
"""A class derived from Value error to indicate the checksum failed."""
|
||||
pass
|
||||
|
||||
|
||||
def restart_on_change(restart_map, stopstart=False, restart_functions=None):
|
||||
"""Restart services based on configuration files changing
|
||||
|
||||
This function is used a decorator, for example::
|
||||
|
||||
@restart_on_change({
|
||||
'/etc/ceph/ceph.conf': [ 'cinder-api', 'cinder-volume' ]
|
||||
'/etc/apache/sites-enabled/*': [ 'apache2' ]
|
||||
})
|
||||
def config_changed():
|
||||
pass # your code here
|
||||
|
||||
In this example, the cinder-api and cinder-volume services
|
||||
would be restarted if /etc/ceph/ceph.conf is changed by the
|
||||
ceph_client_changed function. The apache2 service would be
|
||||
restarted if any file matching the pattern got changed, created
|
||||
or removed. Standard wildcards are supported, see documentation
|
||||
for the 'glob' module for more information.
|
||||
|
||||
@param restart_map: {path_file_name: [service_name, ...]
|
||||
@param stopstart: DEFAULT false; whether to stop, start OR restart
|
||||
@param restart_functions: nonstandard functions to use to restart services
|
||||
{svc: func, ...}
|
||||
@returns result from decorated function
|
||||
"""
|
||||
def wrap(f):
|
||||
@functools.wraps(f)
|
||||
def wrapped_f(*args, **kwargs):
|
||||
return restart_on_change_helper(
|
||||
(lambda: f(*args, **kwargs)), restart_map, stopstart,
|
||||
restart_functions)
|
||||
return wrapped_f
|
||||
return wrap
|
||||
|
||||
|
||||
def restart_on_change_helper(lambda_f, restart_map, stopstart=False,
|
||||
restart_functions=None):
|
||||
"""Helper function to perform the restart_on_change function.
|
||||
|
||||
This is provided for decorators to restart services if files described
|
||||
in the restart_map have changed after an invocation of lambda_f().
|
||||
|
||||
@param lambda_f: function to call.
|
||||
@param restart_map: {file: [service, ...]}
|
||||
@param stopstart: whether to stop, start or restart a service
|
||||
@param restart_functions: nonstandard functions to use to restart services
|
||||
{svc: func, ...}
|
||||
@returns result of lambda_f()
|
||||
"""
|
||||
if restart_functions is None:
|
||||
restart_functions = {}
|
||||
checksums = {path: path_hash(path) for path in restart_map}
|
||||
r = lambda_f()
|
||||
# create a list of lists of the services to restart
|
||||
restarts = [restart_map[path]
|
||||
for path in restart_map
|
||||
if path_hash(path) != checksums[path]]
|
||||
# create a flat list of ordered services without duplicates from lists
|
||||
services_list = list(OrderedDict.fromkeys(itertools.chain(*restarts)))
|
||||
if services_list:
|
||||
actions = ('stop', 'start') if stopstart else ('restart',)
|
||||
for service_name in services_list:
|
||||
if service_name in restart_functions:
|
||||
restart_functions[service_name](service_name)
|
||||
else:
|
||||
for action in actions:
|
||||
service(action, service_name)
|
||||
return r
|
||||
|
||||
|
||||
def lsb_release():
|
||||
"""Return /etc/lsb-release in a dict"""
|
||||
d = {}
|
||||
with open('/etc/lsb-release', 'r') as lsb:
|
||||
for l in lsb:
|
||||
k, v = l.split('=')
|
||||
d[k.strip()] = v.strip()
|
||||
return d
|
||||
|
||||
|
||||
def pwgen(length=None):
|
||||
"""Generate a random pasword."""
|
||||
if length is None:
|
||||
# A random length is ok to use a weak PRNG
|
||||
length = random.choice(range(35, 45))
|
||||
alphanumeric_chars = [
|
||||
l for l in (string.ascii_letters + string.digits)
|
||||
if l not in 'l0QD1vAEIOUaeiou']
|
||||
# Use a crypto-friendly PRNG (e.g. /dev/urandom) for making the
|
||||
# actual password
|
||||
random_generator = random.SystemRandom()
|
||||
random_chars = [
|
||||
random_generator.choice(alphanumeric_chars) for _ in range(length)]
|
||||
return(''.join(random_chars))
|
||||
|
||||
|
||||
def is_phy_iface(interface):
|
||||
"""Returns True if interface is not virtual, otherwise False."""
|
||||
if interface:
|
||||
sys_net = '/sys/class/net'
|
||||
if os.path.isdir(sys_net):
|
||||
for iface in glob.glob(os.path.join(sys_net, '*')):
|
||||
if '/virtual/' in os.path.realpath(iface):
|
||||
continue
|
||||
|
||||
if interface == os.path.basename(iface):
|
||||
return True
|
||||
|
||||
return False
|
||||
|
||||
|
||||
def get_bond_master(interface):
|
||||
"""Returns bond master if interface is bond slave otherwise None.
|
||||
|
||||
NOTE: the provided interface is expected to be physical
|
||||
"""
|
||||
if interface:
|
||||
iface_path = '/sys/class/net/%s' % (interface)
|
||||
if os.path.exists(iface_path):
|
||||
if '/virtual/' in os.path.realpath(iface_path):
|
||||
return None
|
||||
|
||||
master = os.path.join(iface_path, 'master')
|
||||
if os.path.exists(master):
|
||||
master = os.path.realpath(master)
|
||||
# make sure it is a bond master
|
||||
if os.path.exists(os.path.join(master, 'bonding')):
|
||||
return os.path.basename(master)
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def list_nics(nic_type=None):
|
||||
"""Return a list of nics of given type(s)"""
|
||||
if isinstance(nic_type, six.string_types):
|
||||
int_types = [nic_type]
|
||||
else:
|
||||
int_types = nic_type
|
||||
|
||||
interfaces = []
|
||||
if nic_type:
|
||||
for int_type in int_types:
|
||||
cmd = ['ip', 'addr', 'show', 'label', int_type + '*']
|
||||
ip_output = subprocess.check_output(cmd).decode('UTF-8')
|
||||
ip_output = ip_output.split('\n')
|
||||
ip_output = (line for line in ip_output if line)
|
||||
for line in ip_output:
|
||||
if line.split()[1].startswith(int_type):
|
||||
matched = re.search('.*: (' + int_type +
|
||||
r'[0-9]+\.[0-9]+)@.*', line)
|
||||
if matched:
|
||||
iface = matched.groups()[0]
|
||||
else:
|
||||
iface = line.split()[1].replace(":", "")
|
||||
|
||||
if iface not in interfaces:
|
||||
interfaces.append(iface)
|
||||
else:
|
||||
cmd = ['ip', 'a']
|
||||
ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n')
|
||||
ip_output = (line.strip() for line in ip_output if line)
|
||||
|
||||
key = re.compile('^[0-9]+:\s+(.+):')
|
||||
for line in ip_output:
|
||||
matched = re.search(key, line)
|
||||
if matched:
|
||||
iface = matched.group(1)
|
||||
iface = iface.partition("@")[0]
|
||||
if iface not in interfaces:
|
||||
interfaces.append(iface)
|
||||
|
||||
return interfaces
|
||||
|
||||
|
||||
def set_nic_mtu(nic, mtu):
|
||||
"""Set the Maximum Transmission Unit (MTU) on a network interface."""
|
||||
cmd = ['ip', 'link', 'set', nic, 'mtu', mtu]
|
||||
subprocess.check_call(cmd)
|
||||
|
||||
|
||||
def get_nic_mtu(nic):
|
||||
"""Return the Maximum Transmission Unit (MTU) for a network interface."""
|
||||
cmd = ['ip', 'addr', 'show', nic]
|
||||
ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n')
|
||||
mtu = ""
|
||||
for line in ip_output:
|
||||
words = line.split()
|
||||
if 'mtu' in words:
|
||||
mtu = words[words.index("mtu") + 1]
|
||||
return mtu
|
||||
|
||||
|
||||
def get_nic_hwaddr(nic):
|
||||
"""Return the Media Access Control (MAC) for a network interface."""
|
||||
cmd = ['ip', '-o', '-0', 'addr', 'show', nic]
|
||||
ip_output = subprocess.check_output(cmd).decode('UTF-8')
|
||||
hwaddr = ""
|
||||
words = ip_output.split()
|
||||
if 'link/ether' in words:
|
||||
hwaddr = words[words.index('link/ether') + 1]
|
||||
return hwaddr
|
||||
|
||||
|
||||
def cmp_pkgrevno(package, revno, pkgcache=None):
|
||||
"""Compare supplied revno with the revno of the installed package
|
||||
|
||||
* 1 => Installed revno is greater than supplied arg
|
||||
* 0 => Installed revno is the same as supplied arg
|
||||
* -1 => Installed revno is less than supplied arg
|
||||
|
||||
This function imports apt_cache function from charmhelpers.fetch if
|
||||
the pkgcache argument is None. Be sure to add charmhelpers.fetch if
|
||||
you call this function, or pass an apt_pkg.Cache() instance.
|
||||
"""
|
||||
import apt_pkg
|
||||
if not pkgcache:
|
||||
from charmhelpers.fetch import apt_cache
|
||||
pkgcache = apt_cache()
|
||||
pkg = pkgcache[package]
|
||||
return apt_pkg.version_compare(pkg.current_ver.ver_str, revno)
|
||||
|
||||
|
||||
@contextmanager
|
||||
def chdir(directory):
|
||||
"""Change the current working directory to a different directory for a code
|
||||
block and return the previous directory after the block exits. Useful to
|
||||
run commands from a specificed directory.
|
||||
|
||||
:param str directory: The directory path to change to for this context.
|
||||
"""
|
||||
cur = os.getcwd()
|
||||
try:
|
||||
yield os.chdir(directory)
|
||||
finally:
|
||||
os.chdir(cur)
|
||||
|
||||
|
||||
def chownr(path, owner, group, follow_links=True, chowntopdir=False):
|
||||
"""Recursively change user and group ownership of files and directories
|
||||
in given path. Doesn't chown path itself by default, only its children.
|
||||
|
||||
:param str path: The string path to start changing ownership.
|
||||
:param str owner: The owner string to use when looking up the uid.
|
||||
:param str group: The group string to use when looking up the gid.
|
||||
:param bool follow_links: Also Chown links if True
|
||||
:param bool chowntopdir: Also chown path itself if True
|
||||
"""
|
||||
uid = pwd.getpwnam(owner).pw_uid
|
||||
gid = grp.getgrnam(group).gr_gid
|
||||
if follow_links:
|
||||
chown = os.chown
|
||||
else:
|
||||
chown = os.lchown
|
||||
|
||||
if chowntopdir:
|
||||
broken_symlink = os.path.lexists(path) and not os.path.exists(path)
|
||||
if not broken_symlink:
|
||||
chown(path, uid, gid)
|
||||
for root, dirs, files in os.walk(path):
|
||||
for name in dirs + files:
|
||||
full = os.path.join(root, name)
|
||||
broken_symlink = os.path.lexists(full) and not os.path.exists(full)
|
||||
if not broken_symlink:
|
||||
chown(full, uid, gid)
|
||||
|
||||
|
||||
def lchownr(path, owner, group):
|
||||
"""Recursively change user and group ownership of files and directories
|
||||
in a given path, not following symbolic links. See the documentation for
|
||||
'os.lchown' for more information.
|
||||
|
||||
:param str path: The string path to start changing ownership.
|
||||
:param str owner: The owner string to use when looking up the uid.
|
||||
:param str group: The group string to use when looking up the gid.
|
||||
"""
|
||||
chownr(path, owner, group, follow_links=False)
|
||||
|
||||
|
||||
def get_total_ram():
|
||||
"""The total amount of system RAM in bytes.
|
||||
|
||||
This is what is reported by the OS, and may be overcommitted when
|
||||
there are multiple containers hosted on the same machine.
|
||||
"""
|
||||
with open('/proc/meminfo', 'r') as f:
|
||||
for line in f.readlines():
|
||||
if line:
|
||||
key, value, unit = line.split()
|
||||
if key == 'MemTotal:':
|
||||
assert unit == 'kB', 'Unknown unit'
|
||||
return int(value) * 1024 # Classic, not KiB.
|
||||
raise NotImplementedError()
|
|
@ -1,71 +0,0 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
|
||||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
import yaml
|
||||
from charmhelpers.core import fstab
|
||||
from charmhelpers.core import sysctl
|
||||
from charmhelpers.core.host import (
|
||||
add_group,
|
||||
add_user_to_group,
|
||||
fstab_mount,
|
||||
mkdir,
|
||||
)
|
||||
from charmhelpers.core.strutils import bytes_from_string
|
||||
from subprocess import check_output
|
||||
|
||||
|
||||
def hugepage_support(user, group='hugetlb', nr_hugepages=256,
|
||||
max_map_count=65536, mnt_point='/run/hugepages/kvm',
|
||||
pagesize='2MB', mount=True, set_shmmax=False):
|
||||
"""Enable hugepages on system.
|
||||
|
||||
Args:
|
||||
user (str) -- Username to allow access to hugepages to
|
||||
group (str) -- Group name to own hugepages
|
||||
nr_hugepages (int) -- Number of pages to reserve
|
||||
max_map_count (int) -- Number of Virtual Memory Areas a process can own
|
||||
mnt_point (str) -- Directory to mount hugepages on
|
||||
pagesize (str) -- Size of hugepages
|
||||
mount (bool) -- Whether to Mount hugepages
|
||||
"""
|
||||
group_info = add_group(group)
|
||||
gid = group_info.gr_gid
|
||||
add_user_to_group(user, group)
|
||||
if max_map_count < 2 * nr_hugepages:
|
||||
max_map_count = 2 * nr_hugepages
|
||||
sysctl_settings = {
|
||||
'vm.nr_hugepages': nr_hugepages,
|
||||
'vm.max_map_count': max_map_count,
|
||||
'vm.hugetlb_shm_group': gid,
|
||||
}
|
||||
if set_shmmax:
|
||||
shmmax_current = int(check_output(['sysctl', '-n', 'kernel.shmmax']))
|
||||
shmmax_minsize = bytes_from_string(pagesize) * nr_hugepages
|
||||
if shmmax_minsize > shmmax_current:
|
||||
sysctl_settings['kernel.shmmax'] = shmmax_minsize
|
||||
sysctl.create(yaml.dump(sysctl_settings), '/etc/sysctl.d/10-hugepage.conf')
|
||||
mkdir(mnt_point, owner='root', group='root', perms=0o755, force=False)
|
||||
lfstab = fstab.Fstab()
|
||||
fstab_entry = lfstab.get_entry_by_attr('mountpoint', mnt_point)
|
||||
if fstab_entry:
|
||||
lfstab.remove_entry(fstab_entry)
|
||||
entry = lfstab.Entry('nodev', mnt_point, 'hugetlbfs',
|
||||
'mode=1770,gid={},pagesize={}'.format(gid, pagesize), 0, 0)
|
||||
lfstab.add_entry(entry)
|
||||
if mount:
|
||||
fstab_mount(mnt_point)
|
|
@ -1,68 +0,0 @@
|
|||
#!/usr/bin/env python
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
__author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>"
|
||||
|
||||
from charmhelpers.core.hookenv import (
|
||||
log,
|
||||
INFO
|
||||
)
|
||||
|
||||
from subprocess import check_call, check_output
|
||||
import re
|
||||
|
||||
|
||||
def modprobe(module, persist=True):
|
||||
"""Load a kernel module and configure for auto-load on reboot."""
|
||||
cmd = ['modprobe', module]
|
||||
|
||||
log('Loading kernel module %s' % module, level=INFO)
|
||||
|
||||
check_call(cmd)
|
||||
if persist:
|
||||
with open('/etc/modules', 'r+') as modules:
|
||||
if module not in modules.read():
|
||||
modules.write(module)
|
||||
|
||||
|
||||
def rmmod(module, force=False):
|
||||
"""Remove a module from the linux kernel"""
|
||||
cmd = ['rmmod']
|
||||
if force:
|
||||
cmd.append('-f')
|
||||
cmd.append(module)
|
||||
log('Removing kernel module %s' % module, level=INFO)
|
||||
return check_call(cmd)
|
||||
|
||||
|
||||
def lsmod():
|
||||
"""Shows what kernel modules are currently loaded"""
|
||||
return check_output(['lsmod'],
|
||||
universal_newlines=True)
|
||||
|
||||
|
||||
def is_module_loaded(module):
|
||||
"""Checks if a kernel module is already loaded"""
|
||||
matches = re.findall('^%s[ ]+' % module, lsmod(), re.M)
|
||||
return len(matches) > 0
|
||||
|
||||
|
||||
def update_initramfs(version='all'):
|
||||
"""Updates an initramfs image"""
|
||||
return check_call(["update-initramfs", "-k", version, "-u"])
|
|
@ -1,18 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
from .base import * # NOQA
|
||||
from .helpers import * # NOQA
|
|
@ -1,353 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
import os
|
||||
import json
|
||||
from inspect import getargspec
|
||||
from collections import Iterable, OrderedDict
|
||||
|
||||
from charmhelpers.core import host
|
||||
from charmhelpers.core import hookenv
|
||||
|
||||
|
||||
__all__ = ['ServiceManager', 'ManagerCallback',
|
||||
'PortManagerCallback', 'open_ports', 'close_ports', 'manage_ports',
|
||||
'service_restart', 'service_stop']
|
||||
|
||||
|
||||
class ServiceManager(object):
|
||||
def __init__(self, services=None):
|
||||
"""
|
||||
Register a list of services, given their definitions.
|
||||
|
||||
Service definitions are dicts in the following formats (all keys except
|
||||
'service' are optional)::
|
||||
|
||||
{
|
||||
"service": <service name>,
|
||||
"required_data": <list of required data contexts>,
|
||||
"provided_data": <list of provided data contexts>,
|
||||
"data_ready": <one or more callbacks>,
|
||||
"data_lost": <one or more callbacks>,
|
||||
"start": <one or more callbacks>,
|
||||
"stop": <one or more callbacks>,
|
||||
"ports": <list of ports to manage>,
|
||||
}
|
||||
|
||||
The 'required_data' list should contain dicts of required data (or
|
||||
dependency managers that act like dicts and know how to collect the data).
|
||||
Only when all items in the 'required_data' list are populated are the list
|
||||
of 'data_ready' and 'start' callbacks executed. See `is_ready()` for more
|
||||
information.
|
||||
|
||||
The 'provided_data' list should contain relation data providers, most likely
|
||||
a subclass of :class:`charmhelpers.core.services.helpers.RelationContext`,
|
||||
that will indicate a set of data to set on a given relation.
|
||||
|
||||
The 'data_ready' value should be either a single callback, or a list of
|
||||
callbacks, to be called when all items in 'required_data' pass `is_ready()`.
|
||||
Each callback will be called with the service name as the only parameter.
|
||||
After all of the 'data_ready' callbacks are called, the 'start' callbacks
|
||||
are fired.
|
||||
|
||||
The 'data_lost' value should be either a single callback, or a list of
|
||||
callbacks, to be called when a 'required_data' item no longer passes
|
||||
`is_ready()`. Each callback will be called with the service name as the
|
||||
only parameter. After all of the 'data_lost' callbacks are called,
|
||||
the 'stop' callbacks are fired.
|
||||
|
||||
The 'start' value should be either a single callback, or a list of
|
||||
callbacks, to be called when starting the service, after the 'data_ready'
|
||||
callbacks are complete. Each callback will be called with the service
|
||||
name as the only parameter. This defaults to
|
||||
`[host.service_start, services.open_ports]`.
|
||||
|
||||
The 'stop' value should be either a single callback, or a list of
|
||||
callbacks, to be called when stopping the service. If the service is
|
||||
being stopped because it no longer has all of its 'required_data', this
|
||||
will be called after all of the 'data_lost' callbacks are complete.
|
||||
Each callback will be called with the service name as the only parameter.
|
||||
This defaults to `[services.close_ports, host.service_stop]`.
|
||||
|
||||
The 'ports' value should be a list of ports to manage. The default
|
||||
'start' handler will open the ports after the service is started,
|
||||
and the default 'stop' handler will close the ports prior to stopping
|
||||
the service.
|
||||
|
||||
|
||||
Examples:
|
||||
|
||||
The following registers an Upstart service called bingod that depends on
|
||||
a mongodb relation and which runs a custom `db_migrate` function prior to
|
||||
restarting the service, and a Runit service called spadesd::
|
||||
|
||||
manager = services.ServiceManager([
|
||||
{
|
||||
'service': 'bingod',
|
||||
'ports': [80, 443],
|
||||
'required_data': [MongoRelation(), config(), {'my': 'data'}],
|
||||
'data_ready': [
|
||||
services.template(source='bingod.conf'),
|
||||
services.template(source='bingod.ini',
|
||||
target='/etc/bingod.ini',
|
||||
owner='bingo', perms=0400),
|
||||
],
|
||||
},
|
||||
{
|
||||
'service': 'spadesd',
|
||||
'data_ready': services.template(source='spadesd_run.j2',
|
||||
target='/etc/sv/spadesd/run',
|
||||
perms=0555),
|
||||
'start': runit_start,
|
||||
'stop': runit_stop,
|
||||
},
|
||||
])
|
||||
manager.manage()
|
||||
"""
|
||||
self._ready_file = os.path.join(hookenv.charm_dir(), 'READY-SERVICES.json')
|
||||
self._ready = None
|
||||
self.services = OrderedDict()
|
||||
for service in services or []:
|
||||
service_name = service['service']
|
||||
self.services[service_name] = service
|
||||
|
||||
def manage(self):
|
||||
"""
|
||||
Handle the current hook by doing The Right Thing with the registered services.
|
||||
"""
|
||||
hookenv._run_atstart()
|
||||
try:
|
||||
hook_name = hookenv.hook_name()
|
||||
if hook_name == 'stop':
|
||||
self.stop_services()
|
||||
else:
|
||||
self.reconfigure_services()
|
||||
self.provide_data()
|
||||
except SystemExit as x:
|
||||
if x.code is None or x.code == 0:
|
||||
hookenv._run_atexit()
|
||||
hookenv._run_atexit()
|
||||
|
||||
def provide_data(self):
|
||||
"""
|
||||
Set the relation data for each provider in the ``provided_data`` list.
|
||||
|
||||
A provider must have a `name` attribute, which indicates which relation
|
||||
to set data on, and a `provide_data()` method, which returns a dict of
|
||||
data to set.
|
||||
|
||||
The `provide_data()` method can optionally accept two parameters:
|
||||
|
||||
* ``remote_service`` The name of the remote service that the data will
|
||||
be provided to. The `provide_data()` method will be called once
|
||||
for each connected service (not unit). This allows the method to
|
||||
tailor its data to the given service.
|
||||
* ``service_ready`` Whether or not the service definition had all of
|
||||
its requirements met, and thus the ``data_ready`` callbacks run.
|
||||
|
||||
Note that the ``provided_data`` methods are now called **after** the
|
||||
``data_ready`` callbacks are run. This gives the ``data_ready`` callbacks
|
||||
a chance to generate any data necessary for the providing to the remote
|
||||
services.
|
||||
"""
|
||||
for service_name, service in self.services.items():
|
||||
service_ready = self.is_ready(service_name)
|
||||
for provider in service.get('provided_data', []):
|
||||
for relid in hookenv.relation_ids(provider.name):
|
||||
units = hookenv.related_units(relid)
|
||||
if not units:
|
||||
continue
|
||||
remote_service = units[0].split('/')[0]
|
||||
argspec = getargspec(provider.provide_data)
|
||||
if len(argspec.args) > 1:
|
||||
data = provider.provide_data(remote_service, service_ready)
|
||||
else:
|
||||
data = provider.provide_data()
|
||||
if data:
|
||||
hookenv.relation_set(relid, data)
|
||||
|
||||
def reconfigure_services(self, *service_names):
|
||||
"""
|
||||
Update all files for one or more registered services, and,
|
||||
if ready, optionally restart them.
|
||||
|
||||
If no service names are given, reconfigures all registered services.
|
||||
"""
|
||||
for service_name in service_names or self.services.keys():
|
||||
if self.is_ready(service_name):
|
||||
self.fire_event('data_ready', service_name)
|
||||
self.fire_event('start', service_name, default=[
|
||||
service_restart,
|
||||
manage_ports])
|
||||
self.save_ready(service_name)
|
||||
else:
|
||||
if self.was_ready(service_name):
|
||||
self.fire_event('data_lost', service_name)
|
||||
self.fire_event('stop', service_name, default=[
|
||||
manage_ports,
|
||||
service_stop])
|
||||
self.save_lost(service_name)
|
||||
|
||||
def stop_services(self, *service_names):
|
||||
"""
|
||||
Stop one or more registered services, by name.
|
||||
|
||||
If no service names are given, stops all registered services.
|
||||
"""
|
||||
for service_name in service_names or self.services.keys():
|
||||
self.fire_event('stop', service_name, default=[
|
||||
manage_ports,
|
||||
service_stop])
|
||||
|
||||
def get_service(self, service_name):
|
||||
"""
|
||||
Given the name of a registered service, return its service definition.
|
||||
"""
|
||||
service = self.services.get(service_name)
|
||||
if not service:
|
||||
raise KeyError('Service not registered: %s' % service_name)
|
||||
return service
|
||||
|
||||
def fire_event(self, event_name, service_name, default=None):
|
||||
"""
|
||||
Fire a data_ready, data_lost, start, or stop event on a given service.
|
||||
"""
|
||||
service = self.get_service(service_name)
|
||||
callbacks = service.get(event_name, default)
|
||||
if not callbacks:
|
||||
return
|
||||
if not isinstance(callbacks, Iterable):
|
||||
callbacks = [callbacks]
|
||||
for callback in callbacks:
|
||||
if isinstance(callback, ManagerCallback):
|
||||
callback(self, service_name, event_name)
|
||||
else:
|
||||
callback(service_name)
|
||||
|
||||
def is_ready(self, service_name):
|
||||
"""
|
||||
Determine if a registered service is ready, by checking its 'required_data'.
|
||||
|
||||
A 'required_data' item can be any mapping type, and is considered ready
|
||||
if `bool(item)` evaluates as True.
|
||||
"""
|
||||
service = self.get_service(service_name)
|
||||
reqs = service.get('required_data', [])
|
||||
return all(bool(req) for req in reqs)
|
||||
|
||||
def _load_ready_file(self):
|
||||
if self._ready is not None:
|
||||
return
|
||||
if os.path.exists(self._ready_file):
|
||||
with open(self._ready_file) as fp:
|
||||
self._ready = set(json.load(fp))
|
||||
else:
|
||||
self._ready = set()
|
||||
|
||||
def _save_ready_file(self):
|
||||
if self._ready is None:
|
||||
return
|
||||
with open(self._ready_file, 'w') as fp:
|
||||
json.dump(list(self._ready), fp)
|
||||
|
||||
def save_ready(self, service_name):
|
||||
"""
|
||||
Save an indicator that the given service is now data_ready.
|
||||
"""
|
||||
self._load_ready_file()
|
||||
self._ready.add(service_name)
|
||||
self._save_ready_file()
|
||||
|
||||
def save_lost(self, service_name):
|
||||
"""
|
||||
Save an indicator that the given service is no longer data_ready.
|
||||
"""
|
||||
self._load_ready_file()
|
||||
self._ready.discard(service_name)
|
||||
self._save_ready_file()
|
||||
|
||||
def was_ready(self, service_name):
|
||||
"""
|
||||
Determine if the given service was previously data_ready.
|
||||
"""
|
||||
self._load_ready_file()
|
||||
return service_name in self._ready
|
||||
|
||||
|
||||
class ManagerCallback(object):
|
||||
"""
|
||||
Special case of a callback that takes the `ServiceManager` instance
|
||||
in addition to the service name.
|
||||
|
||||
Subclasses should implement `__call__` which should accept three parameters:
|
||||
|
||||
* `manager` The `ServiceManager` instance
|
||||
* `service_name` The name of the service it's being triggered for
|
||||
* `event_name` The name of the event that this callback is handling
|
||||
"""
|
||||
def __call__(self, manager, service_name, event_name):
|
||||
raise NotImplementedError()
|
||||
|
||||
|
||||
class PortManagerCallback(ManagerCallback):
|
||||
"""
|
||||
Callback class that will open or close ports, for use as either
|
||||
a start or stop action.
|
||||
"""
|
||||
def __call__(self, manager, service_name, event_name):
|
||||
service = manager.get_service(service_name)
|
||||
new_ports = service.get('ports', [])
|
||||
port_file = os.path.join(hookenv.charm_dir(), '.{}.ports'.format(service_name))
|
||||
if os.path.exists(port_file):
|
||||
with open(port_file) as fp:
|
||||
old_ports = fp.read().split(',')
|
||||
for old_port in old_ports:
|
||||
if bool(old_port):
|
||||
old_port = int(old_port)
|
||||
if old_port not in new_ports:
|
||||
hookenv.close_port(old_port)
|
||||
with open(port_file, 'w') as fp:
|
||||
fp.write(','.join(str(port) for port in new_ports))
|
||||
for port in new_ports:
|
||||
if event_name == 'start':
|
||||
hookenv.open_port(port)
|
||||
elif event_name == 'stop':
|
||||
hookenv.close_port(port)
|
||||
|
||||
|
||||
def service_stop(service_name):
|
||||
"""
|
||||
Wrapper around host.service_stop to prevent spurious "unknown service"
|
||||
messages in the logs.
|
||||
"""
|
||||
if host.service_running(service_name):
|
||||
host.service_stop(service_name)
|
||||
|
||||
|
||||
def service_restart(service_name):
|
||||
"""
|
||||
Wrapper around host.service_restart to prevent spurious "unknown service"
|
||||
messages in the logs.
|
||||
"""
|
||||
if host.service_available(service_name):
|
||||
if host.service_running(service_name):
|
||||
host.service_restart(service_name)
|
||||
else:
|
||||
host.service_start(service_name)
|
||||
|
||||
|
||||
# Convenience aliases
|
||||
open_ports = close_ports = manage_ports = PortManagerCallback()
|
|
@ -1,292 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
import os
|
||||
import yaml
|
||||
|
||||
from charmhelpers.core import hookenv
|
||||
from charmhelpers.core import host
|
||||
from charmhelpers.core import templating
|
||||
|
||||
from charmhelpers.core.services.base import ManagerCallback
|
||||
|
||||
|
||||
__all__ = ['RelationContext', 'TemplateCallback',
|
||||
'render_template', 'template']
|
||||
|
||||
|
||||
class RelationContext(dict):
|
||||
"""
|
||||
Base class for a context generator that gets relation data from juju.
|
||||
|
||||
Subclasses must provide the attributes `name`, which is the name of the
|
||||
interface of interest, `interface`, which is the type of the interface of
|
||||
interest, and `required_keys`, which is the set of keys required for the
|
||||
relation to be considered complete. The data for all interfaces matching
|
||||
the `name` attribute that are complete will used to populate the dictionary
|
||||
values (see `get_data`, below).
|
||||
|
||||
The generated context will be namespaced under the relation :attr:`name`,
|
||||
to prevent potential naming conflicts.
|
||||
|
||||
:param str name: Override the relation :attr:`name`, since it can vary from charm to charm
|
||||
:param list additional_required_keys: Extend the list of :attr:`required_keys`
|
||||
"""
|
||||
name = None
|
||||
interface = None
|
||||
|
||||
def __init__(self, name=None, additional_required_keys=None):
|
||||
if not hasattr(self, 'required_keys'):
|
||||
self.required_keys = []
|
||||
|
||||
if name is not None:
|
||||
self.name = name
|
||||
if additional_required_keys:
|
||||
self.required_keys.extend(additional_required_keys)
|
||||
self.get_data()
|
||||
|
||||
def __bool__(self):
|
||||
"""
|
||||
Returns True if all of the required_keys are available.
|
||||
"""
|
||||
return self.is_ready()
|
||||
|
||||
__nonzero__ = __bool__
|
||||
|
||||
def __repr__(self):
|
||||
return super(RelationContext, self).__repr__()
|
||||
|
||||
def is_ready(self):
|
||||
"""
|
||||
Returns True if all of the `required_keys` are available from any units.
|
||||
"""
|
||||
ready = len(self.get(self.name, [])) > 0
|
||||
if not ready:
|
||||
hookenv.log('Incomplete relation: {}'.format(self.__class__.__name__), hookenv.DEBUG)
|
||||
return ready
|
||||
|
||||
def _is_ready(self, unit_data):
|
||||
"""
|
||||
Helper method that tests a set of relation data and returns True if
|
||||
all of the `required_keys` are present.
|
||||
"""
|
||||
return set(unit_data.keys()).issuperset(set(self.required_keys))
|
||||
|
||||
def get_data(self):
|
||||
"""
|
||||
Retrieve the relation data for each unit involved in a relation and,
|
||||
if complete, store it in a list under `self[self.name]`. This
|
||||
is automatically called when the RelationContext is instantiated.
|
||||
|
||||
The units are sorted lexographically first by the service ID, then by
|
||||
the unit ID. Thus, if an interface has two other services, 'db:1'
|
||||
and 'db:2', with 'db:1' having two units, 'wordpress/0' and 'wordpress/1',
|
||||
and 'db:2' having one unit, 'mediawiki/0', all of which have a complete
|
||||
set of data, the relation data for the units will be stored in the
|
||||
order: 'wordpress/0', 'wordpress/1', 'mediawiki/0'.
|
||||
|
||||
If you only care about a single unit on the relation, you can just
|
||||
access it as `{{ interface[0]['key'] }}`. However, if you can at all
|
||||
support multiple units on a relation, you should iterate over the list,
|
||||
like::
|
||||
|
||||
{% for unit in interface -%}
|
||||
{{ unit['key'] }}{% if not loop.last %},{% endif %}
|
||||
{%- endfor %}
|
||||
|
||||
Note that since all sets of relation data from all related services and
|
||||
units are in a single list, if you need to know which service or unit a
|
||||
set of data came from, you'll need to extend this class to preserve
|
||||
that information.
|
||||
"""
|
||||
if not hookenv.relation_ids(self.name):
|
||||
return
|
||||
|
||||
ns = self.setdefault(self.name, [])
|
||||
for rid in sorted(hookenv.relation_ids(self.name)):
|
||||
for unit in sorted(hookenv.related_units(rid)):
|
||||
reldata = hookenv.relation_get(rid=rid, unit=unit)
|
||||
if self._is_ready(reldata):
|
||||
ns.append(reldata)
|
||||
|
||||
def provide_data(self):
|
||||
"""
|
||||
Return data to be relation_set for this interface.
|
||||
"""
|
||||
return {}
|
||||
|
||||
|
||||
class MysqlRelation(RelationContext):
|
||||
"""
|
||||
Relation context for the `mysql` interface.
|
||||
|
||||
:param str name: Override the relation :attr:`name`, since it can vary from charm to charm
|
||||
:param list additional_required_keys: Extend the list of :attr:`required_keys`
|
||||
"""
|
||||
name = 'db'
|
||||
interface = 'mysql'
|
||||
|
||||
def __init__(self, *args, **kwargs):
|
||||
self.required_keys = ['host', 'user', 'password', 'database']
|
||||
RelationContext.__init__(self, *args, **kwargs)
|
||||
|
||||
|
||||
class HttpRelation(RelationContext):
|
||||
"""
|
||||
Relation context for the `http` interface.
|
||||
|
||||
:param str name: Override the relation :attr:`name`, since it can vary from charm to charm
|
||||
:param list additional_required_keys: Extend the list of :attr:`required_keys`
|
||||
"""
|
||||
name = 'website'
|
||||
interface = 'http'
|
||||
|
||||
def __init__(self, *args, **kwargs):
|
||||
self.required_keys = ['host', 'port']
|
||||
RelationContext.__init__(self, *args, **kwargs)
|
||||
|
||||
def provide_data(self):
|
||||
return {
|
||||
'host': hookenv.unit_get('private-address'),
|
||||
'port': 80,
|
||||
}
|
||||
|
||||
|
||||
class RequiredConfig(dict):
|
||||
"""
|
||||
Data context that loads config options with one or more mandatory options.
|
||||
|
||||
Once the required options have been changed from their default values, all
|
||||
config options will be available, namespaced under `config` to prevent
|
||||
potential naming conflicts (for example, between a config option and a
|
||||
relation property).
|
||||
|
||||
:param list *args: List of options that must be changed from their default values.
|
||||
"""
|
||||
|
||||
def __init__(self, *args):
|
||||
self.required_options = args
|
||||
self['config'] = hookenv.config()
|
||||
with open(os.path.join(hookenv.charm_dir(), 'config.yaml')) as fp:
|
||||
self.config = yaml.load(fp).get('options', {})
|
||||
|
||||
def __bool__(self):
|
||||
for option in self.required_options:
|
||||
if option not in self['config']:
|
||||
return False
|
||||
current_value = self['config'][option]
|
||||
default_value = self.config[option].get('default')
|
||||
if current_value == default_value:
|
||||
return False
|
||||
if current_value in (None, '') and default_value in (None, ''):
|
||||
return False
|
||||
return True
|
||||
|
||||
def __nonzero__(self):
|
||||
return self.__bool__()
|
||||
|
||||
|
||||
class StoredContext(dict):
|
||||
"""
|
||||
A data context that always returns the data that it was first created with.
|
||||
|
||||
This is useful to do a one-time generation of things like passwords, that
|
||||
will thereafter use the same value that was originally generated, instead
|
||||
of generating a new value each time it is run.
|
||||
"""
|
||||
def __init__(self, file_name, config_data):
|
||||
"""
|
||||
If the file exists, populate `self` with the data from the file.
|
||||
Otherwise, populate with the given data and persist it to the file.
|
||||
"""
|
||||
if os.path.exists(file_name):
|
||||
self.update(self.read_context(file_name))
|
||||
else:
|
||||
self.store_context(file_name, config_data)
|
||||
self.update(config_data)
|
||||
|
||||
def store_context(self, file_name, config_data):
|
||||
if not os.path.isabs(file_name):
|
||||
file_name = os.path.join(hookenv.charm_dir(), file_name)
|
||||
with open(file_name, 'w') as file_stream:
|
||||
os.fchmod(file_stream.fileno(), 0o600)
|
||||
yaml.dump(config_data, file_stream)
|
||||
|
||||
def read_context(self, file_name):
|
||||
if not os.path.isabs(file_name):
|
||||
file_name = os.path.join(hookenv.charm_dir(), file_name)
|
||||
with open(file_name, 'r') as file_stream:
|
||||
data = yaml.load(file_stream)
|
||||
if not data:
|
||||
raise OSError("%s is empty" % file_name)
|
||||
return data
|
||||
|
||||
|
||||
class TemplateCallback(ManagerCallback):
|
||||
"""
|
||||
Callback class that will render a Jinja2 template, for use as a ready
|
||||
action.
|
||||
|
||||
:param str source: The template source file, relative to
|
||||
`$CHARM_DIR/templates`
|
||||
|
||||
:param str target: The target to write the rendered template to (or None)
|
||||
:param str owner: The owner of the rendered file
|
||||
:param str group: The group of the rendered file
|
||||
:param int perms: The permissions of the rendered file
|
||||
:param partial on_change_action: functools partial to be executed when
|
||||
rendered file changes
|
||||
:param jinja2 loader template_loader: A jinja2 template loader
|
||||
|
||||
:return str: The rendered template
|
||||
"""
|
||||
def __init__(self, source, target,
|
||||
owner='root', group='root', perms=0o444,
|
||||
on_change_action=None, template_loader=None):
|
||||
self.source = source
|
||||
self.target = target
|
||||
self.owner = owner
|
||||
self.group = group
|
||||
self.perms = perms
|
||||
self.on_change_action = on_change_action
|
||||
self.template_loader = template_loader
|
||||
|
||||
def __call__(self, manager, service_name, event_name):
|
||||
pre_checksum = ''
|
||||
if self.on_change_action and os.path.isfile(self.target):
|
||||
pre_checksum = host.file_hash(self.target)
|
||||
service = manager.get_service(service_name)
|
||||
context = {'ctx': {}}
|
||||
for ctx in service.get('required_data', []):
|
||||
context.update(ctx)
|
||||
context['ctx'].update(ctx)
|
||||
|
||||
result = templating.render(self.source, self.target, context,
|
||||
self.owner, self.group, self.perms,
|
||||
template_loader=self.template_loader)
|
||||
if self.on_change_action:
|
||||
if pre_checksum == host.file_hash(self.target):
|
||||
hookenv.log(
|
||||
'No change detected: {}'.format(self.target),
|
||||
hookenv.DEBUG)
|
||||
else:
|
||||
self.on_change_action()
|
||||
|
||||
return result
|
||||
|
||||
|
||||
# Convenience aliases for templates
|
||||
render_template = template = TemplateCallback
|
|
@ -1,72 +0,0 @@
|
|||
#!/usr/bin/env python
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
import six
|
||||
import re
|
||||
|
||||
|
||||
def bool_from_string(value):
|
||||
"""Interpret string value as boolean.
|
||||
|
||||
Returns True if value translates to True otherwise False.
|
||||
"""
|
||||
if isinstance(value, six.string_types):
|
||||
value = six.text_type(value)
|
||||
else:
|
||||
msg = "Unable to interpret non-string value '%s' as boolean" % (value)
|
||||
raise ValueError(msg)
|
||||
|
||||
value = value.strip().lower()
|
||||
|
||||
if value in ['y', 'yes', 'true', 't', 'on']:
|
||||
return True
|
||||
elif value in ['n', 'no', 'false', 'f', 'off']:
|
||||
return False
|
||||
|
||||
msg = "Unable to interpret string value '%s' as boolean" % (value)
|
||||
raise ValueError(msg)
|
||||
|
||||
|
||||
def bytes_from_string(value):
|
||||
"""Interpret human readable string value as bytes.
|
||||
|
||||
Returns int
|
||||
"""
|
||||
BYTE_POWER = {
|
||||
'K': 1,
|
||||
'KB': 1,
|
||||
'M': 2,
|
||||
'MB': 2,
|
||||
'G': 3,
|
||||
'GB': 3,
|
||||
'T': 4,
|
||||
'TB': 4,
|
||||
'P': 5,
|
||||
'PB': 5,
|
||||
}
|
||||
if isinstance(value, six.string_types):
|
||||
value = six.text_type(value)
|
||||
else:
|
||||
msg = "Unable to interpret non-string value '%s' as boolean" % (value)
|
||||
raise ValueError(msg)
|
||||
matches = re.match("([0-9]+)([a-zA-Z]+)", value)
|
||||
if not matches:
|
||||
msg = "Unable to interpret string value '%s' as bytes" % (value)
|
||||
raise ValueError(msg)
|
||||
return int(matches.group(1)) * (1024 ** BYTE_POWER[matches.group(2)])
|
|
@ -1,56 +0,0 @@
|
|||
#!/usr/bin/env python
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
import yaml
|
||||
|
||||
from subprocess import check_call
|
||||
|
||||
from charmhelpers.core.hookenv import (
|
||||
log,
|
||||
DEBUG,
|
||||
ERROR,
|
||||
)
|
||||
|
||||
__author__ = 'Jorge Niedbalski R. <jorge.niedbalski@canonical.com>'
|
||||
|
||||
|
||||
def create(sysctl_dict, sysctl_file):
|
||||
"""Creates a sysctl.conf file from a YAML associative array
|
||||
|
||||
:param sysctl_dict: a YAML-formatted string of sysctl options eg "{ 'kernel.max_pid': 1337 }"
|
||||
:type sysctl_dict: str
|
||||
:param sysctl_file: path to the sysctl file to be saved
|
||||
:type sysctl_file: str or unicode
|
||||
:returns: None
|
||||
"""
|
||||
try:
|
||||
sysctl_dict_parsed = yaml.safe_load(sysctl_dict)
|
||||
except yaml.YAMLError:
|
||||
log("Error parsing YAML sysctl_dict: {}".format(sysctl_dict),
|
||||
level=ERROR)
|
||||
return
|
||||
|
||||
with open(sysctl_file, "w") as fd:
|
||||
for key, value in sysctl_dict_parsed.items():
|
||||
fd.write("{}={}\n".format(key, value))
|
||||
|
||||
log("Updating sysctl_file: %s values: %s" % (sysctl_file, sysctl_dict_parsed),
|
||||
level=DEBUG)
|
||||
|
||||
check_call(["sysctl", "-p", sysctl_file])
|
|
@ -1,81 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
import os
|
||||
|
||||
from charmhelpers.core import host
|
||||
from charmhelpers.core import hookenv
|
||||
|
||||
|
||||
def render(source, target, context, owner='root', group='root',
|
||||
perms=0o444, templates_dir=None, encoding='UTF-8', template_loader=None):
|
||||
"""
|
||||
Render a template.
|
||||
|
||||
The `source` path, if not absolute, is relative to the `templates_dir`.
|
||||
|
||||
The `target` path should be absolute. It can also be `None`, in which
|
||||
case no file will be written.
|
||||
|
||||
The context should be a dict containing the values to be replaced in the
|
||||
template.
|
||||
|
||||
The `owner`, `group`, and `perms` options will be passed to `write_file`.
|
||||
|
||||
If omitted, `templates_dir` defaults to the `templates` folder in the charm.
|
||||
|
||||
The rendered template will be written to the file as well as being returned
|
||||
as a string.
|
||||
|
||||
Note: Using this requires python-jinja2; if it is not installed, calling
|
||||
this will attempt to use charmhelpers.fetch.apt_install to install it.
|
||||
"""
|
||||
try:
|
||||
from jinja2 import FileSystemLoader, Environment, exceptions
|
||||
except ImportError:
|
||||
try:
|
||||
from charmhelpers.fetch import apt_install
|
||||
except ImportError:
|
||||
hookenv.log('Could not import jinja2, and could not import '
|
||||
'charmhelpers.fetch to install it',
|
||||
level=hookenv.ERROR)
|
||||
raise
|
||||
apt_install('python-jinja2', fatal=True)
|
||||
from jinja2 import FileSystemLoader, Environment, exceptions
|
||||
|
||||
if template_loader:
|
||||
template_env = Environment(loader=template_loader)
|
||||
else:
|
||||
if templates_dir is None:
|
||||
templates_dir = os.path.join(hookenv.charm_dir(), 'templates')
|
||||
template_env = Environment(loader=FileSystemLoader(templates_dir))
|
||||
try:
|
||||
source = source
|
||||
template = template_env.get_template(source)
|
||||
except exceptions.TemplateNotFound as e:
|
||||
hookenv.log('Could not load template %s from %s.' %
|
||||
(source, templates_dir),
|
||||
level=hookenv.ERROR)
|
||||
raise e
|
||||
content = template.render(context)
|
||||
if target is not None:
|
||||
target_dir = os.path.dirname(target)
|
||||
if not os.path.exists(target_dir):
|
||||
# This is a terrible default directory permission, as the file
|
||||
# or its siblings will often contain secrets.
|
||||
host.mkdir(os.path.dirname(target), owner, group, perms=0o755)
|
||||
host.write_file(target, content.encode(encoding), owner, group, perms)
|
||||
return content
|
|
@ -1,521 +0,0 @@
|
|||
#!/usr/bin/env python
|
||||
# -*- coding: utf-8 -*-
|
||||
#
|
||||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
#
|
||||
#
|
||||
# Authors:
|
||||
# Kapil Thangavelu <kapil.foss@gmail.com>
|
||||
#
|
||||
"""
|
||||
Intro
|
||||
-----
|
||||
|
||||
A simple way to store state in units. This provides a key value
|
||||
storage with support for versioned, transactional operation,
|
||||
and can calculate deltas from previous values to simplify unit logic
|
||||
when processing changes.
|
||||
|
||||
|
||||
Hook Integration
|
||||
----------------
|
||||
|
||||
There are several extant frameworks for hook execution, including
|
||||
|
||||
- charmhelpers.core.hookenv.Hooks
|
||||
- charmhelpers.core.services.ServiceManager
|
||||
|
||||
The storage classes are framework agnostic, one simple integration is
|
||||
via the HookData contextmanager. It will record the current hook
|
||||
execution environment (including relation data, config data, etc.),
|
||||
setup a transaction and allow easy access to the changes from
|
||||
previously seen values. One consequence of the integration is the
|
||||
reservation of particular keys ('rels', 'unit', 'env', 'config',
|
||||
'charm_revisions') for their respective values.
|
||||
|
||||
Here's a fully worked integration example using hookenv.Hooks::
|
||||
|
||||
from charmhelper.core import hookenv, unitdata
|
||||
|
||||
hook_data = unitdata.HookData()
|
||||
db = unitdata.kv()
|
||||
hooks = hookenv.Hooks()
|
||||
|
||||
@hooks.hook
|
||||
def config_changed():
|
||||
# Print all changes to configuration from previously seen
|
||||
# values.
|
||||
for changed, (prev, cur) in hook_data.conf.items():
|
||||
print('config changed', changed,
|
||||
'previous value', prev,
|
||||
'current value', cur)
|
||||
|
||||
# Get some unit specific bookeeping
|
||||
if not db.get('pkg_key'):
|
||||
key = urllib.urlopen('https://example.com/pkg_key').read()
|
||||
db.set('pkg_key', key)
|
||||
|
||||
# Directly access all charm config as a mapping.
|
||||
conf = db.getrange('config', True)
|
||||
|
||||
# Directly access all relation data as a mapping
|
||||
rels = db.getrange('rels', True)
|
||||
|
||||
if __name__ == '__main__':
|
||||
with hook_data():
|
||||
hook.execute()
|
||||
|
||||
|
||||
A more basic integration is via the hook_scope context manager which simply
|
||||
manages transaction scope (and records hook name, and timestamp)::
|
||||
|
||||
>>> from unitdata import kv
|
||||
>>> db = kv()
|
||||
>>> with db.hook_scope('install'):
|
||||
... # do work, in transactional scope.
|
||||
... db.set('x', 1)
|
||||
>>> db.get('x')
|
||||
1
|
||||
|
||||
|
||||
Usage
|
||||
-----
|
||||
|
||||
Values are automatically json de/serialized to preserve basic typing
|
||||
and complex data struct capabilities (dicts, lists, ints, booleans, etc).
|
||||
|
||||
Individual values can be manipulated via get/set::
|
||||
|
||||
>>> kv.set('y', True)
|
||||
>>> kv.get('y')
|
||||
True
|
||||
|
||||
# We can set complex values (dicts, lists) as a single key.
|
||||
>>> kv.set('config', {'a': 1, 'b': True'})
|
||||
|
||||
# Also supports returning dictionaries as a record which
|
||||
# provides attribute access.
|
||||
>>> config = kv.get('config', record=True)
|
||||
>>> config.b
|
||||
True
|
||||
|
||||
|
||||
Groups of keys can be manipulated with update/getrange::
|
||||
|
||||
>>> kv.update({'z': 1, 'y': 2}, prefix="gui.")
|
||||
>>> kv.getrange('gui.', strip=True)
|
||||
{'z': 1, 'y': 2}
|
||||
|
||||
When updating values, its very helpful to understand which values
|
||||
have actually changed and how have they changed. The storage
|
||||
provides a delta method to provide for this::
|
||||
|
||||
>>> data = {'debug': True, 'option': 2}
|
||||
>>> delta = kv.delta(data, 'config.')
|
||||
>>> delta.debug.previous
|
||||
None
|
||||
>>> delta.debug.current
|
||||
True
|
||||
>>> delta
|
||||
{'debug': (None, True), 'option': (None, 2)}
|
||||
|
||||
Note the delta method does not persist the actual change, it needs to
|
||||
be explicitly saved via 'update' method::
|
||||
|
||||
>>> kv.update(data, 'config.')
|
||||
|
||||
Values modified in the context of a hook scope retain historical values
|
||||
associated to the hookname.
|
||||
|
||||
>>> with db.hook_scope('config-changed'):
|
||||
... db.set('x', 42)
|
||||
>>> db.gethistory('x')
|
||||
[(1, u'x', 1, u'install', u'2015-01-21T16:49:30.038372'),
|
||||
(2, u'x', 42, u'config-changed', u'2015-01-21T16:49:30.038786')]
|
||||
|
||||
"""
|
||||
|
||||
import collections
|
||||
import contextlib
|
||||
import datetime
|
||||
import itertools
|
||||
import json
|
||||
import os
|
||||
import pprint
|
||||
import sqlite3
|
||||
import sys
|
||||
|
||||
__author__ = 'Kapil Thangavelu <kapil.foss@gmail.com>'
|
||||
|
||||
|
||||
class Storage(object):
|
||||
"""Simple key value database for local unit state within charms.
|
||||
|
||||
Modifications are not persisted unless :meth:`flush` is called.
|
||||
|
||||
To support dicts, lists, integer, floats, and booleans values
|
||||
are automatically json encoded/decoded.
|
||||
"""
|
||||
def __init__(self, path=None):
|
||||
self.db_path = path
|
||||
if path is None:
|
||||
if 'UNIT_STATE_DB' in os.environ:
|
||||
self.db_path = os.environ['UNIT_STATE_DB']
|
||||
else:
|
||||
self.db_path = os.path.join(
|
||||
os.environ.get('CHARM_DIR', ''), '.unit-state.db')
|
||||
self.conn = sqlite3.connect('%s' % self.db_path)
|
||||
self.cursor = self.conn.cursor()
|
||||
self.revision = None
|
||||
self._closed = False
|
||||
self._init()
|
||||
|
||||
def close(self):
|
||||
if self._closed:
|
||||
return
|
||||
self.flush(False)
|
||||
self.cursor.close()
|
||||
self.conn.close()
|
||||
self._closed = True
|
||||
|
||||
def get(self, key, default=None, record=False):
|
||||
self.cursor.execute('select data from kv where key=?', [key])
|
||||
result = self.cursor.fetchone()
|
||||
if not result:
|
||||
return default
|
||||
if record:
|
||||
return Record(json.loads(result[0]))
|
||||
return json.loads(result[0])
|
||||
|
||||
def getrange(self, key_prefix, strip=False):
|
||||
"""
|
||||
Get a range of keys starting with a common prefix as a mapping of
|
||||
keys to values.
|
||||
|
||||
:param str key_prefix: Common prefix among all keys
|
||||
:param bool strip: Optionally strip the common prefix from the key
|
||||
names in the returned dict
|
||||
:return dict: A (possibly empty) dict of key-value mappings
|
||||
"""
|
||||
self.cursor.execute("select key, data from kv where key like ?",
|
||||
['%s%%' % key_prefix])
|
||||
result = self.cursor.fetchall()
|
||||
|
||||
if not result:
|
||||
return {}
|
||||
if not strip:
|
||||
key_prefix = ''
|
||||
return dict([
|
||||
(k[len(key_prefix):], json.loads(v)) for k, v in result])
|
||||
|
||||
def update(self, mapping, prefix=""):
|
||||
"""
|
||||
Set the values of multiple keys at once.
|
||||
|
||||
:param dict mapping: Mapping of keys to values
|
||||
:param str prefix: Optional prefix to apply to all keys in `mapping`
|
||||
before setting
|
||||
"""
|
||||
for k, v in mapping.items():
|
||||
self.set("%s%s" % (prefix, k), v)
|
||||
|
||||
def unset(self, key):
|
||||
"""
|
||||
Remove a key from the database entirely.
|
||||
"""
|
||||
self.cursor.execute('delete from kv where key=?', [key])
|
||||
if self.revision and self.cursor.rowcount:
|
||||
self.cursor.execute(
|
||||
'insert into kv_revisions values (?, ?, ?)',
|
||||
[key, self.revision, json.dumps('DELETED')])
|
||||
|
||||
def unsetrange(self, keys=None, prefix=""):
|
||||
"""
|
||||
Remove a range of keys starting with a common prefix, from the database
|
||||
entirely.
|
||||
|
||||
:param list keys: List of keys to remove.
|
||||
:param str prefix: Optional prefix to apply to all keys in ``keys``
|
||||
before removing.
|
||||
"""
|
||||
if keys is not None:
|
||||
keys = ['%s%s' % (prefix, key) for key in keys]
|
||||
self.cursor.execute('delete from kv where key in (%s)' % ','.join(['?'] * len(keys)), keys)
|
||||
if self.revision and self.cursor.rowcount:
|
||||
self.cursor.execute(
|
||||
'insert into kv_revisions values %s' % ','.join(['(?, ?, ?)'] * len(keys)),
|
||||
list(itertools.chain.from_iterable((key, self.revision, json.dumps('DELETED')) for key in keys)))
|
||||
else:
|
||||
self.cursor.execute('delete from kv where key like ?',
|
||||
['%s%%' % prefix])
|
||||
if self.revision and self.cursor.rowcount:
|
||||
self.cursor.execute(
|
||||
'insert into kv_revisions values (?, ?, ?)',
|
||||
['%s%%' % prefix, self.revision, json.dumps('DELETED')])
|
||||
|
||||
def set(self, key, value):
|
||||
"""
|
||||
Set a value in the database.
|
||||
|
||||
:param str key: Key to set the value for
|
||||
:param value: Any JSON-serializable value to be set
|
||||
"""
|
||||
serialized = json.dumps(value)
|
||||
|
||||
self.cursor.execute('select data from kv where key=?', [key])
|
||||
exists = self.cursor.fetchone()
|
||||
|
||||
# Skip mutations to the same value
|
||||
if exists:
|
||||
if exists[0] == serialized:
|
||||
return value
|
||||
|
||||
if not exists:
|
||||
self.cursor.execute(
|
||||
'insert into kv (key, data) values (?, ?)',
|
||||
(key, serialized))
|
||||
else:
|
||||
self.cursor.execute('''
|
||||
update kv
|
||||
set data = ?
|
||||
where key = ?''', [serialized, key])
|
||||
|
||||
# Save
|
||||
if not self.revision:
|
||||
return value
|
||||
|
||||
self.cursor.execute(
|
||||
'select 1 from kv_revisions where key=? and revision=?',
|
||||
[key, self.revision])
|
||||
exists = self.cursor.fetchone()
|
||||
|
||||
if not exists:
|
||||
self.cursor.execute(
|
||||
'''insert into kv_revisions (
|
||||
revision, key, data) values (?, ?, ?)''',
|
||||
(self.revision, key, serialized))
|
||||
else:
|
||||
self.cursor.execute(
|
||||
'''
|
||||
update kv_revisions
|
||||
set data = ?
|
||||
where key = ?
|
||||
and revision = ?''',
|
||||
[serialized, key, self.revision])
|
||||
|
||||
return value
|
||||
|
||||
def delta(self, mapping, prefix):
|
||||
"""
|
||||
return a delta containing values that have changed.
|
||||
"""
|
||||
previous = self.getrange(prefix, strip=True)
|
||||
if not previous:
|
||||
pk = set()
|
||||
else:
|
||||
pk = set(previous.keys())
|
||||
ck = set(mapping.keys())
|
||||
delta = DeltaSet()
|
||||
|
||||
# added
|
||||
for k in ck.difference(pk):
|
||||
delta[k] = Delta(None, mapping[k])
|
||||
|
||||
# removed
|
||||
for k in pk.difference(ck):
|
||||
delta[k] = Delta(previous[k], None)
|
||||
|
||||
# changed
|
||||
for k in pk.intersection(ck):
|
||||
c = mapping[k]
|
||||
p = previous[k]
|
||||
if c != p:
|
||||
delta[k] = Delta(p, c)
|
||||
|
||||
return delta
|
||||
|
||||
@contextlib.contextmanager
|
||||
def hook_scope(self, name=""):
|
||||
"""Scope all future interactions to the current hook execution
|
||||
revision."""
|
||||
assert not self.revision
|
||||
self.cursor.execute(
|
||||
'insert into hooks (hook, date) values (?, ?)',
|
||||
(name or sys.argv[0],
|
||||
datetime.datetime.utcnow().isoformat()))
|
||||
self.revision = self.cursor.lastrowid
|
||||
try:
|
||||
yield self.revision
|
||||
self.revision = None
|
||||
except:
|
||||
self.flush(False)
|
||||
self.revision = None
|
||||
raise
|
||||
else:
|
||||
self.flush()
|
||||
|
||||
def flush(self, save=True):
|
||||
if save:
|
||||
self.conn.commit()
|
||||
elif self._closed:
|
||||
return
|
||||
else:
|
||||
self.conn.rollback()
|
||||
|
||||
def _init(self):
|
||||
self.cursor.execute('''
|
||||
create table if not exists kv (
|
||||
key text,
|
||||
data text,
|
||||
primary key (key)
|
||||
)''')
|
||||
self.cursor.execute('''
|
||||
create table if not exists kv_revisions (
|
||||
key text,
|
||||
revision integer,
|
||||
data text,
|
||||
primary key (key, revision)
|
||||
)''')
|
||||
self.cursor.execute('''
|
||||
create table if not exists hooks (
|
||||
version integer primary key autoincrement,
|
||||
hook text,
|
||||
date text
|
||||
)''')
|
||||
self.conn.commit()
|
||||
|
||||
def gethistory(self, key, deserialize=False):
|
||||
self.cursor.execute(
|
||||
'''
|
||||
select kv.revision, kv.key, kv.data, h.hook, h.date
|
||||
from kv_revisions kv,
|
||||
hooks h
|
||||
where kv.key=?
|
||||
and kv.revision = h.version
|
||||
''', [key])
|
||||
if deserialize is False:
|
||||
return self.cursor.fetchall()
|
||||
return map(_parse_history, self.cursor.fetchall())
|
||||
|
||||
def debug(self, fh=sys.stderr):
|
||||
self.cursor.execute('select * from kv')
|
||||
pprint.pprint(self.cursor.fetchall(), stream=fh)
|
||||
self.cursor.execute('select * from kv_revisions')
|
||||
pprint.pprint(self.cursor.fetchall(), stream=fh)
|
||||
|
||||
|
||||
def _parse_history(d):
|
||||
return (d[0], d[1], json.loads(d[2]), d[3],
|
||||
datetime.datetime.strptime(d[-1], "%Y-%m-%dT%H:%M:%S.%f"))
|
||||
|
||||
|
||||
class HookData(object):
|
||||
"""Simple integration for existing hook exec frameworks.
|
||||
|
||||
Records all unit information, and stores deltas for processing
|
||||
by the hook.
|
||||
|
||||
Sample::
|
||||
|
||||
from charmhelper.core import hookenv, unitdata
|
||||
|
||||
changes = unitdata.HookData()
|
||||
db = unitdata.kv()
|
||||
hooks = hookenv.Hooks()
|
||||
|
||||
@hooks.hook
|
||||
def config_changed():
|
||||
# View all changes to configuration
|
||||
for changed, (prev, cur) in changes.conf.items():
|
||||
print('config changed', changed,
|
||||
'previous value', prev,
|
||||
'current value', cur)
|
||||
|
||||
# Get some unit specific bookeeping
|
||||
if not db.get('pkg_key'):
|
||||
key = urllib.urlopen('https://example.com/pkg_key').read()
|
||||
db.set('pkg_key', key)
|
||||
|
||||
if __name__ == '__main__':
|
||||
with changes():
|
||||
hook.execute()
|
||||
|
||||
"""
|
||||
def __init__(self):
|
||||
self.kv = kv()
|
||||
self.conf = None
|
||||
self.rels = None
|
||||
|
||||
@contextlib.contextmanager
|
||||
def __call__(self):
|
||||
from charmhelpers.core import hookenv
|
||||
hook_name = hookenv.hook_name()
|
||||
|
||||
with self.kv.hook_scope(hook_name):
|
||||
self._record_charm_version(hookenv.charm_dir())
|
||||
delta_config, delta_relation = self._record_hook(hookenv)
|
||||
yield self.kv, delta_config, delta_relation
|
||||
|
||||
def _record_charm_version(self, charm_dir):
|
||||
# Record revisions.. charm revisions are meaningless
|
||||
# to charm authors as they don't control the revision.
|
||||
# so logic dependnent on revision is not particularly
|
||||
# useful, however it is useful for debugging analysis.
|
||||
charm_rev = open(
|
||||
os.path.join(charm_dir, 'revision')).read().strip()
|
||||
charm_rev = charm_rev or '0'
|
||||
revs = self.kv.get('charm_revisions', [])
|
||||
if charm_rev not in revs:
|
||||
revs.append(charm_rev.strip() or '0')
|
||||
self.kv.set('charm_revisions', revs)
|
||||
|
||||
def _record_hook(self, hookenv):
|
||||
data = hookenv.execution_environment()
|
||||
self.conf = conf_delta = self.kv.delta(data['conf'], 'config')
|
||||
self.rels = rels_delta = self.kv.delta(data['rels'], 'rels')
|
||||
self.kv.set('env', dict(data['env']))
|
||||
self.kv.set('unit', data['unit'])
|
||||
self.kv.set('relid', data.get('relid'))
|
||||
return conf_delta, rels_delta
|
||||
|
||||
|
||||
class Record(dict):
|
||||
|
||||
__slots__ = ()
|
||||
|
||||
def __getattr__(self, k):
|
||||
if k in self:
|
||||
return self[k]
|
||||
raise AttributeError(k)
|
||||
|
||||
|
||||
class DeltaSet(Record):
|
||||
|
||||
__slots__ = ()
|
||||
|
||||
|
||||
Delta = collections.namedtuple('Delta', ['previous', 'current'])
|
||||
|
||||
|
||||
_KV = None
|
||||
|
||||
|
||||
def kv():
|
||||
global _KV
|
||||
if _KV is None:
|
||||
_KV = Storage()
|
||||
return _KV
|
|
@ -1,464 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
import importlib
|
||||
from tempfile import NamedTemporaryFile
|
||||
import time
|
||||
from yaml import safe_load
|
||||
from charmhelpers.core.host import (
|
||||
lsb_release
|
||||
)
|
||||
import subprocess
|
||||
from charmhelpers.core.hookenv import (
|
||||
config,
|
||||
log,
|
||||
)
|
||||
import os
|
||||
|
||||
import six
|
||||
if six.PY3:
|
||||
from urllib.parse import urlparse, urlunparse
|
||||
else:
|
||||
from urlparse import urlparse, urlunparse
|
||||
|
||||
|
||||
CLOUD_ARCHIVE = """# Ubuntu Cloud Archive
|
||||
deb http://ubuntu-cloud.archive.canonical.com/ubuntu {} main
|
||||
"""
|
||||
PROPOSED_POCKET = """# Proposed
|
||||
deb http://archive.ubuntu.com/ubuntu {}-proposed main universe multiverse restricted
|
||||
"""
|
||||
CLOUD_ARCHIVE_POCKETS = {
|
||||
# Folsom
|
||||
'folsom': 'precise-updates/folsom',
|
||||
'precise-folsom': 'precise-updates/folsom',
|
||||
'precise-folsom/updates': 'precise-updates/folsom',
|
||||
'precise-updates/folsom': 'precise-updates/folsom',
|
||||
'folsom/proposed': 'precise-proposed/folsom',
|
||||
'precise-folsom/proposed': 'precise-proposed/folsom',
|
||||
'precise-proposed/folsom': 'precise-proposed/folsom',
|
||||
# Grizzly
|
||||
'grizzly': 'precise-updates/grizzly',
|
||||
'precise-grizzly': 'precise-updates/grizzly',
|
||||
'precise-grizzly/updates': 'precise-updates/grizzly',
|
||||
'precise-updates/grizzly': 'precise-updates/grizzly',
|
||||
'grizzly/proposed': 'precise-proposed/grizzly',
|
||||
'precise-grizzly/proposed': 'precise-proposed/grizzly',
|
||||
'precise-proposed/grizzly': 'precise-proposed/grizzly',
|
||||
# Havana
|
||||
'havana': 'precise-updates/havana',
|
||||
'precise-havana': 'precise-updates/havana',
|
||||
'precise-havana/updates': 'precise-updates/havana',
|
||||
'precise-updates/havana': 'precise-updates/havana',
|
||||
'havana/proposed': 'precise-proposed/havana',
|
||||
'precise-havana/proposed': 'precise-proposed/havana',
|
||||
'precise-proposed/havana': 'precise-proposed/havana',
|
||||
# Icehouse
|
||||
'icehouse': 'precise-updates/icehouse',
|
||||
'precise-icehouse': 'precise-updates/icehouse',
|
||||
'precise-icehouse/updates': 'precise-updates/icehouse',
|
||||
'precise-updates/icehouse': 'precise-updates/icehouse',
|
||||
'icehouse/proposed': 'precise-proposed/icehouse',
|
||||
'precise-icehouse/proposed': 'precise-proposed/icehouse',
|
||||
'precise-proposed/icehouse': 'precise-proposed/icehouse',
|
||||
# Juno
|
||||
'juno': 'trusty-updates/juno',
|
||||
'trusty-juno': 'trusty-updates/juno',
|
||||
'trusty-juno/updates': 'trusty-updates/juno',
|
||||
'trusty-updates/juno': 'trusty-updates/juno',
|
||||
'juno/proposed': 'trusty-proposed/juno',
|
||||
'trusty-juno/proposed': 'trusty-proposed/juno',
|
||||
'trusty-proposed/juno': 'trusty-proposed/juno',
|
||||
# Kilo
|
||||
'kilo': 'trusty-updates/kilo',
|
||||
'trusty-kilo': 'trusty-updates/kilo',
|
||||
'trusty-kilo/updates': 'trusty-updates/kilo',
|
||||
'trusty-updates/kilo': 'trusty-updates/kilo',
|
||||
'kilo/proposed': 'trusty-proposed/kilo',
|
||||
'trusty-kilo/proposed': 'trusty-proposed/kilo',
|
||||
'trusty-proposed/kilo': 'trusty-proposed/kilo',
|
||||
# Liberty
|
||||
'liberty': 'trusty-updates/liberty',
|
||||
'trusty-liberty': 'trusty-updates/liberty',
|
||||
'trusty-liberty/updates': 'trusty-updates/liberty',
|
||||
'trusty-updates/liberty': 'trusty-updates/liberty',
|
||||
'liberty/proposed': 'trusty-proposed/liberty',
|
||||
'trusty-liberty/proposed': 'trusty-proposed/liberty',
|
||||
'trusty-proposed/liberty': 'trusty-proposed/liberty',
|
||||
# Mitaka
|
||||
'mitaka': 'trusty-updates/mitaka',
|
||||
'trusty-mitaka': 'trusty-updates/mitaka',
|
||||
'trusty-mitaka/updates': 'trusty-updates/mitaka',
|
||||
'trusty-updates/mitaka': 'trusty-updates/mitaka',
|
||||
'mitaka/proposed': 'trusty-proposed/mitaka',
|
||||
'trusty-mitaka/proposed': 'trusty-proposed/mitaka',
|
||||
'trusty-proposed/mitaka': 'trusty-proposed/mitaka',
|
||||
}
|
||||
|
||||
# The order of this list is very important. Handlers should be listed in from
|
||||
# least- to most-specific URL matching.
|
||||
FETCH_HANDLERS = (
|
||||
'charmhelpers.fetch.archiveurl.ArchiveUrlFetchHandler',
|
||||
'charmhelpers.fetch.bzrurl.BzrUrlFetchHandler',
|
||||
'charmhelpers.fetch.giturl.GitUrlFetchHandler',
|
||||
)
|
||||
|
||||
APT_NO_LOCK = 100 # The return code for "couldn't acquire lock" in APT.
|
||||
APT_NO_LOCK_RETRY_DELAY = 10 # Wait 10 seconds between apt lock checks.
|
||||
APT_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times.
|
||||
|
||||
|
||||
class SourceConfigError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
class UnhandledSource(Exception):
|
||||
pass
|
||||
|
||||
|
||||
class AptLockError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
class BaseFetchHandler(object):
|
||||
|
||||
"""Base class for FetchHandler implementations in fetch plugins"""
|
||||
|
||||
def can_handle(self, source):
|
||||
"""Returns True if the source can be handled. Otherwise returns
|
||||
a string explaining why it cannot"""
|
||||
return "Wrong source type"
|
||||
|
||||
def install(self, source):
|
||||
"""Try to download and unpack the source. Return the path to the
|
||||
unpacked files or raise UnhandledSource."""
|
||||
raise UnhandledSource("Wrong source type {}".format(source))
|
||||
|
||||
def parse_url(self, url):
|
||||
return urlparse(url)
|
||||
|
||||
def base_url(self, url):
|
||||
"""Return url without querystring or fragment"""
|
||||
parts = list(self.parse_url(url))
|
||||
parts[4:] = ['' for i in parts[4:]]
|
||||
return urlunparse(parts)
|
||||
|
||||
|
||||
def filter_installed_packages(packages):
|
||||
"""Returns a list of packages that require installation"""
|
||||
cache = apt_cache()
|
||||
_pkgs = []
|
||||
for package in packages:
|
||||
try:
|
||||
p = cache[package]
|
||||
p.current_ver or _pkgs.append(package)
|
||||
except KeyError:
|
||||
log('Package {} has no installation candidate.'.format(package),
|
||||
level='WARNING')
|
||||
_pkgs.append(package)
|
||||
return _pkgs
|
||||
|
||||
|
||||
def apt_cache(in_memory=True):
|
||||
"""Build and return an apt cache"""
|
||||
from apt import apt_pkg
|
||||
apt_pkg.init()
|
||||
if in_memory:
|
||||
apt_pkg.config.set("Dir::Cache::pkgcache", "")
|
||||
apt_pkg.config.set("Dir::Cache::srcpkgcache", "")
|
||||
return apt_pkg.Cache()
|
||||
|
||||
|
||||
def apt_install(packages, options=None, fatal=False):
|
||||
"""Install one or more packages"""
|
||||
if options is None:
|
||||
options = ['--option=Dpkg::Options::=--force-confold']
|
||||
|
||||
cmd = ['apt-get', '--assume-yes']
|
||||
cmd.extend(options)
|
||||
cmd.append('install')
|
||||
if isinstance(packages, six.string_types):
|
||||
cmd.append(packages)
|
||||
else:
|
||||
cmd.extend(packages)
|
||||
log("Installing {} with options: {}".format(packages,
|
||||
options))
|
||||
_run_apt_command(cmd, fatal)
|
||||
|
||||
|
||||
def apt_upgrade(options=None, fatal=False, dist=False):
|
||||
"""Upgrade all packages"""
|
||||
if options is None:
|
||||
options = ['--option=Dpkg::Options::=--force-confold']
|
||||
|
||||
cmd = ['apt-get', '--assume-yes']
|
||||
cmd.extend(options)
|
||||
if dist:
|
||||
cmd.append('dist-upgrade')
|
||||
else:
|
||||
cmd.append('upgrade')
|
||||
log("Upgrading with options: {}".format(options))
|
||||
_run_apt_command(cmd, fatal)
|
||||
|
||||
|
||||
def apt_update(fatal=False):
|
||||
"""Update local apt cache"""
|
||||
cmd = ['apt-get', 'update']
|
||||
_run_apt_command(cmd, fatal)
|
||||
|
||||
|
||||
def apt_purge(packages, fatal=False):
|
||||
"""Purge one or more packages"""
|
||||
cmd = ['apt-get', '--assume-yes', 'purge']
|
||||
if isinstance(packages, six.string_types):
|
||||
cmd.append(packages)
|
||||
else:
|
||||
cmd.extend(packages)
|
||||
log("Purging {}".format(packages))
|
||||
_run_apt_command(cmd, fatal)
|
||||
|
||||
|
||||
def apt_mark(packages, mark, fatal=False):
|
||||
"""Flag one or more packages using apt-mark"""
|
||||
log("Marking {} as {}".format(packages, mark))
|
||||
cmd = ['apt-mark', mark]
|
||||
if isinstance(packages, six.string_types):
|
||||
cmd.append(packages)
|
||||
else:
|
||||
cmd.extend(packages)
|
||||
|
||||
if fatal:
|
||||
subprocess.check_call(cmd, universal_newlines=True)
|
||||
else:
|
||||
subprocess.call(cmd, universal_newlines=True)
|
||||
|
||||
|
||||
def apt_hold(packages, fatal=False):
|
||||
return apt_mark(packages, 'hold', fatal=fatal)
|
||||
|
||||
|
||||
def apt_unhold(packages, fatal=False):
|
||||
return apt_mark(packages, 'unhold', fatal=fatal)
|
||||
|
||||
|
||||
def add_source(source, key=None):
|
||||
"""Add a package source to this system.
|
||||
|
||||
@param source: a URL or sources.list entry, as supported by
|
||||
add-apt-repository(1). Examples::
|
||||
|
||||
ppa:charmers/example
|
||||
deb https://stub:key@private.example.com/ubuntu trusty main
|
||||
|
||||
In addition:
|
||||
'proposed:' may be used to enable the standard 'proposed'
|
||||
pocket for the release.
|
||||
'cloud:' may be used to activate official cloud archive pockets,
|
||||
such as 'cloud:icehouse'
|
||||
'distro' may be used as a noop
|
||||
|
||||
@param key: A key to be added to the system's APT keyring and used
|
||||
to verify the signatures on packages. Ideally, this should be an
|
||||
ASCII format GPG public key including the block headers. A GPG key
|
||||
id may also be used, but be aware that only insecure protocols are
|
||||
available to retrieve the actual public key from a public keyserver
|
||||
placing your Juju environment at risk. ppa and cloud archive keys
|
||||
are securely added automtically, so sould not be provided.
|
||||
"""
|
||||
if source is None:
|
||||
log('Source is not present. Skipping')
|
||||
return
|
||||
|
||||
if (source.startswith('ppa:') or
|
||||
source.startswith('http') or
|
||||
source.startswith('deb ') or
|
||||
source.startswith('cloud-archive:')):
|
||||
subprocess.check_call(['add-apt-repository', '--yes', source])
|
||||
elif source.startswith('cloud:'):
|
||||
apt_install(filter_installed_packages(['ubuntu-cloud-keyring']),
|
||||
fatal=True)
|
||||
pocket = source.split(':')[-1]
|
||||
if pocket not in CLOUD_ARCHIVE_POCKETS:
|
||||
raise SourceConfigError(
|
||||
'Unsupported cloud: source option %s' %
|
||||
pocket)
|
||||
actual_pocket = CLOUD_ARCHIVE_POCKETS[pocket]
|
||||
with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as apt:
|
||||
apt.write(CLOUD_ARCHIVE.format(actual_pocket))
|
||||
elif source == 'proposed':
|
||||
release = lsb_release()['DISTRIB_CODENAME']
|
||||
with open('/etc/apt/sources.list.d/proposed.list', 'w') as apt:
|
||||
apt.write(PROPOSED_POCKET.format(release))
|
||||
elif source == 'distro':
|
||||
pass
|
||||
else:
|
||||
log("Unknown source: {!r}".format(source))
|
||||
|
||||
if key:
|
||||
if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key:
|
||||
with NamedTemporaryFile('w+') as key_file:
|
||||
key_file.write(key)
|
||||
key_file.flush()
|
||||
key_file.seek(0)
|
||||
subprocess.check_call(['apt-key', 'add', '-'], stdin=key_file)
|
||||
else:
|
||||
# Note that hkp: is in no way a secure protocol. Using a
|
||||
# GPG key id is pointless from a security POV unless you
|
||||
# absolutely trust your network and DNS.
|
||||
subprocess.check_call(['apt-key', 'adv', '--keyserver',
|
||||
'hkp://keyserver.ubuntu.com:80', '--recv',
|
||||
key])
|
||||
|
||||
|
||||
def configure_sources(update=False,
|
||||
sources_var='install_sources',
|
||||
keys_var='install_keys'):
|
||||
"""
|
||||
Configure multiple sources from charm configuration.
|
||||
|
||||
The lists are encoded as yaml fragments in the configuration.
|
||||
The frament needs to be included as a string. Sources and their
|
||||
corresponding keys are of the types supported by add_source().
|
||||
|
||||
Example config:
|
||||
install_sources: |
|
||||
- "ppa:foo"
|
||||
- "http://example.com/repo precise main"
|
||||
install_keys: |
|
||||
- null
|
||||
- "a1b2c3d4"
|
||||
|
||||
Note that 'null' (a.k.a. None) should not be quoted.
|
||||
"""
|
||||
sources = safe_load((config(sources_var) or '').strip()) or []
|
||||
keys = safe_load((config(keys_var) or '').strip()) or None
|
||||
|
||||
if isinstance(sources, six.string_types):
|
||||
sources = [sources]
|
||||
|
||||
if keys is None:
|
||||
for source in sources:
|
||||
add_source(source, None)
|
||||
else:
|
||||
if isinstance(keys, six.string_types):
|
||||
keys = [keys]
|
||||
|
||||
if len(sources) != len(keys):
|
||||
raise SourceConfigError(
|
||||
'Install sources and keys lists are different lengths')
|
||||
for source, key in zip(sources, keys):
|
||||
add_source(source, key)
|
||||
if update:
|
||||
apt_update(fatal=True)
|
||||
|
||||
|
||||
def install_remote(source, *args, **kwargs):
|
||||
"""
|
||||
Install a file tree from a remote source
|
||||
|
||||
The specified source should be a url of the form:
|
||||
scheme://[host]/path[#[option=value][&...]]
|
||||
|
||||
Schemes supported are based on this modules submodules.
|
||||
Options supported are submodule-specific.
|
||||
Additional arguments are passed through to the submodule.
|
||||
|
||||
For example::
|
||||
|
||||
dest = install_remote('http://example.com/archive.tgz',
|
||||
checksum='deadbeef',
|
||||
hash_type='sha1')
|
||||
|
||||
This will download `archive.tgz`, validate it using SHA1 and, if
|
||||
the file is ok, extract it and return the directory in which it
|
||||
was extracted. If the checksum fails, it will raise
|
||||
:class:`charmhelpers.core.host.ChecksumError`.
|
||||
"""
|
||||
# We ONLY check for True here because can_handle may return a string
|
||||
# explaining why it can't handle a given source.
|
||||
handlers = [h for h in plugins() if h.can_handle(source) is True]
|
||||
installed_to = None
|
||||
for handler in handlers:
|
||||
try:
|
||||
installed_to = handler.install(source, *args, **kwargs)
|
||||
except UnhandledSource as e:
|
||||
log('Install source attempt unsuccessful: {}'.format(e),
|
||||
level='WARNING')
|
||||
if not installed_to:
|
||||
raise UnhandledSource("No handler found for source {}".format(source))
|
||||
return installed_to
|
||||
|
||||
|
||||
def install_from_config(config_var_name):
|
||||
charm_config = config()
|
||||
source = charm_config[config_var_name]
|
||||
return install_remote(source)
|
||||
|
||||
|
||||
def plugins(fetch_handlers=None):
|
||||
if not fetch_handlers:
|
||||
fetch_handlers = FETCH_HANDLERS
|
||||
plugin_list = []
|
||||
for handler_name in fetch_handlers:
|
||||
package, classname = handler_name.rsplit('.', 1)
|
||||
try:
|
||||
handler_class = getattr(
|
||||
importlib.import_module(package),
|
||||
classname)
|
||||
plugin_list.append(handler_class())
|
||||
except NotImplementedError:
|
||||
# Skip missing plugins so that they can be ommitted from
|
||||
# installation if desired
|
||||
log("FetchHandler {} not found, skipping plugin".format(
|
||||
handler_name))
|
||||
return plugin_list
|
||||
|
||||
|
||||
def _run_apt_command(cmd, fatal=False):
|
||||
"""
|
||||
Run an APT command, checking output and retrying if the fatal flag is set
|
||||
to True.
|
||||
|
||||
:param: cmd: str: The apt command to run.
|
||||
:param: fatal: bool: Whether the command's output should be checked and
|
||||
retried.
|
||||
"""
|
||||
env = os.environ.copy()
|
||||
|
||||
if 'DEBIAN_FRONTEND' not in env:
|
||||
env['DEBIAN_FRONTEND'] = 'noninteractive'
|
||||
|
||||
if fatal:
|
||||
retry_count = 0
|
||||
result = None
|
||||
|
||||
# If the command is considered "fatal", we need to retry if the apt
|
||||
# lock was not acquired.
|
||||
|
||||
while result is None or result == APT_NO_LOCK:
|
||||
try:
|
||||
result = subprocess.check_call(cmd, env=env)
|
||||
except subprocess.CalledProcessError as e:
|
||||
retry_count = retry_count + 1
|
||||
if retry_count > APT_NO_LOCK_RETRY_COUNT:
|
||||
raise
|
||||
result = e.returncode
|
||||
log("Couldn't acquire DPKG lock. Will retry in {} seconds."
|
||||
"".format(APT_NO_LOCK_RETRY_DELAY))
|
||||
time.sleep(APT_NO_LOCK_RETRY_DELAY)
|
||||
|
||||
else:
|
||||
subprocess.call(cmd, env=env)
|
|
@ -1,167 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
import os
|
||||
import hashlib
|
||||
import re
|
||||
|
||||
from charmhelpers.fetch import (
|
||||
BaseFetchHandler,
|
||||
UnhandledSource
|
||||
)
|
||||
from charmhelpers.payload.archive import (
|
||||
get_archive_handler,
|
||||
extract,
|
||||
)
|
||||
from charmhelpers.core.host import mkdir, check_hash
|
||||
|
||||
import six
|
||||
if six.PY3:
|
||||
from urllib.request import (
|
||||
build_opener, install_opener, urlopen, urlretrieve,
|
||||
HTTPPasswordMgrWithDefaultRealm, HTTPBasicAuthHandler,
|
||||
)
|
||||
from urllib.parse import urlparse, urlunparse, parse_qs
|
||||
from urllib.error import URLError
|
||||
else:
|
||||
from urllib import urlretrieve
|
||||
from urllib2 import (
|
||||
build_opener, install_opener, urlopen,
|
||||
HTTPPasswordMgrWithDefaultRealm, HTTPBasicAuthHandler,
|
||||
URLError
|
||||
)
|
||||
from urlparse import urlparse, urlunparse, parse_qs
|
||||
|
||||
|
||||
def splituser(host):
|
||||
'''urllib.splituser(), but six's support of this seems broken'''
|
||||
_userprog = re.compile('^(.*)@(.*)$')
|
||||
match = _userprog.match(host)
|
||||
if match:
|
||||
return match.group(1, 2)
|
||||
return None, host
|
||||
|
||||
|
||||
def splitpasswd(user):
|
||||
'''urllib.splitpasswd(), but six's support of this is missing'''
|
||||
_passwdprog = re.compile('^([^:]*):(.*)$', re.S)
|
||||
match = _passwdprog.match(user)
|
||||
if match:
|
||||
return match.group(1, 2)
|
||||
return user, None
|
||||
|
||||
|
||||
class ArchiveUrlFetchHandler(BaseFetchHandler):
|
||||
"""
|
||||
Handler to download archive files from arbitrary URLs.
|
||||
|
||||
Can fetch from http, https, ftp, and file URLs.
|
||||
|
||||
Can install either tarballs (.tar, .tgz, .tbz2, etc) or zip files.
|
||||
|
||||
Installs the contents of the archive in $CHARM_DIR/fetched/.
|
||||
"""
|
||||
def can_handle(self, source):
|
||||
url_parts = self.parse_url(source)
|
||||
if url_parts.scheme not in ('http', 'https', 'ftp', 'file'):
|
||||
# XXX: Why is this returning a boolean and a string? It's
|
||||
# doomed to fail since "bool(can_handle('foo://'))" will be True.
|
||||
return "Wrong source type"
|
||||
if get_archive_handler(self.base_url(source)):
|
||||
return True
|
||||
return False
|
||||
|
||||
def download(self, source, dest):
|
||||
"""
|
||||
Download an archive file.
|
||||
|
||||
:param str source: URL pointing to an archive file.
|
||||
:param str dest: Local path location to download archive file to.
|
||||
"""
|
||||
# propogate all exceptions
|
||||
# URLError, OSError, etc
|
||||
proto, netloc, path, params, query, fragment = urlparse(source)
|
||||
if proto in ('http', 'https'):
|
||||
auth, barehost = splituser(netloc)
|
||||
if auth is not None:
|
||||
source = urlunparse((proto, barehost, path, params, query, fragment))
|
||||
username, password = splitpasswd(auth)
|
||||
passman = HTTPPasswordMgrWithDefaultRealm()
|
||||
# Realm is set to None in add_password to force the username and password
|
||||
# to be used whatever the realm
|
||||
passman.add_password(None, source, username, password)
|
||||
authhandler = HTTPBasicAuthHandler(passman)
|
||||
opener = build_opener(authhandler)
|
||||
install_opener(opener)
|
||||
response = urlopen(source)
|
||||
try:
|
||||
with open(dest, 'wb') as dest_file:
|
||||
dest_file.write(response.read())
|
||||
except Exception as e:
|
||||
if os.path.isfile(dest):
|
||||
os.unlink(dest)
|
||||
raise e
|
||||
|
||||
# Mandatory file validation via Sha1 or MD5 hashing.
|
||||
def download_and_validate(self, url, hashsum, validate="sha1"):
|
||||
tempfile, headers = urlretrieve(url)
|
||||
check_hash(tempfile, hashsum, validate)
|
||||
return tempfile
|
||||
|
||||
def install(self, source, dest=None, checksum=None, hash_type='sha1'):
|
||||
"""
|
||||
Download and install an archive file, with optional checksum validation.
|
||||
|
||||
The checksum can also be given on the `source` URL's fragment.
|
||||
For example::
|
||||
|
||||
handler.install('http://example.com/file.tgz#sha1=deadbeef')
|
||||
|
||||
:param str source: URL pointing to an archive file.
|
||||
:param str dest: Local destination path to install to. If not given,
|
||||
installs to `$CHARM_DIR/archives/archive_file_name`.
|
||||
:param str checksum: If given, validate the archive file after download.
|
||||
:param str hash_type: Algorithm used to generate `checksum`.
|
||||
Can be any hash alrgorithm supported by :mod:`hashlib`,
|
||||
such as md5, sha1, sha256, sha512, etc.
|
||||
|
||||
"""
|
||||
url_parts = self.parse_url(source)
|
||||
dest_dir = os.path.join(os.environ.get('CHARM_DIR'), 'fetched')
|
||||
if not os.path.exists(dest_dir):
|
||||
mkdir(dest_dir, perms=0o755)
|
||||
dld_file = os.path.join(dest_dir, os.path.basename(url_parts.path))
|
||||
try:
|
||||
self.download(source, dld_file)
|
||||
except URLError as e:
|
||||
raise UnhandledSource(e.reason)
|
||||
except OSError as e:
|
||||
raise UnhandledSource(e.strerror)
|
||||
options = parse_qs(url_parts.fragment)
|
||||
for key, value in options.items():
|
||||
if not six.PY3:
|
||||
algorithms = hashlib.algorithms
|
||||
else:
|
||||
algorithms = hashlib.algorithms_available
|
||||
if key in algorithms:
|
||||
if len(value) != 1:
|
||||
raise TypeError(
|
||||
"Expected 1 hash value, not %d" % len(value))
|
||||
expected = value[0]
|
||||
check_hash(dld_file, expected, key)
|
||||
if checksum:
|
||||
check_hash(dld_file, checksum, hash_type)
|
||||
return extract(dld_file, dest)
|
|
@ -1,68 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
import os
|
||||
from subprocess import check_call
|
||||
from charmhelpers.fetch import (
|
||||
BaseFetchHandler,
|
||||
UnhandledSource,
|
||||
filter_installed_packages,
|
||||
apt_install,
|
||||
)
|
||||
from charmhelpers.core.host import mkdir
|
||||
|
||||
|
||||
if filter_installed_packages(['bzr']) != []:
|
||||
apt_install(['bzr'])
|
||||
if filter_installed_packages(['bzr']) != []:
|
||||
raise NotImplementedError('Unable to install bzr')
|
||||
|
||||
|
||||
class BzrUrlFetchHandler(BaseFetchHandler):
|
||||
"""Handler for bazaar branches via generic and lp URLs"""
|
||||
def can_handle(self, source):
|
||||
url_parts = self.parse_url(source)
|
||||
if url_parts.scheme not in ('bzr+ssh', 'lp', ''):
|
||||
return False
|
||||
elif not url_parts.scheme:
|
||||
return os.path.exists(os.path.join(source, '.bzr'))
|
||||
else:
|
||||
return True
|
||||
|
||||
def branch(self, source, dest):
|
||||
if not self.can_handle(source):
|
||||
raise UnhandledSource("Cannot handle {}".format(source))
|
||||
if os.path.exists(dest):
|
||||
check_call(['bzr', 'pull', '--overwrite', '-d', dest, source])
|
||||
else:
|
||||
check_call(['bzr', 'branch', source, dest])
|
||||
|
||||
def install(self, source, dest=None):
|
||||
url_parts = self.parse_url(source)
|
||||
branch_name = url_parts.path.strip("/").split("/")[-1]
|
||||
if dest:
|
||||
dest_dir = os.path.join(dest, branch_name)
|
||||
else:
|
||||
dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched",
|
||||
branch_name)
|
||||
|
||||
if not os.path.exists(dest_dir):
|
||||
mkdir(dest_dir, perms=0o755)
|
||||
try:
|
||||
self.branch(source, dest_dir)
|
||||
except OSError as e:
|
||||
raise UnhandledSource(e.strerror)
|
||||
return dest_dir
|
|
@ -1,70 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
import os
|
||||
from subprocess import check_call, CalledProcessError
|
||||
from charmhelpers.fetch import (
|
||||
BaseFetchHandler,
|
||||
UnhandledSource,
|
||||
filter_installed_packages,
|
||||
apt_install,
|
||||
)
|
||||
|
||||
if filter_installed_packages(['git']) != []:
|
||||
apt_install(['git'])
|
||||
if filter_installed_packages(['git']) != []:
|
||||
raise NotImplementedError('Unable to install git')
|
||||
|
||||
|
||||
class GitUrlFetchHandler(BaseFetchHandler):
|
||||
"""Handler for git branches via generic and github URLs"""
|
||||
def can_handle(self, source):
|
||||
url_parts = self.parse_url(source)
|
||||
# TODO (mattyw) no support for ssh git@ yet
|
||||
if url_parts.scheme not in ('http', 'https', 'git', ''):
|
||||
return False
|
||||
elif not url_parts.scheme:
|
||||
return os.path.exists(os.path.join(source, '.git'))
|
||||
else:
|
||||
return True
|
||||
|
||||
def clone(self, source, dest, branch="master", depth=None):
|
||||
if not self.can_handle(source):
|
||||
raise UnhandledSource("Cannot handle {}".format(source))
|
||||
|
||||
if os.path.exists(dest):
|
||||
cmd = ['git', '-C', dest, 'pull', source, branch]
|
||||
else:
|
||||
cmd = ['git', 'clone', source, dest, '--branch', branch]
|
||||
if depth:
|
||||
cmd.extend(['--depth', depth])
|
||||
check_call(cmd)
|
||||
|
||||
def install(self, source, branch="master", dest=None, depth=None):
|
||||
url_parts = self.parse_url(source)
|
||||
branch_name = url_parts.path.strip("/").split("/")[-1]
|
||||
if dest:
|
||||
dest_dir = os.path.join(dest, branch_name)
|
||||
else:
|
||||
dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched",
|
||||
branch_name)
|
||||
try:
|
||||
self.clone(source, dest_dir, branch, depth)
|
||||
except CalledProcessError as e:
|
||||
raise UnhandledSource(e)
|
||||
except OSError as e:
|
||||
raise UnhandledSource(e.strerror)
|
||||
return dest_dir
|
|
@ -1,17 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
"Tools for working with files injected into a charm just before deployment."
|
|
@ -1,73 +0,0 @@
|
|||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
import os
|
||||
import tarfile
|
||||
import zipfile
|
||||
from charmhelpers.core import (
|
||||
host,
|
||||
hookenv,
|
||||
)
|
||||
|
||||
|
||||
class ArchiveError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def get_archive_handler(archive_name):
|
||||
if os.path.isfile(archive_name):
|
||||
if tarfile.is_tarfile(archive_name):
|
||||
return extract_tarfile
|
||||
elif zipfile.is_zipfile(archive_name):
|
||||
return extract_zipfile
|
||||
else:
|
||||
# look at the file name
|
||||
for ext in ('.tar', '.tar.gz', '.tgz', 'tar.bz2', '.tbz2', '.tbz'):
|
||||
if archive_name.endswith(ext):
|
||||
return extract_tarfile
|
||||
for ext in ('.zip', '.jar'):
|
||||
if archive_name.endswith(ext):
|
||||
return extract_zipfile
|
||||
|
||||
|
||||
def archive_dest_default(archive_name):
|
||||
archive_file = os.path.basename(archive_name)
|
||||
return os.path.join(hookenv.charm_dir(), "archives", archive_file)
|
||||
|
||||
|
||||
def extract(archive_name, destpath=None):
|
||||
handler = get_archive_handler(archive_name)
|
||||
if handler:
|
||||
if not destpath:
|
||||
destpath = archive_dest_default(archive_name)
|
||||
if not os.path.isdir(destpath):
|
||||
host.mkdir(destpath)
|
||||
handler(archive_name, destpath)
|
||||
return destpath
|
||||
else:
|
||||
raise ArchiveError("No handler for archive")
|
||||
|
||||
|
||||
def extract_tarfile(archive_name, destpath):
|
||||
"Unpack a tar archive, optionally compressed"
|
||||
archive = tarfile.open(archive_name)
|
||||
archive.extractall(destpath)
|
||||
|
||||
|
||||
def extract_zipfile(archive_name, destpath):
|
||||
"Unpack a zip file"
|
||||
archive = zipfile.ZipFile(archive_name)
|
||||
archive.extractall(destpath)
|
|
@ -1,66 +0,0 @@
|
|||
#!/usr/bin/env python
|
||||
|
||||
# Copyright 2014-2015 Canonical Limited.
|
||||
#
|
||||
# This file is part of charm-helpers.
|
||||
#
|
||||
# charm-helpers is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# charm-helpers is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
import os
|
||||
import sys
|
||||
import subprocess
|
||||
from charmhelpers.core import hookenv
|
||||
|
||||
|
||||
def default_execd_dir():
|
||||
return os.path.join(os.environ['CHARM_DIR'], 'exec.d')
|
||||
|
||||
|
||||
def execd_module_paths(execd_dir=None):
|
||||
"""Generate a list of full paths to modules within execd_dir."""
|
||||
if not execd_dir:
|
||||
execd_dir = default_execd_dir()
|
||||
|
||||
if not os.path.exists(execd_dir):
|
||||
return
|
||||
|
||||
for subpath in os.listdir(execd_dir):
|
||||
module = os.path.join(execd_dir, subpath)
|
||||
if os.path.isdir(module):
|
||||
yield module
|
||||
|
||||
|
||||
def execd_submodule_paths(command, execd_dir=None):
|
||||
"""Generate a list of full paths to the specified command within exec_dir.
|
||||
"""
|
||||
for module_path in execd_module_paths(execd_dir):
|
||||
path = os.path.join(module_path, command)
|
||||
if os.access(path, os.X_OK) and os.path.isfile(path):
|
||||
yield path
|
||||
|
||||
|
||||
def execd_run(command, execd_dir=None, die_on_error=False, stderr=None):
|
||||
"""Run command for each module within execd_dir which defines it."""
|
||||
for submodule_path in execd_submodule_paths(command, execd_dir):
|
||||
try:
|
||||
subprocess.check_call(submodule_path, shell=True, stderr=stderr)
|
||||
except subprocess.CalledProcessError as e:
|
||||
hookenv.log("Error ({}) running {}. Output: {}".format(
|
||||
e.returncode, e.cmd, e.output))
|
||||
if die_on_error:
|
||||
sys.exit(e.returncode)
|
||||
|
||||
|
||||
def execd_preinstall(execd_dir=None):
|
||||
"""Run charm-pre-install for each module within execd_dir."""
|
||||
execd_run('charm-pre-install', execd_dir=execd_dir)
|
|
@ -1 +0,0 @@
|
|||
pg_dir_hooks.py
|
|
@ -1 +0,0 @@
|
|||
pg_dir_hooks.py
|
|
@ -1 +0,0 @@
|
|||
pg_dir_hooks.py
|
|
@ -1,128 +0,0 @@
|
|||
# Copyright (c) 2015, PLUMgrid Inc, http://plumgrid.com
|
||||
|
||||
# This file contains the class that generates context
|
||||
# for PLUMgrid template files.
|
||||
|
||||
import re
|
||||
from charmhelpers.contrib.openstack import context
|
||||
from charmhelpers.contrib.openstack.utils import get_host_ip
|
||||
from charmhelpers.core.hookenv import (
|
||||
config,
|
||||
unit_get,
|
||||
)
|
||||
from charmhelpers.core.hookenv import (
|
||||
relation_ids,
|
||||
related_units,
|
||||
relation_get,
|
||||
)
|
||||
from charmhelpers.contrib.network.ip import (
|
||||
is_ip,
|
||||
get_address_in_network,
|
||||
)
|
||||
|
||||
from socket import (
|
||||
gethostname,
|
||||
getfqdn
|
||||
)
|
||||
|
||||
|
||||
def _pg_edge_ips():
|
||||
'''
|
||||
Inspects edge-peer relation and returns the
|
||||
ips of the edge nodes
|
||||
'''
|
||||
return [get_host_ip(rdata['private-address'])
|
||||
for rid in relation_ids("plumgrid")
|
||||
for rdata in
|
||||
(relation_get(rid=rid, unit=unit) for unit in related_units(rid))
|
||||
if 'edge-peer' in rdata]
|
||||
|
||||
|
||||
def _pg_gateway_ips():
|
||||
'''
|
||||
Inspects gateway-peer relation and returns the
|
||||
ips of the gateway nodes
|
||||
'''
|
||||
return [get_host_ip(rdata['private-address'])
|
||||
for rid in relation_ids("plumgrid")
|
||||
for rdata in
|
||||
(relation_get(rid=rid, unit=unit) for unit in related_units(rid))
|
||||
if 'gateway-peer' in rdata]
|
||||
|
||||
|
||||
def _pg_dir_ips():
|
||||
'''
|
||||
Inspects plumgrid-director peer relation and returns the
|
||||
ips of the peer directors
|
||||
'''
|
||||
return [get_host_ip(rdata['private-address'])
|
||||
for rid in relation_ids("director")
|
||||
for rdata in
|
||||
(relation_get(rid=rid, unit=unit) for unit in related_units(rid))
|
||||
if rdata]
|
||||
|
||||
|
||||
class PGDirContext(context.NeutronContext):
|
||||
|
||||
@property
|
||||
def plugin(self):
|
||||
'''
|
||||
Over-riding function in NeutronContext Class to return 'plumgrid'
|
||||
as the neutron plugin.
|
||||
'''
|
||||
return 'plumgrid'
|
||||
|
||||
@property
|
||||
def network_manager(self):
|
||||
'''
|
||||
Over-riding function in NeutronContext Class to return 'neutron'
|
||||
as the network manager.
|
||||
'''
|
||||
return 'neutron'
|
||||
|
||||
def _save_flag_file(self):
|
||||
'''
|
||||
Over-riding function in NeutronContext Class.
|
||||
Function only needed for OVS.
|
||||
'''
|
||||
pass
|
||||
|
||||
def pg_ctxt(self):
|
||||
'''
|
||||
Generated Config for all PLUMgrid templates inside the templates
|
||||
folder.
|
||||
'''
|
||||
pg_ctxt = super(PGDirContext, self).pg_ctxt()
|
||||
if not pg_ctxt:
|
||||
return {}
|
||||
|
||||
conf = config()
|
||||
pg_dir_ips = _pg_dir_ips()
|
||||
pg_dir_ips.append(str(get_address_in_network(network=None,
|
||||
fallback=get_host_ip(unit_get('private-address')))))
|
||||
pg_dir_ips = sorted(pg_dir_ips)
|
||||
pg_ctxt['director_ips'] = pg_dir_ips
|
||||
dir_count = len(pg_dir_ips)
|
||||
pg_ctxt['director_ips_string'] = (str(pg_dir_ips[0]) + ',' +
|
||||
str(pg_dir_ips[1]) + ',' +
|
||||
str(pg_dir_ips[2])
|
||||
if dir_count == 3 else
|
||||
str(pg_dir_ips[0]))
|
||||
PG_VIP = conf['plumgrid-virtual-ip']
|
||||
if is_ip(PG_VIP):
|
||||
pg_ctxt['virtual_ip'] = PG_VIP
|
||||
else:
|
||||
raise ValueError('Invalid PLUMgrid Virtual IP Provided')
|
||||
unit_hostname = gethostname()
|
||||
pg_ctxt['pg_hostname'] = unit_hostname
|
||||
pg_ctxt['pg_fqdn'] = getfqdn()
|
||||
from pg_dir_utils import get_mgmt_interface, get_fabric_interface
|
||||
pg_ctxt['interface'] = get_mgmt_interface()
|
||||
pg_ctxt['fabric_interface'] = get_fabric_interface()
|
||||
pg_ctxt['label'] = unit_hostname
|
||||
pg_ctxt['fabric_mode'] = 'host'
|
||||
virtual_ip_array = re.split('\.', conf['plumgrid-virtual-ip'])
|
||||
pg_ctxt['virtual_router_id'] = virtual_ip_array[3]
|
||||
pg_ctxt['opsvm_ip'] = conf['opsvm-ip']
|
||||
|
||||
return pg_ctxt
|
|
@ -1,231 +0,0 @@
|
|||
#!/usr/bin/python
|
||||
|
||||
# Copyright (c) 2015, PLUMgrid Inc, http://plumgrid.com
|
||||
|
||||
# The hooks of this charm have been symlinked to functions
|
||||
# in this file.
|
||||
|
||||
import sys
|
||||
import time
|
||||
from charmhelpers.core.host import service_running
|
||||
from charmhelpers.contrib.network.ip import is_ip
|
||||
from charmhelpers.core.hookenv import (
|
||||
Hooks,
|
||||
UnregisteredHookError,
|
||||
log,
|
||||
config,
|
||||
relation_set,
|
||||
relation_ids,
|
||||
status_set,
|
||||
is_leader
|
||||
)
|
||||
|
||||
from charmhelpers.fetch import (
|
||||
apt_install,
|
||||
configure_sources,
|
||||
)
|
||||
|
||||
from pg_dir_utils import (
|
||||
register_configs,
|
||||
restart_pg,
|
||||
restart_map,
|
||||
stop_pg,
|
||||
determine_packages,
|
||||
load_iovisor,
|
||||
remove_iovisor,
|
||||
ensure_mtu,
|
||||
add_lcm_key,
|
||||
post_pg_license,
|
||||
fabric_interface_changed,
|
||||
load_iptables,
|
||||
restart_on_change,
|
||||
director_cluster_ready,
|
||||
configure_pg_sources,
|
||||
configure_analyst_opsvm,
|
||||
sapi_post_ips,
|
||||
sapi_post_license,
|
||||
sapi_post_zone_info
|
||||
)
|
||||
|
||||
hooks = Hooks()
|
||||
CONFIGS = register_configs()
|
||||
|
||||
|
||||
@hooks.hook()
|
||||
def install():
|
||||
'''
|
||||
Install hook is run when the charm is first deployed on a node.
|
||||
'''
|
||||
status_set('maintenance', 'Executing pre-install')
|
||||
load_iptables()
|
||||
configure_sources(update=True)
|
||||
status_set('maintenance', 'Installing apt packages')
|
||||
pkgs = determine_packages()
|
||||
for pkg in pkgs:
|
||||
apt_install(pkg, options=['--force-yes'], fatal=True)
|
||||
load_iovisor()
|
||||
ensure_mtu()
|
||||
CONFIGS.write_all()
|
||||
|
||||
|
||||
@hooks.hook('director-relation-joined')
|
||||
@hooks.hook('director-relation-changed')
|
||||
@restart_on_change(restart_map())
|
||||
def dir_joined():
|
||||
'''
|
||||
This hook is run when a unit of director is added.
|
||||
'''
|
||||
if director_cluster_ready():
|
||||
ensure_mtu()
|
||||
CONFIGS.write_all()
|
||||
|
||||
|
||||
@hooks.hook('plumgrid-relation-joined',
|
||||
'plumgrid-relation-changed',
|
||||
'plumgrid-relation-departed')
|
||||
def plumgrid_joined(relation_id=None):
|
||||
'''
|
||||
This hook is run when relation with edge or gateway is created.
|
||||
'''
|
||||
opsvm_ip = config('opsvm-ip')
|
||||
if not is_ip(opsvm_ip):
|
||||
raise ValueError('Invalid OPSVM IP specified!')
|
||||
else:
|
||||
relation_set(relation_id=relation_id, opsvm_ip=opsvm_ip)
|
||||
if is_leader():
|
||||
sapi_post_ips()
|
||||
|
||||
|
||||
@hooks.hook('plumgrid-configs-relation-joined')
|
||||
def plumgrid_configs_joined(relation_id=None):
|
||||
'''
|
||||
This hook is run when relation with neutron-api-plumgrid is created.
|
||||
'''
|
||||
relation_settings = {
|
||||
'plumgrid_virtual_ip': config('plumgrid-virtual-ip'),
|
||||
'plumgrid_username': config('plumgrid-username'),
|
||||
'plumgrid_password': config('plumgrid-password'),
|
||||
}
|
||||
if is_leader():
|
||||
relation_set(relation_id=relation_id,
|
||||
relation_settings=relation_settings)
|
||||
|
||||
|
||||
@hooks.hook('config-changed')
|
||||
def config_changed():
|
||||
'''
|
||||
This hook is run when a config parameter is changed.
|
||||
It also runs on node reboot.
|
||||
'''
|
||||
charm_config = config()
|
||||
if charm_config.changed('lcm-ssh-key'):
|
||||
if add_lcm_key():
|
||||
log("PLUMgrid LCM Key added")
|
||||
if charm_config.changed('plumgrid-license-key'):
|
||||
if is_leader() and post_pg_license():
|
||||
log("PLUMgrid License Posted")
|
||||
# Post PG license to Sol-API
|
||||
sapi_post_license()
|
||||
if charm_config.changed('fabric-interfaces'):
|
||||
if not fabric_interface_changed():
|
||||
log("Fabric interface already set")
|
||||
else:
|
||||
stop_pg()
|
||||
if charm_config.changed('plumgrid-virtual-ip'):
|
||||
CONFIGS.write_all()
|
||||
for rid in relation_ids('plumgrid'):
|
||||
plumgrid_joined(rid)
|
||||
stop_pg()
|
||||
for rid in relation_ids('plumgrid-configs'):
|
||||
plumgrid_configs_joined(rid)
|
||||
if (charm_config.changed('plumgrid-username') or
|
||||
charm_config.changed('plumgrid-password')):
|
||||
for rid in relation_ids('plumgrid-configs'):
|
||||
plumgrid_configs_joined(rid)
|
||||
if (charm_config.changed('install_sources') or
|
||||
charm_config.changed('plumgrid-build') or
|
||||
charm_config.changed('install_keys') or
|
||||
charm_config.changed('iovisor-build')):
|
||||
status_set('maintenance', 'Upgrading apt packages')
|
||||
stop_pg()
|
||||
if charm_config.changed('install_sources'):
|
||||
configure_pg_sources()
|
||||
configure_sources(update=True)
|
||||
pkgs = determine_packages()
|
||||
for pkg in pkgs:
|
||||
apt_install(pkg, options=['--force-yes'], fatal=True)
|
||||
remove_iovisor()
|
||||
load_iovisor()
|
||||
if charm_config.changed('opsvm-ip'):
|
||||
for rid in relation_ids('plumgrid'):
|
||||
plumgrid_joined(rid)
|
||||
stop_pg()
|
||||
if (charm_config.changed('sapi-port') or
|
||||
charm_config.changed('lcm-ip') or
|
||||
charm_config.changed('sapi-zone')):
|
||||
if is_leader():
|
||||
if is_ip(config('lcm-ip')):
|
||||
sapi_post_zone_info()
|
||||
else:
|
||||
raise ValueError('Invalid LCM IP specified!')
|
||||
for rid in relation_ids('plumgrid'):
|
||||
plumgrid_joined(rid)
|
||||
ensure_mtu()
|
||||
CONFIGS.write_all()
|
||||
if not service_running('plumgrid'):
|
||||
restart_pg()
|
||||
|
||||
|
||||
@hooks.hook('start')
|
||||
def start():
|
||||
'''
|
||||
This hook is run when the charm is started.
|
||||
'''
|
||||
configure_analyst_opsvm()
|
||||
if config('plumgrid-license-key') is not None:
|
||||
count = 0
|
||||
while (count < 10):
|
||||
if post_pg_license():
|
||||
break
|
||||
count += 1
|
||||
time.sleep(15)
|
||||
if count == 10:
|
||||
raise ValueError("Error occurred while posting plumgrid license"
|
||||
"key. Please check plumgrid services.")
|
||||
|
||||
|
||||
@hooks.hook('upgrade-charm')
|
||||
@restart_on_change(restart_map())
|
||||
def upgrade_charm():
|
||||
'''
|
||||
This hook is run when the charm is upgraded
|
||||
'''
|
||||
ensure_mtu()
|
||||
CONFIGS.write_all()
|
||||
|
||||
|
||||
@hooks.hook('stop')
|
||||
def stop():
|
||||
'''
|
||||
This hook is run when the charm is destroyed.
|
||||
'''
|
||||
stop_pg()
|
||||
|
||||
|
||||
@hooks.hook('update-status')
|
||||
def update_status():
|
||||
if service_running('plumgrid'):
|
||||
status_set('active', 'Unit is ready')
|
||||
else:
|
||||
status_set('blocked', 'plumgrid service not running')
|
||||
|
||||
|
||||
def main():
|
||||
try:
|
||||
hooks.execute(sys.argv)
|
||||
except UnregisteredHookError as e:
|
||||
log('Unknown hook {} - skipping.'.format(e))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
|
@ -1,637 +0,0 @@
|
|||
# Copyright (c) 2015, PLUMgrid Inc, http://plumgrid.com
|
||||
|
||||
# This file contains functions used by the hooks to deploy PLUMgrid Director.
|
||||
|
||||
import pg_dir_context
|
||||
import subprocess
|
||||
import time
|
||||
import os
|
||||
import json
|
||||
from collections import OrderedDict
|
||||
from socket import gethostname as get_unit_hostname
|
||||
from copy import deepcopy
|
||||
from charmhelpers.contrib.openstack.neutron import neutron_plugin_attribute
|
||||
from charmhelpers.contrib.openstack import templating
|
||||
from charmhelpers.contrib.storage.linux.ceph import modprobe
|
||||
from charmhelpers.core.hookenv import (
|
||||
log,
|
||||
config,
|
||||
unit_get,
|
||||
status_set
|
||||
)
|
||||
from charmhelpers.contrib.network.ip import (
|
||||
get_iface_from_addr,
|
||||
get_bridges,
|
||||
get_bridge_nics,
|
||||
is_ip,
|
||||
get_iface_addr,
|
||||
get_host_ip
|
||||
)
|
||||
from charmhelpers.core.host import (
|
||||
service_start,
|
||||
service_stop,
|
||||
service_running,
|
||||
path_hash,
|
||||
set_nic_mtu
|
||||
)
|
||||
from charmhelpers.fetch import (
|
||||
apt_cache,
|
||||
apt_install
|
||||
)
|
||||
from charmhelpers.contrib.openstack.utils import (
|
||||
os_release,
|
||||
)
|
||||
from pg_dir_context import (
|
||||
_pg_dir_ips,
|
||||
_pg_edge_ips,
|
||||
_pg_gateway_ips
|
||||
)
|
||||
|
||||
SOURCES_LIST = '/etc/apt/sources.list'
|
||||
LXC_CONF = '/etc/libvirt/lxc.conf'
|
||||
TEMPLATES = 'templates/'
|
||||
PG_LXC_DATA_PATH = '/var/lib/libvirt/filesystems/plumgrid-data'
|
||||
PG_LXC_PATH = '/var/lib/libvirt/filesystems/plumgrid'
|
||||
PG_CONF = '%s/conf/pg/plumgrid.conf' % PG_LXC_DATA_PATH
|
||||
PG_KA_CONF = '%s/conf/etc/keepalived.conf' % PG_LXC_DATA_PATH
|
||||
PG_DEF_CONF = '%s/conf/pg/nginx.conf' % PG_LXC_DATA_PATH
|
||||
PG_HN_CONF = '%s/conf/etc/hostname' % PG_LXC_DATA_PATH
|
||||
PG_HS_CONF = '%s/conf/etc/hosts' % PG_LXC_DATA_PATH
|
||||
PG_IFCS_CONF = '%s/conf/pg/ifcs.conf' % PG_LXC_DATA_PATH
|
||||
OPS_CONF = '%s/conf/etc/00-pg.conf' % PG_LXC_DATA_PATH
|
||||
AUTH_KEY_PATH = '%s/root/.ssh/authorized_keys' % PG_LXC_DATA_PATH
|
||||
TEMP_LICENSE_FILE = '/tmp/license'
|
||||
|
||||
# Constant values for OpenStack releases as Canonical-Ubuntu
|
||||
# doesn't have any specific solution version associated
|
||||
OPENSTACK_RELEASE_VERS = {
|
||||
'kilo': '10',
|
||||
'liberty': '11',
|
||||
'mitaka': '12'
|
||||
}
|
||||
|
||||
BASE_RESOURCE_MAP = OrderedDict([
|
||||
(PG_KA_CONF, {
|
||||
'services': ['plumgrid'],
|
||||
'contexts': [pg_dir_context.PGDirContext()],
|
||||
}),
|
||||
(PG_CONF, {
|
||||
'services': ['plumgrid'],
|
||||
'contexts': [pg_dir_context.PGDirContext()],
|
||||
}),
|
||||
(PG_DEF_CONF, {
|
||||
'services': ['plumgrid'],
|
||||
'contexts': [pg_dir_context.PGDirContext()],
|
||||
}),
|
||||
(PG_HN_CONF, {
|
||||
'services': ['plumgrid'],
|
||||
'contexts': [pg_dir_context.PGDirContext()],
|
||||
}),
|
||||
(PG_HS_CONF, {
|
||||
'services': ['plumgrid'],
|
||||
'contexts': [pg_dir_context.PGDirContext()],
|
||||
}),
|
||||
(OPS_CONF, {
|
||||
'services': ['plumgrid'],
|
||||
'contexts': [pg_dir_context.PGDirContext()],
|
||||
}),
|
||||
(PG_IFCS_CONF, {
|
||||
'services': [],
|
||||
'contexts': [pg_dir_context.PGDirContext()],
|
||||
}),
|
||||
])
|
||||
|
||||
|
||||
def configure_pg_sources():
|
||||
'''
|
||||
Returns true if install sources is updated in sources.list file
|
||||
'''
|
||||
try:
|
||||
with open(SOURCES_LIST, 'r+') as sources:
|
||||
all_lines = sources.readlines()
|
||||
sources.seek(0)
|
||||
for i in (line for line in all_lines if "plumgrid" not in line):
|
||||
sources.write(i)
|
||||
sources.truncate()
|
||||
sources.close()
|
||||
except IOError:
|
||||
log('Unable to update /etc/apt/sources.list')
|
||||
|
||||
|
||||
def configure_analyst_opsvm():
|
||||
'''
|
||||
Configures Anaylyst for OPSVM
|
||||
'''
|
||||
if not service_running('plumgrid'):
|
||||
restart_pg()
|
||||
NS_ENTER = ('/opt/local/bin/nsenter -t $(ps ho pid --ppid $(cat '
|
||||
'/var/run/libvirt/lxc/plumgrid.pid)) -m -n -u -i -p ')
|
||||
sigmund_stop = NS_ENTER + '/usr/bin/service plumgrid-sigmund stop'
|
||||
sigmund_status = NS_ENTER \
|
||||
+ '/usr/bin/service plumgrid-sigmund status'
|
||||
sigmund_autoboot = NS_ENTER \
|
||||
+ '/usr/bin/sigmund-configure --ip {0} --start --autoboot' \
|
||||
.format(config('opsvm-ip'))
|
||||
try:
|
||||
status = subprocess.check_output(sigmund_status, shell=True)
|
||||
if 'start/running' in status:
|
||||
if subprocess.call(sigmund_stop, shell=True):
|
||||
log('plumgrid-sigmund couldn\'t be stopped!')
|
||||
return
|
||||
subprocess.check_call(sigmund_autoboot, shell=True)
|
||||
except:
|
||||
log('plumgrid-sigmund couldn\'t be started!')
|
||||
|
||||
|
||||
def determine_packages():
|
||||
'''
|
||||
Returns list of packages required by PLUMgrid director as specified
|
||||
in the neutron_plugins dictionary in charmhelpers.
|
||||
'''
|
||||
pkgs = []
|
||||
tag = 'latest'
|
||||
for pkg in neutron_plugin_attribute('plumgrid', 'packages', 'neutron'):
|
||||
if 'plumgrid' in pkg:
|
||||
tag = config('plumgrid-build')
|
||||
elif pkg == 'iovisor-dkms':
|
||||
tag = config('iovisor-build')
|
||||
|
||||
if tag == 'latest':
|
||||
pkgs.append(pkg)
|
||||
else:
|
||||
if tag in [i.ver_str for i in apt_cache()[pkg].version_list]:
|
||||
pkgs.append('%s=%s' % (pkg, tag))
|
||||
else:
|
||||
error_msg = \
|
||||
"Build version '%s' for package '%s' not available" \
|
||||
% (tag, pkg)
|
||||
raise ValueError(error_msg)
|
||||
return pkgs
|
||||
|
||||
|
||||
def register_configs(release=None):
|
||||
'''
|
||||
Returns an object of the Openstack Tempating Class which contains the
|
||||
the context required for all templates of this charm.
|
||||
'''
|
||||
release = release or os_release('neutron-common', base='kilo')
|
||||
configs = templating.OSConfigRenderer(templates_dir=TEMPLATES,
|
||||
openstack_release=release)
|
||||
for cfg, rscs in resource_map().iteritems():
|
||||
configs.register(cfg, rscs['contexts'])
|
||||
return configs
|
||||
|
||||
|
||||
def resource_map():
|
||||
'''
|
||||
Dynamically generate a map of resources that will be managed for a single
|
||||
hook execution.
|
||||
'''
|
||||
resource_map = deepcopy(BASE_RESOURCE_MAP)
|
||||
return resource_map
|
||||
|
||||
|
||||
def restart_map():
|
||||
'''
|
||||
Constructs a restart map based on charm config settings and relation
|
||||
state.
|
||||
'''
|
||||
return {cfg: rscs['services'] for cfg, rscs in resource_map().iteritems()}
|
||||
|
||||
|
||||
def restart_pg():
|
||||
'''
|
||||
Stops and Starts PLUMgrid service after flushing iptables.
|
||||
'''
|
||||
stop_pg()
|
||||
service_start('plumgrid')
|
||||
time.sleep(3)
|
||||
if not service_running('plumgrid'):
|
||||
if service_running('libvirt-bin'):
|
||||
raise ValueError("plumgrid service couldn't be started")
|
||||
else:
|
||||
if service_start('libvirt-bin'):
|
||||
time.sleep(8)
|
||||
if not service_running('plumgrid') \
|
||||
and not service_start('plumgrid'):
|
||||
raise ValueError("plumgrid service couldn't be started")
|
||||
else:
|
||||
raise ValueError("libvirt-bin service couldn't be started")
|
||||
status_set('active', 'Unit is ready')
|
||||
|
||||
|
||||
def stop_pg():
|
||||
'''
|
||||
Stops PLUMgrid service.
|
||||
'''
|
||||
service_stop('plumgrid')
|
||||
time.sleep(2)
|
||||
|
||||
|
||||
def load_iovisor():
|
||||
'''
|
||||
Loads iovisor kernel module.
|
||||
'''
|
||||
modprobe('iovisor')
|
||||
|
||||
|
||||
def remove_iovisor():
|
||||
'''
|
||||
Removes iovisor kernel module.
|
||||
'''
|
||||
_exec_cmd(cmd=['rmmod', 'iovisor'],
|
||||
error_msg='Error Removing IOVisor Kernel Module')
|
||||
time.sleep(1)
|
||||
|
||||
|
||||
def interface_exists(interface):
|
||||
'''
|
||||
Checks if interface exists on node.
|
||||
'''
|
||||
try:
|
||||
subprocess.check_call(['ip', 'link', 'show', interface],
|
||||
stdout=open(os.devnull, 'w'),
|
||||
stderr=subprocess.STDOUT)
|
||||
except subprocess.CalledProcessError:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def get_mgmt_interface():
|
||||
'''
|
||||
Returns the managment interface.
|
||||
'''
|
||||
mgmt_interface = config('mgmt-interface')
|
||||
if not mgmt_interface:
|
||||
try:
|
||||
return get_iface_from_addr(unit_get('private-address'))
|
||||
except:
|
||||
for bridge_interface in get_bridges():
|
||||
if (get_host_ip(unit_get('private-address'))
|
||||
in get_iface_addr(bridge_interface)):
|
||||
return bridge_interface
|
||||
elif interface_exists(mgmt_interface):
|
||||
return mgmt_interface
|
||||
else:
|
||||
log('Provided managment interface %s does not exist'
|
||||
% mgmt_interface)
|
||||
return get_iface_from_addr(unit_get('private-address'))
|
||||
|
||||
|
||||
def fabric_interface_changed():
|
||||
'''
|
||||
Returns true if interface for node changed.
|
||||
'''
|
||||
fabric_interface = get_fabric_interface()
|
||||
try:
|
||||
with open(PG_IFCS_CONF, 'r') as ifcs:
|
||||
for line in ifcs:
|
||||
if 'fabric_core' in line:
|
||||
if line.split()[0] == fabric_interface:
|
||||
return False
|
||||
except IOError:
|
||||
return True
|
||||
return True
|
||||
|
||||
|
||||
def get_fabric_interface():
|
||||
'''
|
||||
Returns the fabric interface.
|
||||
'''
|
||||
fabric_interfaces = config('fabric-interfaces')
|
||||
if fabric_interfaces == 'MANAGEMENT':
|
||||
return get_mgmt_interface()
|
||||
else:
|
||||
try:
|
||||
all_fabric_interfaces = json.loads(fabric_interfaces)
|
||||
except ValueError:
|
||||
raise ValueError('Invalid json provided for fabric interfaces')
|
||||
hostname = get_unit_hostname()
|
||||
if hostname in all_fabric_interfaces:
|
||||
node_fabric_interface = all_fabric_interfaces[hostname]
|
||||
elif 'DEFAULT' in all_fabric_interfaces:
|
||||
node_fabric_interface = all_fabric_interfaces['DEFAULT']
|
||||
else:
|
||||
raise ValueError('No fabric interface provided for node')
|
||||
if interface_exists(node_fabric_interface):
|
||||
return node_fabric_interface
|
||||
else:
|
||||
log('Provided fabric interface %s does not exist'
|
||||
% node_fabric_interface)
|
||||
raise ValueError('Provided fabric interface does not exist')
|
||||
return node_fabric_interface
|
||||
|
||||
|
||||
def ensure_mtu():
|
||||
'''
|
||||
Ensures required MTU of the underlying networking of the node.
|
||||
'''
|
||||
interface_mtu = config('network-device-mtu')
|
||||
fabric_interface = get_fabric_interface()
|
||||
if fabric_interface in get_bridges():
|
||||
attached_interfaces = get_bridge_nics(fabric_interface)
|
||||
for interface in attached_interfaces:
|
||||
set_nic_mtu(interface, interface_mtu)
|
||||
set_nic_mtu(fabric_interface, interface_mtu)
|
||||
|
||||
|
||||
def _exec_cmd(cmd=None, error_msg='Command exited with ERRORs', fatal=False):
|
||||
'''
|
||||
Function to execute any bash command on the node.
|
||||
'''
|
||||
if cmd is None:
|
||||
log("No command specified")
|
||||
else:
|
||||
if fatal:
|
||||
subprocess.check_call(cmd)
|
||||
else:
|
||||
try:
|
||||
subprocess.check_call(cmd)
|
||||
except subprocess.CalledProcessError:
|
||||
log(error_msg)
|
||||
|
||||
|
||||
def add_lcm_key():
|
||||
'''
|
||||
Adds public key of PLUMgrid-lcm to authorized keys of PLUMgrid Director.
|
||||
'''
|
||||
key = config('lcm-ssh-key')
|
||||
if key == 'null':
|
||||
log('lcm key not specified')
|
||||
return 0
|
||||
file_write_type = 'w+'
|
||||
if os.path.isfile(AUTH_KEY_PATH):
|
||||
file_write_type = 'a'
|
||||
try:
|
||||
fr = open(AUTH_KEY_PATH, 'r')
|
||||
except IOError:
|
||||
log('plumgrid-lxc not installed yet')
|
||||
return 0
|
||||
for line in fr:
|
||||
if key in line:
|
||||
log('key already added')
|
||||
return 0
|
||||
try:
|
||||
fa = open(AUTH_KEY_PATH, file_write_type)
|
||||
except IOError:
|
||||
log('Error opening file to append')
|
||||
return 0
|
||||
fa.write(key)
|
||||
fa.write('\n')
|
||||
fa.close()
|
||||
return 1
|
||||
|
||||
|
||||
def post_pg_license():
|
||||
'''
|
||||
Posts PLUMgrid License if it hasnt been posted already.
|
||||
'''
|
||||
key = config('plumgrid-license-key')
|
||||
if key is None:
|
||||
log('PLUMgrid License Key not specified')
|
||||
return 0
|
||||
PG_VIP = config('plumgrid-virtual-ip')
|
||||
if not is_ip(PG_VIP):
|
||||
raise ValueError('Invalid IP Provided')
|
||||
LICENSE_POST_PATH = 'https://%s/0/tenant_manager/license_key' % PG_VIP
|
||||
LICENSE_GET_PATH = 'https://%s/0/tenant_manager/licenses' % PG_VIP
|
||||
PG_CURL = '%s/opt/pg/scripts/pg_curl.sh' % PG_LXC_PATH
|
||||
license = {"key1": {"license": key}}
|
||||
licence_post_cmd = [
|
||||
PG_CURL,
|
||||
'-u',
|
||||
'plumgrid:plumgrid',
|
||||
LICENSE_POST_PATH,
|
||||
'-d',
|
||||
json.dumps(license)]
|
||||
licence_get_cmd = [PG_CURL, '-u', 'plumgrid:plumgrid', LICENSE_GET_PATH]
|
||||
try:
|
||||
old_license = subprocess.check_output(licence_get_cmd)
|
||||
except subprocess.CalledProcessError:
|
||||
log('No response from specified virtual IP')
|
||||
return 0
|
||||
_exec_cmd(cmd=licence_post_cmd,
|
||||
error_msg='Unable to post License', fatal=False)
|
||||
new_license = subprocess.check_output(licence_get_cmd)
|
||||
if old_license == new_license:
|
||||
log('No change in PLUMgrid License')
|
||||
return 0
|
||||
return 1
|
||||
|
||||
|
||||
def sapi_post_ips():
|
||||
"""
|
||||
Posts PLUMgrid nodes IPs to solutions api server.
|
||||
"""
|
||||
pg_edge_ips = _pg_edge_ips()
|
||||
pg_dir_ips = _pg_dir_ips()
|
||||
pg_gateway_ips = _pg_gateway_ips()
|
||||
pg_dir_ips.append(get_host_ip(unit_get('private-address')))
|
||||
pg_edge_ips = '"edge_ips"' + ':' \
|
||||
+ '"{}"'.format(','.join(str(i) for i in pg_edge_ips))
|
||||
pg_dir_ips = '"director_ips"' + ':' \
|
||||
+ '"{}"'.format(','.join(str(i) for i in pg_dir_ips))
|
||||
pg_gateway_ips = '"gateway_ips"' + ':' \
|
||||
+ '"{}"'.format(','.join(str(i) for i in pg_gateway_ips))
|
||||
opsvm_ip = '"opsvm_ip"' + ':' + '"{}"'.format(config('opsvm-ip'))
|
||||
virtual_ip = '"virtual_ip"' + ':' \
|
||||
+ '"{}"'.format(config('plumgrid-virtual-ip'))
|
||||
JSON_IPS = ','.join([pg_dir_ips, pg_edge_ips, pg_gateway_ips,
|
||||
opsvm_ip, virtual_ip])
|
||||
status = (
|
||||
'curl -H \'Content-Type: application/json\' -X '
|
||||
'PUT -d \'{{{0}}}\' http://{1}' + ':' + '{2}/v1/zones/{3}/allIps'
|
||||
).format(JSON_IPS, config('lcm-ip'), config('sapi-port'),
|
||||
config('sapi-zone'))
|
||||
POST_ZONE_IPs = _exec_cmd_output(
|
||||
status,
|
||||
'Posting Zone IPs to Solutions API server failed!')
|
||||
if POST_ZONE_IPs:
|
||||
if 'success' in POST_ZONE_IPs:
|
||||
log('Successfully posted Zone IPs to Solutions API server!')
|
||||
log(POST_ZONE_IPs)
|
||||
|
||||
|
||||
def _exec_cmd_output(cmd=None, error_msg='Command exited with ERRORs',
|
||||
fatal=False):
|
||||
'''
|
||||
Function to get output from bash command executed on the node.
|
||||
'''
|
||||
if cmd is None:
|
||||
log("No command specified")
|
||||
else:
|
||||
if fatal:
|
||||
return subprocess.check_output(cmd, shell=True)
|
||||
else:
|
||||
try:
|
||||
return subprocess.check_output(cmd, shell=True)
|
||||
except subprocess.CalledProcessError:
|
||||
log(error_msg)
|
||||
return None
|
||||
|
||||
|
||||
def sapi_post_license():
|
||||
'''
|
||||
Posts PLUMgrid License to solutions api server
|
||||
'''
|
||||
username = '"user_name":' + '"{}"'.format(config('plumgrid-username'))
|
||||
password = '"password":' + '"{}"'.format(config('plumgrid-password'))
|
||||
license = '"license":' + '"{}"'.format(config('plumgrid-license-key'))
|
||||
JSON_LICENSE = ','.join([username, password, license])
|
||||
status = (
|
||||
'curl -H \'Content-Type: application/json\' -X '
|
||||
'PUT -d \'{{{0}}}\' http://{1}' + ':' + '{2}/v1/zones/{3}/pgLicense'
|
||||
).format(JSON_LICENSE, config('lcm-ip'), config('sapi-port'),
|
||||
config('sapi-zone'))
|
||||
POST_LICENSE = _exec_cmd_output(
|
||||
status,
|
||||
'Posting PLUMgrid License to Solutions API server failed!')
|
||||
if POST_LICENSE:
|
||||
if 'success' in POST_LICENSE:
|
||||
log('Successfully posted license file for zone "{}"!'
|
||||
.format(config('sapi-zone')))
|
||||
log(POST_LICENSE)
|
||||
|
||||
|
||||
def sapi_post_zone_info():
|
||||
'''
|
||||
Posts zone information to solutions api server
|
||||
'''
|
||||
sol_name = '"solution_name":"Ubuntu OpenStack"'
|
||||
release = config('openstack-release')
|
||||
for key, value in OPENSTACK_RELEASE_VERS.iteritems():
|
||||
if release == value:
|
||||
sol_version = value
|
||||
else:
|
||||
sol_version = 10
|
||||
sol_version = '"solution_version":"{}"'.format(sol_version)
|
||||
pg_ons_version = _exec_cmd_output(
|
||||
'dpkg -l | grep plumgrid | awk \'{print $3}\' | '
|
||||
'sed \'s/-/./\' | cut -f1 -d"-"',
|
||||
'Unable to obtain PG ONS version'
|
||||
).replace('\n', '')
|
||||
pg_ons_version = \
|
||||
'"pg_ons_version":"{}"'.format(pg_ons_version)
|
||||
hypervisor = '"hypervisor":"Ubuntu"'
|
||||
hypervisor_version = \
|
||||
_exec_cmd_output('lsb_release -r | awk \'{print $2}\'',
|
||||
'Unable to obtain solution version'
|
||||
).replace('\n', '')
|
||||
hypervisor_version = '"hypervisor_version":"{}"' \
|
||||
.format(hypervisor_version)
|
||||
kernel_version = _exec_cmd_output(
|
||||
'uname -r',
|
||||
'Unable to obtain kernal version').replace('\n', '')
|
||||
kernel_version = \
|
||||
'"kernel_version":"{}"'.format(kernel_version)
|
||||
cloudapex_path = '/var/lib/libvirt/filesystems/plumgrid/' \
|
||||
'opt/pg/web/cloudApex/modules/appCloudApex' \
|
||||
'/appCloudApex.js'
|
||||
if os.path.isfile(cloudapex_path):
|
||||
pg_cloudapex_version = 'cat ' \
|
||||
+ '{}'.format(cloudapex_path) \
|
||||
+ ' | grep -i appversion | awk \'{print $2}\''
|
||||
pg_cloudapex_version = \
|
||||
_exec_cmd_output(pg_cloudapex_version,
|
||||
'Unable to retrieve CloudApex version'
|
||||
).replace('\n', '')
|
||||
else:
|
||||
log('CloudApex not installed!')
|
||||
pg_cloudapex_version = ''
|
||||
pg_cloudapex_version = \
|
||||
'"pg_cloudapex_version":"{}"'.format(pg_cloudapex_version)
|
||||
JSON_ZONE_INFO = ','.join([
|
||||
sol_name,
|
||||
sol_version,
|
||||
pg_ons_version,
|
||||
hypervisor,
|
||||
hypervisor_version,
|
||||
kernel_version,
|
||||
pg_cloudapex_version,
|
||||
])
|
||||
status = (
|
||||
'curl -H \'Content-Type: application/json\' -X '
|
||||
'PUT -d \'{{{0}}}\' http://{1}:{2}/v1/zones/{3}/zoneinfo'
|
||||
).format(JSON_ZONE_INFO, config('lcm-ip'), config('sapi-port'),
|
||||
config('sapi-zone'))
|
||||
POST_ZONE_INFO = _exec_cmd_output(
|
||||
status,
|
||||
'Posting Zone Information to Solutions API server failed!')
|
||||
if POST_ZONE_INFO:
|
||||
if 'success' in POST_ZONE_INFO:
|
||||
log('Successfully posted Zone information to Solutions API'
|
||||
' server!')
|
||||
log(POST_ZONE_INFO)
|
||||
|
||||
|
||||
def load_iptables():
|
||||
'''
|
||||
Loads iptables rules to allow all PLUMgrid communication.
|
||||
'''
|
||||
network = get_cidr_from_iface(get_mgmt_interface())
|
||||
if network:
|
||||
_exec_cmd(['sudo', 'iptables', '-A', 'INPUT', '-p', 'tcp',
|
||||
'-j', 'ACCEPT', '-s', network, '-d',
|
||||
network, '-m', 'state', '--state', 'NEW'])
|
||||
_exec_cmd(['sudo', 'iptables', '-A', 'INPUT', '-p', 'udp', '-j',
|
||||
'ACCEPT', '-s', network, '-d', network,
|
||||
'-m', 'state', '--state', 'NEW'])
|
||||
_exec_cmd(['sudo', 'iptables', '-I', 'INPUT', '-s', network,
|
||||
'-d', '224.0.0.18/32', '-j', 'ACCEPT'])
|
||||
_exec_cmd(['sudo', 'iptables', '-I', 'INPUT', '-p', 'vrrp', '-j',
|
||||
'ACCEPT'])
|
||||
_exec_cmd(['sudo', 'iptables', '-A', 'INPUT', '-p', 'tcp', '-j',
|
||||
'ACCEPT', '-d', config('plumgrid-virtual-ip'), '-m',
|
||||
'state', '--state', 'NEW'])
|
||||
apt_install('iptables-persistent')
|
||||
|
||||
|
||||
def get_cidr_from_iface(interface):
|
||||
'''
|
||||
Determines Network CIDR from interface.
|
||||
'''
|
||||
if not interface:
|
||||
return None
|
||||
apt_install('ohai')
|
||||
try:
|
||||
os_info = subprocess.check_output(['ohai', '-l', 'fatal'])
|
||||
except OSError:
|
||||
log('Unable to get operating system information')
|
||||
return None
|
||||
try:
|
||||
os_info_json = json.loads(os_info)
|
||||
except ValueError:
|
||||
log('Unable to determine network')
|
||||
return None
|
||||
device = os_info_json['network']['interfaces'].get(interface)
|
||||
if device is not None:
|
||||
if device.get('routes'):
|
||||
routes = device['routes']
|
||||
for net in routes:
|
||||
if 'scope' in net:
|
||||
return net.get('destination')
|
||||
else:
|
||||
return None
|
||||
else:
|
||||
return None
|
||||
|
||||
|
||||
def director_cluster_ready():
|
||||
dirs_count = len(pg_dir_context._pg_dir_ips())
|
||||
return True if dirs_count == 2 else False
|
||||
|
||||
|
||||
def restart_on_change(restart_map):
|
||||
"""
|
||||
Restart services based on configuration files changing
|
||||
"""
|
||||
def wrap(f):
|
||||
def wrapped_f(*args, **kwargs):
|
||||
checksums = {path: path_hash(path) for path in restart_map}
|
||||
f(*args, **kwargs)
|
||||
for path in restart_map:
|
||||
if path_hash(path) != checksums[path]:
|
||||
restart_pg()
|
||||
break
|
||||
return wrapped_f
|
||||
return wrap
|
|
@ -1 +0,0 @@
|
|||
pg_dir_hooks.py
|
|
@ -1 +0,0 @@
|
|||
pg_dir_hooks.py
|
|
@ -1 +0,0 @@
|
|||
pg_dir_hooks.py
|
|
@ -1 +0,0 @@
|
|||
pg_dir_hooks.py
|
|
@ -1 +0,0 @@
|
|||
pg_dir_hooks.py
|
|
@ -1 +0,0 @@
|
|||
pg_dir_hooks.py
|
|
@ -1 +0,0 @@
|
|||
pg_dir_hooks.py
|
|
@ -1 +0,0 @@
|
|||
pg_dir_hooks.py
|
384
icon.svg
384
icon.svg
|
@ -1,384 +0,0 @@
|
|||
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
|
||||
<!-- Created with Inkscape (http://www.inkscape.org/) -->
|
||||
|
||||
<svg
|
||||
xmlns:dc="http://purl.org/dc/elements/1.1/"
|
||||
xmlns:cc="http://creativecommons.org/ns#"
|
||||
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
|
||||
xmlns:svg="http://www.w3.org/2000/svg"
|
||||
xmlns="http://www.w3.org/2000/svg"
|
||||
xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
|
||||
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
|
||||
width="96"
|
||||
height="96"
|
||||
id="svg6517"
|
||||
version="1.1"
|
||||
inkscape:version="0.91 r13725"
|
||||
sodipodi:docname="Director2.svg">
|
||||
<defs
|
||||
id="defs6519">
|
||||
<linearGradient
|
||||
id="Background">
|
||||
<stop
|
||||
id="stop4178"
|
||||
offset="0"
|
||||
style="stop-color:#b8b8b8;stop-opacity:1" />
|
||||
<stop
|
||||
id="stop4180"
|
||||
offset="1"
|
||||
style="stop-color:#c9c9c9;stop-opacity:1" />
|
||||
</linearGradient>
|
||||
<filter
|
||||
style="color-interpolation-filters:sRGB;"
|
||||
inkscape:label="Inner Shadow"
|
||||
id="filter1121">
|
||||
<feFlood
|
||||
flood-opacity="0.59999999999999998"
|
||||
flood-color="rgb(0,0,0)"
|
||||
result="flood"
|
||||
id="feFlood1123" />
|
||||
<feComposite
|
||||
in="flood"
|
||||
in2="SourceGraphic"
|
||||
operator="out"
|
||||
result="composite1"
|
||||
id="feComposite1125" />
|
||||
<feGaussianBlur
|
||||
in="composite1"
|
||||
stdDeviation="1"
|
||||
result="blur"
|
||||
id="feGaussianBlur1127" />
|
||||
<feOffset
|
||||
dx="0"
|
||||
dy="2"
|
||||
result="offset"
|
||||
id="feOffset1129" />
|
||||
<feComposite
|
||||
in="offset"
|
||||
in2="SourceGraphic"
|
||||
operator="atop"
|
||||
result="composite2"
|
||||
id="feComposite1131" />
|
||||
</filter>
|
||||
<filter
|
||||
style="color-interpolation-filters:sRGB;"
|
||||
inkscape:label="Drop Shadow"
|
||||
id="filter950">
|
||||
<feFlood
|
||||
flood-opacity="0.25"
|
||||
flood-color="rgb(0,0,0)"
|
||||
result="flood"
|
||||
id="feFlood952" />
|
||||
<feComposite
|
||||
in="flood"
|
||||
in2="SourceGraphic"
|
||||
operator="in"
|
||||
result="composite1"
|
||||
id="feComposite954" />
|
||||
<feGaussianBlur
|
||||
in="composite1"
|
||||
stdDeviation="1"
|
||||
result="blur"
|
||||
id="feGaussianBlur956" />
|
||||
<feOffset
|
||||
dx="0"
|
||||
dy="1"
|
||||
result="offset"
|
||||
id="feOffset958" />
|
||||
<feComposite
|
||||
in="SourceGraphic"
|
||||
in2="offset"
|
||||
operator="over"
|
||||
result="composite2"
|
||||
id="feComposite960" />
|
||||
</filter>
|
||||
<clipPath
|
||||
clipPathUnits="userSpaceOnUse"
|
||||
id="clipPath873">
|
||||
<g
|
||||
transform="matrix(0,-0.66666667,0.66604479,0,-258.25992,677.00001)"
|
||||
id="g875"
|
||||
inkscape:label="Layer 1"
|
||||
style="fill:#ff00ff;fill-opacity:1;stroke:none;display:inline">
|
||||
<path
|
||||
style="fill:#ff00ff;fill-opacity:1;stroke:none;display:inline"
|
||||
d="m 46.702703,898.22775 50.594594,0 C 138.16216,898.22775 144,904.06497 144,944.92583 l 0,50.73846 c 0,40.86071 -5.83784,46.69791 -46.702703,46.69791 l -50.594594,0 C 5.8378378,1042.3622 0,1036.525 0,995.66429 L 0,944.92583 C 0,904.06497 5.8378378,898.22775 46.702703,898.22775 Z"
|
||||
id="path877"
|
||||
inkscape:connector-curvature="0"
|
||||
sodipodi:nodetypes="sssssssss" />
|
||||
</g>
|
||||
</clipPath>
|
||||
<filter
|
||||
inkscape:collect="always"
|
||||
id="filter891"
|
||||
inkscape:label="Badge Shadow">
|
||||
<feGaussianBlur
|
||||
inkscape:collect="always"
|
||||
stdDeviation="0.71999962"
|
||||
id="feGaussianBlur893" />
|
||||
</filter>
|
||||
</defs>
|
||||
<sodipodi:namedview
|
||||
id="base"
|
||||
pagecolor="#ffffff"
|
||||
bordercolor="#666666"
|
||||
borderopacity="1.0"
|
||||
inkscape:pageopacity="0.0"
|
||||
inkscape:pageshadow="2"
|
||||
inkscape:zoom="4.0745362"
|
||||
inkscape:cx="57.131043"
|
||||
inkscape:cy="49.018169"
|
||||
inkscape:document-units="px"
|
||||
inkscape:current-layer="layer3"
|
||||
showgrid="true"
|
||||
fit-margin-top="0"
|
||||
fit-margin-left="0"
|
||||
fit-margin-right="0"
|
||||
fit-margin-bottom="0"
|
||||
inkscape:window-width="1366"
|
||||
inkscape:window-height="705"
|
||||
inkscape:window-x="-8"
|
||||
inkscape:window-y="-8"
|
||||
inkscape:window-maximized="1"
|
||||
showborder="true"
|
||||
showguides="true"
|
||||
inkscape:guide-bbox="true"
|
||||
inkscape:showpageshadow="false">
|
||||
<inkscape:grid
|
||||
type="xygrid"
|
||||
id="grid821" />
|
||||
<sodipodi:guide
|
||||
orientation="1,0"
|
||||
position="16,48"
|
||||
id="guide823" />
|
||||
<sodipodi:guide
|
||||
orientation="0,1"
|
||||
position="64,80"
|
||||
id="guide825" />
|
||||
<sodipodi:guide
|
||||
orientation="1,0"
|
||||
position="80,40"
|
||||
id="guide827" />
|
||||
<sodipodi:guide
|
||||
orientation="0,1"
|
||||
position="64,16"
|
||||
id="guide829" />
|
||||
</sodipodi:namedview>
|
||||
<metadata
|
||||
id="metadata6522">
|
||||
<rdf:RDF>
|
||||
<cc:Work
|
||||
rdf:about="">
|
||||
<dc:format>image/svg+xml</dc:format>
|
||||
<dc:type
|
||||
rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
|
||||
<dc:title></dc:title>
|
||||
</cc:Work>
|
||||
</rdf:RDF>
|
||||
</metadata>
|
||||
<g
|
||||
inkscape:label="BACKGROUND"
|
||||
inkscape:groupmode="layer"
|
||||
id="layer1"
|
||||
transform="translate(268,-635.29076)"
|
||||
style="display:inline">
|
||||
<path
|
||||
style="fill:#029bd6;fill-opacity:0.90980393;stroke:none;display:inline;filter:url(#filter1121)"
|
||||
d="m -268,700.15563 0,-33.72973 c 0,-27.24324 3.88785,-31.13513 31.10302,-31.13513 l 33.79408,0 c 27.21507,0 31.1029,3.89189 31.1029,31.13513 l 0,33.72973 c 0,27.24325 -3.88783,31.13514 -31.1029,31.13514 l -33.79408,0 C -264.11215,731.29077 -268,727.39888 -268,700.15563 Z"
|
||||
id="path6455"
|
||||
inkscape:connector-curvature="0"
|
||||
sodipodi:nodetypes="sssssssss" />
|
||||
</g>
|
||||
<g
|
||||
inkscape:groupmode="layer"
|
||||
id="layer3"
|
||||
inkscape:label="PLACE YOUR PICTOGRAM HERE"
|
||||
style="display:inline">
|
||||
<rect
|
||||
x="20.77989"
|
||||
y="42.958199"
|
||||
stroke-miterlimit="10"
|
||||
width="56.401024"
|
||||
height="34.449623"
|
||||
id="rect3"
|
||||
style="fill:#ffffff;stroke:#ffffff;stroke-width:0.98441201;stroke-miterlimit:10" />
|
||||
<rect
|
||||
x="-60.724186"
|
||||
y="-46.685535"
|
||||
transform="matrix(-0.9431282,0.33242924,-0.35177172,-0.93608582,0,0)"
|
||||
stroke-miterlimit="10"
|
||||
width="56.193024"
|
||||
height="6.9126363"
|
||||
id="rect5"
|
||||
style="fill:#ffffff;stroke:#ffffff;stroke-width:0.98451632;stroke-miterlimit:10" />
|
||||
<polygon
|
||||
points="68.439,17.669 65.151,18.86 65.151,25.262 68.439,24.068 "
|
||||
id="polygon7"
|
||||
transform="matrix(1.0003019,0,0,0.96877454,1.2930698,0.9773256)"
|
||||
style="fill:#039cd7" />
|
||||
<polygon
|
||||
points="60.926,20.408 57.643,21.602 57.643,27.998 60.926,26.807 "
|
||||
id="polygon9"
|
||||
transform="matrix(1.0003019,0,0,0.96877454,1.2930698,0.9773256)"
|
||||
style="fill:#039cd7" />
|
||||
<polygon
|
||||
points="53.417,23.146 50.134,24.337 50.134,30.739 53.417,29.548 "
|
||||
id="polygon11"
|
||||
transform="matrix(1.0003019,0,0,0.96877454,1.2930698,0.9773256)"
|
||||
style="fill:#039cd7" />
|
||||
<polygon
|
||||
points="45.908,25.888 42.625,27.082 42.625,33.48 45.908,32.287 "
|
||||
id="polygon13"
|
||||
transform="matrix(1.0003019,0,0,0.96877454,1.2930698,0.9773256)"
|
||||
style="fill:#039cd7" />
|
||||
<polygon
|
||||
points="38.399,28.626 35.117,29.82 35.117,36.221 38.399,35.027 "
|
||||
id="polygon15"
|
||||
transform="matrix(1.0003019,0,0,0.96877454,1.2930698,0.9773256)"
|
||||
style="fill:#039cd7" />
|
||||
<polygon
|
||||
points="30.891,31.367 27.603,32.558 27.603,38.959 30.891,37.766 "
|
||||
id="polygon17"
|
||||
transform="matrix(1.0003019,0,0,0.96877454,1.2930698,0.9773256)"
|
||||
style="fill:#039cd7" />
|
||||
<polygon
|
||||
points="23.376,34.105 20.094,35.299 20.094,41.699 23.376,40.507 "
|
||||
id="polygon19"
|
||||
transform="matrix(1.0003019,0,0,0.96877454,1.2930698,0.9773256)"
|
||||
style="fill:#039cd7" />
|
||||
<polygon
|
||||
points="68.677,44.413 72.171,44.407 74.362,50.421 70.866,50.423 "
|
||||
id="polygon21"
|
||||
transform="matrix(1.0003019,0,0,0.96877454,1.2930698,0.9773256)"
|
||||
style="fill:#039cd7" />
|
||||
<polygon
|
||||
points="20.95,44.413 24.439,44.407 26.629,50.421 23.14,50.423 "
|
||||
id="polygon23"
|
||||
transform="matrix(1.0003019,0,0,0.96877454,1.2930698,0.9773256)"
|
||||
style="fill:#039cd7" />
|
||||
<polygon
|
||||
points="28.903,44.413 32.393,44.407 34.583,50.421 31.093,50.423 "
|
||||
id="polygon25"
|
||||
transform="matrix(1.0003019,0,0,0.96877454,1.2930698,0.9773256)"
|
||||
style="fill:#039cd7" />
|
||||
<polygon
|
||||
points="36.857,44.413 40.352,44.407 42.542,50.421 39.047,50.423 "
|
||||
id="polygon27"
|
||||
transform="matrix(1.0003019,0,0,0.96877454,1.2930698,0.9773256)"
|
||||
style="fill:#039cd7" />
|
||||
<polygon
|
||||
points="44.815,44.413 48.305,44.407 50.49,50.421 47,50.423 "
|
||||
id="polygon29"
|
||||
transform="matrix(1.0003019,0,0,0.96877454,1.2930698,0.9773256)"
|
||||
style="fill:#039cd7" />
|
||||
<polygon
|
||||
points="52.77,44.413 56.259,44.407 58.449,50.421 54.959,50.423 "
|
||||
id="polygon31"
|
||||
transform="matrix(1.0003019,0,0,0.96877454,1.2930698,0.9773256)"
|
||||
style="fill:#039cd7" />
|
||||
<polygon
|
||||
points="60.723,44.413 64.218,44.407 66.402,50.421 62.913,50.423 "
|
||||
id="polygon33"
|
||||
transform="matrix(1.0003019,0,0,0.96877454,1.2930698,0.9773256)"
|
||||
style="fill:#039cd7" />
|
||||
<g
|
||||
id="g35"
|
||||
transform="matrix(1.0003019,0,0,0.96877454,1.2930698,0.9773256)">
|
||||
<line
|
||||
stroke-miterlimit="10"
|
||||
x1="23.035"
|
||||
y1="56.598"
|
||||
x2="72.101997"
|
||||
y2="56.598"
|
||||
id="line37"
|
||||
style="fill:#ffffff;stroke:#039cd7;stroke-miterlimit:10" />
|
||||
<line
|
||||
stroke-miterlimit="10"
|
||||
x1="23.035"
|
||||
y1="64.891998"
|
||||
x2="72.101997"
|
||||
y2="64.891998"
|
||||
id="line39"
|
||||
style="fill:#ffffff;stroke:#039cd7;stroke-miterlimit:10" />
|
||||
<line
|
||||
stroke-miterlimit="10"
|
||||
x1="23.035"
|
||||
y1="73.188004"
|
||||
x2="72.101997"
|
||||
y2="73.188004"
|
||||
id="line41"
|
||||
style="fill:#ffffff;stroke:#039cd7;stroke-miterlimit:10" />
|
||||
</g>
|
||||
</g>
|
||||
<g
|
||||
inkscape:groupmode="layer"
|
||||
id="layer2"
|
||||
inkscape:label="BADGE"
|
||||
style="display:none"
|
||||
sodipodi:insensitive="true">
|
||||
<g
|
||||
style="display:inline"
|
||||
transform="translate(-340.00001,-581)"
|
||||
id="g4394"
|
||||
clip-path="none">
|
||||
<g
|
||||
id="g855">
|
||||
<g
|
||||
inkscape:groupmode="maskhelper"
|
||||
id="g870"
|
||||
clip-path="url(#clipPath873)"
|
||||
style="opacity:0.6;filter:url(#filter891)">
|
||||
<path
|
||||
transform="matrix(1.4999992,0,0,1.4999992,-29.999795,-237.54282)"
|
||||
d="m 264,552.36218 a 12,12 0 0 1 -12,12 12,12 0 0 1 -12,-12 12,12 0 0 1 12,-12 12,12 0 0 1 12,12 z"
|
||||
sodipodi:ry="12"
|
||||
sodipodi:rx="12"
|
||||
sodipodi:cy="552.36218"
|
||||
sodipodi:cx="252"
|
||||
id="path844"
|
||||
style="color:#000000;fill:#000000;fill-opacity:1;fill-rule:nonzero;stroke:none;stroke-width:4;marker:none;visibility:visible;display:inline;overflow:visible;enable-background:accumulate"
|
||||
sodipodi:type="arc" />
|
||||
</g>
|
||||
<g
|
||||
id="g862">
|
||||
<path
|
||||
sodipodi:type="arc"
|
||||
style="color:#000000;fill:#f5f5f5;fill-opacity:1;fill-rule:nonzero;stroke:none;stroke-width:4;marker:none;visibility:visible;display:inline;overflow:visible;enable-background:accumulate"
|
||||
id="path4398"
|
||||
sodipodi:cx="252"
|
||||
sodipodi:cy="552.36218"
|
||||
sodipodi:rx="12"
|
||||
sodipodi:ry="12"
|
||||
d="m 264,552.36218 a 12,12 0 0 1 -12,12 12,12 0 0 1 -12,-12 12,12 0 0 1 12,-12 12,12 0 0 1 12,12 z"
|
||||
transform="matrix(1.4999992,0,0,1.4999992,-29.999795,-238.54282)" />
|
||||
<path
|
||||
transform="matrix(1.25,0,0,1.25,33,-100.45273)"
|
||||
d="m 264,552.36218 a 12,12 0 0 1 -12,12 12,12 0 0 1 -12,-12 12,12 0 0 1 12,-12 12,12 0 0 1 12,12 z"
|
||||
sodipodi:ry="12"
|
||||
sodipodi:rx="12"
|
||||
sodipodi:cy="552.36218"
|
||||
sodipodi:cx="252"
|
||||
id="path4400"
|
||||
style="color:#000000;fill:#dd4814;fill-opacity:1;fill-rule:nonzero;stroke:none;stroke-width:4;marker:none;visibility:visible;display:inline;overflow:visible;enable-background:accumulate"
|
||||
sodipodi:type="arc" />
|
||||
<path
|
||||
sodipodi:type="star"
|
||||
style="color:#000000;fill:#f5f5f5;fill-opacity:1;fill-rule:nonzero;stroke:none;stroke-width:3;marker:none;visibility:visible;display:inline;overflow:visible;enable-background:accumulate"
|
||||
id="path4459"
|
||||
sodipodi:sides="5"
|
||||
sodipodi:cx="666.19574"
|
||||
sodipodi:cy="589.50385"
|
||||
sodipodi:r1="7.2431178"
|
||||
sodipodi:r2="4.3458705"
|
||||
sodipodi:arg1="1.0471976"
|
||||
sodipodi:arg2="1.6755161"
|
||||
inkscape:flatsided="false"
|
||||
inkscape:rounded="0.1"
|
||||
inkscape:randomized="0"
|
||||
d="m 669.8173,595.77657 c -0.39132,0.22593 -3.62645,-1.90343 -4.07583,-1.95066 -0.44938,-0.0472 -4.05653,1.36297 -4.39232,1.06062 -0.3358,-0.30235 0.68963,-4.03715 0.59569,-4.47913 -0.0939,-0.44198 -2.5498,-3.43681 -2.36602,-3.8496 0.18379,-0.41279 4.05267,-0.59166 4.44398,-0.81759 0.39132,-0.22593 2.48067,-3.48704 2.93005,-3.4398 0.44938,0.0472 1.81505,3.67147 2.15084,3.97382 0.3358,0.30236 4.08294,1.2817 4.17689,1.72369 0.0939,0.44198 -2.9309,2.86076 -3.11469,3.27355 -0.18379,0.41279 0.0427,4.27917 -0.34859,4.5051 z"
|
||||
transform="matrix(1.511423,-0.16366377,0.16366377,1.511423,-755.37346,-191.93651)" />
|
||||
</g>
|
||||
</g>
|
||||
</g>
|
||||
</g>
|
||||
</svg>
|
Before Width: | Height: | Size: 14 KiB |
|
@ -1,23 +0,0 @@
|
|||
name: plumgrid-director
|
||||
subordinate: false
|
||||
maintainer: Bilal Baqar <bbaqar@plumgrid.com>
|
||||
summary: "PLUMgrid IO Visor Director"
|
||||
description: |
|
||||
The PLUMgrid Director is the control plane of PLUMgrid ONS for OpenStack.
|
||||
The configuration of the virtual network infrastructure for tenants is
|
||||
done through the PLUMgrid Director. The PLUMgrid Director is typically
|
||||
co-located on the OpenStack controller nodes.
|
||||
tags:
|
||||
- openstack
|
||||
requires:
|
||||
plumgrid-plugin:
|
||||
interface: plumgrid-plugin
|
||||
provides:
|
||||
plumgrid:
|
||||
interface: plumgrid
|
||||
plumgrid-configs:
|
||||
interface: plumgrid-configs
|
||||
peers:
|
||||
director:
|
||||
interface: director
|
||||
|
|
@ -1,5 +0,0 @@
|
|||
[nosetests]
|
||||
verbosity=1
|
||||
with-coverage=1
|
||||
cover-erase=1
|
||||
cover-package=hooks
|
|
@ -1,3 +0,0 @@
|
|||
$template ls_json,"{{'{'}}{{'%'}}timestamp:::date-rfc3339,jsonf:@timestamp%,%source:::jsonf:@source_host%,%msg:::json%}"
|
||||
if $syslogtag == 'pg:' and not ($msg contains 'dht_cli_') then @{{ opsvm_ip }}:6000;ls_json
|
||||
:msg, contains, "VM Interface Stats" ~
|
|
@ -1,2 +0,0 @@
|
|||
{{ pg_hostname }}
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
127.0.0.1 localhost
|
||||
127.0.1.1 {{ pg_fqdn }} {{ pg_hostname }}
|
||||
|
||||
# The following lines are desirable for IPv6 capable hosts
|
||||
::1 ip6-localhost ip6-loopback
|
||||
fe00::0 ip6-localnet
|
||||
ff00::0 ip6-mcastprefix
|
||||
ff02::1 ip6-allnodes
|
||||
ff02::2 ip6-allrouters
|
||||
|
|
@ -1,2 +0,0 @@
|
|||
{{ fabric_interface }} = fabric_core host
|
||||
|
|
@ -1,31 +0,0 @@
|
|||
global_defs {
|
||||
router_id {{ pg_hostname }}
|
||||
}
|
||||
|
||||
vrrp_script chk_nginx {
|
||||
script "killall -0 nginx"
|
||||
interval 2
|
||||
}
|
||||
|
||||
vrrp_instance nos {
|
||||
virtual_router_id {{ virtual_router_id }}
|
||||
|
||||
# for electing MASTER, highest priority wins.
|
||||
priority 100
|
||||
state BACKUP
|
||||
nopreempt
|
||||
|
||||
interface {{ interface }}
|
||||
|
||||
virtual_ipaddress {
|
||||
{{ virtual_ip }} dev {{ interface }} label {{ interface }}:1
|
||||
}
|
||||
track_script {
|
||||
chk_nginx
|
||||
}
|
||||
authentication {
|
||||
auth_type PASS
|
||||
auth_pass keepaliv
|
||||
}
|
||||
}
|
||||
|
|
@ -1,161 +0,0 @@
|
|||
upstream sal {
|
||||
server unix:/opt/pg/tmp/sal-web.socket;
|
||||
keepalive 16;
|
||||
}
|
||||
|
||||
upstream websocket {
|
||||
server unix:/opt/pg/tmp/sal-ws.socket;
|
||||
keepalive 16;
|
||||
}
|
||||
|
||||
upstream pgCli {
|
||||
server {{ virtual_ip }}:3000;
|
||||
}
|
||||
|
||||
upstream pgMW {
|
||||
server {{ opsvm_ip }}:4000;
|
||||
}
|
||||
|
||||
map $http_upgrade $connection_upgrade {
|
||||
default upgrade;
|
||||
'' close;
|
||||
}
|
||||
|
||||
lua_socket_log_errors off;
|
||||
#lua_code_cache off;
|
||||
lua_shared_dict rest_servers 16K;
|
||||
lua_shared_dict apache_servers 16K;
|
||||
lua_shared_dict tc_servers 16K;
|
||||
init_by_lua 'lb = require "lb"
|
||||
init_servers = {
|
||||
{% for ip in director_ips -%}
|
||||
["{{ ip }}"] = true,
|
||||
{% endfor -%}
|
||||
}';
|
||||
|
||||
# Redirect http to https
|
||||
server {
|
||||
listen {{ virtual_ip }}:9080;
|
||||
server_name $hostname;
|
||||
return 301 https://$host$request_uri;
|
||||
}
|
||||
|
||||
server {
|
||||
listen {{ virtual_ip }}:443 ssl;
|
||||
ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2;
|
||||
ssl_ciphers AES128-SHA:AES256-SHA:RC4-SHA:DES-CBC3-SHA:RC4-MD5;
|
||||
ssl_certificate /opt/pg/sal/nginx/ssl/default.crt;
|
||||
ssl_certificate_key /opt/pg/sal/nginx/ssl/default.key;
|
||||
#ssl_session_cache shared:SSL:10m;
|
||||
#ssl_session_timeout 10m;
|
||||
|
||||
server_name $hostname;
|
||||
root /opt/pg/web;
|
||||
index login.html;
|
||||
|
||||
location /cli/ {
|
||||
proxy_pass http://pgCli/;
|
||||
proxy_redirect off;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "upgrade";
|
||||
proxy_set_header Host $host;
|
||||
}
|
||||
|
||||
location /mwv0 {
|
||||
proxy_pass http://pgMW;
|
||||
proxy_redirect off;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "upgrade";
|
||||
proxy_set_header Host $host;
|
||||
}
|
||||
|
||||
location ~ /cloudApex {
|
||||
index index.html;
|
||||
}
|
||||
|
||||
location ~* /cloudapex {
|
||||
rewrite (?i)/cloudapex(.*)$ /cloudApex$1 last;
|
||||
}
|
||||
|
||||
location /vtap/ {
|
||||
alias /opt/pg/vtap;
|
||||
}
|
||||
|
||||
# REST API calls start with /v[0-9]/, a keyword, or a capital letter.
|
||||
# Note: Regular expressions have higher precedence than prefix matches
|
||||
# so don't combine with /0/...
|
||||
location ~ ^/(v[0-9]/|pg/|docs|api-docs|[A-Z]) {
|
||||
set $active_upstream "http://sal";
|
||||
access_by_lua 'if ngx.req.get_uri_args()["server"]~=nil then
|
||||
if ngx.req.get_uri_args()["server"]~=ngx.var.host then
|
||||
ngx.var.active_upstream = "https://"..ngx.req.get_uri_args()["server"]..ngx.var.request_uri
|
||||
end
|
||||
end';
|
||||
|
||||
proxy_pass $active_upstream;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Connection "";
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
}
|
||||
|
||||
location /0/ {
|
||||
set $active_upstream "http://sal";
|
||||
access_by_lua 'if ngx.req.get_uri_args()["server"]~=nil then
|
||||
if ngx.req.get_uri_args()["server"]~=ngx.var.host then
|
||||
ngx.var.active_upstream = "https://"..ngx.req.get_uri_args()["server"]..ngx.var.request_uri
|
||||
end
|
||||
end';
|
||||
|
||||
proxy_pass $active_upstream;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Connection "";
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
}
|
||||
|
||||
location /0/websocket {
|
||||
set $active_upstream "http://websocket";
|
||||
access_by_lua 'if ngx.req.get_uri_args()["server"]~=nil then
|
||||
if ngx.req.get_uri_args()["server"]~=ngx.var.host then
|
||||
ngx.var.active_upstream = "https://"..ngx.req.get_uri_args()["server"]..ngx.var.request_uri
|
||||
end
|
||||
end';
|
||||
proxy_pass $active_upstream;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection $connection_upgrade;
|
||||
}
|
||||
}
|
||||
|
||||
server {
|
||||
listen unix:/opt/pg/tmp/sal-rest.socket;
|
||||
|
||||
# debug socket
|
||||
listen 127.0.0.1:9080;
|
||||
|
||||
location / {
|
||||
set $active_upstream "";
|
||||
access_by_lua 'ngx.var.active_upstream = find_next(ngx.shared.rest_servers, 9180)';
|
||||
proxy_pass http://$active_upstream:9180;
|
||||
}
|
||||
|
||||
location /_debug/rest_servers {
|
||||
access_by_lua 'find_next(ngx.shared.rest_servers, 9180)';
|
||||
content_by_lua '
|
||||
for _, ip in pairs(ngx.shared.rest_servers:get_keys()) do
|
||||
ngx.say(ip.."="..ngx.shared.rest_servers:get(ip))
|
||||
end
|
||||
';
|
||||
}
|
||||
|
||||
location /_debug/tc_servers {
|
||||
access_by_lua 'find_next(ngx.shared.tc_servers, 12349)';
|
||||
content_by_lua '
|
||||
for _, ip in pairs(ngx.shared.tc_servers:get_keys()) do
|
||||
ngx.say(ip.."="..ngx.shared.tc_servers:get(ip))
|
||||
end
|
||||
';
|
||||
}
|
||||
}
|
||||
|
|
@ -1,14 +0,0 @@
|
|||
plumgrid_ip={{ director_ips_string }}
|
||||
plumgrid_port=8001
|
||||
mgmt_dev={{ interface }}
|
||||
label={{ label}}
|
||||
plumgrid_rsync_port=2222
|
||||
plumgrid_rest_addr=0.0.0.0:9180
|
||||
fabric_mode={{ fabric_mode }}
|
||||
plumgrid_syslog_ng_ip={{ plumgrid_syslog_ng_ip }}
|
||||
plumgrid_syslog_ng_port={{ plumgrid_syslog_ng_port }}
|
||||
plumgrid_monitor_interval={{ plumgrid_monitor_interval }}
|
||||
start_plumgrid_iovisor=yes
|
||||
start_plumgrid=`/opt/pg/scripts/pg_is_director.sh $plumgrid_ip`
|
||||
location=
|
||||
|
|
@ -1,21 +0,0 @@
|
|||
{% if rabbitmq_host or rabbitmq_hosts -%}
|
||||
rabbit_userid = {{ rabbitmq_user }}
|
||||
rabbit_virtual_host = {{ rabbitmq_virtual_host }}
|
||||
rabbit_password = {{ rabbitmq_password }}
|
||||
{% if rabbitmq_hosts -%}
|
||||
rabbit_hosts = {{ rabbitmq_hosts }}
|
||||
{% if rabbitmq_ha_queues -%}
|
||||
rabbit_ha_queues = True
|
||||
rabbit_durable_queues = False
|
||||
{% endif -%}
|
||||
{% else -%}
|
||||
rabbit_host = {{ rabbitmq_host }}
|
||||
{% endif -%}
|
||||
{% if rabbit_ssl_port -%}
|
||||
rabbit_use_ssl = True
|
||||
rabbit_port = {{ rabbit_ssl_port }}
|
||||
{% if rabbit_ssl_ca -%}
|
||||
kombu_ssl_ca_certs = {{ rabbit_ssl_ca }}
|
||||
{% endif -%}
|
||||
{% endif -%}
|
||||
{% endif -%}
|
|
@ -1,5 +0,0 @@
|
|||
#!/bin/bash
|
||||
|
||||
sudo add-apt-repository ppa:juju/stable -y
|
||||
sudo apt-get update
|
||||
sudo apt-get install amulet python3-requests juju-deployer -y
|
|
@ -1,47 +0,0 @@
|
|||
#!/usr/bin/env python3
|
||||
|
||||
import amulet
|
||||
import requests
|
||||
import unittest
|
||||
|
||||
|
||||
class TestDeployment(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
cls.deployment = amulet.Deployment(series='trusty')
|
||||
cls.deployment.load_bundle_file(
|
||||
bundle_file='tests/files/plumgrid-director.yaml',
|
||||
deployment_name='test')
|
||||
try:
|
||||
cls.deployment.setup(timeout=2000)
|
||||
cls.deployment.sentry.wait()
|
||||
except amulet.helpers.TimeoutError:
|
||||
amulet.raise_status(
|
||||
amulet.SKIP, msg="Environment wasn't stood up in time")
|
||||
except:
|
||||
raise
|
||||
cls.pg_sentry = cls.deployment.sentry['plumgrid-director/0']
|
||||
|
||||
def test_plumgrid_director_gui(self):
|
||||
pg_options = self.deployment.services['plumgrid-director']['options']
|
||||
pg_vip = pg_options['plumgrid-virtual-ip']
|
||||
if not pg_vip:
|
||||
amulet.raise_status(
|
||||
amulet.FAIL, msg='plumgrid virtual ip was not found.')
|
||||
gui_req = requests.get(
|
||||
"https://{}/".format(pg_vip), verify=False)
|
||||
if not gui_req.ok:
|
||||
amulet.raise_status(
|
||||
amulet.FAIL, msg='unable to access plumgrid gui.')
|
||||
|
||||
def test_plumgrid_director_started(self):
|
||||
pg_sentry = self.deployment.sentry['plumgrid-director/0']
|
||||
agent_state = pg_sentry.info['agent-state']
|
||||
if agent_state != 'started':
|
||||
amulet.raise_status(
|
||||
amulet.FAIL,
|
||||
msg='plumgrid director is not in a started state')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
|
@ -1,133 +0,0 @@
|
|||
test:
|
||||
series: 'trusty'
|
||||
relations:
|
||||
- - mysql
|
||||
- keystone
|
||||
- - nova-cloud-controller
|
||||
- mysql
|
||||
- - nova-cloud-controller
|
||||
- rabbitmq-server
|
||||
- - nova-cloud-controller
|
||||
- glance
|
||||
- - nova-cloud-controller
|
||||
- keystone
|
||||
- - nova-compute
|
||||
- nova-cloud-controller
|
||||
- - nova-compute
|
||||
- mysql
|
||||
- - nova-compute
|
||||
- rabbitmq-server
|
||||
- - nova-compute
|
||||
- glance
|
||||
- - glance
|
||||
- mysql
|
||||
- - glance
|
||||
- keystone
|
||||
- - glance
|
||||
- cinder
|
||||
- - mysql
|
||||
- cinder
|
||||
- - cinder
|
||||
- rabbitmq-server
|
||||
- - cinder
|
||||
- nova-cloud-controller
|
||||
- - cinder
|
||||
- keystone
|
||||
- - openstack-dashboard
|
||||
- keystone
|
||||
- - neutron-api
|
||||
- mysql
|
||||
- - neutron-api
|
||||
- keystone
|
||||
- - neutron-api
|
||||
- rabbitmq-server
|
||||
- - neutron-api
|
||||
- nova-cloud-controller
|
||||
- - neutron-api
|
||||
- neutron-api-plumgrid
|
||||
- - neutron-api-plumgrid
|
||||
- plumgrid-edge
|
||||
- - plumgrid-director
|
||||
- plumgrid-edge
|
||||
- - nova-compute
|
||||
- plumgrid-edge
|
||||
- - plumgrid-director
|
||||
- plumgrid-gateway
|
||||
services:
|
||||
mysql:
|
||||
charm: cs:trusty/mysql
|
||||
num_units: 1
|
||||
to: 'lxc:plumgrid-director=0'
|
||||
rabbitmq-server:
|
||||
charm: cs:trusty/rabbitmq-server
|
||||
num_units: 1
|
||||
to: 'lxc:plumgrid-director=0'
|
||||
keystone:
|
||||
charm: cs:trusty/keystone
|
||||
num_units: 1
|
||||
options:
|
||||
admin-password: plumgrid
|
||||
openstack-origin: cloud:trusty-kilo
|
||||
to: 'lxc:plumgrid-director=0'
|
||||
nova-cloud-controller:
|
||||
charm: cs:trusty/nova-cloud-controller
|
||||
num_units: 1
|
||||
options:
|
||||
console-access-protocol: novnc
|
||||
network-manager: Neutron
|
||||
openstack-origin: cloud:trusty-kilo
|
||||
quantum-security-groups: 'yes'
|
||||
to: 'lxc:plumgrid-director=0'
|
||||
glance:
|
||||
charm: cs:trusty/glance
|
||||
num_units: 1
|
||||
options:
|
||||
openstack-origin: cloud:trusty-kilo
|
||||
to: 'lxc:plumgrid-director=0'
|
||||
openstack-dashboard:
|
||||
charm: cs:trusty/openstack-dashboard
|
||||
num_units: 1
|
||||
options:
|
||||
openstack-origin: cloud:trusty-kilo
|
||||
to: 'lxc:plumgrid-director=0'
|
||||
cinder:
|
||||
charm: cs:trusty/cinder
|
||||
num_units: 1
|
||||
options:
|
||||
openstack-origin: cloud:trusty-kilo
|
||||
to: 'lxc:plumgrid-director=0'
|
||||
neutron-api:
|
||||
charm: cs:~plumgrid-team/trusty/neutron-api
|
||||
num_units: 1
|
||||
options:
|
||||
neutron-plugin: plumgrid
|
||||
neutron-security-groups: false
|
||||
openstack-origin: cloud:trusty-kilo
|
||||
plumgrid-password: plumgrid
|
||||
plumgrid-username: plumgrid
|
||||
plumgrid-virtual-ip: 192.168.100.250
|
||||
to: 'lxc:plumgrid-director=0'
|
||||
neutron-api-plumgrid:
|
||||
charm: cs:~plumgrid-team/trusty/neutron-api-plumgrid
|
||||
options:
|
||||
enable-metadata: True
|
||||
plumgrid-director:
|
||||
charm: cs:~plumgrid-team/trusty/plumgrid-director
|
||||
num_units: 1
|
||||
constraints: "root-disk=30G mem=8G cpu-cores=8"
|
||||
options:
|
||||
plumgrid-virtual-ip: 192.168.100.250
|
||||
nova-compute:
|
||||
charm: cs:~plumgrid-team/trusty/nova-compute
|
||||
num_units: 1
|
||||
options:
|
||||
enable-live-migration: true
|
||||
enable-resize: true
|
||||
migration-auth-type: ssh
|
||||
openstack-origin: cloud:trusty-kilo
|
||||
to: '0'
|
||||
plumgrid-edge:
|
||||
charm: cs:~plumgrid-team/trusty/plumgrid-edge
|
||||
plumgrid-gateway:
|
||||
charm: cs:~plumgrid-team/trusty/plumgrid-gateway
|
||||
num_units: 1
|
|
@ -1,123 +0,0 @@
|
|||
test:
|
||||
series: 'trusty'
|
||||
relations:
|
||||
- - mysql
|
||||
- keystone
|
||||
- - nova-cloud-controller
|
||||
- mysql
|
||||
- - nova-cloud-controller
|
||||
- rabbitmq-server
|
||||
- - nova-cloud-controller
|
||||
- glance
|
||||
- - nova-cloud-controller
|
||||
- keystone
|
||||
- - nova-compute
|
||||
- nova-cloud-controller
|
||||
- - nova-compute
|
||||
- mysql
|
||||
- - nova-compute
|
||||
- rabbitmq-server
|
||||
- - nova-compute
|
||||
- glance
|
||||
- - glance
|
||||
- mysql
|
||||
- - glance
|
||||
- keystone
|
||||
- - glance
|
||||
- cinder
|
||||
- - mysql
|
||||
- cinder
|
||||
- - cinder
|
||||
- rabbitmq-server
|
||||
- - cinder
|
||||
- nova-cloud-controller
|
||||
- - cinder
|
||||
- keystone
|
||||
- - openstack-dashboard
|
||||
- keystone
|
||||
- - neutron-api
|
||||
- mysql
|
||||
- - neutron-api
|
||||
- keystone
|
||||
- - neutron-api
|
||||
- rabbitmq-server
|
||||
- - neutron-api
|
||||
- nova-cloud-controller
|
||||
- - neutron-api
|
||||
- neutron-api-plumgrid
|
||||
- - neutron-api-plumgrid
|
||||
- plumgrid-edge
|
||||
- - plumgrid-director
|
||||
- plumgrid-edge
|
||||
- - nova-compute
|
||||
- plumgrid-edge
|
||||
- - plumgrid-director
|
||||
- plumgrid-gateway
|
||||
services:
|
||||
mysql:
|
||||
charm: cs:trusty/mysql
|
||||
num_units: 1
|
||||
rabbitmq-server:
|
||||
charm: cs:trusty/rabbitmq-server
|
||||
num_units: 1
|
||||
keystone:
|
||||
charm: cs:trusty/keystone
|
||||
num_units: 1
|
||||
options:
|
||||
admin-password: plumgrid
|
||||
openstack-origin: cloud:trusty-kilo
|
||||
nova-cloud-controller:
|
||||
charm: cs:trusty/nova-cloud-controller
|
||||
num_units: 1
|
||||
options:
|
||||
console-access-protocol: novnc
|
||||
network-manager: Neutron
|
||||
openstack-origin: cloud:trusty-kilo
|
||||
quantum-security-groups: 'yes'
|
||||
glance:
|
||||
charm: cs:trusty/glance
|
||||
num_units: 1
|
||||
options:
|
||||
openstack-origin: cloud:trusty-kilo
|
||||
openstack-dashboard:
|
||||
charm: cs:trusty/openstack-dashboard
|
||||
num_units: 1
|
||||
options:
|
||||
openstack-origin: cloud:trusty-kilo
|
||||
cinder:
|
||||
charm: cs:trusty/cinder
|
||||
num_units: 1
|
||||
options:
|
||||
openstack-origin: cloud:trusty-kilo
|
||||
neutron-api:
|
||||
charm: cs:~plumgrid-team/trusty/neutron-api
|
||||
num_units: 1
|
||||
options:
|
||||
neutron-plugin: plumgrid
|
||||
neutron-security-groups: false
|
||||
openstack-origin: cloud:trusty-kilo
|
||||
plumgrid-password: plumgrid
|
||||
plumgrid-username: plumgrid
|
||||
plumgrid-virtual-ip: 192.168.100.250
|
||||
neutron-api-plumgrid:
|
||||
charm: cs:~plumgrid-team/trusty/neutron-api-plumgrid
|
||||
options:
|
||||
enable-metadata: True
|
||||
plumgrid-director:
|
||||
charm: cs:~plumgrid-team/trusty/plumgrid-director
|
||||
num_units: 1
|
||||
options:
|
||||
plumgrid-virtual-ip: 192.168.100.250
|
||||
nova-compute:
|
||||
charm: cs:~plumgrid-team/trusty/nova-compute
|
||||
num_units: 1
|
||||
options:
|
||||
enable-live-migration: true
|
||||
enable-resize: true
|
||||
migration-auth-type: ssh
|
||||
openstack-origin: cloud:trusty-kilo
|
||||
plumgrid-edge:
|
||||
charm: cs:~plumgrid-team/trusty/plumgrid-edge
|
||||
plumgrid-gateway:
|
||||
charm: cs:~plumgrid-team/trusty/plumgrid-gateway
|
||||
num_units: 1
|
|
@ -1,2 +0,0 @@
|
|||
makefile:
|
||||
- lint
|
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue