Retire Packaging Deb project repos
This commit is part of a series to retire the Packaging Deb project. Step 2 is to remove all content from the project repos, replacing it with a README notification where to find ongoing work, and how to recover the repo if needed at some future point (as in https://docs.openstack.org/infra/manual/drivers.html#retiring-a-project). Change-Id: I9a1573dadc4d14f9584c7af317918177993a83f0
This commit is contained in:
Tony Breeds
parent
43b5d6d905
commit
9871f32012
|
|
@ -1,3 +0,0 @@
|
|||
[run]
|
||||
branch = True
|
||||
omit = shotgun/test/*
|
||||
|
|
@ -1,76 +0,0 @@
|
|||
*.py[cod]
|
||||
|
||||
# C extensions
|
||||
*.so
|
||||
|
||||
# Packages
|
||||
*.egg
|
||||
*.egg-info
|
||||
dist
|
||||
build
|
||||
.eggs
|
||||
eggs
|
||||
parts
|
||||
var
|
||||
sdist
|
||||
develop-eggs
|
||||
.installed.cfg
|
||||
lib
|
||||
lib64
|
||||
|
||||
# Installer logs
|
||||
pip-log.txt
|
||||
|
||||
# Unit test / coverage reports
|
||||
.coverage
|
||||
.tox
|
||||
nosetests.xml
|
||||
.testrepository
|
||||
.venv
|
||||
|
||||
# Translations
|
||||
*.mo
|
||||
|
||||
# Mr Developer
|
||||
.mr.developer.cfg
|
||||
.project
|
||||
.pydevproject
|
||||
|
||||
# Complexity
|
||||
output/*.html
|
||||
output/*/index.html
|
||||
|
||||
# Sphinx
|
||||
doc/build
|
||||
|
||||
# pbr generates these
|
||||
AUTHORS
|
||||
ChangeLog
|
||||
|
||||
# Editors
|
||||
*~
|
||||
.*.swp
|
||||
.*sw?
|
||||
|
||||
# Cookiecutter
|
||||
output/
|
||||
boilerplate/
|
||||
|
||||
# services' runtime files
|
||||
*.log
|
||||
*.pid
|
||||
|
||||
# Vagrant housekeeping file
|
||||
.vagrant
|
||||
|
||||
lock
|
||||
|
||||
.testrepository
|
||||
.tox
|
||||
.venv
|
||||
.idea
|
||||
.DS_Store
|
||||
test_run/*
|
||||
.cache
|
||||
|
||||
.bashrc
|
||||
|
|
@ -1,4 +0,0 @@
|
|||
[gerrit]
|
||||
host=review.openstack.org
|
||||
port=29418
|
||||
project=openstack/shotgun.git
|
||||
|
|
@ -1,4 +0,0 @@
|
|||
[DEFAULT]
|
||||
test_command=OS_STDOUT_CAPTURE=1 OS_STDERR_CAPTURE=1 OS_TEST_TIMEOUT=60 ${PYTHON:-python} -m subunit.run discover -s shotgun/test/ -p "*.py" $LISTOPT $IDOPTION
|
||||
test_id_option=--load-list $IDFILE
|
||||
test_list_option=--list
|
||||
|
|
@ -1,17 +0,0 @@
|
|||
If you would like to contribute to the development of OpenStack, you must
|
||||
follow the steps in this page:
|
||||
|
||||
http://docs.openstack.org/infra/manual/developers.html
|
||||
|
||||
If you already have a good understanding of how the system works and your
|
||||
OpenStack accounts are set up, you can skip to the development workflow
|
||||
section of this documentation to learn how changes to OpenStack should be
|
||||
submitted for review via the Gerrit tool:
|
||||
|
||||
http://docs.openstack.org/infra/manual/developers.html#development-workflow
|
||||
|
||||
Pull requests submitted through GitHub will be ignored.
|
||||
|
||||
Bugs should be filed on Launchpad, not GitHub:
|
||||
|
||||
https://bugs.launchpad.net/fuel
|
||||
|
|
@ -1,4 +0,0 @@
|
|||
shotgun Style Commandments
|
||||
===============================================
|
||||
|
||||
Read the OpenStack Style Commandments http://docs.openstack.org/developer/hacking/
|
||||
176
LICENSE
176
LICENSE
|
|
@ -1,176 +0,0 @@
|
|||
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
||||
61
MAINTAINERS
61
MAINTAINERS
|
|
@ -1,61 +0,0 @@
|
|||
---
|
||||
description:
|
||||
For Fuel team structure and contribution policy, see [1].
|
||||
|
||||
This is repository level MAINTAINERS file. All contributions to this
|
||||
repository must be approved by one or more Core Reviewers [2].
|
||||
If you are contributing to files (or create new directories) in
|
||||
root folder of this repository, please contact Core Reviewers for
|
||||
review and merge requests.
|
||||
|
||||
If you are contributing to subfolders of this repository, please
|
||||
check 'maintainers' section of this file in order to find maintainers
|
||||
for those specific modules.
|
||||
|
||||
It is mandatory to get +1 from one or more maintainers before asking
|
||||
Core Reviewers for review/merge in order to decrease a load on Core Reviewers [3].
|
||||
Exceptions are when maintainers are actually cores, or when maintainers
|
||||
are not available for some reason (e.g. on vacation).
|
||||
|
||||
[1] https://specs.openstack.org/openstack/fuel-specs/policy/team-structure
|
||||
[2] https://review.openstack.org/#/admin/groups/1144,members
|
||||
[3] http://lists.openstack.org/pipermail/openstack-dev/2015-August/072406.html
|
||||
|
||||
Please keep this file in YAML format in order to allow helper scripts
|
||||
to read this as a configuration data.
|
||||
|
||||
maintainers:
|
||||
|
||||
- ./:
|
||||
- name: Maciej Kwiek
|
||||
email: mkwiek@mirantis.com
|
||||
IRC: mkwiek
|
||||
|
||||
- name: Vladimir Kozhukalov
|
||||
email: vkozhukalov@mirantis.com
|
||||
IRC: kozhukalov
|
||||
|
||||
- specs/:
|
||||
- name: Mikhail Ivanov
|
||||
email: mivanov@mirantis.com
|
||||
IRC: mivanov
|
||||
|
||||
- name: Artem Silenkov
|
||||
email: asilenkov@mirantis.com
|
||||
IRC: asilenkov
|
||||
|
||||
- name: Alexander Tsamutali
|
||||
email: atsamutali@mirantis.com
|
||||
IRC: astsmtl
|
||||
|
||||
- name: Daniil Trishkin
|
||||
email: dtrishkin@mirantis.com
|
||||
IRC: dtrishkin
|
||||
|
||||
- name: Ivan Udovichenko
|
||||
email: iudovichenko@mirantis.com
|
||||
IRC: tlbr
|
||||
|
||||
- name: Igor Yozhikov
|
||||
email: iyozhikov@mirantis.com
|
||||
IRC: IgorYozhikov
|
||||
|
|
@ -1,6 +0,0 @@
|
|||
include AUTHORS
|
||||
include ChangeLog
|
||||
exclude .gitignore
|
||||
exclude .gitreview
|
||||
|
||||
global-exclude *.pyc
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
This project is no longer maintained.
|
||||
|
||||
The contents of this repository are still available in the Git
|
||||
source code management system. To see the contents of this
|
||||
repository before it reached its end of life, please check out the
|
||||
previous commit with "git checkout HEAD^1".
|
||||
|
||||
For ongoing work on maintaining OpenStack packages in the Debian
|
||||
distribution, please see the Debian OpenStack packaging team at
|
||||
https://wiki.debian.org/OpenStack/.
|
||||
|
||||
For any further questions, please email
|
||||
openstack-dev@lists.openstack.org or join #openstack-dev on
|
||||
Freenode.
|
||||
30
README.rst
30
README.rst
|
|
@ -1,30 +0,0 @@
|
|||
===============================
|
||||
shotgun
|
||||
===============================
|
||||
|
||||
Shotgun is diagnostic snapshot generator. Currently it's
|
||||
used mainly in `Fuel <https://github.com/openstack/fuel-web>`_.
|
||||
|
||||
|
||||
-----------------
|
||||
Project resources
|
||||
-----------------
|
||||
|
||||
Project status, bugs, and blueprints are tracked on Launchpad:
|
||||
https://launchpad.net/fuel
|
||||
|
||||
Development documentation is hosted here:
|
||||
https://docs.fuel-infra.org/fuel-dev
|
||||
|
||||
User guide can be found here:
|
||||
http://docs.mirantis.com
|
||||
|
||||
Any additional information can be found on the Fuel's project wiki
|
||||
https://wiki.openstack.org/wiki/Fuel
|
||||
|
||||
Anyone wishing to contribute to shotgun should follow the general
|
||||
OpenStack process. A good reference for it can be found here:
|
||||
https://wiki.openstack.org/wiki/How_To_Contribute
|
||||
|
||||
http://docs.openstack.org/infra/manual/developers.html
|
||||
|
||||
|
|
@ -1,40 +0,0 @@
|
|||
{
|
||||
|
||||
"target": "/tmp/snapshot",
|
||||
"timestamp": "true",
|
||||
"dump_roles": {
|
||||
"master": ["localhost"],
|
||||
"slave": ["srv11-msk.msk.mirantis.net"]
|
||||
},
|
||||
"dump_objects": {
|
||||
"master": [
|
||||
{
|
||||
"type": "file",
|
||||
"path": "/etc/rsyslog.d"
|
||||
},
|
||||
{
|
||||
"type": "postgres",
|
||||
"dbname": "nailgun",
|
||||
"username": "postgres"
|
||||
},
|
||||
],
|
||||
"slave": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "ps auxww"
|
||||
},
|
||||
{
|
||||
"type": "command",
|
||||
"command": "top"
|
||||
},
|
||||
{
|
||||
"type": "command",
|
||||
"command": "ip a"
|
||||
},
|
||||
{
|
||||
"type": "file",
|
||||
"path": "/etc/naily.facts"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
|
|
@ -1,34 +0,0 @@
|
|||
# Copyright 2013 Mirantis, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path[:0] = [os.path.abspath(os.path.join(os.path.dirname(__file__), ".."))]
|
||||
|
||||
from shotgun.config import Config
|
||||
from shotgun.manager import Manager
|
||||
|
||||
logging.basicConfig(level=logging.DEBUG)
|
||||
|
||||
|
||||
with open("snapshot.json", "r") as fo:
|
||||
data = json.loads(fo.read())
|
||||
config = Config(data)
|
||||
|
||||
|
||||
manager = Manager(config)
|
||||
manager.snapshot()
|
||||
|
|
@ -1,17 +0,0 @@
|
|||
dump:
|
||||
local:
|
||||
objects:
|
||||
- type: command
|
||||
command:
|
||||
- cat /etc/fuel_build_id
|
||||
- cat /etc/fuel_build_number
|
||||
- cat /etc/fuel_release
|
||||
- cat /etc/fuel_openstack_version
|
||||
- |
|
||||
rpm -qa | \
|
||||
egrep 'fuel|astute|network-checker|nailgun|packetary|shotgun' | \
|
||||
while read package; do
|
||||
echo
|
||||
echo $package
|
||||
rpm -q --changelog $package | head -10
|
||||
done
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
dump:
|
||||
local:
|
||||
objects:
|
||||
- type: command
|
||||
command:
|
||||
- cat /etc/fuel_build_id
|
||||
- cat /etc/fuel_build_number
|
||||
- cat /etc/fuel_release
|
||||
- cat /etc/fuel_openstack_version
|
||||
- rpm -qa | egrep 'fuel|astute|network-checker|nailgun|packetary|shotgun'
|
||||
|
|
@ -1,9 +0,0 @@
|
|||
# The order of packages is significant, because pip processes them in the order
|
||||
# of appearance. Changing the order has an impact on the overall integration
|
||||
# process, which may cause wedges in the gate later.
|
||||
six>=1.9.0 # MIT
|
||||
cliff!=1.16.0,>=1.15.0 # Apache-2.0
|
||||
|
||||
# Packages beyond this line are not in Global Requirements list
|
||||
# and must be added there.
|
||||
Fabric>=1.7.0
|
||||
38
setup.cfg
38
setup.cfg
|
|
@ -1,38 +0,0 @@
|
|||
[metadata]
|
||||
name = shotgun
|
||||
version = 10.0.0
|
||||
description-file = README.rst
|
||||
description = Shotgun is diagnostic snapshot generator
|
||||
author = OpenStack
|
||||
author_email = openstack-dev@lists.openstack.org
|
||||
|
||||
classifier =
|
||||
Environment :: OpenStack
|
||||
Intended Audience :: Information Technology
|
||||
Intended Audience :: System Administrators
|
||||
License :: OSI Approved :: Apache Software License
|
||||
Operating System :: POSIX :: Linux
|
||||
Programming Language :: Python
|
||||
Programming Language :: Python :: 2
|
||||
Programming Language :: Python :: 2.7
|
||||
|
||||
[files]
|
||||
packages = shotgun
|
||||
|
||||
[entry_points]
|
||||
console_scripts =
|
||||
shotgun=shotgun.cli:main
|
||||
shotgun2=shotgun.cli2:main
|
||||
|
||||
shotgun =
|
||||
snapshot=shotgun.cli2:SnapshotCommand
|
||||
report=shotgun.cli2:ReportCommand
|
||||
short-report=shotgun.cli2:ShortReportCommand
|
||||
|
||||
[global]
|
||||
setup-hooks =
|
||||
pbr.hooks.setup_hook
|
||||
shotgun.hooks.setup_hook
|
||||
|
||||
[wheel]
|
||||
python-tag = py2
|
||||
29
setup.py
29
setup.py
|
|
@ -1,29 +0,0 @@
|
|||
# Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
# implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
# THIS FILE IS MANAGED BY THE GLOBAL REQUIREMENTS REPO - DO NOT EDIT
|
||||
import setuptools
|
||||
|
||||
# In python < 2.7.4, a lazy loading of package `pbr` will break
|
||||
# setuptools if some other modules registered functions in `atexit`.
|
||||
# solution from: http://bugs.python.org/issue15881#msg170215
|
||||
try:
|
||||
import multiprocessing # noqa
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
setuptools.setup(
|
||||
setup_requires=['pbr>=1.8'],
|
||||
pbr=True)
|
||||
|
|
@ -1,20 +0,0 @@
|
|||
# Copyright 2013 Mirantis, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
import pkg_resources
|
||||
|
||||
try:
|
||||
__version__ = pkg_resources.require(__name__)[0].version
|
||||
except pkg_resources.DistributionNotFound as e:
|
||||
__version__ = "Unknown"
|
||||
|
|
@ -1,78 +0,0 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
|
||||
# Copyright 2014 Mirantis, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import logging
|
||||
import warnings
|
||||
|
||||
from shotgun.logger import configure_logger
|
||||
configure_logger()
|
||||
|
||||
from shotgun.config import Config
|
||||
from shotgun.manager import Manager
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def parse_args():
|
||||
"""Parse arguments and return them
|
||||
|
||||
:returns: argparse object
|
||||
"""
|
||||
parser = argparse.ArgumentParser()
|
||||
|
||||
parser.add_argument(
|
||||
'-c',
|
||||
'--config',
|
||||
help='configuration file',
|
||||
required=True)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def read_config(config_path):
|
||||
"""Reads config
|
||||
|
||||
:param config_path: path to configuration file
|
||||
:returns: dict with configuration data
|
||||
"""
|
||||
with open(config_path, "r") as fo:
|
||||
config = json.loads(fo.read())
|
||||
|
||||
return config
|
||||
|
||||
|
||||
def make_snapshot(args):
|
||||
"""Generates snapshot
|
||||
|
||||
:param args: argparse object
|
||||
"""
|
||||
config_object = Config(read_config(args.config))
|
||||
manager = Manager(config_object)
|
||||
snapshot_path = manager.snapshot()
|
||||
logger.info(u'Snapshot path: {0}'.format(snapshot_path))
|
||||
|
||||
|
||||
def main():
|
||||
"""Entry point"""
|
||||
warnings.warn("This command is deprecated. "
|
||||
"Please use 'shotgun2' instead", DeprecationWarning)
|
||||
try:
|
||||
make_snapshot(parse_args())
|
||||
except Exception as e:
|
||||
logger.error(e)
|
||||
return getattr(e, 'errno', 1)
|
||||
108
shotgun/cli2.py
108
shotgun/cli2.py
|
|
@ -1,108 +0,0 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
|
||||
# Copyright 2015 Mirantis, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
import logging
|
||||
import yaml
|
||||
|
||||
from cliff.app import App
|
||||
from cliff.command import Command
|
||||
from cliff.commandmanager import CommandManager
|
||||
from cliff.lister import Lister
|
||||
|
||||
import shotgun
|
||||
from shotgun.config import Config
|
||||
from shotgun.manager import Manager
|
||||
|
||||
|
||||
class Base(object):
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
def initialize_cmd(self, parsed_args):
|
||||
with open(parsed_args.config, "r") as f:
|
||||
self.config = Config(yaml.safe_load(f))
|
||||
self.manager = Manager(self.config)
|
||||
|
||||
|
||||
class SnapshotCommand(Command, Base):
|
||||
|
||||
def get_parser(self, prog_name):
|
||||
parser = super(SnapshotCommand, self).get_parser(prog_name)
|
||||
parser.add_argument(
|
||||
'--config',
|
||||
required=True,
|
||||
help='Path to snapshot config file')
|
||||
return parser
|
||||
|
||||
def take_action(self, parsed_args):
|
||||
"""Generates snapshot
|
||||
|
||||
:param parsed_args: argparse object
|
||||
"""
|
||||
self.initialize_cmd(parsed_args)
|
||||
snapshot_path = self.manager.snapshot()
|
||||
self.log.info(u'Snapshot path: {0}'.format(snapshot_path))
|
||||
|
||||
def run(self, parsed_args):
|
||||
"""Overriden for returning errno from exceptions"""
|
||||
try:
|
||||
return super(SnapshotCommand, self).run(parsed_args)
|
||||
except Exception as err:
|
||||
self.log.error(err)
|
||||
return getattr(err, 'errno', 1)
|
||||
|
||||
|
||||
class ReportCommand(Lister, Base):
|
||||
columns = ['Host', 'Reporter', 'Report']
|
||||
|
||||
def get_parser(self, prog_name):
|
||||
parser = super(ReportCommand, self).get_parser(prog_name)
|
||||
parser.add_argument(
|
||||
'--config',
|
||||
default='/etc/shotgun/report.yaml',
|
||||
help='Path to report config file')
|
||||
return parser
|
||||
|
||||
def take_action(self, parsed_args):
|
||||
self.initialize_cmd(parsed_args)
|
||||
data = [line for line in self.manager.report()]
|
||||
return (self.columns, data)
|
||||
|
||||
|
||||
class ShortReportCommand(Command, Base):
|
||||
|
||||
def get_parser(self, prog_name):
|
||||
parser = super(ShortReportCommand, self).get_parser(prog_name)
|
||||
parser.add_argument(
|
||||
'--config',
|
||||
default='/etc/shotgun/short_report.yaml',
|
||||
help='Path to report config file')
|
||||
return parser
|
||||
|
||||
def take_action(self, parsed_args):
|
||||
self.initialize_cmd(parsed_args)
|
||||
for line in self.manager.report():
|
||||
host, reporter, report = line
|
||||
if report:
|
||||
if reporter:
|
||||
self.app.stdout.write('{0}:\n'.format(reporter))
|
||||
self.app.stdout.write(' {out}\n'.format(out=report))
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
return App(
|
||||
description="Shotgun CLI",
|
||||
version=shotgun.__version__,
|
||||
command_manager=CommandManager('shotgun')
|
||||
).run(argv)
|
||||
|
|
@ -1,121 +0,0 @@
|
|||
# Copyright 2013 Mirantis, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
from collections import deque
|
||||
import copy
|
||||
import logging
|
||||
import time
|
||||
|
||||
import six
|
||||
|
||||
from shotgun import settings
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class Config(object):
|
||||
def __init__(self, data=None):
|
||||
self.data = data or {}
|
||||
self.time = time.localtime()
|
||||
self.offline_hosts = set()
|
||||
self.objs = deque()
|
||||
self.try_again = deque()
|
||||
for properties in six.itervalues(self.data.get('dump', {})):
|
||||
hosts = properties.get('hosts') or [{}]
|
||||
for obj in properties.get('objects', []):
|
||||
for h in hosts:
|
||||
obj_new = copy.deepcopy(obj)
|
||||
obj_new['host'] = h
|
||||
self.objs.append(obj_new)
|
||||
|
||||
def _timestamp(self, name):
|
||||
return "{0}-{1}".format(
|
||||
name,
|
||||
time.strftime('%Y-%m-%d_%H-%M-%S', self.time)
|
||||
)
|
||||
|
||||
@property
|
||||
def target(self):
|
||||
target = self.data.get("target", settings.TARGET)
|
||||
if self.data.get("timestamp", settings.TIMESTAMP):
|
||||
target = self._timestamp(target)
|
||||
return target
|
||||
|
||||
@property
|
||||
def compression_level(self):
|
||||
level = self.data.get("compression_level")
|
||||
if level is None:
|
||||
logger.info(
|
||||
'Compression level is not specified,'
|
||||
' Default %s will be used', settings.COMPRESSION_LEVEL)
|
||||
|
||||
level = settings.COMPRESSION_LEVEL
|
||||
|
||||
return '-{level}'.format(level=level)
|
||||
|
||||
@property
|
||||
def lastdump(self):
|
||||
return self.data.get("lastdump", settings.LASTDUMP)
|
||||
|
||||
@staticmethod
|
||||
def get_network_address(obj):
|
||||
"""Returns network address of object."""
|
||||
return obj['host'].get('address') or obj['host'].get('hostname')
|
||||
|
||||
def on_network_error(self, obj):
|
||||
"""Lets the object to have another attempt for being proccessed."""
|
||||
host = self.get_network_address(obj)
|
||||
logger.debug("Remote host %s is unreachable. "
|
||||
"Processing of its objects postponed.", host)
|
||||
self.try_again.append(obj)
|
||||
self.offline_hosts.add(host)
|
||||
|
||||
@property
|
||||
def objects(self):
|
||||
"""Stateful generator for processing objects.
|
||||
|
||||
It should be used in conjunction with on_network_error() to give
|
||||
another try for objects which threw NetworkError.
|
||||
"""
|
||||
for _ in range(settings.ATTEMPTS):
|
||||
while self.objs:
|
||||
obj = self.objs.popleft()
|
||||
if self.get_network_address(obj) not in self.offline_hosts:
|
||||
yield obj
|
||||
else:
|
||||
self.try_again.append(obj)
|
||||
self.offline_hosts.clear()
|
||||
self.objs, self.try_again = self.try_again, deque()
|
||||
|
||||
for obj in self.objs:
|
||||
obj["type"] = 'offline'
|
||||
host = self.get_network_address(obj)
|
||||
if host not in self.offline_hosts:
|
||||
self.offline_hosts.add(host)
|
||||
yield obj
|
||||
else:
|
||||
logger.debug("Skipping offline object processing: %s", obj)
|
||||
|
||||
@property
|
||||
def timeout(self):
|
||||
"""Timeout for executing commands."""
|
||||
return self.data.get("timeout", settings.DEFAULT_TIMEOUT)
|
||||
|
||||
@property
|
||||
def self_log_object(self):
|
||||
return {
|
||||
"type": "file",
|
||||
"path": settings.LOG_FILE
|
||||
}
|
||||
|
|
@ -1,349 +0,0 @@
|
|||
# Copyright 2013 Mirantis, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
import itertools
|
||||
import logging
|
||||
import os
|
||||
import pprint
|
||||
import pwd
|
||||
import re
|
||||
import socket
|
||||
import stat
|
||||
import sys
|
||||
import xmlrpclib
|
||||
|
||||
import fabric.api
|
||||
import fabric.exceptions
|
||||
|
||||
from shotgun import utils
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class CommandOut(object):
|
||||
stdout = None
|
||||
return_code = None
|
||||
stderr = None
|
||||
output = None
|
||||
|
||||
def __eq__(self, other):
|
||||
return (
|
||||
str(self.stdout) == str(other.stdout) and
|
||||
str(self.stderr) == str(other.stderr) and
|
||||
str(self.return_code) == str(other.return_code) and
|
||||
str(self.output) == str(other.output)
|
||||
)
|
||||
|
||||
|
||||
class Driver(object):
|
||||
|
||||
default_report_message = 'Reporting is not implemented for the driver.'
|
||||
|
||||
@classmethod
|
||||
def getDriver(cls, data, conf):
|
||||
driver_type = data["type"]
|
||||
return {
|
||||
"file": File,
|
||||
"dir": Dir,
|
||||
"postgres": Postgres,
|
||||
"xmlrpc": XmlRpc,
|
||||
"command": Command,
|
||||
"docker_command": DockerCommand,
|
||||
"offline": Offline,
|
||||
}.get(driver_type, cls)(data, conf)
|
||||
|
||||
def __init__(self, data, conf):
|
||||
logger.debug("Initializing driver %s: host=%s",
|
||||
self.__class__.__name__, data.get("host"))
|
||||
|
||||
data_host = data.get("host", {})
|
||||
hostname = data_host.get("hostname")
|
||||
address = data_host.get("address")
|
||||
self.ssh_key = data_host.get("ssh-key")
|
||||
|
||||
# `dest_host` is a IP address or a hostname which is used for network
|
||||
# connection. IP address is preferable for network connection.
|
||||
self.dest_host = address or hostname # destination host
|
||||
|
||||
# `host` is a hostname or an IP address or hostname of the local host
|
||||
# which is used for reporting (e.g. logging reporting); a hostname is
|
||||
# more preferred than an address for reporting. If neither hostname nor
|
||||
# address is used than it is considered that a command (or a file
|
||||
# operation) will be executed on a local host, so we use local hostname
|
||||
# for reporting.
|
||||
self.host = hostname or address or socket.gethostname()
|
||||
|
||||
self.conf = conf
|
||||
self.timeout = data.get("timeout", self.conf.timeout)
|
||||
|
||||
def snapshot(self):
|
||||
raise NotImplementedError
|
||||
|
||||
def report(self):
|
||||
"""Should be generator"""
|
||||
yield (self.host,
|
||||
'Driver: {0}'.format(self.__class__.__name__),
|
||||
self.default_report_message)
|
||||
|
||||
def command(self, command):
|
||||
out = CommandOut()
|
||||
|
||||
raw_stdout = utils.CCStringIO(writers=sys.stdout)
|
||||
try:
|
||||
if self.dest_host:
|
||||
with fabric.api.settings(
|
||||
host_string=self.dest_host, # destination host
|
||||
key_filename=self.ssh_key, # a path to ssh key
|
||||
timeout=2, # connection timeout
|
||||
command_timeout=self.timeout, # command execution timeout
|
||||
warn_only=True, # don't exit on error
|
||||
abort_on_prompts=True, # non-interactive mode
|
||||
use_shell=True,
|
||||
):
|
||||
logger.debug(
|
||||
"Running remote command: host: %s command: %s",
|
||||
self.host, command)
|
||||
try:
|
||||
output = fabric.api.run(command, stdout=raw_stdout)
|
||||
except SystemExit:
|
||||
logger.error("Fabric aborted this iteration")
|
||||
# NOTE(prmtl): because of pty=True (default) and
|
||||
# combine_stderr=True (default) stderr is combined
|
||||
# with stdout
|
||||
out.output = output
|
||||
out.stdout = raw_stdout.getvalue()
|
||||
out.return_code = output.return_code
|
||||
else:
|
||||
logger.debug("Running local command: %s", command)
|
||||
out.return_code, out.stdout, out.stderr = utils.execute(
|
||||
command)
|
||||
out.stdout = out.stdout.decode('utf-8')
|
||||
out.output = out.stdout
|
||||
except fabric.exceptions.NetworkError as e:
|
||||
logger.error("NetworkError occured: %s", str(e))
|
||||
raise
|
||||
except Exception as e:
|
||||
logger.error("Unexpected error occured: %s", str(e))
|
||||
out.stdout = raw_stdout.getvalue()
|
||||
return out
|
||||
|
||||
def get(self, path, target_path):
|
||||
"""Get remote or local file
|
||||
|
||||
target_path must be the directory where to put
|
||||
copied files or directories
|
||||
"""
|
||||
try:
|
||||
if not os.path.exists(target_path):
|
||||
os.makedirs(target_path)
|
||||
if self.dest_host:
|
||||
with fabric.api.settings(
|
||||
host_string=self.dest_host, # destination host
|
||||
key_filename=self.ssh_key, # a path to ssh key
|
||||
timeout=2, # connection timeout
|
||||
warn_only=True, # don't exit on error
|
||||
abort_on_prompts=True, # non-interactive mode
|
||||
):
|
||||
logger.debug("Getting remote file: %s %s",
|
||||
path, target_path)
|
||||
try:
|
||||
return fabric.api.get(path, target_path)
|
||||
except SystemExit:
|
||||
logger.error("Fabric aborted this iteration")
|
||||
else:
|
||||
# NOTE(mkwiek): We need to use shell ln instead of os.symlink
|
||||
# because of wildcards used in shotgun settings. ln utility
|
||||
# will nicely handle wildcards and create all needed symlinks
|
||||
# in target_path directory
|
||||
symlink_command = 'ln -s "{}" "{}"'.format(path, target_path)
|
||||
logger.debug(
|
||||
"Symlinking to local file: {}".format(symlink_command))
|
||||
return utils.execute(symlink_command)
|
||||
except fabric.exceptions.NetworkError as e:
|
||||
logger.error("NetworkError occured: %s", str(e))
|
||||
raise
|
||||
except Exception as e:
|
||||
logger.error("Unexpected error occured: %s", str(e))
|
||||
|
||||
|
||||
class File(Driver):
|
||||
|
||||
def __init__(self, data, conf):
|
||||
super(File, self).__init__(data, conf)
|
||||
self.path = data["path"]
|
||||
self.exclude = data.get('exclude', [])
|
||||
logger.debug("File to get: %s", self.path)
|
||||
self.target_path = str(os.path.join(
|
||||
self.conf.target, self.host,
|
||||
os.path.dirname(self.path).lstrip("/")))
|
||||
self.full_dst_path = os.path.join(
|
||||
self.conf.target, self.host,
|
||||
self.path.lstrip("/"))
|
||||
logger.debug("File to save: %s", self.target_path)
|
||||
|
||||
def snapshot(self):
|
||||
"""Make a snapshot
|
||||
|
||||
Example:
|
||||
self.conf.target IS /target
|
||||
self.host IS host.domain.tld
|
||||
self.path IS /var/log/somedir
|
||||
self.target_path IS /target/host.domain.tld/var/log
|
||||
"""
|
||||
self.get(self.path, self.target_path)
|
||||
|
||||
|
||||
Dir = File
|
||||
|
||||
|
||||
class Postgres(Driver):
|
||||
def __init__(self, data, conf):
|
||||
super(Postgres, self).__init__(data, conf)
|
||||
self.dbhost = data.get("dbhost", "localhost")
|
||||
self.dbname = data["dbname"]
|
||||
self.username = data.get("username", "postgres")
|
||||
self.password = data.get("password")
|
||||
self.target_path = str(os.path.join(self.conf.target,
|
||||
self.host, "pg_dump"))
|
||||
|
||||
def snapshot(self):
|
||||
if self.password:
|
||||
authline = "{host}:{port}:{dbname}:{username}:{password}".format(
|
||||
host=self.dbhost, port="5432", dbname=self.dbname,
|
||||
username=self.username, password=self.password)
|
||||
home_dir = pwd.getpwuid(os.getuid()).pw_dir
|
||||
pgpass = os.path.join(home_dir, ".pgpass")
|
||||
with open(pgpass, "a+") as fo:
|
||||
fo.seek(0)
|
||||
auth = False
|
||||
for line in fo:
|
||||
if re.search(ur"^{0}$".format(authline), line):
|
||||
auth = True
|
||||
break
|
||||
if not auth:
|
||||
fo.seek(0, 2)
|
||||
fo.write("{0}\n".format(authline))
|
||||
os.chmod(pgpass, stat.S_IRUSR + stat.S_IWUSR)
|
||||
temp = self.command("mktemp").output.strip()
|
||||
self.command("pg_dump -h {dbhost} -U {username} -w "
|
||||
"-f {file} {dbname}".format(
|
||||
dbhost=self.dbhost, username=self.username,
|
||||
file=temp, dbname=self.dbname))
|
||||
utils.execute('mkdir -p "{0}"'.format(self.target_path))
|
||||
dump_basename = "{0}_{1}.sql".format(self.dbhost, self.dbname)
|
||||
|
||||
utils.execute('mv -f "{0}" "{1}"'.format(
|
||||
temp,
|
||||
os.path.join(self.target_path, dump_basename)))
|
||||
|
||||
|
||||
class XmlRpc(Driver):
|
||||
def __init__(self, data, conf):
|
||||
super(XmlRpc, self).__init__(data, conf)
|
||||
|
||||
self.server = data.get("server", "localhost")
|
||||
self.methods = data.get("methods", [])
|
||||
self.to_file = data.get("to_file")
|
||||
|
||||
self.target_path = os.path.join(
|
||||
self.conf.target, self.host, "xmlrpc", self.to_file)
|
||||
|
||||
def snapshot(self):
|
||||
utils.execute('mkdir -p "{0}"'.format(os.path.dirname(
|
||||
self.target_path)))
|
||||
|
||||
server = xmlrpclib.Server(self.server)
|
||||
with open(self.target_path, "w") as f:
|
||||
for method in self.methods:
|
||||
if hasattr(server, method):
|
||||
response = getattr(server, method)()
|
||||
response = pprint.pformat(response, indent=2)
|
||||
else:
|
||||
response = "no such method on remote server"
|
||||
|
||||
f.write("===== {0} =====\n{1}\n\n".format(method, response))
|
||||
|
||||
|
||||
class Command(Driver):
|
||||
|
||||
def __init__(self, data, conf):
|
||||
super(Command, self).__init__(data, conf)
|
||||
if isinstance(data["command"], list):
|
||||
self.cmds = data["command"]
|
||||
else:
|
||||
self.cmds = [data["command"]]
|
||||
self.to_file = data.get("to_file", "/dev/null")
|
||||
self.target_path = os.path.join(
|
||||
self.conf.target, self.host, "commands", self.to_file)
|
||||
|
||||
def snapshot(self):
|
||||
for cmd in self.cmds:
|
||||
self._snapshot_single(cmd)
|
||||
|
||||
def _snapshot_single(self, cmd):
|
||||
out = self.command(cmd)
|
||||
utils.execute('mkdir -p "{0}"'.format(os.path.dirname(
|
||||
self.target_path)))
|
||||
with open(self.target_path, "a") as f:
|
||||
f.write("===== COMMAND =====: {0}\n".format(cmd))
|
||||
f.write("===== RETURN CODE =====: {0}\n".format(out.return_code))
|
||||
f.write("===== STDOUT =====:\n")
|
||||
if out.stdout:
|
||||
f.write(out.stdout)
|
||||
f.write("\n===== STDERR =====:\n")
|
||||
if out.stderr:
|
||||
f.write(out.stderr)
|
||||
|
||||
def report(self):
|
||||
for cmd in self.cmds:
|
||||
for report_line in self._report_single(cmd):
|
||||
yield report_line
|
||||
|
||||
def _report_single(self, cmd):
|
||||
return itertools.izip_longest(
|
||||
[self.host],
|
||||
cmd.split('\n'),
|
||||
self.command(cmd).stdout.split('\n'),
|
||||
fillvalue='')
|
||||
|
||||
|
||||
class DockerCommand(Command):
|
||||
def __init__(self, data, conf):
|
||||
super(DockerCommand, self).__init__(data, conf)
|
||||
self.cmds = [
|
||||
'docker exec \\\n'
|
||||
'$(docker ps -q \\\n'
|
||||
' --filter \'name={0}\' \\\n'
|
||||
' --format \'{{{{.Names}}}}\') '
|
||||
'{1}'.format(cnt, cmd)
|
||||
for cnt in data["containers"] for cmd in self.cmds]
|
||||
|
||||
|
||||
class Offline(Driver):
|
||||
|
||||
default_report_message = 'Network error occured. Host might be offline.'
|
||||
|
||||
def __init__(self, data, conf):
|
||||
super(Offline, self).__init__(data, conf)
|
||||
self.target_path = os.path.join(
|
||||
self.conf.target, self.host, "OFFLINE_NODE.txt")
|
||||
|
||||
def snapshot(self):
|
||||
if not os.path.exists(self.target_path):
|
||||
utils.execute('mkdir -p "{0}"'.format(os.path.dirname(
|
||||
self.target_path)))
|
||||
with open(self.target_path, "w") as f:
|
||||
f.write("Host {0} was offline/unreachable during "
|
||||
"logs obtaining.\n".format(self.host))
|
||||
|
|
@ -1,21 +0,0 @@
|
|||
# Copyright 2015 Mirantis, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
|
||||
def setup_hook(config):
|
||||
import pbr
|
||||
import pbr.packaging
|
||||
|
||||
# this monkey patch is to avoid appending git version to version
|
||||
pbr.packaging._get_version_from_git = lambda pre_version: pre_version
|
||||
|
|
@ -1,37 +0,0 @@
|
|||
# Copyright 2013 Mirantis, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
import logging
|
||||
|
||||
from shotgun.settings import LOG_FILE
|
||||
|
||||
|
||||
def configure_logger():
|
||||
"""Configures shotgun logger"""
|
||||
logger = logging.getLogger('shotgun')
|
||||
logger.setLevel(logging.DEBUG)
|
||||
formatter = logging.Formatter(
|
||||
'%(asctime)s %(levelname)s %(process)d (%(module)s) %(message)s',
|
||||
"%Y-%m-%d %H:%M:%S")
|
||||
|
||||
stream_handler = logging.StreamHandler()
|
||||
stream_handler.setLevel(logging.DEBUG)
|
||||
stream_handler.setFormatter(formatter)
|
||||
|
||||
file_handler = logging.FileHandler(LOG_FILE)
|
||||
file_handler.setLevel(logging.DEBUG)
|
||||
file_handler.setFormatter(formatter)
|
||||
|
||||
logger.addHandler(stream_handler)
|
||||
logger.addHandler(file_handler)
|
||||
|
|
@ -1,88 +0,0 @@
|
|||
# Copyright 2013 Mirantis, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
import errno
|
||||
import logging
|
||||
import os
|
||||
import shutil
|
||||
|
||||
import fabric.exceptions
|
||||
|
||||
from shotgun.driver import Driver
|
||||
from shotgun import utils
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class Manager(object):
|
||||
def __init__(self, conf):
|
||||
logger.debug("Initializing snapshot manager")
|
||||
self.conf = conf
|
||||
|
||||
def snapshot(self):
|
||||
logger.debug("Making snapshot")
|
||||
self.clear_target()
|
||||
exclusions = []
|
||||
try:
|
||||
for obj_data in self.conf.objects:
|
||||
logger.debug("Dumping: %s", obj_data)
|
||||
self.action_single(obj_data, action='snapshot')
|
||||
if 'exclude' in obj_data:
|
||||
exclusions += (
|
||||
os.path.join(obj_data['path'], ex).lstrip('/')
|
||||
for ex in obj_data['exclude']
|
||||
)
|
||||
|
||||
logger.debug("Dumping shotgun log "
|
||||
"and archiving dump directory: %s",
|
||||
self.conf.target)
|
||||
self.action_single(self.conf.self_log_object, action='snapshot')
|
||||
|
||||
utils.compress(self.conf.target, self.conf.compression_level,
|
||||
exclude=exclusions)
|
||||
|
||||
with open(self.conf.lastdump, "w") as fo:
|
||||
fo.write("{0}.tar.gz".format(self.conf.target))
|
||||
except IOError as e:
|
||||
if e.errno == errno.ENOSPC:
|
||||
logger.error("Not enough space in "
|
||||
"{} for snapshot".format(self.conf.target))
|
||||
self.clear_target()
|
||||
raise
|
||||
|
||||
return "{0}.tar.gz".format(self.conf.target)
|
||||
|
||||
def action_single(self, obj, action='snapshot'):
|
||||
driver = Driver.getDriver(obj, self.conf)
|
||||
try:
|
||||
return getattr(driver, action)()
|
||||
except fabric.exceptions.NetworkError:
|
||||
self.conf.on_network_error(obj)
|
||||
|
||||
def report(self):
|
||||
logger.debug("Making report")
|
||||
for obj_data in self.conf.objects:
|
||||
logger.debug("Gathering report for: %s", obj_data)
|
||||
for report in self.action_single(obj_data, action='report'):
|
||||
yield report
|
||||
|
||||
def clear_target(self):
|
||||
def on_rmtree_error(function, path, excinfo):
|
||||
msg = "Clearing target failed. function: {}, path: {}, excinfo: {}"
|
||||
logger.error(msg.format(function, path, excinfo))
|
||||
|
||||
if os.path.exists(self.conf.target):
|
||||
shutil.rmtree(os.path.dirname(self.conf.target),
|
||||
onerror=on_rmtree_error)
|
||||
|
|
@ -1,21 +0,0 @@
|
|||
# Copyright 2013 Mirantis, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
TARGET = "/tmp/snapshot"
|
||||
LASTDUMP = "/tmp/snapshot_last"
|
||||
TIMESTAMP = True
|
||||
COMPRESSION_LEVEL = 3
|
||||
LOG_FILE = "/var/log/shotgun.log"
|
||||
DEFAULT_TIMEOUT = 10
|
||||
ATTEMPTS = 2
|
||||
|
|
@ -1,13 +0,0 @@
|
|||
# Copyright 2013 Mirantis, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
|
@ -1,19 +0,0 @@
|
|||
# Copyright 2015 Mirantis, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
from oslotest import base as oslo_base
|
||||
|
||||
|
||||
class BaseTestCase(oslo_base.BaseTestCase):
|
||||
"""Base unit test case for shotgun tests."""
|
||||
|
|
@ -1,158 +0,0 @@
|
|||
# Copyright 2013 Mirantis, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
import time
|
||||
|
||||
import mock
|
||||
|
||||
from shotgun.config import Config
|
||||
from shotgun.test import base
|
||||
|
||||
|
||||
class TestConfig(base.BaseTestCase):
|
||||
|
||||
def test_timestamp(self):
|
||||
t = time.localtime()
|
||||
with mock.patch('shotgun.config.time') as MockedTime:
|
||||
MockedTime.localtime.return_value = t
|
||||
MockedTime.strftime.side_effect = time.strftime
|
||||
conf = Config({})
|
||||
stamped = conf._timestamp("sample")
|
||||
self.assertEqual(
|
||||
stamped,
|
||||
"sample-{0}".format(time.strftime('%Y-%m-%d_%H-%M-%S', t))
|
||||
)
|
||||
|
||||
def test_target_timestamp(self):
|
||||
conf = Config({
|
||||
"target": "/tmp/sample",
|
||||
"timestamp": True
|
||||
})
|
||||
self.assertRegex(
|
||||
conf.target,
|
||||
ur"\/tmp\/sample\-[\d]{4}\-[\d]{2}\-[\d]{2}_"
|
||||
"([\d]{2}\-){2}[\d]{2}",
|
||||
)
|
||||
|
||||
@mock.patch('shotgun.config.settings')
|
||||
def test_timeout(self, m_settings):
|
||||
conf = Config({})
|
||||
self.assertIs(conf.timeout, m_settings.DEFAULT_TIMEOUT)
|
||||
|
||||
def test_pass_default_timeout(self):
|
||||
timeout = 1345
|
||||
conf = Config({
|
||||
'timeout': timeout,
|
||||
})
|
||||
self.assertEqual(conf.timeout, timeout)
|
||||
|
||||
def test_on_network_error(self):
|
||||
data = {
|
||||
"dump": {
|
||||
"master": {
|
||||
"objects":
|
||||
[{"path": "/etc/nailgun",
|
||||
"type": "dir"},
|
||||
],
|
||||
"hosts": [{"ssh-key": "/root/.ssh/id_rsa",
|
||||
"address": "10.109.2.2"}]},
|
||||
}
|
||||
}
|
||||
conf = Config(data)
|
||||
obj = conf.objects.next()
|
||||
host = conf.get_network_address(obj)
|
||||
self.assertNotIn(obj, conf.try_again)
|
||||
self.assertNotIn(host, conf.offline_hosts)
|
||||
conf.on_network_error(obj)
|
||||
self.assertIn(obj, conf.try_again)
|
||||
self.assertIn(host, conf.offline_hosts)
|
||||
|
||||
def test_get_network_address(self):
|
||||
data = {
|
||||
"dump": {
|
||||
"master": {
|
||||
"objects":
|
||||
[{"path": "/etc/nailgun",
|
||||
"type": "dir"},
|
||||
],
|
||||
"hosts": [{"ssh-key": "/root/.ssh/id_rsa",
|
||||
"address": "10.109.2.2"}]},
|
||||
}
|
||||
}
|
||||
conf = Config(data)
|
||||
obj = conf.objects.next()
|
||||
self.assertEqual('10.109.2.2', conf.get_network_address(obj))
|
||||
|
||||
def test_get_network_address_hostname(self):
|
||||
hostname = "fuel.tld"
|
||||
data = {
|
||||
"dump": {
|
||||
"master": {
|
||||
"objects":
|
||||
[{"path": "/etc/nailgun",
|
||||
"type": "dir"},
|
||||
],
|
||||
"hosts": [{"ssh-key": "/root/.ssh/id_rsa",
|
||||
"hostname": hostname}]},
|
||||
}
|
||||
}
|
||||
conf = Config(data)
|
||||
obj = conf.objects.next()
|
||||
self.assertEqual(hostname, conf.get_network_address(obj))
|
||||
|
||||
def test_get_network_address_absent_address_and_hostname(self):
|
||||
data = {
|
||||
"dump": {
|
||||
"master": {
|
||||
"objects":
|
||||
[{"path": "/etc/nailgun",
|
||||
"type": "dir"}]},
|
||||
}
|
||||
}
|
||||
conf = Config(data)
|
||||
obj = conf.objects.next()
|
||||
self.assertIsNone(conf.get_network_address(obj))
|
||||
|
||||
def test_obj_without_hosts(self):
|
||||
data = {
|
||||
"dump": {
|
||||
"fake_role1": {
|
||||
"objects":
|
||||
[{"fake_obj_1": '1'}, {"fake_obj_2": '2'}]},
|
||||
}
|
||||
}
|
||||
conf = Config(data)
|
||||
expected_objs = [
|
||||
{'host': {}, 'fake_obj_1': '1'},
|
||||
{'host': {}, 'fake_obj_2': '2'}]
|
||||
self.assertItemsEqual(expected_objs, conf.objs)
|
||||
|
||||
def test_init(self):
|
||||
data = {
|
||||
"dump": {
|
||||
"fake_role1": {
|
||||
"objects":
|
||||
[{"fake_obj_1": '1'}, {"fake_obj_2": '2'}],
|
||||
"hosts": ["fake_host1", "fake_host2", "fake_host3"]},
|
||||
}
|
||||
}
|
||||
conf = Config(data)
|
||||
expected_objs = [
|
||||
{'host': 'fake_host1', 'fake_obj_1': '1'},
|
||||
{'host': 'fake_host1', 'fake_obj_2': '2'},
|
||||
{'host': 'fake_host2', 'fake_obj_1': '1'},
|
||||
{'host': 'fake_host2', 'fake_obj_2': '2'},
|
||||
{'host': 'fake_host3', 'fake_obj_1': '1'},
|
||||
{'host': 'fake_host3', 'fake_obj_2': '2'}]
|
||||
self.assertItemsEqual(expected_objs, conf.objs)
|
||||
|
|
@ -1,409 +0,0 @@
|
|||
# Copyright 2013 Mirantis, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
import itertools
|
||||
import os
|
||||
import random
|
||||
import sys
|
||||
|
||||
import fabric
|
||||
import mock
|
||||
|
||||
import shotgun
|
||||
from shotgun.test import base
|
||||
|
||||
|
||||
class RunOut(object):
|
||||
return_code = None
|
||||
stderr = None
|
||||
stdout = None
|
||||
|
||||
def __str__(self):
|
||||
return str(self.stdout)
|
||||
|
||||
|
||||
class TestDriver(base.BaseTestCase):
|
||||
def test_driver_factory(self):
|
||||
types = {
|
||||
"file": "File",
|
||||
"dir": "Dir",
|
||||
"postgres": "Postgres",
|
||||
"command": "Command"
|
||||
}
|
||||
for t, n in types.iteritems():
|
||||
with mock.patch("shotgun.driver.%s" % n) as mocked:
|
||||
shotgun.driver.Driver.getDriver({"type": t}, None)
|
||||
mocked.assert_called_with({"type": t}, None)
|
||||
|
||||
@mock.patch('shotgun.driver.utils.CCStringIO')
|
||||
@mock.patch('shotgun.driver.fabric.api.settings')
|
||||
@mock.patch('shotgun.driver.fabric.api.run')
|
||||
def test_driver_remote_command(self, mfabrun, mfabset, mccstring):
|
||||
out = shotgun.driver.CommandOut()
|
||||
out.stdout = "STDOUT"
|
||||
out.return_code = "RETURN_CODE"
|
||||
mccstring.return_value.getvalue.return_value = out.stdout
|
||||
|
||||
runout = RunOut()
|
||||
runout.return_code = "RETURN_CODE"
|
||||
mfabrun.return_value = runout
|
||||
|
||||
command = "COMMAND"
|
||||
|
||||
conf = mock.Mock()
|
||||
driver = shotgun.driver.Driver({
|
||||
"host": {
|
||||
"hostname": "remote_host",
|
||||
"address": "10.109.0.2"
|
||||
}
|
||||
}, conf)
|
||||
result = driver.command(command)
|
||||
|
||||
mfabrun.assert_called_with(
|
||||
command, stdout=mock.ANY)
|
||||
mfabset.assert_called_with(
|
||||
host_string="10.109.0.2",
|
||||
timeout=2,
|
||||
command_timeout=driver.timeout,
|
||||
warn_only=True,
|
||||
key_filename=None,
|
||||
abort_on_prompts=True,
|
||||
use_shell=True)
|
||||
self.assertEqual(result, out)
|
||||
|
||||
@mock.patch('shotgun.driver.fabric.api.run')
|
||||
@mock.patch('shotgun.driver.fabric.api.settings')
|
||||
def test_fabric_use_timout_from_driver(self, mfabset, _):
|
||||
timeout = random.randint(1, 100)
|
||||
conf = mock.Mock()
|
||||
driver = shotgun.driver.Driver({
|
||||
"host": {
|
||||
"hostname": "remote_host",
|
||||
"address": "10.109.0.2"
|
||||
}
|
||||
}, conf)
|
||||
driver.timeout = timeout
|
||||
driver.command("COMMAND")
|
||||
mfabset.assert_called_with(
|
||||
host_string=mock.ANY,
|
||||
timeout=mock.ANY,
|
||||
command_timeout=timeout,
|
||||
warn_only=mock.ANY,
|
||||
key_filename=mock.ANY,
|
||||
abort_on_prompts=mock.ANY,
|
||||
use_shell=mock.ANY)
|
||||
|
||||
@mock.patch('shotgun.driver.utils.execute')
|
||||
def test_driver_local_command(self, mexecute):
|
||||
mexecute.return_value = ("RETURN_CODE", "STDOUT", "STDERR")
|
||||
|
||||
out = shotgun.driver.CommandOut()
|
||||
out.stdout = "STDOUT"
|
||||
out.stderr = "STDERR"
|
||||
out.output = "STDOUT"
|
||||
out.return_code = "RETURN_CODE"
|
||||
|
||||
command = "COMMAND"
|
||||
conf = mock.Mock()
|
||||
driver = shotgun.driver.Driver({}, conf)
|
||||
result = driver.command(command)
|
||||
shotgun.driver.utils.execute.assert_called_with(command)
|
||||
self.assertEqual(result, out)
|
||||
|
||||
@mock.patch('shotgun.driver.utils.CCStringIO')
|
||||
@mock.patch('shotgun.driver.fabric.api.settings')
|
||||
@mock.patch('shotgun.driver.fabric.api.run')
|
||||
def test_command_timeout(self, mfabrun, mfabset, mstringio):
|
||||
mfabrun.side_effect = fabric.exceptions.CommandTimeout(10)
|
||||
|
||||
mstdout = mock.MagicMock()
|
||||
mstdout.getvalue.return_value = 'FULL STDOUT'
|
||||
mstringio.return_value = mstdout
|
||||
|
||||
command = "COMMAND"
|
||||
|
||||
conf = mock.Mock()
|
||||
driver = shotgun.driver.Driver({
|
||||
"host": {
|
||||
"hostname": "remote_host",
|
||||
"address": "10.109.0.2"
|
||||
}
|
||||
}, conf)
|
||||
result = driver.command(command)
|
||||
|
||||
mstringio.assert_has_calls([
|
||||
mock.call(writers=sys.stdout),
|
||||
])
|
||||
mfabrun.assert_called_with(command, stdout=mstdout)
|
||||
self.assertEqual(result.stdout, 'FULL STDOUT')
|
||||
|
||||
@mock.patch('os.path.exists', return_value=False)
|
||||
@mock.patch('os.makedirs')
|
||||
@mock.patch('shotgun.utils.execute')
|
||||
@mock.patch('shotgun.driver.fabric.api.settings')
|
||||
@mock.patch('shotgun.driver.fabric.api.get')
|
||||
def test_driver_get(self, mfabget, mfabset, mexecute, mmakedirs, _):
|
||||
mexecute.return_value = ("RETURN_CODE", "STDOUT", "STDERR")
|
||||
remote_path = "/remote_dir/remote_file"
|
||||
target_path = "/target_dir"
|
||||
conf = mock.Mock()
|
||||
|
||||
driver = shotgun.driver.Driver({
|
||||
"host": {
|
||||
"hostname": "remote_host",
|
||||
"address": "10.109.0.2",
|
||||
"ssh-key": "path_to_key",
|
||||
},
|
||||
}, conf)
|
||||
driver.get(remote_path, target_path)
|
||||
mmakedirs.assert_called_once_with(target_path)
|
||||
mfabget.assert_called_with(remote_path, target_path)
|
||||
|
||||
mfabset.assert_called_with(
|
||||
host_string="10.109.0.2", key_filename="path_to_key",
|
||||
timeout=2, warn_only=True, abort_on_prompts=True)
|
||||
|
||||
mexecute.reset_mock()
|
||||
mmakedirs.reset_mock()
|
||||
driver = shotgun.driver.Driver({}, conf)
|
||||
driver.get(remote_path, target_path)
|
||||
mmakedirs.assert_called_once_with(target_path)
|
||||
mexecute.assert_called_with('ln -s "{}" "{}"'.format(remote_path,
|
||||
target_path))
|
||||
|
||||
def test_use_timeout_from_global_conf(self):
|
||||
data = {}
|
||||
conf = mock.Mock(spec=shotgun.config.Config, target="some_target")
|
||||
cmd_driver = shotgun.driver.Driver(data, conf)
|
||||
self.assertEqual(cmd_driver.timeout, conf.timeout)
|
||||
|
||||
def test_use_command_specific_timeout(self):
|
||||
timeout = 1234
|
||||
data = {
|
||||
"timeout": timeout
|
||||
}
|
||||
conf = mock.Mock(spec=shotgun.config.Config, target="some_target")
|
||||
cmd_driver = shotgun.driver.Driver(data, conf)
|
||||
self.assertEqual(cmd_driver.timeout, timeout)
|
||||
self.assertNotEqual(cmd_driver.timeout, conf.timeout)
|
||||
|
||||
def test_host_when_host_is_specified(self):
|
||||
conf = mock.Mock(spec=shotgun.config.Config, target="some_target")
|
||||
hostname = 'example.com'
|
||||
driver = shotgun.driver.Driver({
|
||||
'host': {
|
||||
'hostname': hostname
|
||||
}
|
||||
}, conf)
|
||||
self.assertEqual(driver.host, hostname)
|
||||
|
||||
def test_host_when_addr_is_specified(self):
|
||||
conf = mock.Mock(spec=shotgun.config.Config, target="some_target")
|
||||
address = '198.51.100.2'
|
||||
driver = shotgun.driver.Driver({
|
||||
'host': {
|
||||
'address': address
|
||||
}
|
||||
}, conf)
|
||||
self.assertEqual(driver.host, address)
|
||||
|
||||
@mock.patch('shotgun.driver.socket.gethostname')
|
||||
def test_host_when_neither_addr_nor_hostname_is_specified(
|
||||
self, mock_hostname):
|
||||
|
||||
conf = mock.Mock(spec=shotgun.config.Config, target="some_target")
|
||||
sentinel = mock.sentinel
|
||||
mock_hostname.return_value = sentinel
|
||||
|
||||
driver = shotgun.driver.Driver({
|
||||
'host': {}
|
||||
}, conf)
|
||||
|
||||
self.assertEqual(driver.host, sentinel)
|
||||
self.assertEqual(mock_hostname.call_count, 1)
|
||||
|
||||
|
||||
class TestFile(base.BaseTestCase):
|
||||
|
||||
@mock.patch('shotgun.driver.Driver.get')
|
||||
def test_snapshot(self, mget):
|
||||
data = {
|
||||
"type": "file",
|
||||
"path": "/remote_dir/remote_file",
|
||||
"host": {
|
||||
"hostname": "remote_host",
|
||||
"address": "10.109.0.2",
|
||||
},
|
||||
}
|
||||
conf = mock.MagicMock()
|
||||
conf.target = "/target"
|
||||
file_driver = shotgun.driver.File(data, conf)
|
||||
|
||||
target_path = "/target/remote_host/remote_dir"
|
||||
file_driver.snapshot()
|
||||
|
||||
mget.assert_called_with(data["path"], target_path)
|
||||
|
||||
|
||||
class TestCommand(base.BaseTestCase):
|
||||
def setUp(self):
|
||||
super(TestCommand, self).setUp()
|
||||
|
||||
self.conf = mock.Mock()
|
||||
self.conf.target = '/some/dir'
|
||||
|
||||
def test_init(self):
|
||||
data = {
|
||||
"host": {"hostname": "somehost"},
|
||||
"command": "some command",
|
||||
"to_file": "some_command.txt"
|
||||
}
|
||||
driver_inst = shotgun.driver.Command(data, self.conf)
|
||||
self.assertListEqual(["some command"], driver_inst.cmds)
|
||||
self.assertEqual("some_command.txt", driver_inst.to_file)
|
||||
self.assertEqual(os.path.join("/some/dir", "somehost",
|
||||
"commands", "some_command.txt"),
|
||||
driver_inst.target_path)
|
||||
data = {
|
||||
"host": {"hostname": "somehost"},
|
||||
"command": ["cmd1", "cmd2"],
|
||||
"to_file": "some_command.txt"
|
||||
}
|
||||
driver_inst = shotgun.driver.Command(data, self.conf)
|
||||
self.assertListEqual(["cmd1", "cmd2"], driver_inst.cmds)
|
||||
|
||||
@mock.patch('shotgun.driver.Command._snapshot_single')
|
||||
def test_snapshot(self, msnap_sing):
|
||||
data = {
|
||||
"command": ["cmd1", "cmd2"],
|
||||
}
|
||||
driver_inst = shotgun.driver.Command(data, self.conf)
|
||||
driver_inst.snapshot()
|
||||
expected = [mock.call("cmd1"), mock.call("cmd2")]
|
||||
self.assertListEqual(expected, msnap_sing.call_args_list)
|
||||
|
||||
@mock.patch('shotgun.driver.open', create=True,
|
||||
new_callable=mock.mock_open)
|
||||
@mock.patch('shotgun.driver.Command.command')
|
||||
@mock.patch('shotgun.utils.execute')
|
||||
def test_snapshot_single(self, mexec, mcom, mopen):
|
||||
mout = mock.Mock()
|
||||
mout.return_code = 0
|
||||
mout.stdout = "stdout"
|
||||
mout.stderr = "stderr"
|
||||
mcom.return_value = mout
|
||||
driver_inst = shotgun.driver.Command({"command": "cmd"}, self.conf)
|
||||
driver_inst._snapshot_single("cmd")
|
||||
expected_write = [
|
||||
mock.call("===== COMMAND =====: cmd\n"),
|
||||
mock.call("===== RETURN CODE =====: 0\n"),
|
||||
mock.call("===== STDOUT =====:\n"),
|
||||
mock.call("stdout"),
|
||||
mock.call("\n===== STDERR =====:\n"),
|
||||
mock.call("stderr"),
|
||||
]
|
||||
file_handle_mock = mopen.return_value.__enter__.return_value
|
||||
self.assertListEqual(expected_write,
|
||||
file_handle_mock.write.call_args_list)
|
||||
|
||||
@mock.patch('shotgun.driver.Command._report_single')
|
||||
def test_report(self, mrepsing):
|
||||
data = {
|
||||
"host": {"hostname": "somehost"},
|
||||
"command": ["cmd1", "cmd2"],
|
||||
}
|
||||
driver_inst = shotgun.driver.Command(data, self.conf)
|
||||
reports = [['r1'], ['r2', 'r3']]
|
||||
mrepsing.side_effect = reports
|
||||
replines = []
|
||||
for repline in driver_inst.report():
|
||||
replines.append(repline)
|
||||
self.assertListEqual(list(itertools.chain(*reports)), replines)
|
||||
|
||||
@mock.patch('shotgun.driver.Command.command')
|
||||
def test_report_single(self, mcom):
|
||||
command_retval = mock.Mock()
|
||||
mcom.return_value = command_retval
|
||||
command_retval.stdout = "r1\nr2\nr3"
|
||||
data = {
|
||||
"host": {"hostname": "somehost"},
|
||||
"command": "cmd1\ncmd2",
|
||||
}
|
||||
driver_inst = shotgun.driver.Command(data, self.conf)
|
||||
expected = [
|
||||
("somehost", "cmd1", "r1"),
|
||||
("", "cmd2", "r2"),
|
||||
("", "", "r3")
|
||||
]
|
||||
result = driver_inst._report_single(data["command"])
|
||||
self.assertListEqual(expected, list(result))
|
||||
|
||||
|
||||
class TestDockerCommand(base.BaseTestCase):
|
||||
def setUp(self):
|
||||
super(TestDockerCommand, self).setUp()
|
||||
|
||||
self.conf = mock.Mock()
|
||||
self.conf.target = '/some/dir'
|
||||
|
||||
def test_init(self):
|
||||
data = {
|
||||
"command": ["cmd1", "cmd2"],
|
||||
"containers": ["cont1", "cont2"],
|
||||
}
|
||||
driver_inst = shotgun.driver.DockerCommand(data, self.conf)
|
||||
template_str = (
|
||||
'docker exec \\\n'
|
||||
'$(docker ps -q \\\n'
|
||||
' --filter \'name={0}\' \\\n'
|
||||
' --format \'{{{{.Names}}}}\') {1}')
|
||||
expected = [
|
||||
template_str.format("cont1", "cmd1"),
|
||||
template_str.format("cont1", "cmd2"),
|
||||
template_str.format("cont2", "cmd1"),
|
||||
template_str.format("cont2", "cmd2"),
|
||||
]
|
||||
self.assertListEqual(expected, driver_inst.cmds)
|
||||
|
||||
|
||||
class TestOffline(base.BaseTestCase):
|
||||
|
||||
@mock.patch('shotgun.driver.open', create=True,
|
||||
new_callable=mock.mock_open)
|
||||
@mock.patch('shotgun.driver.utils.execute', autospec=True)
|
||||
@mock.patch('shotgun.driver.os', autospec=True)
|
||||
def test_snapshot(self, mos, mexec, mopen):
|
||||
data = {
|
||||
"type": "offline",
|
||||
"path": "/remote_dir/remote_file",
|
||||
"host": {
|
||||
"hostname": "remote_host",
|
||||
"address": "10.109.0.2",
|
||||
},
|
||||
}
|
||||
conf = mock.MagicMock()
|
||||
conf.target = "/target"
|
||||
target_path = "/target/remote_host/OFFLINE_NODE.txt"
|
||||
mos.path.exists.return_value = False
|
||||
mos.path.dirname.return_value = '/target/remote_host'
|
||||
mos.path.join.return_value = target_path
|
||||
offline_driver = shotgun.driver.Offline(data, conf)
|
||||
offline_driver.snapshot()
|
||||
file_handle_mock = mopen.return_value.__enter__.return_value
|
||||
file_handle_mock.write.assert_called_once_with(
|
||||
'Host remote_host was offline/unreachable '
|
||||
'during logs obtaining.\n')
|
||||
mopen.assert_called_once_with(target_path, 'w')
|
||||
mexec.assert_called_once_with('mkdir -p "/target/remote_host"')
|
||||
self.assertEqual(target_path, offline_driver.target_path)
|
||||
|
|
@ -1,171 +0,0 @@
|
|||
# Copyright 2015 Mirantis, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
from collections import deque
|
||||
import tempfile
|
||||
|
||||
import fabric.exceptions
|
||||
import mock
|
||||
|
||||
from shotgun.config import Config
|
||||
from shotgun.manager import Manager
|
||||
from shotgun.test import base
|
||||
|
||||
|
||||
class TestManager(base.BaseTestCase):
|
||||
|
||||
@mock.patch('shotgun.manager.Driver.getDriver')
|
||||
@mock.patch('shotgun.manager.utils.compress')
|
||||
def test_snapshot(self, mcompress, mget):
|
||||
data = {
|
||||
"type": "file",
|
||||
"path": "/remote_dir/remote_file",
|
||||
"host": {
|
||||
"address": "remote_host",
|
||||
},
|
||||
}
|
||||
conf = mock.MagicMock()
|
||||
conf.target = "/target/data"
|
||||
conf.objects = [data]
|
||||
conf.lastdump = tempfile.mkstemp()[1]
|
||||
conf.self_log_object = {"type": "file", "path": "/path"}
|
||||
manager = Manager(conf)
|
||||
manager.snapshot()
|
||||
calls = [mock.call(data, conf), mock.call(conf.self_log_object, conf)]
|
||||
mget.assert_has_calls(calls, any_order=True)
|
||||
|
||||
@mock.patch('shotgun.manager.Driver.getDriver')
|
||||
@mock.patch('shotgun.manager.utils.compress')
|
||||
def test_snapshot_network_error(self, mcompress, mget):
|
||||
objs = [
|
||||
{"type": "file",
|
||||
"path": "/remote_file1",
|
||||
"host": {"address": "remote_host1"},
|
||||
},
|
||||
{"type": "dir",
|
||||
"path": "/remote_dir1",
|
||||
"host": {"address": "remote_host1"},
|
||||
},
|
||||
{"type": "file",
|
||||
"path": "/remote_file1",
|
||||
"host": {"address": "remote_host2"},
|
||||
},
|
||||
]
|
||||
drv = mock.MagicMock()
|
||||
drv.snapshot.side_effect = [
|
||||
fabric.exceptions.NetworkError,
|
||||
None,
|
||||
fabric.exceptions.NetworkError,
|
||||
None,
|
||||
None,
|
||||
]
|
||||
mget.return_value = drv
|
||||
conf = Config()
|
||||
conf.objs = deque(objs)
|
||||
offline_obj = {
|
||||
'path': '/remote_file1',
|
||||
'host': {'address': 'remote_host1'},
|
||||
'type': 'offline',
|
||||
}
|
||||
processed_obj = {
|
||||
'path': '/remote_file1',
|
||||
'host': {'address': 'remote_host2'},
|
||||
'type': 'file',
|
||||
}
|
||||
manager = Manager(conf)
|
||||
manager.snapshot()
|
||||
mget.assert_has_calls([mock.call(offline_obj, conf),
|
||||
mock.call(processed_obj, conf),
|
||||
mock.call(offline_obj, conf),
|
||||
mock.call(offline_obj, conf)], any_order=True)
|
||||
|
||||
@mock.patch('shotgun.manager.Manager.action_single')
|
||||
def test_report(self, mock_action):
|
||||
objs = ["o1", "o2"]
|
||||
mock_action.side_effect = [["r1", "r2"], ["r3"]]
|
||||
conf = mock.Mock()
|
||||
conf.objects = objs
|
||||
manager = Manager(conf)
|
||||
manager.action_single = mock_action
|
||||
reports = []
|
||||
for rep in manager.report():
|
||||
reports.append(rep)
|
||||
self.assertEqual(["r1", "r2", "r3"], reports)
|
||||
|
||||
expected_calls = [
|
||||
mock.call('o1', action='report'),
|
||||
mock.call('o2', action='report')]
|
||||
self.assertEqual(expected_calls, mock_action.call_args_list)
|
||||
|
||||
@mock.patch('shotgun.manager.Driver')
|
||||
def test_action_single(self, mock_driver):
|
||||
mock_driver_instance = mock.Mock()
|
||||
mock_driver_instance.report = mock.Mock()
|
||||
mock_driver_instance.snapshot = mock.Mock()
|
||||
mock_driver.getDriver = mock.Mock(return_value=mock_driver_instance)
|
||||
manager = Manager('conf')
|
||||
|
||||
manager.action_single('object', action='report')
|
||||
mock_driver.getDriver.assert_called_once_with('object', 'conf')
|
||||
mock_driver_instance.report.assert_called_once_with()
|
||||
self.assertFalse(mock_driver_instance.snapshot.called)
|
||||
mock_driver.getDriver.reset_mock()
|
||||
|
||||
# default action should be 'snapshot'
|
||||
manager.action_single('object')
|
||||
mock_driver.getDriver.assert_called_once_with('object', 'conf')
|
||||
mock_driver_instance.report.mock_reset()
|
||||
mock_driver_instance.snapshot.assert_called_once_with()
|
||||
self.assertFalse(mock_driver_instance.report.called)
|
||||
|
||||
@mock.patch('shotgun.manager.Manager.action_single')
|
||||
def test_snapshot_rm_without_disk_space(self, mock_action):
|
||||
mock_action.side_effect = IOError(28, "Not enough space")
|
||||
|
||||
data = {
|
||||
"type": "file",
|
||||
"path": "/remote_dir/remote_file",
|
||||
"host": {
|
||||
"address": "remote_host",
|
||||
},
|
||||
}
|
||||
conf = mock.MagicMock()
|
||||
conf.target = "/target/data"
|
||||
conf.objects = [data]
|
||||
conf.lastdump = tempfile.mkstemp()[1]
|
||||
conf.self_log_object = {"type": "file", "path": "/path"}
|
||||
manager = Manager(conf)
|
||||
|
||||
self.assertRaises(IOError, manager.snapshot)
|
||||
|
||||
@mock.patch('shotgun.manager.Manager.action_single')
|
||||
def test_snapshot_doesnt_clean_on_generic_ioerror(self,
|
||||
mock_action):
|
||||
mock_action.side_effect = IOError(1, "Generic error")
|
||||
|
||||
data = {
|
||||
"type": "file",
|
||||
"path": "/remote_dir/remote_file",
|
||||
"host": {
|
||||
"address": "remote_host",
|
||||
},
|
||||
}
|
||||
conf = mock.MagicMock()
|
||||
conf.target = "/target/data"
|
||||
conf.objects = [data]
|
||||
conf.lastdump = tempfile.mkstemp()[1]
|
||||
conf.self_log_object = {"type": "file", "path": "/path"}
|
||||
manager = Manager(conf)
|
||||
|
||||
self.assertRaises(IOError, manager.snapshot)
|
||||
|
|
@ -1,134 +0,0 @@
|
|||
# coding: utf-8
|
||||
|
||||
# Copyright 2015 Mirantis, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
import StringIO
|
||||
|
||||
import mock
|
||||
|
||||
from shotgun.test import base
|
||||
from shotgun import utils
|
||||
|
||||
|
||||
class TestUtils(base.BaseTestCase):
|
||||
|
||||
@mock.patch('shotgun.utils.execute')
|
||||
def test_remove_subdir(self, mexecute):
|
||||
utils.remove('/', ['good', '**/*.py'])
|
||||
mexecute.assert_has_calls([
|
||||
mock.call('shopt -s globstar; rm -rf /good'),
|
||||
mock.call('shopt -s globstar; rm -rf /**/*.py')])
|
||||
|
||||
@mock.patch('shotgun.utils.os.walk')
|
||||
def test_iterfiles(self, mwalk):
|
||||
path = '/root'
|
||||
mwalk.return_value = [
|
||||
(path, '', ('file1', 'file2')),
|
||||
(path + '/sub', '', ('file3',))]
|
||||
|
||||
result = list(utils.iterfiles(path))
|
||||
|
||||
mwalk.assert_called_once_with(path, topdown=True)
|
||||
self.assertEqual(
|
||||
result, ['/root/file1', '/root/file2', '/root/sub/file3'])
|
||||
|
||||
@mock.patch('shotgun.utils.execute')
|
||||
def test_compress(self, mexecute):
|
||||
target = '/path/target'
|
||||
level = '-3'
|
||||
|
||||
mexecute.return_value = (None, None, None)
|
||||
|
||||
utils.compress(target, level)
|
||||
|
||||
compress_call = mexecute.call_args_list[0]
|
||||
rm_call = mexecute.call_args_list[1]
|
||||
|
||||
compress_env = compress_call[1]['env']
|
||||
self.assertEqual(compress_env['XZ_OPT'], level)
|
||||
self.assertEqual(
|
||||
compress_call[0][0],
|
||||
'tar chzf /path/target.tar.gz -C /path target')
|
||||
|
||||
self.assertEqual(rm_call[0][0], 'rm -r /path/target')
|
||||
|
||||
@mock.patch('shotgun.utils.execute')
|
||||
def test_compress_exclude(self, mexecute):
|
||||
target = '/path/target'
|
||||
level = '-3'
|
||||
|
||||
exclusions = ['/path/to/exclude1', '/path/to/exclude2']
|
||||
|
||||
mexecute.return_value = (None, None, None)
|
||||
|
||||
utils.compress(target, level, exclude=exclusions)
|
||||
|
||||
compress_call = mexecute.call_args_list[0]
|
||||
|
||||
compress_env = compress_call[1]['env']
|
||||
self.assertEqual(compress_env['XZ_OPT'], level)
|
||||
self.assertEqual(
|
||||
compress_call[0][0],
|
||||
'tar chzf /path/target.tar.gz -C /path target '
|
||||
'--exclude=/path/to/exclude1 --exclude=/path/to/exclude2')
|
||||
|
||||
|
||||
class TestCCStringIO(base.BaseTestCase):
|
||||
|
||||
def test_no_writers(self):
|
||||
test_string = 'some_string'
|
||||
|
||||
ccstring = utils.CCStringIO()
|
||||
ccstring.write(test_string)
|
||||
|
||||
self.assertEqual(ccstring.getvalue(), test_string)
|
||||
|
||||
def test_with_one_writer(self):
|
||||
test_string = 'some_string'
|
||||
|
||||
writer = StringIO.StringIO()
|
||||
ccstring = utils.CCStringIO(writers=writer)
|
||||
ccstring.write(test_string)
|
||||
|
||||
self.assertEqual(ccstring.getvalue(), test_string)
|
||||
self.assertEqual(writer.getvalue(), test_string)
|
||||
|
||||
def test_with_multiple_writers(self):
|
||||
test_string = 'some_string'
|
||||
|
||||
writer_a = StringIO.StringIO()
|
||||
writer_b = StringIO.StringIO()
|
||||
ccstring = utils.CCStringIO(writers=[writer_a, writer_b])
|
||||
ccstring.write(test_string)
|
||||
|
||||
self.assertEqual(ccstring.getvalue(), test_string)
|
||||
self.assertEqual(writer_a.getvalue(), test_string)
|
||||
self.assertEqual(writer_b.getvalue(), test_string)
|
||||
|
||||
def test_with_writer_and_buffer(self):
|
||||
buffer = 'I am here already'
|
||||
|
||||
writer = StringIO.StringIO()
|
||||
ccstring = utils.CCStringIO(buffer, writers=writer)
|
||||
|
||||
self.assertEqual(ccstring.getvalue(), buffer)
|
||||
self.assertEqual(writer.getvalue(), '')
|
||||
|
||||
def test_non_ascii_output_with_unicode(self):
|
||||
ccstring = utils.CCStringIO()
|
||||
ccstring.write('привет')
|
||||
ccstring.write(u'test')
|
||||
|
||||
self.assertEqual(ccstring.getvalue(), 'приветtest')
|
||||
145
shotgun/utils.py
145
shotgun/utils.py
|
|
@ -1,145 +0,0 @@
|
|||
# Copyright 2013 Mirantis, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
import copy
|
||||
import errno
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import socket
|
||||
from StringIO import StringIO
|
||||
import subprocess
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def hostname():
|
||||
return socket.gethostname()
|
||||
|
||||
|
||||
def is_ip(name):
|
||||
return (re.search(ur"([0-9]{1,3}\.){3}[0-9]{1,3}", name) and True)
|
||||
|
||||
|
||||
def fqdn(name=None):
|
||||
if name:
|
||||
return socket.getfqdn(name)
|
||||
return socket.getfqdn(socket.gethostname())
|
||||
|
||||
|
||||
def iterfiles(path):
|
||||
for root, dirnames, filenames in os.walk(path, topdown=True):
|
||||
for filename in filenames:
|
||||
yield os.path.join(root, filename)
|
||||
|
||||
|
||||
def remove(full_dst_path, excludes):
|
||||
"""Removes subdirs/files using unixs syntax
|
||||
|
||||
full_dst_path is treated as root directory for remove
|
||||
|
||||
:param full_dst_path: str
|
||||
:param excludes: list with excludes paths/files
|
||||
"""
|
||||
for exclude in excludes:
|
||||
path = os.path.join(full_dst_path, exclude.lstrip('/'))
|
||||
logger.debug('Deleting %s', path)
|
||||
execute("shopt -s globstar; rm -rf {0}".format(path))
|
||||
|
||||
|
||||
def is_out_of_space(code, errdata):
|
||||
if code:
|
||||
# Since compression command is actually a pipeline
|
||||
# we cannot use return code to detect out of space condition
|
||||
# so, the only way is to parse stderr
|
||||
return errdata.lower().find('no space left'.encode()) >= 0
|
||||
|
||||
|
||||
def compress(target, level, keep_target=False, exclude=None):
|
||||
"""Runs compression of provided directory
|
||||
|
||||
:param target: directory to compress
|
||||
:param level: level of compression
|
||||
:param keep_target: bool, if True target directory wont be removed
|
||||
"""
|
||||
if exclude is None:
|
||||
exclude = []
|
||||
|
||||
env = copy.deepcopy(os.environ)
|
||||
env['XZ_OPT'] = level
|
||||
env['LANG'] = 'C' # We need non localized output
|
||||
code, _, errdata = execute(
|
||||
"tar chzf {0}.tar.gz -C {1} {2}{3}"
|
||||
"".format(target,
|
||||
os.path.dirname(target),
|
||||
os.path.basename(target),
|
||||
"".join(' --exclude={}'.format(e) for e in exclude)),
|
||||
env=env)
|
||||
if not keep_target:
|
||||
execute("rm -r {0}".format(target))
|
||||
if is_out_of_space(code, errdata):
|
||||
raise IOError(errno.ENOSPC)
|
||||
|
||||
|
||||
def execute(command, env=None):
|
||||
logger.debug("Trying to execute command: %s", command)
|
||||
|
||||
env = env or os.environ
|
||||
env["PATH"] = "/bin:/usr/bin:/sbin:/usr/sbin"
|
||||
|
||||
process = subprocess.Popen(
|
||||
command, env=env, stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE, shell=True)
|
||||
stdout, stderr = process.communicate()
|
||||
return (process.poll(), stdout, stderr)
|
||||
|
||||
|
||||
class CCStringIO(StringIO):
|
||||
"""A "carbon copy" StringIO.
|
||||
|
||||
It's capable of multiplexing its writes to other buffer objects.
|
||||
|
||||
Taken from fabric.tests.mock_streams.CarbonCopy
|
||||
"""
|
||||
|
||||
def __init__(self, buffer='', writers=None):
|
||||
"""CCStringIO initializator
|
||||
|
||||
If ``writers`` is given and is a file-like object or an
|
||||
iterable of same, it/they will be written to whenever this
|
||||
StringIO instance is written to.
|
||||
"""
|
||||
StringIO.__init__(self, buffer)
|
||||
if writers is None:
|
||||
writers = []
|
||||
elif hasattr(writers, 'write'):
|
||||
writers = [writers]
|
||||
self.writers = writers
|
||||
|
||||
def write(self, s):
|
||||
# unfortunately, fabric writes into StringIO both so-called
|
||||
# bytestrings and unicode strings. obviously, bytestrings may
|
||||
# contain non-ascii symbols. that leads to type-conversion
|
||||
# issue when we use string's join (inside getvalue()) with
|
||||
# a list of both unicodes and bytestrings. in order to avoid
|
||||
# this issue we should convert all input unicode strings into
|
||||
# utf-8 bytestrings (let's assume that slaves encoding is utf-8
|
||||
# too so we won't have encoding mess in the output file).
|
||||
if isinstance(s, unicode):
|
||||
s = s.encode('utf-8')
|
||||
|
||||
StringIO.write(self, s)
|
||||
for writer in self.writers:
|
||||
writer.write(s)
|
||||
|
|
@ -1,50 +0,0 @@
|
|||
%define name shotgun
|
||||
%{!?version: %define version 10.0.0}
|
||||
%{!?release: %define release 1}
|
||||
|
||||
Name: %{name}
|
||||
Summary: Shotgun package
|
||||
Version: %{version}
|
||||
Release: %{release}
|
||||
Source0: %{name}-%{version}.tar.gz
|
||||
URL: http://mirantis.com
|
||||
License: Apache
|
||||
Group: Development/Libraries
|
||||
BuildRoot: %{_tmppath}/%{name}-%{version}-buildroot
|
||||
Prefix: %{_prefix}
|
||||
BuildRequires: python-setuptools
|
||||
BuildRequires: python-pbr >= 1.8
|
||||
Requires: postgresql
|
||||
Requires: python-cliff >= 1.7.0
|
||||
Requires: python-fabric >= 1.10.0
|
||||
Requires: python-argparse
|
||||
Requires: python-six >= 1.9.0
|
||||
Requires: tar
|
||||
Requires: gzip
|
||||
Requires: bzip2
|
||||
Requires: openssh-clients
|
||||
Requires: xz
|
||||
BuildArch: noarch
|
||||
|
||||
%description
|
||||
Shotgun package.
|
||||
|
||||
%prep
|
||||
%setup -cq -n %{name}-%{version}
|
||||
|
||||
%build
|
||||
cd %{_builddir}/%{name}-%{version} && python setup.py build
|
||||
|
||||
%install
|
||||
cd %{_builddir}/%{name}-%{version} && python setup.py install --single-version-externally-managed -O1 --root=$RPM_BUILD_ROOT --record=%{_builddir}/%{name}-%{version}/INSTALLED_FILES
|
||||
install -d -m 755 %{buildroot}%{_sysconfdir}/shotgun
|
||||
install -p -D -m 644 %{_builddir}/%{name}-%{version}/etc/report.yaml %{buildroot}%{_sysconfdir}/shotgun/report.yaml
|
||||
install -p -D -m 644 %{_builddir}/%{name}-%{version}/etc/short_report.yaml %{buildroot}%{_sysconfdir}/shotgun/short_report.yaml
|
||||
|
||||
%clean
|
||||
rm -rf $RPM_BUILD_ROOT
|
||||
|
||||
%files -f %{_builddir}/%{name}-%{version}/INSTALLED_FILES
|
||||
%defattr(-,root,root)
|
||||
%{_sysconfdir}/shotgun/report.yaml
|
||||
%{_sysconfdir}/shotgun/short_report.yaml
|
||||
|
|
@ -1,8 +0,0 @@
|
|||
# The order of packages is significant, because pip processes them in the order
|
||||
# of appearance. Changing the order has an impact on the overall integration
|
||||
# process, which may cause wedges in the gate later.
|
||||
hacking<0.11,>=0.10.2 # Apache-2.0
|
||||
coverage>=3.6 # Apache-2.0
|
||||
os-testr>=0.4.1 # Apache-2.0
|
||||
oslotest>=1.10.0 # Apache-2.0
|
||||
mock>=1.2 # BSD
|
||||
38
tox.ini
38
tox.ini
|
|
@ -1,38 +0,0 @@
|
|||
[tox]
|
||||
minversion = 1.6
|
||||
skipsdist = True
|
||||
envlist = py27,pep8
|
||||
|
||||
[testenv]
|
||||
usedevelop = True
|
||||
install_command = pip install --allow-external -U {opts} {packages}
|
||||
setenv = VIRTUAL_ENV={envdir}
|
||||
deps = -r{toxinidir}/requirements.txt
|
||||
-r{toxinidir}/test-requirements.txt
|
||||
commands =
|
||||
ostestr {posargs}
|
||||
|
||||
[testenv:pep8]
|
||||
commands =
|
||||
flake8 {posargs:shotgun}
|
||||
|
||||
[testenv:cover]
|
||||
setenv = VIRTUAL_ENV={envdir}
|
||||
commands =
|
||||
python setup.py test --coverage {posargs}
|
||||
|
||||
[testenv:venv]
|
||||
commands = {posargs:}
|
||||
|
||||
[testenv:devenv]
|
||||
envdir = devenv
|
||||
usedevelop = True
|
||||
|
||||
[flake8]
|
||||
exclude = .venv,.git,.tox,dist,doc,*lib/python*,*egg,build,tools,__init__.py,docs
|
||||
show-pep8 = True
|
||||
show-source = True
|
||||
count = True
|
||||
|
||||
[hacking]
|
||||
import_exceptions = testtools.matchers
|
||||
Loading…
Reference in New Issue