Merge tox, tests and other support files

Change-Id: I5a4759e36089f1f4fab0c75412c94d051d8b16a7
This commit is contained in:
Monty Taylor 2017-10-04 13:06:22 -05:00
parent 65293358a0
commit a4ee1a3f09
No known key found for this signature in database
GPG Key ID: 7BAE94BC7141A594
259 changed files with 1049 additions and 2203 deletions

1
.gitignore vendored
View File

@ -29,6 +29,7 @@ cover/*
.tox
nosetests.xml
.testrepository
.stestr
# Translations
*.mo

View File

@ -1,3 +1,6 @@
# Format is:
# <preferred e-mail> <other e-mail 1>
# <preferred e-mail> <other e-mail 2>
# <preferred e-mail> <other e-mail 2>
<corvus@inaugust.com> <jeblair@redhat.com>
<corvus@inaugust.com> <jeblair@linux.vnet.ibm.com>
<corvus@inaugust.com> <jeblair@hp.com>

3
.stestr.conf Normal file
View File

@ -0,0 +1,3 @@
[DEFAULT]
test_path=./openstack/cloud/tests/unit
top_dir=./

View File

@ -1,8 +0,0 @@
[DEFAULT]
test_command=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-60} \
${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./openstack/tests/unit} $LISTOPT $IDOPTION
test_id_option=--load-list $IDFILE
test_list_option=--list
group_regex=([^\.]+\.)+

View File

@ -1,16 +1,45 @@
If you would like to contribute to the development of OpenStack,
you must follow the steps in this page:
.. _contributing:
http://docs.openstack.org/infra/manual/developers.html
===================================
Contributing to python-openstacksdk
===================================
Once those steps have been completed, changes to OpenStack
should be submitted for review via the Gerrit tool, following
the workflow documented at:
If you're interested in contributing to the python-openstacksdk project,
the following will help get you started.
http://docs.openstack.org/infra/manual/developers.html#development-workflow
Contributor License Agreement
-----------------------------
.. index::
single: license; agreement
In order to contribute to the python-openstacksdk project, you need to have
signed OpenStack's contributor's agreement.
Please read `DeveloperWorkflow`_ before sending your first patch for review.
Pull requests submitted through GitHub will be ignored.
Bugs should be filed on Launchpad, not GitHub:
.. seealso::
https://bugs.launchpad.net/python-openstacksdk
* http://wiki.openstack.org/HowToContribute
* http://wiki.openstack.org/CLA
.. _DeveloperWorkflow: http://docs.openstack.org/infra/manual/developers.html#development-workflow
Project Hosting Details
-------------------------
Project Documentation
http://docs.openstack.org/sdks/python/openstacksdk/
Bug tracker
http://storyboard.openstack.org
Mailing list (prefix subjects with ``[sdk]`` for faster responses)
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Code Hosting
https://git.openstack.org/cgit/openstack/python-openstacksdk
Code Review
https://review.openstack.org/#/q/status:open+project:openstack/python-openstacksdk,n,z

View File

@ -1,4 +1,49 @@
python-openstacksdk Style Commandments
======================================
openstacksdk Style Commandments
===============================
Read the OpenStack Style Commandments https://docs.openstack.org/hacking/latest/
Read the OpenStack Style Commandments
http://docs.openstack.org/developer/hacking/
Indentation
-----------
PEP-8 allows for 'visual' indentation. Do not use it. Visual indentation looks
like this:
.. code-block:: python
return_value = self.some_method(arg1, arg1,
arg3, arg4)
Visual indentation makes refactoring the code base unneccesarily hard.
Instead of visual indentation, use this:
.. code-block:: python
return_value = self.some_method(
arg1, arg1, arg3, arg4)
That way, if some_method ever needs to be renamed, the only line that needs
to be touched is the line with some_method. Additionaly, if you need to
line break at the top of a block, please indent the continuation line
an additional 4 spaces, like this:
.. code-block:: python
for val in self.some_method(
arg1, arg1, arg3, arg4):
self.do_something_awesome()
Neither of these are 'mandated' by PEP-8. However, they are prevailing styles
within this code base.
Unit Tests
----------
Unit tests should be virtually instant. If a unit test takes more than 1 second
to run, it is a bad unit test. Honestly, 1 second is too slow.
All unit test classes should subclass `openstack.cloud.tests.unit.base.BaseTestCase`. The
base TestCase class takes care of properly creating `OpenStackCloud` objects
in a way that protects against local environment.

View File

@ -1,36 +1,119 @@
OpenStack Python SDK
====================
openstacksdk
============
The ``python-openstacksdk`` is a collection of libraries for building
applications to work with OpenStack clouds. The project aims to provide
a consistent and complete set of interactions with OpenStack's many
services, along with complete documentation, examples, and tools.
openstacksdk is a client library for for building applications to work
with OpenStack clouds. The project aims to provide a consistent and
complete set of interactions with OpenStack's many services, along with
complete documentation, examples, and tools.
This SDK is under active development, and in the interests of providing
a high-quality interface, the APIs provided in this release may differ
from those provided in future release.
It also contains a simple interface layer. Clouds can do many things, but
there are probably only about 10 of them that most people care about with any
regularity. If you want to do complicated things, the per-service oriented
portions of the SDK are for you. However, if what you want is to be able to
write an application that talks to clouds no matter what crazy choices the
deployer has made in an attempt to be more hipster than their self-entitled
narcissist peers, then the ``openstack.cloud`` layer is for you.
Usage
-----
A Brief History
---------------
The following example simply connects to an OpenStack cloud and lists
the containers in the Object Store service.::
openstacksdk started its life as three different libraries: shade,
os-client-config and python-openstacksdk.
from openstack import connection
conn = connection.Connection(auth_url="http://openstack:5000/v3",
project_name="big_project",
username="SDK_user",
password="Super5ecretPassw0rd")
for container in conn.object_store.containers():
print(container.name)
``shade`` started its life as some code inside of OpenStack Infra's nodepool
project, and as some code inside of Ansible. Ansible had a bunch of different
OpenStack related modules, and there was a ton of duplicated code. Eventually,
between refactoring that duplication into an internal library, and adding logic
and features that the OpenStack Infra team had developed to run client
applications at scale, it turned out that we'd written nine-tenths of what we'd
need to have a standalone library.
Documentation
-------------
``os-client-config`` was a library for collecting client configuration for
using an OpenStack cloud in a consistent and comprehensive manner.
In parallel, the python-openstacksdk team was working on a library to expose
the OpenStack APIs to developers in a consistent and predictable manner. After
a while it became clear that there was value in both a high-level layer that
contains business logic, a lower-level SDK that exposes services and their
resources as Python objects, and also to be able to make direct REST calls
when needed with a properly configured Session or Adapter from python-requests.
This led to the merger of the three projects.
Documentation is available at
http://developer.openstack.org/sdks/python/openstacksdk/
The contents of the shade library have been moved into ``openstack.cloud``
and os-client-config has been moved in to ``openstack.config``. The next
release of shade will be a thin compatibility layer that subclasses the objects
from ``openstack.cloud`` and provides different argument defaults where needed
for compat. Similarly the next release of os-client-config will be a compat
layer shim around ``openstack.config``.
License
-------
openstack.config
================
Apache 2.0
``openstack.config`` will find cloud configuration for as few as 1 clouds and
as many as you want to put in a config file. It will read environment variables
and config files, and it also contains some vendor specific default values so
that you don't have to know extra info to use OpenStack
* If you have a config file, you will get the clouds listed in it
* If you have environment variables, you will get a cloud named `envvars`
* If you have neither, you will get a cloud named `defaults` with base defaults
Sometimes an example is nice.
Create a ``clouds.yaml`` file:
.. code-block:: yaml
clouds:
mordred:
region_name: Dallas
auth:
username: 'mordred'
password: XXXXXXX
project_name: 'shade'
auth_url: 'https://identity.example.com'
Please note: ``openstack.config`` will look for a file called ``clouds.yaml``
in the following locations:
* Current Directory
* ``~/.config/openstack``
* ``/etc/openstack``
More information at https://developer.openstack.org/sdks/python/openstacksdk/users/config
openstack.cloud
===============
Create a server using objects configured with the ``clouds.yaml`` file:
.. code-block:: python
import openstack.cloud
# Initialize and turn on debug logging
openstack.cloud.simple_logging(debug=True)
# Initialize cloud
# Cloud configs are read with openstack.config
cloud = openstack.cloud.openstack_cloud(cloud='mordred')
# Upload an image to the cloud
image = cloud.create_image(
'ubuntu-trusty', filename='ubuntu-trusty.qcow2', wait=True)
# Find a flavor with at least 512M of RAM
flavor = cloud.get_flavor_by_ram(512)
# Boot a server, wait for it to boot, and then do whatever is needed
# to get a public ip for it.
cloud.create_server(
'my-server', image=image, flavor=flavor, wait=True, auto_ip=True)
Links
=====
* `Issue Tracker <https://storyboard.openstack.org/#!/project/760>`_
* `Code Review <https://review.openstack.org/#/q/status:open+project:openstack/python-openstacksdk,n,z>`_
* `Documentation <https://developer.openstack.org/sdks/python/openstacksdk/>`_
* `PyPI <https://pypi.python.org/pypi/python-openstacksdk/>`_
* `Mailing list <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>`_

54
devstack/plugin.sh Normal file
View File

@ -0,0 +1,54 @@
# Install and configure **openstacksdk** library in devstack
#
# To enable openstacksdk in devstack add an entry to local.conf that looks like
#
# [[local|localrc]]
# enable_plugin openstacksdk git://git.openstack.org/openstack/python-openstacksdk
function preinstall_openstacksdk {
:
}
function install_openstacksdk {
if use_library_from_git "python-openstacksdk"; then
# don't clone, it'll be done by the plugin install
setup_dev_lib "python-openstacksdk"
else
pip_install "python-openstacksdk"
fi
}
function configure_openstacksdk {
:
}
function initialize_openstacksdk {
:
}
function unstack_openstacksdk {
:
}
function clean_openstacksdk {
:
}
# This is the main for plugin.sh
if [[ "$1" == "stack" && "$2" == "pre-install" ]]; then
preinstall_openstacksdk
elif [[ "$1" == "stack" && "$2" == "install" ]]; then
install_openstacksdk
elif [[ "$1" == "stack" && "$2" == "post-config" ]]; then
configure_openstacksdk
elif [[ "$1" == "stack" && "$2" == "extra" ]]; then
initialize_openstacksdk
fi
if [[ "$1" == "unstack" ]]; then
unstack_openstacksdk
fi
if [[ "$1" == "clean" ]]; then
clean_openstacksdk
fi

View File

@ -19,16 +19,24 @@ import openstackdocstheme
sys.path.insert(0, os.path.abspath('../..'))
sys.path.insert(0, os.path.abspath('.'))
# -- General configuration ----------------------------------------------------
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
extensions = [
'sphinx.ext.autodoc',
'sphinx.ext.intersphinx',
'openstackdocstheme',
'enforcer'
]
# openstackdocstheme options
repository_name = 'openstack/python-openstacksdk'
bug_project = '760'
bug_tag = ''
html_last_updated_fmt = '%Y-%m-%d %H:%M'
html_theme = 'openstackdocs'
# When True, this will raise an exception that kills sphinx-build.
enforcer_warnings_as_errors = True
@ -47,18 +55,7 @@ master_doc = 'index'
# General information about the project.
project = u'python-openstacksdk'
copyright = u'2015, OpenStack Foundation'
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# "version" and "release" are used by the "log-a-bug" feature
#
# The short X.Y version.
version = '1.0'
# The full version, including alpha/beta/rc tags.
release = '1.0'
copyright = u'2017, Various members of the OpenStack Foundation'
# A few variables have to be set for the log-a-bug feature.
# giturl: The location of conf.py on Git. Must be set manually.
@ -101,13 +98,6 @@ exclude_patterns = []
# -- Options for HTML output ----------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
html_theme = 'openstackdocs'
# Add any paths that contain custom themes here, relative to this directory.
html_theme_path = [openstackdocstheme.get_html_theme_path()]
# Don't let openstackdocstheme insert TOCs automatically.
theme_include_auto_toc = False
@ -124,9 +114,5 @@ latex_documents = [
u'OpenStack Foundation', 'manual'),
]
# Example configuration for intersphinx: refer to the Python standard library.
intersphinx_mapping = {'https://docs.python.org/3/': None,
'http://docs.python-requests.org/en/master/': None}
# Include both the class and __init__ docstrings when describing the class
autoclass_content = "both"

View File

@ -1,24 +1,24 @@
********************************
Shade Developer Coding Standards
********************************
========================================
OpenStack SDK Developer Coding Standards
========================================
In the beginning, there were no guidelines. And it was good. But that
didn't last long. As more and more people added more and more code,
we realized that we needed a set of coding standards to make sure that
the shade API at least *attempted* to display some form of consistency.
the openstacksdk API at least *attempted* to display some form of consistency.
Thus, these coding standards/guidelines were developed. Note that not
all of shade adheres to these standards just yet. Some older code has
all of openstacksdk adheres to these standards just yet. Some older code has
not been updated because we need to maintain backward compatibility.
Some of it just hasn't been changed yet. But be clear, all new code
*must* adhere to these guidelines.
Below are the patterns that we expect Shade developers to follow.
Below are the patterns that we expect openstacksdk developers to follow.
Release Notes
=============
Shade uses `reno <http://docs.openstack.org/developer/reno/>`_ for
openstacksdk uses `reno <http://docs.openstack.org/developer/reno/>`_ for
managing its release notes. A new release note should be added to
your contribution anytime you add new API calls, fix significant bugs,
add new functionality or parameters to existing API calls, or make any
@ -29,8 +29,17 @@ It is *not* necessary to add release notes for minor fixes, such as
correction of documentation typos, minor code cleanup or reorganization,
or any other change that a user would not notice through normal usage.
API Methods
===========
Exceptions
==========
Exceptions should NEVER be wrapped and re-raised inside of a new exception.
This removes important debug information from the user. All of the exceptions
should be raised correctly the first time.
openstack.cloud API Methods
===========================
The `openstack.cloud` layer has some specific rules:
- When an API call acts on a resource that has both a unique ID and a
name, that API call should accept either identifier with a name_or_id
@ -50,21 +59,8 @@ API Methods
- Deleting a resource should return True if the delete succeeded, or False
if the resource was not found.
Exceptions
==========
All underlying client exceptions must be captured and converted to an
`OpenStackCloudException` or one of its derivatives.
REST Calls
============
All interactions with the cloud should be done with direct REST using
the appropriate `keystoneauth1.adapter.Adapter`. See Glance and Swift
calls for examples.
Returned Resources
==================
------------------
Complex objects returned to the caller must be a `munch.Munch` type. The
`openstack.cloud._adapter.Adapter` class makes resources into `munch.Munch`.
@ -72,19 +68,20 @@ Complex objects returned to the caller must be a `munch.Munch` type. The
All objects should be normalized. It is shade's purpose in life to make
OpenStack consistent for end users, and this means not trusting the clouds
to return consistent objects. There should be a normalize function in
`shade/_normalize.py` that is applied to objects before returning them to
the user. See :doc:`../user/model` for further details on object model requirements.
`openstack/cloud/_normalize.py` that is applied to objects before returning
them to the user. See :doc:`../user/model` for further details on object model
requirements.
Fields should not be in the normalization contract if we cannot commit to
providing them to all users.
Fields should be renamed in normalization to be consistent with
the rest of openstack.cloud. For instance, nothing in shade exposes the legacy OpenStack
concept of "tenant" to a user, but instead uses "project" even if the
cloud uses tenant.
the rest of `openstack.cloud`. For instance, nothing in `openstack.cloud`
exposes the legacy OpenStack concept of "tenant" to a user, but instead uses
"project" even if the cloud in question uses tenant.
Nova vs. Neutron
================
----------------
- Recognize that not all cloud providers support Neutron, so never
assume it will be present. If a task can be handled by either
@ -101,8 +98,10 @@ Tests
- New API methods *must* have unit tests!
- New unit tests should only mock at the REST layer using `requests_mock`.
Any mocking of shade itself or of legacy client libraries should be
considered legacy and to be avoided.
Any mocking of openstacksdk itself should be considered legacy and to be
avoided. Exceptions to this rule can be made when attempting to test the
internals of a logical shim where the inputs and output of the method aren't
actually impacted by remote content.
- Functional tests should be added, when possible.

View File

@ -13,6 +13,14 @@ software development kit for the programs which make up the OpenStack
community. It is a set of Python-based libraries, documentation, examples,
and tools released under the Apache 2 license.
Contribution Mechanics
----------------------
.. toctree::
:maxdepth: 2
contributing
Contacting the Developers
-------------------------
@ -33,6 +41,17 @@ mailing list fields questions of all types on OpenStack. Using the
``[python-openstacksdk]`` filter to begin your email subject will ensure
that the message gets to SDK developers.
Coding Standards
----------------
We are a bit stricter than usual in the coding standards department. It's a
good idea to read through the :doc:`coding <coding>` section.
.. toctree::
:maxdepth: 2
coding
Development Environment
-----------------------

View File

@ -1 +0,0 @@
.. include:: ../../ChangeLog

View File

@ -13,6 +13,10 @@ For Users
:maxdepth: 2
users/index
install/index
user/index
.. TODO(shade) merge users/index and user/index into user/index
For Contributors
----------------
@ -20,7 +24,9 @@ For Contributors
.. toctree::
:maxdepth: 2
contributors/index
contributor/index
.. include:: ../../README.rst
General Information
-------------------
@ -31,4 +37,4 @@ General information about the SDK including a glossary and release history.
:maxdepth: 1
Glossary of Terms <glossary>
Release History <history>
Release Notes <releasenotes>

View File

@ -0,0 +1,12 @@
============
Installation
============
At the command line::
$ pip install python-openstacksdk
Or, if you have virtualenv wrapper installed::
$ mkvirtualenv python-openstacksdk
$ pip install python-openstacksdk

View File

@ -0,0 +1,6 @@
=============
Release Notes
=============
Release notes for `python-openstacksdk` can be found at
http://docs.openstack.org/releasenotes/python-openstacksdk/

View File

@ -9,4 +9,4 @@
using
vendor-support
network-config
releasenotes
reference

View File

@ -5,6 +5,7 @@
.. toctree::
:maxdepth: 2
config
usage
logging
model

View File

@ -66,7 +66,7 @@ then
echo "Using existing Ansible source repo"
else
echo "Installing Ansible source repo at $ENVDIR"
git clone --recursive git://github.com/ansible/ansible.git ${ENVDIR}/ansible
git clone --recursive https://github.com/ansible/ansible.git ${ENVDIR}/ansible
fi
source $ENVDIR/ansible/hacking/env-setup
else
@ -91,4 +91,4 @@ then
exit 1
fi
ansible-playbook -vvv ./shade/tests/ansible/run.yml -e "cloud=${CLOUD} image=${IMAGE}" ${tag_opt}
ansible-playbook -vvv ./openstack/tests/ansible/run.yml -e "cloud=${CLOUD} image=${IMAGE}" ${tag_opt}

View File

@ -1,646 +0,0 @@
# Copyright 2010-2011 OpenStack Foundation
# Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import collections
import time
import uuid
import fixtures
import mock
import os
import openstack.config as occ
from requests import structures
from requests_mock.contrib import fixture as rm_fixture
from six.moves import urllib
import tempfile
import openstack.cloud.openstackcloud
from openstack.cloud.tests import base
_ProjectData = collections.namedtuple(
'ProjectData',
'project_id, project_name, enabled, domain_id, description, '
'json_response, json_request')
_UserData = collections.namedtuple(
'UserData',
'user_id, password, name, email, description, domain_id, enabled, '
'json_response, json_request')
_GroupData = collections.namedtuple(
'GroupData',
'group_id, group_name, domain_id, description, json_response, '
'json_request')
_DomainData = collections.namedtuple(
'DomainData',
'domain_id, domain_name, description, json_response, '
'json_request')
_ServiceData = collections.namedtuple(
'Servicedata',
'service_id, service_name, service_type, description, enabled, '
'json_response_v3, json_response_v2, json_request')
_EndpointDataV3 = collections.namedtuple(
'EndpointData',
'endpoint_id, service_id, interface, region, url, enabled, '
'json_response, json_request')
_EndpointDataV2 = collections.namedtuple(
'EndpointData',
'endpoint_id, service_id, region, public_url, internal_url, '
'admin_url, v3_endpoint_list, json_response, '
'json_request')
# NOTE(notmorgan): Shade does not support domain-specific roles
# This should eventually be fixed if it becomes a main-stream feature.
_RoleData = collections.namedtuple(
'RoleData',
'role_id, role_name, json_response, json_request')
class BaseTestCase(base.TestCase):
def setUp(self, cloud_config_fixture='clouds.yaml'):
"""Run before each test method to initialize test environment."""
super(BaseTestCase, self).setUp()
# Sleeps are for real testing, but unit tests shouldn't need them
realsleep = time.sleep
def _nosleep(seconds):
return realsleep(seconds * 0.0001)
self.sleep_fixture = self.useFixture(fixtures.MonkeyPatch(
'time.sleep',
_nosleep))
self.fixtures_directory = 'shade/tests/unit/fixtures'
# Isolate os-client-config from test environment
config = tempfile.NamedTemporaryFile(delete=False)
cloud_path = '%s/clouds/%s' % (self.fixtures_directory,
cloud_config_fixture)
with open(cloud_path, 'rb') as f:
content = f.read()
config.write(content)
config.close()
vendor = tempfile.NamedTemporaryFile(delete=False)
vendor.write(b'{}')
vendor.close()
# set record mode depending on environment
record_mode = os.environ.get('BETAMAX_RECORD_FIXTURES', False)
if record_mode:
self.record_fixtures = 'new_episodes'
else:
self.record_fixtures = None
test_cloud = os.environ.get('SHADE_OS_CLOUD', '_test_cloud_')
self.config = occ.OpenStackConfig(
config_files=[config.name],
vendor_files=[vendor.name],
secure_files=['non-existant'])
self.cloud_config = self.config.get_one_cloud(
cloud=test_cloud, validate=False)
self.cloud = openstack.cloud.OpenStackCloud(
cloud_config=self.cloud_config,
log_inner_exceptions=True)
self.strict_cloud = openstack.cloud.OpenStackCloud(
cloud_config=self.cloud_config,
log_inner_exceptions=True,
strict=True)
self.op_cloud = openstack.cloud.OperatorCloud(
cloud_config=self.cloud_config,
log_inner_exceptions=True)
class TestCase(BaseTestCase):
def setUp(self, cloud_config_fixture='clouds.yaml'):
super(TestCase, self).setUp(cloud_config_fixture=cloud_config_fixture)
self.session_fixture = self.useFixture(fixtures.MonkeyPatch(
'os_client_config.cloud_config.CloudConfig.get_session',
mock.Mock()))
class RequestsMockTestCase(BaseTestCase):
def setUp(self, cloud_config_fixture='clouds.yaml'):
super(RequestsMockTestCase, self).setUp(
cloud_config_fixture=cloud_config_fixture)
# FIXME(notmorgan): Convert the uri_registry, discovery.json, and
# use of keystone_v3/v2 to a proper fixtures.Fixture. For now this
# is acceptable, but eventually this should become it's own fixture
# that encapsulates the registry, registering the URIs, and
# assert_calls (and calling assert_calls every test case that uses
# it on cleanup). Subclassing here could be 100% eliminated in the
# future allowing any class to simply
# self.useFixture(openstack.cloud.RequestsMockFixture) and get all
# the benefits.
# NOTE(notmorgan): use an ordered dict here to ensure we preserve the
# order in which items are added to the uri_registry. This makes
# the behavior more consistent when dealing with ensuring the
# requests_mock uri/query_string matchers are ordered and parse the
# request in the correct orders.
self._uri_registry = collections.OrderedDict()
self.discovery_json = os.path.join(
self.fixtures_directory, 'discovery.json')
self.use_keystone_v3()
self.__register_uris_called = False
def get_mock_url(self, service_type, interface='public', resource=None,
append=None, base_url_append=None,
qs_elements=None):
endpoint_url = self.cloud.endpoint_for(
service_type=service_type, interface=interface)
# Strip trailing slashes, so as not to produce double-slashes below
if endpoint_url.endswith('/'):
endpoint_url = endpoint_url[:-1]
to_join = [endpoint_url]
qs = ''
if base_url_append:
to_join.append(base_url_append)
if resource:
to_join.append(resource)
to_join.extend(append or [])
if qs_elements is not None:
qs = '?%s' % '&'.join(qs_elements)
return '%(uri)s%(qs)s' % {'uri': '/'.join(to_join), 'qs': qs}
def mock_for_keystone_projects(self, project=None, v3=True,
list_get=False, id_get=False,
project_list=None, project_count=None):
if project:
assert not (project_list or project_count)
elif project_list:
assert not (project or project_count)
elif project_count:
assert not (project or project_list)
else:
raise Exception('Must specify a project, project_list, '
'or project_count')
assert list_get or id_get
base_url_append = 'v3' if v3 else None
if project:
project_list = [project]
elif project_count:
# Generate multiple projects
project_list = [self._get_project_data(v3=v3)
for c in range(0, project_count)]
uri_mock_list = []
if list_get:
uri_mock_list.append(
dict(method='GET',
uri=self.get_mock_url(
service_type='identity',
interface='admin',
resource='projects',
base_url_append=base_url_append),
status_code=200,
json={'projects': [p.json_response['project']
for p in project_list]})
)
if id_get:
for p in project_list:
uri_mock_list.append(
dict(method='GET',
uri=self.get_mock_url(
service_type='identity',
interface='admin',
resource='projects',
append=[p.project_id],
base_url_append=base_url_append),
status_code=200,
json=p.json_response)
)
self.__do_register_uris(uri_mock_list)
return project_list
def _get_project_data(self, project_name=None, enabled=None,
domain_id=None, description=None, v3=True,
project_id=None):
project_name = project_name or self.getUniqueString('projectName')
project_id = uuid.UUID(project_id or uuid.uuid4().hex).hex
response = {'id': project_id, 'name': project_name}
request = {'name': project_name}
domain_id = (domain_id or uuid.uuid4().hex) if v3 else None
if domain_id:
request['domain_id'] = domain_id
response['domain_id'] = domain_id
if enabled is not None:
enabled = bool(enabled)
response['enabled'] = enabled
request['enabled'] = enabled
response.setdefault('enabled', True)
request.setdefault('enabled', True)
if description:
response['description'] = description
request['description'] = description
request.setdefault('description', None)
if v3:
project_key = 'project'
else:
project_key = 'tenant'
return _ProjectData(project_id, project_name, enabled, domain_id,
description, {project_key: response},
{project_key: request})
def _get_group_data(self, name=None, domain_id=None, description=None):
group_id = uuid.uuid4().hex
name = name or self.getUniqueString('groupname')
domain_id = uuid.UUID(domain_id or uuid.uuid4().hex).hex
response = {'id': group_id, 'name': name, 'domain_id': domain_id}
request = {'name': name, 'domain_id': domain_id}
if description is not None:
response['description'] = description
request['description'] = description
return _GroupData(group_id, name, domain_id, description,
{'group': response}, {'group': request})
def _get_user_data(self, name=None, password=None, **kwargs):
name = name or self.getUniqueString('username')
password = password or self.getUniqueString('user_password')
user_id = uuid.uuid4().hex
response = {'name': name, 'id': user_id}
request = {'name': name, 'password': password}
if kwargs.get('domain_id'):
kwargs['domain_id'] = uuid.UUID(kwargs['domain_id']).hex
response['domain_id'] = kwargs.pop('domain_id')
request['domain_id'] = response['domain_id']
response['email'] = kwargs.pop('email', None)
request['email'] = response['email']
response['enabled'] = kwargs.pop('enabled', True)
request['enabled'] = response['enabled']
response['description'] = kwargs.pop('description', None)
if response['description']:
request['description'] = response['description']
self.assertIs(0, len(kwargs), message='extra key-word args received '
'on _get_user_data')
return _UserData(user_id, password, name, response['email'],
response['description'], response.get('domain_id'),
response.get('enabled'), {'user': response},
{'user': request})
def _get_domain_data(self, domain_name=None, description=None,
enabled=None):
domain_id = uuid.uuid4().hex
domain_name = domain_name or self.getUniqueString('domainName')
response = {'id': domain_id, 'name': domain_name}
request = {'name': domain_name}
if enabled is not None:
request['enabled'] = bool(enabled)
response['enabled'] = bool(enabled)
if description:
response['description'] = description
request['description'] = description
response.setdefault('enabled', True)
return _DomainData(domain_id, domain_name, description,
{'domain': response}, {'domain': request})
def _get_service_data(self, type=None, name=None, description=None,
enabled=True):
service_id = uuid.uuid4().hex
name = name or uuid.uuid4().hex
type = type or uuid.uuid4().hex
response = {'id': service_id, 'name': name, 'type': type,
'enabled': enabled}
if description is not None:
response['description'] = description
request = response.copy()
request.pop('id')
return _ServiceData(service_id, name, type, description, enabled,
{'service': response},
{'OS-KSADM:service': response}, request)
def _get_endpoint_v3_data(self, service_id=None, region=None,
url=None, interface=None, enabled=True):
endpoint_id = uuid.uuid4().hex
service_id = service_id or uuid.uuid4().hex
region = region or uuid.uuid4().hex
url = url or 'https://example.com/'
interface = interface or uuid.uuid4().hex
response = {'id': endpoint_id, 'service_id': service_id,
'region': region, 'interface': interface,
'url': url, 'enabled': enabled}
request = response.copy()
request.pop('id')
response['region_id'] = response['region']
return _EndpointDataV3(endpoint_id, service_id, interface, region,
url, enabled, {'endpoint': response},
{'endpoint': request})
def _get_endpoint_v2_data(self, service_id=None, region=None,
public_url=None, admin_url=None,
internal_url=None):
endpoint_id = uuid.uuid4().hex
service_id = service_id or uuid.uuid4().hex
region = region or uuid.uuid4().hex
response = {'id': endpoint_id, 'service_id': service_id,
'region': region}
v3_endpoints = {}
request = response.copy()
request.pop('id')
if admin_url:
response['adminURL'] = admin_url
v3_endpoints['admin'] = self._get_endpoint_v3_data(
service_id, region, public_url, interface='admin')
if internal_url:
response['internalURL'] = internal_url
v3_endpoints['internal'] = self._get_endpoint_v3_data(
service_id, region, internal_url, interface='internal')
if public_url:
response['publicURL'] = public_url
v3_endpoints['public'] = self._get_endpoint_v3_data(
service_id, region, public_url, interface='public')
request = response.copy()
request.pop('id')
for u in ('publicURL', 'internalURL', 'adminURL'):
if request.get(u):
request[u.lower()] = request.pop(u)
return _EndpointDataV2(endpoint_id, service_id, region, public_url,
internal_url, admin_url, v3_endpoints,
{'endpoint': response}, {'endpoint': request})
def _get_role_data(self, role_name=None):
role_id = uuid.uuid4().hex
role_name = role_name or uuid.uuid4().hex
request = {'name': role_name}
response = request.copy()
response['id'] = role_id
return _RoleData(role_id, role_name, {'role': response},
{'role': request})
def use_keystone_v3(self, catalog='catalog-v3.json'):
self.adapter = self.useFixture(rm_fixture.Fixture())
self.calls = []
self._uri_registry.clear()
self.__do_register_uris([
dict(method='GET', uri='https://identity.example.com/',
text=open(self.discovery_json, 'r').read()),
dict(method='POST',
uri='https://identity.example.com/v3/auth/tokens',
headers={
'X-Subject-Token': self.getUniqueString('KeystoneToken')},
text=open(os.path.join(
self.fixtures_directory, catalog), 'r').read()
),
])
self._make_test_cloud(identity_api_version='3')
def use_keystone_v2(self):
self.adapter = self.useFixture(rm_fixture.Fixture())
self.calls = []
self._uri_registry.clear()
self.__do_register_uris([
dict(method='GET', uri='https://identity.example.com/',
text=open(self.discovery_json, 'r').read()),
dict(method='POST', uri='https://identity.example.com/v2.0/tokens',
text=open(os.path.join(
self.fixtures_directory, 'catalog-v2.json'), 'r').read()
),
])
self._make_test_cloud(cloud_name='_test_cloud_v2_',
identity_api_version='2.0')
def _make_test_cloud(self, cloud_name='_test_cloud_', **kwargs):
test_cloud = os.environ.get('SHADE_OS_CLOUD', cloud_name)
self.cloud_config = self.config.get_one_cloud(
cloud=test_cloud, validate=True, **kwargs)
self.cloud = openstack.cloud.OpenStackCloud(
cloud_config=self.cloud_config,
log_inner_exceptions=True)
self.op_cloud = openstack.cloud.OperatorCloud(
cloud_config=self.cloud_config,
log_inner_exceptions=True)
def get_glance_discovery_mock_dict(
self, image_version_json='image-version.json'):
discovery_fixture = os.path.join(
self.fixtures_directory, image_version_json)
return dict(method='GET', uri='https://image.example.com/',
status_code=300,
text=open(discovery_fixture, 'r').read())
def get_designate_discovery_mock_dict(self):
discovery_fixture = os.path.join(
self.fixtures_directory, "dns.json")
return dict(method='GET', uri="https://dns.example.com/",
text=open(discovery_fixture, 'r').read())
def get_ironic_discovery_mock_dict(self):
discovery_fixture = os.path.join(
self.fixtures_directory, "baremetal.json")
return dict(method='GET', uri="https://bare-metal.example.com/",
text=open(discovery_fixture, 'r').read())
def use_glance(self, image_version_json='image-version.json'):
# NOTE(notmorgan): This method is only meant to be used in "setUp"
# where the ordering of the url being registered is tightly controlled
# if the functionality of .use_glance is meant to be used during an
# actual test case, use .get_glance_discovery_mock and apply to the
# right location in the mock_uris when calling .register_uris
self.__do_register_uris([
self.get_glance_discovery_mock_dict(image_version_json)])
def use_designate(self):
# NOTE(slaweq): This method is only meant to be used in "setUp"
# where the ordering of the url being registered is tightly controlled
# if the functionality of .use_designate is meant to be used during an
# actual test case, use .get_designate_discovery_mock and apply to the
# right location in the mock_uris when calling .register_uris
self.__do_register_uris([
self.get_designate_discovery_mock_dict()])
def use_ironic(self):
# NOTE(TheJulia): This method is only meant to be used in "setUp"
# where the ordering of the url being registered is tightly controlled
# if the functionality of .use_ironic is meant to be used during an
# actual test case, use .get_ironic_discovery_mock and apply to the
# right location in the mock_uris when calling .register_uris
self.__do_register_uris([
self.get_ironic_discovery_mock_dict()])
def register_uris(self, uri_mock_list=None):
"""Mock a list of URIs and responses via requests mock.
This method may be called only once per test-case to avoid odd
and difficult to debug interactions. Discovery and Auth request mocking
happens separately from this method.
:param uri_mock_list: List of dictionaries that template out what is
passed to requests_mock fixture's `register_uri`.
Format is:
{'method': <HTTP_METHOD>,
'uri': <URI to be mocked>,
...
}
Common keys to pass in the dictionary:
* json: the json response (dict)
* status_code: the HTTP status (int)
* validate: The request body (dict) to
validate with assert_calls
all key-word arguments that are valid to send to
requests_mock are supported.
This list should be in the order in which calls
are made. When `assert_calls` is executed, order
here will be validated. Duplicate URIs and
Methods are allowed and will be collapsed into a
single matcher. Each response will be returned
in order as the URI+Method is hit.
:type uri_mock_list: list
:return: None
"""
assert not self.__register_uris_called
self.__do_register_uris(uri_mock_list or [])
self.__register_uris_called = True
def __do_register_uris(self, uri_mock_list=None):
for to_mock in uri_mock_list:
kw_params = {k: to_mock.pop(k)
for k in ('request_headers', 'complete_qs',
'_real_http')
if k in to_mock}
method = to_mock.pop('method')
uri = to_mock.pop('uri')
# NOTE(notmorgan): make sure the delimiter is non-url-safe, in this
# case "|" is used so that the split can be a bit easier on
# maintainers of this code.
key = '{method}|{uri}|{params}'.format(
method=method, uri=uri, params=kw_params)
validate = to_mock.pop('validate', {})
valid_keys = set(['json', 'headers', 'params'])
invalid_keys = set(validate.keys()) - valid_keys
if invalid_keys:
raise TypeError(
"Invalid values passed to validate: {keys}".format(
keys=invalid_keys))
headers = structures.CaseInsensitiveDict(to_mock.pop('headers',
{}))
if 'content-type' not in headers:
headers[u'content-type'] = 'application/json'
to_mock['headers'] = headers
self.calls += [
dict(
method=method,
url=uri, **validate)
]
self._uri_registry.setdefault(
key, {'response_list': [], 'kw_params': kw_params})
if self._uri_registry[key]['kw_params'] != kw_params:
raise AssertionError(
'PROGRAMMING ERROR: key-word-params '
'should be part of the uri_key and cannot change, '
'it will affect the matcher in requests_mock. '
'%(old)r != %(new)r' %
{'old': self._uri_registry[key]['kw_params'],
'new': kw_params})
self._uri_registry[key]['response_list'].append(to_mock)
for mocked, params in self._uri_registry.items():
mock_method, mock_uri, _ignored = mocked.split('|', 2)
self.adapter.register_uri(
mock_method, mock_uri, params['response_list'],
**params['kw_params'])
def assert_calls(self, stop_after=None, do_count=True):
for (x, (call, history)) in enumerate(
zip(self.calls, self.adapter.request_history)):
if stop_after and x > stop_after:
break
call_uri_parts = urllib.parse.urlparse(call['url'])
history_uri_parts = urllib.parse.urlparse(history.url)
self.assertEqual(
(call['method'], call_uri_parts.scheme, call_uri_parts.netloc,
call_uri_parts.path, call_uri_parts.params,
urllib.parse.parse_qs(call_uri_parts.query)),
(history.method, history_uri_parts.scheme,
history_uri_parts.netloc, history_uri_parts.path,
history_uri_parts.params,
urllib.parse.parse_qs(history_uri_parts.query)),
('REST mismatch on call %(index)d. Expected %(call)r. '
'Got %(history)r). '
'NOTE: query string order differences wont cause mismatch' %
{
'index': x,
'call': '{method} {url}'.format(method=call['method'],
url=call['url']),
'history': '{method} {url}'.format(
method=history.method,
url=history.url)})
)
if 'json' in call:
self.assertEqual(
call['json'], history.json(),
'json content mismatch in call {index}'.format(index=x))
# headers in a call isn't exhaustive - it's checking to make sure
# a specific header or headers are there, not that they are the
# only headers
if 'headers' in call:
for key, value in call['headers'].items():
self.assertEqual(
value, history.headers[key],
'header mismatch in call {index}'.format(index=x))
if do_count:
self.assertEqual(
len(self.calls), len(self.adapter.request_history))
class IronicTestCase(RequestsMockTestCase):
def setUp(self):
super(IronicTestCase, self).setUp()
self.use_ironic()
self.uuid = str(uuid.uuid4())
self.name = self.getUniqueString('name')
def get_mock_url(self, resource=None, append=None, qs_elements=None):
return super(IronicTestCase, self).get_mock_url(
service_type='baremetal', interface='public', resource=resource,
append=append, base_url_append='v1', qs_elements=qs_elements)

View File

@ -12,14 +12,16 @@
# License for the specific language governing permissions and limitations
# under the License.
export SHADE_DIR="$BASE/new/shade"
# TODO(shade) Rework for Zuul v3
cd $SHADE_DIR
sudo chown -R jenkins:stack $SHADE_DIR
export OPENSTACKSDK_DIR="$BASE/new/python-openstacksdk"
cd $OPENSTACKSDK_DIR
sudo chown -R jenkins:stack $OPENSTACKSDK_DIR
echo "Running shade Ansible test suite"
if [ ${SHADE_ANSIBLE_DEV:-0} -eq 1 ]
if [ ${OPENSTACKSDK_ANSIBLE_DEV:-0} -eq 1 ]
then
# Use the upstream development version of Ansible
set +e

View File

@ -10,6 +10,8 @@
# License for the specific language governing permissions and limitations
# under the License.
# TODO(shade) Merge this with openstack.tests.functional.base
import os
import openstack.config as occ
@ -22,9 +24,9 @@ class BaseFunctionalTestCase(base.TestCase):
def setUp(self):
super(BaseFunctionalTestCase, self).setUp()
self._demo_name = os.environ.get('SHADE_DEMO_CLOUD', 'devstack')
self._demo_name = os.environ.get('OPENSTACKSDK_DEMO_CLOUD', 'devstack')
self._op_name = os.environ.get(
'SHADE_OPERATOR_CLOUD', 'devstack-admin')
'OPENSTACKSDK_OPERATOR_CLOUD', 'devstack-admin')
self.config = occ.OpenStackConfig()
self._set_user_cloud()
@ -51,7 +53,7 @@ class BaseFunctionalTestCase(base.TestCase):
images = self.user_cloud.list_images()
self.add_info_on_exception('images', images)
image_name = os.environ.get('SHADE_IMAGE')
image_name = os.environ.get('OPENSTACKSDK_IMAGE')
if image_name:
for image in images:
if image.name == image_name:
@ -80,7 +82,7 @@ class KeystoneBaseFunctionalTestCase(BaseFunctionalTestCase):
def setUp(self):
super(KeystoneBaseFunctionalTestCase, self).setUp()
use_keystone_v2 = os.environ.get('SHADE_USE_KEYSTONE_V2', False)
use_keystone_v2 = os.environ.get('OPENSTACKSDK_USE_KEYSTONE_V2', False)
if use_keystone_v2:
# keystone v2 has special behavior for the admin
# interface and some of the operations, so make a new cloud

View File

@ -12,10 +12,12 @@
# License for the specific language governing permissions and limitations
# under the License.
export SHADE_DIR="$BASE/new/shade"
# TODO(shade) Rework for zuul v3
cd $SHADE_DIR
sudo chown -R jenkins:stack $SHADE_DIR
export OPENSTACKSDK_DIR="$BASE/new/shade"
cd $OPENSTACKSDK_DIR
sudo chown -R jenkins:stack $OPENSTACKSDK_DIR
CLOUDS_YAML=/etc/openstack/clouds.yaml
@ -30,7 +32,7 @@ fi
# Devstack runs both keystone v2 and v3. An environment variable is set
# within the shade keystone v2 job that tells us which version we should
# test against.
if [ ${SHADE_USE_KEYSTONE_V2:-0} -eq 1 ]
if [ ${OPENSTACKSDK_USE_KEYSTONE_V2:-0} -eq 1 ]
then
sudo sed -ie "s/identity_api_version: '3'/identity_api_version: '2.0'/g" $CLOUDS_YAML
sudo sed -ie '/^.*domain_id.*$/d' $CLOUDS_YAML

View File

@ -38,14 +38,15 @@ class TestDevstack(base.BaseFunctionalTestCase):
]
def test_has_service(self):
if os.environ.get('SHADE_HAS_{env}'.format(env=self.env), '0') == '1':
if os.environ.get(
'OPENSTACKSDK_HAS_{env}'.format(env=self.env), '0') == '1':
self.assertTrue(self.user_cloud.has_service(self.service))
class TestKeystoneVersion(base.BaseFunctionalTestCase):
def test_keystone_version(self):
use_keystone_v2 = os.environ.get('SHADE_USE_KEYSTONE_V2', False)
use_keystone_v2 = os.environ.get('OPENSTACKSDK_USE_KEYSTONE_V2', False)
if use_keystone_v2 and use_keystone_v2 != '0':
self.assertEqual('2.0', self.identity_version)
else:

Some files were not shown because too many files have changed in this diff Show More