Initialize from cinder

This commit is contained in:
Ben Swartzlander 2013-08-08 10:34:06 -04:00
parent 0483210d09
commit f99ef92c90
532 changed files with 240158 additions and 0 deletions

12
CONTRIBUTING.md Normal file
View File

@ -0,0 +1,12 @@
If you would like to contribute to the development of OpenStack,
you must follow the steps in the "If you're a developer, start here"
section of this page: [http://wiki.openstack.org/HowToContribute](http://wiki.openstack.org/HowToContribute#If_you.27re_a_developer.2C_start_here:)
Once those steps have been completed, changes to OpenStack
should be submitted for review via the Gerrit tool, following
the workflow documented at [http://wiki.openstack.org/GerritWorkflow](http://wiki.openstack.org/GerritWorkflow).
Pull requests submitted through GitHub will be ignored.
Bugs should be filed [on Launchpad](https://bugs.launchpad.net/cinder),
not in GitHub's issue tracker.

275
HACKING.rst Normal file
View File

@ -0,0 +1,275 @@
Cinder Style Commandments
=======================
- Step 1: Read http://www.python.org/dev/peps/pep-0008/
- Step 2: Read http://www.python.org/dev/peps/pep-0008/ again
- Step 3: Read on
General
-------
- Put two newlines between top-level code (funcs, classes, etc)
- Put one newline between methods in classes and anywhere else
- Long lines should be wrapped in parentheses
in preference to using a backslash for line continuation.
- Do not write "except:", use "except Exception:" at the very least
- Include your name with TODOs as in "#TODO(termie)"
- Do not shadow a built-in or reserved word. Example::
def list():
return [1, 2, 3]
mylist = list() # BAD, shadows `list` built-in
class Foo(object):
def list(self):
return [1, 2, 3]
mylist = Foo().list() # OKAY, does not shadow built-in
- Use the "is not" operator when testing for unequal identities. Example::
if not X is Y: # BAD, intended behavior is ambiguous
pass
if X is not Y: # OKAY, intuitive
pass
- Use the "not in" operator for evaluating membership in a collection. Example::
if not X in Y: # BAD, intended behavior is ambiguous
pass
if X not in Y: # OKAY, intuitive
pass
if not (X in Y or X in Z): # OKAY, still better than all those 'not's
pass
Imports
-------
- Do not import objects, only modules (*)
- Do not import more than one module per line (*)
- Do not make relative imports
- Order your imports by the full module path
- Organize your imports according to the following template
(*) exceptions are:
- imports from ``migrate`` package
- imports from ``sqlalchemy`` package
- imports from ``cinder.db.sqlalchemy.session`` module
Example::
# vim: tabstop=4 shiftwidth=4 softtabstop=4
{{stdlib imports in human alphabetical order}}
\n
{{third-party lib imports in human alphabetical order}}
\n
{{cinder imports in human alphabetical order}}
\n
\n
{{begin your code}}
Human Alphabetical Order Examples
---------------------------------
Example::
import httplib
import logging
import random
import StringIO
import time
import unittest
import eventlet
import webob.exc
import cinder.api.ec2
from cinder.api import openstack
from cinder.auth import users
from cinder.endpoint import cloud
import cinder.flags
from cinder import test
Docstrings
----------
Example::
"""A one line docstring looks like this and ends in a period."""
"""A multi line docstring has a one-line summary, less than 80 characters.
Then a new paragraph after a newline that explains in more detail any
general information about the function, class or method. Example usages
are also great to have here if it is a complex class for function.
When writing the docstring for a class, an extra line should be placed
after the closing quotations. For more in-depth explanations for these
decisions see http://www.python.org/dev/peps/pep-0257/
If you are going to describe parameters and return values, use Sphinx, the
appropriate syntax is as follows.
:param foo: the foo parameter
:param bar: the bar parameter
:returns: return_type -- description of the return value
:returns: description of the return value
:raises: AttributeError, KeyError
"""
Dictionaries/Lists
------------------
If a dictionary (dict) or list object is longer than 80 characters, its items
should be split with newlines. Embedded iterables should have their items
indented. Additionally, the last item in the dictionary should have a trailing
comma. This increases readability and simplifies future diffs.
Example::
my_dictionary = {
"image": {
"name": "Just a Snapshot",
"size": 2749573,
"properties": {
"user_id": 12,
"arch": "x86_64",
},
"things": [
"thing_one",
"thing_two",
],
"status": "ACTIVE",
},
}
Calling Methods
---------------
Calls to methods 80 characters or longer should format each argument with
newlines. This is not a requirement, but a guideline::
unnecessarily_long_function_name('string one',
'string two',
kwarg1=constants.ACTIVE,
kwarg2=['a', 'b', 'c'])
Rather than constructing parameters inline, it is better to break things up::
list_of_strings = [
'what_a_long_string',
'not as long',
]
dict_of_numbers = {
'one': 1,
'two': 2,
'twenty four': 24,
}
object_one.call_a_method('string three',
'string four',
kwarg1=list_of_strings,
kwarg2=dict_of_numbers)
Internationalization (i18n) Strings
-----------------------------------
In order to support multiple languages, we have a mechanism to support
automatic translations of exception and log strings.
Example::
msg = _("An error occurred")
raise HTTPBadRequest(explanation=msg)
If you have a variable to place within the string, first internationalize the
template string then do the replacement.
Example::
msg = _("Missing parameter: %s") % ("flavor",)
LOG.error(msg)
If you have multiple variables to place in the string, use keyword parameters.
This helps our translators reorder parameters when needed.
Example::
msg = _("The server with id %(s_id)s has no key %(m_key)s")
LOG.error(msg % {"s_id": "1234", "m_key": "imageId"})
Creating Unit Tests
-------------------
For every new feature, unit tests should be created that both test and
(implicitly) document the usage of said feature. If submitting a patch for a
bug that had no unit test, a new passing unit test should be added. If a
submitted bug fix does have a unit test, be sure to add a new one that fails
without the patch and passes with the patch.
For more information on creating unit tests and utilizing the testing
infrastructure in OpenStack Cinder, please read cinder/testing/README.rst.
openstack-common
----------------
A number of modules from openstack-common are imported into the project.
These modules are "incubating" in openstack-common and are kept in sync
with the help of openstack-common's update.py script. See:
http://wiki.openstack.org/CommonLibrary#Incubation
The copy of the code should never be directly modified here. Please
always update openstack-common first and then run the script to copy
the changes across.
OpenStack Trademark
-------------------
OpenStack is a registered trademark of OpenStack, LLC, and uses the
following capitalization:
OpenStack
Commit Messages
---------------
Using a common format for commit messages will help keep our git history
readable. Follow these guidelines:
First, provide a brief summary (it is recommended to keep the commit title
under 50 chars).
The first line of the commit message should provide an accurate
description of the change, not just a reference to a bug or
blueprint. It must be followed by a single blank line.
If the change relates to a specific driver (libvirt, xenapi, qpid, etc...),
begin the first line of the commit message with the driver name, lowercased,
followed by a colon.
Following your brief summary, provide a more detailed description of
the patch, manually wrapping the text at 72 characters. This
description should provide enough detail that one does not have to
refer to external resources to determine its high-level functionality.
Once you use 'git review', two lines will be appended to the commit
message: a blank line followed by a 'Change-Id'. This is important
to correlate this commit with a specific review in Gerrit, and it
should not be modified.
For further information on constructing high quality commit messages,
and how to split up commits into a series of changes, consult the
project wiki:
http://wiki.openstack.org/GitCommitMessages

176
LICENSE Normal file
View File

@ -0,0 +1,176 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.

6
MANIFEST.in Normal file
View File

@ -0,0 +1,6 @@
include AUTHORS
include ChangeLog
exclude .gitignore
exclude .gitreview
global-exclude *.pyc

21
README.rst Normal file
View File

@ -0,0 +1,21 @@
The Choose Your Own Adventure README for Cinder
===============================================
You have come across a storage service for an open cloud computing service.
It has identified itself as "Cinder." It was abstracted from the Nova project.
To monitor it from a distance: follow `@openstack <http://twitter.com/openstack>`_ on twitter.
To tame it for use in your own cloud: read http://docs.openstack.org
To study its anatomy: read http://cinder.openstack.org
To dissect it in detail: visit http://github.com/openstack/cinder
To taunt it with its weaknesses: use http://bugs.launchpad.net/cinder
To watch it: http://jenkins.openstack.org
To hack at it: read HACKING
To cry over its pylint problems: http://jenkins.openstack.org/job/cinder-pylint/violations

2
babel.cfg Normal file
View File

@ -0,0 +1,2 @@
[python: **.py]

70
bin/cinder-all Executable file
View File

@ -0,0 +1,70 @@
#!/usr/bin/env python
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2011 OpenStack, LLC
# Copyright 2010 United States Government as represented by the
# Administrator of the National Aeronautics and Space Administration.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Starter script for All cinder services.
This script attempts to start all the cinder services in one process. Each
service is started in its own greenthread. Please note that exceptions and
sys.exit() on the starting of a service are logged and the script will
continue attempting to launch the rest of the services.
"""
import eventlet
eventlet.monkey_patch()
import os
import sys
possible_topdir = os.path.normpath(os.path.join(os.path.abspath(
sys.argv[0]), os.pardir, os.pardir))
if os.path.exists(os.path.join(possible_topdir, "cinder", "__init__.py")):
sys.path.insert(0, possible_topdir)
from cinder.openstack.common import gettextutils
gettextutils.install('cinder')
from cinder import flags
from cinder.openstack.common import log as logging
from cinder import service
from cinder import utils
if __name__ == '__main__':
flags.parse_args(sys.argv)
logging.setup("cinder")
LOG = logging.getLogger('cinder.all')
utils.monkey_patch()
servers = []
# cinder-api
try:
servers.append(service.WSGIService('osapi_volume'))
except (Exception, SystemExit):
LOG.exception(_('Failed to load osapi_volume'))
for binary in ['cinder-volume', 'cinder-scheduler']:
try:
servers.append(service.Service.create(binary=binary))
except (Exception, SystemExit):
LOG.exception(_('Failed to load %s'), binary)
service.serve(*servers)
service.wait()

52
bin/cinder-api Executable file
View File

@ -0,0 +1,52 @@
#!/usr/bin/env python
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2010 United States Government as represented by the
# Administrator of the National Aeronautics and Space Administration.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Starter script for Cinder OS API."""
# NOTE(jdg): If we port over multi worker code from Nova
# we'll need to set monkey_patch(os=False), unless
# eventlet is updated/released to fix the root issue
import eventlet
eventlet.monkey_patch()
import os
import sys
possible_topdir = os.path.normpath(os.path.join(os.path.abspath(
sys.argv[0]), os.pardir, os.pardir))
if os.path.exists(os.path.join(possible_topdir, "cinder", "__init__.py")):
sys.path.insert(0, possible_topdir)
from cinder.openstack.common import gettextutils
gettextutils.install('cinder')
from cinder import flags
from cinder.openstack.common import log as logging
from cinder import service
from cinder import utils
if __name__ == '__main__':
flags.parse_args(sys.argv)
logging.setup("cinder")
utils.monkey_patch()
server = service.WSGIService('osapi_volume')
service.serve(server)
service.wait()

50
bin/cinder-backup Executable file
View File

@ -0,0 +1,50 @@
#!/usr/bin/env python
# Copyright (C) 2012 Hewlett-Packard Development Company, L.P.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""Starter script for Cinder Volume Backup."""
import os
import sys
import eventlet
eventlet.monkey_patch()
# If ../cinder/__init__.py exists, add ../ to Python search path, so that
# it will override what happens to be installed in /usr/(local/)lib/python...
possible_topdir = os.path.normpath(os.path.join(os.path.abspath(sys.argv[0]),
os.pardir,
os.pardir))
if os.path.exists(os.path.join(possible_topdir, 'cinder', '__init__.py')):
sys.path.insert(0, possible_topdir)
from cinder.openstack.common import gettextutils
gettextutils.install('cinder')
from cinder import flags
from cinder.openstack.common import log as logging
from cinder import service
from cinder import utils
if __name__ == '__main__':
flags.parse_args(sys.argv)
logging.setup("cinder")
utils.monkey_patch()
launcher = service.ProcessLauncher()
server = service.Service.create(binary='cinder-backup')
launcher.launch_server(server)
launcher.wait()

76
bin/cinder-clear-rabbit-queues Executable file
View File

@ -0,0 +1,76 @@
#!/usr/bin/env python
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright (c) 2011 OpenStack, LLC.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""Admin/debug script to wipe rabbitMQ (AMQP) queues cinder uses.
This can be used if you need to change durable options on queues,
or to wipe all messages in the queue system if things are in a
serious bad way.
"""
import datetime
import os
import sys
import time
# If ../cinder/__init__.py exists, add ../ to Python search path, so that
# it will override what happens to be installed in /usr/(local/)lib/python...
POSSIBLE_TOPDIR = os.path.normpath(os.path.join(os.path.abspath(sys.argv[0]),
os.pardir,
os.pardir))
if os.path.exists(os.path.join(POSSIBLE_TOPDIR, 'cinder', '__init__.py')):
sys.path.insert(0, POSSIBLE_TOPDIR)
from cinder.openstack.common import gettextutils
gettextutils.install('cinder')
from oslo.config import cfg
from cinder import context
from cinder import exception
from cinder import flags
from cinder.openstack.common import log as logging
from cinder.openstack.common import rpc
delete_exchange_opt = \
cfg.BoolOpt('delete_exchange',
default=False,
help='delete cinder exchange too.')
FLAGS = flags.FLAGS
FLAGS.register_cli_opt(delete_exchange_opt)
def delete_exchange(exch):
conn = rpc.create_connection()
x = conn.get_channel()
x.exchange_delete(exch)
def delete_queues(queues):
conn = rpc.create_connection()
x = conn.get_channel()
for q in queues:
x.queue_delete(q)
if __name__ == '__main__':
args = flags.parse_args(sys.argv)
logging.setup("cinder")
delete_queues(args[1:])
if FLAGS.delete_exchange:
delete_exchange(FLAGS.control_exchange)

820
bin/cinder-manage Executable file
View File

@ -0,0 +1,820 @@
#!/usr/bin/env python
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright (c) 2011 X.commerce, a business unit of eBay Inc.
# Copyright 2010 United States Government as represented by the
# Administrator of the National Aeronautics and Space Administration.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
# Interactive shell based on Django:
#
# Copyright (c) 2005, the Lawrence Journal-World
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright notice,
# this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
#
# 3. Neither the name of Django nor the names of its contributors may be
# used to endorse or promote products derived from this software without
# specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
"""
CLI interface for cinder management.
"""
import os
import sys
import uuid
from sqlalchemy import create_engine, MetaData, Table
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker
# If ../cinder/__init__.py exists, add ../ to Python search path, so that
# it will override what happens to be installed in /usr/(local/)lib/python...
POSSIBLE_TOPDIR = os.path.normpath(os.path.join(os.path.abspath(sys.argv[0]),
os.pardir,
os.pardir))
if os.path.exists(os.path.join(POSSIBLE_TOPDIR, 'cinder', '__init__.py')):
sys.path.insert(0, POSSIBLE_TOPDIR)
from cinder.openstack.common import gettextutils
gettextutils.install('cinder')
from oslo.config import cfg
from cinder import context
from cinder import db
from cinder.db import migration
from cinder import exception
from cinder import flags
from cinder.openstack.common import log as logging
from cinder.openstack.common import rpc
from cinder.openstack.common import uuidutils
from cinder import utils
from cinder import version
FLAGS = flags.FLAGS
# Decorators for actions
def args(*args, **kwargs):
def _decorator(func):
func.__dict__.setdefault('args', []).insert(0, (args, kwargs))
return func
return _decorator
def param2id(object_id):
"""Helper function to convert various id types to internal id.
args: [object_id], e.g. 'vol-0000000a' or 'volume-0000000a' or '10'
"""
if uuidutils.is_uuid_like(object_id):
return object_id
elif '-' in object_id:
# FIXME(ja): mapping occurs in nova?
pass
else:
return int(object_id)
class ShellCommands(object):
def bpython(self):
"""Runs a bpython shell.
Falls back to Ipython/python shell if unavailable"""
self.run('bpython')
def ipython(self):
"""Runs an Ipython shell.
Falls back to Python shell if unavailable"""
self.run('ipython')
def python(self):
"""Runs a python shell.
Falls back to Python shell if unavailable"""
self.run('python')
@args('--shell', dest="shell",
metavar='<bpython|ipython|python>',
help='Python shell')
def run(self, shell=None):
"""Runs a Python interactive interpreter."""
if not shell:
shell = 'bpython'
if shell == 'bpython':
try:
import bpython
bpython.embed()
except ImportError:
shell = 'ipython'
if shell == 'ipython':
try:
import IPython
# Explicitly pass an empty list as arguments, because
# otherwise IPython would use sys.argv from this script.
shell = IPython.Shell.IPShell(argv=[])
shell.mainloop()
except ImportError:
shell = 'python'
if shell == 'python':
import code
try:
# Try activating rlcompleter, because it's handy.
import readline
except ImportError:
pass
else:
# We don't have to wrap the following import in a 'try',
# because we already know 'readline' was imported successfully.
import rlcompleter
readline.parse_and_bind("tab:complete")
code.interact()
@args('--path', required=True, help='Script path')
def script(self, path):
"""Runs the script from the specifed path with flags set properly.
arguments: path"""
exec(compile(open(path).read(), path, 'exec'), locals(), globals())
def _db_error(caught_exception):
print caught_exception
print _("The above error may show that the database has not "
"been created.\nPlease create a database using "
"'cinder-manage db sync' before running this command.")
exit(1)
class HostCommands(object):
"""List hosts."""
@args('zone', nargs='?', default=None,
help='Availability Zone (default: %(default)s)')
def list(self, zone=None):
"""Show a list of all physical hosts. Filter by zone.
args: [zone]"""
print "%-25s\t%-15s" % (_('host'),
_('zone'))
ctxt = context.get_admin_context()
services = db.service_get_all(ctxt)
if zone:
services = [s for s in services if s['availability_zone'] == zone]
hosts = []
for srv in services:
if not [h for h in hosts if h['host'] == srv['host']]:
hosts.append(srv)
for h in hosts:
print "%-25s\t%-15s" % (h['host'], h['availability_zone'])
class DbCommands(object):
"""Class for managing the database."""
def __init__(self):
pass
@args('version', nargs='?', default=None,
help='Database version')
def sync(self, version=None):
"""Sync the database up to the most recent version."""
return migration.db_sync(version)
def version(self):
"""Print the current database version."""
print migration.db_version()
class VersionCommands(object):
"""Class for exposing the codebase version."""
def __init__(self):
pass
def list(self):
print(version.version_string())
def __call__(self):
self.list()
class ImportCommands(object):
"""Methods for importing Nova volumes to Cinder.
EXPECTATIONS:
These methods will do two things:
1. Import relevant Nova DB info in to Cinder
2. Import persistent tgt files from Nova to Cinder (see copy_tgt_files)
If you're using VG's (local storage) for your backend YOU MUST install
Cinder on the same node that you're migrating from.
"""
def __init__(self):
pass
def _map_table(self, table):
class Mapper(declarative_base()):
__table__ = table
return Mapper
def _open_session(self, con_info):
# Note(jdg): The echo option below sets whether to dispaly db command
# debug info.
engine = create_engine(con_info,
convert_unicode=True,
echo=False)
session = sessionmaker(bind=engine)
return (session(), engine)
def _backup_cinder_db(self):
#First, dump the dest_db as a backup incase this goes wrong
cinder_dump = utils.execute('mysqldump', 'cinder')
if 'Dump completed on' in cinder_dump[0]:
with open('./cinder_db_bkup.sql', 'w+') as fo:
for line in cinder_dump:
fo.write(line)
else:
raise exception.InvalidResults()
def _import_db(self, src_db, dest_db, backup_db):
# Remember order matters due to FK's
table_list = ['sm_flavors',
'sm_backend_config',
'snapshots',
'volume_types',
'volumes',
'iscsi_targets',
'sm_volume',
'volume_metadata',
'volume_type_extra_specs']
quota_table_list = ['quota_classes',
'quota_usages',
'quotas',
'reservations']
if backup_db > 0:
if 'mysql:' not in dest_db:
print (_('Sorry, only mysql backups are supported!'))
raise exception.InvalidRequest()
else:
self._backup_cinder_db()
(src, src_engine) = self._open_session(src_db)
src_meta = MetaData(bind=src_engine)
(dest, dest_engine) = self._open_session(dest_db)
# First make sure nova is at Folsom
table = Table('migrate_version', src_meta, autoload=True)
if src.query(table).first().version < 132:
print (_('ERROR: Specified Nova DB is not at a compatible '
'migration version!\nNova must be at Folsom or newer '
'to import into Cinder database.'))
sys.exit(2)
for table_name in table_list:
print (_('Importing table %s...') % table_name)
table = Table(table_name, src_meta, autoload=True)
new_row = self._map_table(table)
columns = table.columns.keys()
for row in src.query(table).all():
data = dict([(str(column), getattr(row, column))
for column in columns])
dest.add(new_row(**data))
dest.commit()
for table_name in quota_table_list:
print (_('Importing table %s...') % table_name)
table = Table(table_name, src_meta, autoload=True)
new_row = self._map_table(table)
columns = table.columns.keys()
for row in src.query(table).all():
if row.resource == 'gigabytes' or row.resource == 'volumes':
data = dict([(str(column), getattr(row, column))
for column in columns])
dest.add(new_row(**data))
dest.commit()
@args('src', metavar='<Nova DB>',
help='db-engine://db_user[:passwd]@db_host[:port]\t\t'
'example: mysql://root:secrete@192.168.137.1')
@args('dest', metavar='<Cinder DB>',
help='db-engine://db_user[:passwd]@db_host[:port]\t\t'
'example: mysql://root:secrete@192.168.137.1')
@args('--backup', metavar='<0|1>', choices=[0, 1], default=1,
help='Perform mysqldump of cinder db before writing to it'
' (default: %(default)d)')
def import_db(self, src_db, dest_db, backup_db=1):
"""Import relevant volume DB entries from Nova into Cinder.
NOTE:
Your Cinder DB should be clean WRT volume entries.
NOTE:
We take an sqldump of the cinder DB before mods
If you're not using mysql, set backup_db=0
and create your own backup.
"""
src_db = '%s/nova' % src_db
dest_db = '%s/cinder' % dest_db
self._import_db(src_db, dest_db, backup_db)
@args('src',
help='e.g. (login@src_host:]/opt/stack/nova/volumes/)')
@args('dest', nargs='?', default=None,
help='e.g. (login@src_host:/opt/stack/cinder/volumes/) '
'optional, if emitted, \'volume_dir\' in config will be used')
def copy_ptgt_files(self, src_tgts, dest_tgts=None):
"""Copy persistent scsi tgt files from nova to cinder.
Default destination is FLAGS.volume_dir or state_path/volumes/
PREREQUISITES:
Persistent tgts were introduced in Folsom. If you're running
Essex or other release, this script is unnecessary.
NOTE:
If you're using local VG's and LVM for your nova volume backend
there's no point in copying these files over. Leave them on
your Nova system as they won't do any good here.
"""
if dest_tgts is None:
try:
dest_tgts = FLAGS.volumes_dir
except Exception:
dest_tgts = '%s/volumes' % FLAGS.state_path
utils.execute('rsync', '-avz', src_tgts, dest_tgts)
class VolumeCommands(object):
"""Methods for dealing with a cloud in an odd state."""
@args('volume_id',
help='Volume ID to be deleted')
def delete(self, volume_id):
"""Delete a volume, bypassing the check that it
must be available."""
ctxt = context.get_admin_context()
volume = db.volume_get(ctxt, param2id(volume_id))
host = volume['host']
if not host:
print "Volume not yet assigned to host."
print "Deleting volume from database and skipping rpc."
db.volume_destroy(ctxt, param2id(volume_id))
return
if volume['status'] == 'in-use':
print "Volume is in-use."
print "Detach volume from instance and then try again."
return
rpc.cast(ctxt,
rpc.queue_get_for(ctxt, FLAGS.volume_topic, host),
{"method": "delete_volume",
"args": {"volume_id": volume['id']}})
@args('volume_id',
help='Volume ID to be reattached')
def reattach(self, volume_id):
"""Re-attach a volume that has previously been attached
to an instance. Typically called after a compute host
has been rebooted."""
ctxt = context.get_admin_context()
volume = db.volume_get(ctxt, param2id(volume_id))
if not volume['instance_id']:
print "volume is not attached to an instance"
return
instance = db.instance_get(ctxt, volume['instance_id'])
host = instance['host']
rpc.cast(ctxt,
rpc.queue_get_for(ctxt, FLAGS.compute_topic, host),
{"method": "attach_volume",
"args": {"instance_id": instance['id'],
"volume_id": volume['id'],
"mountpoint": volume['mountpoint']}})
class StorageManagerCommands(object):
"""Class for mangaging Storage Backends and Flavors."""
@args('flavor', nargs='?',
help='flavor to be listed')
def flavor_list(self, flavor=None):
ctxt = context.get_admin_context()
try:
if flavor is None:
flavors = db.sm_flavor_get_all(ctxt)
else:
flavors = db.sm_flavor_get(ctxt, flavor)
except exception.NotFound as ex:
print "error: %s" % ex
sys.exit(2)
print "%-18s\t%-20s\t%s" % (_('id'),
_('Label'),
_('Description'))
for flav in flavors:
print "%-18s\t%-20s\t%s" % (
flav['id'],
flav['label'],
flav['description'])
@args('label', help='flavor label')
@args('desc', help='flavor description')
def flavor_create(self, label, desc):
# TODO(renukaapte) flavor name must be unique
try:
db.sm_flavor_create(context.get_admin_context(),
dict(label=label,
description=desc))
except exception.DBError, e:
_db_error(e)
@args('label', help='label of flavor to be deleted')
def flavor_delete(self, label):
try:
db.sm_flavor_delete(context.get_admin_context(), label)
except exception.DBError, e:
_db_error(e)
def _splitfun(self, item):
i = item.split("=")
return i[0:2]
@args('backend_conf_id', nargs='?', default=None)
def backend_list(self, backend_conf_id=None):
ctxt = context.get_admin_context()
try:
if backend_conf_id is None:
backends = db.sm_backend_conf_get_all(ctxt)
else:
backends = db.sm_backend_conf_get(ctxt, backend_conf_id)
except exception.NotFound as ex:
print "error: %s" % ex
sys.exit(2)
print "%-5s\t%-10s\t%-40s\t%-10s\t%s" % (_('id'),
_('Flavor id'),
_('SR UUID'),
_('SR Type'),
_('Config Parameters'),)
for b in backends:
print "%-5s\t%-10s\t%-40s\t%-10s\t%s" % (b['id'],
b['flavor_id'],
b['sr_uuid'],
b['sr_type'],
b['config_params'],)
@args('flavor_label')
@args('sr_type')
@args('args', nargs='*')
def backend_add(self, flavor_label, sr_type, *args):
# TODO(renukaapte) Add backend_introduce.
ctxt = context.get_admin_context()
params = dict(map(self._splitfun, args))
sr_uuid = uuid.uuid4()
if flavor_label is None:
print "error: backend needs to be associated with flavor"
sys.exit(2)
try:
flavors = db.sm_flavor_get(ctxt, flavor_label)
except exception.NotFound as ex:
print "error: %s" % ex
sys.exit(2)
config_params = " ".join(
['%s=%s' % (key, params[key]) for key in params])
if 'sr_uuid' in params:
sr_uuid = params['sr_uuid']
try:
backend = db.sm_backend_conf_get_by_sr(ctxt, sr_uuid)
except exception.DBError, e:
_db_error(e)
if backend:
print 'Backend config found. Would you like to recreate this?'
print '(WARNING:Recreating will destroy all VDIs on backend!!)'
c = raw_input('Proceed? (y/n) ')
if c == 'y' or c == 'Y':
try:
db.sm_backend_conf_update(
ctxt, backend['id'],
dict(created=False,
flavor_id=flavors['id'],
sr_type=sr_type,
config_params=config_params))
except exception.DBError, e:
_db_error(e)
return
else:
print 'Backend config not found. Would you like to create it?'
print '(WARNING: Creating will destroy all data on backend!!!)'
c = raw_input('Proceed? (y/n) ')
if c == 'y' or c == 'Y':
try:
db.sm_backend_conf_create(ctxt,
dict(flavor_id=flavors['id'],
sr_uuid=sr_uuid,
sr_type=sr_type,
config_params=config_params))
except exception.DBError, e:
_db_error(e)
@args('backend_conf_id')
def backend_remove(self, backend_conf_id):
try:
db.sm_backend_conf_delete(context.get_admin_context(),
backend_conf_id)
except exception.DBError, e:
_db_error(e)
class ConfigCommands(object):
"""Class for exposing the flags defined by flag_file(s)."""
def __init__(self):
pass
def list(self):
for key, value in FLAGS.iteritems():
if value is not None:
print '%s = %s' % (key, value)
class GetLogCommands(object):
"""Get logging information."""
def errors(self):
"""Get all of the errors from the log files."""
error_found = 0
if FLAGS.log_dir:
logs = [x for x in os.listdir(FLAGS.log_dir) if x.endswith('.log')]
for file in logs:
log_file = os.path.join(FLAGS.log_dir, file)
lines = [line.strip() for line in open(log_file, "r")]
lines.reverse()
print_name = 0
for index, line in enumerate(lines):
if line.find(" ERROR ") > 0:
error_found += 1
if print_name == 0:
print log_file + ":-"
print_name = 1
print "Line %d : %s" % (len(lines) - index, line)
if error_found == 0:
print "No errors in logfiles!"
@args('num_entries', nargs='?', type=int, default=10,
help='Number of entries to list (default: %(default)d)')
def syslog(self, num_entries=10):
"""Get <num_entries> of the cinder syslog events."""
entries = int(num_entries)
count = 0
log_file = ''
if os.path.exists('/var/log/syslog'):
log_file = '/var/log/syslog'
elif os.path.exists('/var/log/messages'):
log_file = '/var/log/messages'
else:
print "Unable to find system log file!"
sys.exit(1)
lines = [line.strip() for line in open(log_file, "r")]
lines.reverse()
print "Last %s cinder syslog entries:-" % (entries)
for line in lines:
if line.find("cinder") > 0:
count += 1
print "%s" % (line)
if count == entries:
break
if count == 0:
print "No cinder entries in syslog!"
class BackupCommands(object):
"""Methods for managing backups."""
def list(self):
"""List all backups (including ones in progress) and the host
on which the backup operation is running."""
ctxt = context.get_admin_context()
backups = db.backup_get_all(ctxt)
hdr = "%-32s\t%-32s\t%-32s\t%-24s\t%-24s\t%-12s\t%-12s\t%-12s\t%-12s"
print hdr % (_('ID'),
_('User ID'),
_('Project ID'),
_('Host'),
_('Name'),
_('Container'),
_('Status'),
_('Size'),
_('Object Count'))
res = "%-32s\t%-32s\t%-32s\t%-24s\t%-24s\t%-12s\t%-12s\t%-12d\t%-12d"
for backup in backups:
object_count = 0
if backup['object_count'] is not None:
object_count = backup['object_count']
print res % (backup['id'],
backup['user_id'],
backup['project_id'],
backup['host'],
backup['display_name'],
backup['container'],
backup['status'],
backup['size'],
object_count)
class ServiceCommands(object):
"""Methods for managing services."""
def list(self):
"""Show a list of all cinder services."""
ctxt = context.get_admin_context()
services = db.service_get_all(ctxt)
print_format = "%-16s %-36s %-16s %-10s %-5s %-10s"
print print_format % (
_('Binary'),
_('Host'),
_('Zone'),
_('Status'),
_('State'),
_('Updated At'))
for svc in services:
alive = utils.service_is_up(svc)
art = ":-)" if alive else "XXX"
status = 'enabled'
if svc['disabled']:
status = 'disabled'
print print_format % (svc['binary'], svc['host'].partition('.')[0],
svc['availability_zone'], status, art,
svc['updated_at'])
CATEGORIES = {
'backup': BackupCommands,
'config': ConfigCommands,
'db': DbCommands,
'host': HostCommands,
'logs': GetLogCommands,
'service': ServiceCommands,
'shell': ShellCommands,
'sm': StorageManagerCommands,
'version': VersionCommands,
'volume': VolumeCommands,
'migrate': ImportCommands,
}
def methods_of(obj):
"""Get all callable methods of an object that don't start with underscore
returns a list of tuples of the form (method_name, method)"""
result = []
for i in dir(obj):
if callable(getattr(obj, i)) and not i.startswith('_'):
result.append((i, getattr(obj, i)))
return result
def add_command_parsers(subparsers):
for category in CATEGORIES:
command_object = CATEGORIES[category]()
parser = subparsers.add_parser(category)
parser.set_defaults(command_object=command_object)
category_subparsers = parser.add_subparsers(dest='action')
for (action, action_fn) in methods_of(command_object):
parser = category_subparsers.add_parser(action)
action_kwargs = []
for args, kwargs in getattr(action_fn, 'args', []):
parser.add_argument(*args, **kwargs)
parser.set_defaults(action_fn=action_fn)
parser.set_defaults(action_kwargs=action_kwargs)
category_opt = cfg.SubCommandOpt('category',
title='Command categories',
handler=add_command_parsers)
def get_arg_string(args):
arg = None
if args[0] == '-':
# (Note)zhiteng: args starts with FLAGS.oparser.prefix_chars
# is optional args. Notice that cfg module takes care of
# actual ArgParser so prefix_chars is always '-'.
if args[1] == '-':
# This is long optional arg
arg = args[2:]
else:
arg = args[3:]
else:
arg = args
return arg
def fetch_func_args(func):
fn_args = []
for args, kwargs in getattr(func, 'args', []):
arg = get_arg_string(args[0])
fn_args.append(getattr(FLAGS.category, arg))
return fn_args
def main():
"""Parse options and call the appropriate class/method."""
FLAGS.register_cli_opt(category_opt)
script_name = sys.argv[0]
if len(sys.argv) < 2:
print(_("\nOpenStack Cinder version: %(version)s\n") %
{'version': version.version_string()})
print script_name + " category action [<args>]"
print _("Available categories:")
for category in CATEGORIES:
print "\t%s" % category
sys.exit(2)
try:
flags.parse_args(sys.argv)
logging.setup("cinder")
except cfg.ConfigFilesNotFoundError:
cfgfile = FLAGS.config_file[-1] if FLAGS.config_file else None
if cfgfile and not os.access(cfgfile, os.R_OK):
st = os.stat(cfgfile)
print _("Could not read %s. Re-running with sudo") % cfgfile
try:
os.execvp('sudo', ['sudo', '-u', '#%s' % st.st_uid] + sys.argv)
except Exception:
print _('sudo failed, continuing as if nothing happened')
print _('Please re-run cinder-manage as root.')
sys.exit(2)
fn = FLAGS.category.action_fn
fn_args = fetch_func_args(fn)
fn(*fn_args)
if __name__ == '__main__':
main()

128
bin/cinder-rootwrap Executable file
View File

@ -0,0 +1,128 @@
#!/usr/bin/env python
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright (c) 2011 OpenStack Foundation.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""Root wrapper for OpenStack services
Filters which commands a service is allowed to run as another user.
To use this with cinder, you should set the following in
cinder.conf:
rootwrap_config=/etc/cinder/rootwrap.conf
You also need to let the cinder user run cinder-rootwrap
as root in sudoers:
cinder ALL = (root) NOPASSWD: /usr/bin/cinder-rootwrap
/etc/cinder/rootwrap.conf *
Service packaging should deploy .filters files only on nodes where
they are needed, to avoid allowing more than is necessary.
"""
import ConfigParser
import logging
import os
import pwd
import signal
import subprocess
import sys
RC_UNAUTHORIZED = 99
RC_NOCOMMAND = 98
RC_BADCONFIG = 97
RC_NOEXECFOUND = 96
def _subprocess_setup():
# Python installs a SIGPIPE handler by default. This is usually not what
# non-Python subprocesses expect.
signal.signal(signal.SIGPIPE, signal.SIG_DFL)
def _exit_error(execname, message, errorcode, log=True):
print "%s: %s" % (execname, message)
if log:
logging.error(message)
sys.exit(errorcode)
if __name__ == '__main__':
# Split arguments, require at least a command
execname = sys.argv.pop(0)
if len(sys.argv) < 2:
_exit_error(execname, "No command specified", RC_NOCOMMAND, log=False)
configfile = sys.argv.pop(0)
userargs = sys.argv[:]
# Add ../ to sys.path to allow running from branch
possible_topdir = os.path.normpath(os.path.join(os.path.abspath(execname),
os.pardir, os.pardir))
if os.path.exists(os.path.join(possible_topdir, "cinder", "__init__.py")):
sys.path.insert(0, possible_topdir)
from cinder.openstack.common.rootwrap import wrapper
# Load configuration
try:
rawconfig = ConfigParser.RawConfigParser()
rawconfig.read(configfile)
config = wrapper.RootwrapConfig(rawconfig)
except ValueError as exc:
msg = "Incorrect value in %s: %s" % (configfile, exc.message)
_exit_error(execname, msg, RC_BADCONFIG, log=False)
except ConfigParser.Error:
_exit_error(execname, "Incorrect configuration file: %s" % configfile,
RC_BADCONFIG, log=False)
if config.use_syslog:
wrapper.setup_syslog(execname,
config.syslog_log_facility,
config.syslog_log_level)
# Execute command if it matches any of the loaded filters
filters = wrapper.load_filters(config.filters_path)
try:
filtermatch = wrapper.match_filter(filters, userargs,
exec_dirs=config.exec_dirs)
if filtermatch:
command = filtermatch.get_command(userargs,
exec_dirs=config.exec_dirs)
if config.use_syslog:
logging.info("(%s > %s) Executing %s (filter match = %s)" % (
os.getlogin(), pwd.getpwuid(os.getuid())[0],
command, filtermatch.name))
obj = subprocess.Popen(command,
stdin=sys.stdin,
stdout=sys.stdout,
stderr=sys.stderr,
preexec_fn=_subprocess_setup,
env=filtermatch.get_environment(userargs))
obj.wait()
sys.exit(obj.returncode)
except wrapper.FilterMatchNotExecutable as exc:
msg = ("Executable not found: %s (filter match = %s)"
% (exc.match.exec_path, exc.match.name))
_exit_error(execname, msg, RC_NOEXECFOUND, log=config.use_syslog)
except wrapper.NoFilterMatched:
msg = ("Unauthorized command: %s (no filter matched)"
% ' '.join(userargs))
_exit_error(execname, msg, RC_UNAUTHORIZED, log=config.use_syslog)

53
bin/cinder-rpc-zmq-receiver Executable file
View File

@ -0,0 +1,53 @@
#!/usr/bin/env python
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2011 OpenStack Foundation
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import eventlet
eventlet.monkey_patch()
import contextlib
import os
import sys
# If ../cinder/__init__.py exists, add ../ to Python search path, so that
# it will override what happens to be installed in /usr/(local/)lib/python...
POSSIBLE_TOPDIR = os.path.normpath(os.path.join(os.path.abspath(sys.argv[0]),
os.pardir,
os.pardir))
if os.path.exists(os.path.join(POSSIBLE_TOPDIR, 'cinder', '__init__.py')):
sys.path.insert(0, POSSIBLE_TOPDIR)
from oslo.config import cfg
from cinder.openstack.common import log as logging
from cinder.openstack.common import rpc
from cinder.openstack.common.rpc import impl_zmq
CONF = cfg.CONF
CONF.register_opts(rpc.rpc_opts)
CONF.register_opts(impl_zmq.zmq_opts)
def main():
CONF(sys.argv[1:], project='cinder')
logging.setup("cinder")
with contextlib.closing(impl_zmq.ZmqProxy(CONF)) as reactor:
reactor.consume_in_thread()
reactor.wait()
if __name__ == '__main__':
main()

50
bin/cinder-scheduler Executable file
View File

@ -0,0 +1,50 @@
#!/usr/bin/env python
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2010 United States Government as represented by the
# Administrator of the National Aeronautics and Space Administration.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""Starter script for Cinder Scheduler."""
import eventlet
eventlet.monkey_patch()
import os
import sys
# If ../cinder/__init__.py exists, add ../ to Python search path, so that
# it will override what happens to be installed in /usr/(local/)lib/python...
possible_topdir = os.path.normpath(os.path.join(os.path.abspath(sys.argv[0]),
os.pardir,
os.pardir))
if os.path.exists(os.path.join(possible_topdir, 'cinder', '__init__.py')):
sys.path.insert(0, possible_topdir)
from cinder.openstack.common import gettextutils
gettextutils.install('cinder')
from cinder import flags
from cinder.openstack.common import log as logging
from cinder import service
from cinder import utils
if __name__ == '__main__':
flags.parse_args(sys.argv)
logging.setup("cinder")
utils.monkey_patch()
server = service.Service.create(binary='cinder-scheduler')
service.serve(server)
service.wait()

60
bin/cinder-share Executable file
View File

@ -0,0 +1,60 @@
#!/usr/bin/env python
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2013 NetApp
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""Starter script for Cinder Share."""
import eventlet
eventlet.monkey_patch()
import os
import sys
# If ../cinder/__init__.py exists, add ../ to Python search path, so that
# it will override what happens to be installed in /usr/(local/)lib/python...
possible_topdir = os.path.normpath(os.path.join(os.path.abspath(sys.argv[0]),
os.pardir,
os.pardir))
if os.path.exists(os.path.join(possible_topdir, 'cinder', '__init__.py')):
sys.path.insert(0, possible_topdir)
from cinder.openstack.common import gettextutils
gettextutils.install('cinder')
from cinder import flags
from cinder.openstack.common import log as logging
from cinder import service
from cinder import utils
FLAGS = flags.FLAGS
if __name__ == '__main__':
flags.parse_args(sys.argv)
logging.setup("cinder")
utils.monkey_patch()
launcher = service.ProcessLauncher()
if FLAGS.enabled_share_backends:
for backend in FLAGS.enabled_share_backends:
host = "%s@%s" % (FLAGS.host, backend)
server = service.Service.create(
host=host,
service_name=backend)
launcher.launch_server(server)
else:
server = service.Service.create(binary='cinder-share')
launcher.launch_server(server)
launcher.wait()

61
bin/cinder-volume Executable file
View File

@ -0,0 +1,61 @@
#!/usr/bin/env python
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2010 United States Government as represented by the
# Administrator of the National Aeronautics and Space Administration.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""Starter script for Cinder Volume."""
import eventlet
eventlet.monkey_patch()
import os
import sys
# If ../cinder/__init__.py exists, add ../ to Python search path, so that
# it will override what happens to be installed in /usr/(local/)lib/python...
possible_topdir = os.path.normpath(os.path.join(os.path.abspath(sys.argv[0]),
os.pardir,
os.pardir))
if os.path.exists(os.path.join(possible_topdir, 'cinder', '__init__.py')):
sys.path.insert(0, possible_topdir)
from cinder.openstack.common import gettextutils
gettextutils.install('cinder')
from cinder import flags
from cinder.openstack.common import log as logging
from cinder import service
from cinder import utils
FLAGS = flags.FLAGS
if __name__ == '__main__':
flags.parse_args(sys.argv)
logging.setup("cinder")
utils.monkey_patch()
launcher = service.ProcessLauncher()
if FLAGS.enabled_backends:
for backend in FLAGS.enabled_backends:
host = "%s@%s" % (FLAGS.host, backend)
server = service.Service.create(
host=host,
service_name=backend)
launcher.launch_server(server)
else:
server = service.Service.create(binary='cinder-volume')
launcher.launch_server(server)
launcher.wait()

101
bin/cinder-volume-usage-audit Executable file
View File

@ -0,0 +1,101 @@
#!/usr/bin/env python
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright (c) 2011 OpenStack, LLC.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""Cron script to generate usage notifications for volumes existing during
the audit period.
Together with the notifications generated by volumes
create/delete/resize, over that time period, this allows an external
system consuming usage notification feeds to calculate volume usage
for each tenant.
Time periods are specified as 'hour', 'month', 'day' or 'year'
hour = previous hour. If run at 9:07am, will generate usage for 8-9am.
month = previous month. If the script is run April 1, it will generate
usages for March 1 through March 31.
day = previous day. if run on July 4th, it generates usages for July 3rd.
year = previous year. If run on Jan 1, it generates usages for
Jan 1 through Dec 31 of the previous year.
"""
import os
import sys
import traceback
# If ../cinder/__init__.py exists, add ../ to Python search path, so that
# it will override what happens to be installed in /usr/(local/)lib/python...
POSSIBLE_TOPDIR = os.path.normpath(os.path.join(os.path.abspath(sys.argv[0]),
os.pardir,
os.pardir))
if os.path.exists(os.path.join(POSSIBLE_TOPDIR, 'cinder', '__init__.py')):
sys.path.insert(0, POSSIBLE_TOPDIR)
from cinder.openstack.common import gettextutils
gettextutils.install('cinder')
from cinder import context
from cinder import db
from cinder import flags
from cinder.openstack.common import log as logging
from cinder.openstack.common import rpc
from cinder import utils
import cinder.volume.utils
FLAGS = flags.FLAGS
if __name__ == '__main__':
admin_context = context.get_admin_context()
flags.parse_args(sys.argv)
logging.setup("cinder")
begin, end = utils.last_completed_audit_period()
print _("Starting volume usage audit")
msg = _("Creating usages for %(begin_period)s until %(end_period)s")
print (msg % {"begin_period": str(begin), "end_period": str(end)})
extra_info = {
'audit_period_beginning': str(begin),
'audit_period_ending': str(end),
}
volumes = db.volume_get_active_by_window(admin_context,
begin,
end)
print _("Found %d volumes") % len(volumes)
for volume_ref in volumes:
try:
cinder.volume.utils.notify_usage_exists(
admin_context, volume_ref)
except Exception, e:
print traceback.format_exc(e)
snapshots = db.snapshot_get_active_by_window(admin_context,
begin,
end)
print _("Found %d snapshots") % len(snapshots)
for snapshot_ref in snapshots:
try:
cinder.volume.utils.notify_about_snapshot_usage(admin_context,
snapshot_ref,
'exists',
extra_info)
except Exception, e:
print traceback.fromat_exc(e)
print _("Volume usage audit completed")

32
cinder/__init__.py Normal file
View File

@ -0,0 +1,32 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2010 United States Government as represented by the
# Administrator of the National Aeronautics and Space Administration.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""
:mod:`cinder` -- Cloud IaaS Platform
===================================
.. automodule:: cinder
:platform: Unix
:synopsis: Infrastructure-as-a-Service Cloud platform.
.. moduleauthor:: Jesse Andrews <jesse@ansolabs.com>
.. moduleauthor:: Devin Carlen <devin.carlen@gmail.com>
.. moduleauthor:: Vishvananda Ishaya <vishvananda@gmail.com>
.. moduleauthor:: Joshua McKenty <joshua@cognition.ca>
.. moduleauthor:: Manish Singh <yosh@gimp.org>
.. moduleauthor:: Andy Smith <andy@anarkystic.com>
"""

32
cinder/api/__init__.py Normal file
View File

@ -0,0 +1,32 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2010 United States Government as represented by the
# Administrator of the National Aeronautics and Space Administration.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import paste.urlmap
from cinder import flags
FLAGS = flags.FLAGS
def root_app_factory(loader, global_conf, **local_conf):
if not FLAGS.enable_v1_api:
del local_conf['/v1']
if not FLAGS.enable_v2_api:
del local_conf['/v2']
return paste.urlmap.urlmap_factory(loader, global_conf, **local_conf)

36
cinder/api/auth.py Normal file
View File

@ -0,0 +1,36 @@
# Copyright (c) 2013 OpenStack, LLC.
#
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from cinder.api.middleware import auth
from cinder.openstack.common import log as logging
LOG = logging.getLogger(__name__)
class CinderKeystoneContext(auth.CinderKeystoneContext):
def __init__(self, application):
LOG.warn(_('cinder.api.auth:CinderKeystoneContext is deprecated. '
'Please use '
'cinder.api.middleware.auth:CinderKeystoneContext '
'instead.'))
super(CinderKeystoneContext, self).__init__(application)
def pipeline_factory(loader, global_conf, **local_conf):
LOG.warn(_('cinder.api.auth:pipeline_factory is deprecated. Please use '
'cinder.api.middleware.auth:pipeline_factory instead.'))
auth.pipeline_factory(loader, global_conf, **local_conf)

314
cinder/api/common.py Normal file
View File

@ -0,0 +1,314 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2010 OpenStack LLC.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import os
import re
import urlparse
import webob
from cinder.api.openstack import wsgi
from cinder.api import xmlutil
from cinder import flags
from cinder.openstack.common import log as logging
from cinder import utils
LOG = logging.getLogger(__name__)
FLAGS = flags.FLAGS
XML_NS_V1 = 'http://docs.openstack.org/volume/api/v1'
def get_pagination_params(request):
"""Return marker, limit tuple from request.
:param request: `wsgi.Request` possibly containing 'marker' and 'limit'
GET variables. 'marker' is the id of the last element
the client has seen, and 'limit' is the maximum number
of items to return. If 'limit' is not specified, 0, or
> max_limit, we default to max_limit. Negative values
for either marker or limit will cause
exc.HTTPBadRequest() exceptions to be raised.
"""
params = {}
if 'limit' in request.GET:
params['limit'] = _get_limit_param(request)
if 'marker' in request.GET:
params['marker'] = _get_marker_param(request)
return params
def _get_limit_param(request):
"""Extract integer limit from request or fail"""
try:
limit = int(request.GET['limit'])
except ValueError:
msg = _('limit param must be an integer')
raise webob.exc.HTTPBadRequest(explanation=msg)
if limit < 0:
msg = _('limit param must be positive')
raise webob.exc.HTTPBadRequest(explanation=msg)
return limit
def _get_marker_param(request):
"""Extract marker id from request or fail"""
return request.GET['marker']
def limited(items, request, max_limit=FLAGS.osapi_max_limit):
"""Return a slice of items according to requested offset and limit.
:param items: A sliceable entity
:param request: ``wsgi.Request`` possibly containing 'offset' and 'limit'
GET variables. 'offset' is where to start in the list,
and 'limit' is the maximum number of items to return. If
'limit' is not specified, 0, or > max_limit, we default
to max_limit. Negative values for either offset or limit
will cause exc.HTTPBadRequest() exceptions to be raised.
:kwarg max_limit: The maximum number of items to return from 'items'
"""
try:
offset = int(request.GET.get('offset', 0))
except ValueError:
msg = _('offset param must be an integer')
raise webob.exc.HTTPBadRequest(explanation=msg)
try:
limit = int(request.GET.get('limit', max_limit))
except ValueError:
msg = _('limit param must be an integer')
raise webob.exc.HTTPBadRequest(explanation=msg)
if limit < 0:
msg = _('limit param must be positive')
raise webob.exc.HTTPBadRequest(explanation=msg)
if offset < 0:
msg = _('offset param must be positive')
raise webob.exc.HTTPBadRequest(explanation=msg)
limit = min(max_limit, limit or max_limit)
range_end = offset + limit
return items[offset:range_end]
def limited_by_marker(items, request, max_limit=FLAGS.osapi_max_limit):
"""Return a slice of items according to the requested marker and limit."""
params = get_pagination_params(request)
limit = params.get('limit', max_limit)
marker = params.get('marker')
limit = min(max_limit, limit)
start_index = 0
if marker:
start_index = -1
for i, item in enumerate(items):
if 'flavorid' in item:
if item['flavorid'] == marker:
start_index = i + 1
break
elif item['id'] == marker or item.get('uuid') == marker:
start_index = i + 1
break
if start_index < 0:
msg = _('marker [%s] not found') % marker
raise webob.exc.HTTPBadRequest(explanation=msg)
range_end = start_index + limit
return items[start_index:range_end]
def remove_version_from_href(href):
"""Removes the first api version from the href.
Given: 'http://www.cinder.com/v1.1/123'
Returns: 'http://www.cinder.com/123'
Given: 'http://www.cinder.com/v1.1'
Returns: 'http://www.cinder.com'
"""
parsed_url = urlparse.urlsplit(href)
url_parts = parsed_url.path.split('/', 2)
# NOTE: this should match vX.X or vX
expression = re.compile(r'^v([0-9]+|[0-9]+\.[0-9]+)(/.*|$)')
if expression.match(url_parts[1]):
del url_parts[1]
new_path = '/'.join(url_parts)
if new_path == parsed_url.path:
msg = _('href %s does not contain version') % href
LOG.debug(msg)
raise ValueError(msg)
parsed_url = list(parsed_url)
parsed_url[2] = new_path
return urlparse.urlunsplit(parsed_url)
def dict_to_query_str(params):
# TODO(throughnothing): we should just use urllib.urlencode instead of this
# But currently we don't work with urlencoded url's
param_str = ""
for key, val in params.iteritems():
param_str = param_str + '='.join([str(key), str(val)]) + '&'
return param_str.rstrip('&')
class ViewBuilder(object):
"""Model API responses as dictionaries."""
_collection_name = None
def _get_links(self, request, identifier):
return [{"rel": "self",
"href": self._get_href_link(request, identifier), },
{"rel": "bookmark",
"href": self._get_bookmark_link(request, identifier), }]
def _get_next_link(self, request, identifier):
"""Return href string with proper limit and marker params."""
params = request.params.copy()
params["marker"] = identifier
prefix = self._update_link_prefix(request.application_url,
FLAGS.osapi_volume_base_URL)
url = os.path.join(prefix,
request.environ["cinder.context"].project_id,
self._collection_name)
return "%s?%s" % (url, dict_to_query_str(params))
def _get_href_link(self, request, identifier):
"""Return an href string pointing to this object."""
prefix = self._update_link_prefix(request.application_url,
FLAGS.osapi_volume_base_URL)
return os.path.join(prefix,
request.environ["cinder.context"].project_id,
self._collection_name,
str(identifier))
def _get_bookmark_link(self, request, identifier):
"""Create a URL that refers to a specific resource."""
base_url = remove_version_from_href(request.application_url)
base_url = self._update_link_prefix(base_url,
FLAGS.osapi_volume_base_URL)
return os.path.join(base_url,
request.environ["cinder.context"].project_id,
self._collection_name,
str(identifier))
def _get_collection_links(self, request, items, id_key="uuid"):
"""Retrieve 'next' link, if applicable."""
links = []
limit = int(request.params.get("limit", 0))
if limit and limit == len(items):
last_item = items[-1]
if id_key in last_item:
last_item_id = last_item[id_key]
else:
last_item_id = last_item["id"]
links.append({
"rel": "next",
"href": self._get_next_link(request, last_item_id),
})
return links
def _update_link_prefix(self, orig_url, prefix):
if not prefix:
return orig_url
url_parts = list(urlparse.urlsplit(orig_url))
prefix_parts = list(urlparse.urlsplit(prefix))
url_parts[0:2] = prefix_parts[0:2]
return urlparse.urlunsplit(url_parts)
class MetadataDeserializer(wsgi.MetadataXMLDeserializer):
def deserialize(self, text):
dom = utils.safe_minidom_parse_string(text)
metadata_node = self.find_first_child_named(dom, "metadata")
metadata = self.extract_metadata(metadata_node)
return {'body': {'metadata': metadata}}
class MetaItemDeserializer(wsgi.MetadataXMLDeserializer):
def deserialize(self, text):
dom = utils.safe_minidom_parse_string(text)
metadata_item = self.extract_metadata(dom)
return {'body': {'meta': metadata_item}}
class MetadataXMLDeserializer(wsgi.XMLDeserializer):
def extract_metadata(self, metadata_node):
"""Marshal the metadata attribute of a parsed request"""
if metadata_node is None:
return {}
metadata = {}
for meta_node in self.find_children_named(metadata_node, "meta"):
key = meta_node.getAttribute("key")
metadata[key] = self.extract_text(meta_node)
return metadata
def _extract_metadata_container(self, datastring):
dom = utils.safe_minidom_parse_string(datastring)
metadata_node = self.find_first_child_named(dom, "metadata")
metadata = self.extract_metadata(metadata_node)
return {'body': {'metadata': metadata}}
def create(self, datastring):
return self._extract_metadata_container(datastring)
def update_all(self, datastring):
return self._extract_metadata_container(datastring)
def update(self, datastring):
dom = utils.safe_minidom_parse_string(datastring)
metadata_item = self.extract_metadata(dom)
return {'body': {'meta': metadata_item}}
metadata_nsmap = {None: xmlutil.XMLNS_V11}
class MetaItemTemplate(xmlutil.TemplateBuilder):
def construct(self):
sel = xmlutil.Selector('meta', xmlutil.get_items, 0)
root = xmlutil.TemplateElement('meta', selector=sel)
root.set('key', 0)
root.text = 1
return xmlutil.MasterTemplate(root, 1, nsmap=metadata_nsmap)
class MetadataTemplateElement(xmlutil.TemplateElement):
def will_render(self, datum):
return True
class MetadataTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = MetadataTemplateElement('metadata', selector='metadata')
elem = xmlutil.SubTemplateElement(root, 'meta',
selector=xmlutil.get_items)
elem.set('key', 0)
elem.text = 1
return xmlutil.MasterTemplate(root, 1, nsmap=metadata_nsmap)

View File

@ -0,0 +1,39 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2011 Justin Santa Barbara
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""Contrib contains extensions that are shipped with cinder.
It can't be called 'extensions' because that causes namespacing problems.
"""
from cinder.api import extensions
from cinder import flags
from cinder.openstack.common import log as logging
FLAGS = flags.FLAGS
LOG = logging.getLogger(__name__)
def standard_extensions(ext_mgr):
extensions.load_standard_extensions(ext_mgr, LOG, __path__, __package__)
def select_extensions(ext_mgr):
extensions.load_standard_extensions(ext_mgr, LOG, __path__, __package__,
FLAGS.osapi_volume_ext_list)

View File

@ -0,0 +1,174 @@
# Copyright 2012 OpenStack, LLC.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import webob
from webob import exc
from cinder.api import extensions
from cinder.api.openstack import wsgi
from cinder import db
from cinder import exception
from cinder.openstack.common import log as logging
from cinder import volume
LOG = logging.getLogger(__name__)
class AdminController(wsgi.Controller):
"""Abstract base class for AdminControllers."""
collection = None # api collection to extend
# FIXME(clayg): this will be hard to keep up-to-date
# Concrete classes can expand or over-ride
valid_status = set([
'creating',
'available',
'deleting',
'error',
'error_deleting',
])
def __init__(self, *args, **kwargs):
super(AdminController, self).__init__(*args, **kwargs)
# singular name of the resource
self.resource_name = self.collection.rstrip('s')
self.volume_api = volume.API()
def _update(self, *args, **kwargs):
raise NotImplementedError()
def _get(self, *args, **kwargs):
raise NotImplementedError()
def _delete(self, *args, **kwargs):
raise NotImplementedError()
def validate_update(self, body):
update = {}
try:
update['status'] = body['status']
except (TypeError, KeyError):
raise exc.HTTPBadRequest("Must specify 'status'")
if update['status'] not in self.valid_status:
raise exc.HTTPBadRequest("Must specify a valid status")
return update
def authorize(self, context, action_name):
# e.g. "snapshot_admin_actions:reset_status"
action = '%s_admin_actions:%s' % (self.resource_name, action_name)
extensions.extension_authorizer('volume', action)(context)
@wsgi.action('os-reset_status')
def _reset_status(self, req, id, body):
"""Reset status on the resource."""
context = req.environ['cinder.context']
self.authorize(context, 'reset_status')
update = self.validate_update(body['os-reset_status'])
msg = _("Updating %(resource)s '%(id)s' with '%(update)r'")
LOG.debug(msg, {'resource': self.resource_name, 'id': id,
'update': update})
try:
self._update(context, id, update)
except exception.NotFound, e:
raise exc.HTTPNotFound(e)
return webob.Response(status_int=202)
@wsgi.action('os-force_delete')
def _force_delete(self, req, id, body):
"""Delete a resource, bypassing the check that it must be available."""
context = req.environ['cinder.context']
self.authorize(context, 'force_delete')
try:
resource = self._get(context, id)
except exception.NotFound:
raise exc.HTTPNotFound()
self._delete(context, resource, force=True)
return webob.Response(status_int=202)
class VolumeAdminController(AdminController):
"""AdminController for Volumes."""
collection = 'volumes'
valid_status = AdminController.valid_status.union(
set(['attaching', 'in-use', 'detaching']))
def _update(self, *args, **kwargs):
db.volume_update(*args, **kwargs)
def _get(self, *args, **kwargs):
return self.volume_api.get(*args, **kwargs)
def _delete(self, *args, **kwargs):
return self.volume_api.delete(*args, **kwargs)
def validate_update(self, body):
update = super(VolumeAdminController, self).validate_update(body)
if 'attach_status' in body:
if body['attach_status'] not in ('detached', 'attached'):
raise exc.HTTPBadRequest("Must specify a valid attach_status")
update['attach_status'] = body['attach_status']
return update
@wsgi.action('os-force_detach')
def _force_detach(self, req, id, body):
"""
Roll back a bad detach after the volume been disconnected from
the hypervisor.
"""
context = req.environ['cinder.context']
self.authorize(context, 'force_detach')
try:
volume = self._get(context, id)
except exception.NotFound:
raise exc.HTTPNotFound()
self.volume_api.terminate_connection(context, volume,
{}, force=True)
self.volume_api.detach(context, volume)
return webob.Response(status_int=202)
class SnapshotAdminController(AdminController):
"""AdminController for Snapshots."""
collection = 'snapshots'
def _update(self, *args, **kwargs):
db.snapshot_update(*args, **kwargs)
def _get(self, *args, **kwargs):
return self.volume_api.get_snapshot(*args, **kwargs)
def _delete(self, *args, **kwargs):
return self.volume_api.delete_snapshot(*args, **kwargs)
class Admin_actions(extensions.ExtensionDescriptor):
"""Enable admin actions."""
name = "AdminActions"
alias = "os-admin-actions"
namespace = "http://docs.openstack.org/volume/ext/admin-actions/api/v1.1"
updated = "2012-08-25T00:00:00+00:00"
def get_controller_extensions(self):
exts = []
for class_ in (VolumeAdminController, SnapshotAdminController):
controller = class_()
extension = extensions.ControllerExtension(
self, class_.collection, controller)
exts.append(extension)
return exts

View File

@ -0,0 +1,278 @@
# Copyright (C) 2012 Hewlett-Packard Development Company, L.P.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""The backups api."""
import webob
from webob import exc
from xml.dom import minidom
from cinder.api import common
from cinder.api import extensions
from cinder.api.openstack import wsgi
from cinder.api.views import backups as backup_views
from cinder.api import xmlutil
from cinder import backup as backupAPI
from cinder import exception
from cinder import flags
from cinder.openstack.common import log as logging
FLAGS = flags.FLAGS
LOG = logging.getLogger(__name__)
def make_backup(elem):
elem.set('id')
elem.set('status')
elem.set('size')
elem.set('container')
elem.set('volume_id')
elem.set('object_count')
elem.set('availability_zone')
elem.set('created_at')
elem.set('name')
elem.set('description')
elem.set('fail_reason')
def make_backup_restore(elem):
elem.set('backup_id')
elem.set('volume_id')
class BackupTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('backup', selector='backup')
make_backup(root)
alias = Backups.alias
namespace = Backups.namespace
return xmlutil.MasterTemplate(root, 1, nsmap={alias: namespace})
class BackupsTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('backups')
elem = xmlutil.SubTemplateElement(root, 'backup', selector='backups')
make_backup(elem)
alias = Backups.alias
namespace = Backups.namespace
return xmlutil.MasterTemplate(root, 1, nsmap={alias: namespace})
class BackupRestoreTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('restore', selector='restore')
make_backup_restore(root)
alias = Backups.alias
namespace = Backups.namespace
return xmlutil.MasterTemplate(root, 1, nsmap={alias: namespace})
class CreateDeserializer(wsgi.MetadataXMLDeserializer):
def default(self, string):
dom = minidom.parseString(string)
backup = self._extract_backup(dom)
return {'body': {'backup': backup}}
def _extract_backup(self, node):
backup = {}
backup_node = self.find_first_child_named(node, 'backup')
attributes = ['container', 'display_name',
'display_description', 'volume_id']
for attr in attributes:
if backup_node.getAttribute(attr):
backup[attr] = backup_node.getAttribute(attr)
return backup
class RestoreDeserializer(wsgi.MetadataXMLDeserializer):
def default(self, string):
dom = minidom.parseString(string)
restore = self._extract_restore(dom)
return {'body': {'restore': restore}}
def _extract_restore(self, node):
restore = {}
restore_node = self.find_first_child_named(node, 'restore')
if restore_node.getAttribute('volume_id'):
restore['volume_id'] = restore_node.getAttribute('volume_id')
return restore
class BackupsController(wsgi.Controller):
"""The Backups API controller for the OpenStack API."""
_view_builder_class = backup_views.ViewBuilder
def __init__(self):
self.backup_api = backupAPI.API()
super(BackupsController, self).__init__()
@wsgi.serializers(xml=BackupTemplate)
def show(self, req, id):
"""Return data about the given backup."""
LOG.debug(_('show called for member %s'), id)
context = req.environ['cinder.context']
try:
backup = self.backup_api.get(context, backup_id=id)
except exception.BackupNotFound as error:
raise exc.HTTPNotFound(explanation=unicode(error))
return self._view_builder.detail(req, backup)
def delete(self, req, id):
"""Delete a backup."""
LOG.debug(_('delete called for member %s'), id)
context = req.environ['cinder.context']
LOG.audit(_('Delete backup with id: %s'), id, context=context)
try:
self.backup_api.delete(context, id)
except exception.BackupNotFound as error:
raise exc.HTTPNotFound(explanation=unicode(error))
except exception.InvalidBackup as error:
raise exc.HTTPBadRequest(explanation=unicode(error))
return webob.Response(status_int=202)
@wsgi.serializers(xml=BackupsTemplate)
def index(self, req):
"""Returns a summary list of backups."""
return self._get_backups(req, is_detail=False)
@wsgi.serializers(xml=BackupsTemplate)
def detail(self, req):
"""Returns a detailed list of backups."""
return self._get_backups(req, is_detail=True)
def _get_backups(self, req, is_detail):
"""Returns a list of backups, transformed through view builder."""
context = req.environ['cinder.context']
backups = self.backup_api.get_all(context)
limited_list = common.limited(backups, req)
if is_detail:
backups = self._view_builder.detail_list(req, limited_list)
else:
backups = self._view_builder.summary_list(req, limited_list)
return backups
# TODO(frankm): Add some checks here including
# - whether requested volume_id exists so we can return some errors
# immediately
# - maybe also do validation of swift container name
@wsgi.response(202)
@wsgi.serializers(xml=BackupTemplate)
@wsgi.deserializers(xml=CreateDeserializer)
def create(self, req, body):
"""Create a new backup."""
LOG.debug(_('Creating new backup %s'), body)
if not self.is_valid_body(body, 'backup'):
raise exc.HTTPBadRequest()
context = req.environ['cinder.context']
try:
backup = body['backup']
volume_id = backup['volume_id']
except KeyError:
msg = _("Incorrect request body format")
raise exc.HTTPBadRequest(explanation=msg)
container = backup.get('container', None)
name = backup.get('name', None)
description = backup.get('description', None)
LOG.audit(_("Creating backup of volume %(volume_id)s in container"
" %(container)s"), locals(), context=context)
try:
new_backup = self.backup_api.create(context, name, description,
volume_id, container)
except exception.InvalidVolume as error:
raise exc.HTTPBadRequest(explanation=unicode(error))
except exception.VolumeNotFound as error:
raise exc.HTTPNotFound(explanation=unicode(error))
retval = self._view_builder.summary(req, dict(new_backup.iteritems()))
return retval
@wsgi.response(202)
@wsgi.serializers(xml=BackupRestoreTemplate)
@wsgi.deserializers(xml=RestoreDeserializer)
def restore(self, req, id, body):
"""Restore an existing backup to a volume."""
backup_id = id
LOG.debug(_('Restoring backup %(backup_id)s (%(body)s)') % locals())
if not self.is_valid_body(body, 'restore'):
raise exc.HTTPBadRequest()
context = req.environ['cinder.context']
try:
restore = body['restore']
except KeyError:
msg = _("Incorrect request body format")
raise exc.HTTPBadRequest(explanation=msg)
volume_id = restore.get('volume_id', None)
LOG.audit(_("Restoring backup %(backup_id)s to volume %(volume_id)s"),
locals(), context=context)
try:
new_restore = self.backup_api.restore(context,
backup_id=backup_id,
volume_id=volume_id)
except exception.InvalidInput as error:
raise exc.HTTPBadRequest(explanation=unicode(error))
except exception.InvalidVolume as error:
raise exc.HTTPBadRequest(explanation=unicode(error))
except exception.InvalidBackup as error:
raise exc.HTTPBadRequest(explanation=unicode(error))
except exception.BackupNotFound as error:
raise exc.HTTPNotFound(explanation=unicode(error))
except exception.VolumeNotFound as error:
raise exc.HTTPNotFound(explanation=unicode(error))
except exception.VolumeSizeExceedsAvailableQuota as error:
raise exc.HTTPRequestEntityTooLarge(
explanation=error.message, headers={'Retry-After': 0})
except exception.VolumeLimitExceeded as error:
raise exc.HTTPRequestEntityTooLarge(
explanation=error.message, headers={'Retry-After': 0})
retval = self._view_builder.restore_summary(
req, dict(new_restore.iteritems()))
return retval
class Backups(extensions.ExtensionDescriptor):
"""Backups support."""
name = 'Backups'
alias = 'backups'
namespace = 'http://docs.openstack.org/volume/ext/backups/api/v1'
updated = '2012-12-12T00:00:00+00:00'
def get_resources(self):
resources = []
res = extensions.ResourceExtension(
Backups.alias, BackupsController(),
collection_actions={'detail': 'GET'},
member_actions={'restore': 'POST'})
resources.append(res)
return resources

View File

@ -0,0 +1,125 @@
# Copyright 2012 OpenStack, LLC.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""The Extended Snapshot Attributes API extension."""
from webob import exc
from cinder.api import extensions
from cinder.api.openstack import wsgi
from cinder.api import xmlutil
from cinder import exception
from cinder import flags
from cinder.openstack.common import log as logging
from cinder import volume
FLAGS = flags.FLAGS
LOG = logging.getLogger(__name__)
authorize = extensions.soft_extension_authorizer(
'volume',
'extended_snapshot_attributes')
class ExtendedSnapshotAttributesController(wsgi.Controller):
def __init__(self, *args, **kwargs):
super(ExtendedSnapshotAttributesController, self).__init__(*args,
**kwargs)
self.volume_api = volume.API()
def _get_snapshots(self, context):
snapshots = self.volume_api.get_all_snapshots(context)
rval = dict((snapshot['id'], snapshot) for snapshot in snapshots)
return rval
def _extend_snapshot(self, context, snapshot, data):
for attr in ['project_id', 'progress']:
key = "%s:%s" % (Extended_snapshot_attributes.alias, attr)
snapshot[key] = data[attr]
@wsgi.extends
def show(self, req, resp_obj, id):
context = req.environ['cinder.context']
if authorize(context):
# Attach our slave template to the response object
resp_obj.attach(xml=ExtendedSnapshotAttributeTemplate())
try:
snapshot = self.volume_api.get_snapshot(context, id)
except exception.NotFound:
explanation = _("Snapshot not found.")
raise exc.HTTPNotFound(explanation=explanation)
self._extend_snapshot(context, resp_obj.obj['snapshot'], snapshot)
@wsgi.extends
def detail(self, req, resp_obj):
context = req.environ['cinder.context']
if authorize(context):
# Attach our slave template to the response object
resp_obj.attach(xml=ExtendedSnapshotAttributesTemplate())
snapshots = list(resp_obj.obj.get('snapshots', []))
db_snapshots = self._get_snapshots(context)
for snapshot_object in snapshots:
try:
snapshot_data = db_snapshots[snapshot_object['id']]
except KeyError:
continue
self._extend_snapshot(context, snapshot_object, snapshot_data)
class Extended_snapshot_attributes(extensions.ExtensionDescriptor):
"""Extended SnapshotAttributes support."""
name = "ExtendedSnapshotAttributes"
alias = "os-extended-snapshot-attributes"
namespace = ("http://docs.openstack.org/volume/ext/"
"extended_snapshot_attributes/api/v1")
updated = "2012-06-19T00:00:00+00:00"
def get_controller_extensions(self):
controller = ExtendedSnapshotAttributesController()
extension = extensions.ControllerExtension(self, 'snapshots',
controller)
return [extension]
def make_snapshot(elem):
elem.set('{%s}project_id' % Extended_snapshot_attributes.namespace,
'%s:project_id' % Extended_snapshot_attributes.alias)
elem.set('{%s}progress' % Extended_snapshot_attributes.namespace,
'%s:progress' % Extended_snapshot_attributes.alias)
class ExtendedSnapshotAttributeTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('snapshot', selector='snapshot')
make_snapshot(root)
alias = Extended_snapshot_attributes.alias
namespace = Extended_snapshot_attributes.namespace
return xmlutil.SlaveTemplate(root, 1, nsmap={alias: namespace})
class ExtendedSnapshotAttributesTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('snapshots')
elem = xmlutil.SubTemplateElement(root, 'snapshot',
selector='snapshots')
make_snapshot(elem)
alias = Extended_snapshot_attributes.alias
namespace = Extended_snapshot_attributes.namespace
return xmlutil.SlaveTemplate(root, 1, nsmap={alias: namespace})

265
cinder/api/contrib/hosts.py Normal file
View File

@ -0,0 +1,265 @@
# Copyright (c) 2011 OpenStack, LLC.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""The hosts admin extension."""
import webob.exc
from xml.parsers import expat
from cinder.api import extensions
from cinder.api.openstack import wsgi
from cinder.api import xmlutil
from cinder import db
from cinder import exception
from cinder import flags
from cinder.openstack.common import log as logging
from cinder.openstack.common import timeutils
from cinder import utils
from cinder.volume import api as volume_api
FLAGS = flags.FLAGS
LOG = logging.getLogger(__name__)
authorize = extensions.extension_authorizer('volume', 'hosts')
class HostIndexTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('hosts')
elem = xmlutil.SubTemplateElement(root, 'host', selector='hosts')
elem.set('service-status')
elem.set('service')
elem.set('zone')
elem.set('service-state')
elem.set('host_name')
elem.set('last-update')
return xmlutil.MasterTemplate(root, 1)
class HostUpdateTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('host')
root.set('host')
root.set('status')
return xmlutil.MasterTemplate(root, 1)
class HostActionTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('host')
root.set('host')
return xmlutil.MasterTemplate(root, 1)
class HostShowTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('host')
elem = xmlutil.make_flat_dict('resource', selector='host',
subselector='resource')
root.append(elem)
return xmlutil.MasterTemplate(root, 1)
class HostDeserializer(wsgi.XMLDeserializer):
def default(self, string):
try:
node = utils.safe_minidom_parse_string(string)
except expat.ExpatError:
msg = _("cannot understand XML")
raise exception.MalformedRequestBody(reason=msg)
updates = {}
for child in node.childNodes[0].childNodes:
updates[child.tagName] = self.extract_text(child)
return dict(body=updates)
def _list_hosts(req, service=None):
"""Returns a summary list of hosts."""
curr_time = timeutils.utcnow()
context = req.environ['cinder.context']
services = db.service_get_all(context, False)
zone = ''
if 'zone' in req.GET:
zone = req.GET['zone']
if zone:
services = [s for s in services if s['availability_zone'] == zone]
hosts = []
for host in services:
delta = curr_time - (host['updated_at'] or host['created_at'])
alive = abs(utils.total_seconds(delta)) <= FLAGS.service_down_time
status = (alive and "available") or "unavailable"
active = 'enabled'
if host['disabled']:
active = 'disabled'
LOG.debug('status, active and update: %s, %s, %s' %
(status, active, host['updated_at']))
hosts.append({'host_name': host['host'],
'service': host['topic'],
'zone': host['availability_zone'],
'service-status': status,
'service-state': active,
'last-update': host['updated_at']})
if service:
hosts = [host for host in hosts
if host["service"] == service]
return hosts
def check_host(fn):
"""Makes sure that the host exists."""
def wrapped(self, req, id, service=None, *args, **kwargs):
listed_hosts = _list_hosts(req, service)
hosts = [h["host_name"] for h in listed_hosts]
if id in hosts:
return fn(self, req, id, *args, **kwargs)
else:
message = _("Host '%s' could not be found.") % id
raise webob.exc.HTTPNotFound(explanation=message)
return wrapped
class HostController(object):
"""The Hosts API controller for the OpenStack API."""
def __init__(self):
self.api = volume_api.HostAPI()
super(HostController, self).__init__()
@wsgi.serializers(xml=HostIndexTemplate)
def index(self, req):
authorize(req.environ['cinder.context'])
return {'hosts': _list_hosts(req)}
@wsgi.serializers(xml=HostUpdateTemplate)
@wsgi.deserializers(xml=HostDeserializer)
@check_host
def update(self, req, id, body):
authorize(req.environ['cinder.context'])
update_values = {}
for raw_key, raw_val in body.iteritems():
key = raw_key.lower().strip()
val = raw_val.lower().strip()
if key == "status":
if val in ("enable", "disable"):
update_values['status'] = val.startswith("enable")
else:
explanation = _("Invalid status: '%s'") % raw_val
raise webob.exc.HTTPBadRequest(explanation=explanation)
else:
explanation = _("Invalid update setting: '%s'") % raw_key
raise webob.exc.HTTPBadRequest(explanation=explanation)
update_setters = {'status': self._set_enabled_status}
result = {}
for key, value in update_values.iteritems():
result.update(update_setters[key](req, id, value))
return result
def _set_enabled_status(self, req, host, enabled):
"""Sets the specified host's ability to accept new volumes."""
context = req.environ['cinder.context']
state = "enabled" if enabled else "disabled"
LOG.audit(_("Setting host %(host)s to %(state)s.") % locals())
result = self.api.set_host_enabled(context,
host=host,
enabled=enabled)
if result not in ("enabled", "disabled"):
# An error message was returned
raise webob.exc.HTTPBadRequest(explanation=result)
return {"host": host, "status": result}
@wsgi.serializers(xml=HostShowTemplate)
def show(self, req, id):
"""Shows the volume usage info given by hosts.
:param context: security context
:param host: hostname
:returns: expected to use HostShowTemplate.
ex.::
{'host': {'resource':D},..}
D: {'host': 'hostname','project': 'admin',
'volume_count': 1, 'total_volume_gb': 2048}
"""
host = id
context = req.environ['cinder.context']
if not context.is_admin:
msg = _("Describe-resource is admin only functionality")
raise webob.exc.HTTPForbidden(explanation=msg)
try:
host_ref = db.service_get_by_host_and_topic(context,
host,
FLAGS.volume_topic)
except exception.ServiceNotFound:
raise webob.exc.HTTPNotFound(explanation=_("Host not found"))
# Getting total available/used resource
# TODO(jdg): Add summary info for Snapshots
volume_refs = db.volume_get_all_by_host(context, host_ref['host'])
(count, sum) = db.volume_data_get_for_host(context,
host_ref['host'])
snap_count_total = 0
snap_sum_total = 0
resources = [{'resource': {'host': host, 'project': '(total)',
'volume_count': str(count),
'total_volume_gb': str(sum),
'snapshot_count': str(snap_count_total),
'total_snapshot_gb': str(snap_sum_total)}}]
project_ids = [v['project_id'] for v in volume_refs]
project_ids = list(set(project_ids))
for project_id in project_ids:
(count, sum) = db.volume_data_get_for_project(context, project_id)
(snap_count, snap_sum) = db.snapshot_data_get_for_project(
context,
project_id)
resources.append(
{'resource':
{'host': host,
'project': project_id,
'volume_count': str(count),
'total_volume_gb': str(sum),
'snapshot_count': str(snap_count),
'total_snapshot_gb': str(snap_sum)}})
snap_count_total += int(snap_count)
snap_sum_total += int(snap_sum)
resources[0]['resource']['snapshot_count'] = str(snap_count_total)
resources[0]['resource']['total_snapshot_gb'] = str(snap_sum_total)
return {"host": resources}
class Hosts(extensions.ExtensionDescriptor):
"""Admin-only host administration"""
name = "Hosts"
alias = "os-hosts"
namespace = "http://docs.openstack.org/volume/ext/hosts/api/v1.1"
updated = "2011-06-29T00:00:00+00:00"
def get_resources(self):
resources = [extensions.ResourceExtension('os-hosts',
HostController(),
collection_actions={
'update': 'PUT'},
member_actions={
'startup': 'GET',
'shutdown': 'GET',
'reboot': 'GET'})]
return resources

View File

@ -0,0 +1,31 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright (c) 2012 NTT.
# Copyright (c) 2012 OpenStack, LLC.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""The Create Volume from Image extension."""
from cinder.api import extensions
class Image_create(extensions.ExtensionDescriptor):
"""Allow creating a volume from an image in the Create Volume v1 API"""
name = "CreateVolumeExtension"
alias = "os-image-create"
namespace = "http://docs.openstack.org/volume/ext/image-create/api/v1"
updated = "2012-08-13T00:00:00+00:00"

View File

@ -0,0 +1,103 @@
# Copyright 2012 OpenStack LLC.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import webob
from cinder.api import extensions
from cinder.api.openstack import wsgi
from cinder.api import xmlutil
from cinder import db
from cinder import exception
from cinder import quota
QUOTAS = quota.QUOTAS
authorize = extensions.extension_authorizer('volume', 'quota_classes')
class QuotaClassTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('quota_class_set',
selector='quota_class_set')
root.set('id')
for resource in QUOTAS.resources:
elem = xmlutil.SubTemplateElement(root, resource)
elem.text = resource
return xmlutil.MasterTemplate(root, 1)
class QuotaClassSetsController(object):
def _format_quota_set(self, quota_class, quota_set):
"""Convert the quota object to a result dict"""
result = dict(id=str(quota_class))
for resource in QUOTAS.resources:
result[resource] = quota_set[resource]
return dict(quota_class_set=result)
@wsgi.serializers(xml=QuotaClassTemplate)
def show(self, req, id):
context = req.environ['cinder.context']
authorize(context)
try:
db.sqlalchemy.api.authorize_quota_class_context(context, id)
except exception.NotAuthorized:
raise webob.exc.HTTPForbidden()
return self._format_quota_set(id,
QUOTAS.get_class_quotas(context, id))
@wsgi.serializers(xml=QuotaClassTemplate)
def update(self, req, id, body):
context = req.environ['cinder.context']
authorize(context)
quota_class = id
for key in body['quota_class_set'].keys():
if key in QUOTAS:
value = int(body['quota_class_set'][key])
try:
db.quota_class_update(context, quota_class, key, value)
except exception.QuotaClassNotFound:
db.quota_class_create(context, quota_class, key, value)
except exception.AdminRequired:
raise webob.exc.HTTPForbidden()
return {'quota_class_set': QUOTAS.get_class_quotas(context,
quota_class)}
class Quota_classes(extensions.ExtensionDescriptor):
"""Quota classes management support"""
name = "QuotaClasses"
alias = "os-quota-class-sets"
namespace = ("http://docs.openstack.org/volume/ext/"
"quota-classes-sets/api/v1.1")
updated = "2012-03-12T00:00:00+00:00"
def get_resources(self):
resources = []
res = extensions.ResourceExtension('os-quota-class-sets',
QuotaClassSetsController())
resources.append(res)
return resources

View File

@ -0,0 +1,125 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2011 OpenStack LLC.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import webob
from cinder.api import extensions
from cinder.api.openstack import wsgi
from cinder.api import xmlutil
from cinder import db
from cinder.db.sqlalchemy import api as sqlalchemy_api
from cinder import exception
from cinder import quota
QUOTAS = quota.QUOTAS
authorize_update = extensions.extension_authorizer('compute', 'quotas:update')
authorize_show = extensions.extension_authorizer('compute', 'quotas:show')
class QuotaTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('quota_set', selector='quota_set')
root.set('id')
for resource in QUOTAS.resources:
elem = xmlutil.SubTemplateElement(root, resource)
elem.text = resource
return xmlutil.MasterTemplate(root, 1)
class QuotaSetsController(object):
def _format_quota_set(self, project_id, quota_set):
"""Convert the quota object to a result dict"""
result = dict(id=str(project_id))
for resource in QUOTAS.resources:
result[resource] = quota_set[resource]
return dict(quota_set=result)
def _validate_quota_limit(self, limit):
# NOTE: -1 is a flag value for unlimited
if limit < -1:
msg = _("Quota limit must be -1 or greater.")
raise webob.exc.HTTPBadRequest(explanation=msg)
def _get_quotas(self, context, id, usages=False):
values = QUOTAS.get_project_quotas(context, id, usages=usages)
if usages:
return values
else:
return dict((k, v['limit']) for k, v in values.items())
@wsgi.serializers(xml=QuotaTemplate)
def show(self, req, id):
context = req.environ['cinder.context']
authorize_show(context)
try:
sqlalchemy_api.authorize_project_context(context, id)
except exception.NotAuthorized:
raise webob.exc.HTTPForbidden()
return self._format_quota_set(id, self._get_quotas(context, id))
@wsgi.serializers(xml=QuotaTemplate)
def update(self, req, id, body):
context = req.environ['cinder.context']
authorize_update(context)
project_id = id
for key in body['quota_set'].keys():
if key in QUOTAS:
value = int(body['quota_set'][key])
self._validate_quota_limit(value)
try:
db.quota_update(context, project_id, key, value)
except exception.ProjectQuotaNotFound:
db.quota_create(context, project_id, key, value)
except exception.AdminRequired:
raise webob.exc.HTTPForbidden()
return {'quota_set': self._get_quotas(context, id)}
@wsgi.serializers(xml=QuotaTemplate)
def defaults(self, req, id):
context = req.environ['cinder.context']
authorize_show(context)
return self._format_quota_set(id, QUOTAS.get_defaults(context))
class Quotas(extensions.ExtensionDescriptor):
"""Quotas management support"""
name = "Quotas"
alias = "os-quota-sets"
namespace = "http://docs.openstack.org/compute/ext/quotas-sets/api/v1.1"
updated = "2011-08-08T00:00:00+00:00"
def get_resources(self):
resources = []
res = extensions.ResourceExtension('os-quota-sets',
QuotaSetsController(),
member_actions={'defaults': 'GET'})
resources.append(res)
return resources

View File

@ -0,0 +1,139 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2012 IBM Corp.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import webob.exc
from cinder.api import extensions
from cinder.api.openstack import wsgi
from cinder.api import xmlutil
from cinder import db
from cinder import exception
from cinder.openstack.common import log as logging
from cinder.openstack.common import timeutils
from cinder import utils
LOG = logging.getLogger(__name__)
authorize = extensions.extension_authorizer('volume', 'services')
class ServicesIndexTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('services')
elem = xmlutil.SubTemplateElement(root, 'service', selector='services')
elem.set('binary')
elem.set('host')
elem.set('zone')
elem.set('status')
elem.set('state')
elem.set('update_at')
return xmlutil.MasterTemplate(root, 1)
class ServicesUpdateTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('host')
root.set('host')
root.set('service')
root.set('disabled')
return xmlutil.MasterTemplate(root, 1)
class ServiceController(object):
@wsgi.serializers(xml=ServicesIndexTemplate)
def index(self, req):
"""
Return a list of all running services. Filter by host & service name.
"""
context = req.environ['cinder.context']
authorize(context)
now = timeutils.utcnow()
services = db.service_get_all(context)
host = ''
if 'host' in req.GET:
host = req.GET['host']
service = ''
if 'service' in req.GET:
service = req.GET['service']
if host:
services = [s for s in services if s['host'] == host]
if service:
services = [s for s in services if s['binary'] == service]
svcs = []
for svc in services:
delta = now - (svc['updated_at'] or svc['created_at'])
alive = abs(utils.total_seconds(delta))
art = (alive and "up") or "down"
active = 'enabled'
if svc['disabled']:
active = 'disabled'
svcs.append({"binary": svc['binary'], 'host': svc['host'],
'zone': svc['availability_zone'],
'status': active, 'state': art,
'updated_at': svc['updated_at']})
return {'services': svcs}
@wsgi.serializers(xml=ServicesUpdateTemplate)
def update(self, req, id, body):
"""Enable/Disable scheduling for a service"""
context = req.environ['cinder.context']
authorize(context)
if id == "enable":
disabled = False
elif id == "disable":
disabled = True
else:
raise webob.exc.HTTPNotFound("Unknown action")
try:
host = body['host']
service = body['service']
except (TypeError, KeyError):
raise webob.exc.HTTPBadRequest()
try:
svc = db.service_get_by_args(context, host, service)
if not svc:
raise webob.exc.HTTPNotFound('Unknown service')
db.service_update(context, svc['id'], {'disabled': disabled})
except exception.ServiceNotFound:
raise webob.exc.HTTPNotFound("service not found")
return {'host': host, 'service': service, 'disabled': disabled}
class Services(extensions.ExtensionDescriptor):
"""Services support"""
name = "Services"
alias = "os-services"
namespace = "http://docs.openstack.org/volume/ext/services/api/v2"
updated = "2012-10-28T00:00:00-00:00"
def get_resources(self):
resources = []
resource = extensions.ResourceExtension('os-services',
ServiceController())
resources.append(resource)
return resources

View File

@ -0,0 +1,80 @@
# Copyright 2013 NetApp.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import webob
from cinder.api import extensions
from cinder.api.openstack import wsgi
from cinder import exception
from cinder import share
class ShareActionsController(wsgi.Controller):
def __init__(self, *args, **kwargs):
super(ShareActionsController, self).__init__(*args, **kwargs)
self.share_api = share.API()
@wsgi.action('os-allow_access')
def _allow_access(self, req, id, body):
"""Add share access rule."""
context = req.environ['cinder.context']
share = self.share_api.get(context, id)
access_type = body['os-allow_access']['access_type']
access_to = body['os-allow_access']['access_to']
self.share_api.allow_access(context, share, access_type, access_to)
return webob.Response(status_int=202)
@wsgi.action('os-deny_access')
def _deny_access(self, req, id, body):
"""Remove access rule."""
context = req.environ['cinder.context']
access_id = body['os-deny_access']['access_id']
try:
access = self.share_api.access_get(context, access_id)
if access.share_id != id:
raise exception.NotFound()
share = self.share_api.get(context, id)
except exception.NotFound, error:
raise webob.exc.HTTPNotFound(explanation=unicode(error))
self.share_api.deny_access(context, share, access)
return webob.Response(status_int=202)
@wsgi.action('os-access_list')
def _access_list(self, req, id, body):
"""list access rules."""
context = req.environ['cinder.context']
share = self.share_api.get(context, id)
access_list = self.share_api.access_get_all(context, share)
return {'access_list': access_list}
class Share_actions(extensions.ExtensionDescriptor):
"""Enable share actions."""
name = 'ShareActions'
alias = 'share-actions'
namespace = ''
updated = '2012-08-14T00:00:00+00:00'
def get_controller_extensions(self):
controller = ShareActionsController()
extension = extensions.ControllerExtension(self, 'shares',
controller)
return [extension]

View File

@ -0,0 +1,181 @@
# Copyright 2013 NetApp
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""The share snapshots api."""
import webob
from webob import exc
from cinder.api import common
from cinder.api.contrib import shares
from cinder.api import extensions
from cinder.api.openstack import wsgi
from cinder.api.views import share_snapshots as snapshot_views
from cinder.api import xmlutil
from cinder import exception
from cinder.openstack.common import log as logging
from cinder import share
LOG = logging.getLogger(__name__)
def make_snapshot(elem):
elem.set('id')
elem.set('size')
elem.set('status')
elem.set('name')
elem.set('description')
elem.set('share_proto')
elem.set('export_location')
class SnapshotTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('share-snapshot',
selector='share-snapshot')
make_snapshot(root)
return xmlutil.MasterTemplate(root, 1)
class SnapshotsTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('share-snapshots')
elem = xmlutil.SubTemplateElement(root, 'share-snapshot',
selector='share-snapshots')
make_snapshot(elem)
return xmlutil.MasterTemplate(root, 1)
class ShareSnapshotsController(wsgi.Controller):
"""The Share Snapshots API controller for the OpenStack API."""
_view_builder_class = snapshot_views.ViewBuilder
def __init__(self):
super(ShareSnapshotsController, self).__init__()
self.share_api = share.API()
@wsgi.serializers(xml=SnapshotTemplate)
def show(self, req, id):
"""Return data about the given snapshot."""
context = req.environ['cinder.context']
try:
snapshot = self.share_api.get_snapshot(context, id)
except exception.NotFound:
raise exc.HTTPNotFound()
return self._view_builder.detail(req, snapshot)
def delete(self, req, id):
"""Delete a snapshot."""
context = req.environ['cinder.context']
LOG.audit(_("Delete snapshot with id: %s"), id, context=context)
try:
snapshot = self.share_api.get_snapshot(context, id)
self.share_api.delete_snapshot(context, snapshot)
except exception.NotFound:
raise exc.HTTPNotFound()
return webob.Response(status_int=202)
@wsgi.serializers(xml=SnapshotsTemplate)
def index(self, req):
"""Returns a summary list of snapshots."""
return self._get_snapshots(req, is_detail=False)
@wsgi.serializers(xml=SnapshotsTemplate)
def detail(self, req):
"""Returns a detailed list of snapshots."""
return self._get_snapshots(req, is_detail=True)
def _get_snapshots(self, req, is_detail):
"""Returns a list of snapshots."""
context = req.environ['cinder.context']
search_opts = {}
search_opts.update(req.GET)
# NOTE(rushiagr): v2 API allows name instead of display_name
if 'name' in search_opts:
search_opts['display_name'] = search_opts['name']
del search_opts['name']
shares.remove_invalid_options(context, search_opts,
self._get_snapshots_search_options())
snapshots = self.share_api.get_all_snapshots(context,
search_opts=search_opts)
limited_list = common.limited(snapshots, req)
if is_detail:
snapshots = self._view_builder.detail_list(req, limited_list)
else:
snapshots = self._view_builder.summary_list(req, limited_list)
return snapshots
def _get_snapshots_search_options(self):
"""Return share search options allowed by non-admin."""
return ('name', 'status', 'share_id')
@wsgi.response(202)
@wsgi.serializers(xml=SnapshotTemplate)
def create(self, req, body):
"""Creates a new snapshot."""
context = req.environ['cinder.context']
if not self.is_valid_body(body, 'share-snapshot'):
raise exc.HTTPUnprocessableEntity()
snapshot = body['share-snapshot']
share_id = snapshot['share_id']
share = self.share_api.get(context, share_id)
msg = _("Create snapshot from share %s")
LOG.audit(msg, share_id, context=context)
# NOTE(rushiagr): v2 API allows name instead of display_name
if 'name' in snapshot:
snapshot['display_name'] = snapshot.get('name')
del snapshot['name']
# NOTE(rushiagr): v2 API allows description instead of
# display_description
if 'description' in snapshot:
snapshot['display_description'] = snapshot.get('description')
del snapshot['description']
new_snapshot = self.share_api.create_snapshot(
context,
share,
snapshot.get('display_name'),
snapshot.get('display_description'))
return self._view_builder.summary(req, dict(new_snapshot.iteritems()))
class Share_snapshots(extensions.ExtensionDescriptor):
"""Enable share snapshtos API."""
name = 'ShareSnapshots'
alias = 'share-snapshots'
namespace = ''
updated = '2013-03-01T00:00:00+00:00'
def get_resources(self):
controller = ShareSnapshotsController()
resource = extensions.ResourceExtension(
'share-snapshots', controller,
collection_actions={'detail': 'GET'})
return [resource]

View File

@ -0,0 +1,215 @@
# Copyright 2013 NetApp
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""The shares api."""
import webob
from webob import exc
from cinder.api import common
from cinder.api import extensions
from cinder.api.openstack import wsgi
from cinder.api.views import shares as share_views
from cinder.api import xmlutil
from cinder import exception
from cinder.openstack.common import log as logging
from cinder import share
LOG = logging.getLogger(__name__)
def make_share(elem):
elem.set('id')
elem.set('size')
elem.set('availability_zone')
elem.set('status')
elem.set('name')
elem.set('description')
elem.set('share_proto')
elem.set('export_location')
def remove_invalid_options(context, search_options, allowed_search_options):
"""Remove search options that are not valid for non-admin API/context."""
if context.is_admin:
# Allow all options
return
# Otherwise, strip out all unknown options
unknown_options = [opt for opt in search_options
if opt not in allowed_search_options]
bad_options = ", ".join(unknown_options)
log_msg = _("Removing options '%(bad_options)s' from query") % locals()
LOG.debug(log_msg)
for opt in unknown_options:
del search_options[opt]
class ShareTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('share', selector='share')
make_share(root)
return xmlutil.MasterTemplate(root, 1)
class SharesTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('shares')
elem = xmlutil.SubTemplateElement(root, 'share', selector='shares')
make_share(elem)
return xmlutil.MasterTemplate(root, 1)
class ShareController(wsgi.Controller):
"""The Shares API controller for the OpenStack API."""
_view_builder_class = share_views.ViewBuilder
def __init__(self):
super(ShareController, self).__init__()
self.share_api = share.API()
@wsgi.serializers(xml=ShareTemplate)
def show(self, req, id):
"""Return data about the given share."""
context = req.environ['cinder.context']
try:
share = self.share_api.get(context, id)
except exception.NotFound:
raise exc.HTTPNotFound()
return self._view_builder.detail(req, share)
def delete(self, req, id):
"""Delete a share."""
context = req.environ['cinder.context']
LOG.audit(_("Delete share with id: %s"), id, context=context)
try:
share = self.share_api.get(context, id)
self.share_api.delete(context, share)
except exception.NotFound:
raise exc.HTTPNotFound()
return webob.Response(status_int=202)
@wsgi.serializers(xml=SharesTemplate)
def index(self, req):
"""Returns a summary list of shares."""
return self._get_shares(req, is_detail=False)
@wsgi.serializers(xml=SharesTemplate)
def detail(self, req):
"""Returns a detailed list of shares."""
return self._get_shares(req, is_detail=True)
def _get_shares(self, req, is_detail):
"""Returns a list of shares, transformed through view
builder.
"""
context = req.environ['cinder.context']
search_opts = {}
search_opts.update(req.GET)
# NOTE(rushiagr): v2 API allows name instead of display_name
if 'name' in search_opts:
search_opts['display_name'] = search_opts['name']
del search_opts['name']
remove_invalid_options(context, search_opts,
self._get_share_search_options())
shares = self.share_api.get_all(context, search_opts=search_opts)
limited_list = common.limited(shares, req)
if is_detail:
shares = self._view_builder.detail_list(req, limited_list)
else:
shares = self._view_builder.summary_list(req, limited_list)
return shares
def _get_share_search_options(self):
"""Return share search options allowed by non-admin."""
return ('name', 'status')
@wsgi.serializers(xml=ShareTemplate)
def create(self, req, body):
"""Creates a new share."""
context = req.environ['cinder.context']
if not self.is_valid_body(body, 'share'):
raise exc.HTTPUnprocessableEntity()
share = body['share']
# NOTE(rushiagr): v2 API allows name instead of display_name
if share.get('name'):
share['display_name'] = share.get('name')
del share['name']
# NOTE(rushiagr): v2 API allows description instead of
# display_description
if share.get('description'):
share['display_description'] = share.get('description')
del share['description']
size = share['size']
share_proto = share['share_proto'].upper()
msg = (_("Create %(share_proto)s share of %(size)s GB") %
{'share_proto': share_proto, 'size': size})
LOG.audit(msg, context=context)
kwargs = {}
kwargs['availability_zone'] = share.get('availability_zone')
snapshot_id = share.get('snapshot_id')
if snapshot_id is not None:
kwargs['snapshot'] = self.share_api.get_snapshot(context,
snapshot_id)
else:
kwargs['snapshot'] = None
display_name = share.get('display_name')
display_description = share.get('display_description')
new_share = self.share_api.create(context,
share_proto,
size,
display_name,
display_description,
**kwargs)
# TODO(vish): Instance should be None at db layer instead of
# trying to lazy load, but for now we turn it into
# a dict to avoid an error.
return self._view_builder.summary(req, dict(new_share.iteritems()))
class Shares(extensions.ExtensionDescriptor):
"""Enable share API."""
name = 'Shares'
alias = 'shares'
namespace = ''
updated = '2013-01-29T00:00:00+00:00'
def get_resources(self):
controller = ShareController()
resource = extensions.ResourceExtension(
'shares', controller, collection_actions={'detail': 'GET'},
member_actions={'action': 'POST'})
return [resource]

View File

@ -0,0 +1,162 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright (c) 2011 Zadara Storage Inc.
# Copyright (c) 2011 OpenStack LLC.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""The volume types extra specs extension"""
import webob
from cinder.api import extensions
from cinder.api.openstack import wsgi
from cinder.api import xmlutil
from cinder import db
from cinder import exception
from cinder.openstack.common.notifier import api as notifier_api
from cinder.volume import volume_types
authorize = extensions.extension_authorizer('volume', 'types_extra_specs')
class VolumeTypeExtraSpecsTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.make_flat_dict('extra_specs', selector='extra_specs')
return xmlutil.MasterTemplate(root, 1)
class VolumeTypeExtraSpecTemplate(xmlutil.TemplateBuilder):
def construct(self):
tagname = xmlutil.Selector('key')
def extraspec_sel(obj, do_raise=False):
# Have to extract the key and value for later use...
key, value = obj.items()[0]
return dict(key=key, value=value)
root = xmlutil.TemplateElement(tagname, selector=extraspec_sel)
root.text = 'value'
return xmlutil.MasterTemplate(root, 1)
class VolumeTypeExtraSpecsController(wsgi.Controller):
""" The volume type extra specs API controller for the OpenStack API """
def _get_extra_specs(self, context, type_id):
extra_specs = db.volume_type_extra_specs_get(context, type_id)
specs_dict = {}
for key, value in extra_specs.iteritems():
specs_dict[key] = value
return dict(extra_specs=specs_dict)
def _check_type(self, context, type_id):
try:
volume_types.get_volume_type(context, type_id)
except exception.NotFound as ex:
raise webob.exc.HTTPNotFound(explanation=unicode(ex))
@wsgi.serializers(xml=VolumeTypeExtraSpecsTemplate)
def index(self, req, type_id):
""" Returns the list of extra specs for a given volume type """
context = req.environ['cinder.context']
authorize(context)
self._check_type(context, type_id)
return self._get_extra_specs(context, type_id)
@wsgi.serializers(xml=VolumeTypeExtraSpecsTemplate)
def create(self, req, type_id, body=None):
context = req.environ['cinder.context']
authorize(context)
if not self.is_valid_body(body, 'extra_specs'):
raise webob.exc.HTTPBadRequest()
self._check_type(context, type_id)
specs = body['extra_specs']
db.volume_type_extra_specs_update_or_create(context,
type_id,
specs)
notifier_info = dict(type_id=type_id, specs=specs)
notifier_api.notify(context, 'volumeTypeExtraSpecs',
'volume_type_extra_specs.create',
notifier_api.INFO, notifier_info)
return body
@wsgi.serializers(xml=VolumeTypeExtraSpecTemplate)
def update(self, req, type_id, id, body=None):
context = req.environ['cinder.context']
authorize(context)
if not body:
expl = _('Request body empty')
raise webob.exc.HTTPBadRequest(explanation=expl)
self._check_type(context, type_id)
if id not in body:
expl = _('Request body and URI mismatch')
raise webob.exc.HTTPBadRequest(explanation=expl)
if len(body) > 1:
expl = _('Request body contains too many items')
raise webob.exc.HTTPBadRequest(explanation=expl)
db.volume_type_extra_specs_update_or_create(context,
type_id,
body)
notifier_info = dict(type_id=type_id, id=id)
notifier_api.notify(context, 'volumeTypeExtraSpecs',
'volume_type_extra_specs.update',
notifier_api.INFO, notifier_info)
return body
@wsgi.serializers(xml=VolumeTypeExtraSpecTemplate)
def show(self, req, type_id, id):
"""Return a single extra spec item."""
context = req.environ['cinder.context']
authorize(context)
self._check_type(context, type_id)
specs = self._get_extra_specs(context, type_id)
if id in specs['extra_specs']:
return {id: specs['extra_specs'][id]}
else:
raise webob.exc.HTTPNotFound()
def delete(self, req, type_id, id):
""" Deletes an existing extra spec """
context = req.environ['cinder.context']
self._check_type(context, type_id)
authorize(context)
db.volume_type_extra_specs_delete(context, type_id, id)
notifier_info = dict(type_id=type_id, id=id)
notifier_api.notify(context, 'volumeTypeExtraSpecs',
'volume_type_extra_specs.delete',
notifier_api.INFO, notifier_info)
return webob.Response(status_int=202)
class Types_extra_specs(extensions.ExtensionDescriptor):
"""Types extra specs support"""
name = "TypesExtraSpecs"
alias = "os-types-extra-specs"
namespace = "http://docs.openstack.org/volume/ext/types-extra-specs/api/v1"
updated = "2011-08-24T00:00:00+00:00"
def get_resources(self):
resources = []
res = extensions.ResourceExtension('extra_specs',
VolumeTypeExtraSpecsController(),
parent=dict(member_name='type',
collection_name='types')
)
resources.append(res)
return resources

View File

@ -0,0 +1,122 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright (c) 2011 Zadara Storage Inc.
# Copyright (c) 2011 OpenStack LLC.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""The volume types manage extension."""
import webob
from cinder.api import extensions
from cinder.api.openstack import wsgi
from cinder.api.v1 import types
from cinder.api.views import types as views_types
from cinder import exception
from cinder.openstack.common.notifier import api as notifier_api
from cinder.volume import volume_types
authorize = extensions.extension_authorizer('volume', 'types_manage')
class VolumeTypesManageController(wsgi.Controller):
"""The volume types API controller for the OpenStack API."""
_view_builder_class = views_types.ViewBuilder
def _notify_voloume_type_error(self, context, method, payload):
notifier_api.notify(context,
'volumeType',
method,
notifier_api.ERROR,
payload)
@wsgi.action("create")
@wsgi.serializers(xml=types.VolumeTypeTemplate)
def _create(self, req, body):
"""Creates a new volume type."""
context = req.environ['cinder.context']
authorize(context)
if not self.is_valid_body(body, 'volume_type'):
raise webob.exc.HTTPBadRequest()
vol_type = body['volume_type']
name = vol_type.get('name', None)
specs = vol_type.get('extra_specs', {})
if name is None or name == "":
raise webob.exc.HTTPBadRequest()
try:
volume_types.create(context, name, specs)
vol_type = volume_types.get_volume_type_by_name(context, name)
notifier_info = dict(volume_types=vol_type)
notifier_api.notify(context, 'volumeType',
'volume_type.create',
notifier_api.INFO, notifier_info)
except exception.VolumeTypeExists as err:
notifier_err = dict(volume_types=vol_type, error_message=str(err))
self._notify_voloume_type_error(context,
'volume_type.create',
notifier_err)
raise webob.exc.HTTPConflict(explanation=str(err))
except exception.NotFound as err:
notifier_err = dict(volume_types=vol_type, error_message=str(err))
self._notify_voloume_type_error(context,
'volume_type.create',
notifier_err)
raise webob.exc.HTTPNotFound()
return self._view_builder.show(req, vol_type)
@wsgi.action("delete")
def _delete(self, req, id):
"""Deletes an existing volume type."""
context = req.environ['cinder.context']
authorize(context)
try:
vol_type = volume_types.get_volume_type(context, id)
volume_types.destroy(context, vol_type['id'])
notifier_info = dict(volume_types=vol_type)
notifier_api.notify(context, 'volumeType',
'volume_type.delete',
notifier_api.INFO, notifier_info)
except exception.NotFound as err:
notifier_err = dict(id=id, error_message=str(err))
self._notify_voloume_type_error(context,
'volume_type.delete',
notifier_err)
raise webob.exc.HTTPNotFound()
return webob.Response(status_int=202)
class Types_manage(extensions.ExtensionDescriptor):
"""Types manage support."""
name = "TypesManage"
alias = "os-types-manage"
namespace = "http://docs.openstack.org/volume/ext/types-manage/api/v1"
updated = "2011-08-24T00:00:00+00:00"
def get_controller_extensions(self):
controller = VolumeTypesManageController()
extension = extensions.ControllerExtension(self, 'types', controller)
return [extension]

View File

@ -0,0 +1,204 @@
# Copyright 2012 OpenStack, LLC.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import webob
from cinder.api import extensions
from cinder.api.openstack import wsgi
from cinder.api import xmlutil
from cinder import exception
from cinder import flags
from cinder.openstack.common import log as logging
from cinder.openstack.common.rpc import common as rpc_common
from cinder import utils
from cinder import volume
FLAGS = flags.FLAGS
LOG = logging.getLogger(__name__)
def authorize(context, action_name):
action = 'volume_actions:%s' % action_name
extensions.extension_authorizer('volume', action)(context)
class VolumeToImageSerializer(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('os-volume_upload_image',
selector='os-volume_upload_image')
root.set('id')
root.set('updated_at')
root.set('status')
root.set('display_description')
root.set('size')
root.set('volume_type')
root.set('image_id')
root.set('container_format')
root.set('disk_format')
root.set('image_name')
return xmlutil.MasterTemplate(root, 1)
class VolumeToImageDeserializer(wsgi.XMLDeserializer):
"""Deserializer to handle xml-formatted requests."""
def default(self, string):
dom = utils.safe_minidom_parse_string(string)
action_node = dom.childNodes[0]
action_name = action_node.tagName
action_data = {}
attributes = ["force", "image_name", "container_format", "disk_format"]
for attr in attributes:
if action_node.hasAttribute(attr):
action_data[attr] = action_node.getAttribute(attr)
if 'force' in action_data and action_data['force'] == 'True':
action_data['force'] = True
return {'body': {action_name: action_data}}
class VolumeActionsController(wsgi.Controller):
def __init__(self, *args, **kwargs):
super(VolumeActionsController, self).__init__(*args, **kwargs)
self.volume_api = volume.API()
@wsgi.action('os-attach')
def _attach(self, req, id, body):
"""Add attachment metadata."""
context = req.environ['cinder.context']
volume = self.volume_api.get(context, id)
instance_uuid = body['os-attach']['instance_uuid']
mountpoint = body['os-attach']['mountpoint']
self.volume_api.attach(context, volume,
instance_uuid, mountpoint)
return webob.Response(status_int=202)
@wsgi.action('os-detach')
def _detach(self, req, id, body):
"""Clear attachment metadata."""
context = req.environ['cinder.context']
volume = self.volume_api.get(context, id)
self.volume_api.detach(context, volume)
return webob.Response(status_int=202)
@wsgi.action('os-reserve')
def _reserve(self, req, id, body):
"""Mark volume as reserved."""
context = req.environ['cinder.context']
volume = self.volume_api.get(context, id)
self.volume_api.reserve_volume(context, volume)
return webob.Response(status_int=202)
@wsgi.action('os-unreserve')
def _unreserve(self, req, id, body):
"""Unmark volume as reserved."""
context = req.environ['cinder.context']
volume = self.volume_api.get(context, id)
self.volume_api.unreserve_volume(context, volume)
return webob.Response(status_int=202)
@wsgi.action('os-begin_detaching')
def _begin_detaching(self, req, id, body):
"""Update volume status to 'detaching'."""
context = req.environ['cinder.context']
volume = self.volume_api.get(context, id)
self.volume_api.begin_detaching(context, volume)
return webob.Response(status_int=202)
@wsgi.action('os-roll_detaching')
def _roll_detaching(self, req, id, body):
"""Roll back volume status to 'in-use'."""
context = req.environ['cinder.context']
volume = self.volume_api.get(context, id)
self.volume_api.roll_detaching(context, volume)
return webob.Response(status_int=202)
@wsgi.action('os-initialize_connection')
def _initialize_connection(self, req, id, body):
"""Initialize volume attachment."""
context = req.environ['cinder.context']
volume = self.volume_api.get(context, id)
connector = body['os-initialize_connection']['connector']
info = self.volume_api.initialize_connection(context,
volume,
connector)
return {'connection_info': info}
@wsgi.action('os-terminate_connection')
def _terminate_connection(self, req, id, body):
"""Terminate volume attachment."""
context = req.environ['cinder.context']
volume = self.volume_api.get(context, id)
connector = body['os-terminate_connection']['connector']
self.volume_api.terminate_connection(context, volume, connector)
return webob.Response(status_int=202)
@wsgi.response(202)
@wsgi.action('os-volume_upload_image')
@wsgi.serializers(xml=VolumeToImageSerializer)
@wsgi.deserializers(xml=VolumeToImageDeserializer)
def _volume_upload_image(self, req, id, body):
"""Uploads the specified volume to image service."""
context = req.environ['cinder.context']
try:
params = body['os-volume_upload_image']
except (TypeError, KeyError):
msg = _("Invalid request body")
raise webob.exc.HTTPBadRequest(explanation=msg)
if not params.get("image_name"):
msg = _("No image_name was specified in request.")
raise webob.exc.HTTPBadRequest(explanation=msg)
force = params.get('force', False)
try:
volume = self.volume_api.get(context, id)
except exception.VolumeNotFound, error:
raise webob.exc.HTTPNotFound(explanation=unicode(error))
authorize(context, "upload_image")
image_metadata = {"container_format": params.get("container_format",
"bare"),
"disk_format": params.get("disk_format", "raw"),
"name": params["image_name"]}
try:
response = self.volume_api.copy_volume_to_image(context,
volume,
image_metadata,
force)
except exception.InvalidVolume, error:
raise webob.exc.HTTPBadRequest(explanation=unicode(error))
except ValueError, error:
raise webob.exc.HTTPBadRequest(explanation=unicode(error))
except rpc_common.RemoteError as error:
msg = "%(err_type)s: %(err_msg)s" % {'err_type': error.exc_type,
'err_msg': error.value}
raise webob.exc.HTTPBadRequest(explanation=msg)
return {'os-volume_upload_image': response}
class Volume_actions(extensions.ExtensionDescriptor):
"""Enable volume actions
"""
name = "VolumeActions"
alias = "os-volume-actions"
namespace = "http://docs.openstack.org/volume/ext/volume-actions/api/v1.1"
updated = "2012-05-31T00:00:00+00:00"
def get_controller_extensions(self):
controller = VolumeActionsController()
extension = extensions.ControllerExtension(self, 'volumes', controller)
return [extension]

View File

@ -0,0 +1,93 @@
# Copyright 2012 OpenStack, LLC.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from cinder.api import extensions
from cinder.api.openstack import wsgi
from cinder.api import xmlutil
from cinder.openstack.common import log as logging
from cinder import volume
LOG = logging.getLogger(__name__)
authorize = extensions.soft_extension_authorizer('volume',
'volume_host_attribute')
class VolumeHostAttributeController(wsgi.Controller):
def __init__(self, *args, **kwargs):
super(VolumeHostAttributeController, self).__init__(*args, **kwargs)
self.volume_api = volume.API()
def _add_volume_host_attribute(self, context, resp_volume):
try:
db_volume = self.volume_api.get(context, resp_volume['id'])
except Exception:
return
else:
key = "%s:host" % Volume_host_attribute.alias
resp_volume[key] = db_volume['host']
@wsgi.extends
def show(self, req, resp_obj, id):
context = req.environ['cinder.context']
if authorize(context):
resp_obj.attach(xml=VolumeHostAttributeTemplate())
self._add_volume_host_attribute(context, resp_obj.obj['volume'])
@wsgi.extends
def detail(self, req, resp_obj):
context = req.environ['cinder.context']
if authorize(context):
resp_obj.attach(xml=VolumeListHostAttributeTemplate())
for volume in list(resp_obj.obj['volumes']):
self._add_volume_host_attribute(context, volume)
class Volume_host_attribute(extensions.ExtensionDescriptor):
"""Expose host as an attribute of a volume."""
name = "VolumeHostAttribute"
alias = "os-vol-host-attr"
namespace = ("http://docs.openstack.org/volume/ext/"
"volume_host_attribute/api/v1")
updated = "2011-11-03T00:00:00+00:00"
def get_controller_extensions(self):
controller = VolumeHostAttributeController()
extension = extensions.ControllerExtension(self, 'volumes', controller)
return [extension]
def make_volume(elem):
elem.set('{%s}host' % Volume_host_attribute.namespace,
'%s:host' % Volume_host_attribute.alias)
class VolumeHostAttributeTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('volume', selector='volume')
make_volume(root)
alias = Volume_host_attribute.alias
namespace = Volume_host_attribute.namespace
return xmlutil.SlaveTemplate(root, 1, nsmap={alias: namespace})
class VolumeListHostAttributeTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('volumes')
elem = xmlutil.SubTemplateElement(root, 'volume', selector='volumes')
make_volume(elem)
alias = Volume_host_attribute.alias
namespace = Volume_host_attribute.namespace
return xmlutil.SlaveTemplate(root, 1, nsmap={alias: namespace})

View File

@ -0,0 +1,106 @@
# Copyright 2012 OpenStack, LLC.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""The Volume Image Metadata API extension."""
from cinder.api import extensions
from cinder.api.openstack import wsgi
from cinder.api import xmlutil
from cinder import volume
authorize = extensions.soft_extension_authorizer('volume',
'volume_image_metadata')
class VolumeImageMetadataController(wsgi.Controller):
def __init__(self, *args, **kwargs):
super(VolumeImageMetadataController, self).__init__(*args, **kwargs)
self.volume_api = volume.API()
def _add_image_metadata(self, context, resp_volume):
try:
image_meta = self.volume_api.get_volume_image_metadata(
context, resp_volume)
except Exception:
return
else:
if image_meta:
resp_volume['volume_image_metadata'] = dict(
image_meta.iteritems())
@wsgi.extends
def show(self, req, resp_obj, id):
context = req.environ['cinder.context']
if authorize(context):
resp_obj.attach(xml=VolumeImageMetadataTemplate())
self._add_image_metadata(context, resp_obj.obj['volume'])
@wsgi.extends
def detail(self, req, resp_obj):
context = req.environ['cinder.context']
if authorize(context):
resp_obj.attach(xml=VolumesImageMetadataTemplate())
for volume in list(resp_obj.obj.get('volumes', [])):
self._add_image_metadata(context, volume)
class Volume_image_metadata(extensions.ExtensionDescriptor):
"""Show image metadata associated with the volume"""
name = "VolumeImageMetadata"
alias = "os-vol-image-meta"
namespace = ("http://docs.openstack.org/volume/ext/"
"volume_image_metadata/api/v1")
updated = "2012-12-07T00:00:00+00:00"
def get_controller_extensions(self):
controller = VolumeImageMetadataController()
extension = extensions.ControllerExtension(self, 'volumes', controller)
return [extension]
class VolumeImageMetadataMetadataTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('volume_image_metadata',
selector='volume_image_metadata')
elem = xmlutil.SubTemplateElement(root, 'meta',
selector=xmlutil.get_items)
elem.set('key', 0)
elem.text = 1
return xmlutil.MasterTemplate(root, 1)
class VolumeImageMetadataTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('volume', selector='volume')
root.append(VolumeImageMetadataMetadataTemplate())
alias = Volume_image_metadata.alias
namespace = Volume_image_metadata.namespace
return xmlutil.SlaveTemplate(root, 1, nsmap={alias: namespace})
class VolumesImageMetadataTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('volumes')
elem = xmlutil.SubTemplateElement(root, 'volume', selector='volume')
elem.append(VolumeImageMetadataMetadataTemplate())
alias = Volume_image_metadata.alias
namespace = Volume_image_metadata.namespace
return xmlutil.SlaveTemplate(root, 1, nsmap={alias: namespace})

View File

@ -0,0 +1,91 @@
# Copyright 2012 OpenStack, LLC.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from cinder.api import extensions
from cinder.api.openstack import wsgi
from cinder.api import xmlutil
from cinder import volume
authorize = extensions.soft_extension_authorizer('volume',
'volume_tenant_attribute')
class VolumeTenantAttributeController(wsgi.Controller):
def __init__(self, *args, **kwargs):
super(VolumeTenantAttributeController, self).__init__(*args, **kwargs)
self.volume_api = volume.API()
def _add_volume_tenant_attribute(self, context, resp_volume):
try:
db_volume = self.volume_api.get(context, resp_volume['id'])
except Exception:
return
else:
key = "%s:tenant_id" % Volume_tenant_attribute.alias
resp_volume[key] = db_volume['project_id']
@wsgi.extends
def show(self, req, resp_obj, id):
context = req.environ['cinder.context']
if authorize(context):
resp_obj.attach(xml=VolumeTenantAttributeTemplate())
self._add_volume_tenant_attribute(context, resp_obj.obj['volume'])
@wsgi.extends
def detail(self, req, resp_obj):
context = req.environ['cinder.context']
if authorize(context):
resp_obj.attach(xml=VolumeListTenantAttributeTemplate())
for volume in list(resp_obj.obj['volumes']):
self._add_volume_tenant_attribute(context, volume)
class Volume_tenant_attribute(extensions.ExtensionDescriptor):
"""Expose the internal project_id as an attribute of a volume."""
name = "VolumeTenantAttribute"
alias = "os-vol-tenant-attr"
namespace = ("http://docs.openstack.org/volume/ext/"
"volume_tenant_attribute/api/v1")
updated = "2011-11-03T00:00:00+00:00"
def get_controller_extensions(self):
controller = VolumeTenantAttributeController()
extension = extensions.ControllerExtension(self, 'volumes', controller)
return [extension]
def make_volume(elem):
elem.set('{%s}tenant_id' % Volume_tenant_attribute.namespace,
'%s:tenant_id' % Volume_tenant_attribute.alias)
class VolumeTenantAttributeTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('volume', selector='volume')
make_volume(root)
alias = Volume_tenant_attribute.alias
namespace = Volume_tenant_attribute.namespace
return xmlutil.SlaveTemplate(root, 1, nsmap={alias: namespace})
class VolumeListTenantAttributeTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('volumes')
elem = xmlutil.SubTemplateElement(root, 'volume', selector='volumes')
make_volume(elem)
alias = Volume_tenant_attribute.alias
namespace = Volume_tenant_attribute.namespace
return xmlutil.SlaveTemplate(root, 1, nsmap={alias: namespace})

407
cinder/api/extensions.py Normal file
View File

@ -0,0 +1,407 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2011 OpenStack LLC.
# Copyright 2011 Justin Santa Barbara
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import os
import webob.dec
import webob.exc
import cinder.api.openstack
from cinder.api.openstack import wsgi
from cinder.api import xmlutil
from cinder import exception
from cinder import flags
from cinder.openstack.common import exception as common_exception
from cinder.openstack.common import importutils
from cinder.openstack.common import log as logging
import cinder.policy
LOG = logging.getLogger(__name__)
FLAGS = flags.FLAGS
class ExtensionDescriptor(object):
"""Base class that defines the contract for extensions.
Note that you don't have to derive from this class to have a valid
extension; it is purely a convenience.
"""
# The name of the extension, e.g., 'Fox In Socks'
name = None
# The alias for the extension, e.g., 'FOXNSOX'
alias = None
# Description comes from the docstring for the class
# The XML namespace for the extension, e.g.,
# 'http://www.fox.in.socks/api/ext/pie/v1.0'
namespace = None
# The timestamp when the extension was last updated, e.g.,
# '2011-01-22T13:25:27-06:00'
updated = None
def __init__(self, ext_mgr):
"""Register extension with the extension manager."""
ext_mgr.register(self)
def get_resources(self):
"""List of extensions.ResourceExtension extension objects.
Resources define new nouns, and are accessible through URLs.
"""
resources = []
return resources
def get_controller_extensions(self):
"""List of extensions.ControllerExtension extension objects.
Controller extensions are used to extend existing controllers.
"""
controller_exts = []
return controller_exts
@classmethod
def nsmap(cls):
"""Synthesize a namespace map from extension."""
# Start with a base nsmap
nsmap = ext_nsmap.copy()
# Add the namespace for the extension
nsmap[cls.alias] = cls.namespace
return nsmap
@classmethod
def xmlname(cls, name):
"""Synthesize element and attribute names."""
return '{%s}%s' % (cls.namespace, name)
def make_ext(elem):
elem.set('name')
elem.set('namespace')
elem.set('alias')
elem.set('updated')
desc = xmlutil.SubTemplateElement(elem, 'description')
desc.text = 'description'
xmlutil.make_links(elem, 'links')
ext_nsmap = {None: xmlutil.XMLNS_COMMON_V10, 'atom': xmlutil.XMLNS_ATOM}
class ExtensionTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('extension', selector='extension')
make_ext(root)
return xmlutil.MasterTemplate(root, 1, nsmap=ext_nsmap)
class ExtensionsTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('extensions')
elem = xmlutil.SubTemplateElement(root, 'extension',
selector='extensions')
make_ext(elem)
return xmlutil.MasterTemplate(root, 1, nsmap=ext_nsmap)
class ExtensionsResource(wsgi.Resource):
def __init__(self, extension_manager):
self.extension_manager = extension_manager
super(ExtensionsResource, self).__init__(None)
def _translate(self, ext):
ext_data = {}
ext_data['name'] = ext.name
ext_data['alias'] = ext.alias
ext_data['description'] = ext.__doc__
ext_data['namespace'] = ext.namespace
ext_data['updated'] = ext.updated
ext_data['links'] = [] # TODO(dprince): implement extension links
return ext_data
@wsgi.serializers(xml=ExtensionsTemplate)
def index(self, req):
extensions = []
for _alias, ext in self.extension_manager.extensions.iteritems():
extensions.append(self._translate(ext))
return dict(extensions=extensions)
@wsgi.serializers(xml=ExtensionTemplate)
def show(self, req, id):
try:
# NOTE(dprince): the extensions alias is used as the 'id' for show
ext = self.extension_manager.extensions[id]
except KeyError:
raise webob.exc.HTTPNotFound()
return dict(extension=self._translate(ext))
def delete(self, req, id):
raise webob.exc.HTTPNotFound()
def create(self, req):
raise webob.exc.HTTPNotFound()
class ExtensionManager(object):
"""Load extensions from the configured extension path.
See cinder/tests/api/extensions/foxinsocks/extension.py for an
example extension implementation.
"""
def __init__(self):
LOG.audit(_('Initializing extension manager.'))
self.cls_list = FLAGS.osapi_volume_extension
self.extensions = {}
self._load_extensions()
def is_loaded(self, alias):
return alias in self.extensions
def register(self, ext):
# Do nothing if the extension doesn't check out
if not self._check_extension(ext):
return
alias = ext.alias
LOG.audit(_('Loaded extension: %s'), alias)
if alias in self.extensions:
raise exception.Error("Found duplicate extension: %s" % alias)
self.extensions[alias] = ext
def get_resources(self):
"""Returns a list of ResourceExtension objects."""
resources = []
resources.append(ResourceExtension('extensions',
ExtensionsResource(self)))
for ext in self.extensions.values():
try:
resources.extend(ext.get_resources())
except AttributeError:
# NOTE(dprince): Extension aren't required to have resource
# extensions
pass
return resources
def get_controller_extensions(self):
"""Returns a list of ControllerExtension objects."""
controller_exts = []
for ext in self.extensions.values():
try:
get_ext_method = ext.get_controller_extensions
except AttributeError:
# NOTE(Vek): Extensions aren't required to have
# controller extensions
continue
controller_exts.extend(get_ext_method())
return controller_exts
def _check_extension(self, extension):
"""Checks for required methods in extension objects."""
try:
LOG.debug(_('Ext name: %s'), extension.name)
LOG.debug(_('Ext alias: %s'), extension.alias)
LOG.debug(_('Ext description: %s'),
' '.join(extension.__doc__.strip().split()))
LOG.debug(_('Ext namespace: %s'), extension.namespace)
LOG.debug(_('Ext updated: %s'), extension.updated)
except AttributeError as ex:
LOG.exception(_("Exception loading extension: %s"), unicode(ex))
return False
return True
def load_extension(self, ext_factory):
"""Execute an extension factory.
Loads an extension. The 'ext_factory' is the name of a
callable that will be imported and called with one
argument--the extension manager. The factory callable is
expected to call the register() method at least once.
"""
LOG.debug(_("Loading extension %s"), ext_factory)
# Load the factory
factory = importutils.import_class(ext_factory)
# Call it
LOG.debug(_("Calling extension factory %s"), ext_factory)
factory(self)
def _load_extensions(self):
"""Load extensions specified on the command line."""
extensions = list(self.cls_list)
# NOTE(thingee): Backwards compat for the old extension loader path.
# We can drop this post-grizzly in the H release.
old_contrib_path = ('cinder.api.openstack.volume.contrib.'
'standard_extensions')
new_contrib_path = 'cinder.api.contrib.standard_extensions'
if old_contrib_path in extensions:
LOG.warn(_('osapi_volume_extension is set to deprecated path: %s'),
old_contrib_path)
LOG.warn(_('Please set your flag or cinder.conf settings for '
'osapi_volume_extension to: %s'), new_contrib_path)
extensions = [e.replace(old_contrib_path, new_contrib_path)
for e in extensions]
for ext_factory in extensions:
try:
self.load_extension(ext_factory)
except Exception as exc:
LOG.warn(_('Failed to load extension %(ext_factory)s: '
'%(exc)s') % locals())
class ControllerExtension(object):
"""Extend core controllers of cinder OpenStack API.
Provide a way to extend existing cinder OpenStack API core
controllers.
"""
def __init__(self, extension, collection, controller):
self.extension = extension
self.collection = collection
self.controller = controller
class ResourceExtension(object):
"""Add top level resources to the OpenStack API in cinder."""
def __init__(self, collection, controller, parent=None,
collection_actions=None, member_actions=None,
custom_routes_fn=None):
if not collection_actions:
collection_actions = {}
if not member_actions:
member_actions = {}
self.collection = collection
self.controller = controller
self.parent = parent
self.collection_actions = collection_actions
self.member_actions = member_actions
self.custom_routes_fn = custom_routes_fn
def load_standard_extensions(ext_mgr, logger, path, package, ext_list=None):
"""Registers all standard API extensions."""
# Walk through all the modules in our directory...
our_dir = path[0]
for dirpath, dirnames, filenames in os.walk(our_dir):
# Compute the relative package name from the dirpath
relpath = os.path.relpath(dirpath, our_dir)
if relpath == '.':
relpkg = ''
else:
relpkg = '.%s' % '.'.join(relpath.split(os.sep))
# Now, consider each file in turn, only considering .py files
for fname in filenames:
root, ext = os.path.splitext(fname)
# Skip __init__ and anything that's not .py
if ext != '.py' or root == '__init__':
continue
# Try loading it
classname = "%s%s" % (root[0].upper(), root[1:])
classpath = ("%s%s.%s.%s" %
(package, relpkg, root, classname))
if ext_list is not None and classname not in ext_list:
logger.debug("Skipping extension: %s" % classpath)
continue
try:
ext_mgr.load_extension(classpath)
except Exception as exc:
logger.warn(_('Failed to load extension %(classpath)s: '
'%(exc)s') % locals())
# Now, let's consider any subdirectories we may have...
subdirs = []
for dname in dirnames:
# Skip it if it does not have __init__.py
if not os.path.exists(os.path.join(dirpath, dname,
'__init__.py')):
continue
# If it has extension(), delegate...
ext_name = ("%s%s.%s.extension" %
(package, relpkg, dname))
try:
ext = importutils.import_class(ext_name)
except common_exception.NotFound:
# extension() doesn't exist on it, so we'll explore
# the directory for ourselves
subdirs.append(dname)
else:
try:
ext(ext_mgr)
except Exception as exc:
logger.warn(_('Failed to load extension %(ext_name)s: '
'%(exc)s') % locals())
# Update the list of directories we'll explore...
dirnames[:] = subdirs
def extension_authorizer(api_name, extension_name):
def authorize(context, target=None):
if target is None:
target = {'project_id': context.project_id,
'user_id': context.user_id}
action = '%s_extension:%s' % (api_name, extension_name)
cinder.policy.enforce(context, action, target)
return authorize
def soft_extension_authorizer(api_name, extension_name):
hard_authorize = extension_authorizer(api_name, extension_name)
def authorize(context):
try:
hard_authorize(context)
return True
except exception.NotAuthorized:
return False
return authorize

View File

@ -0,0 +1,16 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2011 OpenStack LLC.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.

View File

@ -0,0 +1,140 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2010 OpenStack LLC.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""
Common Auth Middleware.
"""
import os
from oslo.config import cfg
import webob.dec
import webob.exc
from cinder.api.openstack import wsgi
from cinder import context
from cinder import flags
from cinder.openstack.common import log as logging
from cinder import wsgi as base_wsgi
use_forwarded_for_opt = cfg.BoolOpt(
'use_forwarded_for',
default=False,
help='Treat X-Forwarded-For as the canonical remote address. '
'Only enable this if you have a sanitizing proxy.')
FLAGS = flags.FLAGS
FLAGS.register_opt(use_forwarded_for_opt)
LOG = logging.getLogger(__name__)
def pipeline_factory(loader, global_conf, **local_conf):
"""A paste pipeline replica that keys off of auth_strategy."""
pipeline = local_conf[FLAGS.auth_strategy]
if not FLAGS.api_rate_limit:
limit_name = FLAGS.auth_strategy + '_nolimit'
pipeline = local_conf.get(limit_name, pipeline)
pipeline = pipeline.split()
filters = [loader.get_filter(n) for n in pipeline[:-1]]
app = loader.get_app(pipeline[-1])
filters.reverse()
for filter in filters:
app = filter(app)
return app
class InjectContext(base_wsgi.Middleware):
"""Add a 'cinder.context' to WSGI environ."""
def __init__(self, context, *args, **kwargs):
self.context = context
super(InjectContext, self).__init__(*args, **kwargs)
@webob.dec.wsgify(RequestClass=base_wsgi.Request)
def __call__(self, req):
req.environ['cinder.context'] = self.context
return self.application
class CinderKeystoneContext(base_wsgi.Middleware):
"""Make a request context from keystone headers"""
@webob.dec.wsgify(RequestClass=base_wsgi.Request)
def __call__(self, req):
user_id = req.headers.get('X_USER')
user_id = req.headers.get('X_USER_ID', user_id)
if user_id is None:
LOG.debug("Neither X_USER_ID nor X_USER found in request")
return webob.exc.HTTPUnauthorized()
# get the roles
roles = [r.strip() for r in req.headers.get('X_ROLE', '').split(',')]
if 'X_TENANT_ID' in req.headers:
# This is the new header since Keystone went to ID/Name
project_id = req.headers['X_TENANT_ID']
else:
# This is for legacy compatibility
project_id = req.headers['X_TENANT']
# Get the auth token
auth_token = req.headers.get('X_AUTH_TOKEN',
req.headers.get('X_STORAGE_TOKEN'))
# Build a context, including the auth_token...
remote_address = req.remote_addr
if FLAGS.use_forwarded_for:
remote_address = req.headers.get('X-Forwarded-For', remote_address)
ctx = context.RequestContext(user_id,
project_id,
roles=roles,
auth_token=auth_token,
remote_address=remote_address)
req.environ['cinder.context'] = ctx
return self.application
class NoAuthMiddleware(base_wsgi.Middleware):
"""Return a fake token if one isn't specified."""
@webob.dec.wsgify(RequestClass=wsgi.Request)
def __call__(self, req):
if 'X-Auth-Token' not in req.headers:
user_id = req.headers.get('X-Auth-User', 'admin')
project_id = req.headers.get('X-Auth-Project-Id', 'admin')
os_url = os.path.join(req.url, project_id)
res = webob.Response()
# NOTE(vish): This is expecting and returning Auth(1.1), whereas
# keystone uses 2.0 auth. We should probably allow
# 2.0 auth here as well.
res.headers['X-Auth-Token'] = '%s:%s' % (user_id, project_id)
res.headers['X-Server-Management-Url'] = os_url
res.content_type = 'text/plain'
res.status = '204'
return res
token = req.headers['X-Auth-Token']
user_id, _sep, project_id = token.partition(':')
project_id = project_id or user_id
remote_address = getattr(req, 'remote_address', '127.0.0.1')
if FLAGS.use_forwarded_for:
remote_address = req.headers.get('X-Forwarded-For', remote_address)
ctx = context.RequestContext(user_id,
project_id,
is_admin=True,
remote_address=remote_address)
req.environ['cinder.context'] = ctx
return self.application

View File

@ -0,0 +1,75 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2010 United States Government as represented by the
# Administrator of the National Aeronautics and Space Administration.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import webob.dec
import webob.exc
from cinder.api.openstack import wsgi
from cinder.openstack.common import log as logging
from cinder import utils
from cinder import wsgi as base_wsgi
LOG = logging.getLogger(__name__)
class FaultWrapper(base_wsgi.Middleware):
"""Calls down the middleware stack, making exceptions into faults."""
_status_to_type = {}
@staticmethod
def status_to_type(status):
if not FaultWrapper._status_to_type:
for clazz in utils.walk_class_hierarchy(webob.exc.HTTPError):
FaultWrapper._status_to_type[clazz.code] = clazz
return FaultWrapper._status_to_type.get(
status, webob.exc.HTTPInternalServerError)()
def _error(self, inner, req):
LOG.exception(_("Caught error: %s"), unicode(inner))
safe = getattr(inner, 'safe', False)
headers = getattr(inner, 'headers', None)
status = getattr(inner, 'code', 500)
if status is None:
status = 500
msg_dict = dict(url=req.url, status=status)
LOG.info(_("%(url)s returned with HTTP %(status)d") % msg_dict)
outer = self.status_to_type(status)
if headers:
outer.headers = headers
# NOTE(johannes): We leave the explanation empty here on
# purpose. It could possibly have sensitive information
# that should not be returned back to the user. See
# bugs 868360 and 874472
# NOTE(eglynn): However, it would be over-conservative and
# inconsistent with the EC2 API to hide every exception,
# including those that are safe to expose, see bug 1021373
if safe:
outer.explanation = '%s: %s' % (inner.__class__.__name__,
unicode(inner))
return wsgi.Fault(outer)
@webob.dec.wsgify(RequestClass=wsgi.Request)
def __call__(self, req):
try:
return req.get_response(self.application)
except Exception as ex:
return self._error(ex, req)

View File

@ -0,0 +1,83 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright (c) 2012 OpenStack, LLC
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""
Request Body limiting middleware.
"""
from oslo.config import cfg
import webob.dec
import webob.exc
from cinder import flags
from cinder.openstack.common import log as logging
from cinder import wsgi
#default request size is 112k
max_request_body_size_opt = cfg.IntOpt('osapi_max_request_body_size',
default=114688,
help='Max size for body of a request')
FLAGS = flags.FLAGS
FLAGS.register_opt(max_request_body_size_opt)
LOG = logging.getLogger(__name__)
class LimitingReader(object):
"""Reader to limit the size of an incoming request."""
def __init__(self, data, limit):
"""
:param data: Underlying data object
:param limit: maximum number of bytes the reader should allow
"""
self.data = data
self.limit = limit
self.bytes_read = 0
def __iter__(self):
for chunk in self.data:
self.bytes_read += len(chunk)
if self.bytes_read > self.limit:
msg = _("Request is too large.")
raise webob.exc.HTTPRequestEntityTooLarge(explanation=msg)
else:
yield chunk
def read(self, i=None):
result = self.data.read(i)
self.bytes_read += len(result)
if self.bytes_read > self.limit:
msg = _("Request is too large.")
raise webob.exc.HTTPRequestEntityTooLarge(explanation=msg)
return result
class RequestBodySizeLimiter(wsgi.Middleware):
"""Add a 'cinder.context' to WSGI environ."""
def __init__(self, *args, **kwargs):
super(RequestBodySizeLimiter, self).__init__(*args, **kwargs)
@webob.dec.wsgify(RequestClass=wsgi.Request)
def __call__(self, req):
if req.content_length > FLAGS.osapi_max_request_body_size:
msg = _("Request is too large.")
raise webob.exc.HTTPRequestEntityTooLarge(explanation=msg)
if req.content_length is None and req.is_body_readable:
limiter = LimitingReader(req.body_file,
FLAGS.osapi_max_request_body_size)
req.body_file = limiter
return self.application

View File

@ -0,0 +1,130 @@
# Copyright (c) 2013 OpenStack, LLC.
#
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""
WSGI middleware for OpenStack API controllers.
"""
import routes
from cinder.api.middleware import fault
from cinder.api.openstack import wsgi
from cinder.openstack.common import log as logging
from cinder import utils
from cinder import wsgi as base_wsgi
LOG = logging.getLogger(__name__)
class APIMapper(routes.Mapper):
def routematch(self, url=None, environ=None):
if url is "":
result = self._match("", environ)
return result[0], result[1]
return routes.Mapper.routematch(self, url, environ)
class ProjectMapper(APIMapper):
def resource(self, member_name, collection_name, **kwargs):
if 'parent_resource' not in kwargs:
kwargs['path_prefix'] = '{project_id}/'
else:
parent_resource = kwargs['parent_resource']
p_collection = parent_resource['collection_name']
p_member = parent_resource['member_name']
kwargs['path_prefix'] = '{project_id}/%s/:%s_id' % (p_collection,
p_member)
routes.Mapper.resource(self,
member_name,
collection_name,
**kwargs)
class APIRouter(base_wsgi.Router):
"""
Routes requests on the OpenStack API to the appropriate controller
and method.
"""
ExtensionManager = None # override in subclasses
@classmethod
def factory(cls, global_config, **local_config):
"""Simple paste factory, :class:`cinder.wsgi.Router` doesn't have"""
return cls()
def __init__(self, ext_mgr=None):
if ext_mgr is None:
if self.ExtensionManager:
ext_mgr = self.ExtensionManager()
else:
raise Exception(_("Must specify an ExtensionManager class"))
mapper = ProjectMapper()
self.resources = {}
self._setup_routes(mapper, ext_mgr)
self._setup_ext_routes(mapper, ext_mgr)
self._setup_extensions(ext_mgr)
super(APIRouter, self).__init__(mapper)
def _setup_ext_routes(self, mapper, ext_mgr):
for resource in ext_mgr.get_resources():
LOG.debug(_('Extended resource: %s'),
resource.collection)
wsgi_resource = wsgi.Resource(resource.controller)
self.resources[resource.collection] = wsgi_resource
kargs = dict(
controller=wsgi_resource,
collection=resource.collection_actions,
member=resource.member_actions)
if resource.parent:
kargs['parent_resource'] = resource.parent
mapper.resource(resource.collection, resource.collection, **kargs)
if resource.custom_routes_fn:
resource.custom_routes_fn(mapper, wsgi_resource)
def _setup_extensions(self, ext_mgr):
for extension in ext_mgr.get_controller_extensions():
ext_name = extension.extension.name
collection = extension.collection
controller = extension.controller
if collection not in self.resources:
LOG.warning(_('Extension %(ext_name)s: Cannot extend '
'resource %(collection)s: No such resource') %
locals())
continue
LOG.debug(_('Extension %(ext_name)s extending resource: '
'%(collection)s') % locals())
resource = self.resources[collection]
resource.register_actions(controller)
resource.register_extensions(controller)
def _setup_routes(self, mapper, ext_mgr):
raise NotImplementedError
class FaultWrapper(fault.FaultWrapper):
def __init__(self, application):
LOG.warn(_('cinder.api.openstack:FaultWrapper is deprecated. Please '
'use cinder.api.middleware.fault:FaultWrapper instead.'))
super(FaultWrapper, self).__init__(application)

View File

@ -0,0 +1,27 @@
# Copyright (c) 2013 OpenStack, LLC.
#
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from cinder.api import urlmap
from cinder.openstack.common import log as logging
LOG = logging.getLogger(__name__)
def urlmap_factory(loader, global_conf, **local_conf):
LOG.warn(_('cinder.api.openstack.urlmap:urlmap_factory is deprecated. '
'Please use cinder.api.urlmap:urlmap_factory instead.'))
urlmap.urlmap_factory(loader, global_conf, **local_conf)

View File

@ -0,0 +1,27 @@
# Copyright (c) 2013 OpenStack, LLC.
#
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from cinder.api.v1.router import APIRouter as v1_router
from cinder.openstack.common import log as logging
LOG = logging.getLogger(__name__)
class APIRouter(v1_router):
def __init__(self, ext_mgr=None):
LOG.warn(_('cinder.api.openstack.volume:APIRouter is deprecated. '
'Please use cinder.api.v1.router:APIRouter instead.'))
super(APIRouter, self).__init__(ext_mgr)

View File

@ -0,0 +1,29 @@
# Copyright (c) 2013 OpenStack, LLC.
#
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from cinder.api import versions
from cinder.openstack.common import log as logging
LOG = logging.getLogger(__name__)
class Versions(versions.Versions):
def __init__(self):
LOG.warn(_('cinder.api.openstack.volume.versions.Versions is '
'deprecated. Please use cinder.api.versions.Versions '
'instead.'))
super(Versions, self).__init__()

1144
cinder/api/openstack/wsgi.py Normal file

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,141 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
-*- rnc -*-
RELAX NG Compact Syntax Grammar for the
Atom Format Specification Version 11
-->
<grammar xmlns:xhtml="http://www.w3.org/1999/xhtml" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:s="http://www.ascc.net/xml/schematron" xmlns="http://relaxng.org/ns/structure/1.0" datatypeLibrary="http://www.w3.org/2001/XMLSchema-datatypes">
<start>
<choice>
<ref name="atomLink"/>
</choice>
</start>
<!-- Common attributes -->
<define name="atomCommonAttributes">
<optional>
<attribute name="xml:base">
<ref name="atomUri"/>
</attribute>
</optional>
<optional>
<attribute name="xml:lang">
<ref name="atomLanguageTag"/>
</attribute>
</optional>
<zeroOrMore>
<ref name="undefinedAttribute"/>
</zeroOrMore>
</define>
<!-- atom:link -->
<define name="atomLink">
<element name="atom:link">
<ref name="atomCommonAttributes"/>
<attribute name="href">
<ref name="atomUri"/>
</attribute>
<optional>
<attribute name="rel">
<choice>
<ref name="atomNCName"/>
<ref name="atomUri"/>
</choice>
</attribute>
</optional>
<optional>
<attribute name="type">
<ref name="atomMediaType"/>
</attribute>
</optional>
<optional>
<attribute name="hreflang">
<ref name="atomLanguageTag"/>
</attribute>
</optional>
<optional>
<attribute name="title"/>
</optional>
<optional>
<attribute name="length"/>
</optional>
<ref name="undefinedContent"/>
</element>
</define>
<!-- Low-level simple types -->
<define name="atomNCName">
<data type="string">
<param name="minLength">1</param>
<param name="pattern">[^:]*</param>
</data>
</define>
<!-- Whatever a media type is, it contains at least one slash -->
<define name="atomMediaType">
<data type="string">
<param name="pattern">.+/.+</param>
</data>
</define>
<!-- As defined in RFC 3066 -->
<define name="atomLanguageTag">
<data type="string">
<param name="pattern">[A-Za-z]{1,8}(-[A-Za-z0-9]{1,8})*</param>
</data>
</define>
<!--
Unconstrained; it's not entirely clear how IRI fit into
xsd:anyURI so let's not try to constrain it here
-->
<define name="atomUri">
<text/>
</define>
<!-- Other Extensibility -->
<define name="undefinedAttribute">
<attribute>
<anyName>
<except>
<name>xml:base</name>
<name>xml:lang</name>
<nsName ns=""/>
</except>
</anyName>
</attribute>
</define>
<define name="undefinedContent">
<zeroOrMore>
<choice>
<text/>
<ref name="anyForeignElement"/>
</choice>
</zeroOrMore>
</define>
<define name="anyElement">
<element>
<anyName/>
<zeroOrMore>
<choice>
<attribute>
<anyName/>
</attribute>
<text/>
<ref name="anyElement"/>
</choice>
</zeroOrMore>
</element>
</define>
<define name="anyForeignElement">
<element>
<anyName>
<except>
<nsName ns="http://www.w3.org/2005/Atom"/>
</except>
</anyName>
<zeroOrMore>
<choice>
<attribute>
<anyName/>
</attribute>
<text/>
<ref name="anyElement"/>
</choice>
</zeroOrMore>
</element>
</define>
</grammar>

View File

@ -0,0 +1,11 @@
<element name="extension" ns="http://docs.openstack.org/common/api/v1.0"
xmlns="http://relaxng.org/ns/structure/1.0">
<attribute name="alias"> <text/> </attribute>
<attribute name="name"> <text/> </attribute>
<attribute name="namespace"> <text/> </attribute>
<attribute name="updated"> <text/> </attribute>
<element name="description"> <text/> </element>
<zeroOrMore>
<externalRef href="../atom-link.rng"/>
</zeroOrMore>
</element>

View File

@ -0,0 +1,6 @@
<element name="extensions" xmlns="http://relaxng.org/ns/structure/1.0"
ns="http://docs.openstack.org/common/api/v1.0">
<zeroOrMore>
<externalRef href="extension.rng"/>
</zeroOrMore>
</element>

View File

@ -0,0 +1,28 @@
<element name="limits" ns="http://docs.openstack.org/common/api/v1.0"
xmlns="http://relaxng.org/ns/structure/1.0">
<element name="rates">
<zeroOrMore>
<element name="rate">
<attribute name="uri"> <text/> </attribute>
<attribute name="regex"> <text/> </attribute>
<zeroOrMore>
<element name="limit">
<attribute name="value"> <text/> </attribute>
<attribute name="verb"> <text/> </attribute>
<attribute name="remaining"> <text/> </attribute>
<attribute name="unit"> <text/> </attribute>
<attribute name="next-available"> <text/> </attribute>
</element>
</zeroOrMore>
</element>
</zeroOrMore>
</element>
<element name="absolute">
<zeroOrMore>
<element name="limit">
<attribute name="name"> <text/> </attribute>
<attribute name="value"> <text/> </attribute>
</element>
</zeroOrMore>
</element>
</element>

View File

@ -0,0 +1,9 @@
<element name="metadata" ns="http://docs.openstack.org/compute/api/v1.1"
xmlns="http://relaxng.org/ns/structure/1.0">
<zeroOrMore>
<element name="meta">
<attribute name="key"> <text/> </attribute>
<text/>
</element>
</zeroOrMore>
</element>

28
cinder/api/sizelimit.py Normal file
View File

@ -0,0 +1,28 @@
# Copyright (c) 2013 OpenStack, LLC.
#
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from cinder.api.middleware import sizelimit
from cinder.openstack.common import log as logging
LOG = logging.getLogger(__name__)
class RequestBodySizeLimiter(sizelimit.RequestBodySizeLimiter):
def __init__(self, *args, **kwargs):
LOG.warn(_('cinder.api.sizelimit:RequestBodySizeLimiter is '
'deprecated. Please use cinder.api.middleware.sizelimit:'
'RequestBodySizeLimiter instead'))
super(RequestBodySizeLimiter, self).__init__(*args, **kwargs)

297
cinder/api/urlmap.py Normal file
View File

@ -0,0 +1,297 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2011 OpenStack LLC.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import paste.urlmap
import re
import urllib2
from cinder.api.openstack import wsgi
from cinder.openstack.common import log as logging
_quoted_string_re = r'"[^"\\]*(?:\\.[^"\\]*)*"'
_option_header_piece_re = re.compile(
r';\s*([^\s;=]+|%s)\s*'
r'(?:=\s*([^;]+|%s))?\s*' %
(_quoted_string_re, _quoted_string_re))
LOG = logging.getLogger(__name__)
def unquote_header_value(value):
"""Unquotes a header value.
This does not use the real unquoting but what browsers are actually
using for quoting.
:param value: the header value to unquote.
"""
if value and value[0] == value[-1] == '"':
# this is not the real unquoting, but fixing this so that the
# RFC is met will result in bugs with internet explorer and
# probably some other browsers as well. IE for example is
# uploading files with "C:\foo\bar.txt" as filename
value = value[1:-1]
return value
def parse_list_header(value):
"""Parse lists as described by RFC 2068 Section 2.
In particular, parse comma-separated lists where the elements of
the list may include quoted-strings. A quoted-string could
contain a comma. A non-quoted string could have quotes in the
middle. Quotes are removed automatically after parsing.
The return value is a standard :class:`list`:
>>> parse_list_header('token, "quoted value"')
['token', 'quoted value']
:param value: a string with a list header.
:return: :class:`list`
"""
result = []
for item in urllib2.parse_http_list(value):
if item[:1] == item[-1:] == '"':
item = unquote_header_value(item[1:-1])
result.append(item)
return result
def parse_options_header(value):
"""Parse a ``Content-Type`` like header into a tuple with the content
type and the options:
>>> parse_options_header('Content-Type: text/html; mimetype=text/html')
('Content-Type:', {'mimetype': 'text/html'})
:param value: the header to parse.
:return: (str, options)
"""
def _tokenize(string):
for match in _option_header_piece_re.finditer(string):
key, value = match.groups()
key = unquote_header_value(key)
if value is not None:
value = unquote_header_value(value)
yield key, value
if not value:
return '', {}
parts = _tokenize(';' + value)
name = parts.next()[0]
extra = dict(parts)
return name, extra
class Accept(object):
def __init__(self, value):
self._content_types = [parse_options_header(v) for v in
parse_list_header(value)]
def best_match(self, supported_content_types):
# FIXME: Should we have a more sophisticated matching algorithm that
# takes into account the version as well?
best_quality = -1
best_content_type = None
best_params = {}
best_match = '*/*'
for content_type in supported_content_types:
for content_mask, params in self._content_types:
try:
quality = float(params.get('q', 1))
except ValueError:
continue
if quality < best_quality:
continue
elif best_quality == quality:
if best_match.count('*') <= content_mask.count('*'):
continue
if self._match_mask(content_mask, content_type):
best_quality = quality
best_content_type = content_type
best_params = params
best_match = content_mask
return best_content_type, best_params
def content_type_params(self, best_content_type):
"""Find parameters in Accept header for given content type."""
for content_type, params in self._content_types:
if best_content_type == content_type:
return params
return {}
def _match_mask(self, mask, content_type):
if '*' not in mask:
return content_type == mask
if mask == '*/*':
return True
mask_major = mask[:-2]
content_type_major = content_type.split('/', 1)[0]
return content_type_major == mask_major
def urlmap_factory(loader, global_conf, **local_conf):
if 'not_found_app' in local_conf:
not_found_app = local_conf.pop('not_found_app')
else:
not_found_app = global_conf.get('not_found_app')
if not_found_app:
not_found_app = loader.get_app(not_found_app, global_conf=global_conf)
urlmap = URLMap(not_found_app=not_found_app)
for path, app_name in local_conf.items():
path = paste.urlmap.parse_path_expression(path)
app = loader.get_app(app_name, global_conf=global_conf)
urlmap[path] = app
return urlmap
class URLMap(paste.urlmap.URLMap):
def _match(self, host, port, path_info):
"""Find longest match for a given URL path."""
for (domain, app_url), app in self.applications:
if domain and domain != host and domain != host + ':' + port:
continue
if (path_info == app_url or path_info.startswith(app_url + '/')):
return app, app_url
return None, None
def _set_script_name(self, app, app_url):
def wrap(environ, start_response):
environ['SCRIPT_NAME'] += app_url
return app(environ, start_response)
return wrap
def _munge_path(self, app, path_info, app_url):
def wrap(environ, start_response):
environ['SCRIPT_NAME'] += app_url
environ['PATH_INFO'] = path_info[len(app_url):]
return app(environ, start_response)
return wrap
def _path_strategy(self, host, port, path_info):
"""Check path suffix for MIME type and path prefix for API version."""
mime_type = app = app_url = None
parts = path_info.rsplit('.', 1)
if len(parts) > 1:
possible_type = 'application/' + parts[1]
if possible_type in wsgi.SUPPORTED_CONTENT_TYPES:
mime_type = possible_type
parts = path_info.split('/')
if len(parts) > 1:
possible_app, possible_app_url = self._match(host, port, path_info)
# Don't use prefix if it ends up matching default
if possible_app and possible_app_url:
app_url = possible_app_url
app = self._munge_path(possible_app, path_info, app_url)
return mime_type, app, app_url
def _content_type_strategy(self, host, port, environ):
"""Check Content-Type header for API version."""
app = None
params = parse_options_header(environ.get('CONTENT_TYPE', ''))[1]
if 'version' in params:
app, app_url = self._match(host, port, '/v' + params['version'])
if app:
app = self._set_script_name(app, app_url)
return app
def _accept_strategy(self, host, port, environ, supported_content_types):
"""Check Accept header for best matching MIME type and API version."""
accept = Accept(environ.get('HTTP_ACCEPT', ''))
app = None
# Find the best match in the Accept header
mime_type, params = accept.best_match(supported_content_types)
if 'version' in params:
app, app_url = self._match(host, port, '/v' + params['version'])
if app:
app = self._set_script_name(app, app_url)
return mime_type, app
def __call__(self, environ, start_response):
host = environ.get('HTTP_HOST', environ.get('SERVER_NAME')).lower()
if ':' in host:
host, port = host.split(':', 1)
else:
if environ['wsgi.url_scheme'] == 'http':
port = '80'
else:
port = '443'
path_info = environ['PATH_INFO']
path_info = self.normalize_url(path_info, False)[1]
# The MIME type for the response is determined in one of two ways:
# 1) URL path suffix (eg /servers/detail.json)
# 2) Accept header (eg application/json;q=0.8, application/xml;q=0.2)
# The API version is determined in one of three ways:
# 1) URL path prefix (eg /v1.1/tenant/servers/detail)
# 2) Content-Type header (eg application/json;version=1.1)
# 3) Accept header (eg application/json;q=0.8;version=1.1)
supported_content_types = list(wsgi.SUPPORTED_CONTENT_TYPES)
mime_type, app, app_url = self._path_strategy(host, port, path_info)
# Accept application/atom+xml for the index query of each API
# version mount point as well as the root index
if (app_url and app_url + '/' == path_info) or path_info == '/':
supported_content_types.append('application/atom+xml')
if not app:
app = self._content_type_strategy(host, port, environ)
if not mime_type or not app:
possible_mime_type, possible_app = self._accept_strategy(
host, port, environ, supported_content_types)
if possible_mime_type and not mime_type:
mime_type = possible_mime_type
if possible_app and not app:
app = possible_app
if not mime_type:
mime_type = 'application/json'
if not app:
# Didn't match a particular version, probably matches default
app, app_url = self._match(host, port, path_info)
if app:
app = self._munge_path(app, path_info, app_url)
if app:
environ['cinder.best_content_type'] = mime_type
return app(environ, start_response)
environ['paste.urlmap_object'] = self
return self.not_found_application(environ, start_response)

View File

482
cinder/api/v1/limits.py Normal file
View File

@ -0,0 +1,482 @@
# Copyright 2011 OpenStack LLC.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""
Module dedicated functions/classes dealing with rate limiting requests.
"""
import collections
import copy
import httplib
import math
import re
import time
import webob.dec
import webob.exc
from cinder.api.openstack import wsgi
from cinder.api.views import limits as limits_views
from cinder.api import xmlutil
from cinder.openstack.common import importutils
from cinder.openstack.common import jsonutils
from cinder import quota
from cinder import wsgi as base_wsgi
QUOTAS = quota.QUOTAS
# Convenience constants for the limits dictionary passed to Limiter().
PER_SECOND = 1
PER_MINUTE = 60
PER_HOUR = 60 * 60
PER_DAY = 60 * 60 * 24
limits_nsmap = {None: xmlutil.XMLNS_COMMON_V10, 'atom': xmlutil.XMLNS_ATOM}
class LimitsTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('limits', selector='limits')
rates = xmlutil.SubTemplateElement(root, 'rates')
rate = xmlutil.SubTemplateElement(rates, 'rate', selector='rate')
rate.set('uri', 'uri')
rate.set('regex', 'regex')
limit = xmlutil.SubTemplateElement(rate, 'limit', selector='limit')
limit.set('value', 'value')
limit.set('verb', 'verb')
limit.set('remaining', 'remaining')
limit.set('unit', 'unit')
limit.set('next-available', 'next-available')
absolute = xmlutil.SubTemplateElement(root, 'absolute',
selector='absolute')
limit = xmlutil.SubTemplateElement(absolute, 'limit',
selector=xmlutil.get_items)
limit.set('name', 0)
limit.set('value', 1)
return xmlutil.MasterTemplate(root, 1, nsmap=limits_nsmap)
class LimitsController(object):
"""
Controller for accessing limits in the OpenStack API.
"""
@wsgi.serializers(xml=LimitsTemplate)
def index(self, req):
"""
Return all global and rate limit information.
"""
context = req.environ['cinder.context']
quotas = QUOTAS.get_project_quotas(context, context.project_id,
usages=False)
abs_limits = dict((k, v['limit']) for k, v in quotas.items())
rate_limits = req.environ.get("cinder.limits", [])
builder = self._get_view_builder(req)
return builder.build(rate_limits, abs_limits)
def _get_view_builder(self, req):
return limits_views.ViewBuilder()
def create_resource():
return wsgi.Resource(LimitsController())
class Limit(object):
"""
Stores information about a limit for HTTP requests.
"""
UNITS = {
1: "SECOND",
60: "MINUTE",
60 * 60: "HOUR",
60 * 60 * 24: "DAY",
}
UNIT_MAP = dict([(v, k) for k, v in UNITS.items()])
def __init__(self, verb, uri, regex, value, unit):
"""
Initialize a new `Limit`.
@param verb: HTTP verb (POST, PUT, etc.)
@param uri: Human-readable URI
@param regex: Regular expression format for this limit
@param value: Integer number of requests which can be made
@param unit: Unit of measure for the value parameter
"""
self.verb = verb
self.uri = uri
self.regex = regex
self.value = int(value)
self.unit = unit
self.unit_string = self.display_unit().lower()
self.remaining = int(value)
if value <= 0:
raise ValueError("Limit value must be > 0")
self.last_request = None
self.next_request = None
self.water_level = 0
self.capacity = self.unit
self.request_value = float(self.capacity) / float(self.value)
msg = _("Only %(value)s %(verb)s request(s) can be "
"made to %(uri)s every %(unit_string)s.")
self.error_message = msg % self.__dict__
def __call__(self, verb, url):
"""
Represents a call to this limit from a relevant request.
@param verb: string http verb (POST, GET, etc.)
@param url: string URL
"""
if self.verb != verb or not re.match(self.regex, url):
return
now = self._get_time()
if self.last_request is None:
self.last_request = now
leak_value = now - self.last_request
self.water_level -= leak_value
self.water_level = max(self.water_level, 0)
self.water_level += self.request_value
difference = self.water_level - self.capacity
self.last_request = now
if difference > 0:
self.water_level -= self.request_value
self.next_request = now + difference
return difference
cap = self.capacity
water = self.water_level
val = self.value
self.remaining = math.floor(((cap - water) / cap) * val)
self.next_request = now
def _get_time(self):
"""Retrieve the current time. Broken out for testability."""
return time.time()
def display_unit(self):
"""Display the string name of the unit."""
return self.UNITS.get(self.unit, "UNKNOWN")
def display(self):
"""Return a useful representation of this class."""
return {
"verb": self.verb,
"URI": self.uri,
"regex": self.regex,
"value": self.value,
"remaining": int(self.remaining),
"unit": self.display_unit(),
"resetTime": int(self.next_request or self._get_time()),
}
# "Limit" format is a dictionary with the HTTP verb, human-readable URI,
# a regular-expression to match, value and unit of measure (PER_DAY, etc.)
DEFAULT_LIMITS = [
Limit("POST", "*", ".*", 10, PER_MINUTE),
Limit("POST", "*/servers", "^/servers", 50, PER_DAY),
Limit("PUT", "*", ".*", 10, PER_MINUTE),
Limit("GET", "*changes-since*", ".*changes-since.*", 3, PER_MINUTE),
Limit("DELETE", "*", ".*", 100, PER_MINUTE),
]
class RateLimitingMiddleware(base_wsgi.Middleware):
"""
Rate-limits requests passing through this middleware. All limit information
is stored in memory for this implementation.
"""
def __init__(self, application, limits=None, limiter=None, **kwargs):
"""
Initialize new `RateLimitingMiddleware`, which wraps the given WSGI
application and sets up the given limits.
@param application: WSGI application to wrap
@param limits: String describing limits
@param limiter: String identifying class for representing limits
Other parameters are passed to the constructor for the limiter.
"""
base_wsgi.Middleware.__init__(self, application)
# Select the limiter class
if limiter is None:
limiter = Limiter
else:
limiter = importutils.import_class(limiter)
# Parse the limits, if any are provided
if limits is not None:
limits = limiter.parse_limits(limits)
self._limiter = limiter(limits or DEFAULT_LIMITS, **kwargs)
@webob.dec.wsgify(RequestClass=wsgi.Request)
def __call__(self, req):
"""
Represents a single call through this middleware. We should record the
request if we have a limit relevant to it. If no limit is relevant to
the request, ignore it.
If the request should be rate limited, return a fault telling the user
they are over the limit and need to retry later.
"""
verb = req.method
url = req.url
context = req.environ.get("cinder.context")
if context:
username = context.user_id
else:
username = None
delay, error = self._limiter.check_for_delay(verb, url, username)
if delay:
msg = _("This request was rate-limited.")
retry = time.time() + delay
return wsgi.OverLimitFault(msg, error, retry)
req.environ["cinder.limits"] = self._limiter.get_limits(username)
return self.application
class Limiter(object):
"""
Rate-limit checking class which handles limits in memory.
"""
def __init__(self, limits, **kwargs):
"""
Initialize the new `Limiter`.
@param limits: List of `Limit` objects
"""
self.limits = copy.deepcopy(limits)
self.levels = collections.defaultdict(lambda: copy.deepcopy(limits))
# Pick up any per-user limit information
for key, value in kwargs.items():
if key.startswith('user:'):
username = key[5:]
self.levels[username] = self.parse_limits(value)
def get_limits(self, username=None):
"""
Return the limits for a given user.
"""
return [limit.display() for limit in self.levels[username]]
def check_for_delay(self, verb, url, username=None):
"""
Check the given verb/user/user triplet for limit.
@return: Tuple of delay (in seconds) and error message (or None, None)
"""
delays = []
for limit in self.levels[username]:
delay = limit(verb, url)
if delay:
delays.append((delay, limit.error_message))
if delays:
delays.sort()
return delays[0]
return None, None
# Note: This method gets called before the class is instantiated,
# so this must be either a static method or a class method. It is
# used to develop a list of limits to feed to the constructor. We
# put this in the class so that subclasses can override the
# default limit parsing.
@staticmethod
def parse_limits(limits):
"""
Convert a string into a list of Limit instances. This
implementation expects a semicolon-separated sequence of
parenthesized groups, where each group contains a
comma-separated sequence consisting of HTTP method,
user-readable URI, a URI reg-exp, an integer number of
requests which can be made, and a unit of measure. Valid
values for the latter are "SECOND", "MINUTE", "HOUR", and
"DAY".
@return: List of Limit instances.
"""
# Handle empty limit strings
limits = limits.strip()
if not limits:
return []
# Split up the limits by semicolon
result = []
for group in limits.split(';'):
group = group.strip()
if group[:1] != '(' or group[-1:] != ')':
raise ValueError("Limit rules must be surrounded by "
"parentheses")
group = group[1:-1]
# Extract the Limit arguments
args = [a.strip() for a in group.split(',')]
if len(args) != 5:
raise ValueError("Limit rules must contain the following "
"arguments: verb, uri, regex, value, unit")
# Pull out the arguments
verb, uri, regex, value, unit = args
# Upper-case the verb
verb = verb.upper()
# Convert value--raises ValueError if it's not integer
value = int(value)
# Convert unit
unit = unit.upper()
if unit not in Limit.UNIT_MAP:
raise ValueError("Invalid units specified")
unit = Limit.UNIT_MAP[unit]
# Build a limit
result.append(Limit(verb, uri, regex, value, unit))
return result
class WsgiLimiter(object):
"""
Rate-limit checking from a WSGI application. Uses an in-memory `Limiter`.
To use, POST ``/<username>`` with JSON data such as::
{
"verb" : GET,
"path" : "/servers"
}
and receive a 204 No Content, or a 403 Forbidden with an X-Wait-Seconds
header containing the number of seconds to wait before the action would
succeed.
"""
def __init__(self, limits=None):
"""
Initialize the new `WsgiLimiter`.
@param limits: List of `Limit` objects
"""
self._limiter = Limiter(limits or DEFAULT_LIMITS)
@webob.dec.wsgify(RequestClass=wsgi.Request)
def __call__(self, request):
"""
Handles a call to this application. Returns 204 if the request is
acceptable to the limiter, else a 403 is returned with a relevant
header indicating when the request *will* succeed.
"""
if request.method != "POST":
raise webob.exc.HTTPMethodNotAllowed()
try:
info = dict(jsonutils.loads(request.body))
except ValueError:
raise webob.exc.HTTPBadRequest()
username = request.path_info_pop()
verb = info.get("verb")
path = info.get("path")
delay, error = self._limiter.check_for_delay(verb, path, username)
if delay:
headers = {"X-Wait-Seconds": "%.2f" % delay}
return webob.exc.HTTPForbidden(headers=headers, explanation=error)
else:
return webob.exc.HTTPNoContent()
class WsgiLimiterProxy(object):
"""
Rate-limit requests based on answers from a remote source.
"""
def __init__(self, limiter_address):
"""
Initialize the new `WsgiLimiterProxy`.
@param limiter_address: IP/port combination of where to request limit
"""
self.limiter_address = limiter_address
def check_for_delay(self, verb, path, username=None):
body = jsonutils.dumps({"verb": verb, "path": path})
headers = {"Content-Type": "application/json"}
conn = httplib.HTTPConnection(self.limiter_address)
if username:
conn.request("POST", "/%s" % (username), body, headers)
else:
conn.request("POST", "/", body, headers)
resp = conn.getresponse()
if 200 >= resp.status < 300:
return None, None
return resp.getheader("X-Wait-Seconds"), resp.read() or None
# Note: This method gets called before the class is instantiated,
# so this must be either a static method or a class method. It is
# used to develop a list of limits to feed to the constructor.
# This implementation returns an empty list, since all limit
# decisions are made by a remote server.
@staticmethod
def parse_limits(limits):
"""
Ignore a limits string--simply doesn't apply for the limit
proxy.
@return: Empty list.
"""
return []

95
cinder/api/v1/router.py Normal file
View File

@ -0,0 +1,95 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2011 OpenStack LLC.
# Copyright 2011 United States Government as represented by the
# Administrator of the National Aeronautics and Space Administration.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""
WSGI middleware for OpenStack Volume API.
"""
from cinder.api import extensions
import cinder.api.openstack
from cinder.api.v1 import limits
from cinder.api.v1 import snapshot_metadata
from cinder.api.v1 import snapshots
from cinder.api.v1 import types
from cinder.api.v1 import volume_metadata
from cinder.api.v1 import volumes
from cinder.api import versions
from cinder.openstack.common import log as logging
LOG = logging.getLogger(__name__)
class APIRouter(cinder.api.openstack.APIRouter):
"""
Routes requests on the OpenStack API to the appropriate controller
and method.
"""
ExtensionManager = extensions.ExtensionManager
def _setup_routes(self, mapper, ext_mgr):
self.resources['versions'] = versions.create_resource()
mapper.connect("versions", "/",
controller=self.resources['versions'],
action='show')
mapper.redirect("", "/")
self.resources['volumes'] = volumes.create_resource(ext_mgr)
mapper.resource("volume", "volumes",
controller=self.resources['volumes'],
collection={'detail': 'GET'},
member={'action': 'POST'})
self.resources['types'] = types.create_resource()
mapper.resource("type", "types",
controller=self.resources['types'])
self.resources['snapshots'] = snapshots.create_resource(ext_mgr)
mapper.resource("snapshot", "snapshots",
controller=self.resources['snapshots'],
collection={'detail': 'GET'},
member={'action': 'POST'})
self.resources['snapshot_metadata'] = \
snapshot_metadata.create_resource()
snapshot_metadata_controller = self.resources['snapshot_metadata']
mapper.resource("snapshot_metadata", "metadata",
controller=snapshot_metadata_controller,
parent_resource=dict(member_name='snapshot',
collection_name='snapshots'))
self.resources['limits'] = limits.create_resource()
mapper.resource("limit", "limits",
controller=self.resources['limits'])
self.resources['volume_metadata'] = \
volume_metadata.create_resource()
volume_metadata_controller = self.resources['volume_metadata']
mapper.resource("volume_metadata", "metadata",
controller=volume_metadata_controller,
parent_resource=dict(member_name='volume',
collection_name='volumes'))
mapper.connect("metadata",
"/{project_id}/volumes/{volume_id}/metadata",
controller=volume_metadata_controller,
action='update_all',
conditions={"method": ['PUT']})

View File

@ -0,0 +1,164 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2011 OpenStack LLC.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import webob
from cinder.api import common
from cinder.api.openstack import wsgi
from cinder import exception
from cinder import volume
from webob import exc
class Controller(object):
""" The volume metadata API controller for the OpenStack API """
def __init__(self):
self.volume_api = volume.API()
super(Controller, self).__init__()
def _get_metadata(self, context, snapshot_id):
try:
snapshot = self.volume_api.get_snapshot(context, snapshot_id)
meta = self.volume_api.get_snapshot_metadata(context, snapshot)
except exception.SnapshotNotFound:
msg = _('snapshot does not exist')
raise exc.HTTPNotFound(explanation=msg)
return meta
@wsgi.serializers(xml=common.MetadataTemplate)
def index(self, req, snapshot_id):
""" Returns the list of metadata for a given snapshot"""
context = req.environ['cinder.context']
return {'metadata': self._get_metadata(context, snapshot_id)}
@wsgi.serializers(xml=common.MetadataTemplate)
@wsgi.deserializers(xml=common.MetadataDeserializer)
def create(self, req, snapshot_id, body):
try:
metadata = body['metadata']
except (KeyError, TypeError):
msg = _("Malformed request body")
raise exc.HTTPBadRequest(explanation=msg)
context = req.environ['cinder.context']
new_metadata = self._update_snapshot_metadata(context,
snapshot_id,
metadata,
delete=False)
return {'metadata': new_metadata}
@wsgi.serializers(xml=common.MetaItemTemplate)
@wsgi.deserializers(xml=common.MetaItemDeserializer)
def update(self, req, snapshot_id, id, body):
try:
meta_item = body['meta']
except (TypeError, KeyError):
expl = _('Malformed request body')
raise exc.HTTPBadRequest(explanation=expl)
if id not in meta_item:
expl = _('Request body and URI mismatch')
raise exc.HTTPBadRequest(explanation=expl)
if len(meta_item) > 1:
expl = _('Request body contains too many items')
raise exc.HTTPBadRequest(explanation=expl)
context = req.environ['cinder.context']
self._update_snapshot_metadata(context,
snapshot_id,
meta_item,
delete=False)
return {'meta': meta_item}
@wsgi.serializers(xml=common.MetadataTemplate)
@wsgi.deserializers(xml=common.MetadataDeserializer)
def update_all(self, req, snapshot_id, body):
try:
metadata = body['metadata']
except (TypeError, KeyError):
expl = _('Malformed request body')
raise exc.HTTPBadRequest(explanation=expl)
context = req.environ['cinder.context']
new_metadata = self._update_snapshot_metadata(context,
snapshot_id,
metadata,
delete=True)
return {'metadata': new_metadata}
def _update_snapshot_metadata(self, context,
snapshot_id, metadata,
delete=False):
try:
snapshot = self.volume_api.get_snapshot(context, snapshot_id)
return self.volume_api.update_snapshot_metadata(context,
snapshot,
metadata,
delete)
except exception.SnapshotNotFound:
msg = _('snapshot does not exist')
raise exc.HTTPNotFound(explanation=msg)
except (ValueError, AttributeError):
msg = _("Malformed request body")
raise exc.HTTPBadRequest(explanation=msg)
except exception.InvalidVolumeMetadata as error:
raise exc.HTTPBadRequest(explanation=unicode(error))
except exception.InvalidVolumeMetadataSize as error:
raise exc.HTTPRequestEntityTooLarge(explanation=unicode(error))
@wsgi.serializers(xml=common.MetaItemTemplate)
def show(self, req, snapshot_id, id):
""" Return a single metadata item """
context = req.environ['cinder.context']
data = self._get_metadata(context, snapshot_id)
try:
return {'meta': {id: data[id]}}
except KeyError:
msg = _("Metadata item was not found")
raise exc.HTTPNotFound(explanation=msg)
def delete(self, req, snapshot_id, id):
""" Deletes an existing metadata """
context = req.environ['cinder.context']
metadata = self._get_metadata(context, snapshot_id)
if id not in metadata:
msg = _("Metadata item was not found")
raise exc.HTTPNotFound(explanation=msg)
try:
snapshot = self.volume_api.get_snapshot(context, snapshot_id)
self.volume_api.delete_snapshot_metadata(context, snapshot, id)
except exception.SnapshotNotFound:
msg = _('snapshot does not exist')
raise exc.HTTPNotFound(explanation=msg)
return webob.Response(status_int=200)
def create_resource():
return wsgi.Resource(Controller())

234
cinder/api/v1/snapshots.py Normal file
View File

@ -0,0 +1,234 @@
# Copyright 2011 Justin Santa Barbara
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""The volumes snapshots api."""
import webob
from webob import exc
from cinder.api import common
from cinder.api.openstack import wsgi
from cinder.api.v1 import volumes
from cinder.api import xmlutil
from cinder import exception
from cinder import flags
from cinder.openstack.common import log as logging
from cinder.openstack.common import strutils
from cinder import utils
from cinder import volume
LOG = logging.getLogger(__name__)
FLAGS = flags.FLAGS
def _translate_snapshot_detail_view(context, snapshot):
"""Maps keys for snapshots details view."""
d = _translate_snapshot_summary_view(context, snapshot)
# NOTE(gagupta): No additional data / lookups at the moment
return d
def _translate_snapshot_summary_view(context, snapshot):
"""Maps keys for snapshots summary view."""
d = {}
d['id'] = snapshot['id']
d['created_at'] = snapshot['created_at']
d['display_name'] = snapshot['display_name']
d['display_description'] = snapshot['display_description']
d['volume_id'] = snapshot['volume_id']
d['status'] = snapshot['status']
d['size'] = snapshot['volume_size']
if snapshot.get('snapshot_metadata'):
metadata = snapshot.get('snapshot_metadata')
d['metadata'] = dict((item['key'], item['value']) for item in metadata)
# avoid circular ref when vol is a Volume instance
elif snapshot.get('metadata') and isinstance(snapshot.get('metadata'),
dict):
d['metadata'] = snapshot['metadata']
else:
d['metadata'] = {}
return d
def make_snapshot(elem):
elem.set('id')
elem.set('status')
elem.set('size')
elem.set('created_at')
elem.set('display_name')
elem.set('display_description')
elem.set('volume_id')
elem.append(common.MetadataTemplate())
class SnapshotTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('snapshot', selector='snapshot')
make_snapshot(root)
return xmlutil.MasterTemplate(root, 1)
class SnapshotsTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('snapshots')
elem = xmlutil.SubTemplateElement(root, 'snapshot',
selector='snapshots')
make_snapshot(elem)
return xmlutil.MasterTemplate(root, 1)
class SnapshotsController(wsgi.Controller):
"""The Volumes API controller for the OpenStack API."""
def __init__(self, ext_mgr=None):
self.volume_api = volume.API()
self.ext_mgr = ext_mgr
super(SnapshotsController, self).__init__()
@wsgi.serializers(xml=SnapshotTemplate)
def show(self, req, id):
"""Return data about the given snapshot."""
context = req.environ['cinder.context']
try:
vol = self.volume_api.get_snapshot(context, id)
except exception.NotFound:
raise exc.HTTPNotFound()
return {'snapshot': _translate_snapshot_detail_view(context, vol)}
def delete(self, req, id):
"""Delete a snapshot."""
context = req.environ['cinder.context']
LOG.audit(_("Delete snapshot with id: %s"), id, context=context)
try:
snapshot = self.volume_api.get_snapshot(context, id)
self.volume_api.delete_snapshot(context, snapshot)
except exception.NotFound:
raise exc.HTTPNotFound()
return webob.Response(status_int=202)
@wsgi.serializers(xml=SnapshotsTemplate)
def index(self, req):
"""Returns a summary list of snapshots."""
return self._items(req, entity_maker=_translate_snapshot_summary_view)
@wsgi.serializers(xml=SnapshotsTemplate)
def detail(self, req):
"""Returns a detailed list of snapshots."""
return self._items(req, entity_maker=_translate_snapshot_detail_view)
def _items(self, req, entity_maker):
"""Returns a list of snapshots, transformed through entity_maker."""
context = req.environ['cinder.context']
search_opts = {}
search_opts.update(req.GET)
allowed_search_options = ('status', 'volume_id', 'display_name')
volumes.remove_invalid_options(context, search_opts,
allowed_search_options)
snapshots = self.volume_api.get_all_snapshots(context,
search_opts=search_opts)
limited_list = common.limited(snapshots, req)
res = [entity_maker(context, snapshot) for snapshot in limited_list]
return {'snapshots': res}
@wsgi.serializers(xml=SnapshotTemplate)
def create(self, req, body):
"""Creates a new snapshot."""
kwargs = {}
context = req.environ['cinder.context']
if not self.is_valid_body(body, 'snapshot'):
raise exc.HTTPUnprocessableEntity()
snapshot = body['snapshot']
kwargs['metadata'] = snapshot.get('metadata', None)
volume_id = snapshot['volume_id']
volume = self.volume_api.get(context, volume_id)
force = snapshot.get('force', False)
msg = _("Create snapshot from volume %s")
LOG.audit(msg, volume_id, context=context)
if not utils.is_valid_boolstr(force):
msg = _("Invalid value '%s' for force. ") % force
raise exception.InvalidParameterValue(err=msg)
if strutils.bool_from_string(force):
new_snapshot = self.volume_api.create_snapshot_force(
context,
volume,
snapshot.get('display_name'),
snapshot.get('display_description'),
**kwargs)
else:
new_snapshot = self.volume_api.create_snapshot(
context,
volume,
snapshot.get('display_name'),
snapshot.get('display_description'),
**kwargs)
retval = _translate_snapshot_detail_view(context, new_snapshot)
return {'snapshot': retval}
@wsgi.serializers(xml=SnapshotTemplate)
def update(self, req, id, body):
"""Update a snapshot."""
context = req.environ['cinder.context']
if not body:
raise exc.HTTPUnprocessableEntity()
if 'snapshot' not in body:
raise exc.HTTPUnprocessableEntity()
snapshot = body['snapshot']
update_dict = {}
valid_update_keys = (
'display_name',
'display_description',
)
for key in valid_update_keys:
if key in snapshot:
update_dict[key] = snapshot[key]
try:
snapshot = self.volume_api.get_snapshot(context, id)
self.volume_api.update_snapshot(context, snapshot, update_dict)
except exception.NotFound:
raise exc.HTTPNotFound()
snapshot.update(update_dict)
return {'snapshot': _translate_snapshot_detail_view(context, snapshot)}
def create_resource(ext_mgr):
return wsgi.Resource(SnapshotsController(ext_mgr))

80
cinder/api/v1/types.py Normal file
View File

@ -0,0 +1,80 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright (c) 2011 Zadara Storage Inc.
# Copyright (c) 2011 OpenStack LLC.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""The volume type & volume types extra specs extension."""
from webob import exc
from cinder.api.openstack import wsgi
from cinder.api.views import types as views_types
from cinder.api import xmlutil
from cinder import exception
from cinder.volume import volume_types
def make_voltype(elem):
elem.set('id')
elem.set('name')
extra_specs = xmlutil.make_flat_dict('extra_specs', selector='extra_specs')
elem.append(extra_specs)
class VolumeTypeTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('volume_type', selector='volume_type')
make_voltype(root)
return xmlutil.MasterTemplate(root, 1)
class VolumeTypesTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('volume_types')
elem = xmlutil.SubTemplateElement(root, 'volume_type',
selector='volume_types')
make_voltype(elem)
return xmlutil.MasterTemplate(root, 1)
class VolumeTypesController(wsgi.Controller):
"""The volume types API controller for the OpenStack API."""
_view_builder_class = views_types.ViewBuilder
@wsgi.serializers(xml=VolumeTypesTemplate)
def index(self, req):
"""Returns the list of volume types."""
context = req.environ['cinder.context']
vol_types = volume_types.get_all_types(context).values()
return self._view_builder.index(req, vol_types)
@wsgi.serializers(xml=VolumeTypeTemplate)
def show(self, req, id):
"""Return a single volume type item."""
context = req.environ['cinder.context']
try:
vol_type = volume_types.get_volume_type(context, id)
except exception.NotFound:
raise exc.HTTPNotFound()
# TODO(bcwaldon): remove str cast once we use uuids
vol_type['id'] = str(vol_type['id'])
return self._view_builder.show(req, vol_type)
def create_resource():
return wsgi.Resource(VolumeTypesController())

View File

@ -0,0 +1,164 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2011 OpenStack LLC.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import webob
from cinder.api import common
from cinder.api.openstack import wsgi
from cinder import exception
from cinder import volume
from webob import exc
class Controller(object):
""" The volume metadata API controller for the OpenStack API """
def __init__(self):
self.volume_api = volume.API()
super(Controller, self).__init__()
def _get_metadata(self, context, volume_id):
try:
volume = self.volume_api.get(context, volume_id)
meta = self.volume_api.get_volume_metadata(context, volume)
except exception.VolumeNotFound:
msg = _('volume does not exist')
raise exc.HTTPNotFound(explanation=msg)
return meta
@wsgi.serializers(xml=common.MetadataTemplate)
def index(self, req, volume_id):
""" Returns the list of metadata for a given volume"""
context = req.environ['cinder.context']
return {'metadata': self._get_metadata(context, volume_id)}
@wsgi.serializers(xml=common.MetadataTemplate)
@wsgi.deserializers(xml=common.MetadataDeserializer)
def create(self, req, volume_id, body):
try:
metadata = body['metadata']
except (KeyError, TypeError):
msg = _("Malformed request body")
raise exc.HTTPBadRequest(explanation=msg)
context = req.environ['cinder.context']
new_metadata = self._update_volume_metadata(context,
volume_id,
metadata,
delete=False)
return {'metadata': new_metadata}
@wsgi.serializers(xml=common.MetaItemTemplate)
@wsgi.deserializers(xml=common.MetaItemDeserializer)
def update(self, req, volume_id, id, body):
try:
meta_item = body['meta']
except (TypeError, KeyError):
expl = _('Malformed request body')
raise exc.HTTPBadRequest(explanation=expl)
if id not in meta_item:
expl = _('Request body and URI mismatch')
raise exc.HTTPBadRequest(explanation=expl)
if len(meta_item) > 1:
expl = _('Request body contains too many items')
raise exc.HTTPBadRequest(explanation=expl)
context = req.environ['cinder.context']
self._update_volume_metadata(context,
volume_id,
meta_item,
delete=False)
return {'meta': meta_item}
@wsgi.serializers(xml=common.MetadataTemplate)
@wsgi.deserializers(xml=common.MetadataDeserializer)
def update_all(self, req, volume_id, body):
try:
metadata = body['metadata']
except (TypeError, KeyError):
expl = _('Malformed request body')
raise exc.HTTPBadRequest(explanation=expl)
context = req.environ['cinder.context']
new_metadata = self._update_volume_metadata(context,
volume_id,
metadata,
delete=True)
return {'metadata': new_metadata}
def _update_volume_metadata(self, context,
volume_id, metadata,
delete=False):
try:
volume = self.volume_api.get(context, volume_id)
return self.volume_api.update_volume_metadata(context,
volume,
metadata,
delete)
except exception.VolumeNotFound:
msg = _('volume does not exist')
raise exc.HTTPNotFound(explanation=msg)
except (ValueError, AttributeError):
msg = _("Malformed request body")
raise exc.HTTPBadRequest(explanation=msg)
except exception.InvalidVolumeMetadata as error:
raise exc.HTTPBadRequest(explanation=unicode(error))
except exception.InvalidVolumeMetadataSize as error:
raise exc.HTTPRequestEntityTooLarge(explanation=unicode(error))
@wsgi.serializers(xml=common.MetaItemTemplate)
def show(self, req, volume_id, id):
""" Return a single metadata item """
context = req.environ['cinder.context']
data = self._get_metadata(context, volume_id)
try:
return {'meta': {id: data[id]}}
except KeyError:
msg = _("Metadata item was not found")
raise exc.HTTPNotFound(explanation=msg)
def delete(self, req, volume_id, id):
""" Deletes an existing metadata """
context = req.environ['cinder.context']
metadata = self._get_metadata(context, volume_id)
if id not in metadata:
msg = _("Metadata item was not found")
raise exc.HTTPNotFound(explanation=msg)
try:
volume = self.volume_api.get(context, volume_id)
self.volume_api.delete_volume_metadata(context, volume, id)
except exception.VolumeNotFound:
msg = _('volume does not exist')
raise exc.HTTPNotFound(explanation=msg)
return webob.Response(status_int=200)
def create_resource():
return wsgi.Resource(Controller())

421
cinder/api/v1/volumes.py Normal file
View File

@ -0,0 +1,421 @@
# Copyright 2011 Justin Santa Barbara
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""The volumes api."""
import webob
from webob import exc
from cinder.api import common
from cinder.api.openstack import wsgi
from cinder.api import xmlutil
from cinder import exception
from cinder import flags
from cinder.openstack.common import log as logging
from cinder.openstack.common import uuidutils
from cinder import utils
from cinder import volume
from cinder.volume import volume_types
LOG = logging.getLogger(__name__)
FLAGS = flags.FLAGS
def _translate_attachment_detail_view(_context, vol):
"""Maps keys for attachment details view."""
d = _translate_attachment_summary_view(_context, vol)
# No additional data / lookups at the moment
return d
def _translate_attachment_summary_view(_context, vol):
"""Maps keys for attachment summary view."""
d = {}
volume_id = vol['id']
# NOTE(justinsb): We use the volume id as the id of the attachment object
d['id'] = volume_id
d['volume_id'] = volume_id
d['server_id'] = vol['instance_uuid']
if vol.get('mountpoint'):
d['device'] = vol['mountpoint']
return d
def _translate_volume_detail_view(context, vol, image_id=None):
"""Maps keys for volumes details view."""
d = _translate_volume_summary_view(context, vol, image_id)
# No additional data / lookups at the moment
return d
def _translate_volume_summary_view(context, vol, image_id=None):
"""Maps keys for volumes summary view."""
d = {}
d['id'] = vol['id']
d['status'] = vol['status']
d['size'] = vol['size']
d['availability_zone'] = vol['availability_zone']
d['created_at'] = vol['created_at']
d['attachments'] = []
if vol['attach_status'] == 'attached':
attachment = _translate_attachment_detail_view(context, vol)
d['attachments'].append(attachment)
d['display_name'] = vol['display_name']
d['display_description'] = vol['display_description']
if vol['volume_type_id'] and vol.get('volume_type'):
d['volume_type'] = vol['volume_type']['name']
else:
# TODO(bcwaldon): remove str cast once we use uuids
d['volume_type'] = str(vol['volume_type_id'])
d['snapshot_id'] = vol['snapshot_id']
d['source_volid'] = vol['source_volid']
if image_id:
d['image_id'] = image_id
LOG.audit(_("vol=%s"), vol, context=context)
if vol.get('volume_metadata'):
metadata = vol.get('volume_metadata')
d['metadata'] = dict((item['key'], item['value']) for item in metadata)
# avoid circular ref when vol is a Volume instance
elif vol.get('metadata') and isinstance(vol.get('metadata'), dict):
d['metadata'] = vol['metadata']
else:
d['metadata'] = {}
if vol.get('volume_glance_metadata'):
d['bootable'] = 'true'
else:
d['bootable'] = 'false'
return d
def make_attachment(elem):
elem.set('id')
elem.set('server_id')
elem.set('volume_id')
elem.set('device')
def make_volume(elem):
elem.set('id')
elem.set('status')
elem.set('size')
elem.set('availability_zone')
elem.set('created_at')
elem.set('display_name')
elem.set('display_description')
elem.set('volume_type')
elem.set('snapshot_id')
elem.set('source_volid')
attachments = xmlutil.SubTemplateElement(elem, 'attachments')
attachment = xmlutil.SubTemplateElement(attachments, 'attachment',
selector='attachments')
make_attachment(attachment)
# Attach metadata node
elem.append(common.MetadataTemplate())
volume_nsmap = {None: xmlutil.XMLNS_VOLUME_V1, 'atom': xmlutil.XMLNS_ATOM}
class VolumeTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('volume', selector='volume')
make_volume(root)
return xmlutil.MasterTemplate(root, 1, nsmap=volume_nsmap)
class VolumesTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('volumes')
elem = xmlutil.SubTemplateElement(root, 'volume', selector='volumes')
make_volume(elem)
return xmlutil.MasterTemplate(root, 1, nsmap=volume_nsmap)
class CommonDeserializer(wsgi.MetadataXMLDeserializer):
"""Common deserializer to handle xml-formatted volume requests.
Handles standard volume attributes as well as the optional metadata
attribute
"""
metadata_deserializer = common.MetadataXMLDeserializer()
def _extract_volume(self, node):
"""Marshal the volume attribute of a parsed request."""
volume = {}
volume_node = self.find_first_child_named(node, 'volume')
attributes = ['display_name', 'display_description', 'size',
'volume_type', 'availability_zone']
for attr in attributes:
if volume_node.getAttribute(attr):
volume[attr] = volume_node.getAttribute(attr)
metadata_node = self.find_first_child_named(volume_node, 'metadata')
if metadata_node is not None:
volume['metadata'] = self.extract_metadata(metadata_node)
return volume
class CreateDeserializer(CommonDeserializer):
"""Deserializer to handle xml-formatted create volume requests.
Handles standard volume attributes as well as the optional metadata
attribute
"""
def default(self, string):
"""Deserialize an xml-formatted volume create request."""
dom = utils.safe_minidom_parse_string(string)
volume = self._extract_volume(dom)
return {'body': {'volume': volume}}
class VolumeController(wsgi.Controller):
"""The Volumes API controller for the OpenStack API."""
def __init__(self, ext_mgr):
self.volume_api = volume.API()
self.ext_mgr = ext_mgr
super(VolumeController, self).__init__()
@wsgi.serializers(xml=VolumeTemplate)
def show(self, req, id):
"""Return data about the given volume."""
context = req.environ['cinder.context']
try:
vol = self.volume_api.get(context, id)
except exception.NotFound:
raise exc.HTTPNotFound()
return {'volume': _translate_volume_detail_view(context, vol)}
def delete(self, req, id):
"""Delete a volume."""
context = req.environ['cinder.context']
LOG.audit(_("Delete volume with id: %s"), id, context=context)
try:
volume = self.volume_api.get(context, id)
self.volume_api.delete(context, volume)
except exception.NotFound:
raise exc.HTTPNotFound()
return webob.Response(status_int=202)
@wsgi.serializers(xml=VolumesTemplate)
def index(self, req):
"""Returns a summary list of volumes."""
return self._items(req, entity_maker=_translate_volume_summary_view)
@wsgi.serializers(xml=VolumesTemplate)
def detail(self, req):
"""Returns a detailed list of volumes."""
return self._items(req, entity_maker=_translate_volume_detail_view)
def _items(self, req, entity_maker):
"""Returns a list of volumes, transformed through entity_maker."""
search_opts = {}
search_opts.update(req.GET)
context = req.environ['cinder.context']
remove_invalid_options(context,
search_opts, self._get_volume_search_options())
volumes = self.volume_api.get_all(context, marker=None, limit=None,
sort_key='created_at',
sort_dir='desc', filters=search_opts)
limited_list = common.limited(volumes, req)
res = [entity_maker(context, vol) for vol in limited_list]
return {'volumes': res}
def _image_uuid_from_href(self, image_href):
# If the image href was generated by nova api, strip image_href
# down to an id.
try:
image_uuid = image_href.split('/').pop()
except (TypeError, AttributeError):
msg = _("Invalid imageRef provided.")
raise exc.HTTPBadRequest(explanation=msg)
if not uuidutils.is_uuid_like(image_uuid):
msg = _("Invalid imageRef provided.")
raise exc.HTTPBadRequest(explanation=msg)
return image_uuid
@wsgi.serializers(xml=VolumeTemplate)
@wsgi.deserializers(xml=CreateDeserializer)
def create(self, req, body):
"""Creates a new volume."""
if not self.is_valid_body(body, 'volume'):
raise exc.HTTPUnprocessableEntity()
context = req.environ['cinder.context']
volume = body['volume']
kwargs = {}
req_volume_type = volume.get('volume_type', None)
if req_volume_type:
if not uuidutils.is_uuid_like(req_volume_type):
try:
kwargs['volume_type'] = \
volume_types.get_volume_type_by_name(
context, req_volume_type)
except exception.VolumeTypeNotFound:
explanation = 'Volume type not found.'
raise exc.HTTPNotFound(explanation=explanation)
else:
try:
kwargs['volume_type'] = volume_types.get_volume_type(
context, req_volume_type)
except exception.VolumeTypeNotFound:
explanation = 'Volume type not found.'
raise exc.HTTPNotFound(explanation=explanation)
kwargs['metadata'] = volume.get('metadata', None)
snapshot_id = volume.get('snapshot_id')
if snapshot_id is not None:
kwargs['snapshot'] = self.volume_api.get_snapshot(context,
snapshot_id)
else:
kwargs['snapshot'] = None
source_volid = volume.get('source_volid')
if source_volid is not None:
kwargs['source_volume'] = self.volume_api.get_volume(context,
source_volid)
else:
kwargs['source_volume'] = None
size = volume.get('size', None)
if size is None and kwargs['snapshot'] is not None:
size = kwargs['snapshot']['volume_size']
elif size is None and kwargs['source_volume'] is not None:
size = kwargs['source_volume']['size']
LOG.audit(_("Create volume of %s GB"), size, context=context)
image_href = None
image_uuid = None
if self.ext_mgr.is_loaded('os-image-create'):
image_href = volume.get('imageRef')
if image_href:
image_uuid = self._image_uuid_from_href(image_href)
kwargs['image_id'] = image_uuid
kwargs['availability_zone'] = volume.get('availability_zone', None)
new_volume = self.volume_api.create(context,
size,
volume.get('display_name'),
volume.get('display_description'),
**kwargs)
# TODO(vish): Instance should be None at db layer instead of
# trying to lazy load, but for now we turn it into
# a dict to avoid an error.
retval = _translate_volume_detail_view(context,
dict(new_volume.iteritems()),
image_uuid)
return {'volume': retval}
def _get_volume_search_options(self):
"""Return volume search options allowed by non-admin."""
return ('display_name', 'status')
@wsgi.serializers(xml=VolumeTemplate)
def update(self, req, id, body):
"""Update a volume."""
context = req.environ['cinder.context']
if not body:
raise exc.HTTPUnprocessableEntity()
if 'volume' not in body:
raise exc.HTTPUnprocessableEntity()
volume = body['volume']
update_dict = {}
valid_update_keys = (
'display_name',
'display_description',
'metadata',
)
for key in valid_update_keys:
if key in volume:
update_dict[key] = volume[key]
try:
volume = self.volume_api.get(context, id)
self.volume_api.update(context, volume, update_dict)
except exception.NotFound:
raise exc.HTTPNotFound()
volume.update(update_dict)
return {'volume': _translate_volume_detail_view(context, volume)}
def create_resource(ext_mgr):
return wsgi.Resource(VolumeController(ext_mgr))
def remove_invalid_options(context, search_options, allowed_search_options):
"""Remove search options that are not valid for non-admin API/context."""
if context.is_admin:
# Allow all options
return
# Otherwise, strip out all unknown options
unknown_options = [opt for opt in search_options
if opt not in allowed_search_options]
bad_options = ", ".join(unknown_options)
log_msg = _("Removing options '%(bad_options)s' from query") % locals()
LOG.debug(log_msg)
for opt in unknown_options:
del search_options[opt]

View File

482
cinder/api/v2/limits.py Normal file
View File

@ -0,0 +1,482 @@
# Copyright 2011 OpenStack LLC.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""
Module dedicated functions/classes dealing with rate limiting requests.
"""
import collections
import copy
import httplib
import math
import re
import time
import webob.dec
import webob.exc
from cinder.api.openstack import wsgi
from cinder.api.views import limits as limits_views
from cinder.api import xmlutil
from cinder.openstack.common import importutils
from cinder.openstack.common import jsonutils
from cinder import quota
from cinder import wsgi as base_wsgi
QUOTAS = quota.QUOTAS
# Convenience constants for the limits dictionary passed to Limiter().
PER_SECOND = 1
PER_MINUTE = 60
PER_HOUR = 60 * 60
PER_DAY = 60 * 60 * 24
limits_nsmap = {None: xmlutil.XMLNS_COMMON_V10, 'atom': xmlutil.XMLNS_ATOM}
class LimitsTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('limits', selector='limits')
rates = xmlutil.SubTemplateElement(root, 'rates')
rate = xmlutil.SubTemplateElement(rates, 'rate', selector='rate')
rate.set('uri', 'uri')
rate.set('regex', 'regex')
limit = xmlutil.SubTemplateElement(rate, 'limit', selector='limit')
limit.set('value', 'value')
limit.set('verb', 'verb')
limit.set('remaining', 'remaining')
limit.set('unit', 'unit')
limit.set('next-available', 'next-available')
absolute = xmlutil.SubTemplateElement(root, 'absolute',
selector='absolute')
limit = xmlutil.SubTemplateElement(absolute, 'limit',
selector=xmlutil.get_items)
limit.set('name', 0)
limit.set('value', 1)
return xmlutil.MasterTemplate(root, 1, nsmap=limits_nsmap)
class LimitsController(object):
"""
Controller for accessing limits in the OpenStack API.
"""
@wsgi.serializers(xml=LimitsTemplate)
def index(self, req):
"""
Return all global and rate limit information.
"""
context = req.environ['cinder.context']
quotas = QUOTAS.get_project_quotas(context, context.project_id,
usages=False)
abs_limits = dict((k, v['limit']) for k, v in quotas.items())
rate_limits = req.environ.get("cinder.limits", [])
builder = self._get_view_builder(req)
return builder.build(rate_limits, abs_limits)
def _get_view_builder(self, req):
return limits_views.ViewBuilder()
def create_resource():
return wsgi.Resource(LimitsController())
class Limit(object):
"""
Stores information about a limit for HTTP requests.
"""
UNITS = {
1: "SECOND",
60: "MINUTE",
60 * 60: "HOUR",
60 * 60 * 24: "DAY",
}
UNIT_MAP = dict([(v, k) for k, v in UNITS.items()])
def __init__(self, verb, uri, regex, value, unit):
"""
Initialize a new `Limit`.
@param verb: HTTP verb (POST, PUT, etc.)
@param uri: Human-readable URI
@param regex: Regular expression format for this limit
@param value: Integer number of requests which can be made
@param unit: Unit of measure for the value parameter
"""
self.verb = verb
self.uri = uri
self.regex = regex
self.value = int(value)
self.unit = unit
self.unit_string = self.display_unit().lower()
self.remaining = int(value)
if value <= 0:
raise ValueError("Limit value must be > 0")
self.last_request = None
self.next_request = None
self.water_level = 0
self.capacity = self.unit
self.request_value = float(self.capacity) / float(self.value)
msg = _("Only %(value)s %(verb)s request(s) can be "
"made to %(uri)s every %(unit_string)s.")
self.error_message = msg % self.__dict__
def __call__(self, verb, url):
"""
Represents a call to this limit from a relevant request.
@param verb: string http verb (POST, GET, etc.)
@param url: string URL
"""
if self.verb != verb or not re.match(self.regex, url):
return
now = self._get_time()
if self.last_request is None:
self.last_request = now
leak_value = now - self.last_request
self.water_level -= leak_value
self.water_level = max(self.water_level, 0)
self.water_level += self.request_value
difference = self.water_level - self.capacity
self.last_request = now
if difference > 0:
self.water_level -= self.request_value
self.next_request = now + difference
return difference
cap = self.capacity
water = self.water_level
val = self.value
self.remaining = math.floor(((cap - water) / cap) * val)
self.next_request = now
def _get_time(self):
"""Retrieve the current time. Broken out for testability."""
return time.time()
def display_unit(self):
"""Display the string name of the unit."""
return self.UNITS.get(self.unit, "UNKNOWN")
def display(self):
"""Return a useful representation of this class."""
return {
"verb": self.verb,
"URI": self.uri,
"regex": self.regex,
"value": self.value,
"remaining": int(self.remaining),
"unit": self.display_unit(),
"resetTime": int(self.next_request or self._get_time()),
}
# "Limit" format is a dictionary with the HTTP verb, human-readable URI,
# a regular-expression to match, value and unit of measure (PER_DAY, etc.)
DEFAULT_LIMITS = [
Limit("POST", "*", ".*", 10, PER_MINUTE),
Limit("POST", "*/servers", "^/servers", 50, PER_DAY),
Limit("PUT", "*", ".*", 10, PER_MINUTE),
Limit("GET", "*changes-since*", ".*changes-since.*", 3, PER_MINUTE),
Limit("DELETE", "*", ".*", 100, PER_MINUTE),
]
class RateLimitingMiddleware(base_wsgi.Middleware):
"""
Rate-limits requests passing through this middleware. All limit information
is stored in memory for this implementation.
"""
def __init__(self, application, limits=None, limiter=None, **kwargs):
"""
Initialize new `RateLimitingMiddleware`, which wraps the given WSGI
application and sets up the given limits.
@param application: WSGI application to wrap
@param limits: String describing limits
@param limiter: String identifying class for representing limits
Other parameters are passed to the constructor for the limiter.
"""
base_wsgi.Middleware.__init__(self, application)
# Select the limiter class
if limiter is None:
limiter = Limiter
else:
limiter = importutils.import_class(limiter)
# Parse the limits, if any are provided
if limits is not None:
limits = limiter.parse_limits(limits)
self._limiter = limiter(limits or DEFAULT_LIMITS, **kwargs)
@webob.dec.wsgify(RequestClass=wsgi.Request)
def __call__(self, req):
"""
Represents a single call through this middleware. We should record the
request if we have a limit relevant to it. If no limit is relevant to
the request, ignore it.
If the request should be rate limited, return a fault telling the user
they are over the limit and need to retry later.
"""
verb = req.method
url = req.url
context = req.environ.get("cinder.context")
if context:
username = context.user_id
else:
username = None
delay, error = self._limiter.check_for_delay(verb, url, username)
if delay:
msg = _("This request was rate-limited.")
retry = time.time() + delay
return wsgi.OverLimitFault(msg, error, retry)
req.environ["cinder.limits"] = self._limiter.get_limits(username)
return self.application
class Limiter(object):
"""
Rate-limit checking class which handles limits in memory.
"""
def __init__(self, limits, **kwargs):
"""
Initialize the new `Limiter`.
@param limits: List of `Limit` objects
"""
self.limits = copy.deepcopy(limits)
self.levels = collections.defaultdict(lambda: copy.deepcopy(limits))
# Pick up any per-user limit information
for key, value in kwargs.items():
if key.startswith('user:'):
username = key[5:]
self.levels[username] = self.parse_limits(value)
def get_limits(self, username=None):
"""
Return the limits for a given user.
"""
return [limit.display() for limit in self.levels[username]]
def check_for_delay(self, verb, url, username=None):
"""
Check the given verb/user/user triplet for limit.
@return: Tuple of delay (in seconds) and error message (or None, None)
"""
delays = []
for limit in self.levels[username]:
delay = limit(verb, url)
if delay:
delays.append((delay, limit.error_message))
if delays:
delays.sort()
return delays[0]
return None, None
# Note: This method gets called before the class is instantiated,
# so this must be either a static method or a class method. It is
# used to develop a list of limits to feed to the constructor. We
# put this in the class so that subclasses can override the
# default limit parsing.
@staticmethod
def parse_limits(limits):
"""
Convert a string into a list of Limit instances. This
implementation expects a semicolon-separated sequence of
parenthesized groups, where each group contains a
comma-separated sequence consisting of HTTP method,
user-readable URI, a URI reg-exp, an integer number of
requests which can be made, and a unit of measure. Valid
values for the latter are "SECOND", "MINUTE", "HOUR", and
"DAY".
@return: List of Limit instances.
"""
# Handle empty limit strings
limits = limits.strip()
if not limits:
return []
# Split up the limits by semicolon
result = []
for group in limits.split(';'):
group = group.strip()
if group[:1] != '(' or group[-1:] != ')':
raise ValueError("Limit rules must be surrounded by "
"parentheses")
group = group[1:-1]
# Extract the Limit arguments
args = [a.strip() for a in group.split(',')]
if len(args) != 5:
raise ValueError("Limit rules must contain the following "
"arguments: verb, uri, regex, value, unit")
# Pull out the arguments
verb, uri, regex, value, unit = args
# Upper-case the verb
verb = verb.upper()
# Convert value--raises ValueError if it's not integer
value = int(value)
# Convert unit
unit = unit.upper()
if unit not in Limit.UNIT_MAP:
raise ValueError("Invalid units specified")
unit = Limit.UNIT_MAP[unit]
# Build a limit
result.append(Limit(verb, uri, regex, value, unit))
return result
class WsgiLimiter(object):
"""
Rate-limit checking from a WSGI application. Uses an in-memory `Limiter`.
To use, POST ``/<username>`` with JSON data such as::
{
"verb" : GET,
"path" : "/servers"
}
and receive a 204 No Content, or a 403 Forbidden with an X-Wait-Seconds
header containing the number of seconds to wait before the action would
succeed.
"""
def __init__(self, limits=None):
"""
Initialize the new `WsgiLimiter`.
@param limits: List of `Limit` objects
"""
self._limiter = Limiter(limits or DEFAULT_LIMITS)
@webob.dec.wsgify(RequestClass=wsgi.Request)
def __call__(self, request):
"""
Handles a call to this application. Returns 204 if the request is
acceptable to the limiter, else a 403 is returned with a relevant
header indicating when the request *will* succeed.
"""
if request.method != "POST":
raise webob.exc.HTTPMethodNotAllowed()
try:
info = dict(jsonutils.loads(request.body))
except ValueError:
raise webob.exc.HTTPBadRequest()
username = request.path_info_pop()
verb = info.get("verb")
path = info.get("path")
delay, error = self._limiter.check_for_delay(verb, path, username)
if delay:
headers = {"X-Wait-Seconds": "%.2f" % delay}
return webob.exc.HTTPForbidden(headers=headers, explanation=error)
else:
return webob.exc.HTTPNoContent()
class WsgiLimiterProxy(object):
"""
Rate-limit requests based on answers from a remote source.
"""
def __init__(self, limiter_address):
"""
Initialize the new `WsgiLimiterProxy`.
@param limiter_address: IP/port combination of where to request limit
"""
self.limiter_address = limiter_address
def check_for_delay(self, verb, path, username=None):
body = jsonutils.dumps({"verb": verb, "path": path})
headers = {"Content-Type": "application/json"}
conn = httplib.HTTPConnection(self.limiter_address)
if username:
conn.request("POST", "/%s" % (username), body, headers)
else:
conn.request("POST", "/", body, headers)
resp = conn.getresponse()
if 200 >= resp.status < 300:
return None, None
return resp.getheader("X-Wait-Seconds"), resp.read() or None
# Note: This method gets called before the class is instantiated,
# so this must be either a static method or a class method. It is
# used to develop a list of limits to feed to the constructor.
# This implementation returns an empty list, since all limit
# decisions are made by a remote server.
@staticmethod
def parse_limits(limits):
"""
Ignore a limits string--simply doesn't apply for the limit
proxy.
@return: Empty list.
"""
return []

70
cinder/api/v2/router.py Normal file
View File

@ -0,0 +1,70 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2011 OpenStack LLC.
# Copyright 2011 United States Government as represented by the
# Administrator of the National Aeronautics and Space Administration.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""
WSGI middleware for OpenStack Volume API.
"""
from cinder.api import extensions
import cinder.api.openstack
from cinder.api.v2 import limits
from cinder.api.v2 import snapshots
from cinder.api.v2 import types
from cinder.api.v2 import volumes
from cinder.api import versions
from cinder.openstack.common import log as logging
LOG = logging.getLogger(__name__)
class APIRouter(cinder.api.openstack.APIRouter):
"""
Routes requests on the OpenStack API to the appropriate controller
and method.
"""
ExtensionManager = extensions.ExtensionManager
def _setup_routes(self, mapper, ext_mgr):
self.resources['versions'] = versions.create_resource()
mapper.connect("versions", "/",
controller=self.resources['versions'],
action='show')
mapper.redirect("", "/")
self.resources['volumes'] = volumes.create_resource(ext_mgr)
mapper.resource("volume", "volumes",
controller=self.resources['volumes'],
collection={'detail': 'GET'},
member={'action': 'POST'})
self.resources['types'] = types.create_resource()
mapper.resource("type", "types",
controller=self.resources['types'])
self.resources['snapshots'] = snapshots.create_resource(ext_mgr)
mapper.resource("snapshot", "snapshots",
controller=self.resources['snapshots'],
collection={'detail': 'GET'},
member={'action': 'POST'})
self.resources['limits'] = limits.create_resource()
mapper.resource("limit", "limits",
controller=self.resources['limits'])

View File

@ -0,0 +1,164 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2011 OpenStack LLC.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import webob
from cinder.api import common
from cinder.api.openstack import wsgi
from cinder import exception
from cinder import volume
from webob import exc
class Controller(object):
""" The volume metadata API controller for the OpenStack API """
def __init__(self):
self.volume_api = volume.API()
super(Controller, self).__init__()
def _get_metadata(self, context, snapshot_id):
try:
snapshot = self.volume_api.get_snapshot(context, snapshot_id)
meta = self.volume_api.get_snapshot_metadata(context, snapshot)
except exception.SnapshotNotFound:
msg = _('snapshot does not exist')
raise exc.HTTPNotFound(explanation=msg)
return meta
@wsgi.serializers(xml=common.MetadataTemplate)
def index(self, req, snapshot_id):
""" Returns the list of metadata for a given snapshot"""
context = req.environ['cinder.context']
return {'metadata': self._get_metadata(context, snapshot_id)}
@wsgi.serializers(xml=common.MetadataTemplate)
@wsgi.deserializers(xml=common.MetadataDeserializer)
def create(self, req, snapshot_id, body):
try:
metadata = body['metadata']
except (KeyError, TypeError):
msg = _("Malformed request body")
raise exc.HTTPBadRequest(explanation=msg)
context = req.environ['cinder.context']
new_metadata = self._update_snapshot_metadata(context,
snapshot_id,
metadata,
delete=False)
return {'metadata': new_metadata}
@wsgi.serializers(xml=common.MetaItemTemplate)
@wsgi.deserializers(xml=common.MetaItemDeserializer)
def update(self, req, snapshot_id, id, body):
try:
meta_item = body['meta']
except (TypeError, KeyError):
expl = _('Malformed request body')
raise exc.HTTPBadRequest(explanation=expl)
if id not in meta_item:
expl = _('Request body and URI mismatch')
raise exc.HTTPBadRequest(explanation=expl)
if len(meta_item) > 1:
expl = _('Request body contains too many items')
raise exc.HTTPBadRequest(explanation=expl)
context = req.environ['cinder.context']
self._update_snapshot_metadata(context,
snapshot_id,
meta_item,
delete=False)
return {'meta': meta_item}
@wsgi.serializers(xml=common.MetadataTemplate)
@wsgi.deserializers(xml=common.MetadataDeserializer)
def update_all(self, req, snapshot_id, body):
try:
metadata = body['metadata']
except (TypeError, KeyError):
expl = _('Malformed request body')
raise exc.HTTPBadRequest(explanation=expl)
context = req.environ['cinder.context']
new_metadata = self._update_snapshot_metadata(context,
snapshot_id,
metadata,
delete=True)
return {'metadata': new_metadata}
def _update_snapshot_metadata(self, context,
snapshot_id, metadata,
delete=False):
try:
snapshot = self.volume_api.get_snapshot(context, snapshot_id)
return self.volume_api.update_snapshot_metadata(context,
snapshot,
metadata,
delete)
except exception.SnapshotNotFound:
msg = _('snapshot does not exist')
raise exc.HTTPNotFound(explanation=msg)
except (ValueError, AttributeError):
msg = _("Malformed request body")
raise exc.HTTPBadRequest(explanation=msg)
except exception.InvalidVolumeMetadata as error:
raise exc.HTTPBadRequest(explanation=unicode(error))
except exception.InvalidVolumeMetadataSize as error:
raise exc.HTTPRequestEntityTooLarge(explanation=unicode(error))
@wsgi.serializers(xml=common.MetaItemTemplate)
def show(self, req, snapshot_id, id):
""" Return a single metadata item """
context = req.environ['cinder.context']
data = self._get_metadata(context, snapshot_id)
try:
return {'meta': {id: data[id]}}
except KeyError:
msg = _("Metadata item was not found")
raise exc.HTTPNotFound(explanation=msg)
def delete(self, req, snapshot_id, id):
""" Deletes an existing metadata """
context = req.environ['cinder.context']
metadata = self._get_metadata(context, snapshot_id)
if id not in metadata:
msg = _("Metadata item was not found")
raise exc.HTTPNotFound(explanation=msg)
try:
snapshot = self.volume_api.get_snapshot(context, snapshot_id)
self.volume_api.delete_snapshot_metadata(context, snapshot, id)
except exception.SnapshotNotFound:
msg = _('snapshot does not exist')
raise exc.HTTPNotFound(explanation=msg)
return webob.Response(status_int=200)
def create_resource():
return wsgi.Resource(Controller())

257
cinder/api/v2/snapshots.py Normal file
View File

@ -0,0 +1,257 @@
# Copyright 2011 Justin Santa Barbara
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""The volumes snapshots api."""
import webob
from webob import exc
from cinder.api import common
from cinder.api.openstack import wsgi
from cinder.api.v2 import volumes
from cinder.api import xmlutil
from cinder import exception
from cinder import flags
from cinder.openstack.common import log as logging
from cinder.openstack.common import strutils
from cinder import utils
from cinder import volume
LOG = logging.getLogger(__name__)
FLAGS = flags.FLAGS
def _translate_snapshot_detail_view(context, snapshot):
"""Maps keys for snapshots details view."""
d = _translate_snapshot_summary_view(context, snapshot)
# NOTE(gagupta): No additional data / lookups at the moment
return d
def _translate_snapshot_summary_view(context, snapshot):
"""Maps keys for snapshots summary view."""
d = {}
d['id'] = snapshot['id']
d['created_at'] = snapshot['created_at']
d['name'] = snapshot['display_name']
d['description'] = snapshot['display_description']
d['volume_id'] = snapshot['volume_id']
d['status'] = snapshot['status']
d['size'] = snapshot['volume_size']
if snapshot.get('snapshot_metadata'):
metadata = snapshot.get('snapshot_metadata')
d['metadata'] = dict((item['key'], item['value']) for item in metadata)
# avoid circular ref when vol is a Volume instance
elif snapshot.get('metadata') and isinstance(snapshot.get('metadata'),
dict):
d['metadata'] = snapshot['metadata']
else:
d['metadata'] = {}
return d
def make_snapshot(elem):
elem.set('id')
elem.set('status')
elem.set('size')
elem.set('created_at')
elem.set('name')
elem.set('description')
elem.set('volume_id')
elem.append(common.MetadataTemplate())
class SnapshotTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('snapshot', selector='snapshot')
make_snapshot(root)
return xmlutil.MasterTemplate(root, 1)
class SnapshotsTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('snapshots')
elem = xmlutil.SubTemplateElement(root, 'snapshot',
selector='snapshots')
make_snapshot(elem)
return xmlutil.MasterTemplate(root, 1)
class SnapshotsController(wsgi.Controller):
"""The Volumes API controller for the OpenStack API."""
def __init__(self, ext_mgr=None):
self.volume_api = volume.API()
self.ext_mgr = ext_mgr
super(SnapshotsController, self).__init__()
@wsgi.serializers(xml=SnapshotTemplate)
def show(self, req, id):
"""Return data about the given snapshot."""
context = req.environ['cinder.context']
try:
vol = self.volume_api.get_snapshot(context, id)
except exception.NotFound:
raise exc.HTTPNotFound()
return {'snapshot': _translate_snapshot_detail_view(context, vol)}
def delete(self, req, id):
"""Delete a snapshot."""
context = req.environ['cinder.context']
LOG.audit(_("Delete snapshot with id: %s"), id, context=context)
try:
snapshot = self.volume_api.get_snapshot(context, id)
self.volume_api.delete_snapshot(context, snapshot)
except exception.NotFound:
raise exc.HTTPNotFound()
return webob.Response(status_int=202)
@wsgi.serializers(xml=SnapshotsTemplate)
def index(self, req):
"""Returns a summary list of snapshots."""
return self._items(req, entity_maker=_translate_snapshot_summary_view)
@wsgi.serializers(xml=SnapshotsTemplate)
def detail(self, req):
"""Returns a detailed list of snapshots."""
return self._items(req, entity_maker=_translate_snapshot_detail_view)
def _items(self, req, entity_maker):
"""Returns a list of snapshots, transformed through entity_maker."""
context = req.environ['cinder.context']
search_opts = {}
search_opts.update(req.GET)
allowed_search_options = ('status', 'volume_id', 'name')
volumes.remove_invalid_options(context, search_opts,
allowed_search_options)
# NOTE(thingee): v2 API allows name instead of display_name
if 'name' in search_opts:
search_opts['display_name'] = search_opts['name']
del search_opts['name']
snapshots = self.volume_api.get_all_snapshots(context,
search_opts=search_opts)
limited_list = common.limited(snapshots, req)
res = [entity_maker(context, snapshot) for snapshot in limited_list]
return {'snapshots': res}
@wsgi.response(202)
@wsgi.serializers(xml=SnapshotTemplate)
def create(self, req, body):
"""Creates a new snapshot."""
kwargs = {}
context = req.environ['cinder.context']
if not self.is_valid_body(body, 'snapshot'):
raise exc.HTTPBadRequest()
snapshot = body['snapshot']
kwargs['metadata'] = snapshot.get('metadata', None)
volume_id = snapshot['volume_id']
volume = self.volume_api.get(context, volume_id)
force = snapshot.get('force', False)
msg = _("Create snapshot from volume %s")
LOG.audit(msg, volume_id, context=context)
# NOTE(thingee): v2 API allows name instead of display_name
if 'name' in snapshot:
snapshot['display_name'] = snapshot.get('name')
del snapshot['name']
if not utils.is_valid_boolstr(force):
msg = _("Invalid value '%s' for force. ") % force
raise exception.InvalidParameterValue(err=msg)
if strutils.bool_from_string(force):
new_snapshot = self.volume_api.create_snapshot_force(
context,
volume,
snapshot.get('display_name'),
snapshot.get('description'),
**kwargs)
else:
new_snapshot = self.volume_api.create_snapshot(
context,
volume,
snapshot.get('display_name'),
snapshot.get('description'),
**kwargs)
retval = _translate_snapshot_detail_view(context, new_snapshot)
return {'snapshot': retval}
@wsgi.serializers(xml=SnapshotTemplate)
def update(self, req, id, body):
"""Update a snapshot."""
context = req.environ['cinder.context']
if not body:
raise exc.HTTPBadRequest()
if 'snapshot' not in body:
raise exc.HTTPBadRequest()
snapshot = body['snapshot']
update_dict = {}
valid_update_keys = (
'name',
'description',
'display_description',
)
# NOTE(thingee): v2 API allows description instead of
# display_description
if 'description' in snapshot:
snapshot['display_description'] = snapshot['description']
del snapshot['description']
for key in valid_update_keys:
if key in snapshot:
update_dict[key] = snapshot[key]
# NOTE(thingee): v2 API allows name instead of display_name
if 'name' in update_dict:
update_dict['display_name'] = update_dict['name']
del update_dict['name']
try:
snapshot = self.volume_api.get_snapshot(context, id)
self.volume_api.update_snapshot(context, snapshot, update_dict)
except exception.NotFound:
raise exc.HTTPNotFound()
snapshot.update(update_dict)
return {'snapshot': _translate_snapshot_detail_view(context, snapshot)}
def create_resource(ext_mgr):
return wsgi.Resource(SnapshotsController(ext_mgr))

80
cinder/api/v2/types.py Normal file
View File

@ -0,0 +1,80 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright (c) 2011 Zadara Storage Inc.
# Copyright (c) 2011 OpenStack LLC.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""The volume type & volume types extra specs extension."""
from webob import exc
from cinder.api.openstack import wsgi
from cinder.api.views import types as views_types
from cinder.api import xmlutil
from cinder import exception
from cinder.volume import volume_types
def make_voltype(elem):
elem.set('id')
elem.set('name')
extra_specs = xmlutil.make_flat_dict('extra_specs', selector='extra_specs')
elem.append(extra_specs)
class VolumeTypeTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('volume_type', selector='volume_type')
make_voltype(root)
return xmlutil.MasterTemplate(root, 1)
class VolumeTypesTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('volume_types')
elem = xmlutil.SubTemplateElement(root, 'volume_type',
selector='volume_types')
make_voltype(elem)
return xmlutil.MasterTemplate(root, 1)
class VolumeTypesController(wsgi.Controller):
"""The volume types API controller for the OpenStack API."""
_view_builder_class = views_types.ViewBuilder
@wsgi.serializers(xml=VolumeTypesTemplate)
def index(self, req):
"""Returns the list of volume types."""
context = req.environ['cinder.context']
vol_types = volume_types.get_all_types(context).values()
return self._view_builder.index(req, vol_types)
@wsgi.serializers(xml=VolumeTypeTemplate)
def show(self, req, id):
"""Return a single volume type item."""
context = req.environ['cinder.context']
try:
vol_type = volume_types.get_volume_type(context, id)
except exception.NotFound:
raise exc.HTTPNotFound()
# TODO(bcwaldon): remove str cast once we use uuids
vol_type['id'] = str(vol_type['id'])
return self._view_builder.show(req, vol_type)
def create_resource():
return wsgi.Resource(VolumeTypesController())

View File

@ -0,0 +1,16 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2012 OpenStack LLC.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.

View File

@ -0,0 +1,122 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2012 OpenStack LLC.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from cinder.api import common
from cinder.openstack.common import log as logging
LOG = logging.getLogger(__name__)
class ViewBuilder(common.ViewBuilder):
"""Model a server API response as a python dictionary."""
_collection_name = "volumes"
def __init__(self):
"""Initialize view builder."""
super(ViewBuilder, self).__init__()
def summary_list(self, request, volumes):
"""Show a list of volumes without many details."""
return self._list_view(self.summary, request, volumes)
def detail_list(self, request, volumes):
"""Detailed view of a list of volumes."""
return self._list_view(self.detail, request, volumes)
def summary(self, request, volume):
"""Generic, non-detailed view of an volume."""
return {
'volume': {
'id': volume['id'],
'name': volume['display_name'],
'links': self._get_links(request,
volume['id']),
},
}
def detail(self, request, volume):
"""Detailed view of a single volume."""
return {
'volume': {
'id': volume.get('id'),
'status': volume.get('status'),
'size': volume.get('size'),
'availability_zone': volume.get('availability_zone'),
'created_at': volume.get('created_at'),
'attachments': self._get_attachments(volume),
'name': volume.get('display_name'),
'description': volume.get('display_description'),
'volume_type': self._get_volume_type(volume),
'snapshot_id': volume.get('snapshot_id'),
'source_volid': volume.get('source_volid'),
'metadata': self._get_volume_metadata(volume),
'links': self._get_links(request, volume['id'])
}
}
def _get_attachments(self, volume):
"""Retrieves the attachments of the volume object"""
attachments = []
if volume['attach_status'] == 'attached':
d = {}
volume_id = volume['id']
# note(justinsb): we use the volume id as the id of the attachments
# object
d['id'] = volume_id
d['volume_id'] = volume_id
d['server_id'] = volume['instance_uuid']
if volume.get('mountpoint'):
d['device'] = volume['mountpoint']
attachments.append(d)
return attachments
def _get_volume_metadata(self, volume):
"""Retrieves the metadata of the volume object"""
if volume.get('volume_metadata'):
metadata = volume.get('volume_metadata')
return dict((item['key'], item['value']) for item in metadata)
# avoid circular ref when vol is a Volume instance
elif volume.get('metadata') and isinstance(volume.get('metadata'),
dict):
return volume['metadata']
return {}
def _get_volume_type(self, volume):
"""Retrieves the type the volume object is"""
if volume['volume_type_id'] and volume.get('volume_type'):
return volume['volume_type']['name']
else:
return volume['volume_type_id']
def _list_view(self, func, request, volumes):
"""Provide a view for a list of volumes."""
volumes_list = [func(request, volume)['volume'] for volume in volumes]
volumes_links = self._get_collection_links(request,
volumes,
self._collection_name)
volumes_dict = dict(volumes=volumes_list)
if volumes_links:
volumes_dict['volumes_links'] = volumes_links
return volumes_dict

362
cinder/api/v2/volumes.py Normal file
View File

@ -0,0 +1,362 @@
# Copyright 2011 Justin Santa Barbara
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""The volumes api."""
import webob
from webob import exc
from cinder.api import common
from cinder.api.openstack import wsgi
from cinder.api.v2.views import volumes as volume_views
from cinder.api import xmlutil
from cinder import exception
from cinder import flags
from cinder.openstack.common import log as logging
from cinder.openstack.common import uuidutils
from cinder import utils
from cinder import volume
from cinder.volume import volume_types
LOG = logging.getLogger(__name__)
FLAGS = flags.FLAGS
def make_attachment(elem):
elem.set('id')
elem.set('server_id')
elem.set('volume_id')
elem.set('device')
def make_volume(elem):
elem.set('id')
elem.set('status')
elem.set('size')
elem.set('availability_zone')
elem.set('created_at')
elem.set('name')
elem.set('description')
elem.set('volume_type')
elem.set('snapshot_id')
elem.set('source_volid')
attachments = xmlutil.SubTemplateElement(elem, 'attachments')
attachment = xmlutil.SubTemplateElement(attachments, 'attachment',
selector='attachments')
make_attachment(attachment)
# Attach metadata node
elem.append(common.MetadataTemplate())
volume_nsmap = {None: xmlutil.XMLNS_VOLUME_V2, 'atom': xmlutil.XMLNS_ATOM}
class VolumeTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('volume', selector='volume')
make_volume(root)
return xmlutil.MasterTemplate(root, 1, nsmap=volume_nsmap)
class VolumesTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('volumes')
elem = xmlutil.SubTemplateElement(root, 'volume', selector='volumes')
make_volume(elem)
return xmlutil.MasterTemplate(root, 1, nsmap=volume_nsmap)
class CommonDeserializer(wsgi.MetadataXMLDeserializer):
"""Common deserializer to handle xml-formatted volume requests.
Handles standard volume attributes as well as the optional metadata
attribute
"""
metadata_deserializer = common.MetadataXMLDeserializer()
def _extract_volume(self, node):
"""Marshal the volume attribute of a parsed request."""
volume = {}
volume_node = self.find_first_child_named(node, 'volume')
attributes = ['name', 'description', 'size',
'volume_type', 'availability_zone']
for attr in attributes:
if volume_node.getAttribute(attr):
volume[attr] = volume_node.getAttribute(attr)
metadata_node = self.find_first_child_named(volume_node, 'metadata')
if metadata_node is not None:
volume['metadata'] = self.extract_metadata(metadata_node)
return volume
class CreateDeserializer(CommonDeserializer):
"""Deserializer to handle xml-formatted create volume requests.
Handles standard volume attributes as well as the optional metadata
attribute
"""
def default(self, string):
"""Deserialize an xml-formatted volume create request."""
dom = utils.safe_minidom_parse_string(string)
volume = self._extract_volume(dom)
return {'body': {'volume': volume}}
class VolumeController(wsgi.Controller):
"""The Volumes API controller for the OpenStack API."""
_view_builder_class = volume_views.ViewBuilder
def __init__(self, ext_mgr):
self.volume_api = volume.API()
self.ext_mgr = ext_mgr
super(VolumeController, self).__init__()
@wsgi.serializers(xml=VolumeTemplate)
def show(self, req, id):
"""Return data about the given volume."""
context = req.environ['cinder.context']
try:
vol = self.volume_api.get(context, id)
except exception.NotFound:
raise exc.HTTPNotFound()
return self._view_builder.detail(req, vol)
def delete(self, req, id):
"""Delete a volume."""
context = req.environ['cinder.context']
LOG.audit(_("Delete volume with id: %s"), id, context=context)
try:
volume = self.volume_api.get(context, id)
self.volume_api.delete(context, volume)
except exception.NotFound:
raise exc.HTTPNotFound()
return webob.Response(status_int=202)
@wsgi.serializers(xml=VolumesTemplate)
def index(self, req):
"""Returns a summary list of volumes."""
return self._get_volumes(req, is_detail=False)
@wsgi.serializers(xml=VolumesTemplate)
def detail(self, req):
"""Returns a detailed list of volumes."""
return self._get_volumes(req, is_detail=True)
def _get_volumes(self, req, is_detail):
"""Returns a list of volumes, transformed through view builder."""
context = req.environ['cinder.context']
params = req.params.copy()
marker = params.pop('marker', None)
limit = params.pop('limit', None)
sort_key = params.pop('sort_key', 'created_at')
sort_dir = params.pop('sort_dir', 'desc')
params.pop('offset', None)
filters = params
remove_invalid_options(context,
filters, self._get_volume_filter_options())
# NOTE(thingee): v2 API allows name instead of display_name
if 'name' in filters:
filters['display_name'] = filters['name']
del filters['name']
volumes = self.volume_api.get_all(context, marker, limit, sort_key,
sort_dir, filters)
limited_list = common.limited(volumes, req)
if is_detail:
volumes = self._view_builder.detail_list(req, limited_list)
else:
volumes = self._view_builder.summary_list(req, limited_list)
return volumes
def _image_uuid_from_href(self, image_href):
# If the image href was generated by nova api, strip image_href
# down to an id.
try:
image_uuid = image_href.split('/').pop()
except (TypeError, AttributeError):
msg = _("Invalid imageRef provided.")
raise exc.HTTPBadRequest(explanation=msg)
if not uuidutils.is_uuid_like(image_uuid):
msg = _("Invalid imageRef provided.")
raise exc.HTTPBadRequest(explanation=msg)
return image_uuid
@wsgi.response(202)
@wsgi.serializers(xml=VolumeTemplate)
@wsgi.deserializers(xml=CreateDeserializer)
def create(self, req, body):
"""Creates a new volume."""
if not self.is_valid_body(body, 'volume'):
raise exc.HTTPBadRequest()
context = req.environ['cinder.context']
volume = body['volume']
kwargs = {}
# NOTE(thingee): v2 API allows name instead of display_name
if volume.get('name'):
volume['display_name'] = volume.get('name')
del volume['name']
# NOTE(thingee): v2 API allows description instead of description
if volume.get('description'):
volume['display_description'] = volume.get('description')
del volume['description']
req_volume_type = volume.get('volume_type', None)
if req_volume_type:
try:
kwargs['volume_type'] = volume_types.get_volume_type(
context, req_volume_type)
except exception.VolumeTypeNotFound:
explanation = 'Volume type not found.'
raise exc.HTTPNotFound(explanation=explanation)
kwargs['metadata'] = volume.get('metadata', None)
snapshot_id = volume.get('snapshot_id')
if snapshot_id is not None:
kwargs['snapshot'] = self.volume_api.get_snapshot(context,
snapshot_id)
else:
kwargs['snapshot'] = None
source_volid = volume.get('source_volid')
if source_volid is not None:
kwargs['source_volume'] = self.volume_api.get_volume(context,
source_volid)
else:
kwargs['source_volume'] = None
size = volume.get('size', None)
if size is None and kwargs['snapshot'] is not None:
size = kwargs['snapshot']['volume_size']
elif size is None and kwargs['source_volume'] is not None:
size = kwargs['source_volume']['size']
LOG.audit(_("Create volume of %s GB"), size, context=context)
image_href = None
image_uuid = None
if self.ext_mgr.is_loaded('os-image-create'):
image_href = volume.get('imageRef')
if image_href:
image_uuid = self._image_uuid_from_href(image_href)
kwargs['image_id'] = image_uuid
kwargs['availability_zone'] = volume.get('availability_zone', None)
new_volume = self.volume_api.create(context,
size,
volume.get('display_name'),
volume.get('display_description'),
**kwargs)
# TODO(vish): Instance should be None at db layer instead of
# trying to lazy load, but for now we turn it into
# a dict to avoid an error.
retval = self._view_builder.summary(req, dict(new_volume.iteritems()))
return retval
def _get_volume_filter_options(self):
"""Return volume search options allowed by non-admin."""
return ('name', 'status')
@wsgi.serializers(xml=VolumeTemplate)
def update(self, req, id, body):
"""Update a volume."""
context = req.environ['cinder.context']
if not body:
raise exc.HTTPBadRequest()
if 'volume' not in body:
raise exc.HTTPBadRequest()
volume = body['volume']
update_dict = {}
valid_update_keys = (
'name',
'description',
'metadata',
)
for key in valid_update_keys:
if key in volume:
update_dict[key] = volume[key]
# NOTE(thingee): v2 API allows name instead of display_name
if 'name' in update_dict:
update_dict['display_name'] = update_dict['name']
del update_dict['name']
# NOTE(thingee): v2 API allows name instead of display_name
if 'description' in update_dict:
update_dict['display_description'] = update_dict['description']
del update_dict['description']
try:
volume = self.volume_api.get(context, id)
self.volume_api.update(context, volume, update_dict)
except exception.NotFound:
raise exc.HTTPNotFound()
volume.update(update_dict)
return self._view_builder.detail(req, volume)
def create_resource(ext_mgr):
return wsgi.Resource(VolumeController(ext_mgr))
def remove_invalid_options(context, filters, allowed_search_options):
"""Remove search options that are not valid for non-admin API/context."""
if context.is_admin:
# Allow all options
return
# Otherwise, strip out all unknown options
unknown_options = [opt for opt in filters
if opt not in allowed_search_options]
bad_options = ", ".join(unknown_options)
log_msg = _("Removing options '%s' from query") % bad_options
LOG.debug(log_msg)
for opt in unknown_options:
del filters[opt]

282
cinder/api/versions.py Normal file
View File

@ -0,0 +1,282 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2010 OpenStack LLC.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import datetime
from lxml import etree
from cinder.api.openstack import wsgi
from cinder.api.views import versions as views_versions
from cinder.api import xmlutil
from cinder import flags
FLAGS = flags.FLAGS
_KNOWN_VERSIONS = {
"v2.0": {
"id": "v2.0",
"status": "CURRENT",
"updated": "2012-11-21T11:33:21Z",
"links": [
{
"rel": "describedby",
"type": "application/pdf",
"href": "http://jorgew.github.com/block-storage-api/"
"content/os-block-storage-1.0.pdf",
},
{
"rel": "describedby",
"type": "application/vnd.sun.wadl+xml",
#(anthony) FIXME
"href": "http://docs.rackspacecloud.com/"
"servers/api/v1.1/application.wadl",
},
],
"media-types": [
{
"base": "application/xml",
"type": "application/vnd.openstack.volume+xml;version=1",
},
{
"base": "application/json",
"type": "application/vnd.openstack.volume+json;version=1",
}
],
},
"v1.0": {
"id": "v1.0",
"status": "CURRENT",
"updated": "2012-01-04T11:33:21Z",
"links": [
{
"rel": "describedby",
"type": "application/pdf",
"href": "http://jorgew.github.com/block-storage-api/"
"content/os-block-storage-1.0.pdf",
},
{
"rel": "describedby",
"type": "application/vnd.sun.wadl+xml",
#(anthony) FIXME
"href": "http://docs.rackspacecloud.com/"
"servers/api/v1.1/application.wadl",
},
],
"media-types": [
{
"base": "application/xml",
"type": "application/vnd.openstack.volume+xml;version=1",
},
{
"base": "application/json",
"type": "application/vnd.openstack.volume+json;version=1",
}
],
}
}
def get_supported_versions():
versions = {}
if FLAGS.enable_v1_api:
versions['v1.0'] = _KNOWN_VERSIONS['v1.0']
if FLAGS.enable_v2_api:
versions['v2.0'] = _KNOWN_VERSIONS['v2.0']
return versions
class MediaTypesTemplateElement(xmlutil.TemplateElement):
def will_render(self, datum):
return 'media-types' in datum
def make_version(elem):
elem.set('id')
elem.set('status')
elem.set('updated')
mts = MediaTypesTemplateElement('media-types')
elem.append(mts)
mt = xmlutil.SubTemplateElement(mts, 'media-type', selector='media-types')
mt.set('base')
mt.set('type')
xmlutil.make_links(elem, 'links')
version_nsmap = {None: xmlutil.XMLNS_COMMON_V10, 'atom': xmlutil.XMLNS_ATOM}
class VersionTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('version', selector='version')
make_version(root)
return xmlutil.MasterTemplate(root, 1, nsmap=version_nsmap)
class VersionsTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('versions')
elem = xmlutil.SubTemplateElement(root, 'version', selector='versions')
make_version(elem)
return xmlutil.MasterTemplate(root, 1, nsmap=version_nsmap)
class ChoicesTemplate(xmlutil.TemplateBuilder):
def construct(self):
root = xmlutil.TemplateElement('choices')
elem = xmlutil.SubTemplateElement(root, 'version', selector='choices')
make_version(elem)
return xmlutil.MasterTemplate(root, 1, nsmap=version_nsmap)
class AtomSerializer(wsgi.XMLDictSerializer):
NSMAP = {None: xmlutil.XMLNS_ATOM}
def __init__(self, metadata=None, xmlns=None):
self.metadata = metadata or {}
if not xmlns:
self.xmlns = wsgi.XMLNS_ATOM
else:
self.xmlns = xmlns
def _get_most_recent_update(self, versions):
recent = None
for version in versions:
updated = datetime.datetime.strptime(version['updated'],
'%Y-%m-%dT%H:%M:%SZ')
if not recent:
recent = updated
elif updated > recent:
recent = updated
return recent.strftime('%Y-%m-%dT%H:%M:%SZ')
def _get_base_url(self, link_href):
# Make sure no trailing /
link_href = link_href.rstrip('/')
return link_href.rsplit('/', 1)[0] + '/'
def _create_feed(self, versions, feed_title, feed_id):
feed = etree.Element('feed', nsmap=self.NSMAP)
title = etree.SubElement(feed, 'title')
title.set('type', 'text')
title.text = feed_title
# Set this updated to the most recently updated version
recent = self._get_most_recent_update(versions)
etree.SubElement(feed, 'updated').text = recent
etree.SubElement(feed, 'id').text = feed_id
link = etree.SubElement(feed, 'link')
link.set('rel', 'self')
link.set('href', feed_id)
author = etree.SubElement(feed, 'author')
etree.SubElement(author, 'name').text = 'Rackspace'
etree.SubElement(author, 'uri').text = 'http://www.rackspace.com/'
for version in versions:
feed.append(self._create_version_entry(version))
return feed
def _create_version_entry(self, version):
entry = etree.Element('entry')
etree.SubElement(entry, 'id').text = version['links'][0]['href']
title = etree.SubElement(entry, 'title')
title.set('type', 'text')
title.text = 'Version %s' % version['id']
etree.SubElement(entry, 'updated').text = version['updated']
for link in version['links']:
link_elem = etree.SubElement(entry, 'link')
link_elem.set('rel', link['rel'])
link_elem.set('href', link['href'])
if 'type' in link:
link_elem.set('type', link['type'])
content = etree.SubElement(entry, 'content')
content.set('type', 'text')
content.text = 'Version %s %s (%s)' % (version['id'],
version['status'],
version['updated'])
return entry
class VersionsAtomSerializer(AtomSerializer):
def default(self, data):
versions = data['versions']
feed_id = self._get_base_url(versions[0]['links'][0]['href'])
feed = self._create_feed(versions, 'Available API Versions', feed_id)
return self._to_xml(feed)
class VersionAtomSerializer(AtomSerializer):
def default(self, data):
version = data['version']
feed_id = version['links'][0]['href']
feed = self._create_feed([version], 'About This Version', feed_id)
return self._to_xml(feed)
class Versions(wsgi.Resource):
def __init__(self):
super(Versions, self).__init__(None)
@wsgi.serializers(xml=VersionsTemplate,
atom=VersionsAtomSerializer)
def index(self, req):
"""Return all versions."""
builder = views_versions.get_view_builder(req)
return builder.build_versions(get_supported_versions())
@wsgi.serializers(xml=ChoicesTemplate)
@wsgi.response(300)
def multi(self, req):
"""Return multiple choices."""
builder = views_versions.get_view_builder(req)
return builder.build_choices(get_supported_versions(), req)
def get_action_args(self, request_environment):
"""Parse dictionary created by routes library."""
args = {}
if request_environment['PATH_INFO'] == '/':
args['action'] = 'index'
else:
args['action'] = 'multi'
return args
class VolumeVersionV1(object):
@wsgi.serializers(xml=VersionTemplate,
atom=VersionAtomSerializer)
def show(self, req):
builder = views_versions.get_view_builder(req)
return builder.build_version(_KNOWN_VERSIONS['v1.0'])
def create_resource():
return wsgi.Resource(VolumeVersionV1())

View File

@ -0,0 +1,16 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2011 OpenStack LLC.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.

View File

@ -0,0 +1,90 @@
# Copyright (C) 2012 Hewlett-Packard Development Company, L.P.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from cinder.api import common
from cinder.openstack.common import log as logging
LOG = logging.getLogger(__name__)
class ViewBuilder(common.ViewBuilder):
"""Model backup API responses as a python dictionary."""
_collection_name = "backups"
def __init__(self):
"""Initialize view builder."""
super(ViewBuilder, self).__init__()
def summary_list(self, request, backups):
"""Show a list of backups without many details."""
return self._list_view(self.summary, request, backups)
def detail_list(self, request, backups):
"""Detailed view of a list of backups ."""
return self._list_view(self.detail, request, backups)
def summary(self, request, backup):
"""Generic, non-detailed view of a backup."""
return {
'backup': {
'id': backup['id'],
'name': backup['display_name'],
'links': self._get_links(request,
backup['id']),
},
}
def restore_summary(self, request, restore):
"""Generic, non-detailed view of a restore."""
return {
'restore': {
'backup_id': restore['backup_id'],
'volume_id': restore['volume_id'],
},
}
def detail(self, request, backup):
"""Detailed view of a single backup."""
return {
'backup': {
'id': backup.get('id'),
'status': backup.get('status'),
'size': backup.get('size'),
'object_count': backup.get('object_count'),
'availability_zone': backup.get('availability_zone'),
'container': backup.get('container'),
'created_at': backup.get('created_at'),
'name': backup.get('display_name'),
'description': backup.get('display_description'),
'fail_reason': backup.get('fail_reason'),
'volume_id': backup.get('volume_id'),
'links': self._get_links(request, backup['id'])
}
}
def _list_view(self, func, request, backups):
"""Provide a view for a list of backups."""
backups_list = [func(request, backup)['backup'] for backup in backups]
backups_links = self._get_collection_links(request,
backups,
self._collection_name)
backups_dict = dict(backups=backups_list)
if backups_links:
backups_dict['backups_links'] = backups_links
return backups_dict

100
cinder/api/views/limits.py Normal file
View File

@ -0,0 +1,100 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2010-2011 OpenStack LLC.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import datetime
from cinder.openstack.common import timeutils
class ViewBuilder(object):
"""OpenStack API base limits view builder."""
def build(self, rate_limits, absolute_limits):
rate_limits = self._build_rate_limits(rate_limits)
absolute_limits = self._build_absolute_limits(absolute_limits)
output = {
"limits": {
"rate": rate_limits,
"absolute": absolute_limits,
},
}
return output
def _build_absolute_limits(self, absolute_limits):
"""Builder for absolute limits
absolute_limits should be given as a dict of limits.
For example: {"ram": 512, "gigabytes": 1024}.
"""
limit_names = {
"ram": ["maxTotalRAMSize"],
"instances": ["maxTotalInstances"],
"cores": ["maxTotalCores"],
"gigabytes": ["maxTotalVolumeGigabytes"],
"volumes": ["maxTotalVolumes"],
"key_pairs": ["maxTotalKeypairs"],
"floating_ips": ["maxTotalFloatingIps"],
"metadata_items": ["maxServerMeta", "maxImageMeta"],
"injected_files": ["maxPersonality"],
"injected_file_content_bytes": ["maxPersonalitySize"],
}
limits = {}
for name, value in absolute_limits.iteritems():
if name in limit_names and value is not None:
for name in limit_names[name]:
limits[name] = value
return limits
def _build_rate_limits(self, rate_limits):
limits = []
for rate_limit in rate_limits:
_rate_limit_key = None
_rate_limit = self._build_rate_limit(rate_limit)
# check for existing key
for limit in limits:
if (limit["uri"] == rate_limit["URI"] and
limit["regex"] == rate_limit["regex"]):
_rate_limit_key = limit
break
# ensure we have a key if we didn't find one
if not _rate_limit_key:
_rate_limit_key = {
"uri": rate_limit["URI"],
"regex": rate_limit["regex"],
"limit": [],
}
limits.append(_rate_limit_key)
_rate_limit_key["limit"].append(_rate_limit)
return limits
def _build_rate_limit(self, rate_limit):
_get_utc = datetime.datetime.utcfromtimestamp
next_avail = _get_utc(rate_limit["resetTime"])
return {
"verb": rate_limit["verb"],
"value": rate_limit["value"],
"remaining": int(rate_limit["remaining"]),
"unit": rate_limit["unit"],
"next-available": timeutils.isotime(at=next_avail),
}

View File

@ -0,0 +1,74 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2013 NetApp
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from cinder.api import common
from cinder.openstack.common import log as logging
class ViewBuilder(common.ViewBuilder):
"""Model a server API response as a python dictionary."""
_collection_name = 'share-snapshots'
def summary_list(self, request, snapshots):
"""Show a list of share snapshots without many details."""
return self._list_view(self.summary, request, snapshots)
def detail_list(self, request, snapshots):
"""Detailed view of a list of share snapshots."""
return self._list_view(self.detail, request, snapshots)
def summary(self, request, snapshot):
"""Generic, non-detailed view of an share snapshot."""
return {
'share-snapshot': {
'id': snapshot.get('id'),
'name': snapshot.get('display_name'),
'links': self._get_links(request, snapshot['id'])
}
}
def detail(self, request, snapshot):
"""Detailed view of a single share snapshot."""
return {
'share-snapshot': {
'id': snapshot.get('id'),
'share_id': snapshot.get('share_id'),
'share_size': snapshot.get('share_size'),
'created_at': snapshot.get('created_at'),
'status': snapshot.get('status'),
'name': snapshot.get('display_name'),
'description': snapshot.get('display_description'),
'share_proto': snapshot.get('share_proto'),
'export_location': snapshot.get('export_location'),
'links': self._get_links(request, snapshot['id'])
}
}
def _list_view(self, func, request, snapshots):
"""Provide a view for a list of share snapshots."""
snapshots_list = [func(request, snapshot)['share-snapshot']
for snapshot in snapshots]
snapshots_links = self._get_collection_links(request,
snapshots,
self._collection_name)
snapshots_dict = {self._collection_name: snapshots_list}
if snapshots_links:
snapshots_dict['share_snapshots_links'] = snapshots_links
return snapshots_dict

View File

@ -0,0 +1,74 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2013 OpenStack LLC.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from cinder.api import common
from cinder.openstack.common import log as logging
class ViewBuilder(common.ViewBuilder):
"""Model a server API response as a python dictionary."""
_collection_name = 'shares'
def summary_list(self, request, shares):
"""Show a list of shares without many details."""
return self._list_view(self.summary, request, shares)
def detail_list(self, request, shares):
"""Detailed view of a list of shares."""
return self._list_view(self.detail, request, shares)
def summary(self, request, share):
"""Generic, non-detailed view of an share."""
return {
'share': {
'id': share.get('id'),
'name': share.get('display_name'),
'links': self._get_links(request, share['id'])
}
}
def detail(self, request, share):
"""Detailed view of a single share."""
return {
'share': {
'id': share.get('id'),
'size': share.get('size'),
'availability_zone': share.get('availability_zone'),
'created_at': share.get('created_at'),
'status': share.get('status'),
'name': share.get('display_name'),
'description': share.get('display_description'),
'snapshot_id': share.get('snapshot_id'),
'share_proto': share.get('share_proto'),
'export_location': share.get('export_location'),
'links': self._get_links(request, share['id'])
}
}
def _list_view(self, func, request, shares):
"""Provide a view for a list of shares."""
shares_list = [func(request, share)['share'] for share in shares]
shares_links = self._get_collection_links(request,
shares,
self._collection_name)
shares_dict = dict(shares=shares_list)
if shares_links:
shares_dict['shares_links'] = shares_links
return shares_dict

34
cinder/api/views/types.py Normal file
View File

@ -0,0 +1,34 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2012 Red Hat, Inc.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from cinder.api import common
class ViewBuilder(common.ViewBuilder):
def show(self, request, volume_type, brief=False):
"""Trim away extraneous volume type attributes."""
trimmed = dict(id=volume_type.get('id'),
name=volume_type.get('name'),
extra_specs=volume_type.get('extra_specs'))
return trimmed if brief else dict(volume_type=trimmed)
def index(self, request, volume_types):
"""Index over trimmed volume types"""
volume_types_list = [self.show(request, volume_type, True)
for volume_type in volume_types]
return dict(volume_types=volume_types_list)

View File

@ -0,0 +1,82 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2010-2011 OpenStack LLC.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import copy
import os
def get_view_builder(req):
base_url = req.application_url
return ViewBuilder(base_url)
class ViewBuilder(object):
def __init__(self, base_url):
"""
:param base_url: url of the root wsgi application
"""
self.base_url = base_url
def build_choices(self, VERSIONS, req):
version_objs = []
for version in VERSIONS:
version = VERSIONS[version]
version_objs.append({
"id": version['id'],
"status": version['status'],
"links": [{"rel": "self",
"href": self.generate_href(req.path), }, ],
"media-types": version['media-types'], })
return dict(choices=version_objs)
def build_versions(self, versions):
version_objs = []
for version in sorted(versions.keys()):
version = versions[version]
version_objs.append({
"id": version['id'],
"status": version['status'],
"updated": version['updated'],
"links": self._build_links(version), })
return dict(versions=version_objs)
def build_version(self, version):
reval = copy.deepcopy(version)
reval['links'].insert(0, {
"rel": "self",
"href": self.base_url.rstrip('/') + '/', })
return dict(version=reval)
def _build_links(self, version_data):
"""Generate a container of links that refer to the provided version."""
href = self.generate_href()
links = [{'rel': 'self',
'href': href, }, ]
return links
def generate_href(self, path=None):
"""Create an url that refers to a specific version_number."""
version_number = 'v1'
if path:
path = path.strip('/')
return os.path.join(self.base_url, version_number, path)
else:
return os.path.join(self.base_url, version_number) + '/'

911
cinder/api/xmlutil.py Normal file
View File

@ -0,0 +1,911 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2011 OpenStack LLC.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import os.path
from lxml import etree
from cinder import utils
XMLNS_V10 = 'http://docs.rackspacecloud.com/servers/api/v1.0'
XMLNS_V11 = 'http://docs.openstack.org/compute/api/v1.1'
XMLNS_COMMON_V10 = 'http://docs.openstack.org/common/api/v1.0'
XMLNS_ATOM = 'http://www.w3.org/2005/Atom'
XMLNS_VOLUME_V1 = 'http://docs.openstack.org/volume/api/v1'
XMLNS_VOLUME_V2 = ('http://docs.openstack.org/api/openstack-volume/2.0/'
'content')
XMLNS_SHARE_V1 = ''
def validate_schema(xml, schema_name):
if isinstance(xml, str):
xml = etree.fromstring(xml)
base_path = 'cinder/api/schemas/v1.1/'
if schema_name in ('atom', 'atom-link'):
base_path = 'cinder/api/schemas/'
schema_path = os.path.join(utils.cinderdir(),
'%s%s.rng' % (base_path, schema_name))
schema_doc = etree.parse(schema_path)
relaxng = etree.RelaxNG(schema_doc)
relaxng.assertValid(xml)
class Selector(object):
"""Selects datum to operate on from an object."""
def __init__(self, *chain):
"""Initialize the selector.
Each argument is a subsequent index into the object.
"""
self.chain = chain
def __repr__(self):
"""Return a representation of the selector."""
return "Selector" + repr(self.chain)
def __call__(self, obj, do_raise=False):
"""Select a datum to operate on.
Selects the relevant datum within the object.
:param obj: The object from which to select the object.
:param do_raise: If False (the default), return None if the
indexed datum does not exist. Otherwise,
raise a KeyError.
"""
# Walk the selector list
for elem in self.chain:
# If it's callable, call it
if callable(elem):
obj = elem(obj)
else:
# Use indexing
try:
obj = obj[elem]
except (KeyError, IndexError):
# No sense going any further
if do_raise:
# Convert to a KeyError, for consistency
raise KeyError(elem)
return None
# Return the finally-selected object
return obj
def get_items(obj):
"""Get items in obj."""
return list(obj.items())
class EmptyStringSelector(Selector):
"""Returns the empty string if Selector would return None."""
def __call__(self, obj, do_raise=False):
"""Returns empty string if the selected value does not exist."""
try:
return super(EmptyStringSelector, self).__call__(obj, True)
except KeyError:
return ""
class ConstantSelector(object):
"""Returns a constant."""
def __init__(self, value):
"""Initialize the selector.
:param value: The value to return.
"""
self.value = value
def __repr__(self):
"""Return a representation of the selector."""
return repr(self.value)
def __call__(self, _obj, _do_raise=False):
"""Select a datum to operate on.
Returns a constant value. Compatible with
Selector.__call__().
"""
return self.value
class TemplateElement(object):
"""Represent an element in the template."""
def __init__(self, tag, attrib=None, selector=None, subselector=None,
**extra):
"""Initialize an element.
Initializes an element in the template. Keyword arguments
specify attributes to be set on the element; values must be
callables. See TemplateElement.set() for more information.
:param tag: The name of the tag to create.
:param attrib: An optional dictionary of element attributes.
:param selector: An optional callable taking an object and
optional boolean do_raise indicator and
returning the object bound to the element.
:param subselector: An optional callable taking an object and
optional boolean do_raise indicator and
returning the object bound to the element.
This is used to further refine the datum
object returned by selector in the event
that it is a list of objects.
"""
# Convert selector into a Selector
if selector is None:
selector = Selector()
elif not callable(selector):
selector = Selector(selector)
# Convert subselector into a Selector
if subselector is not None and not callable(subselector):
subselector = Selector(subselector)
self.tag = tag
self.selector = selector
self.subselector = subselector
self.attrib = {}
self._text = None
self._children = []
self._childmap = {}
# Run the incoming attributes through set() so that they
# become selectorized
if not attrib:
attrib = {}
attrib.update(extra)
for k, v in attrib.items():
self.set(k, v)
def __repr__(self):
"""Return a representation of the template element."""
return ('<%s.%s %r at %#x>' %
(self.__class__.__module__, self.__class__.__name__,
self.tag, id(self)))
def __len__(self):
"""Return the number of child elements."""
return len(self._children)
def __contains__(self, key):
"""Determine whether a child node named by key exists."""
return key in self._childmap
def __getitem__(self, idx):
"""Retrieve a child node by index or name."""
if isinstance(idx, basestring):
# Allow access by node name
return self._childmap[idx]
else:
return self._children[idx]
def append(self, elem):
"""Append a child to the element."""
# Unwrap templates...
elem = elem.unwrap()
# Avoid duplications
if elem.tag in self._childmap:
raise KeyError(elem.tag)
self._children.append(elem)
self._childmap[elem.tag] = elem
def extend(self, elems):
"""Append children to the element."""
# Pre-evaluate the elements
elemmap = {}
elemlist = []
for elem in elems:
# Unwrap templates...
elem = elem.unwrap()
# Avoid duplications
if elem.tag in self._childmap or elem.tag in elemmap:
raise KeyError(elem.tag)
elemmap[elem.tag] = elem
elemlist.append(elem)
# Update the children
self._children.extend(elemlist)
self._childmap.update(elemmap)
def insert(self, idx, elem):
"""Insert a child element at the given index."""
# Unwrap templates...
elem = elem.unwrap()
# Avoid duplications
if elem.tag in self._childmap:
raise KeyError(elem.tag)
self._children.insert(idx, elem)
self._childmap[elem.tag] = elem
def remove(self, elem):
"""Remove a child element."""
# Unwrap templates...
elem = elem.unwrap()
# Check if element exists
if elem.tag not in self._childmap or self._childmap[elem.tag] != elem:
raise ValueError(_('element is not a child'))
self._children.remove(elem)
del self._childmap[elem.tag]
def get(self, key):
"""Get an attribute.
Returns a callable which performs datum selection.
:param key: The name of the attribute to get.
"""
return self.attrib[key]
def set(self, key, value=None):
"""Set an attribute.
:param key: The name of the attribute to set.
:param value: A callable taking an object and optional boolean
do_raise indicator and returning the datum bound
to the attribute. If None, a Selector() will be
constructed from the key. If a string, a
Selector() will be constructed from the string.
"""
# Convert value to a selector
if value is None:
value = Selector(key)
elif not callable(value):
value = Selector(value)
self.attrib[key] = value
def keys(self):
"""Return the attribute names."""
return self.attrib.keys()
def items(self):
"""Return the attribute names and values."""
return self.attrib.items()
def unwrap(self):
"""Unwraps a template to return a template element."""
# We are a template element
return self
def wrap(self):
"""Wraps a template element to return a template."""
# Wrap in a basic Template
return Template(self)
def apply(self, elem, obj):
"""Apply text and attributes to an etree.Element.
Applies the text and attribute instructions in the template
element to an etree.Element instance.
:param elem: An etree.Element instance.
:param obj: The base object associated with this template
element.
"""
# Start with the text...
if self.text is not None:
elem.text = unicode(self.text(obj))
# Now set up all the attributes...
for key, value in self.attrib.items():
try:
elem.set(key, unicode(value(obj, True)))
except KeyError:
# Attribute has no value, so don't include it
pass
def _render(self, parent, datum, patches, nsmap):
"""Internal rendering.
Renders the template node into an etree.Element object.
Returns the etree.Element object.
:param parent: The parent etree.Element instance.
:param datum: The datum associated with this template element.
:param patches: A list of other template elements that must
also be applied.
:param nsmap: An optional namespace dictionary to be
associated with the etree.Element instance.
"""
# Allocate a node
if callable(self.tag):
tagname = self.tag(datum)
else:
tagname = self.tag
elem = etree.Element(tagname, nsmap=nsmap)
# If we have a parent, append the node to the parent
if parent is not None:
parent.append(elem)
# If the datum is None, do nothing else
if datum is None:
return elem
# Apply this template element to the element
self.apply(elem, datum)
# Additionally, apply the patches
for patch in patches:
patch.apply(elem, datum)
# We have fully rendered the element; return it
return elem
def render(self, parent, obj, patches=[], nsmap=None):
"""Render an object.
Renders an object against this template node. Returns a list
of two-item tuples, where the first item is an etree.Element
instance and the second item is the datum associated with that
instance.
:param parent: The parent for the etree.Element instances.
:param obj: The object to render this template element
against.
:param patches: A list of other template elements to apply
when rendering this template element.
:param nsmap: An optional namespace dictionary to attach to
the etree.Element instances.
"""
# First, get the datum we're rendering
data = None if obj is None else self.selector(obj)
# Check if we should render at all
if not self.will_render(data):
return []
elif data is None:
return [(self._render(parent, None, patches, nsmap), None)]
# Make the data into a list if it isn't already
if not isinstance(data, list):
data = [data]
elif parent is None:
raise ValueError(_('root element selecting a list'))
# Render all the elements
elems = []
for datum in data:
if self.subselector is not None:
datum = self.subselector(datum)
elems.append((self._render(parent, datum, patches, nsmap), datum))
# Return all the elements rendered, as well as the
# corresponding datum for the next step down the tree
return elems
def will_render(self, datum):
"""Hook method.
An overridable hook method to determine whether this template
element will be rendered at all. By default, returns False
(inhibiting rendering) if the datum is None.
:param datum: The datum associated with this template element.
"""
# Don't render if datum is None
return datum is not None
def _text_get(self):
"""Template element text.
Either None or a callable taking an object and optional
boolean do_raise indicator and returning the datum bound to
the text of the template element.
"""
return self._text
def _text_set(self, value):
# Convert value to a selector
if value is not None and not callable(value):
value = Selector(value)
self._text = value
def _text_del(self):
self._text = None
text = property(_text_get, _text_set, _text_del)
def tree(self):
"""Return string representation of the template tree.
Returns a representation of the template rooted at this
element as a string, suitable for inclusion in debug logs.
"""
# Build the inner contents of the tag...
contents = [self.tag, '!selector=%r' % self.selector]
# Add the text...
if self.text is not None:
contents.append('!text=%r' % self.text)
# Add all the other attributes
for key, value in self.attrib.items():
contents.append('%s=%r' % (key, value))
# If there are no children, return it as a closed tag
if len(self) == 0:
return '<%s/>' % ' '.join([str(i) for i in contents])
# OK, recurse to our children
children = [c.tree() for c in self]
# Return the result
return ('<%s>%s</%s>' %
(' '.join(contents), ''.join(children), self.tag))
def SubTemplateElement(parent, tag, attrib=None, selector=None,
subselector=None, **extra):
"""Create a template element as a child of another.
Corresponds to the etree.SubElement interface. Parameters are as
for TemplateElement, with the addition of the parent.
"""
# Convert attributes
attrib = attrib or {}
attrib.update(extra)
# Get a TemplateElement
elem = TemplateElement(tag, attrib=attrib, selector=selector,
subselector=subselector)
# Append the parent safely
if parent is not None:
parent.append(elem)
return elem
class Template(object):
"""Represent a template."""
def __init__(self, root, nsmap=None):
"""Initialize a template.
:param root: The root element of the template.
:param nsmap: An optional namespace dictionary to be
associated with the root element of the
template.
"""
self.root = root.unwrap() if root is not None else None
self.nsmap = nsmap or {}
self.serialize_options = dict(encoding='UTF-8', xml_declaration=True)
def _serialize(self, parent, obj, siblings, nsmap=None):
"""Internal serialization.
Recursive routine to build a tree of etree.Element instances
from an object based on the template. Returns the first
etree.Element instance rendered, or None.
:param parent: The parent etree.Element instance. Can be
None.
:param obj: The object to render.
:param siblings: The TemplateElement instances against which
to render the object.
:param nsmap: An optional namespace dictionary to be
associated with the etree.Element instance
rendered.
"""
# First step, render the element
elems = siblings[0].render(parent, obj, siblings[1:], nsmap)
# Now, recurse to all child elements
seen = set()
for idx, sibling in enumerate(siblings):
for child in sibling:
# Have we handled this child already?
if child.tag in seen:
continue
seen.add(child.tag)
# Determine the child's siblings
nieces = [child]
for sib in siblings[idx + 1:]:
if child.tag in sib:
nieces.append(sib[child.tag])
# Now we recurse for every data element
for elem, datum in elems:
self._serialize(elem, datum, nieces)
# Return the first element; at the top level, this will be the
# root element
if elems:
return elems[0][0]
def serialize(self, obj, *args, **kwargs):
"""Serialize an object.
Serializes an object against the template. Returns a string
with the serialized XML. Positional and keyword arguments are
passed to etree.tostring().
:param obj: The object to serialize.
"""
elem = self.make_tree(obj)
if elem is None:
return ''
for k, v in self.serialize_options.items():
kwargs.setdefault(k, v)
# Serialize it into XML
return etree.tostring(elem, *args, **kwargs)
def make_tree(self, obj):
"""Create a tree.
Serializes an object against the template. Returns an Element
node with appropriate children.
:param obj: The object to serialize.
"""
# If the template is empty, return the empty string
if self.root is None:
return None
# Get the siblings and nsmap of the root element
siblings = self._siblings()
nsmap = self._nsmap()
# Form the element tree
return self._serialize(None, obj, siblings, nsmap)
def _siblings(self):
"""Hook method for computing root siblings.
An overridable hook method to return the siblings of the root
element. By default, this is the root element itself.
"""
return [self.root]
def _nsmap(self):
"""Hook method for computing the namespace dictionary.
An overridable hook method to return the namespace dictionary.
"""
return self.nsmap.copy()
def unwrap(self):
"""Unwraps a template to return a template element."""
# Return the root element
return self.root
def wrap(self):
"""Wraps a template element to return a template."""
# We are a template
return self
def apply(self, master):
"""Hook method for determining slave applicability.
An overridable hook method used to determine if this template
is applicable as a slave to a given master template.
:param master: The master template to test.
"""
return True
def tree(self):
"""Return string representation of the template tree.
Returns a representation of the template as a string, suitable
for inclusion in debug logs.
"""
return "%r: %s" % (self, self.root.tree())
class MasterTemplate(Template):
"""Represent a master template.
Master templates are versioned derivatives of templates that
additionally allow slave templates to be attached. Slave
templates allow modification of the serialized result without
directly changing the master.
"""
def __init__(self, root, version, nsmap=None):
"""Initialize a master template.
:param root: The root element of the template.
:param version: The version number of the template.
:param nsmap: An optional namespace dictionary to be
associated with the root element of the
template.
"""
super(MasterTemplate, self).__init__(root, nsmap)
self.version = version
self.slaves = []
def __repr__(self):
"""Return string representation of the template."""
return ("<%s.%s object version %s at %#x>" %
(self.__class__.__module__, self.__class__.__name__,
self.version, id(self)))
def _siblings(self):
"""Hook method for computing root siblings.
An overridable hook method to return the siblings of the root
element. This is the root element plus the root elements of
all the slave templates.
"""
return [self.root] + [slave.root for slave in self.slaves]
def _nsmap(self):
"""Hook method for computing the namespace dictionary.
An overridable hook method to return the namespace dictionary.
The namespace dictionary is computed by taking the master
template's namespace dictionary and updating it from all the
slave templates.
"""
nsmap = self.nsmap.copy()
for slave in self.slaves:
nsmap.update(slave._nsmap())
return nsmap
def attach(self, *slaves):
"""Attach one or more slave templates.
Attaches one or more slave templates to the master template.
Slave templates must have a root element with the same tag as
the master template. The slave template's apply() method will
be called to determine if the slave should be applied to this
master; if it returns False, that slave will be skipped.
(This allows filtering of slaves based on the version of the
master template.)
"""
slave_list = []
for slave in slaves:
slave = slave.wrap()
# Make sure we have a tree match
if slave.root.tag != self.root.tag:
slavetag = slave.root.tag
mastertag = self.root.tag
msg = _("Template tree mismatch; adding slave %(slavetag)s "
"to master %(mastertag)s") % locals()
raise ValueError(msg)
# Make sure slave applies to this template
if not slave.apply(self):
continue
slave_list.append(slave)
# Add the slaves
self.slaves.extend(slave_list)
def copy(self):
"""Return a copy of this master template."""
# Return a copy of the MasterTemplate
tmp = self.__class__(self.root, self.version, self.nsmap)
tmp.slaves = self.slaves[:]
return tmp
class SlaveTemplate(Template):
"""Represent a slave template.
Slave templates are versioned derivatives of templates. Each
slave has a minimum version and optional maximum version of the
master template to which they can be attached.
"""
def __init__(self, root, min_vers, max_vers=None, nsmap=None):
"""Initialize a slave template.
:param root: The root element of the template.
:param min_vers: The minimum permissible version of the master
template for this slave template to apply.
:param max_vers: An optional upper bound for the master
template version.
:param nsmap: An optional namespace dictionary to be
associated with the root element of the
template.
"""
super(SlaveTemplate, self).__init__(root, nsmap)
self.min_vers = min_vers
self.max_vers = max_vers
def __repr__(self):
"""Return string representation of the template."""
return ("<%s.%s object versions %s-%s at %#x>" %
(self.__class__.__module__, self.__class__.__name__,
self.min_vers, self.max_vers, id(self)))
def apply(self, master):
"""Hook method for determining slave applicability.
An overridable hook method used to determine if this template
is applicable as a slave to a given master template. This
version requires the master template to have a version number
between min_vers and max_vers.
:param master: The master template to test.
"""
# Does the master meet our minimum version requirement?
if master.version < self.min_vers:
return False
# How about our maximum version requirement?
if self.max_vers is not None and master.version > self.max_vers:
return False
return True
class TemplateBuilder(object):
"""Template builder.
This class exists to allow templates to be lazily built without
having to build them each time they are needed. It must be
subclassed, and the subclass must implement the construct()
method, which must return a Template (or subclass) instance. The
constructor will always return the template returned by
construct(), or, if it has a copy() method, a copy of that
template.
"""
_tmpl = None
def __new__(cls, copy=True):
"""Construct and return a template.
:param copy: If True (the default), a copy of the template
will be constructed and returned, if possible.
"""
# Do we need to construct the template?
if cls._tmpl is None:
tmp = super(TemplateBuilder, cls).__new__(cls)
# Construct the template
cls._tmpl = tmp.construct()
# If the template has a copy attribute, return the result of
# calling it
if copy and hasattr(cls._tmpl, 'copy'):
return cls._tmpl.copy()
# Return the template
return cls._tmpl
def construct(self):
"""Construct a template.
Called to construct a template instance, which it must return.
Only called once.
"""
raise NotImplementedError(_("subclasses must implement construct()!"))
def make_links(parent, selector=None):
"""
Attach an Atom <links> element to the parent.
"""
elem = SubTemplateElement(parent, '{%s}link' % XMLNS_ATOM,
selector=selector)
elem.set('rel')
elem.set('type')
elem.set('href')
# Just for completeness...
return elem
def make_flat_dict(name, selector=None, subselector=None, ns=None):
"""
Utility for simple XML templates that traditionally used
XMLDictSerializer with no metadata. Returns a template element
where the top-level element has the given tag name, and where
sub-elements have tag names derived from the object's keys and
text derived from the object's values. This only works for flat
dictionary objects, not dictionaries containing nested lists or
dictionaries.
"""
# Set up the names we need...
if ns is None:
elemname = name
tagname = Selector(0)
else:
elemname = '{%s}%s' % (ns, name)
tagname = lambda obj, do_raise=False: '{%s}%s' % (ns, obj[0])
if selector is None:
selector = name
# Build the root element
root = TemplateElement(elemname, selector=selector,
subselector=subselector)
# Build an element to represent all the keys and values
elem = SubTemplateElement(root, tagname, selector=get_items)
elem.text = 1
# Return the template
return root

23
cinder/backup/__init__.py Normal file
View File

@ -0,0 +1,23 @@
# Copyright (C) 2012 Hewlett-Packard Development Company, L.P.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
# Importing full names to not pollute the namespace and cause possible
# collisions with use of 'from cinder.backup import <foo>' elsewhere.
import cinder.flags
import cinder.openstack.common.importutils
API = cinder.openstack.common.importutils.import_class(
cinder.flags.FLAGS.backup_api_class)

171
cinder/backup/api.py Normal file
View File

@ -0,0 +1,171 @@
# Copyright (C) 2012 Hewlett-Packard Development Company, L.P.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""
Handles all requests relating to the volume backups service.
"""
from eventlet import greenthread
from cinder.backup import rpcapi as backup_rpcapi
from cinder.db import base
from cinder import exception
from cinder import flags
from cinder.openstack.common import log as logging
import cinder.volume
FLAGS = flags.FLAGS
LOG = logging.getLogger(__name__)
class API(base.Base):
"""API for interacting with the volume backup manager."""
def __init__(self, db_driver=None):
self.backup_rpcapi = backup_rpcapi.BackupAPI()
self.volume_api = cinder.volume.API()
super(API, self).__init__(db_driver)
def get(self, context, backup_id):
rv = self.db.backup_get(context, backup_id)
return dict(rv.iteritems())
def delete(self, context, backup_id):
"""
Make the RPC call to delete a volume backup.
"""
backup = self.get(context, backup_id)
if backup['status'] not in ['available', 'error']:
msg = _('Backup status must be available or error')
raise exception.InvalidBackup(reason=msg)
self.db.backup_update(context, backup_id, {'status': 'deleting'})
self.backup_rpcapi.delete_backup(context,
backup['host'],
backup['id'])
# TODO(moorehef): Add support for search_opts, discarded atm
def get_all(self, context, search_opts={}):
if context.is_admin:
backups = self.db.backup_get_all(context)
else:
backups = self.db.backup_get_all_by_project(context,
context.project_id)
return backups
def create(self, context, name, description, volume_id,
container, availability_zone=None):
"""
Make the RPC call to create a volume backup.
"""
volume = self.volume_api.get(context, volume_id)
if volume['status'] != "available":
msg = _('Volume to be backed up must be available')
raise exception.InvalidVolume(reason=msg)
self.db.volume_update(context, volume_id, {'status': 'backing-up'})
options = {'user_id': context.user_id,
'project_id': context.project_id,
'display_name': name,
'display_description': description,
'volume_id': volume_id,
'status': 'creating',
'container': container,
'size': volume['size'],
# TODO(DuncanT): This will need de-managling once
# multi-backend lands
'host': volume['host'], }
backup = self.db.backup_create(context, options)
#TODO(DuncanT): In future, when we have a generic local attach,
# this can go via the scheduler, which enables
# better load ballancing and isolation of services
self.backup_rpcapi.create_backup(context,
backup['host'],
backup['id'],
volume_id)
return backup
def restore(self, context, backup_id, volume_id=None):
"""
Make the RPC call to restore a volume backup.
"""
backup = self.get(context, backup_id)
if backup['status'] != 'available':
msg = _('Backup status must be available')
raise exception.InvalidBackup(reason=msg)
size = backup['size']
if size is None:
msg = _('Backup to be restored has invalid size')
raise exception.InvalidBackup(reason=msg)
# Create a volume if none specified. If a volume is specified check
# it is large enough for the backup
if volume_id is None:
name = 'restore_backup_%s' % backup_id
description = 'auto-created_from_restore_from_swift'
LOG.audit(_("Creating volume of %(size)s GB for restore of "
"backup %(backup_id)s"), locals(), context=context)
volume = self.volume_api.create(context, size, name, description)
volume_id = volume['id']
while True:
volume = self.volume_api.get(context, volume_id)
if volume['status'] != 'creating':
break
greenthread.sleep(1)
else:
volume = self.volume_api.get(context, volume_id)
volume_size = volume['size']
if volume_size < size:
err = _('volume size %(volume_size)d is too small to restore '
'backup of size %(size)d.') % locals()
raise exception.InvalidVolume(reason=err)
if volume['status'] != "available":
msg = _('Volume to be restored to must be available')
raise exception.InvalidVolume(reason=msg)
LOG.debug('Checking backup size %s against volume size %s',
size, volume['size'])
if size > volume['size']:
msg = _('Volume to be restored to is smaller '
'than the backup to be restored')
raise exception.InvalidVolume(reason=msg)
LOG.audit(_("Overwriting volume %(volume_id)s with restore of "
"backup %(backup_id)s"), locals(), context=context)
# Setting the status here rather than setting at start and unrolling
# for each error condition, it should be a very small window
self.db.backup_update(context, backup_id, {'status': 'restoring'})
self.db.volume_update(context, volume_id, {'status':
'restoring-backup'})
self.backup_rpcapi.restore_backup(context,
backup['host'],
backup['id'],
volume_id)
d = {'backup_id': backup_id,
'volume_id': volume_id, }
return d

264
cinder/backup/manager.py Executable file
View File

@ -0,0 +1,264 @@
# Copyright (C) 2012 Hewlett-Packard Development Company, L.P.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""
Backup manager manages volume backups.
Volume Backups are full copies of persistent volumes stored in Swift object
storage. They are usable without the original object being available. A
volume backup can be restored to the original volume it was created from or
any other available volume with a minimum size of the original volume.
Volume backups can be created, restored, deleted and listed.
**Related Flags**
:backup_topic: What :mod:`rpc` topic to listen to (default:
`cinder-backup`).
:backup_manager: The module name of a class derived from
:class:`manager.Manager` (default:
:class:`cinder.backup.manager.Manager`).
"""
from oslo.config import cfg
from cinder import context
from cinder import exception
from cinder import flags
from cinder import manager
from cinder.openstack.common import excutils
from cinder.openstack.common import importutils
from cinder.openstack.common import log as logging
LOG = logging.getLogger(__name__)
backup_manager_opts = [
cfg.StrOpt('backup_service',
default='cinder.backup.services.swift',
help='Service to use for backups.'),
]
FLAGS = flags.FLAGS
FLAGS.register_opts(backup_manager_opts)
class BackupManager(manager.SchedulerDependentManager):
"""Manages backup of block storage devices."""
RPC_API_VERSION = '1.0'
def __init__(self, service_name=None, *args, **kwargs):
self.service = importutils.import_module(FLAGS.backup_service)
self.az = FLAGS.storage_availability_zone
self.volume_manager = importutils.import_object(FLAGS.volume_manager)
self.driver = self.volume_manager.driver
super(BackupManager, self).__init__(service_name='backup',
*args, **kwargs)
self.driver.db = self.db
def init_host(self):
"""Do any initialization that needs to be run if this is a
standalone service."""
ctxt = context.get_admin_context()
self.driver.do_setup(ctxt)
self.driver.check_for_setup_error()
LOG.info(_("Cleaning up incomplete backup operations"))
volumes = self.db.volume_get_all_by_host(ctxt, self.host)
for volume in volumes:
if volume['status'] == 'backing-up':
LOG.info(_('Resetting volume %s to available '
'(was backing-up)') % volume['id'])
self.volume_manager.detach_volume(ctxt, volume['id'])
if volume['status'] == 'restoring-backup':
LOG.info(_('Resetting volume %s to error_restoring '
'(was restoring-backup)') % volume['id'])
self.volume_manager.detach_volume(ctxt, volume['id'])
self.db.volume_update(ctxt, volume['id'],
{'status': 'error_restoring'})
# TODO(smulcahy) implement full resume of backup and restore
# operations on restart (rather than simply resetting)
backups = self.db.backup_get_all_by_host(ctxt, self.host)
for backup in backups:
if backup['status'] == 'creating':
LOG.info(_('Resetting backup %s to error '
'(was creating)') % backup['id'])
err = 'incomplete backup reset on manager restart'
self.db.backup_update(ctxt, backup['id'], {'status': 'error',
'fail_reason': err})
if backup['status'] == 'restoring':
LOG.info(_('Resetting backup %s to available '
'(was restoring)') % backup['id'])
self.db.backup_update(ctxt, backup['id'],
{'status': 'available'})
if backup['status'] == 'deleting':
LOG.info(_('Resuming delete on backup: %s') % backup['id'])
self.delete_backup(ctxt, backup['id'])
def create_backup(self, context, backup_id):
"""
Create volume backups using configured backup service.
"""
backup = self.db.backup_get(context, backup_id)
volume_id = backup['volume_id']
volume = self.db.volume_get(context, volume_id)
LOG.info(_('create_backup started, backup: %(backup_id)s for '
'volume: %(volume_id)s') % locals())
self.db.backup_update(context, backup_id, {'host': self.host,
'service':
FLAGS.backup_service})
expected_status = 'backing-up'
actual_status = volume['status']
if actual_status != expected_status:
err = _('create_backup aborted, expected volume status '
'%(expected_status)s but got %(actual_status)s') % locals()
self.db.backup_update(context, backup_id, {'status': 'error',
'fail_reason': err})
raise exception.InvalidVolume(reason=err)
expected_status = 'creating'
actual_status = backup['status']
if actual_status != expected_status:
err = _('create_backup aborted, expected backup status '
'%(expected_status)s but got %(actual_status)s') % locals()
self.db.volume_update(context, volume_id, {'status': 'available'})
self.db.backup_update(context, backup_id, {'status': 'error',
'fail_reason': err})
raise exception.InvalidBackup(reason=err)
try:
backup_service = self.service.get_backup_service(context)
self.driver.backup_volume(context, backup, backup_service)
except Exception as err:
with excutils.save_and_reraise_exception():
self.db.volume_update(context, volume_id,
{'status': 'available'})
self.db.backup_update(context, backup_id,
{'status': 'error',
'fail_reason': unicode(err)})
self.db.volume_update(context, volume_id, {'status': 'available'})
self.db.backup_update(context, backup_id, {'status': 'available',
'size': volume['size'],
'availability_zone':
self.az})
LOG.info(_('create_backup finished. backup: %s'), backup_id)
def restore_backup(self, context, backup_id, volume_id):
"""
Restore volume backups from configured backup service.
"""
LOG.info(_('restore_backup started, restoring backup: %(backup_id)s'
' to volume: %(volume_id)s') % locals())
backup = self.db.backup_get(context, backup_id)
volume = self.db.volume_get(context, volume_id)
self.db.backup_update(context, backup_id, {'host': self.host})
expected_status = 'restoring-backup'
actual_status = volume['status']
if actual_status != expected_status:
err = _('restore_backup aborted, expected volume status '
'%(expected_status)s but got %(actual_status)s') % locals()
self.db.backup_update(context, backup_id, {'status': 'available'})
raise exception.InvalidVolume(reason=err)
expected_status = 'restoring'
actual_status = backup['status']
if actual_status != expected_status:
err = _('restore_backup aborted, expected backup status '
'%(expected_status)s but got %(actual_status)s') % locals()
self.db.backup_update(context, backup_id, {'status': 'error',
'fail_reason': err})
self.db.volume_update(context, volume_id, {'status': 'error'})
raise exception.InvalidBackup(reason=err)
if volume['size'] > backup['size']:
LOG.warn('volume: %s, size: %d is larger than backup: %s, '
'size: %d, continuing with restore',
volume['id'], volume['size'],
backup['id'], backup['size'])
backup_service = backup['service']
configured_service = FLAGS.backup_service
if backup_service != configured_service:
err = _('restore_backup aborted, the backup service currently'
' configured [%(configured_service)s] is not the'
' backup service that was used to create this'
' backup [%(backup_service)s]') % locals()
self.db.backup_update(context, backup_id, {'status': 'available'})
self.db.volume_update(context, volume_id, {'status': 'error'})
raise exception.InvalidBackup(reason=err)
try:
backup_service = self.service.get_backup_service(context)
self.driver.restore_backup(context, backup, volume,
backup_service)
except Exception as err:
with excutils.save_and_reraise_exception():
self.db.volume_update(context, volume_id,
{'status': 'error_restoring'})
self.db.backup_update(context, backup_id,
{'status': 'available'})
self.db.volume_update(context, volume_id, {'status': 'available'})
self.db.backup_update(context, backup_id, {'status': 'available'})
LOG.info(_('restore_backup finished, backup: %(backup_id)s restored'
' to volume: %(volume_id)s') % locals())
def delete_backup(self, context, backup_id):
"""
Delete volume backup from configured backup service.
"""
backup = self.db.backup_get(context, backup_id)
LOG.info(_('delete_backup started, backup: %s'), backup_id)
self.db.backup_update(context, backup_id, {'host': self.host})
expected_status = 'deleting'
actual_status = backup['status']
if actual_status != expected_status:
err = _('delete_backup aborted, expected backup status '
'%(expected_status)s but got %(actual_status)s') % locals()
self.db.backup_update(context, backup_id, {'status': 'error',
'fail_reason': err})
raise exception.InvalidBackup(reason=err)
backup_service = backup['service']
if backup_service is not None:
configured_service = FLAGS.backup_service
if backup_service != configured_service:
err = _('delete_backup aborted, the backup service currently'
' configured [%(configured_service)s] is not the'
' backup service that was used to create this'
' backup [%(backup_service)s]') % locals()
self.db.backup_update(context, backup_id,
{'status': 'error'})
raise exception.InvalidBackup(reason=err)
try:
backup_service = self.service.get_backup_service(context)
backup_service.delete(backup)
except Exception as err:
with excutils.save_and_reraise_exception():
self.db.backup_update(context, backup_id,
{'status': 'error',
'fail_reason':
unicode(err)})
context = context.elevated()
self.db.backup_destroy(context, backup_id)
LOG.info(_('delete_backup finished, backup %s deleted'), backup_id)

73
cinder/backup/rpcapi.py Normal file
View File

@ -0,0 +1,73 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright (C) 2012 Hewlett-Packard Development Company, L.P.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""
Client side of the volume backup RPC API.
"""
from cinder import flags
from cinder.openstack.common import log as logging
from cinder.openstack.common import rpc
import cinder.openstack.common.rpc.proxy
LOG = logging.getLogger(__name__)
FLAGS = flags.FLAGS
class BackupAPI(cinder.openstack.common.rpc.proxy.RpcProxy):
'''Client side of the volume rpc API.
API version history:
1.0 - Initial version.
'''
BASE_RPC_API_VERSION = '1.0'
def __init__(self):
super(BackupAPI, self).__init__(
topic=FLAGS.backup_topic,
default_version=self.BASE_RPC_API_VERSION)
def create_backup(self, ctxt, host, backup_id, volume_id):
LOG.debug("create_backup in rpcapi backup_id %s", backup_id)
topic = rpc.queue_get_for(ctxt, self.topic, host)
LOG.debug("create queue topic=%s", topic)
self.cast(ctxt,
self.make_msg('create_backup',
backup_id=backup_id),
topic=topic)
def restore_backup(self, ctxt, host, backup_id, volume_id):
LOG.debug("restore_backup in rpcapi backup_id %s", backup_id)
topic = rpc.queue_get_for(ctxt, self.topic, host)
LOG.debug("restore queue topic=%s", topic)
self.cast(ctxt,
self.make_msg('restore_backup',
backup_id=backup_id,
volume_id=volume_id),
topic=topic)
def delete_backup(self, ctxt, host, backup_id):
LOG.debug("delete_backup rpcapi backup_id %s", backup_id)
topic = rpc.queue_get_for(ctxt, self.topic, host)
self.cast(ctxt,
self.make_msg('delete_backup',
backup_id=backup_id),
topic=topic)

View File

@ -0,0 +1,14 @@
# Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.

View File

@ -0,0 +1,384 @@
# Copyright (C) 2012 Hewlett-Packard Development Company, L.P.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""Implementation of a backup service that uses Swift as the backend
**Related Flags**
:backup_swift_url: The URL of the Swift endpoint (default:
localhost:8080).
:backup_swift_object_size: The size in bytes of the Swift objects used
for volume backups (default: 52428800).
:backup_swift_retry_attempts: The number of retries to make for Swift
operations (default: 10).
:backup_swift_retry_backoff: The backoff time in seconds between retrying
failed Swift operations (default: 10).
:backup_compression_algorithm: Compression algorithm to use for volume
backups. Supported options are:
None (to disable), zlib and bz2 (default: zlib)
"""
import hashlib
import httplib
import json
import os
import socket
import StringIO
import eventlet
from oslo.config import cfg
from cinder.db import base
from cinder import exception
from cinder import flags
from cinder.openstack.common import log as logging
from cinder.openstack.common import timeutils
from swiftclient import client as swift
LOG = logging.getLogger(__name__)
swiftbackup_service_opts = [
cfg.StrOpt('backup_swift_url',
default='http://localhost:8080/v1/AUTH_',
help='The URL of the Swift endpoint'),
cfg.StrOpt('backup_swift_container',
default='volumebackups',
help='The default Swift container to use'),
cfg.IntOpt('backup_swift_object_size',
default=52428800,
help='The size in bytes of Swift backup objects'),
cfg.IntOpt('backup_swift_retry_attempts',
default=3,
help='The number of retries to make for Swift operations'),
cfg.IntOpt('backup_swift_retry_backoff',
default=2,
help='The backoff time in seconds between Swift retries'),
cfg.StrOpt('backup_compression_algorithm',
default='zlib',
help='Compression algorithm (None to disable)'),
]
FLAGS = flags.FLAGS
FLAGS.register_opts(swiftbackup_service_opts)
class SwiftBackupService(base.Base):
"""Provides backup, restore and delete of backup objects within Swift."""
SERVICE_VERSION = '1.0.0'
SERVICE_VERSION_MAPPING = {'1.0.0': '_restore_v1'}
def _get_compressor(self, algorithm):
try:
if algorithm.lower() in ('none', 'off', 'no'):
return None
elif algorithm.lower() in ('zlib', 'gzip'):
import zlib as compressor
return compressor
elif algorithm.lower() in ('bz2', 'bzip2'):
import bz2 as compressor
return compressor
except ImportError:
pass
err = _('unsupported compression algorithm: %s') % algorithm
raise ValueError(unicode(err))
def __init__(self, context, db_driver=None):
self.context = context
self.swift_url = '%s%s' % (FLAGS.backup_swift_url,
self.context.project_id)
self.az = FLAGS.storage_availability_zone
self.data_block_size_bytes = FLAGS.backup_swift_object_size
self.swift_attempts = FLAGS.backup_swift_retry_attempts
self.swift_backoff = FLAGS.backup_swift_retry_backoff
self.compressor = \
self._get_compressor(FLAGS.backup_compression_algorithm)
self.conn = swift.Connection(None, None, None,
retries=self.swift_attempts,
preauthurl=self.swift_url,
preauthtoken=self.context.auth_token,
starting_backoff=self.swift_backoff)
super(SwiftBackupService, self).__init__(db_driver)
def _check_container_exists(self, container):
LOG.debug(_('_check_container_exists: container: %s') % container)
try:
self.conn.head_container(container)
except swift.ClientException as error:
if error.http_status == httplib.NOT_FOUND:
LOG.debug(_('container %s does not exist') % container)
return False
else:
raise
else:
LOG.debug(_('container %s exists') % container)
return True
def _create_container(self, context, backup):
backup_id = backup['id']
container = backup['container']
LOG.debug(_('_create_container started, container: %(container)s,'
'backup: %(backup_id)s') % locals())
if container is None:
container = FLAGS.backup_swift_container
self.db.backup_update(context, backup_id, {'container': container})
if not self._check_container_exists(container):
self.conn.put_container(container)
return container
def _generate_swift_object_name_prefix(self, backup):
az = 'az_%s' % self.az
backup_name = '%s_backup_%s' % (az, backup['id'])
volume = 'volume_%s' % (backup['volume_id'])
timestamp = timeutils.strtime(fmt="%Y%m%d%H%M%S")
prefix = volume + '/' + timestamp + '/' + backup_name
LOG.debug(_('_generate_swift_object_name_prefix: %s') % prefix)
return prefix
def _generate_object_names(self, backup):
prefix = backup['service_metadata']
swift_objects = self.conn.get_container(backup['container'],
prefix=prefix,
full_listing=True)[1]
swift_object_names = []
for swift_object in swift_objects:
swift_object_names.append(swift_object['name'])
LOG.debug(_('generated object list: %s') % swift_object_names)
return swift_object_names
def _metadata_filename(self, backup):
swift_object_name = backup['service_metadata']
filename = '%s_metadata' % swift_object_name
return filename
def _write_metadata(self, backup, volume_id, container, object_list):
filename = self._metadata_filename(backup)
LOG.debug(_('_write_metadata started, container name: %(container)s,'
' metadata filename: %(filename)s') % locals())
metadata = {}
metadata['version'] = self.SERVICE_VERSION
metadata['backup_id'] = backup['id']
metadata['volume_id'] = volume_id
metadata['backup_name'] = backup['display_name']
metadata['backup_description'] = backup['display_description']
metadata['created_at'] = str(backup['created_at'])
metadata['objects'] = object_list
metadata_json = json.dumps(metadata, sort_keys=True, indent=2)
reader = StringIO.StringIO(metadata_json)
etag = self.conn.put_object(container, filename, reader)
md5 = hashlib.md5(metadata_json).hexdigest()
if etag != md5:
err = _('error writing metadata file to swift, MD5 of metadata'
' file in swift [%(etag)s] is not the same as MD5 of '
'metadata file sent to swift [%(md5)s]') % locals()
raise exception.InvalidBackup(reason=err)
LOG.debug(_('_write_metadata finished'))
def _read_metadata(self, backup):
container = backup['container']
filename = self._metadata_filename(backup)
LOG.debug(_('_read_metadata started, container name: %(container)s, '
'metadata filename: %(filename)s') % locals())
(resp, body) = self.conn.get_object(container, filename)
metadata = json.loads(body)
LOG.debug(_('_read_metadata finished (%s)') % metadata)
return metadata
def backup(self, backup, volume_file):
"""Backup the given volume to swift using the given backup metadata."""
backup_id = backup['id']
volume_id = backup['volume_id']
volume = self.db.volume_get(self.context, volume_id)
if volume['size'] <= 0:
err = _('volume size %d is invalid.') % volume['size']
raise exception.InvalidVolume(reason=err)
try:
container = self._create_container(self.context, backup)
except socket.error as err:
raise exception.SwiftConnectionFailed(reason=str(err))
object_prefix = self._generate_swift_object_name_prefix(backup)
backup['service_metadata'] = object_prefix
self.db.backup_update(self.context, backup_id, {'service_metadata':
object_prefix})
volume_size_bytes = volume['size'] * 1024 * 1024 * 1024
availability_zone = self.az
LOG.debug(_('starting backup of volume: %(volume_id)s to swift,'
' volume size: %(volume_size_bytes)d, swift object names'
' prefix %(object_prefix)s, availability zone:'
' %(availability_zone)s') % locals())
object_id = 1
object_list = []
while True:
data_block_size_bytes = self.data_block_size_bytes
object_name = '%s-%05d' % (object_prefix, object_id)
obj = {}
obj[object_name] = {}
obj[object_name]['offset'] = volume_file.tell()
data = volume_file.read(data_block_size_bytes)
obj[object_name]['length'] = len(data)
if data == '':
break
LOG.debug(_('reading chunk of data from volume'))
if self.compressor is not None:
algorithm = FLAGS.backup_compression_algorithm.lower()
obj[object_name]['compression'] = algorithm
data_size_bytes = len(data)
data = self.compressor.compress(data)
comp_size_bytes = len(data)
LOG.debug(_('compressed %(data_size_bytes)d bytes of data'
' to %(comp_size_bytes)d bytes using '
'%(algorithm)s') % locals())
else:
LOG.debug(_('not compressing data'))
obj[object_name]['compression'] = 'none'
reader = StringIO.StringIO(data)
LOG.debug(_('About to put_object'))
try:
etag = self.conn.put_object(container, object_name, reader)
except socket.error as err:
raise exception.SwiftConnectionFailed(reason=str(err))
LOG.debug(_('swift MD5 for %(object_name)s: %(etag)s') % locals())
md5 = hashlib.md5(data).hexdigest()
obj[object_name]['md5'] = md5
LOG.debug(_('backup MD5 for %(object_name)s: %(md5)s') % locals())
if etag != md5:
err = _('error writing object to swift, MD5 of object in '
'swift %(etag)s is not the same as MD5 of object sent '
'to swift %(md5)s') % locals()
raise exception.InvalidBackup(reason=err)
object_list.append(obj)
object_id += 1
LOG.debug(_('Calling eventlet.sleep(0)'))
eventlet.sleep(0)
try:
self._write_metadata(backup, volume_id, container, object_list)
except socket.error as err:
raise exception.SwiftConnectionFailed(reason=str(err))
self.db.backup_update(self.context, backup_id, {'object_count':
object_id})
LOG.debug(_('backup %s finished.') % backup_id)
def _restore_v1(self, backup, volume_id, metadata, volume_file):
"""Restore a v1 swift volume backup from swift."""
backup_id = backup['id']
LOG.debug(_('v1 swift volume backup restore of %s started'), backup_id)
container = backup['container']
metadata_objects = metadata['objects']
metadata_object_names = []
for metadata_object in metadata_objects:
metadata_object_names.extend(metadata_object.keys())
LOG.debug(_('metadata_object_names = %s') % metadata_object_names)
prune_list = [self._metadata_filename(backup)]
swift_object_names = [swift_object_name for swift_object_name in
self._generate_object_names(backup)
if swift_object_name not in prune_list]
if sorted(swift_object_names) != sorted(metadata_object_names):
err = _('restore_backup aborted, actual swift object list in '
'swift does not match object list stored in metadata')
raise exception.InvalidBackup(reason=err)
for metadata_object in metadata_objects:
object_name = metadata_object.keys()[0]
LOG.debug(_('restoring object from swift. backup: %(backup_id)s, '
'container: %(container)s, swift object name: '
'%(object_name)s, volume: %(volume_id)s') % locals())
try:
(resp, body) = self.conn.get_object(container, object_name)
except socket.error as err:
raise exception.SwiftConnectionFailed(reason=str(err))
compression_algorithm = metadata_object[object_name]['compression']
decompressor = self._get_compressor(compression_algorithm)
if decompressor is not None:
LOG.debug(_('decompressing data using %s algorithm') %
compression_algorithm)
decompressed = decompressor.decompress(body)
volume_file.write(decompressed)
else:
volume_file.write(body)
# force flush every write to avoid long blocking write on close
volume_file.flush()
os.fsync(volume_file.fileno())
# Restoring a backup to a volume can take some time. Yield so other
# threads can run, allowing for among other things the service
# status to be updated
eventlet.sleep(0)
LOG.debug(_('v1 swift volume backup restore of %s finished'),
backup_id)
def restore(self, backup, volume_id, volume_file):
"""Restore the given volume backup from swift."""
backup_id = backup['id']
container = backup['container']
object_prefix = backup['service_metadata']
LOG.debug(_('starting restore of backup %(object_prefix)s from swift'
' container: %(container)s, to volume %(volume_id)s, '
'backup: %(backup_id)s') % locals())
try:
metadata = self._read_metadata(backup)
except socket.error as err:
raise exception.SwiftConnectionFailed(reason=str(err))
metadata_version = metadata['version']
LOG.debug(_('Restoring swift backup version %s'), metadata_version)
try:
restore_func = getattr(self, self.SERVICE_VERSION_MAPPING.get(
metadata_version))
except TypeError:
err = (_('No support to restore swift backup version %s')
% metadata_version)
raise exception.InvalidBackup(reason=err)
restore_func(backup, volume_id, metadata, volume_file)
LOG.debug(_('restore %(backup_id)s to %(volume_id)s finished.') %
locals())
def delete(self, backup):
"""Delete the given backup from swift."""
container = backup['container']
LOG.debug('delete started, backup: %s, container: %s, prefix: %s',
backup['id'], container, backup['service_metadata'])
if container is not None:
swift_object_names = []
try:
swift_object_names = self._generate_object_names(backup)
except Exception:
LOG.warn(_('swift error while listing objects, continuing'
' with delete'))
for swift_object_name in swift_object_names:
try:
self.conn.delete_object(container, swift_object_name)
except socket.error as err:
raise exception.SwiftConnectionFailed(reason=str(err))
except Exception:
LOG.warn(_('swift error while deleting object %s, '
'continuing with delete') % swift_object_name)
else:
LOG.debug(_('deleted swift object: %(swift_object_name)s'
' in container: %(container)s') % locals())
# Deleting a backup's objects from swift can take some time.
# Yield so other threads can run
eventlet.sleep(0)
LOG.debug(_('delete %s finished') % backup['id'])
def get_backup_service(context):
return SwiftBackupService(context)

16
cinder/brick/__init__.py Normal file
View File

@ -0,0 +1,16 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2013 OpenStack Foundation.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.

View File

@ -0,0 +1,16 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2013 OpenStack Foundation.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.

468
cinder/brick/iscsi/iscsi.py Normal file
View File

@ -0,0 +1,468 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2010 United States Government as represented by the
# Administrator of the National Aeronautics and Space Administration.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""
Helper code for the iSCSI volume driver.
"""
import os
import re
from oslo.config import cfg
from cinder import exception
from cinder import flags
from cinder.openstack.common import log as logging
from cinder import utils
from cinder.volume import utils as volume_utils
LOG = logging.getLogger(__name__)
iscsi_helper_opt = [cfg.StrOpt('iscsi_helper',
default='tgtadm',
help='iscsi target user-land tool to use'),
cfg.StrOpt('volumes_dir',
default='$state_path/volumes',
help='Volume configuration file storage '
'directory'),
cfg.StrOpt('iet_conf',
default='/etc/iet/ietd.conf',
help='IET configuration file'),
cfg.StrOpt('lio_initiator_iqns',
default='',
help=('Comma-separatd list of initiator IQNs '
'allowed to connect to the '
'iSCSI target. (From Nova compute nodes.)'
)
),
cfg.StrOpt('iscsi_iotype',
default='fileio',
help=('Sets the behavior of the iSCSI target '
'to either perform blockio or fileio '
'optionally, auto can be set and Cinder '
'will autodetect type of backing device')
)
]
FLAGS = flags.FLAGS
FLAGS.register_opts(iscsi_helper_opt)
FLAGS.import_opt('volume_name_template', 'cinder.db')
class TargetAdmin(object):
"""iSCSI target administration.
Base class for iSCSI target admin helpers.
"""
def __init__(self, cmd, execute):
self._cmd = cmd
self.set_execute(execute)
def set_execute(self, execute):
"""Set the function to be used to execute commands."""
self._execute = execute
def _run(self, *args, **kwargs):
self._execute(self._cmd, *args, run_as_root=True, **kwargs)
def create_iscsi_target(self, name, tid, lun, path,
chap_auth=None, **kwargs):
"""Create a iSCSI target and logical unit"""
raise NotImplementedError()
def remove_iscsi_target(self, tid, lun, vol_id, **kwargs):
"""Remove a iSCSI target and logical unit"""
raise NotImplementedError()
def _new_target(self, name, tid, **kwargs):
"""Create a new iSCSI target."""
raise NotImplementedError()
def _delete_target(self, tid, **kwargs):
"""Delete a target."""
raise NotImplementedError()
def show_target(self, tid, iqn=None, **kwargs):
"""Query the given target ID."""
raise NotImplementedError()
def _new_logicalunit(self, tid, lun, path, **kwargs):
"""Create a new LUN on a target using the supplied path."""
raise NotImplementedError()
def _delete_logicalunit(self, tid, lun, **kwargs):
"""Delete a logical unit from a target."""
raise NotImplementedError()
class TgtAdm(TargetAdmin):
"""iSCSI target administration using tgtadm."""
def __init__(self, execute=utils.execute):
super(TgtAdm, self).__init__('tgtadm', execute)
def _get_target(self, iqn):
(out, err) = self._execute('tgt-admin', '--show', run_as_root=True)
lines = out.split('\n')
for line in lines:
if iqn in line:
parsed = line.split()
tid = parsed[1]
return tid[:-1]
return None
def create_iscsi_target(self, name, tid, lun, path,
chap_auth=None, **kwargs):
# Note(jdg) tid and lun aren't used by TgtAdm but remain for
# compatibility
utils.ensure_tree(FLAGS.volumes_dir)
vol_id = name.split(':')[1]
if chap_auth is None:
volume_conf = """
<target %s>
backing-store %s
</target>
""" % (name, path)
else:
volume_conf = """
<target %s>
backing-store %s
%s
</target>
""" % (name, path, chap_auth)
LOG.info(_('Creating iscsi_target for: %s') % vol_id)
volumes_dir = FLAGS.volumes_dir
volume_path = os.path.join(volumes_dir, vol_id)
f = open(volume_path, 'w+')
f.write(volume_conf)
f.close()
old_persist_file = None
old_name = kwargs.get('old_name', None)
if old_name is not None:
old_persist_file = os.path.join(volumes_dir, old_name)
try:
(out, err) = self._execute('tgt-admin',
'--update',
name,
run_as_root=True)
except exception.ProcessExecutionError, e:
LOG.error(_("Failed to create iscsi target for volume "
"id:%(vol_id)s.") % locals())
#Don't forget to remove the persistent file we created
os.unlink(volume_path)
raise exception.ISCSITargetCreateFailed(volume_id=vol_id)
iqn = '%s%s' % (FLAGS.iscsi_target_prefix, vol_id)
tid = self._get_target(iqn)
if tid is None:
LOG.error(_("Failed to create iscsi target for volume "
"id:%(vol_id)s. Please ensure your tgtd config file "
"contains 'include %(volumes_dir)s/*'") % locals())
raise exception.NotFound()
if old_persist_file is not None and os.path.exists(old_persist_file):
os.unlink(old_persist_file)
return tid
def remove_iscsi_target(self, tid, lun, vol_id, **kwargs):
LOG.info(_('Removing iscsi_target for: %s') % vol_id)
vol_uuid_file = FLAGS.volume_name_template % vol_id
volume_path = os.path.join(FLAGS.volumes_dir, vol_uuid_file)
if os.path.isfile(volume_path):
iqn = '%s%s' % (FLAGS.iscsi_target_prefix,
vol_uuid_file)
else:
raise exception.ISCSITargetRemoveFailed(volume_id=vol_id)
try:
# NOTE(vish): --force is a workaround for bug:
# https://bugs.launchpad.net/cinder/+bug/1159948
self._execute('tgt-admin',
'--force',
'--delete',
iqn,
run_as_root=True)
except exception.ProcessExecutionError, e:
LOG.error(_("Failed to remove iscsi target for volume "
"id:%(vol_id)s.") % locals())
raise exception.ISCSITargetRemoveFailed(volume_id=vol_id)
os.unlink(volume_path)
def show_target(self, tid, iqn=None, **kwargs):
if iqn is None:
raise exception.InvalidParameterValue(
err=_('valid iqn needed for show_target'))
tid = self._get_target(iqn)
if tid is None:
raise exception.NotFound()
class IetAdm(TargetAdmin):
"""iSCSI target administration using ietadm."""
def __init__(self, execute=utils.execute):
super(IetAdm, self).__init__('ietadm', execute)
def _iotype(self, path):
if FLAGS.iscsi_iotype == 'auto':
return 'blockio' if volume_utils.is_block(path) else 'fileio'
else:
return FLAGS.iscsi_iotype
def create_iscsi_target(self, name, tid, lun, path,
chap_auth=None, **kwargs):
# NOTE (jdg): Address bug: 1175207
kwargs.pop('old_name', None)
self._new_target(name, tid, **kwargs)
self._new_logicalunit(tid, lun, path, **kwargs)
if chap_auth is not None:
(type, username, password) = chap_auth.split()
self._new_auth(tid, type, username, password, **kwargs)
conf_file = FLAGS.iet_conf
if os.path.exists(conf_file):
try:
volume_conf = """
Target %s
%s
Lun 0 Path=%s,Type=%s
""" % (name, chap_auth, path, self._iotype(path))
with utils.temporary_chown(conf_file):
f = open(conf_file, 'a+')
f.write(volume_conf)
f.close()
except exception.ProcessExecutionError, e:
vol_id = name.split(':')[1]
LOG.error(_("Failed to create iscsi target for volume "
"id:%(vol_id)s.") % locals())
raise exception.ISCSITargetCreateFailed(volume_id=vol_id)
return tid
def remove_iscsi_target(self, tid, lun, vol_id, **kwargs):
LOG.info(_('Removing iscsi_target for volume: %s') % vol_id)
self._delete_logicalunit(tid, lun, **kwargs)
self._delete_target(tid, **kwargs)
vol_uuid_file = FLAGS.volume_name_template % vol_id
conf_file = FLAGS.iet_conf
if os.path.exists(conf_file):
with utils.temporary_chown(conf_file):
try:
iet_conf_text = open(conf_file, 'r+')
full_txt = iet_conf_text.readlines()
new_iet_conf_txt = []
count = 0
for line in full_txt:
if count > 0:
count -= 1
continue
elif re.search(vol_uuid_file, line):
count = 2
continue
else:
new_iet_conf_txt.append(line)
iet_conf_text.seek(0)
iet_conf_text.truncate(0)
iet_conf_text.writelines(new_iet_conf_txt)
finally:
iet_conf_text.close()
def _new_target(self, name, tid, **kwargs):
self._run('--op', 'new',
'--tid=%s' % tid,
'--params', 'Name=%s' % name,
**kwargs)
def _delete_target(self, tid, **kwargs):
self._run('--op', 'delete',
'--tid=%s' % tid,
**kwargs)
def show_target(self, tid, iqn=None, **kwargs):
self._run('--op', 'show',
'--tid=%s' % tid,
**kwargs)
def _new_logicalunit(self, tid, lun, path, **kwargs):
self._run('--op', 'new',
'--tid=%s' % tid,
'--lun=%d' % lun,
'--params', 'Path=%s,Type=%s' % (path, self._iotype(path)),
**kwargs)
def _delete_logicalunit(self, tid, lun, **kwargs):
self._run('--op', 'delete',
'--tid=%s' % tid,
'--lun=%d' % lun,
**kwargs)
def _new_auth(self, tid, type, username, password, **kwargs):
self._run('--op', 'new',
'--tid=%s' % tid,
'--user',
'--params=%s=%s,Password=%s' % (type, username, password),
**kwargs)
class FakeIscsiHelper(object):
def __init__(self):
self.tid = 1
def set_execute(self, execute):
self._execute = execute
def create_iscsi_target(self, *args, **kwargs):
self.tid += 1
return self.tid
class LioAdm(TargetAdmin):
"""iSCSI target administration for LIO using python-rtslib."""
def __init__(self, execute=utils.execute):
super(LioAdm, self).__init__('rtstool', execute)
try:
self._execute('rtstool', 'verify')
except (OSError, exception.ProcessExecutionError):
LOG.error(_('rtstool is not installed correctly'))
raise
def _get_target(self, iqn):
(out, err) = self._execute('rtstool',
'get-targets',
run_as_root=True)
lines = out.split('\n')
for line in lines:
if iqn in line:
return line
return None
def create_iscsi_target(self, name, tid, lun, path,
chap_auth=None, **kwargs):
# tid and lun are not used
vol_id = name.split(':')[1]
LOG.info(_('Creating iscsi_target for volume: %s') % vol_id)
# rtstool requires chap_auth, but unit tests don't provide it
chap_auth_userid = 'test_id'
chap_auth_password = 'test_pass'
if chap_auth != None:
(chap_auth_userid, chap_auth_password) = chap_auth.split(' ')[1:]
extra_args = []
if FLAGS.lio_initiator_iqns:
extra_args.append(FLAGS.lio_initiator_iqns)
try:
command_args = ['rtstool',
'create',
path,
name,
chap_auth_userid,
chap_auth_password]
if extra_args != []:
command_args += extra_args
self._execute(*command_args, run_as_root=True)
except exception.ProcessExecutionError as e:
LOG.error(_("Failed to create iscsi target for volume "
"id:%(vol_id)s.") % locals())
LOG.error("%s" % str(e))
raise exception.ISCSITargetCreateFailed(volume_id=vol_id)
iqn = '%s%s' % (FLAGS.iscsi_target_prefix, vol_id)
tid = self._get_target(iqn)
if tid is None:
LOG.error(_("Failed to create iscsi target for volume "
"id:%(vol_id)s.") % locals())
raise exception.NotFound()
return tid
def remove_iscsi_target(self, tid, lun, vol_id, **kwargs):
LOG.info(_('Removing iscsi_target: %s') % vol_id)
vol_uuid_name = 'volume-%s' % vol_id
iqn = '%s%s' % (FLAGS.iscsi_target_prefix, vol_uuid_name)
try:
self._execute('rtstool',
'delete',
iqn,
run_as_root=True)
except exception.ProcessExecutionError as e:
LOG.error(_("Failed to remove iscsi target for volume "
"id:%(vol_id)s.") % locals())
LOG.error("%s" % str(e))
raise exception.ISCSITargetRemoveFailed(volume_id=vol_id)
def show_target(self, tid, iqn=None, **kwargs):
if iqn is None:
raise exception.InvalidParameterValue(
err=_('valid iqn needed for show_target'))
tid = self._get_target(iqn)
if tid is None:
raise exception.NotFound()
def initialize_connection(self, volume, connector):
volume_iqn = volume['provider_location'].split(' ')[1]
(auth_method, auth_user, auth_pass) = \
volume['provider_auth'].split(' ', 3)
# Add initiator iqns to target ACL
try:
self._execute('rtstool', 'add-initiator',
volume_iqn,
auth_user,
auth_pass,
connector['initiator'],
run_as_root=True)
except exception.ProcessExecutionError as e:
LOG.error(_("Failed to add initiator iqn %s to target") %
connector['initiator'])
raise exception.ISCSITargetAttachFailed(volume_id=volume['id'])
def get_target_admin():
if FLAGS.iscsi_helper == 'tgtadm':
return TgtAdm()
elif FLAGS.iscsi_helper == 'fake':
return FakeIscsiHelper()
elif FLAGS.iscsi_helper == 'lioadm':
return LioAdm()
else:
return IetAdm()

View File

@ -0,0 +1,16 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2013 OpenStack Foundation.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.

View File

@ -0,0 +1,368 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2013 OpenStack Foundation.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""
LVM class for performing LVM operations.
"""
import math
from itertools import izip
from cinder.openstack.common.gettextutils import _
from cinder.openstack.common import log as logging
from cinder.openstack.common import processutils as putils
LOG = logging.getLogger(__name__)
class VolumeGroupNotFound(Exception):
def __init__(self, vg_name):
message = (_('Unable to find Volume Group: %s') % vg_name)
super(VolumeGroupNotFound, self).__init__(message)
class VolumeGroupCreationFailed(Exception):
def __init__(self, vg_name):
message = (_('Failed to create Volume Group: %s') % vg_name)
super(VolumeGroupCreationFailed, self).__init__(message)
class LVM(object):
"""LVM object to enable various LVM related operations."""
def __init__(self, vg_name, create_vg=False,
physical_volumes=None):
"""Initialize the LVM object.
The LVM object is based on an LVM VolumeGroup, one instantiation
for each VolumeGroup you have/use.
:param vg_name: Name of existing VG or VG to create
:param create_vg: Indicates the VG doesn't exist
and we want to create it
:param physical_volumes: List of PVs to build VG on
"""
self.vg_name = vg_name
self.pv_list = []
self.lv_list = []
self.vg_size = 0
self.vg_available_space = 0
self.vg_lv_count = 0
self.vg_uuid = None
if create_vg and physical_volumes is not None:
self.pv_list = physical_volumes
try:
self._create_vg(physical_volumes)
except putils.ProcessExecutionError as err:
LOG.exception(_('Error creating Volume Group'))
LOG.error(_('Cmd :%s') % err.cmd)
LOG.error(_('StdOut :%s') % err.stdout)
LOG.error(_('StdErr :%s') % err.stderr)
raise VolumeGroupCreationFailed(vg_name=self.vg_name)
if self._vg_exists() is False:
LOG.error(_('Unable to locate Volume Group %s') % vg_name)
raise VolumeGroupNotFound(vg_name=vg_name)
def _size_str(self, size_in_g):
if '.00' in size_in_g:
size_in_g = size_in_g.replace('.00', '')
if int(size_in_g) == 0:
return '100M'
return '%sG' % size_in_g
def _vg_exists(self):
"""Simple check to see if VG exists.
:returns: True if vg specified in object exists, else False
"""
exists = False
cmd = ['vgs', '--noheadings', '-o', 'name']
(out, err) = putils.execute(*cmd, root_helper='sudo', run_as_root=True)
if out is not None:
volume_groups = out.split()
if self.vg_name in volume_groups:
exists = True
return exists
def _create_vg(self, pv_list):
cmd = ['vgcreate', self.vg_name, ','.join(pv_list)]
putils.execute(*cmd, root_helper='sudo', run_as_root=True)
def _get_vg_uuid(self):
(out, err) = putils.execute('vgs', '--noheadings',
'-o uuid', self.vg_name)
if out is not None:
return out.split()
else:
return []
@staticmethod
def supports_thin_provisioning():
"""Static method to check for thin LVM support on a system.
:returns: True if supported, False otherwise
"""
cmd = ['vgs', '--version']
(out, err) = putils.execute(*cmd, root_helper='sudo', run_as_root=True)
lines = out.split('\n')
for line in lines:
if 'LVM version' in line:
version_list = line.split()
version = version_list[2]
if '(2)' in version:
version = version.replace('(2)', '')
version_tuple = tuple(map(int, version.split('.')))
if version_tuple >= (2, 2, 95):
return True
return False
@staticmethod
def get_all_volumes(vg_name=None):
"""Static method to get all LV's on a system.
:param vg_name: optional, gathers info for only the specified VG
:returns: List of Dictionaries with LV info
"""
cmd = ['lvs', '--noheadings', '-o', 'vg_name,name,size']
if vg_name is not None:
cmd += [vg_name]
(out, err) = putils.execute(*cmd, root_helper='sudo', run_as_root=True)
lv_list = []
if out is not None:
volumes = out.split()
for vg, name, size in izip(*[iter(volumes)] * 3):
lv_list.append({"vg": vg, "name": name, "size": size})
return lv_list
def get_volumes(self):
"""Get all LV's associated with this instantiation (VG).
:returns: List of Dictionaries with LV info
"""
self.lv_list = self.get_all_volumes(self.vg_name)
return self.lv_list
def get_volume(self, name):
"""Get reference object of volume specified by name.
:returns: dict representation of Logical Volume if exists
"""
ref_list = self.get_volumes()
for r in ref_list:
if r['name'] == name:
return r
@staticmethod
def get_all_physical_volumes(vg_name=None):
"""Static method to get all PVs on a system.
:param vg_name: optional, gathers info for only the specified VG
:returns: List of Dictionaries with PV info
"""
cmd = ['pvs', '--noheadings',
'-o', 'vg_name,name,size,free',
'--separator', ':']
if vg_name is not None:
cmd += [vg_name]
(out, err) = putils.execute(*cmd, root_helper='sudo', run_as_root=True)
pv_list = []
if out is not None:
pvs = out.split()
for pv in pvs:
fields = pv.split(':')
pv_list.append({'vg': fields[0],
'name': fields[1],
'size': fields[2],
'available': fields[3]})
return pv_list
def get_physical_volumes(self):
"""Get all PVs associated with this instantiation (VG).
:returns: List of Dictionaries with PV info
"""
self.pv_list = self.get_all_physical_volumes(self.vg_name)
return self.pv_list
@staticmethod
def get_all_volume_groups(vg_name=None):
"""Static method to get all VGs on a system.
:param vg_name: optional, gathers info for only the specified VG
:returns: List of Dictionaries with VG info
"""
cmd = ['vgs', '--noheadings',
'-o', 'name,size,free,lv_count,uuid',
'--separator', ':']
if vg_name is not None:
cmd += [vg_name]
(out, err) = putils.execute(*cmd, root_helper='sudo', run_as_root=True)
vg_list = []
if out is not None:
vgs = out.split()
for vg in vgs:
fields = vg.split(':')
vg_list.append({'name': fields[0],
'size': fields[1],
'available': fields[2],
'lv_count': fields[3],
'uuid': fields[4]})
return vg_list
def update_volume_group_info(self):
"""Update VG info for this instantiation.
Used to update member fields of object and
provide a dict of info for caller.
:returns: Dictionaries of VG info
"""
vg_list = self.get_all_volume_groups(self.vg_name)
if len(vg_list) != 1:
LOG.error(_('Unable to find VG: %s') % self.vg_name)
raise VolumeGroupNotFound(vg_name=self.vg_name)
self.vg_size = vg_list[0]['size']
self.vg_available_space = vg_list[0]['available']
self.vg_lv_count = vg_list[0]['lv_count']
self.vg_uuid = vg_list[0]['uuid']
return vg_list[0]
def create_thin_pool(self, name=None, size_str=0):
"""Creates a thin provisioning pool for this VG.
:param name: Name to use for pool, default is "<vg-name>-pool"
:param size_str: Size to allocate for pool, default is entire VG
"""
if not self.supports_thin_provisioning():
LOG.error(_('Requested to setup thin provisioning, '
'however current LVM version does not '
'support it.'))
return None
if name is None:
name = '%s-pool' % self.vg_name
if size_str == 0:
self.update_volume_group_info()
size_str = self.vg_size
self.create_volume(name, size_str, 'thin')
def create_volume(self, name, size_str, lv_type='default', mirror_count=0):
"""Creates a logical volume on the object's VG.
:param name: Name to use when creating Logical Volume
:param size_str: Size to use when creating Logical Volume
:param lv_type: Type of Volume (default or thin)
:param mirror_count: Use LVM mirroring with specified count
"""
size = self._size_str(size_str)
cmd = ['lvcreate', '-n', name, self.vg_name]
if lv_type == 'thin':
cmd += ['-T', '-V', size]
else:
cmd += ['-L', size]
if mirror_count > 0:
cmd += ['-m', mirror_count, '--nosync']
terras = int(size[:-1]) / 1024.0
if terras >= 1.5:
rsize = int(2 ** math.ceil(math.log(terras) / math.log(2)))
# NOTE(vish): Next power of two for region size. See:
# http://red.ht/U2BPOD
cmd += ['-R', str(rsize)]
putils.execute(*cmd,
root_helper='sudo',
run_as_root=True)
def create_lv_snapshot(self, name, source_lv_name, lv_type='default'):
"""Creates a snapshot of a logical volume.
:param name: Name to assign to new snapshot
:param source_lv_name: Name of Logical Volume to snapshot
:param lv_type: Type of LV (default or thin)
"""
source_lvref = self.get_volume(source_lv_name)
if source_lvref is None:
LOG.error(_("Unable to find LV: %s") % source_lv_name)
return False
cmd = ['lvcreate', '--name', name,
'--snapshot', '%s/%s' % (self.vg_name, source_lv_name)]
if lv_type != 'thin':
size = source_lvref['size']
cmd += ['-L', size]
putils.execute(*cmd,
root_helper='sudo',
run_as_root=True)
def delete(self, name):
"""Delete logical volume or snapshot.
:param name: Name of LV to delete
"""
putils.execute('lvremove',
'-f',
'%s/%s' % (self.vg_name, name),
root_helper='sudo', run_as_root=True)
def revert(self, snapshot_name):
"""Revert an LV from snapshot.
:param snapshot_name: Name of snapshot to revert
"""
putils.execute('lvconvert', '--merge',
snapshot_name, root_helper='sudo',
run_as_root=True)

15
cinder/common/__init__.py Normal file
View File

@ -0,0 +1,15 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright (c) 2011 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.

128
cinder/common/sqlalchemyutils.py Executable file
View File

@ -0,0 +1,128 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2010 United States Government as represented by the
# Administrator of the National Aeronautics and Space Administration.
# Copyright 2010-2011 OpenStack LLC.
# Copyright 2012 Justin Santa Barbara
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""Implementation of paginate query."""
import sqlalchemy
from cinder import exception
from cinder.openstack.common import log as logging
LOG = logging.getLogger(__name__)
# copied from glance/db/sqlalchemy/api.py
def paginate_query(query, model, limit, sort_keys, marker=None,
sort_dir=None, sort_dirs=None):
"""Returns a query with sorting / pagination criteria added.
Pagination works by requiring a unique sort_key, specified by sort_keys.
(If sort_keys is not unique, then we risk looping through values.)
We use the last row in the previous page as the 'marker' for pagination.
So we must return values that follow the passed marker in the order.
With a single-valued sort_key, this would be easy: sort_key > X.
With a compound-values sort_key, (k1, k2, k3) we must do this to repeat
the lexicographical ordering:
(k1 > X1) or (k1 == X1 && k2 > X2) or (k1 == X1 && k2 == X2 && k3 > X3)
We also have to cope with different sort_directions.
Typically, the id of the last row is used as the client-facing pagination
marker, then the actual marker object must be fetched from the db and
passed in to us as marker.
:param query: the query object to which we should add paging/sorting
:param model: the ORM model class
:param limit: maximum number of items to return
:param sort_keys: array of attributes by which results should be sorted
:param marker: the last item of the previous page; we returns the next
results after this value.
:param sort_dir: direction in which results should be sorted (asc, desc)
:param sort_dirs: per-column array of sort_dirs, corresponding to sort_keys
:rtype: sqlalchemy.orm.query.Query
:return: The query with sorting/pagination added.
"""
if 'id' not in sort_keys:
# TODO(justinsb): If this ever gives a false-positive, check
# the actual primary key, rather than assuming its id
LOG.warn(_('Id not in sort_keys; is sort_keys unique?'))
assert(not (sort_dir and sort_dirs))
# Default the sort direction to ascending
if sort_dirs is None and sort_dir is None:
sort_dir = 'asc'
# Ensure a per-column sort direction
if sort_dirs is None:
sort_dirs = [sort_dir for _sort_key in sort_keys]
assert(len(sort_dirs) == len(sort_keys))
# Add sorting
for current_sort_key, current_sort_dir in zip(sort_keys, sort_dirs):
sort_dir_func = {
'asc': sqlalchemy.asc,
'desc': sqlalchemy.desc,
}[current_sort_dir]
try:
sort_key_attr = getattr(model, current_sort_key)
except AttributeError:
raise exception.InvalidInput(reason='Invalid sort key')
query = query.order_by(sort_dir_func(sort_key_attr))
# Add pagination
if marker is not None:
marker_values = []
for sort_key in sort_keys:
v = getattr(marker, sort_key)
marker_values.append(v)
# Build up an array of sort criteria as in the docstring
criteria_list = []
for i in xrange(0, len(sort_keys)):
crit_attrs = []
for j in xrange(0, i):
model_attr = getattr(model, sort_keys[j])
crit_attrs.append((model_attr == marker_values[j]))
model_attr = getattr(model, sort_keys[i])
if sort_dirs[i] == 'desc':
crit_attrs.append((model_attr < marker_values[i]))
elif sort_dirs[i] == 'asc':
crit_attrs.append((model_attr > marker_values[i]))
else:
raise ValueError(_("Unknown sort direction, "
"must be 'desc' or 'asc'"))
criteria = sqlalchemy.sql.and_(*crit_attrs)
criteria_list.append(criteria)
f = sqlalchemy.sql.or_(*criteria_list)
query = query.filter(f)
if limit is not None:
query = query.limit(limit)
return query

View File

View File

@ -0,0 +1,44 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2010 OpenStack LLC.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""Possible states for host aggregates.
An aggregate may be 'created', in which case the admin has triggered its
creation, but the underlying hypervisor pool has not actually being set up
yet. An aggregate may be 'changing', meaning that the underlying hypervisor
pool is being setup. An aggregate may be 'active', in which case the underlying
hypervisor pool is up and running. An aggregate may be 'dismissed' when it has
no hosts and it has been deleted. An aggregate may be in 'error' in all other
cases.
A 'created' aggregate becomes 'changing' during the first request of
adding a host. During a 'changing' status no other requests will be accepted;
this is to allow the hypervisor layer to instantiate the underlying pool
without any potential race condition that may incur in master/slave-based
configurations. The aggregate goes into the 'active' state when the underlying
pool has been correctly instantiated.
All other operations (e.g. add/remove hosts) that succeed will keep the
aggregate in the 'active' state. If a number of continuous requests fail,
an 'active' aggregate goes into an 'error' state. To recover from such a state,
admin intervention is required. Currently an error state is irreversible,
that is, in order to recover from it an aggregate must be deleted.
"""
CREATED = 'created'
CHANGING = 'changing'
ACTIVE = 'active'
ERROR = 'error'
DISMISSED = 'dismissed'

155
cinder/context.py Normal file
View File

@ -0,0 +1,155 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2011 OpenStack LLC.
# Copyright 2010 United States Government as represented by the
# Administrator of the National Aeronautics and Space Administration.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""RequestContext: context for requests that persist through all of cinder."""
import copy
import uuid
from cinder.openstack.common import local
from cinder.openstack.common import log as logging
from cinder.openstack.common import timeutils
from cinder import policy
LOG = logging.getLogger(__name__)
def generate_request_id():
return 'req-' + str(uuid.uuid4())
class RequestContext(object):
"""Security context and request information.
Represents the user taking a given action within the system.
"""
def __init__(self, user_id, project_id, is_admin=None, read_deleted="no",
roles=None, remote_address=None, timestamp=None,
request_id=None, auth_token=None, overwrite=True,
quota_class=None, **kwargs):
"""
:param read_deleted: 'no' indicates deleted records are hidden, 'yes'
indicates deleted records are visible, 'only' indicates that
*only* deleted records are visible.
:param overwrite: Set to False to ensure that the greenthread local
copy of the index is not overwritten.
:param kwargs: Extra arguments that might be present, but we ignore
because they possibly came in from older rpc messages.
"""
if kwargs:
LOG.warn(_('Arguments dropped when creating context: %s') %
str(kwargs))
self.user_id = user_id
self.project_id = project_id
self.roles = roles or []
self.is_admin = is_admin
if self.is_admin is None:
self.is_admin = policy.check_is_admin(self.roles)
elif self.is_admin and 'admin' not in self.roles:
self.roles.append('admin')
self.read_deleted = read_deleted
self.remote_address = remote_address
if not timestamp:
timestamp = timeutils.utcnow()
if isinstance(timestamp, basestring):
timestamp = timeutils.parse_strtime(timestamp)
self.timestamp = timestamp
if not request_id:
request_id = generate_request_id()
self.request_id = request_id
self.auth_token = auth_token
self.quota_class = quota_class
if overwrite or not hasattr(local.store, 'context'):
self.update_store()
def _get_read_deleted(self):
return self._read_deleted
def _set_read_deleted(self, read_deleted):
if read_deleted not in ('no', 'yes', 'only'):
raise ValueError(_("read_deleted can only be one of 'no', "
"'yes' or 'only', not %r") % read_deleted)
self._read_deleted = read_deleted
def _del_read_deleted(self):
del self._read_deleted
read_deleted = property(_get_read_deleted, _set_read_deleted,
_del_read_deleted)
def update_store(self):
local.store.context = self
def to_dict(self):
return {'user_id': self.user_id,
'project_id': self.project_id,
'is_admin': self.is_admin,
'read_deleted': self.read_deleted,
'roles': self.roles,
'remote_address': self.remote_address,
'timestamp': timeutils.strtime(self.timestamp),
'request_id': self.request_id,
'auth_token': self.auth_token,
'quota_class': self.quota_class,
'tenant': self.tenant,
'user': self.user}
@classmethod
def from_dict(cls, values):
return cls(**values)
def elevated(self, read_deleted=None, overwrite=False):
"""Return a version of this context with admin flag set."""
context = copy.copy(self)
context.is_admin = True
if 'admin' not in context.roles:
context.roles.append('admin')
if read_deleted is not None:
context.read_deleted = read_deleted
return context
# NOTE(sirp): the openstack/common version of RequestContext uses
# tenant/user whereas the Cinder version uses project_id/user_id. We need
# this shim in order to use context-aware code from openstack/common, like
# logging, until we make the switch to using openstack/common's version of
# RequestContext.
@property
def tenant(self):
return self.project_id
@property
def user(self):
return self.user_id
def get_admin_context(read_deleted="no"):
return RequestContext(user_id=None,
project_id=None,
is_admin=True,
read_deleted=read_deleted,
overwrite=False)

23
cinder/db/__init__.py Normal file
View File

@ -0,0 +1,23 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2010 United States Government as represented by the
# Administrator of the National Aeronautics and Space Administration.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""
DB abstraction for Cinder
"""
from cinder.db.api import *

883
cinder/db/api.py Normal file
View File

@ -0,0 +1,883 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright (c) 2011 X.commerce, a business unit of eBay Inc.
# Copyright 2010 United States Government as represented by the
# Administrator of the National Aeronautics and Space Administration.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""Defines interface for DB access.
The underlying driver is loaded as a :class:`LazyPluggable`.
Functions in this module are imported into the cinder.db namespace. Call these
functions from cinder.db namespace, not the cinder.db.api namespace.
All functions in this module return objects that implement a dictionary-like
interface. Currently, many of these objects are sqlalchemy objects that
implement a dictionary interface. However, a future goal is to have all of
these objects be simple dictionaries.
**Related Flags**
:db_backend: string to lookup in the list of LazyPluggable backends.
`sqlalchemy` is the only supported backend right now.
:sql_connection: string specifying the sqlalchemy connection to use, like:
`sqlite:///var/lib/cinder/cinder.sqlite`.
:enable_new_services: when adding a new service to the database, is it in the
pool of available hardware (Default: True)
"""
from oslo.config import cfg
from cinder import exception
from cinder import flags
from cinder import utils
db_opts = [
cfg.StrOpt('db_backend',
default='sqlalchemy',
help='The backend to use for db'),
cfg.BoolOpt('enable_new_services',
default=True,
help='Services to be added to the available pool on create'),
cfg.StrOpt('volume_name_template',
default='volume-%s',
help='Template string to be used to generate volume names'),
cfg.StrOpt('share_name_template',
default='share-%s',
help='Template string to be used to generate share names'),
cfg.StrOpt('share_snapshot_name_template',
default='share-snapshot-%s',
help='Template string to be used to generate share snapshot '
'names'),
cfg.StrOpt('snapshot_name_template',
default='snapshot-%s',
help='Template string to be used to generate snapshot names'),
cfg.StrOpt('backup_name_template',
default='backup-%s',
help='Template string to be used to generate backup names'), ]
FLAGS = flags.FLAGS
FLAGS.register_opts(db_opts)
IMPL = utils.LazyPluggable('db_backend',
sqlalchemy='cinder.db.sqlalchemy.api')
class NoMoreTargets(exception.CinderException):
"""No more available targets"""
pass
###################
def service_destroy(context, service_id):
"""Destroy the service or raise if it does not exist."""
return IMPL.service_destroy(context, service_id)
def service_get(context, service_id):
"""Get a service or raise if it does not exist."""
return IMPL.service_get(context, service_id)
def service_get_by_host_and_topic(context, host, topic):
"""Get a service by host it's on and topic it listens to."""
return IMPL.service_get_by_host_and_topic(context, host, topic)
def service_get_all(context, disabled=None):
"""Get all services."""
return IMPL.service_get_all(context, disabled)
def service_get_all_by_topic(context, topic):
"""Get all services for a given topic."""
return IMPL.service_get_all_by_topic(context, topic)
def service_get_all_by_host(context, host):
"""Get all services for a given host."""
return IMPL.service_get_all_by_host(context, host)
def service_get_all_volume_sorted(context):
"""Get all volume services sorted by volume count.
:returns: a list of (Service, volume_count) tuples.
"""
return IMPL.service_get_all_volume_sorted(context)
def service_get_all_share_sorted(context):
"""Get all share services sorted by share count.
:returns: a list of (Service, share_count) tuples.
"""
return IMPL.service_get_all_share_sorted(context)
def service_get_by_args(context, host, binary):
"""Get the state of an service by node name and binary."""
return IMPL.service_get_by_args(context, host, binary)
def service_create(context, values):
"""Create a service from the values dictionary."""
return IMPL.service_create(context, values)
def service_update(context, service_id, values):
"""Set the given properties on an service and update it.
Raises NotFound if service does not exist.
"""
return IMPL.service_update(context, service_id, values)
###################
def migration_update(context, id, values):
"""Update a migration instance."""
return IMPL.migration_update(context, id, values)
def migration_create(context, values):
"""Create a migration record."""
return IMPL.migration_create(context, values)
def migration_get(context, migration_id):
"""Finds a migration by the id."""
return IMPL.migration_get(context, migration_id)
def migration_get_by_instance_and_status(context, instance_uuid, status):
"""Finds a migration by the instance uuid its migrating."""
return IMPL.migration_get_by_instance_and_status(context,
instance_uuid,
status)
def migration_get_all_unconfirmed(context, confirm_window):
"""Finds all unconfirmed migrations within the confirmation window."""
return IMPL.migration_get_all_unconfirmed(context, confirm_window)
###################
def iscsi_target_count_by_host(context, host):
"""Return count of export devices."""
return IMPL.iscsi_target_count_by_host(context, host)
def iscsi_target_create_safe(context, values):
"""Create an iscsi_target from the values dictionary.
The device is not returned. If the create violates the unique
constraints because the iscsi_target and host already exist,
no exception is raised.
"""
return IMPL.iscsi_target_create_safe(context, values)
###############
def volume_allocate_iscsi_target(context, volume_id, host):
"""Atomically allocate a free iscsi_target from the pool."""
return IMPL.volume_allocate_iscsi_target(context, volume_id, host)
def volume_attached(context, volume_id, instance_id, mountpoint):
"""Ensure that a volume is set as attached."""
return IMPL.volume_attached(context, volume_id, instance_id, mountpoint)
def volume_create(context, values):
"""Create a volume from the values dictionary."""
return IMPL.volume_create(context, values)
def volume_data_get_for_host(context, host, session=None):
"""Get (volume_count, gigabytes) for project."""
return IMPL.volume_data_get_for_host(context,
host,
session)
def volume_data_get_for_project(context, project_id, session=None):
"""Get (volume_count, gigabytes) for project."""
return IMPL.volume_data_get_for_project(context,
project_id,
session)
def volume_destroy(context, volume_id):
"""Destroy the volume or raise if it does not exist."""
return IMPL.volume_destroy(context, volume_id)
def volume_detached(context, volume_id):
"""Ensure that a volume is set as detached."""
return IMPL.volume_detached(context, volume_id)
def volume_get(context, volume_id):
"""Get a volume or raise if it does not exist."""
return IMPL.volume_get(context, volume_id)
def volume_get_all(context, marker, limit, sort_key, sort_dir):
"""Get all volumes."""
return IMPL.volume_get_all(context, marker, limit, sort_key, sort_dir)
def volume_get_all_by_host(context, host):
"""Get all volumes belonging to a host."""
return IMPL.volume_get_all_by_host(context, host)
def volume_get_all_by_instance_uuid(context, instance_uuid):
"""Get all volumes belonging to a instance."""
return IMPL.volume_get_all_by_instance_uuid(context, instance_uuid)
def volume_get_all_by_project(context, project_id, marker, limit, sort_key,
sort_dir):
"""Get all volumes belonging to a project."""
return IMPL.volume_get_all_by_project(context, project_id, marker, limit,
sort_key, sort_dir)
def volume_get_iscsi_target_num(context, volume_id):
"""Get the target num (tid) allocated to the volume."""
return IMPL.volume_get_iscsi_target_num(context, volume_id)
def volume_update(context, volume_id, values):
"""Set the given properties on an volume and update it.
Raises NotFound if volume does not exist.
"""
return IMPL.volume_update(context, volume_id, values)
####################
def snapshot_create(context, values):
"""Create a snapshot from the values dictionary."""
return IMPL.snapshot_create(context, values)
def snapshot_destroy(context, snapshot_id):
"""Destroy the snapshot or raise if it does not exist."""
return IMPL.snapshot_destroy(context, snapshot_id)
def snapshot_get(context, snapshot_id):
"""Get a snapshot or raise if it does not exist."""
return IMPL.snapshot_get(context, snapshot_id)
def snapshot_get_all(context):
"""Get all snapshots."""
return IMPL.snapshot_get_all(context)
def snapshot_get_all_by_project(context, project_id):
"""Get all snapshots belonging to a project."""
return IMPL.snapshot_get_all_by_project(context, project_id)
def snapshot_get_all_for_volume(context, volume_id):
"""Get all snapshots for a volume."""
return IMPL.snapshot_get_all_for_volume(context, volume_id)
def snapshot_update(context, snapshot_id, values):
"""Set the given properties on an snapshot and update it.
Raises NotFound if snapshot does not exist.
"""
return IMPL.snapshot_update(context, snapshot_id, values)
def snapshot_data_get_for_project(context, project_id, session=None):
"""Get count and gigabytes used for snapshots for specified project."""
return IMPL.snapshot_data_get_for_project(context,
project_id,
session)
def snapshot_get_active_by_window(context, begin, end=None, project_id=None):
"""Get all the snapshots inside the window.
Specifying a project_id will filter for a certain project."""
return IMPL.snapshot_get_active_by_window(context, begin, end, project_id)
####################
def snapshot_metadata_get(context, snapshot_id):
"""Get all metadata for a snapshot."""
return IMPL.snapshot_metadata_get(context, snapshot_id)
def snapshot_metadata_delete(context, snapshot_id, key):
"""Delete the given metadata item."""
IMPL.snapshot_metadata_delete(context, snapshot_id, key)
def snapshot_metadata_update(context, snapshot_id, metadata, delete):
"""Update metadata if it exists, otherwise create it."""
IMPL.snapshot_metadata_update(context, snapshot_id, metadata, delete)
####################
def volume_metadata_get(context, volume_id):
"""Get all metadata for a volume."""
return IMPL.volume_metadata_get(context, volume_id)
def volume_metadata_delete(context, volume_id, key):
"""Delete the given metadata item."""
IMPL.volume_metadata_delete(context, volume_id, key)
def volume_metadata_update(context, volume_id, metadata, delete):
"""Update metadata if it exists, otherwise create it."""
IMPL.volume_metadata_update(context, volume_id, metadata, delete)
##################
def volume_type_create(context, values):
"""Create a new volume type."""
return IMPL.volume_type_create(context, values)
def volume_type_get_all(context, inactive=False):
"""Get all volume types."""
return IMPL.volume_type_get_all(context, inactive)
def volume_type_get(context, id):
"""Get volume type by id."""
return IMPL.volume_type_get(context, id)
def volume_type_get_by_name(context, name):
"""Get volume type by name."""
return IMPL.volume_type_get_by_name(context, name)
def volume_type_destroy(context, id):
"""Delete a volume type."""
return IMPL.volume_type_destroy(context, id)
def volume_get_active_by_window(context, begin, end=None, project_id=None):
"""Get all the volumes inside the window.
Specifying a project_id will filter for a certain project."""
return IMPL.volume_get_active_by_window(context, begin, end, project_id)
####################
def volume_type_extra_specs_get(context, volume_type_id):
"""Get all extra specs for a volume type."""
return IMPL.volume_type_extra_specs_get(context, volume_type_id)
def volume_type_extra_specs_delete(context, volume_type_id, key):
"""Delete the given extra specs item."""
IMPL.volume_type_extra_specs_delete(context, volume_type_id, key)
def volume_type_extra_specs_update_or_create(context,
volume_type_id,
extra_specs):
"""Create or update volume type extra specs. This adds or modifies the
key/value pairs specified in the extra specs dict argument"""
IMPL.volume_type_extra_specs_update_or_create(context,
volume_type_id,
extra_specs)
###################
def volume_glance_metadata_create(context, volume_id, key, value):
"""Update the Glance metadata for the specified volume."""
return IMPL.volume_glance_metadata_create(context,
volume_id,
key,
value)
def volume_glance_metadata_get(context, volume_id):
"""Return the glance metadata for a volume."""
return IMPL.volume_glance_metadata_get(context, volume_id)
def volume_snapshot_glance_metadata_get(context, snapshot_id):
"""Return the Glance metadata for the specified snapshot."""
return IMPL.volume_snapshot_glance_metadata_get(context, snapshot_id)
def volume_glance_metadata_copy_to_snapshot(context, snapshot_id, volume_id):
"""
Update the Glance metadata for a snapshot by copying all of the key:value
pairs from the originating volume. This is so that a volume created from
the snapshot will retain the original metadata.
"""
return IMPL.volume_glance_metadata_copy_to_snapshot(context, snapshot_id,
volume_id)
def volume_glance_metadata_copy_to_volume(context, volume_id, snapshot_id):
"""
Update the Glance metadata from a volume (created from a snapshot) by
copying all of the key:value pairs from the originating snapshot. This is
so that the Glance metadata from the original volume is retained.
"""
return IMPL.volume_glance_metadata_copy_to_volume(context, volume_id,
snapshot_id)
def volume_glance_metadata_delete_by_volume(context, volume_id):
"""Delete the glance metadata for a volume."""
return IMPL.volume_glance_metadata_delete_by_volume(context, volume_id)
def volume_glance_metadata_delete_by_snapshot(context, snapshot_id):
"""Delete the glance metadata for a snapshot."""
return IMPL.volume_glance_metadata_delete_by_snapshot(context, snapshot_id)
def volume_glance_metadata_copy_from_volume_to_volume(context,
src_volume_id,
volume_id):
"""
Update the Glance metadata for a volume by copying all of the key:value
pairs from the originating volume. This is so that a volume created from
the volume (clone) will retain the original metadata.
"""
return IMPL.volume_glance_metadata_copy_from_volume_to_volume(
context,
src_volume_id,
volume_id)
###################
def sm_backend_conf_create(context, values):
"""Create a new SM Backend Config entry."""
return IMPL.sm_backend_conf_create(context, values)
def sm_backend_conf_update(context, sm_backend_conf_id, values):
"""Update a SM Backend Config entry."""
return IMPL.sm_backend_conf_update(context, sm_backend_conf_id, values)
def sm_backend_conf_delete(context, sm_backend_conf_id):
"""Delete a SM Backend Config."""
return IMPL.sm_backend_conf_delete(context, sm_backend_conf_id)
def sm_backend_conf_get(context, sm_backend_conf_id):
"""Get a specific SM Backend Config."""
return IMPL.sm_backend_conf_get(context, sm_backend_conf_id)
def sm_backend_conf_get_by_sr(context, sr_uuid):
"""Get a specific SM Backend Config."""
return IMPL.sm_backend_conf_get_by_sr(context, sr_uuid)
def sm_backend_conf_get_all(context):
"""Get all SM Backend Configs."""
return IMPL.sm_backend_conf_get_all(context)
####################
def sm_flavor_create(context, values):
"""Create a new SM Flavor entry."""
return IMPL.sm_flavor_create(context, values)
def sm_flavor_update(context, sm_flavor_id, values):
"""Update a SM Flavor entry."""
return IMPL.sm_flavor_update(context, values)
def sm_flavor_delete(context, sm_flavor_id):
"""Delete a SM Flavor."""
return IMPL.sm_flavor_delete(context, sm_flavor_id)
def sm_flavor_get(context, sm_flavor):
"""Get a specific SM Flavor."""
return IMPL.sm_flavor_get(context, sm_flavor)
def sm_flavor_get_all(context):
"""Get all SM Flavors."""
return IMPL.sm_flavor_get_all(context)
####################
def sm_volume_create(context, values):
"""Create a new child Zone entry."""
return IMPL.sm_volume_create(context, values)
def sm_volume_update(context, volume_id, values):
"""Update a child Zone entry."""
return IMPL.sm_volume_update(context, values)
def sm_volume_delete(context, volume_id):
"""Delete a child Zone."""
return IMPL.sm_volume_delete(context, volume_id)
def sm_volume_get(context, volume_id):
"""Get a specific child Zone."""
return IMPL.sm_volume_get(context, volume_id)
def sm_volume_get_all(context):
"""Get all child Zones."""
return IMPL.sm_volume_get_all(context)
###################
def quota_create(context, project_id, resource, limit):
"""Create a quota for the given project and resource."""
return IMPL.quota_create(context, project_id, resource, limit)
def quota_get(context, project_id, resource):
"""Retrieve a quota or raise if it does not exist."""
return IMPL.quota_get(context, project_id, resource)
def quota_get_all_by_project(context, project_id):
"""Retrieve all quotas associated with a given project."""
return IMPL.quota_get_all_by_project(context, project_id)
def quota_update(context, project_id, resource, limit):
"""Update a quota or raise if it does not exist."""
return IMPL.quota_update(context, project_id, resource, limit)
def quota_destroy(context, project_id, resource):
"""Destroy the quota or raise if it does not exist."""
return IMPL.quota_destroy(context, project_id, resource)
###################
def quota_class_create(context, class_name, resource, limit):
"""Create a quota class for the given name and resource."""
return IMPL.quota_class_create(context, class_name, resource, limit)
def quota_class_get(context, class_name, resource):
"""Retrieve a quota class or raise if it does not exist."""
return IMPL.quota_class_get(context, class_name, resource)
def quota_class_get_all_by_name(context, class_name):
"""Retrieve all quotas associated with a given quota class."""
return IMPL.quota_class_get_all_by_name(context, class_name)
def quota_class_update(context, class_name, resource, limit):
"""Update a quota class or raise if it does not exist."""
return IMPL.quota_class_update(context, class_name, resource, limit)
def quota_class_destroy(context, class_name, resource):
"""Destroy the quota class or raise if it does not exist."""
return IMPL.quota_class_destroy(context, class_name, resource)
def quota_class_destroy_all_by_name(context, class_name):
"""Destroy all quotas associated with a given quota class."""
return IMPL.quota_class_destroy_all_by_name(context, class_name)
###################
def quota_usage_create(context, project_id, resource, in_use, reserved,
until_refresh):
"""Create a quota usage for the given project and resource."""
return IMPL.quota_usage_create(context, project_id, resource,
in_use, reserved, until_refresh)
def quota_usage_get(context, project_id, resource):
"""Retrieve a quota usage or raise if it does not exist."""
return IMPL.quota_usage_get(context, project_id, resource)
def quota_usage_get_all_by_project(context, project_id):
"""Retrieve all usage associated with a given resource."""
return IMPL.quota_usage_get_all_by_project(context, project_id)
###################
def reservation_create(context, uuid, usage, project_id, resource, delta,
expire):
"""Create a reservation for the given project and resource."""
return IMPL.reservation_create(context, uuid, usage, project_id,
resource, delta, expire)
def reservation_get(context, uuid):
"""Retrieve a reservation or raise if it does not exist."""
return IMPL.reservation_get(context, uuid)
def reservation_get_all_by_project(context, project_id):
"""Retrieve all reservations associated with a given project."""
return IMPL.reservation_get_all_by_project(context, project_id)
def reservation_destroy(context, uuid):
"""Destroy the reservation or raise if it does not exist."""
return IMPL.reservation_destroy(context, uuid)
###################
def quota_reserve(context, resources, quotas, deltas, expire,
until_refresh, max_age, project_id=None):
"""Check quotas and create appropriate reservations."""
return IMPL.quota_reserve(context, resources, quotas, deltas, expire,
until_refresh, max_age, project_id=project_id)
def reservation_commit(context, reservations, project_id=None):
"""Commit quota reservations."""
return IMPL.reservation_commit(context, reservations,
project_id=project_id)
def reservation_rollback(context, reservations, project_id=None):
"""Roll back quota reservations."""
return IMPL.reservation_rollback(context, reservations,
project_id=project_id)
def quota_destroy_all_by_project(context, project_id):
"""Destroy all quotas associated with a given project."""
return IMPL.quota_destroy_all_by_project(context, project_id)
def reservation_expire(context):
"""Roll back any expired reservations."""
return IMPL.reservation_expire(context)
###################
def backup_get(context, backup_id):
"""Get a backup or raise if it does not exist."""
return IMPL.backup_get(context, backup_id)
def backup_get_all(context):
"""Get all backups."""
return IMPL.backup_get_all(context)
def backup_get_all_by_host(context, host):
"""Get all backups belonging to a host."""
return IMPL.backup_get_all_by_host(context, host)
def backup_create(context, values):
"""Create a backup from the values dictionary."""
return IMPL.backup_create(context, values)
def backup_get_all_by_project(context, project_id):
"""Get all backups belonging to a project."""
return IMPL.backup_get_all_by_project(context, project_id)
def backup_update(context, backup_id, values):
"""
Set the given properties on a backup and update it.
Raises NotFound if backup does not exist.
"""
return IMPL.backup_update(context, backup_id, values)
def backup_destroy(context, backup_id):
"""Destroy the backup or raise if it does not exist."""
return IMPL.backup_destroy(context, backup_id)
####################
def share_create(context, values):
"""Create new share."""
return IMPL.share_create(context, values)
def share_update(context, share_id, values):
"""Update share fields."""
return IMPL.share_update(context, share_id, values)
def share_get(context, share_id):
"""Get share by id."""
return IMPL.share_get(context, share_id)
def share_get_all(context):
"""Get all shares."""
return IMPL.share_get_all(context)
def share_get_all_by_host(context, host):
"""Returns all shares with given host."""
return IMPL.share_get_all_by_host(context, host)
def share_get_all_by_project(context, project_id):
"""Returns all shares with given project ID."""
return IMPL.share_get_all_by_project(context, project_id)
def share_delete(context, share_id):
"""Delete share."""
return IMPL.share_delete(context, share_id)
###################
def share_access_create(context, values):
"""Allow access to share."""
return IMPL.share_access_create(context, values)
def share_access_get(context, access_id):
"""Allow access to share."""
return IMPL.share_access_get(context, access_id)
def share_access_get_all_for_share(context, share_id):
"""Allow access to share."""
return IMPL.share_access_get_all_for_share(context, share_id)
def share_access_delete(context, access_id):
"""Deny access to share."""
return IMPL.share_access_delete(context, access_id)
def share_access_update(context, access_id, values):
"""Update access record."""
return IMPL.share_access_update(context, access_id, values)
####################
def share_snapshot_create(context, values):
"""Create a snapshot from the values dictionary."""
return IMPL.share_snapshot_create(context, values)
def share_snapshot_destroy(context, snapshot_id):
"""Destroy the snapshot or raise if it does not exist."""
return IMPL.share_snapshot_destroy(context, snapshot_id)
def share_snapshot_get(context, snapshot_id):
"""Get a snapshot or raise if it does not exist."""
return IMPL.share_snapshot_get(context, snapshot_id)
def share_snapshot_get_all(context):
"""Get all snapshots."""
return IMPL.share_snapshot_get_all(context)
def share_snapshot_get_all_by_project(context, project_id):
"""Get all snapshots belonging to a project."""
return IMPL.share_snapshot_get_all_by_project(context, project_id)
def share_snapshot_get_all_for_share(context, share_id):
"""Get all snapshots for a share."""
return IMPL.share_snapshot_get_all_for_share(context, share_id)
def share_snapshot_update(context, snapshot_id, values):
"""Set the given properties on an snapshot and update it.
Raises NotFound if snapshot does not exist.
"""
return IMPL.share_snapshot_update(context, snapshot_id, values)
def share_snapshot_data_get_for_project(context, project_id, session=None):
"""Get count and gigabytes used for snapshots for specified project."""
return IMPL.share_snapshot_data_get_for_project(context,
project_id,
session=None)
####################

Some files were not shown because too many files have changed in this diff Show More