Add documentation for GateKeeper
The purpose of GateKeeper mostly relates to the development of new swift code, so I threw together a guide for development_middleware that covers some basics with a eye towards metadata handling in-particular. I also fixed up some missing autodoc's, split out middleware autodoc and added some ref's here and about so I could link to them from the development_middleware guide. DocImpact Change-Id: I20dd942ea8df9e33c3e794cb49669ffa1332c63e
This commit is contained in:
Clay Gerrard
Peter Portante
parent
f538006bbf
commit
63f8c2284a
|
|
@ -771,6 +771,8 @@ delay_reaping 0 Normally, the reaper begins deleting
|
|||
2592000 = 30 days, for example.
|
||||
================== =============== =========================================
|
||||
|
||||
.. _proxy-server-config:
|
||||
|
||||
--------------------------
|
||||
Proxy Server Configuration
|
||||
--------------------------
|
||||
|
|
@ -828,6 +830,15 @@ log_custom_handlers None Comma separated list of functions
|
|||
handlers.
|
||||
eventlet_debug false If true, turn on debug logging
|
||||
for eventlet
|
||||
|
||||
expose_info true Enables exposing configuration
|
||||
settings via HTTP GET /info.
|
||||
|
||||
admin_key Key to use for admin calls that
|
||||
are HMAC signed. Default
|
||||
is empty, which will
|
||||
disable admin calls to
|
||||
/info.
|
||||
============================ =============== =============================
|
||||
|
||||
[proxy-server]
|
||||
|
|
|
|||
|
|
@ -0,0 +1,251 @@
|
|||
=======================
|
||||
Middleware and Metadata
|
||||
=======================
|
||||
|
||||
----------------
|
||||
Using Middleware
|
||||
----------------
|
||||
|
||||
`Python WSGI Middleware`_ (or just "middleware") can be used to "wrap"
|
||||
the request and response of a Python WSGI application (i.e. a webapp,
|
||||
or REST/HTTP API), like Swift's WSGI servers (proxy-server,
|
||||
account-server, container-server, object-server). Swift uses middleware
|
||||
to add (sometimes optional) behaviors to the Swift WSGI servers.
|
||||
|
||||
.. _Python WSGI Middleware: http://www.python.org/dev/peps/pep-0333/#middleware-components-that-play-both-sides
|
||||
|
||||
Middleware can be added to the Swift WSGI servers by modifying their
|
||||
`paste`_ configuration file. The majority of Swift middleware is applied
|
||||
to the :ref:`proxy-server`.
|
||||
|
||||
.. _paste: http://pythonpaste.org/
|
||||
|
||||
Given the following basic configuration::
|
||||
|
||||
[DEFAULT]
|
||||
log_level = DEBUG
|
||||
user = <your-user-name>
|
||||
|
||||
[pipeline:main]
|
||||
pipeline = proxy-server
|
||||
|
||||
[app:proxy-server]
|
||||
use = egg:swift#proxy
|
||||
|
||||
You could add the :ref:`healthcheck` middleware by adding a section for
|
||||
that filter and adding it to the pipeline::
|
||||
|
||||
[DEFAULT]
|
||||
log_level = DEBUG
|
||||
user = <your-user-name>
|
||||
|
||||
[pipeline:main]
|
||||
pipeline = healthcheck proxy-server
|
||||
|
||||
[filter:healthcheck]
|
||||
use = egg:swift#healthcheck
|
||||
|
||||
[app:proxy-server]
|
||||
use = egg:swift#proxy
|
||||
|
||||
|
||||
Some middleware is required and will be inserted into your pipeline
|
||||
automatically by core swift code (e.g. the proxy-server will insert
|
||||
:ref:`catch_errors` and :ref:`gatekeeper` at the start of the pipeline if they
|
||||
are not already present). You can see which features are available on a given
|
||||
Swift endpoint (including middleware) using the :ref:`discoverability`
|
||||
interface.
|
||||
|
||||
|
||||
----------------------------
|
||||
Creating Your Own Middleware
|
||||
----------------------------
|
||||
|
||||
The best way to see how to write middleware is to look at examples.
|
||||
|
||||
Many optional features in Swift are implemented as
|
||||
:ref:`common_middleware` and provided in ``swift.common.middleware``, but
|
||||
Swift middleware may be packaged and distributed as a separate project.
|
||||
Some examples are listed on the :ref:`associated_projects` page.
|
||||
|
||||
A contrived middleware example that modifies request behavior by
|
||||
inspecting custom HTTP headers (e.g. X-Webhook) and uses :ref:`sysmeta`
|
||||
to persist data to backend storage as well as common patterns like a
|
||||
:func:`.get_container_info` cache/query and :func:`.wsgify` decorator is
|
||||
presented below::
|
||||
|
||||
from swift.common.http import is_success
|
||||
from swift.common.swob import wsgify
|
||||
from swift.common.utils import split_path, get_logger
|
||||
from swift.common.request_helper import get_sys_meta_prefix
|
||||
from swift.proxy.controllers.base import get_container_info
|
||||
|
||||
from eventlet import Timeout
|
||||
from eventlet.green import urllib2
|
||||
|
||||
# x-container-sysmeta-webhook
|
||||
SYSMETA_WEBHOOK = get_sys_meta_prefix('container') + 'webhook'
|
||||
|
||||
|
||||
class WebhookMiddleware(object):
|
||||
|
||||
def __init__(self, app, conf):
|
||||
self.app = app
|
||||
self.logger = get_logger(conf, log_route='webhook')
|
||||
|
||||
@wsgify
|
||||
def __call__(self, req):
|
||||
obj = None
|
||||
try:
|
||||
(version, account, container, obj) = \
|
||||
split_path(req.path_info, 4, 4, True)
|
||||
except ValueError:
|
||||
# not an object request
|
||||
pass
|
||||
if 'x-webhook' in req.headers:
|
||||
# translate user's request header to sysmeta
|
||||
req.headers[SYSMETA_WEBHOOK] = \
|
||||
req.headers['x-webhook']
|
||||
if 'x-remove-webhook' in req.headers:
|
||||
# empty value will tombstone sysmeta
|
||||
req.headers[SYSMETA_WEBHOOK] = ''
|
||||
# account and object storage will ignore x-container-sysmeta-*
|
||||
resp = req.get_response(self.app)
|
||||
if obj and is_success(resp.status_int) and req.method == 'PUT':
|
||||
container_info = get_container_info(req.environ, self.app)
|
||||
# container_info may have our new sysmeta key
|
||||
webhook = container_info['sysmeta'].get('webhook')
|
||||
if webhook:
|
||||
# create a POST request with obj name as body
|
||||
webhook_req = urllib2.Request(webhook, data=obj)
|
||||
with Timeout(20):
|
||||
try:
|
||||
urllib2.urlopen(webhook_req).read()
|
||||
except (Exception, Timeout):
|
||||
self.logger.exception(
|
||||
'failed POST to webhook %s' % webhook)
|
||||
else:
|
||||
self.logger.info(
|
||||
'successfully called webhook %s' % webhook)
|
||||
if 'x-container-sysmeta-webhook' in resp.headers:
|
||||
# translate sysmeta from the backend resp to
|
||||
# user-visible client resp header
|
||||
resp.headers['x-webhook'] = resp.headers[SYSMETA_WEBHOOK]
|
||||
return resp
|
||||
|
||||
|
||||
def webhook_factory(global_conf, **local_conf):
|
||||
conf = global_conf.copy()
|
||||
conf.update(local_conf)
|
||||
def webhook_filter(app, conf):
|
||||
return WebhookMiddleware(app)
|
||||
return webhook_filter
|
||||
|
||||
In practice this middleware will call the url stored on the container as
|
||||
X-Webhook on all successful object uploads.
|
||||
|
||||
If this example was at ``<swift-repo>/swift/common/middleware/webhook.py`` -
|
||||
you could add it to your proxy by creating a new filter section and
|
||||
adding it to the pipeline::
|
||||
|
||||
[DEFAULT]
|
||||
log_level = DEBUG
|
||||
user = <your-user-name>
|
||||
|
||||
[pipeline:main]
|
||||
pipeline = healthcheck webhook proxy-server
|
||||
|
||||
[filter:webhook]
|
||||
paste.filter_factory = swift.common.middleware.webhook:webhook_factory
|
||||
|
||||
[filter:healthcheck]
|
||||
use = egg:swift#healthcheck
|
||||
|
||||
[app:proxy-server]
|
||||
use = egg:swift#proxy
|
||||
|
||||
Most python packages expose middleware as entrypoints. See `PasteDeploy`_
|
||||
documentation for more information about the syntax of the ``use`` option.
|
||||
All middleware included with Swift is installed to support the ``egg:swift``
|
||||
syntax.
|
||||
|
||||
.. _PasteDeploy: http://pythonpaste.org/deploy/#egg-uris
|
||||
|
||||
Middleware may advertize its availability and capabilities via Swift's
|
||||
:ref:`discoverability` support by using
|
||||
:func:`.register_swift_info`::
|
||||
|
||||
from swift.common.utils import register_swift_info
|
||||
def webhook_factory(global_conf, **local_conf):
|
||||
register_swift_info('webhook')
|
||||
def webhook_filter(app):
|
||||
return WebhookMiddleware(app)
|
||||
return webhook_filter
|
||||
|
||||
|
||||
--------------
|
||||
Swift Metadata
|
||||
--------------
|
||||
|
||||
Generally speaking metadata is information about a resource that is
|
||||
associated with the resource but is not the data contained in the
|
||||
resource itself - which is set and retrieved via HTTP headers. (e.g. the
|
||||
"Content-Type" of a Swift object that is returned in HTTP response
|
||||
headers)
|
||||
|
||||
All user resources in Swift (i.e. account, container, objects) can have
|
||||
user metadata associated with them. Middleware may also persist custom
|
||||
metadata to accounts and containers safely using System Metadata. Some
|
||||
core swift features which predate sysmeta have added exceptions for
|
||||
custom non-user metadata headers (e.g. :ref:`acls`,
|
||||
:ref:`large-objects`)
|
||||
|
||||
^^^^^^^^^^^^^
|
||||
User Metadata
|
||||
^^^^^^^^^^^^^
|
||||
|
||||
User metadata takes the form of ``X-<type>-Meta-<key>: <value>``, where
|
||||
``<type>`` depends on the resources type (i.e. Account, Container, Object)
|
||||
and ``<key>`` and ``<value>`` are set by the client.
|
||||
|
||||
User metadata should generally be reserved for use by the client or
|
||||
client applications. An perfect example use-case for user metadata is
|
||||
`python-swiftclient`_'s ``X-Object-Meta-Mtime`` which it stores on
|
||||
object it uploads to implement its ``--changed`` option which will only
|
||||
upload files that have changed since the last upload.
|
||||
|
||||
.. _python-swiftclient: https://github.com/openstack/python-swiftclient
|
||||
|
||||
New middleware should avoid storing metadata within the User Metadata
|
||||
namespace to avoid potential conflict with existing user metadata when
|
||||
introducing new metadata keys. An example of legacy middleware that
|
||||
borrows the user metadata namespace is :ref:`tempurl`. An example of
|
||||
middleware which uses custom non-user metadata to avoid the user
|
||||
metadata namespace is :ref:`slo-doc`.
|
||||
|
||||
.. _sysmeta:
|
||||
|
||||
^^^^^^^^^^^^^^^
|
||||
System Metadata
|
||||
^^^^^^^^^^^^^^^
|
||||
|
||||
System metadata takes the form of ``X-<type>-Sysmeta-<key>: <value>``,
|
||||
where ``<type>`` depends on the resources type (i.e. Account, Container,
|
||||
Object) and ``<key>`` and ``<value>`` are set by trusted code running in a
|
||||
Swift WSGI Server.
|
||||
|
||||
All headers on client requests in the form of ``X-<type>-Sysmeta-<key>``
|
||||
will be dropped from the request before being processed by any
|
||||
middleware. All headers on responses from back-end systems in the form
|
||||
of ``X-<type>-Sysmeta-<key>`` will be removed after all middleware has
|
||||
processed the response but before the response is sent to the client.
|
||||
See :ref:`gatekeeper` middleware for more information.
|
||||
|
||||
System metadata provides a means to store potentially private custom
|
||||
metadata with associated Swift resources in a safe and secure fashion
|
||||
without actually having to plumb custom metadata through the core swift
|
||||
servers. The incoming filtering ensures that the namespace can not be
|
||||
modified directly by client requests, and the outgoing filter ensures
|
||||
that removing middleware that uses a specific system metadata key
|
||||
renders it benign. New middleware should take advantage of system
|
||||
metadata.
|
||||
|
|
@ -66,6 +66,7 @@ Developer Documentation
|
|||
development_guidelines
|
||||
development_saio
|
||||
development_auth
|
||||
development_middleware
|
||||
development_ondisk_backends
|
||||
|
||||
Administrator Documentation
|
||||
|
|
@ -93,6 +94,7 @@ Source Documentation
|
|||
db
|
||||
object
|
||||
misc
|
||||
middleware
|
||||
|
||||
|
||||
Indices and tables
|
||||
|
|
|
|||
|
|
@ -0,0 +1,196 @@
|
|||
.. _common_middleware:
|
||||
|
||||
**********
|
||||
Middleware
|
||||
**********
|
||||
|
||||
.. _common_tempauth:
|
||||
|
||||
TempAuth
|
||||
========
|
||||
|
||||
.. automodule:: swift.common.middleware.tempauth
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
KeystoneAuth
|
||||
============
|
||||
|
||||
.. automodule:: swift.common.middleware.keystoneauth
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
.. _healthcheck:
|
||||
|
||||
Healthcheck
|
||||
===========
|
||||
|
||||
.. automodule:: swift.common.middleware.healthcheck
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
.. _recon:
|
||||
|
||||
Recon
|
||||
===========
|
||||
|
||||
.. automodule:: swift.common.middleware.recon
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
.. _memecached:
|
||||
|
||||
Ratelimit
|
||||
=========
|
||||
|
||||
.. automodule:: swift.common.middleware.ratelimit
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
StaticWeb
|
||||
=========
|
||||
|
||||
.. automodule:: swift.common.middleware.staticweb
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
.. _tempurl:
|
||||
|
||||
TempURL
|
||||
=======
|
||||
|
||||
.. automodule:: swift.common.middleware.tempurl
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
FormPost
|
||||
========
|
||||
|
||||
.. automodule:: swift.common.middleware.formpost
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
Domain Remap
|
||||
============
|
||||
|
||||
.. automodule:: swift.common.middleware.domain_remap
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
CNAME Lookup
|
||||
============
|
||||
|
||||
.. automodule:: swift.common.middleware.cname_lookup
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
Cross Domain Policies
|
||||
=====================
|
||||
|
||||
.. automodule:: swift.common.middleware.crossdomain
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
Name Check (Forbidden Character Filter)
|
||||
=======================================
|
||||
|
||||
.. automodule:: swift.common.middleware.name_check
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
Memcache
|
||||
========
|
||||
|
||||
.. automodule:: swift.common.middleware.memcache
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
Proxy Logging
|
||||
=============
|
||||
|
||||
.. automodule:: swift.common.middleware.proxy_logging
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
.. _catch_errors:
|
||||
|
||||
CatchErrors
|
||||
=============
|
||||
|
||||
.. automodule:: swift.common.middleware.catch_errors
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
.. _gatekeeper:
|
||||
|
||||
GateKeeper
|
||||
=============
|
||||
|
||||
.. automodule:: swift.common.middleware.gatekeeper
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
Bulk Operations (Delete and Archive Auto Extraction)
|
||||
====================================================
|
||||
|
||||
.. automodule:: swift.common.middleware.bulk
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
Container Quotas
|
||||
================
|
||||
|
||||
.. automodule:: swift.common.middleware.container_quotas
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
Account Quotas
|
||||
==============
|
||||
|
||||
.. automodule:: swift.common.middleware.account_quotas
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
.. _slo-doc:
|
||||
|
||||
Static Large Objects
|
||||
====================
|
||||
|
||||
.. automodule:: swift.common.middleware.slo
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
List Endpoints
|
||||
==============
|
||||
|
||||
.. automodule:: swift.common.middleware.list_endpoints
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
Container Sync Middleware
|
||||
=========================
|
||||
|
||||
.. automodule:: swift.common.middleware.container_sync
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
.. _discoverability:
|
||||
|
||||
Discoverability
|
||||
===============
|
||||
|
||||
Swift will by default provide clients with an interface providing details
|
||||
about the installation. Unless disabled (i.e ``expose_info=false`` in
|
||||
:ref:`proxy-server-config`), a GET request to ``/info`` will return configuration
|
||||
data in JSON format. An example response::
|
||||
|
||||
{"swift": {"version": "1.11.0"}, "staticweb": {}, "tempurl": {}}
|
||||
|
||||
This would signify to the client that swift version 1.11.0 is running and that
|
||||
staticweb and tempurl are available in this installation.
|
||||
|
||||
There may be administrator-only information available via ``/info``. To
|
||||
retrieve it, one must use an HMAC-signed request, similar to TempURL.
|
||||
The signature may be produced like so::
|
||||
|
||||
swift-temp-url GET 3600 /info secret 2>/dev/null | sed s/temp_url/swiftinfo/g
|
||||
|
|
@ -33,25 +33,8 @@ Utils
|
|||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
.. _common_tempauth:
|
||||
|
||||
TempAuth
|
||||
========
|
||||
|
||||
.. automodule:: swift.common.middleware.tempauth
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
.. _acls:
|
||||
|
||||
KeystoneAuth
|
||||
============
|
||||
|
||||
.. automodule:: swift.common.middleware.keystoneauth
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
|
||||
ACLs
|
||||
====
|
||||
|
||||
|
|
@ -68,6 +51,25 @@ WSGI
|
|||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
.. _swob:
|
||||
|
||||
Swob
|
||||
====
|
||||
|
||||
.. automodule:: swift.common.swob
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
.. _request_helpers:
|
||||
|
||||
Request Helpers
|
||||
===============
|
||||
|
||||
.. automodule:: swift.common.request_helpers
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
|
||||
.. _direct_client:
|
||||
|
||||
Direct Client
|
||||
|
|
@ -97,26 +99,6 @@ Buffered HTTP
|
|||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
.. _healthcheck:
|
||||
|
||||
Healthcheck
|
||||
===========
|
||||
|
||||
.. automodule:: swift.common.middleware.healthcheck
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
.. _recon:
|
||||
|
||||
Recon
|
||||
===========
|
||||
|
||||
.. automodule:: swift.common.middleware.recon
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
.. _memecached:
|
||||
|
||||
MemCacheD
|
||||
=========
|
||||
|
||||
|
|
@ -124,99 +106,6 @@ MemCacheD
|
|||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
Manager
|
||||
=========
|
||||
|
||||
.. automodule:: swift.common.manager
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
Ratelimit
|
||||
=========
|
||||
|
||||
.. automodule:: swift.common.middleware.ratelimit
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
StaticWeb
|
||||
=========
|
||||
|
||||
.. automodule:: swift.common.middleware.staticweb
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
TempURL
|
||||
=======
|
||||
|
||||
.. automodule:: swift.common.middleware.tempurl
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
FormPost
|
||||
========
|
||||
|
||||
.. automodule:: swift.common.middleware.formpost
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
Domain Remap
|
||||
============
|
||||
|
||||
.. automodule:: swift.common.middleware.domain_remap
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
CNAME Lookup
|
||||
============
|
||||
|
||||
.. automodule:: swift.common.middleware.cname_lookup
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
Proxy Logging
|
||||
=============
|
||||
|
||||
.. automodule:: swift.common.middleware.proxy_logging
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
Bulk Operations (Delete and Archive Auto Extraction)
|
||||
====================================================
|
||||
|
||||
.. automodule:: swift.common.middleware.bulk
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
Container Quotas
|
||||
================
|
||||
|
||||
.. automodule:: swift.common.middleware.container_quotas
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
Account Quotas
|
||||
==============
|
||||
|
||||
.. automodule:: swift.common.middleware.account_quotas
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
.. _slo-doc:
|
||||
|
||||
Static Large Objects
|
||||
====================
|
||||
|
||||
.. automodule:: swift.common.middleware.slo
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
List Endpoints
|
||||
==============
|
||||
|
||||
.. automodule:: swift.common.middleware.list_endpoints
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
Container Sync Realms
|
||||
=====================
|
||||
|
||||
|
|
@ -224,28 +113,9 @@ Container Sync Realms
|
|||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
Container Sync Middleware
|
||||
=========================
|
||||
Manager
|
||||
=========
|
||||
|
||||
.. automodule:: swift.common.middleware.container_sync
|
||||
.. automodule:: swift.common.manager
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
Discoverability
|
||||
===============
|
||||
|
||||
Swift can optionally be configured to provide clients with an interface
|
||||
providing details about the installation. If configured, a GET request to
|
||||
/info will return configuration data in JSON format. An example
|
||||
response::
|
||||
|
||||
{"swift": {"version": "1.8.1"}, "staticweb": {}, "tempurl": {}}
|
||||
|
||||
This would signify to the client that swift version 1.8.1 is running and that
|
||||
staticweb and tempurl are available in this installation.
|
||||
|
||||
There may be administrator-only information available via /info. To
|
||||
retrieve it, one must use an HMAC-signed request, similar to TempURL.
|
||||
The signature may be produced like so:
|
||||
|
||||
swift-temp-url GET 3600 /info secret 2>/dev/null | sed s/temp_url/swiftinfo/g
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
.. _large-objects:
|
||||
|
||||
====================
|
||||
Large Object Support
|
||||
====================
|
||||
|
|
|
|||
|
|
@ -13,3 +13,40 @@ Proxy Server
|
|||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
|
||||
.. _proxy-controllers:
|
||||
|
||||
Proxy Controllers
|
||||
=================
|
||||
|
||||
Base
|
||||
~~~~
|
||||
|
||||
.. automodule:: swift.proxy.controllers.base
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
|
||||
Account
|
||||
~~~~~~~
|
||||
|
||||
.. automodule:: swift.proxy.controllers.account
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
|
||||
Container
|
||||
~~~~~~~~~
|
||||
|
||||
.. automodule:: swift.proxy.controllers.container
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
|
||||
Object
|
||||
~~~~~~
|
||||
|
||||
.. automodule:: swift.proxy.controllers.obj
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
|
|
|
|||
|
|
@ -405,12 +405,12 @@ class InternalClient(object):
|
|||
:param account: The container's account.
|
||||
:param container: Container to check.
|
||||
|
||||
:returns : True if container exists, false otherwise.
|
||||
|
||||
:raises UnexpectedResponse: Exception raised when requests fail
|
||||
to get a response with an acceptable status
|
||||
:raises Exception: Exception is raised when code fails in an
|
||||
unexpected way.
|
||||
|
||||
:returns : True if container exists, false otherwise.
|
||||
"""
|
||||
|
||||
path = self.make_path(account, container)
|
||||
|
|
|
|||
|
|
@ -36,11 +36,9 @@ from swift.common.utils import get_logger
|
|||
from swift.common.request_helpers import remove_items, get_sys_meta_prefix
|
||||
import re
|
||||
|
||||
"""
|
||||
A list of python regular expressions that will be used to
|
||||
match against inbound request headers. Matching headers will
|
||||
be removed from the request.
|
||||
"""
|
||||
#: A list of python regular expressions that will be used to
|
||||
#: match against inbound request headers. Matching headers will
|
||||
#: be removed from the request.
|
||||
# Exclude headers starting with a sysmeta prefix.
|
||||
# If adding to this list, note that these are regex patterns,
|
||||
# so use a trailing $ to constrain to an exact header match
|
||||
|
|
@ -52,11 +50,9 @@ inbound_exclusions = [get_sys_meta_prefix('account'),
|
|||
# for system metadata being applied to objects
|
||||
|
||||
|
||||
"""
|
||||
A list of python regular expressions that will be used to
|
||||
match against outbound response headers. Matching headers will
|
||||
be removed from the response.
|
||||
"""
|
||||
#: A list of python regular expressions that will be used to
|
||||
#: match against outbound response headers. Matching headers will
|
||||
#: be removed from the response.
|
||||
outbound_exclusions = inbound_exclusions
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -15,14 +15,17 @@
|
|||
'''
|
||||
Created on February 27, 2012
|
||||
|
||||
A filter that disallows any paths that contain defined forbidden characters
|
||||
or that exceed a defined length.
|
||||
A filter that disallows any paths that contain defined forbidden characters or
|
||||
that exceed a defined length.
|
||||
|
||||
Place in proxy filter before proxy, e.g.
|
||||
Place early in the proxy-server pipeline after the left-most occurrence of the
|
||||
``proxy-logging`` middleware (if present) and before the final
|
||||
``proxy-logging`` middleware (if present) or the ``proxy-serer`` app itself,
|
||||
e.g.::
|
||||
|
||||
[pipeline:main]
|
||||
pipeline = catch_errors healthcheck name_check cache ratelimit tempauth sos
|
||||
proxy-logging proxy-server
|
||||
pipeline = catch_errors healthcheck proxy-logging name_check cache \
|
||||
ratelimit tempauth sos proxy-logging proxy-server
|
||||
|
||||
[filter:name_check]
|
||||
use = egg:swift#name_check
|
||||
|
|
|
|||
|
|
@ -40,7 +40,7 @@ added. For example::
|
|||
use = egg:swift#staticweb
|
||||
|
||||
Any publicly readable containers (for example, ``X-Container-Read: .r:*``, see
|
||||
`acls`_ for more information on this) will be checked for
|
||||
:ref:`acls` for more information on this) will be checked for
|
||||
X-Container-Meta-Web-Index and X-Container-Meta-Web-Error header values::
|
||||
|
||||
X-Container-Meta-Web-Index <index.name>
|
||||
|
|
|
|||
|
|
@ -186,7 +186,8 @@ def remove_items(headers, condition):
|
|||
|
||||
:param headers: a dict of headers
|
||||
:param condition: a function that will be passed the header key as a
|
||||
single argument and should return True if the header is to be removed.
|
||||
single argument and should return True if the header
|
||||
is to be removed.
|
||||
:returns: a dict, possibly empty, of headers that have been removed
|
||||
"""
|
||||
removed = {}
|
||||
|
|
|
|||
|
|
@ -223,7 +223,7 @@ class PipelineWrapper(object):
|
|||
:param entry_point_name: entry point of middleware or app (Swift only)
|
||||
|
||||
:returns: True if entry_point_name is first in pipeline, False
|
||||
otherwise
|
||||
otherwise
|
||||
"""
|
||||
try:
|
||||
first_ctx = self.context.filter_contexts[0]
|
||||
|
|
|
|||
|
|
@ -250,8 +250,11 @@ def get_object_info(env, app, path=None, swift_source=None):
|
|||
"""
|
||||
Get the info structure for an object, based on env and app.
|
||||
This is useful to middlewares.
|
||||
Note: This call bypasses auth. Success does not imply that the
|
||||
request has authorization to the object.
|
||||
|
||||
.. note::
|
||||
|
||||
This call bypasses auth. Success does not imply that the request has
|
||||
authorization to the object.
|
||||
"""
|
||||
(version, account, container, obj) = \
|
||||
split_path(path or env['PATH_INFO'], 4, 4, True)
|
||||
|
|
@ -266,8 +269,11 @@ def get_container_info(env, app, swift_source=None):
|
|||
"""
|
||||
Get the info structure for a container, based on env and app.
|
||||
This is useful to middlewares.
|
||||
Note: This call bypasses auth. Success does not imply that the
|
||||
request has authorization to the account.
|
||||
|
||||
.. note::
|
||||
|
||||
This call bypasses auth. Success does not imply that the request has
|
||||
authorization to the account.
|
||||
"""
|
||||
(version, account, container, unused) = \
|
||||
split_path(env['PATH_INFO'], 3, 4, True)
|
||||
|
|
@ -282,8 +288,11 @@ def get_account_info(env, app, swift_source=None):
|
|||
"""
|
||||
Get the info structure for an account, based on env and app.
|
||||
This is useful to middlewares.
|
||||
Note: This call bypasses auth. Success does not imply that the
|
||||
request has authorization to the container.
|
||||
|
||||
.. note::
|
||||
|
||||
This call bypasses auth. Success does not imply that the request has
|
||||
authorization to the container.
|
||||
"""
|
||||
(version, account, _junk, _junk) = \
|
||||
split_path(env['PATH_INFO'], 2, 4, True)
|
||||
|
|
@ -591,9 +600,11 @@ class GetOrHeadHandler(object):
|
|||
def fast_forward(self, num_bytes):
|
||||
"""
|
||||
Will skip num_bytes into the current ranges.
|
||||
|
||||
:params num_bytes: the number of bytes that have already been read on
|
||||
this request. This will change the Range header
|
||||
so that the next req will start where it left off.
|
||||
|
||||
:raises NotImplementedError: if this is a multirange request
|
||||
:raises ValueError: if invalid range header
|
||||
:raises HTTPRequestedRangeNotSatisfiable: if begin + num_bytes
|
||||
|
|
|
|||
Loading…
Reference in New Issue