openstack
/
swift
Code Issues Proposed changes

Merge "Add handler for CORS "actual requests""

This commit is contained in:
Jenkins 2013-01-23 22:18:11 +00:00 committed by Gerrit Code Review
parent 5a4d488729 89ee10bd92
commit 77a562dbf1
7 changed files with 447 additions and 64 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

151
doc/source/cors.rst Normal file
View File

@ -0,0 +1,151 @@
====
CORS
====
CORS_ is a mechanisim to allow code running in a browser (Javascript for
example) make requests to a domain other then the one from where it originated.
Swift supports CORS requests to containers and objects.
CORS metadata is held on the container only. The values given apply to the
container itself and all objects within it.
The supported headers are,
+---------------------------------------------+-------------------------------+
|Metadata | Use |
+==============================================+==============================+
|X-Container-Meta-Access-Control-Allow-Origin | Origins to be allowed to |
| | make Cross Origin Requests, |
| | space separated. |
+----------------------------------------------+------------------------------+
|X-Container-Meta-Access-Control-Max-Age | Max age for the Origin to |
| | hold the preflight results. |
+----------------------------------------------+------------------------------+
|X-Container-Meta-Access-Control-Allow-Headers | Headers to be allowed in |
| | actual request by browser, |
| | space seperated. |
+----------------------------------------------+------------------------------+
|X-Container-Meta-Access-Control-Expose-Headers| Headers exposed to the user |
| | agent (e.g. browser) in the |
| | the actual request response. |
| | Space seperated. |
+----------------------------------------------+------------------------------+
Before a browser issues an actual request it may issue a `preflight request`_.
The preflight request is an OPTIONS call to verify the Origin is allowed to
make the request. The sequence of events are,
* Browser makes OPTIONS request to Swift
* Swift returns 200/401 to browser based on allowed origins
* If 200, browser makes the "actual request" to Swift, i.e. PUT, POST, DELETE,
HEAD, GET
When a browser receives a response to an actual request it only exposes those
headers listed in the ``Access-Control-Expose-Headers`` header. By default Swift
returns the following values for this header,
* "simple response headers" as listed on
http://www.w3.org/TR/cors/#simple-response-header
* the headers ``etag``, ``x-timestamp``, ``x-trans-id``
* all metadata headers (``X-Container-Meta-*`` for containers and
``X-Object-Meta-*`` for objects)
* headers listed in ``X-Container-Meta-Access-Control-Expose-Headers``
-----------------
Sample Javascript
-----------------
To see some CORS Javascript in action download the `test CORS page`_ (source
below). Host it on a webserver and take note of the protocol and hostname
(origin) you'll be using to request the page, e.g. http://localhost.
Locate a container you'd like to query. Needless to say the Swift cluster
hosting this container should have CORS support. Append the origin of the
test page to the container's ``X-Container-Meta-Access-Control-Allow-Origin``
header,::
curl -X POST -H 'X-Auth-Token: xxx' \
-H 'X-Container-Meta-Access-Control-Allow-Origin: http://localhost' \
http://192.168.56.3:8080/v1/AUTH_test/cont1
At this point the container is now accessable to CORS clients hosted on
http://localhost. Open the test CORS page in your browser.
#. Populate the Token field
#. Populate the URL field with the URL of either a container or object
#. Select the request method
#. Hit Submit
Assuming the request succeeds you should see the response header and body. If
something went wrong the response status will be 0.
.. _test CORS page:
Test CORS Page
--------------
::
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Test CORS</title>
</head>
<body>
Token<br><input id="token" type="text" size="64"><br><br>
Method<br>
<select id="method">
<option value="GET">GET</option>
<option value="HEAD">HEAD</option>
<option value="POST">POST</option>
<option value="DELETE">DELETE</option>
<option value="PUT">PUT</option>
</select><br><br>
URL (Container or Object)<br><input id="url" size="64" type="text"><br><br>
<input id="submit" type="button" value="Submit" onclick="submit(); return false;">
<pre id="response_headers"></pre>
<p>
<hr>
<pre id="response_body"></pre>
<script type="text/javascript">
function submit() {
var token = document.getElementById('token').value;
var method = document.getElementById('method').value;
var url = document.getElementById('url').value;
document.getElementById('response_headers').textContent = null;
document.getElementById('response_body').textContent = null;
var request = new XMLHttpRequest();
request.onreadystatechange = function (oEvent) {
if (request.readyState == 4) {
responseHeaders = 'Status: ' + request.status;
responseHeaders = responseHeaders + '\nStatus Text: ' + request.statusText;
responseHeaders = responseHeaders + '\n\n' + request.getAllResponseHeaders();
document.getElementById('response_headers').textContent = responseHeaders;
document.getElementById('response_body').textContent = request.responseText;
}
}
request.open(method, url);
request.setRequestHeader('X-Auth-Token', token);
request.send(null);
}
</script>
</body>
</html>
.. _CORS: https://developer.mozilla.org/en-US/docs/HTTP/Access_control_CORS
.. _preflight request: https://developer.mozilla.org/en-US/docs/HTTP/Access_control_CORS#Preflighted_requests

1
doc/source/index.rst
View File

@ -53,6 +53,7 @@ Overview and Concepts
overview_object_versioning
overview_container_sync
overview_expiring_objects
cors
associated_projects
Developer Documentation

31
doc/source/misc.rst
View File

@ -172,34 +172,3 @@ Proxy Logging
:members:
:show-inheritance:
CORS Headers
============
Cross Origin RequestS or CORS allows the browser to make requests against
Swift from another origin via the browser. This enables the use of HTML5
forms and javascript uploads to swift. The owner of a container can set
three headers:
+---------------------------------------------+-------------------------------+
|Metadata | Use |
+=============================================+===============================+
|X-Container-Meta-Access-Control-Allow-Origin | Origins to be allowed to |
| | make Cross Origin Requests, |
| | space separated |
+---------------------------------------------+-------------------------------+
|X-Container-Meta-Access-Control-Max-Age | Max age for the Origin to |
| | hold the preflight results. |
+---------------------------------------------+-------------------------------+
|X-Container-Meta-Access-Control-Allow-Headers| Headers to be allowed in |
| | actual request by browser. |
+---------------------------------------------+-------------------------------+
When the browser does a request it can issue a preflight request. The
preflight request is the OPTIONS call that verifies the Origin is allowed
to make the request.
* Browser makes OPTIONS request to Swift
* Swift returns 200/401 to browser based on allowed origins
* If 200, browser makes PUT, POST, DELETE, HEAD, GET request to Swift
CORS should be used in conjunction with TempURL and FormPost.

137
swift/proxy/controllers/base.py
View File

@ -113,6 +113,8 @@ def headers_to_container_info(headers, status_int=HTTP_OK):
'x-container-meta-access-control-allow-origin'),
'allow_headers': headers.get(
'x-container-meta-access-control-allow-headers'),
'expose_headers': headers.get(
'x-container-meta-access-control-expose-headers'),
'max_age': headers.get(
'x-container-meta-access-control-max-age')
},
@ -122,6 +124,70 @@ def headers_to_container_info(headers, status_int=HTTP_OK):
}
def cors_validation(func):
"""
Decorator to check if the request is a CORS request and if so, if it's
valid.
:param func: function to check
"""
@functools.wraps(func)
def wrapped(*a, **kw):
controller = a[0]
req = a[1]
# The logic here was interpreted from
# http://www.w3.org/TR/cors/#resource-requests
# Is this a CORS request?
req_origin = req.headers.get('Origin', None)
if req_origin:
# Yes, this is a CORS request so test if the origin is allowed
container_info = \
controller.container_info(controller.account_name,
controller.container_name)
cors_info = container_info.get('cors', {})
if not controller.is_origin_allowed(cors_info, req_origin):
# invalid CORS request
return Response(status=HTTP_UNAUTHORIZED)
# Call through to the decorated method
resp = func(*a, **kw)
# Expose,
# - simple response headers,
# http://www.w3.org/TR/cors/#simple-response-header
# - swift specific: etag, x-timestamp, x-trans-id
# - user metadata headers
# - headers provided by the user in
# x-container-meta-access-control-expose-headers
expose_headers = ['cache-control', 'content-language',
'content-type', 'expires', 'last-modified',
'pragma', 'etag', 'x-timestamp', 'x-trans-id']
for header in resp.headers:
if header.startswith('x-container-meta') or \
header.startswith('x-object-meta'):
expose_headers.append(header.lower())
if cors_info.get('expose_headers'):
expose_headers.extend(
[a.strip()
for a in cors_info['expose_headers'].split(' ')
if a.strip()])
resp.headers['Access-Control-Expose-Headers'] = \
', '.join(expose_headers)
# The user agent won't process the response if the Allow-Origin
# header isn't included
resp.headers['Access-Control-Allow-Origin'] = req_origin
return resp
else:
# Not a CORS request so make the call as normal
return func(*a, **kw)
return wrapped
class Controller(object):
"""Base WSGI controller class for the proxy"""
server_type = 'Base'
@ -694,49 +760,76 @@ class Controller(object):
return self.best_response(req, statuses, reasons, bodies,
'%s %s' % (server_type, req.method))
def OPTIONS_base(self, req):
def is_origin_allowed(self, cors_info, origin):
"""
Is the given Origin allowed to make requests to this resource
:param cors_info: the resource's CORS related metadata headers
:param origin: the origin making the request
:return: True or False
"""
allowed_origins = set()
if cors_info.get('allow_origin'):
allowed_origins.update(
[a.strip()
for a in cors_info['allow_origin'].split(' ')
if a.strip()])
if self.app.cors_allow_origin:
allowed_origins.update(self.app.cors_allow_origin)
return origin in allowed_origins or '*' in allowed_origins
@public
def OPTIONS(self, req):
"""
Base handler for OPTIONS requests
:param req: swob.Request object
:returns: swob.Response object
"""
# Prepare the default response
headers = {'Allow': ', '.join(self.allowed_methods)}
resp = Response(status=200, request=req,
headers=headers)
resp = Response(status=200, request=req, headers=headers)
# If this isn't a CORS pre-flight request then return now
req_origin_value = req.headers.get('Origin', None)
if not req_origin_value:
# NOT a CORS request
return resp
# CORS preflight request
# This is a CORS preflight request so check it's allowed
try:
container_info = \
self.container_info(self.account_name, self.container_name)
except AttributeError:
container_info = {}
# This should only happen for requests to the Account. A future
# change could allow CORS requests to the Account level as well.
return resp
cors = container_info.get('cors', {})
allowed_origins = set()
if cors.get('allow_origin'):
allowed_origins.update(cors['allow_origin'].split(' '))
if self.app.cors_allow_origin:
allowed_origins.update(self.app.cors_allow_origin)
if (req_origin_value not in allowed_origins and
'*' not in allowed_origins) or (
# If the CORS origin isn't allowed return a 401
if not self.is_origin_allowed(cors, req_origin_value) or (
req.headers.get('Access-Control-Request-Method') not in
self.allowed_methods):
resp.status = HTTP_UNAUTHORIZED
return resp # CORS preflight request that isn't valid
return resp
# Always allow the x-auth-token header. This ensures
# clients can always make a request to the resource.
allow_headers = set()
if cors.get('allow_headers'):
allow_headers.update(
[a.strip()
for a in cors['allow_headers'].split(' ')
if a.strip()])
allow_headers.add('x-auth-token')
# Populate the response with the CORS preflight headers
headers['access-control-allow-origin'] = req_origin_value
if cors.get('max_age') is not None:
headers['access-control-max-age'] = cors.get('max_age')
headers['access-control-allow-methods'] = ', '.join(
self.allowed_methods)
if cors.get('allow_headers'):
headers['access-control-allow-headers'] = cors.get('allow_headers')
headers['access-control-allow-methods'] = \
', '.join(self.allowed_methods)
headers['access-control-allow-headers'] = ', '.join(allow_headers)
resp.headers = headers
return resp
@public
def OPTIONS(self, req):
return self.OPTIONS_base(req)
return resp

7
swift/proxy/controllers/container.py
View File

@ -32,7 +32,7 @@ from swift.common.utils import normalize_timestamp, public
from swift.common.constraints import check_metadata, MAX_CONTAINER_NAME_LENGTH
from swift.common.http import HTTP_ACCEPTED
from swift.proxy.controllers.base import Controller, delay_denial, \
get_container_memcache_key, headers_to_container_info
get_container_memcache_key, headers_to_container_info, cors_validation
from swift.common.swob import HTTPBadRequest, HTTPForbidden, \
HTTPNotFound
@ -95,17 +95,20 @@ class ContainerController(Controller):
@public
@delay_denial
@cors_validation
def GET(self, req):
"""Handler for HTTP GET requests."""
return self.GETorHEAD(req)
@public
@delay_denial
@cors_validation
def HEAD(self, req):
"""Handler for HTTP HEAD requests."""
return self.GETorHEAD(req)
@public
@cors_validation
def PUT(self, req):
"""HTTP PUT request handler."""
error_response = \
@ -151,6 +154,7 @@ class ContainerController(Controller):
return resp
@public
@cors_validation
def POST(self, req):
"""HTTP POST request handler."""
error_response = \
@ -177,6 +181,7 @@ class ContainerController(Controller):
return resp
@public
@cors_validation
def DELETE(self, req):
"""HTTP DELETE request handler."""
account_partition, accounts, container_count = \

9
swift/proxy/controllers/obj.py
View File

@ -49,7 +49,8 @@ from swift.common.http import is_success, is_client_error, HTTP_CONTINUE, \
HTTP_CREATED, HTTP_MULTIPLE_CHOICES, HTTP_NOT_FOUND, \
HTTP_INTERNAL_SERVER_ERROR, HTTP_SERVICE_UNAVAILABLE, \
HTTP_INSUFFICIENT_STORAGE
from swift.proxy.controllers.base import Controller, delay_denial
from swift.proxy.controllers.base import Controller, delay_denial, \
cors_validation
from swift.common.swob import HTTPAccepted, HTTPBadRequest, HTTPNotFound, \
HTTPPreconditionFailed, HTTPRequestEntityTooLarge, HTTPRequestTimeout, \
HTTPServerError, HTTPServiceUnavailable, Request, Response, \
@ -405,18 +406,21 @@ class ObjectController(Controller):
return resp
@public
@cors_validation
@delay_denial
def GET(self, req):
"""Handler for HTTP GET requests."""
return self.GETorHEAD(req)
@public
@cors_validation
@delay_denial
def HEAD(self, req):
"""Handler for HTTP HEAD requests."""
return self.GETorHEAD(req)
@public
@cors_validation
@delay_denial
def POST(self, req):
"""HTTP POST request handler."""
@ -541,6 +545,7 @@ class ObjectController(Controller):
_('Expect: 100-continue on %s') % path)
@public
@cors_validation
@delay_denial
def PUT(self, req):
"""HTTP PUT request handler."""
@ -838,6 +843,7 @@ class ObjectController(Controller):
return resp
@public
@cors_validation
@delay_denial
def DELETE(self, req):
"""HTTP DELETE request handler."""
@ -936,6 +942,7 @@ class ObjectController(Controller):
return resp
@public
@cors_validation
@delay_denial
def COPY(self, req):
"""HTTP COPY request handler."""

175
test/unit/proxy/test_server.py
View File

@ -52,7 +52,7 @@ from swift.common.utils import mkdirs, normalize_timestamp, NullLogger
from swift.common.wsgi import monkey_patch_mimetools
from swift.proxy.controllers.obj import SegmentedIterable
from swift.proxy.controllers.base import get_container_memcache_key, \
get_account_memcache_key
get_account_memcache_key, cors_validation
import swift.proxy.controllers
from swift.common.swob import Request, Response, HTTPNotFound, \
HTTPUnauthorized
@ -184,6 +184,17 @@ def teardown():
Request.__del__ = Request._orig_del
def sortHeaderNames(headerNames):
"""
Return the given string of header names sorted.
headerName: a comma-delimited list of header names
"""
headers = [a.strip() for a in headerNames.split(',') if a.strip()]
headers.sort()
return ', '.join(headers)
def fake_http_connect(*code_iter, **kwargs):
class FakeConn(object):
@ -3690,8 +3701,8 @@ class TestObjectController(unittest.TestCase):
7)
self.assertEquals('999', resp.headers['access-control-max-age'])
self.assertEquals(
'x-foo',
resp.headers['access-control-allow-headers'])
'x-auth-token, x-foo',
sortHeaderNames(resp.headers['access-control-allow-headers']))
req = Request.blank(
'/a/c/o.jpg',
{'REQUEST_METHOD': 'OPTIONS'},
@ -3750,8 +3761,73 @@ class TestObjectController(unittest.TestCase):
7)
self.assertEquals('999', resp.headers['access-control-max-age'])
self.assertEquals(
'x-foo',
resp.headers['access-control-allow-headers'])
'x-auth-token, x-foo',
sortHeaderNames(resp.headers['access-control-allow-headers']))
def test_CORS_invalid_origin(self):
with save_globals():
controller = proxy_server.ObjectController(self.app, 'a', 'c', 'o')
def stubContainerInfo(*args):
return {
'cors': {
'allow_origin': 'http://baz'
}
}
controller.container_info = stubContainerInfo
def objectGET(controller, req):
return Response()
req = Request.blank(
'/a/c/o.jpg',
{'REQUEST_METHOD': 'GET'},
headers={'Origin': 'http://foo.bar'})
resp = cors_validation(objectGET)(controller, req)
self.assertEquals(401, resp.status_int)
def test_CORS_valid(self):
with save_globals():
controller = proxy_server.ObjectController(self.app, 'a', 'c', 'o')
def stubContainerInfo(*args):
return {
'cors': {
'allow_origin': 'http://foo.bar'
}
}
controller.container_info = stubContainerInfo
def objectGET(controller, req):
return Response(headers={
'X-Object-Meta-Color': 'red',
'X-Super-Secret': 'hush',
})
req = Request.blank(
'/a/c/o.jpg',
{'REQUEST_METHOD': 'GET'},
headers={'Origin': 'http://foo.bar'})
resp = cors_validation(objectGET)(controller, req)
self.assertEquals(200, resp.status_int)
self.assertEquals('http://foo.bar',
resp.headers['access-control-allow-origin'])
self.assertEquals('red', resp.headers['x-object-meta-color'])
# X-Super-Secret is in the response, but not "exposed"
self.assertEquals('hush', resp.headers['x-super-secret'])
self.assertTrue('access-control-expose-headers' in resp.headers)
exposed = set(
h.strip() for h in
resp.headers['access-control-expose-headers'].split(','))
expected_exposed = set(['cache-control', 'content-language',
'content-type', 'expires', 'last-modified',
'pragma', 'etag', 'x-timestamp',
'x-trans-id', 'x-object-meta-color'])
self.assertEquals(expected_exposed, exposed)
class TestContainerController(unittest.TestCase):
@ -4296,8 +4372,8 @@ class TestContainerController(unittest.TestCase):
6)
self.assertEquals('999', resp.headers['access-control-max-age'])
self.assertEquals(
'x-foo',
resp.headers['access-control-allow-headers'])
'x-auth-token, x-foo',
sortHeaderNames(resp.headers['access-control-allow-headers']))
req = Request.blank(
'/a/c',
{'REQUEST_METHOD': 'OPTIONS'},
@ -4357,8 +4433,73 @@ class TestContainerController(unittest.TestCase):
6)
self.assertEquals('999', resp.headers['access-control-max-age'])
self.assertEquals(
'x-foo',
resp.headers['access-control-allow-headers'])
'x-auth-token, x-foo',
sortHeaderNames(resp.headers['access-control-allow-headers']))
def test_CORS_invalid_origin(self):
with save_globals():
controller = proxy_server.ContainerController(self.app, 'a', 'c')
def stubContainerInfo(*args):
return {
'cors': {
'allow_origin': 'http://baz'
}
}
controller.container_info = stubContainerInfo
def containerGET(controller, req):
return Response()
req = Request.blank(
'/a/c/o.jpg',
{'REQUEST_METHOD': 'GET'},
headers={'Origin': 'http://foo.bar'})
resp = cors_validation(containerGET)(controller, req)
self.assertEquals(401, resp.status_int)
def test_CORS_valid(self):
with save_globals():
controller = proxy_server.ContainerController(self.app, 'a', 'c')
def stubContainerInfo(*args):
return {
'cors': {
'allow_origin': 'http://foo.bar'
}
}
controller.container_info = stubContainerInfo
def containerGET(controller, req):
return Response(headers={
'X-Container-Meta-Color': 'red',
'X-Super-Secret': 'hush',
})
req = Request.blank(
'/a/c',
{'REQUEST_METHOD': 'GET'},
headers={'Origin': 'http://foo.bar'})
resp = cors_validation(containerGET)(controller, req)
self.assertEquals(200, resp.status_int)
self.assertEquals('http://foo.bar',
resp.headers['access-control-allow-origin'])
self.assertEquals('red', resp.headers['x-container-meta-color'])
# X-Super-Secret is in the response, but not "exposed"
self.assertEquals('hush', resp.headers['x-super-secret'])
self.assertTrue('access-control-expose-headers' in resp.headers)
exposed = set(
h.strip() for h in
resp.headers['access-control-expose-headers'].split(','))
expected_exposed = set(['cache-control', 'content-language',
'content-type', 'expires', 'last-modified',
'pragma', 'etag', 'x-timestamp',
'x-trans-id', 'x-container-meta-color'])
self.assertEquals(expected_exposed, exposed)
class TestAccountController(unittest.TestCase):
@ -4394,6 +4535,22 @@ class TestAccountController(unittest.TestCase):
self.assertTrue(
verb in resp.headers['Allow'])
self.assertEquals(len(resp.headers['Allow'].split(', ')), 4)
# Test a CORS OPTIONS request (i.e. including Origin and
# Access-Control-Request-Method headers)
self.app.allow_account_management = False
controller = proxy_server.AccountController(self.app, 'account')
req = Request.blank('/account', {'REQUEST_METHOD': 'OPTIONS'},
headers = {'Origin': 'http://foo.com',
'Access-Control-Request-Method': 'GET'})
req.content_length = 0
resp = controller.OPTIONS(req)
self.assertEquals(200, resp.status_int)
for verb in 'OPTIONS GET POST HEAD'.split():
self.assertTrue(
verb in resp.headers['Allow'])
self.assertEquals(len(resp.headers['Allow'].split(', ')), 4)
self.app.allow_account_management = True
controller = proxy_server.AccountController(self.app, 'account')
req = Request.blank('/account', {'REQUEST_METHOD': 'OPTIONS'})