openstack
/
deb-python-wsgi-intercept
Code Issues Proposed changes

Merge pull request #45 from cdent/response-header-check 

Response header check
This commit is contained in:
Chris Dent 2016-09-27 10:44:02 +01:00 committed by GitHub
parent e58af412b6 ac5f41a03e
commit 7a0851f738
8 changed files with 305 additions and 77 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
README
View File

@ -1 +1 @@
README.md
README.rst

38
README.md
View File

@ -1,38 +0,0 @@
wsgi-intercept
======================
[![travis](https://secure.travis-ci.org/cdent/wsgi-intercept.png)](https://secure.travis-ci.org/cdent/wsgi-intercept)
Documentation is available on [Read The
Docs](http://wsgi-intercept.readthedocs.org/en/latest/).
What is it?
===========
wsgi_intercept installs a WSGI application in place of a real host for
testing while still preserving HTTP semantics. See the
[PyPI page](http://pypi.python.org/pypi/wsgi_intercept) page for more details.
It works by intercepting the connection handling in http client
libraries.
Supported Libraries
-------------------
For Python 2.7 the following libraries are supported:
* `urllib2`
* `httplib`
* `httplib2`
* `requests`
* `urllib3`
In Python 3:
* `urllib.request`
* `http.client`
* `httplib2`
* `requests`
* `urllib3`
If you are using Python 2 and need support for a different HTTP
client, require a version of `wsgi_intercept<0.6`.

139
README.rst Normal file
View File

@ -0,0 +1,139 @@
Installs a WSGI application in place of a real host for testing.
Introduction
============
Testing a WSGI application sometimes involves starting a server at a
local host and port, then pointing your test code to that address.
Instead, this library lets you intercept calls to any specific host/port
combination and redirect them into a `WSGI application`_ importable by
your test program. Thus, you can avoid spawning multiple processes or
threads to test your Web app.
Supported Libaries
==================
``wsgi_intercept`` works with a variety of HTTP clients in Python 2.7,
3.3 and beyond, and in pypy.
* urllib2
* urllib.request
* httplib
* http.client
* httplib2
* requests
* urllib3
How Does It Work?
=================
``wsgi_intercept`` works by replacing ``httplib.HTTPConnection`` with a
subclass, ``wsgi_intercept.WSGI_HTTPConnection``. This class then
redirects specific server/port combinations into a WSGI application by
emulating a socket. If no intercept is registered for the host and port
requested, those requests are passed on to the standard handler.
The easiest way to use an intercept is to import an appropriate subclass
of ``~wsgi_intercept.interceptor.Interceptor`` and use that as a
context manager over web requests that use the library associated with
the subclass. For example::
import httplib2
from wsgi_intercept.interceptor import Httplib2Interceptor
from mywsgiapp import app
def load_app():
return app
http = httplib2.Http()
with Httplib2Interceptor(load_app, host='example.com', port=80) as url:
response, content = http.request('%s%s' % (url, '/path'))
assert response.status == 200
The interceptor class may aslo be used directly to install intercepts.
See the module documentation for more information.
Older versions required that the functions ``add_wsgi_intercept(host,
port, app_create_fn, script_name='')`` and ``remove_wsgi_intercept(host,port)``
be used to specify which URLs should be redirected into what applications.
These methods are still available, but the ``Interceptor`` classes are likely
easier to use for most use cases.
.. note:: ``app_create_fn`` is a *function object* returning a WSGI
application; ``script_name`` becomes ``SCRIPT_NAME`` in the WSGI
app's environment, if set.
.. note:: If ``http_proxy`` or ``https_proxy`` is set in the environment
this can cause difficulties with some of the intercepted libraries.
If requests or urllib is being used, these will raise an exception
if one of those variables is set.
.. note:: If ``wsgi_intercept.STRICT_RESPONSE_HEADERS`` is set to ``True``
then response headers sent by an application will be checked to
make sure they are of the type ``str`` native to the version of
Python, as required by pep 3333. The default is ``False`` (to
preserve backwards compatibility)
Install
=======
::
pip install -U wsgi_intercept
Packages Intercepted
====================
Unfortunately each of the HTTP client libraries use their own specific
mechanism for making HTTP call-outs, so individual implementations are
needed. At this time there are implementations for ``httplib2``,
``urllib3`` and ``requests`` in both Python 2 and 3, ``urllib2`` and
``httplib`` in Python 2 and ``urllib.request`` and ``http.client``
in Python 3.
If you are using Python 2 and need support for a different HTTP
client, require a version of ``wsgi_intercept<0.6``. Earlier versions
include support for ``webtest``, ``webunit`` and ``zope.testbrowser``.
The best way to figure out how to use interception is to inspect
`the tests`_. More comprehensive documentation available upon
request.
.. _the tests: https://github.com/cdent/wsgi-intercept/tree/master/test
History
=======
Pursuant to Ian Bicking's `"best Web testing framework"`_ post, Titus
Brown put together an `in-process HTTP-to-WSGI interception mechanism`_
for his own Web testing system, twill. Because the mechanism is pretty
generic -- it works at the httplib level -- Titus decided to try adding
it into all of the *other* Python Web testing frameworks.
The Python 2 version of wsgi-intercept was the result. Kumar McMillan
later took over maintenance.
The current version is tested with Python 2.7, 3.3, 3.4, 3.5 and pypy
and was assembled by `Chris Dent`_. Testing and documentation improvements
from `Sasha Hart`_.
.. _"best Web testing framework":
http://blog.ianbicking.org/best-of-the-web-app-test-frameworks.html
.. _in-process HTTP-to-WSGI interception mechanism:
http://www.advogato.org/person/titus/diary.html?start=119
.. _WSGI application: http://www.python.org/peps/pep-3333.html
.. _Chris Dent: https://github.com/cdent
.. _Sasha Hart: https://github.com/sashahart
Project Home
============
This project lives on `GitHub`_. Please submit all bugs, patches,
failing tests, et cetera using the Issue Tracker.
Additional documentation is available on `Read The Docs`_.
.. _GitHub: http://github.com/cdent/wsgi-intercept
.. _Read The Docs: http://wsgi-intercept.readthedocs.org/en/latest/

8
setup.py
View File

@ -1,7 +1,9 @@
import wsgi_intercept
from setuptools import setup, find_packages
VERSION = '1.4.0'
README = open('README.rst').read()
CLASSIFIERS = """
Environment :: Web Environment
Intended Audience :: Developers
@ -20,7 +22,7 @@ Topic :: Software Development :: Testing
META = {
'name': 'wsgi_intercept',
'version': wsgi_intercept.__version__,
'version': VERSION,
'author': 'Titus Brown, Kumar McMillan, Chris Dent, Sasha Hart',
'author_email': 'cdent@peermore.com',
'description':
@ -28,7 +30,7 @@ META = {
'real URI for testing.',
# What will the name be?
'url': 'http://pypi.python.org/pypi/wsgi_intercept',
'long_description': wsgi_intercept.__doc__,
'long_description': README,
'license': 'MIT License',
'classifiers': CLASSIFIERS,
'packages': find_packages(),

7
tox.ini
View File

@ -1,7 +1,7 @@
[tox]
minversion = 1.6
skipsdist = True
envlist = py27,py33,py34,py35,pypy,pep8,docs
envlist = py27,py33,py34,py35,pypy,pep8,docs,readme
[testenv]
deps = .[testing]
@ -20,6 +20,11 @@ commands =
whitelist_externals =
rm
[testenv:readme]
deps = .
whitelist_externals = bash
commands = bash -c "python -c 'import sys, wsgi_intercept; sys.stdout.write(wsgi_intercept.__doc__)' > README.rst"
[flake8]
exclude=.venv,.git,.tox,dist,*egg,*.egg-info,build,examples,docs
show-source = True

90
wsgi_intercept/__init__.py
View File

@ -3,13 +3,27 @@
Introduction
============
Testing a WSGI application normally involves starting a server at a
Testing a WSGI application sometimes involves starting a server at a
local host and port, then pointing your test code to that address.
Instead, this library lets you intercept calls to any specific host/port
combination and redirect them into a `WSGI application`_ importable by
your test program. Thus, you can avoid spawning multiple processes or
threads to test your Web app.
Supported Libaries
==================
``wsgi_intercept`` works with a variety of HTTP clients in Python 2.7,
3.3 and beyond, and in pypy.
* urllib2
* urllib.request
* httplib
* http.client
* httplib2
* requests
* urllib3
How Does It Work?
=================
@ -45,14 +59,21 @@ be used to specify which URLs should be redirected into what applications.
These methods are still available, but the ``Interceptor`` classes are likely
easier to use for most use cases.
Note especially that ``app_create_fn`` is a *function object* returning a WSGI
application; ``script_name`` becomes ``SCRIPT_NAME`` in the WSGI app's
environment, if set.
.. note:: ``app_create_fn`` is a *function object* returning a WSGI
application; ``script_name`` becomes ``SCRIPT_NAME`` in the WSGI
app's environment, if set.
.. note:: If ``http_proxy`` or ``https_proxy`` is set in the environment
this can cause difficulties with some of the intercepted libraries.
If requests or urllib is being used, these will raise an exception
if one of those variables is set.
.. note:: If ``wsgi_intercept.STRICT_RESPONSE_HEADERS`` is set to ``True``
then response headers sent by an application will be checked to
make sure they are of the type ``str`` native to the version of
Python, as required by pep 3333. The default is ``False`` (to
preserve backwards compatibility)
Note also that if ``http_proxy`` or ``https_proxy`` is set in the environment
this can cause difficulties with some of the intercepted libraries. If
requests or urllib is being used, these will raise an exception if one of
those variables is set.
Install
=======
@ -64,17 +85,16 @@ Install
Packages Intercepted
====================
Unfortunately each of the Web testing frameworks uses its own specific
Unfortunately each of the HTTP client libraries use their own specific
mechanism for making HTTP call-outs, so individual implementations are
needed. At this time there are implementations for ``httplib2`` and
``requests`` in both Python 2 and 3, ``urllib2`` and ``httplib``
in Python 2 and ``urllib.request`` and ``http.client`` in Python 3.
needed. At this time there are implementations for ``httplib2``,
``urllib3`` and ``requests`` in both Python 2 and 3, ``urllib2`` and
``httplib`` in Python 2 and ``urllib.request`` and ``http.client``
in Python 3.
If you are using Python 2 and need support for a different HTTP
client, require a version of ``wsgi_intercept<0.6``. Earlier versions
include support for ``webtest``, ``webunit`` and ``zope.testbrowser``.
It is quite likely that support for these versions will be relatively
easy to add back in to the new version.
The best way to figure out how to use interception is to inspect
`the tests`_. More comprehensive documentation available upon
@ -117,32 +137,28 @@ Additional documentation is available on `Read The Docs`_.
.. _GitHub: http://github.com/cdent/wsgi-intercept
.. _Read The Docs: http://wsgi-intercept.readthedocs.org/en/latest/
"""
from __future__ import print_function
import sys
import traceback
from io import BytesIO
__version__ = '1.3.2'
try:
from http.client import HTTPConnection, HTTPSConnection
except ImportError:
from httplib import HTTPConnection, HTTPSConnection
try:
from io import BytesIO
except ImportError:
from StringIO import StringIO as BytesIO
# Don't use six here because it is unquote_to_bytes that we want in
# Python 3.
try:
from urllib.parse import unquote_to_bytes as url_unquote
except ImportError:
from urllib import unquote as url_unquote
import six
from six.moves.http_client import HTTPConnection, HTTPSConnection
# Set this to True to cause response headers from the intercepted
# app to be confirmed as bytestrings, behaving as some wsgi servers.
STRICT_RESPONSE_HEADERS = False
debuglevel = 0
# 1 basic
@ -210,7 +226,7 @@ def make_environ(inp, host, port, script_name):
environ = {}
method_line = inp.readline()
if sys.version_info[0] > 2:
if six.PY3:
method_line = method_line.decode('ISO-8859-1')
content_type = None
@ -290,7 +306,7 @@ def make_environ(inp, host, port, script_name):
# do to be like a server. Later various libraries will be forced
# to decode and then reencode to get the UTF-8 that everyone
# wants.
if sys.version_info[0] > 2:
if six.PY3:
path_info = path_info.decode('latin-1')
environ.update({
@ -411,7 +427,8 @@ class wsgi_fake_socket:
def start_response(status, headers, exc_info=None):
# construct the HTTP request.
self.output.write(b"HTTP/1.0 " + status.encode('utf-8') + b"\n")
self.output.write(
b"HTTP/1.0 " + status.encode('ISO-8859-1') + b"\n")
# Keep the reference of the headers list to write them only
# when the whole application have been processed
self.headers = headers
@ -436,12 +453,17 @@ class wsgi_fake_socket:
# send the headers
for k, v in self.headers:
if STRICT_RESPONSE_HEADERS:
if not (isinstance(k, str) and isinstance(v, str)):
raise TypeError(
"Header has a key '%s' or value '%s' "
"which is not a native str." % (k, v))
try:
k = k.encode('utf-8')
k = k.encode('ISO-8859-1')
except AttributeError:
pass
try:
v = v.encode('utf-8')
v = v.encode('ISO-8859-1')
except AttributeError:
pass
self.output.write(k + b': ' + v + b"\n")

5
wsgi_intercept/tests/__init__.py
View File

@ -1,4 +1,9 @@
import os
import wsgi_intercept
# Ensure that our test apps are sending strict headers.
wsgi_intercept.STRICT_RESPONSE_HEADERS = True
if os.environ.get('USER') == 'cdent':
import warnings

93
wsgi_intercept/tests/test_response_headers.py Normal file
View File

@ -0,0 +1,93 @@
"""Test response header validations.
Response headers are supposed to be bytestrings and some servers,
notably will experience an error if they are given headers with
the wrong form. Since wsgi-intercept is standing in as a server,
it should behave like one on this front. At the moment it does
not. There are tests for how it delivers request headers, but
not the other way round. Let's write some tests to fix that.
"""
import py.test
import requests
import six
import wsgi_intercept
from wsgi_intercept.interceptor import RequestsInterceptor
class HeaderApp(object):
"""A simple app that returns whatever headers we give it."""
def __init__(self, headers):
self.headers = headers
def __call__(self, environ, start_response):
headers = []
for header in self.headers:
headers.append((header, self.headers[header]))
start_response('200 OK', headers)
return ['']
def app(headers):
return HeaderApp(headers)
def test_header_app():
"""Make sure the header apps returns headers.
Many libraries normalize headers to strings so we're not
going to get exact matches.
"""
header_value = 'alpha'
header_value_str = 'alpha'
def header_app():
return app({'request-id': header_value})
with RequestsInterceptor(header_app) as url:
response = requests.get(url)
assert response.headers['request-id'] == header_value_str
def test_encoding_violation():
"""If the header is unicode we expect boom."""
header_key = 'request-id'
if six.PY2:
header_value = u'alpha'
else:
header_value = b'alpha'
# we expect our http library to give us a str
returned_header = 'alpha'
def header_app():
return app({header_key: header_value})
# save original
strict_response_headers = wsgi_intercept.STRICT_RESPONSE_HEADERS
# With STRICT_RESPONSE_HEADERS True, response headers must be
# native str.
with RequestsInterceptor(header_app) as url:
wsgi_intercept.STRICT_RESPONSE_HEADERS = True
with py.test.raises(TypeError) as error:
response = requests.get(url)
assert (str(error.value) ==
"Header has a key '%s' or value '%s' "
"which is not a native str." % (header_key, header_value))
# When False, other types of strings are okay.
wsgi_intercept.STRICT_RESPONSE_HEADERS = False
response = requests.get(url)
assert response.headers['request-id'] == returned_header
# reset back to saved original
wsgi_intercept.STRICT_RESPONSE_HEADERS = \
strict_response_headers