17 KiB
Proxies and Wrappers
Underlying the implementation of the decorators created by the wrapt module is a wrapper class which acts as a transparent object proxy. This document describes the object proxy and the various custom wrappers provided.
Object Proxy
The object proxy class is available as
wrapt.ObjectProxy
. The class would not normally be used
directly, but as a base class to custom object proxies or wrappers which
add behaviour which overrides that of the original object. When an
object proxy is used, it will pass through any actions performed on the
proxy through to the wrapped object.
>>> table = {}
>>> proxy = wrapt.ObjectProxy(table)
>>> proxy['key-1'] = 'value-1'
>>> proxy['key-2'] = 'value-2'
>>> proxy.keys()
['key-2', 'key-1']
>>> table.keys()
['key-2', 'key-1']
>>> isinstance(proxy, dict)
True
>>> dir(proxy)
['__class__', '__cmp__', '__contains__', '__delattr__', '__delitem__',
'__doc__', '__eq__', '__format__', '__ge__', '__getattribute__',
'__getitem__', '__gt__', '__hash__', '__init__', '__iter__', '__le__',
'__len__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__',
'__repr__', '__setattr__', '__setitem__', '__sizeof__', '__str__',
'__subclasshook__', 'clear', 'copy', 'fromkeys', 'get', 'has_key',
'items', 'iteritems', 'iterkeys', 'itervalues', 'keys', 'pop',
'popitem', 'setdefault', 'update', 'values']
This ability for a proxy to stand in for the original goes as far as arithmetic operations, rich comparison and hashing.
>>> value = 1
>>> proxy = wrapt.ObjectProxy(value)
>>> proxy + 1
2
>>> int(proxy)
1
>>> hash(proxy)
1
>>> hash(value)
1
>>> proxy < 2
True
>>> proxy == 0
False
Do note however, that when wrapping an object proxy around a literal value, the original value is effectively copied into the proxy object and any operation which updates the value will only update the value held by the proxy object.
>>> value = 1
>>> proxy = wrapt.ObjectProxy(value)
>>> type(proxy)
<type 'ObjectProxy'>
>>> proxy += 1
>>> type(proxy)
<type 'ObjectProxy'>
>>> print(proxy)
2
>>> print(value)
1
Object wrappers may therefore have limited use in conjunction with literal values.
Type Comparison
The type of an instance of the object proxy will be
ObjectProxy
, or that of any derived class type if creating
a custom object proxy.
>>> value = 1
>>> proxy = wrapt.ObjectProxy(value)
>>> type(proxy)
<type 'ObjectProxy'>
>>> class CustomProxy(wrapt.ObjectProxy):
... pass
>>> proxy = CustomProxy(1)
>>> type(proxy)
<class '__main__.CustomProxy'>
Direct type comparisons in Python are generally frowned upon and
allowance for 'duck typing' preferred. Instead of direct type
comparison, the isinstance()
function would therefore be
used. Using isinstance()
, comparison of the type of the
object proxy will properly evaluate against the wrapped object.
>>> isinstance(proxy, int)
True
This works because the __class__
attribute actually
returns the class type for the wrapped object.
>>> proxy.__class__
<type 'int'>
Note that isinstance()
will still also succeed if
comparing to the ObjectProxy
type. It is therefore still
possible to use isinstance()
to determine if an object is
an object proxy.
>>> isinstance(proxy, wrapt.ObjectProxy)
True
>>> class CustomProxy(wrapt.ObjectProxy):
... pass
>>> proxy = CustomProxy(1)
>>> isinstance(proxy, wrapt.ObjectProxy)
True
>>> isinstance(proxy, CustomProxy)
True
Custom Object Proxies
A custom proxy is where one creates a derived object proxy and overrides some specific behaviour of the proxy.
def function():
print('executing', function.__name__)
class CallableWrapper(wrapt.ObjectProxy):
def __call__(self, *args, **kwargs):
print('entering', self.__wrapped__.__name__)
try:
return self.__wrapped__(*args, **kwargs)
finally:
print('exiting', self.__wrapped__.__name__)
>>> proxy = CallableWrapper(function)
>>> proxy()
('entering', 'function')
('executing', 'function')
('exiting', 'function')
Any method of the original wrapped object can be overridden,
including special Python methods such as __call__()
. If it
is necessary to change what happens when a specific attribute of the
wrapped object is accessed, then properties can be used.
If it is necessary to access the original wrapped object from within
an overridden method or property, then self.__wrapped__
is
used.
Proxy Object Attributes
When an attempt is made to access an attribute from the proxy, the same named attribute would in normal circumstances be accessed from the wrapped object. When updating an attributes value, or deleting the attribute, that change will also be reflected in the wrapped object.
>>> proxy = CallableWrapper(function)
>>> hasattr(function, 'attribute')
False
>>> hasattr(proxy, 'attribute')
False
>>> proxy.attribute = 1
>>> hasattr(function, 'attribute')
True
>>> hasattr(proxy, 'attribute')
True
>>> function.attribute
1
>>> proxy.attribute
1
If an attribute was updated on the wrapped object directly, that change is still reflected in what is available via the proxy.
>>> function.attribute = 2
>>> function.attribute
2
>>> proxy.attribute
2
If creating a custom proxy and it needs to keep attributes of its own
which should not be saved through to the wrapped object, those
attributes should be prefixed with _self_
.
def function():
print('executing', function.__name__)
class CallableWrapper(wrapt.ObjectProxy):
def __init__(self, wrapped, wrapper):
super(CallableWrapper, self).__init__(wrapped)
self._self_wrapper = wrapper
def __call__(self, *args, **kwargs):
return self._self_wrapper(self.__wrapped__, args, kwargs)
def wrapper(wrapped, args, kwargs):
print('entering', wrapped.__name__)
try:
return wrapped(*args, **kwargs)
finally:
print('exiting', wrapped.__name__)
>>> proxy = CallableWrapper(function, wrapper)
>>> proxy._self_wrapper
<function wrapper at 0x1005961b8>
>>> function._self_wrapper
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'function' object has no attribute '_self_wrapper'
If an attribute local to the proxy must be available under a name
without this special prefix, then a @property
can be used
in the class definition.
class CustomProxy(wrapt.ObjectProxy):
def __init__(self, wrapped):
super(CustomProxy, self).__init__(wrapped)
self._self_attribute = 1
@property
def attribute(self):
return self._self_attribute
@attribute.setter
def attribute(self, value):
self._self_attribute = value
@attribute.deleter
def attribute(self):
del self._self_attribute
>>> proxy = CustomProxy(1)
>>> print proxy.attribute
1
>>> proxy.attribute = 2
>>> print proxy.attribute
2
>>> del proxy.attribute
>>> print proxy.attribute
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'int' object has no attribute 'attribute'
Alternatively, the attribute can be specified as a class attribute,
with that then being overridden if necessary, with a specific value in
the __init__()
method of the class.
class CustomProxy(wrapt.ObjectProxy):
attribute = None
def __init__(self, wrapped):
super(CustomProxy, self).__init__(wrapped)
self.attribute = 1
>>> proxy = CustomProxy(1)
>>> print proxy.attribute
1
>>> proxy.attribute = 2
>>> print proxy.attribute
2
>>> del proxy.attribute
>>> print proxy.attribute
None
Just be aware that although the attribute can be deleted from the instance of the custom proxy, lookup will then fallback to using the class attribute.
Function Wrappers
Although an ObjectProxy
can be used to wrap a function,
it doesn't do anything special in respect of bound methods. If
attempting to use a custom object proxy to wrap instance methods, class
methods or static methods, it would be necessary to override the
appropriate descriptor protocol methods in order to be able to intercept
and modify in any way the execution of the wrapped function.
class BoundCallableWrapper(wrapt.ObjectProxy):
def __init__(self, wrapped, wrapper):
super(BoundCallableWrapper, self).__init__(wrapped)
self._self_wrapper = wrapper
def __get__(self, instance, owner):
return self
def __call__(self, *args, **kwargs):
return self._self_wrapper(self.__wrapped__, args, kwargs)
class CallableWrapper(wrapt.ObjectProxy):
def __init__(self, wrapped, wrapper):
super(CallableWrapper, self).__init__(wrapped)
self._self_wrapper = wrapper
def __get__(self, instance, owner):
function = self.__wrapped__.__get__(instance, owner)
return BoundCallableWrapper(function, self._self_wrapper)
def __call__(self, *args, **kwargs):
return self._self_wrapper(self.__wrapped__, args, kwargs)
The CallableWrapper.__call__()
method would therefore be
invoked when CallableWrapper
is used around a regular
function. The BoundCallableWrapper.__call__()
would instead
be what is invoked for a bound method, the instance of
BoundCallableWrapper
having being created when the original
wrapped method was bound to the class instance.
This specific pattern is actually the basis of what is required to
implement a robust function wrapper for use in implementing a decorator.
Because it is a fundamental pattern, a predefined version is available
as wrapt.FunctionWrapper
.
As with the illustrative example above, FunctionWrapper
class accepts two key arguments:
wrapped
- The function being wrapped.wrapper
- A wrapper function to be called when the wrapped function is invoked.
Although in prior examples the wrapper function was shown as
accepting three positional arguments of the wrapped function and the
args and kwargs for when the wrapped function was called, when using
FunctionWrapper
, it is expected that the wrapper function
accepts four arguments. These are:
wrapped
- The wrapped function which in turns needs to be called by your wrapper function.instance
- The object to which the wrapped function was bound when it was called.args
- The list of positional arguments supplied when the decorated function was called.kwargs
- The dictionary of keyword arguments supplied when the decorated function was called.
When FunctionWrapper
is applied to a normal function or
static method, the wrapper function when called will be passed
None
as the instance
argument.
When applied to an instance method, the wrapper function when called
will be passed the instance of the class the method is being called on
as the instance
argument. This will be the case even when
the instance method was called explicitly via the class and the instance
passed as the first argument. That is, the instance will never be passed
as part of args
.
When applied to a class method, the wrapper function when called will
be passed the class type as the instance
argument.
When applied to a class, the wrapper function when called will be
passed None
as the instance
argument. The
wrapped
argument in this case will be the class.
The above rules can be summarised with the following example.
import inspect
def wrapper(wrapped, instance, args, kwargs):
if instance is None:
if inspect.isclass(wrapped):
# Decorator was applied to a class.
return wrapped(*args, **kwargs)
else:
# Decorator was applied to a function or staticmethod.
return wrapped(*args, **kwargs)
else:
if inspect.isclass(instance):
# Decorator was applied to a classmethod.
return wrapped(*args, **kwargs)
else:
# Decorator was applied to an instancemethod.
return wrapped(*args, **kwargs)
Using these checks it is therefore possible to create a universal function wrapper that can be applied in all situations. It is no longer necessary to create different variants of function wrappers for normal functions and instance methods.
In all cases, the wrapped function passed to the wrapper function is
called in the same way, with args
and kwargs
being passed. The instance
argument doesn't need to be used
in calling the wrapped function.
A simple decorator factory implementation which makes use of
FunctionWrapper
to delegate execution of the wrapped
function to the wrapper function would be:
def function_wrapper(wrapper):
@functools.wraps(wrapper)
def _wrapper(wrapped):
return FunctionWrapper(wrapped, wrapper)
return _wrapper
It would be used like:
@function_wrapper
def wrapper(wrapped, instance, args, kwargs):
return wrapped(*args, **kwargs)
@wrapper
def function():
pass
This example of a simplified decorator factory is made available as
wrapt.function_wrapper
. Although it is usable in its own
right, it is preferable that wrapt.decorator
be used to
create decorators as it provides additional features. The
@function_wrapper
decorator would generally be used more
when performing monkey patching and needing to dynamically create
function wrappers.
@function_wrapper
def wrapper(wrapped, instance, args, kwargs):
return wrapped(*args, **kwargs)
callback = wrapper(fetch_callback())
Custom Function Wrappers
If it is necessary to implement a custom function wrapper in order to
override the behaviour of a wrapped function, it is possible to still
derive from the wrapt.FunctionWrapper
class. That binding
of functions can occur, does however complicate this. This is because
the bound function is a separate object implemented as a different
type.
The type of the separate bound function wrapper is
wrapt.BoundFunctionWrapper
. If the behaviour for the bound
function also needs to be overridden, a derived version of this class
will also need to be created. The derived custom function wrapper will
then need to indicate that this second type should be used when creating
the bound function wrapper, rather than the default. This is done via
the __bound_function_wrapper__
attribute of the class.
class CustomBoundFunctionWrapper(wrapt.BoundFunctionWrapper):
def __init__(self, *args, **kwargs):
self._self_attribute = self._self_parent._self_attribute
super(CustomBoundFunctionWrapper, self).__init__(*args, **kwargs)
def __call__(self, *args, **kwargs):
if self._self_attribute:
...
return super(CustomBoundFunctionWrapper, self).__call__(*args, **kwargs)
class CustomFunctionWrapper(wrapt.FunctionWrapper):
__bound_function_wrapper__ = CustomBoundFunctionWrapper
def __init__(self, wrapped, wrapper, attribute):
super(CustomFunctionWrapper, self).__init__(wrapped, wrapper)
self._self_attribute = attribute
The set of arguments used to initialize an instance of a bound
function wrapper object should be treated as a private implementation
detail. This means that if a custom bound function wrapper needs to
implement an __init__()
method, it should pass through all
arguments as *args
and **kwargs
. It should not
use specific named parameters.
If the instance of the custom bound function wrapper needs to access
any special attributes originally supplied to the custom function
wrapper when created, it should use self._self_parent
to
access the parent object to retrieve them.
Alternatively, it would be necessary to define the custom bound
function wrapper type in a function closure and assign
__bound_function_wrapper__
dynamically against the instance
of the custom bound function wrapper.
def custom_bound_function_wrapper(attribute):
class CustomBoundFunctionWrapper(wrapt.BoundFunctionWrapper):
def __init__(self, *args, **kwargs):
self._self_attribute = attribute
super(CustomBoundFunctionWrapper, self).__init__(*args, **kwargs)
def __call__(self, *args, **kwargs):
if self._self_attribute:
...
return super(CustomBoundFunctionWrapper, self).__call__(*args, **kwargs)
class CustomFunctionWrapper(wrapt.FunctionWrapper):
def __init__(self, wrapped, wrapper, attribute):
super(CustomFunctionWrapper, self).__init__(wrapped, wrapper)
self._self_attribute = attribute
self.__bound_function_wrapper__ = custom_bound_function_wrapper(attribute)