The abstract base classes previously defined in 'collections' were moved
to 'collections.abc' in 3.3. The aliases will be removed in 3.10.
Preempt this change now with a simple find-replace:
$ ag -l 'collections.($TYPES)' | \
xargs sed -i 's/\(collections\)\.\($TYPES\)/\1.abc.\2/g'
Where $TYPES is the list of moved ABCs from [1].
[1] https://docs.python.org/3/library/collections.abc.html
Change-Id: Ia282479bb1d466bd2189ebb21b51d91e89b9581e
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Six is in use to help us to keep support for python 2.7.
Since the ussuri cycle we decide to remove the python 2.7
support so we can go ahead and also remove six usage from
the python code.
Review process and help
-----------------------
Removing six introduce a lot of changes and an huge amount of modified files
To simplify reviews we decided to split changes into several patches to avoid
painful reviews and avoid mistakes.
To review this patch you can use the six documentation [1] to obtain help and
understand choices.
Additional informations
-----------------------
Changes related to 'six.b(data)' [2]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
six.b [2] encode the given datas in latin-1 in python3 so I did the same
things in this patch.
Latin-1 is equal to iso-8859-1 [3].
This encoding is the default encoding [4] of certain descriptive HTTP
headers.
I suggest to keep latin-1 for the moment and to move to another encoding
in a follow-up patch if needed to move to most powerful encoding (utf8).
HTML4 support utf8 charset and utf8 is the default charset for HTML5 [5].
Note that this commit message is autogenerated and not necesserly contains
changes related to 'six.b'
[1] https://six.readthedocs.io/
[2] https://six.readthedocs.io/#six.b
[3] https://docs.python.org/3/library/codecs.html#standard-encodings
[4] https://www.w3schools.com/charsets/ref_html_8859.asp
[5] https://www.w3schools.com/html/html_charset.asp
Patch 6 of a serie of 28 patches
Change-Id: Ic56cf6da5ff9ab8f4bd218e6516b87f47165206d
Allow use policy in code to resource type's rule.
Also add test for override the in-code resource type rule in json
file.
Partially-Implements: bp policy-in-code
Change-Id: Id6c21732e66de6c421427ded98de52f5da0a4db2
pydoc is part of the standard library and is much more robust at formatting
docstings than any trivial hand-rolled munger could be. For example, it
correctly preserves indentation within the docstring. Use it when
generating descriptions from docstrings for the API.
Change-Id: Ib565a64d990a45c652a9475255c37805f86070b4
When __new__ returns an instance of the class (or a subclass), Python will
call __init__ on it afterwards. So if we return an initialised object of a
subclass, we end up calling both __new__ and __init__ twice. Avoid this by
calling object.__new__ and passing the desired subclass instead in
ResourceInfo and Parameter. Equivalent changes for Resource and Template
have been in place for some time.
Change-Id: I4c3f0bc9c1a39b9f0b964ccf1c6c638f86b3753e
A TripleO environment typically contains hundreds of resource type
mappings. And a TripleO deployment typically contains hundreds of nested
stacks. The result is typically tens of thousands of ResourceInfo objects
all loaded in memory at the same time.
This change saves memory by using slots for these classes instead of
__dict__. I'd expect this to save on the order of tens of megabytes of RAM
in a TripleO deployment - comparatively modest, but an easy win given that
it is such a simple change.
Change-Id: Ia0f17be794618d7b41c463e1992755947c56d4d1
Partial-Bug: #1626675
NotImplementedError is the name of the exception
(https://docs.python.org/2/library/exceptions.html).
NotImplemented is the name of a constant
(https://docs.python.org/2/library/constants.html).
>>> raise NotImplemented()
Traceback (most recent call last):
File "<pyshell#0>", line 1, in <module>
raise NotImplemented()
TypeError: 'NotImplementedType' object is not callable
>>> raise NotImplementedError()
Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
raise NotImplementedError()
NotImplementedError
This patch fix it.
Change-Id: I939eaa4b4b7c574f7a6447725e3a6ad5b128f1b7
Closes-Bug: #1339855
Currently this is reflected in the user_env_as_dict output, which means
not only is it stored in the DB (which we want), but also passed around
to nested stack (which we don't want because it's derived from the parent
stack hidden parameters), and also to the user via stack environment show
(the API returns it but heatclient currently hides it), which we also don't
want because it's not a valid key in user-provided environments.
Change-Id: If5821ccb4a8bbf98012a2541ddd3c8e91455e5cc
Closes-Bug: #1590507
Contrary to popular belief, LOG.exception() is not a method to which you
pass an exception in order to log it. Rather, you pass the message to be
logged at ERROR level, and the exception is retrieved automatically via
sys.exc_info().
Change-Id: I197cf94ada34a7ce80fc4026a99d95cd50823882
Now if extension is not available, user gets
message 'Service endpoint not in service catalog'
which is incorrect. Patch corrects this behavior
and adds improved messages with more info.
Change-Id: I9b6c0d2921519590ef105be35e31db29cd1e3ecc
In Python3, dict.keys() returns a view object rather than a list. This
behaves differently in that changes to the dict also modify the view, and
in that the view type interacts with various operators in different ways to
lists.
One universally correct transformation to preserve Python2 behaviour in
Python3 would be to replace all instances of d.keys() with
list(six.iterkeys(d)), and indeed we did. However, like many automatic
transformations the results are usually unsightly, invariably inefficient,
and frequently absurd. Not least because list(d.keys()) and indeed list(d)
are also equivalent.
This patch changes to using the simplest correct method of accessing the
data we want in each case.
This reverts or rewrites most of commit
4ace95ad47.
Change-Id: Iba3cf48246d8cbc958d8fb577cd700a218b0bebf
fnmatch is not thread safe in python <2.7.10, let's use the safe
one in oslo.utils
Change-Id: I09c8e25b584a253fa19c9f546188b419c2d5b35b
ref: https://hg.python.org/cpython/rev/fe12c34c39eb
ResourceInfo should hold only a weak reference to the ResourceRegistry
it is held in, otherwise there is a cycle that means the entire registry
has to wait for garbage collection in order to be deallocated.
Change-Id: I708aa1e9bac696eb16e9ef21089ca6e14e84e067
Partial-Bug: #1570974
The commit 08431c7c06 added a reference to
the Environment object from the ResourceRegistry object, which is itself
an attribute of the environment. This causes a reference cycle that
means that the environment and everything referenced by it (including
the potentially very large files dict) will only be deallocated when it
is garbage collected since the reference counts will never hit zero.
This has probably been contributing substantially to memory
fragmentation.
Change-Id: Ib251b5f5ffc07fe1a06f5e44124024831d4ba1b2
Partial-Bug: #1570974
Warnings that a resource is not supported are now prefaced
with the resource name and status (e.g., DEPRECATED). In
addition, the warning is now logged via LOG.warn(_LW()).
Previously, warnings.warn(six.text_type()) was used. In
addition to being routed to the log file, it included a
confusing reference to the last line of invocation, which
was the fragment "info.value.support_status.message))".
Change-Id: I006120a395007d88f3ba04a4af89283a1a077c3f
Closes-Bug: #1524512
When a client is not installed, Heat should behave as if the service was
unavailable. We recently started exposing exceptions when checking for
service availability, which displayed that it fails if the client plugin
returs None. Let's test and handle that case.
Change-Id: I71d5fd5be2efd7734d8569e4d26ab1b512453343
Closes-Bug: #1543759
Restricts update/replace of resource based on
environment 'restricted_actions' entry for a resource.
This also adds a 'preview_update' method to Resource
class which is now leveraged by stack-preview.
Change-Id: Iea80a6572aa696ee607682e5113204d1c82389fa
Blueprint: stack-update-restrict
Modify get_class to ensure that user-defined resources cannot result in
reads from the local filesystem. Only resources defined by the operator
in the global environment should read local files.
Change-Id: I845e7d23c73242a4a4c9c40599690ab705c75caa
Closes-Bug: #1496277
Implement a new mechanism to allow specifying a target in the
environment to send events to. It adds zaqar as the first
implementation.
Depends-On: Ie04f9204f3ba0f75de32253f096f439c512cddee
Change-Id: Icfc3864e08693cb4b4f921641af380b39bcf0bc0
This allows mapping several resource type to a single type in the
resource registry.
Mostly useful for testing (map many to OS::Heat::None).
Needed to improve the heat-templates gate.
Change-Id: I5b56c71b5575b73432b0e24df1dc8f8402c17a0c
Closes-Bug: #1499748
Related-Bug: #1492942
We were previously using get_class for two different purposes - to get a
resource plugin on which we could perform introspection to obtain the
properties and attributes schema, and to get a resource plugin we could
instantiate to create a Resource object. These are both the same except in
the case of a TemplateResource, where having two different use cases for
the same piece of code was adding considerable extra complexity. Combining
the use cases in this way also made the error handling confusing (leading
to bug 1518458).
This change separates out the two cases.
Change-Id: I3bde081cd4537810c8c6a0948ab447c3fb7ca9bc
Related-Bug: #1447194
Related-Bug: #1518458
Related-Bug: #1508115
Rather than return None when doing get_resource_info() on resource types
that don't exist, raise an EntityNotFound exception. We can then catch that
and turn it into StackValidationFailed where appropriate, but we don't make
assumptions about where the code is being called from at the point where we
raise the exception.
Change-Id: Idf6f837befea8ccb10fdaf9196490da3dd5be1fb
Related-Bug: #1447194
Related-Bug: #1518458
This reverts commit 9a4d719ea1 and creates a
new exception type, InvalidGlobalResource, that is raised when a
non-existent template is referenced in the global environment and which
results in an HTTP 500 error.
This fixes the cases where we could return 404 or 500 errors as a result of
template validation failures, when we should return a 400 error.
Change-Id: I3c7b3182957448bb13514696cc2e12f5aaddd253
Closes-Bug: #1518458
Related-Bug: #1447194
The process of bringing us into compliance with the H405 pep8 rule has left
us with a lot of docstrings that either don't follow the recommendations of
PEP0257, are misleading, wrong, or nonsensical. This patch attempts to fix
them up.
Change-Id: Icfe90b5ba3102a7e13ab659038a8b2af9019e9e6
Currently we log all template resources every time any API call
happens which indirectly creates an Environment object, which
results in a lot of duplicate bloat in the heat logs (not even
at debug level..)
So, instead log all environment resources (e.g those from resource
plugins and the global environment) only when we start the
service, and log user-provided template resources only when we
do the template validation on create/update, for only the root stack,
because we pass a derived subset environment down to all children.
We also switch to using the string representation of the ResourceInfo
objects, as this contains a bit more useful info than the current
log format.
Change-Id: I20007c1ad6bf8b1479c1d60a5c392b8b9e1c06c4
Closes-Bug: #1499330
Rodolfo Alonso Hernandez