The abstract base classes previously defined in 'collections' were moved
to 'collections.abc' in 3.3. The aliases will be removed in 3.10.
Preempt this change now with a simple find-replace:
$ ag -l 'collections.($TYPES)' | \
xargs sed -i 's/\(collections\)\.\($TYPES\)/\1.abc.\2/g'
Where $TYPES is the list of moved ABCs from [1].
[1] https://docs.python.org/3/library/collections.abc.html
Change-Id: Ia282479bb1d466bd2189ebb21b51d91e89b9581e
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This change allows a Macro returns a 'null' value (as opposed to None)
that makes it effectively cease to exist when that makes sense. If it
appears as a list item, that list item is dropped. If it appears as a
value in a dict, the corresponding key is removed from the dict. If it
appears as a property value, the property is treated as if the user had
not mentioned it in the template.
In all other circumstances (as an argument to a Function, as the top
level of some other field, like an output value), the result will still
appear as None.
A null value is represented internally by the Ellipsis singleton (to
distinguish it from None, which is a value that may appear in the
template from a user or be returned from a Function).
Change-Id: Iaed0982e0db902f6eaf8f986c12b4885bd77e8b9
Story: 2007388
Task: 38976
Six is in use to help us to keep support for python 2.7.
Since the ussuri cycle we decide to remove the python 2.7
support so we can go ahead and also remove six usage from
the python code.
Review process and help
-----------------------
Removing six introduce a lot of changes and an huge amount of modified files
To simplify reviews we decided to split changes into several patches to avoid
painful reviews and avoid mistakes.
To review this patch you can use the six documentation [1] to obtain help and
understand choices.
Additional informations
-----------------------
Changes related to 'six.b(data)' [2]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
six.b [2] encode the given datas in latin-1 in python3 so I did the same
things in this patch.
Latin-1 is equal to iso-8859-1 [3].
This encoding is the default encoding [4] of certain descriptive HTTP
headers.
I suggest to keep latin-1 for the moment and to move to another encoding
in a follow-up patch if needed to move to most powerful encoding (utf8).
HTML4 support utf8 charset and utf8 is the default charset for HTML5 [5].
Note that this commit message is autogenerated and not necesserly contains
changes related to 'six.b'
[1] https://six.readthedocs.io/
[2] https://six.readthedocs.io/#six.b
[3] https://docs.python.org/3/library/codecs.html#standard-encodings
[4] https://www.w3schools.com/charsets/ref_html_8859.asp
[5] https://www.w3schools.com/html/html_charset.asp
Patch 6 of a serie of 28 patches
Change-Id: Ic56cf6da5ff9ab8f4bd218e6516b87f47165206d
Previously when calling function.validate() we passed the path to the
function in the template (used for debugging purposes) as a string. This
change allows us to pass it as a list of path components, as we do
elsewhere, so that higher-level code that catches StackValidationFailed
can deal with its components separately.
Change-Id: I017aa6f7511b8478ef8273522ab8087684ae71c6
When the dep_attrs function was written, it was used only in convergence
after checking a single resource. However, now we also use it to generate
data for the ResourceProxy objects, which is often done for all resources
simultaneously. That means doing O(n^2) dep_attrs() calls, which can really
slow things down when there is a large number of resources with complex
properties (or metadata).
This change adds an all_dep_attrs() call, which runs as fast as dep_attrs()
on everything except get_attr functions and their arguments, but only needs
to be called once instead of once per resource. (The get_attr function can
in future override the default implementation of all_dep_attrs() to be as
efficient as dep_attrs() as well.) The resulting data is cached in the
ResourceDefinition or OutputDefinition so that subsequent calls to their
get_attr() methods with different (or the same) resource names will use the
existing data.
Change-Id: If95f4c04b841519ce3d7492211f2696588c0ed48
Partially-Implements: blueprint stack-definition
Closes-Bug: #1684272
This unifies the 'lightweight stack' used in convergence with the how
things work the rest of the time: we now always obtain resource data from
the StackDefinition's ResourceProxy objects. This means that when we are
checking an individual resource, we will never create all of the other
Resource objects for the stack in memory (although we already avoided
loading data for them from the database) - now we will only create
ResourceProxy objects as needed.
Change-Id: Id7472557e26d172df88841ff7f20afdd7f5bfada
Implements: blueprint stack-definition
Due to unfortunate copy-pasta, the original implementation of
function.dep_attrs() iterates over items in a dictionary instead of values.
Dictionary keys cannot have dependencies - they must be hashable and
Function objects are not. So for each dictionary key we call dep_attrs()
recursively 3 times as many times as we need to (for the key/value tuple,
the key, and the value). For template snippets that are dominated by
dictionaries, this means it took three times as long and used three times
as much memory and twice as much stack space as it needed to.
Change-Id: I13781540483daf88202d221a9f517746eebf0346
Partial-Bug: #1684272
During stack updates in the legacy path, we copy resource definitions
back and forth between templates. In the case where the definitions
contain macros (which in practice means the If macro for conditionals),
they may rely on external state (in practice, the conditional
definitions) that is not available in the template they're being copied
into (e.g. in the case of an If macro referencing a new condition).
This change means that when we copy a template, the macros get resolved
so that only the chosen path of the If macro is represented.
This resolves the issue where trying to signal a resource during an
update fails when one of the already-updated resources in the template
contains an If macro that refers to a condition definition that is
newly-added in the new template.
Change-Id: I6d08507f43b0fcc4c0b5e848e97fa26033d839b2
Closes-Bug: #1699463
To generate its NodeData, a Resource must know which of its attributes
other parts of the template have dependencies on, which it determines by
asking the intrinsic functions in the template. Currently, the get_attr
function when used with no arguments depends on all of the attributes of
the resource (except 'show'), and it obtains this list from the Resource
itself (via the attributes_schema). This presents a chicken-and-egg problem
if we want to give Functions access only to a ResourceProxy based on the
NodeData.
Resolve this by returning a constant sentinel value (indicating that all
attributes are required) to the Resource, which can determine for itself
what attributes it has.
Change-Id: If209b589278fe1facb06e157fe76209fdd42a8a2
Partially-Implements: blueprint stack-definition
Error with path to the function is returned now in the
case of failed validation.
Closes-Bug: #1625723
Change-Id: Ib96a944c751543f3eb7cb5f8ba09911fc5cae586
Just as we store a weakref to the stack in the Function class, we must
also do the same with the template in the Macro class. Otherwise we end
up with circular references that don't get cleaned up until the garbage
collector runs.
Change-Id: Ia615de961f52e9fc5ef90e0668720b42c864a605
Closes-Bug: #1623706
Now that we raise them in only a single place each, there's not a lot of
value in having a custom exception type for condition-related errors in the
template any more, and in fact there was no code to handle them
appropriately in heat-api so it would have resulted in 500 errors to the
client if they ever escaped heat-engine anyway.
Just use the existing StackValidationFailed exception type. This provides
more flexible control over the path than InvalidConditionReference did.
Also, raising ValueError in the Invalid function means that
StackValidationFailed will be raised with the path, so it will be much
easier for users to find where they have used an invalid function.
Change-Id: Iaa6ca030fc144efc75e707910f84bfa7385fa9e2
A Macro differs from a function in that it modifies the syntax tree
directly, so that the result of inserting a macro in the tree might not be
just different output, but actually be based on resolving different
functions. A macro can remove syntax from the tree, modify it or fabricate
it from whole cloth prior to parsing.
The proposed If 'function' for implementing conditionals is a macro.
Arguably the Repeat function should be implemented as a macro also.
Change-Id: I2c5afe31d509ffedc9e0b2ef237f52d04687e0be
Partially-Implements: blueprint support-conditions-function
This changes:
1. Support 'Conditions' for AWSTemplateFormatVersion: 2010-09-09
2. Support 'conditions' for heat_template_version: 2016-10-14
3. There is no need to add a new HeatTemplateFormatVersion template,
because we decide to support conditions in AWSTemplateFormatVersion
and heat_template_version: 2016-10-14, so remove the
HeatTemplateFormatVersion.2016-10-14
4. Move the definition of function 'equals' to hot/functions.py
5. Mark 'equals' as condition function which supported in conditions.
Change-Id: I2e7bdfa1c2052e75f35f2bd0003cdc170188d8b8
Blueprint: support-conditions-function
Functions can't return functions, so we don't need to use `while`.
This doesn't change behaviour of code, just makes it more understandable
and convenient.
Change-Id: Ia1dac5a9583cb99db23a8838412e5bc02e93ef4c
Remove the incorrect __hash__ function added to the Function class in
46233d2c2a and make the class unhashable. Fix
the one place we were trying to hash it, which is when it appeared in the
deletion_policy of a resource.
Change-Id: I3d47c99470a2404183bccb80269a18623aa79041
The process of bringing us into compliance with the H405 pep8 rule has left
us with a lot of docstrings that either don't follow the recommendations of
PEP0257, are misleading, wrong, or nonsensical. This patch attempts to fix
them up.
Change-Id: Icfe90b5ba3102a7e13ab659038a8b2af9019e9e6
According to [1], A class that overrides __eq__() and does not define
__hash__() will have its __hash__() implicitly set to None. When the
__hash__() method of a class is None, instances of the class will raise
an appropriate TypeError when a program attempts to retrieve their hash
value, and will also be correctly identified as unhashable when checking
isinstance(obj, collections.Hashable).
A few other classes also implement __eq__ but we don't seem to use them
anywhere as keys in dictionaries, so they don't need to implement the
__hash__ function yet.
[1] https://docs.python.org/3.4/reference/datamodel.html?highlight=hash#object.__hash__
partial blueprint heat-python34-support
Change-Id: Idf24d462f386927496950e65a24e41b9aef69281
Circular references cause practically every bit of data that Heat uses to
remain in memory until the completion of an operation, and even then to
only be freed once the loop is detected by the garbage collector. By
breaking all of the loops using weak references, we can ensure that things
will get freed when they are no longer referenced without the need to wait
for garbage collection (which should also take a lot less time). This
change removes the loops from Resource and Function objects back to the
Stack.
Change-Id: Ibf80e95e69a2f27ed29754a2e0f1125e8eed0775
Closes-Bug: #1454873
For python3 compatible, using six.string_types instead of
basestring(). six.string_types represent basestring in
python2 and str in python3.
Change-Id: Ib4ba3d164f782601944023b99d6cc1ee2763ee85
According to http://docs.openstack.org/developer/hacking/ it should
be preferred to use six.add_metaclass instead of __metaclass__.
Change-Id: I1775035ea998b78a52c28ffe71dda2ede1377b31
All template functions contain only 'result' function for resolving
snippet of template. Current patch presents initial implementation of
validate method and support function 'validate' similar to 'resolve'.
Related-Bug: #1273490
Change-Id: Id81493bd72e75da746101c4c741eaf846abd8360
These functions can be substituted for their corresponding syntax in the
parse tree and lazily evaluated at the time their output is needed. Thus,
no distinction between statically-resolvable and runtime-resolvable
functions is required.
Change-Id: I3e89b6ee0d17b6755b35ab52b52a22a8d3db6c29