This had a similar problem as when null values were passed explicitly
before the fix for bug 1709620.
Change-Id: I02fb4c8815bc24d4c0c339022aa857dc344d58e9
Story: 2011108
Task: 49921
Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib.
Change-Id: I8f764e9ba46a4e2055be61eb0fe97d155ab1c70e
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
Six is in use to help us to keep support for python 2.7.
Since the ussuri cycle we decide to remove the python 2.7
support so we can go ahead and also remove six usage from
the python code.
Review process and help
-----------------------
Removing six introduce a lot of changes and an huge amount of modified files
To simplify reviews we decided to split changes into several patches to avoid
painful reviews and avoid mistakes.
To review this patch you can use the six documentation [1] to obtain help and
understand choices.
Additional informations
-----------------------
Changes related to 'six.b(data)' [2]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
six.b [2] encode the given datas in latin-1 in python3 so I did the same
things in this patch.
Latin-1 is equal to iso-8859-1 [3].
This encoding is the default encoding [4] of certain descriptive HTTP
headers.
I suggest to keep latin-1 for the moment and to move to another encoding
in a follow-up patch if needed to move to most powerful encoding (utf8).
HTML4 support utf8 charset and utf8 is the default charset for HTML5 [5].
Note that this commit message is autogenerated and not necesserly contains
changes related to 'six.b'
[1] https://six.readthedocs.io/
[2] https://six.readthedocs.io/#six.b
[3] https://docs.python.org/3/library/codecs.html#standard-encodings
[4] https://www.w3schools.com/charsets/ref_html_8859.asp
[5] https://www.w3schools.com/html/html_charset.asp
Patch 26 of a serie of 28 patches
Change-Id: I861e30278e66b680cd64ef659ff889414d4e55fb
The patch https://review.openstack.org/#/c/527001/ bypassed the None
list but did not consider the case where all the for_each lists are
getting their values from attributes and therefore are None.
This patch set fix it by also skipping the check if all the values
are None -- as len(value_lens) would be 0 in that case.
Closes-bug: #1732934
Change-Id: Iba574bbd3877f9ca1ca755c384b3ddb47aec711c
Attributes may return "None" before the resource is created,
so the length check for for_each lists in repeat function will fail.
Bybass the length check for the None list
Change-Id: Ida81060e62340b3ea37ba0fdd7443dc88ba6c5aa
Closes-bug: #1732934
Previously when calling function.validate() we passed the path to the
function in the template (used for debugging purposes) as a string. This
change allows us to pass it as a list of path components, as we do
elsewhere, so that higher-level code that catches StackValidationFailed
can deal with its components separately.
Change-Id: I017aa6f7511b8478ef8273522ab8087684ae71c6
When we add a resource to a template using add_resource(), strip any
references to resources that are not present in the template from
'depends_on'. This means that resources that are present and depended on
but conditionally disabled in the source template will not be depended
on in the destination template, where they're not present at all (and
thus would cause errors when the template is used).
Change-Id: I58a64cd523cc0d1da8cd39793f0c839189458d99
Closes-Bug: #1701677
Related-Bug: #1649900
This unifies the 'lightweight stack' used in convergence with the how
things work the rest of the time: we now always obtain resource data from
the StackDefinition's ResourceProxy objects. This means that when we are
checking an individual resource, we will never create all of the other
Resource objects for the stack in memory (although we already avoided
loading data for them from the database) - now we will only create
ResourceProxy objects as needed.
Change-Id: Id7472557e26d172df88841ff7f20afdd7f5bfada
Implements: blueprint stack-definition
Since function.dep_attrs() returns logical resource names (rather
than actual objects), we can just as easily use the StackDefinition to
calculate it instead of the Stack and Resource objects.
In the legacy path, we must ensure we use the StackDefinition from the
*new* stack to determine which attributes to include in the NodeData, since
that's what we're going to be using it for. In the convergence path the
current stack definition already contains the new template.
Also, update the *new* stack's definition with the NodeData obtained from
completed resources (in addition to the existing stack's), so that that
data may be used in calculating the dep_attrs for future resources. This is
required when get_attr functions are nested in the template.
Change-Id: I23efcc091eae53470f7f9cb3ca21e09f00f43808
Partially-Implements: blueprint stack-definition
Before returning the stack outputs, update the StackDefinition with the
resource data needed to resolve them. This will enable us to switch to
resolving outputs via the StackDefinition in a subsequent patch.
Change-Id: Ide3920385d9c4841c18024e1eb1456eed5c8eef5
Partially-Implements: blueprint stack-definition
Adds a new section 'permutations' for 'repeat'
function to decide whether to do nested loops
to iterate over all the permutations of
the elements in the given lists, the default value
is True if no specify and keep the same behavior as
before.
For example:
repeat:
template:
network: %net%
port: %port%
ip: %ip%
for_each:
%net%: [n1, n2]
%port%: [p1, p2]
%ip%: [ip1, ip2]
permutations: False
Will be resolve to:
[{network: n1, port: p1, ip: ip1},
{network: n2, port: p2, ip: ip2}]
Change-Id: I2a008400fb71453f6a78656f2e041ae2efa098a2
Blueprint: improve-repeat-function
Ensure that attributes that are referenced in outputs get cached (and
therefore stored in the database) even when they are not referenced by
other resources.
Change-Id: I667ab04f91edddef5c5dbec0a89d465110c312b4
Closes-Bug: #1660831
This helps with dynamically building templates, and is similar to the
add_resource method.
Change-Id: I51ce5c913438083d6d43eb6d8a8bf55cfa8ea3ee
Related-Bug: #1660831
Add new function 'contains' to check whether
the specific value is in a sequence. The resolve
result of the new function is a boolean value, so
we put it as a condition function too.
Change-Id: I90074070dd917df13345fb7c8d4ed09e4a76e864
Blueprint: support-contains-function
This replaces the deprecated (in python 3.2) unittest.TestCase
method assertRaisesRegexp() with assertRaisesRegex()
Change-Id: I405875738337433a36cba78023534615dea8b7b6
Adds function 'list_concat_unique' to concat
several lists using python's extend function
and make sure without repeating items.
Blueprint: support-list-concat-unique-function
Change-Id: Ia0e35eb578ebe91543c644155a86030b3258118b
Now the function 'list_concat' allows string type
args, like {'list_concat': 'I am string'} or
{'list_concat': ['v1', 'v2']}, this patch fixes the
validation to avoid string type args.
Change-Id: If6d73a90e25f38a37c0503fe0b1aaa93a6fbc380
Closes-Bug: #1685434
The already existing str_replace_strict function raises an error if a
param is not present in the template. str_replace_vstrict, newly added
in this patch, is identical but also raises an error if any of the
params have an empty value.
Change-Id: I5407135cc0435cfbad2d18964fe2119c999f67a3
This function uses python's extend function for lists in order to
concatenate lists. So, given the input: [['v1', 'v2'], ['v3', 'v4']]
the result would be ['v1', 'v2', 'v3', 'v4'].
This comes as an alternative to using yaql's concat.
Change-Id: I082833e73388540b29f71b78293fd36fe04b7b92
Query strings are not _required_ to have slashes escaped, and in fact RFC
3986 suggests that "it is sometimes better for usability to avoid
percent-escaping those characters" (Section 3.4).
TripleO is currently unable to use the make_url function because its
database URLs contain slashes in the query, and end up in config files that
don't allow percent characters. This is probably not going to be an
isolated occurrence, so don't escape slashes in the query string.
Change-Id: I72f84e737b042ecfaabf5639c6164d46a072b423
A large proportion of uses of the str_replace function is to build URLs out
of various components from various sources. This is invariably brittle,
with a failure to escape special characters, deal with IPv6 addresses, and
so on. The make_url function provides a both a tidier interface and correct
handling of these edge cases.
Change-Id: I61b6dff01cd509b3d1c54bca118632c276569f4e
The facade_resource intrinsic function needs to access the parent resource
of a stack, which it does via the stack.parent_resource property. Accessing
this for the first time loads the parent stack and all of its resources if
they were not already. This made sense when all nested stacks in a tree
were handled in-memory at the same time, but now that they are processed by
separate engines it is inefficient.
This change moves responsibility for lazy-loading the parent stack to a
separate ParentResourceProxy class, and makes access more efficient again
by avoiding the loading of resources if the stack was not already in
memory. It also resolves a circular reference between nested stacks and
their parent stacks.
We now have a well-defined API behind which we can potentially make further
efficiency improvements while giving third-party Function plugin developers
confidence that we won't break them without notice.
Change-Id: Ibfd80544889778f3499bcbe421b83f0a5aa6a7f7
Partially-Implements: blueprint stack-definition
This will allow the snapshotting of attribute/refid values to occur on both
the legacy and convergence paths (currently it is used only for
convergence).
Change-Id: I9a8fce9c6d22d84ec967087b62bff77f5a6de3db
Partially-Implements: blueprint stack-definition
Formalise the format for the output data from a node in the convergence
graph (i.e. resource reference ID, attributes, &c.) by creating an object
with an API rather than ad-hoc dicts.
Change-Id: I7a705b41046bfbf81777e233e56aba24f3166510
Partially-Implements: blueprint stack-definition
If a resource is conditionally defined, depending on it raises an error
in the case it's not defined. This patch fixes that by checking if the
resource is present in the template regardless of the conditions.
Change-Id: Iefae1fcea720bee4ed69ad1a5fe403d52d54433c
Closes-Bug: #1649900
We should allow the case where the keys collide, but the
replacement is the same as the current key (e.g a noop).
This is useful when you want to do a map_replace based on a
user parameter, then the default can be to do a noop replace
but the user parameter may also be any other (non colliding)
value.
Change-Id: I8cc5761c219616b4f8c18b3f44c4bf864c5457f1
Closes-Bug: #1652034
A modulo constraint will be used to restrict numeric values to
leave a certain remainder when divided with certain divisor.
E.g. we can use this to constrain values to even/odd numbers.
Change-Id: I9d7db4307be2a2b93cc928cf5912af7b49c72076
In many cases, a user would rather see an error result if a
str_replace param is not substituted in the template, rather than the
function silently doing nothing.
Since str_replace is set in its ways, introduce a new function
str_replace_strict which behaves identically to str_replace except
that a ValueError is raised if any of the param's are not found
in the template.
Change-Id: I8b8c69bb49dfeb74e05af4871602c20493b081eb
Atrributes would return `None` before the resource is
created, hence resource validation would fail when
using repeat function. Use empty list instead to avoid
it.
Change-Id: I93f65d6604944ccae40f65ef52754dcec69e9886
Closes-Bug: #1629899
Zane Bitter