The abstract base classes previously defined in 'collections' were moved
to 'collections.abc' in 3.3. The aliases will be removed in 3.10.
Preempt this change now with a simple find-replace:
$ ag -l 'collections.($TYPES)' | \
xargs sed -i 's/\(collections\)\.\($TYPES\)/\1.abc.\2/g'
Where $TYPES is the list of moved ABCs from [1].
[1] https://docs.python.org/3/library/collections.abc.html
Change-Id: Ia282479bb1d466bd2189ebb21b51d91e89b9581e
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Six is in use to help us to keep support for python 2.7.
Since the ussuri cycle we decide to remove the python 2.7
support so we can go ahead and also remove six usage from
the python code.
Review process and help
-----------------------
Removing six introduce a lot of changes and an huge amount of modified files
To simplify reviews we decided to split changes into several patches to avoid
painful reviews and avoid mistakes.
To review this patch you can use the six documentation [1] to obtain help and
understand choices.
Additional informations
-----------------------
Changes related to 'six.b(data)' [2]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
six.b [2] encode the given datas in latin-1 in python3 so I did the same
things in this patch.
Latin-1 is equal to iso-8859-1 [3].
This encoding is the default encoding [4] of certain descriptive HTTP
headers.
I suggest to keep latin-1 for the moment and to move to another encoding
in a follow-up patch if needed to move to most powerful encoding (utf8).
HTML4 support utf8 charset and utf8 is the default charset for HTML5 [5].
Note that this commit message is autogenerated and not necesserly contains
changes related to 'six.b'
[1] https://six.readthedocs.io/
[2] https://six.readthedocs.io/#six.b
[3] https://docs.python.org/3/library/codecs.html#standard-encodings
[4] https://www.w3schools.com/charsets/ref_html_8859.asp
[5] https://www.w3schools.com/html/html_charset.asp
Patch 5 of a serie of 28 patches
Change-Id: Idb037ded55698790fc1658896f1e2dcdce89f3f9
Ensure that if the user provides non-ASCII descriptions or e.g. allowed
values in a template, that we can print them correctly wherever they
appear in API output (such as in error messages). Also allow all default
error messages to be localised.
Change-Id: Id2c309a33634b35a4f1f8b7ddf252db22bc46625
Story: #2003096
Task: 23188
This reverts the commits f5c32ad8fd and
14fdf72b000c82a80abb2587189dd7c6c7dfa0a0e.
The constraint never worked and the stuff to pass the template to
constraints (which was broken because we actually passed a
ResourceDefinition instead) is a pain. Just get rid of it rather than
fix it.
Change-Id: I4e1e787ad94ac1951f472ea066a9b1c9d3e03611
Closes-Bug: #1661403
A modulo constraint will be used to restrict numeric values to
leave a certain remainder when divided with certain divisor.
E.g. we can use this to constrain values to even/odd numbers.
Change-Id: I9d7db4307be2a2b93cc928cf5912af7b49c72076
The OS::Heat::Value resource provides a "value" attribute that may be
calculated through references to other parameters or resources. Other
resources may then reference this value.
The primary motivation is to avoid having to create nested templates
that only exist to execute these kinds of data transformations.
This resource's "type" property is optional but recommended.
Change-Id: I72fdab8261336644a9545668fac6f4e49a785e36
The validate method for a constraint only accepts the value being
verified. This is sufficient for most data checks (range, regex) and the
calls to one of the service clients. But it is limiting in the sense
that it prevents us from ever having constraints that span multiple
parameters (such as ensuring that two values are compatible).
What caused this is the atttempt to add a constraint to ensure a
parameter's value is a valid Heat resource type. To verify this, the
constraint code needs access to the resolved environment. I opted to
pass the entire template down instead to make things more flexible.
The call to the BaseCustomConstraint (which appears to be the entry
point for most plugins) will attempt to use the new signature with the
template, but will fall back to the original call for backward
compatibility.
Change-Id: I5dc94c964b35578e84da641278a4175a37038498
Add a new "immutable" boolean field to the parameters section in a HOT
template. A value of True will result in the engine rejecting
stack-updates that include changes to that parameter. When not
specified in the template, "immutable" defaults to False to ensure
backwards compatibility with old templates.
blueprint: immutable-parameters
Change-Id: If17df0b97e96ff2a1926451634e4744d6c3e26f1
The oslo.utils.reflection.get_class_name() handles more variations
of where a class name may come from (on) python 2 and python 3.
Its usage allows getting more accurate class names so we'd better use it.
Change-Id: Iff0e109a558d0185f126781369bac216da930bca
Since we replace not found error with EntityNotFound instead,
we can now consider use a common function for validate_with_client in most
of constraints. For some constraints that have a more complex validation will
overwrite the function(This is what we already used).
Change-Id: I1c554ba7848279cd60db1661fe7ebbd5b72235d8
* Sort the output for consistent checking in tests
* don't use %(x)r, instead use %('x')s. repr flattens
the unicode literal too along with the string.
partial blueprint heat-python34-support
Change-Id: I77b1c4a41aaf3d54008ec0a24be219e0292131d0
Currently heat caches validation results with the following key:
<service name> + <value>.
The patch changes this to
<service name> + <tenant_id> + <value>.
It allows names to be safely used and not be confused with
values received from other tenants.
Change-Id: Idd3fcf6afb3990be7df25fa1c5ef0c58222a5967
Closes-bug: #1478525
The patch adds support of oslo.caching in heat and enables caching
when heat requests OS services and validates custom constraints.
So it allows to increase performance of validation (cause we
don't need to call requests for the same values) and
handle the cases when stack have a lot of instances (and
glance fails because heat is ddosing it).
So the logic introduced in this patch is the following:
- heat request value from validation cache
- if value is found and it is not expired then we return it
otherwise we need request client plugin about validation
- if value is valid then store result in cache
otherwise do not store it.
The patch also introduces list of configurations that needs to be
specified when using caching.
The configuration for caching is modular. It consists from
global toggle for caching (disabled by default because in-memory
cache is leaking and memcached is not installed by default) and
local toggles for specific piece of functionality
(constraints cache). Local toggle is enabled by default but it
won't be activated without global toggle(enabled=True in
heat.conf).
DocImpact
Implements: blueprint constraint-validation-cache
Change-Id: I0a71c52a5c003f0fc6be1762647ffe0146a3b387
The current schema of constraints allow to use default values for
required=True. However we never use it except in unittests, because
user should set it always, which looks like not required property.
This patch adds checking for constraint schema class with warning
message.
Change-Id: Ic6ace30742e887069f119a14ffb091f6bac9e682
When user provides it's own parameter's value, there is no
reason to check custom constraint on default value of this
parameter. And if there is no user value, default value
validates anyway. So custom_constraint validation of default
values is unnecessary.
Change-Id: Ied3feefc7d062ddb20d42cfc5b2c30be4fdc3227
Closes-bug: #1394100
For python3 compatible, using six.string_types instead of
basestring(). six.string_types represent basestring in
python2 and str in python3.
Change-Id: Ib4ba3d164f782601944023b99d6cc1ee2763ee85
Since i18n.install() is deprecated, remove it from heat codes and
import i18n._() to where it needed.
blueprint oslo-i18n
Change-Id: Icefada18b5a33112b425cd90d31d3a6a5f06188a
Schema.to_schema_type() should convert values passed to it to the
appropriate type. So if the user passed a version number of 9.4 and
they defined it in the template with a string type, we should get the
string '9.4' back, instead of the error "The value '9.4' is invalid for
type 'string'".
Co-Authored-By: Richard Lee <rblee88@gmail.com>
Closes-Bug: #1353701
Change-Id: If7abd129dc7819c6ac01f9bf7dfdbe397ec09c65
Move InvalidSchemaError to heat.common.exception and pass a keyword
argument so that it can be serialized properly.
Change-Id: Iada3273b2b0b129bc0125b68fecf1e0efc081611
Closes-Bug: #1336740
Having the Stack create the Clients object means that custom constraints
do not have access to the already created clients. This greatly
complicates test mocking, and results in unnecessarily created
constraint-scoped clients.
This change makes the Clients be created in RequestContext on demand.
Constraints use the Clients associated with the context.
Mocking client create calls is simplified, generally needing to
call _create once instead of MultipleTimes()
Change-Id: If67e2d6761003558b32909c4655fadb4d0630b86
This commit caused a regression in tripleo when trying to signal
waitconditions.
This reverts commit de8b071acf.
Fixes-bug: #1344079
Change-Id: Ib87eb81b687497a6669da38cb43b4f525cafa754
Having the Stack create the Clients object means that custom constraints
do not have access to the already created clients. This greatly
complicates test mocking, and results in unnecessarily created
constraint-scoped clients.
This change makes the Clients be created in RequestContext on demand.
Constraints use the Clients associated with the context.
Mocking client create calls is simplified, generally needing to
call _create once instead of MultipleTimes()
Change-Id: If22f3258aa9ff906bd60977cc9c2752f18c11c65
Mutables in the method params defaults might cause errors and
that's why it's anti-pattern in most of the cases and should be
removed.
Change-Id: I8bc284f12ce72082a0482410ec2c20c2fc087a4b
Closes-Bug: #1327473
This patch makes Constraint objects aware of the data type of the
owning Schema object so constraint validation can apply potential
type casts based on the data type.
This is especially useful and necessary for an AllowedValues
constraint and numeric values, where items may be defined as
strings or numbers and cause comparison issues with user values
at runtime.
Change-Id: Id1884b4968993fc9356e7ba4ad03dced4cd00d4e
Closes-Bug: #1321540
This patch fixes all the missed and newly introduced violations of the H306
style checking rule and enables it in tox.
Change-Id: I5566e66421d5397c0103958f02ea38e6e3c68892
Implements: blueprint reduce-flake8-ignored-rules (partial)
This patch moves validation of constraints out of the constructor of
schema objects into a separate method that can be invoked on the
schema objects after creation. This gives us more control over when
validation shall be invoked.
This patch fixes an issue with custom constraints that could not be
validated when having a default value, since they require the RPC
context to be present. However, the context was not present in the
places where needed.
The short-sighted fix would have been to change a lot of method
signatures to pass the context properly, but that did not seem to be
really clean. The better fix seemed to be to move validation into
a separate method that can be invoked with the proper context. This
will also give us more control on when we do validation, and which
validation we do.
This patch
Closes-Bug: #1314240
This patch will also enable to fix
Partial-Bug: #1314401
For that bug it will be necessary to switch off validation under
certain conditions, which was not possible before when doing
validation right in the constructor.
Change-Id: I19e1fb520551eb42bf36bb380f1cc28c81bbaedd
This refactors constraints to use a shared base class that includes the
error message when validation errors happen.
Co-Authored-By: ala.rezmerita@cloudwatt.com
Closes-Bug: #1297371
Change-Id: Ia382f53d85d83c8c474ad8bda93c6d65f16cfca9
Change exception.py to accommodate future enablement of lazy translation.
Specifically, encode unicode messages to byte strings as gettextutil.Message
doesn't support __str__.
Change-Id: I16ab67c32d2050d83c1b51b84b24c0a60675aff7
Replace str with six.text_type so that unicode exception messages can be
safely processed, and the change is PY3 compatible.
Closes-bug: #1295443
Change-Id: Ic27ec365b82797df37ae53f38d5d31e70cb65c67
This changes stack creation to return an error message instead of an
exception when (some) validation fails. It doesn't convert all validation
errors to keep the patch at a reasonable size.
Change-Id: I6364bccacbc1d447a99ea61565c96fcc90bd21f9
Partial-Bug: #1276623
This change will add the ability to return the parameter label
with the parameter in the template validate call.
Change-Id: If48db3bf7f25396485f5bc8bb1cc8aeed4f8e7b0
Implements: blueprint add-parameter-label-to-template
This enables validation of of the parameters section of the template, so
that operations like template-validate now return a proper error if
something is wrong.
Closes-Bug: #1265862
Co-Authored-By: cedric.soulas@cloudwatt.com
Change-Id: Ic5d67f18d7bc53fd3e4e765ca221e21be0b9546d
This adds a new registry of constraints and a CustomConstraint class to
be able to validate against external services.
blueprint param-constraints
Co-Authored-By: cedric.soulas@cloudwatt.com
Change-Id: Iffaf2c1179eaf5d88cdea02b63782cea9f3fc616
This makes context optionally available in properties and during
constraint validation, to be able to add more advanced constraints which
make API calls.
blueprint param-constraints
Change-Id: I6ec1d6bc952d82029638caea32567036e29e4881
This patch is last in a series of patches to unify the code for
property and parameter validation based on a common Schema class.
The current patch refactors the implementation of the Parameters
Schema to be a sub-class of the common Schema class in constraints.py
so that most of the schema validation code can be shared for property
and parameter validation and no two different code paths for mostly
indentical functionality exist.
Change-Id: I4a1dc2e65e7b118347933bdf18aba49c058aa28b
Closes-Bug: #1230229
Implements: blueprint schema-code-consolidation
Stephen Finucane