Commit Graph

160 Commits

Author SHA1 Message Date
Tobias Urdin c253b38e21 Move CircularDependencyException to common
Moves the exception from inside the engine code
to the common code so that we can use it in the
API fault middleware.

Change-Id: I017b95153c358829501f6a5740918cdb005fb32f
2023-02-27 12:22:38 +00:00
Sean McGinnis fd6cf83554
Use unittest.mock instead of third party mock
Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib.

Change-Id: I8f764e9ba46a4e2055be61eb0fe97d155ab1c70e
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-05-05 08:42:11 -05:00
Hervé Beraud 8c96a4d856 Remove six and python 2.7 full support
Six is in use to help us to keep support for python 2.7.
Since the ussuri cycle we decide to remove the python 2.7
support so we can go ahead and also remove six usage from
the python code.

Review process and help
-----------------------
Removing six introduce a lot of changes and an huge amount of modified files
To simplify reviews we decided to split changes into several patches to avoid
painful reviews and avoid mistakes.

To review this patch you can use the six documentation [1] to obtain help and
understand choices.

Additional informations
-----------------------
Changes related to 'six.b(data)' [2]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

six.b [2] encode the given datas in latin-1 in python3 so I did the same
things in this patch.

Latin-1 is equal to iso-8859-1 [3].

This encoding is the default encoding [4] of certain descriptive HTTP
headers.

I suggest to keep latin-1 for the moment and to move to another encoding
in a follow-up patch if needed to move to most powerful encoding (utf8).

HTML4 support utf8 charset and utf8 is the default charset for HTML5 [5].

Note that this commit message is autogenerated and not necesserly contains
changes related to 'six.b'

[1] https://six.readthedocs.io/
[2] https://six.readthedocs.io/#six.b
[3] https://docs.python.org/3/library/codecs.html#standard-encodings
[4] https://www.w3schools.com/charsets/ref_html_8859.asp
[5] https://www.w3schools.com/html/html_charset.asp

Patch 28 of a serie of 28 patches
six fully removed now!
Thank you six for the rendered services!

Change-Id: If44ee4b565cc9390fa0422fba4dda080b4f90b98
2020-04-23 14:49:12 +02:00
Zane Bitter a29ccdcdb0 Handle unicode in constraints
Ensure that if the user provides non-ASCII descriptions or e.g. allowed
values in a template, that we can print them correctly wherever they
appear in API output (such as in error messages). Also allow all default
error messages to be localised.

Change-Id: Id2c309a33634b35a4f1f8b7ddf252db22bc46625
Story: #2003096
Task: 23188
2019-01-29 19:21:28 +13:00
Zuul cb3967d10a Merge "Change NovaClientPlugin to use MicroversionMixin" 2018-07-05 06:10:43 +00:00
rabi 72f18efe02 Change NovaClientPlugin to use MicroversionMixin
We don't need to maintain a list of versions supported in nova client
plugin. Supporting a latest microversion means supporting all
versions earlier.

Change-Id: I5df1ccd3ebf47b0772c903dbae0613b4f8041634
Story; #1764757
Task: 17358
2018-06-29 09:37:44 +05:30
Thomas Herve 820b5a8d84 Handle invalid depends field
Raise a proper error when something wrong is passed in a depends_on
field.

Change-Id: I9d27cc48a7f4c050fcc89ab5d07b19329d228955
Story: 2002544
Task: 22103
2018-06-25 15:07:33 +02:00
Thomas Herve 9f73a232d7 Return nested parameters for resource group.
This refactors the building of schema from parameter validation to use a
new method (which doesn't keep stacks in memory), and use that new
method for providing proper schema for resource group when the size is
0.

Change-Id: Id3020e8f3fd94e2cef413d5eb9de9d1cd16ddeaa
Closes-Bug: #1751074
Closes-Bug: #1626025
2018-02-26 14:57:01 +01:00
Zane Bitter 316b80e1d5 Ensure the whole ResourceDefinition is validated
We used to validate intrinsic functions in the ResourceDefinition by just
passing it to function.validate(). That worked when it also acted as a
CloudFormation template snippet but, since that behaviour was removed by
2c6fc7bcd6, that call now does nothing. This
replaces it with a validate() method on the ResourceDefinition itself.

This also improves path reporting for validation errors in deletion or
update policies.

Change-Id: I58c788002136ddf9bded309366e2072823b3ac77
Closes-Bug: #1732798
2017-11-16 18:17:23 -05:00
Zuul 0af0bcd2d1 Merge "Added tags attribute to the template parameter" 2017-11-16 12:29:33 +00:00
Steven Hardy 313faad5f1 Return Environment from validate_template
This is to enable preview of the merged environment
without merging the environment on the client side.

Related-Bug: #1635409
Change-Id: I7ec3af729a65164230153021f438bf226cc5e858
2017-11-14 17:35:52 +00:00
Saravanan KR a1a0609e8e Added tags attribute to the template parameter
In some scenarios, it is required to categorize the parameter.
Associating tags with the parameter, it is will be easy to
categorize the parameter and add simple validations to users.
This patch introduces a new attribute 'tags' with the parameters,
which is a list of strings.

Change-Id: I20fc95d606b0b8a08d3b46bf33f4860bff49c74f
2017-11-13 15:51:19 +05:30
Zane Bitter 76aa7c87cb Check for circular dependencies in template validation
We ought to iterate over the dependency list in the validate_template call,
to check for any circular dependencies. This check was removed by
d12cbe7959 in order to avoid trying to
calculate implicit dependencies added by resource plugins (as e.g. many
Neutron resources do) at a time when we don't have sufficient data to do a
good job of it anyway.

However, the previous patch d12cbe7959 split
apart the calculation of explicit and implicit dependencies, so we can
easily do this while still checking for circular references in the
explicit dependencies. This patch splits the dependency calculation into
two phases - the first using only data available from the template, and the
second querying the resources for implicit dependencies.

Use strict_validate = False as the criterion for when to avoid calculating
implicit dependencies, since this corresponds exactly to when we don't have
enough data to do so.

Change-Id: I21c63dcc8d1cad20dbc237b472670753779b0ff8
Closes-bug: #1691575
2017-07-21 10:27:52 -04:00
ricolin 7cbf528da5 Allow only validate external resource template
When we choose only validate resource template, and external resource
id was set, we should consider that external resource template is
valid (since we only required `external_id` from that resource
template format). So the result of this validate will be None
(template validate passed).
Closes-Bug: #1702977

Change-Id: I8b33c957a7185a0f52356f412b13609ba1a7b469
2017-07-20 13:16:38 +08:00
Jenkins 7698eb0501 Merge "Load all templates for generating parameter schema" 2017-06-19 04:57:30 +00:00
Jenkins 32fcda12bf Merge "Use mock in test_validate.py" 2017-05-31 08:44:53 +00:00
Thomas Herve 7221cd2eba Load all templates for generating parameter schema
To be able to handle parameter defaults properly, we need to load all
possible templates when merging the environments, to generate an
approximate schema.

Closes-Bug: #1669571
Change-Id: Idab781d3e6af92990c357a727aa60ec0aa8f5f05
2017-05-30 13:15:57 +02:00
Saravanan KR b15b7e568c Add ParameterGroups for the nested stack during stack.validate
ParameterGroups is added only for the parent stack and all
the nested stack are missing this information. Added
ParameterGroups for all the nested stack also, if present.

Change-Id: I032144733bde916f8de8644121b9fb1ef29baef2
2017-05-18 15:10:32 +05:30
Xiao Liang 2a6dc1a3d0 Check if image_identifier is UUID
Check if image_identifier can be image UUID before calling glance-api.
This could save some time if image name is supplied.

Change-Id: Ied531f35d723a818ab3387582a53a2a96dfb7668
2017-01-06 11:43:45 +08:00
shizhihui c054a6943d Use mock in test_validate.py
In heat, some unit tests still use mox3,
which will make the test speed more slower,
so I decide to remove it.

The file: heat/tests/test_validate.py

Change-Id: I5ca027d3d47ba072039eb5874b87de7703bdb0a4
2016-11-18 11:40:40 +08:00
Zane Bitter 9ce052f67b Lazy-load outputs
Rather than passing around an explicit flag to say whether a Stack object
should parse its outputs during initialisation, just parse them the first
time they are actually needed.

Change-Id: I4b8a84883902261592fbfe8d5d5b1892702a3d03
2016-09-06 08:48:48 -04:00
Zane Bitter bc1adff98e Add unit tests for invalid keys in resources
This is only working more or less by coincidence at the moment, so add
tests to ensure that it doesn't get broken by future changes.

Change-Id: I9d78d2b4f8adcd0ba9d6929c62e3975688498aaa
2016-09-01 17:08:42 -04:00
Jenkins e660cea228 Merge "Improve validation during template.parse" 2016-07-08 08:36:41 +00:00
Oleksii Chuprykov d6e36c2dd2 Improve validation during template.parse
Now error with a path to the problem part of template will be
raised while parsing template. It affects both resource
definitions and outputs parsing. Note, that during parsing of
template we only instantiate the function objects, but doesn't
make any function validation. The function validation still
doesn't return any path with error. This will be fixed in the
other patch.

Change-Id: Ic5b960e79a54e88087803bc092f614911d7e995a
2016-07-06 18:48:24 +03:00
Dmitriy Uvarenkov 558a8cd1e8 Correct message when extension is not available
Now if extension is not available, user gets
message 'Service endpoint not in service catalog'
which is incorrect. Patch corrects this behavior
and adds improved messages with more info.

Change-Id: I9b6c0d2921519590ef105be35e31db29cd1e3ecc
2016-06-27 11:09:32 +03:00
Kanagaraj Manickam a7b8378e7f Adds tag support for glance image resource plugin
implements blueprint heat-glance-image-tag-support

Change-Id: I8fc64e411d78b9abe7795aafdcb901fe110e6610
2016-05-11 18:41:30 +05:30
Kanagaraj Manickam 29e5d96d50 Adds ignore_errors for template_validate in engine
It enhance the validate_template to ignore the given
set of errors while the template is validated.

implements blueprint heat-template-validate-improvements

Change-Id: Id6d85664ae576b6a5d18dbbe84ddf0122fbfaa40
2016-02-04 17:08:24 +05:30
Rabi Mishra 29692ee4f5 Use custom find functions for image lookup
Use get interface and custom find functions to look for
images by id/name. Find interface is now implemented in
the client plugin itslef, as glanceclient v2 does not
support it. Once it's implemented in galnceclient v2,
we can leverage images.find interface directly.

This also leverges openstack.common.apicient.exceptions.

Change-Id: Ibf85495b9d5533c759c4e63284806f11e6e45413
2016-01-27 09:56:01 +05:30
huangtianhua 4f19c8a26c Define engine service in setUp()
Init test engine service in setUp() instead of defining
in each tests.

Change-Id: I6ecb5355b0d07248a0d3d5a908c35a6d33d13ed1
2015-11-19 01:15:46 +00:00
ricolin 3b0ba9c633 Change namespace for Nova resources and tests
Change namespace to more specified.
Use path to specify resource or test should be enough.

Change-Id: I8f42e4ab446b0369d4c53fa4d3a734689ab541e3
2015-11-18 08:42:29 +08:00
Steven Hardy ed64822d4d Convert validate_template validation path
Currently the validation code for template-validate is different to
that used for create/preview/update, which can lead to inconsistent
results as the implementations have diverged.

So instead align more closely with the actual validation, which
should also enable easier validation of nested stacks which is
currently not possible.

Change-Id: Ibf93a170ab381a42a46ea414c3b134cbe0c3f232
Closes-Bug: #1467573
2015-09-21 14:27:16 +01:00
Steven Hardy 4ddd4cb2bd Always return default via template-validate "Default"
Currently we redefine the value of Default when a user value is
provided, which means the data returned doesn't match the schema
defined in the template.  Instead always return the actual default
and add a "Value" key that contains the user value, if provided.

Change-Id: If4ed0b733d4e61c84167063c36cdcb234b001136
Closes-Bug: #1497310
2015-09-18 18:59:31 +01:00
Kanagaraj Manickam 10e55e1945 Engine method validate_template fails for unavailable resource type
When validate_template() method is called with unavailable resource type,
it will fails the template validation.

Change-Id: Id141b7e63d7a58fade8a29d0dd5c6a1f9c847f5e
Closes-Bug: #1473025
2015-09-01 16:45:08 +05:30
Sirushti Murugesan 800f38aac4 py34: test_hot and test_validate
* use hashlib.algorithms_available in PY3 and hashlib.algorithms in PY2
* use e.args[0] instead of six.text_type(e) which will otherwise
  result in the unicode literal being flattened.
* add stubs to prevent actual API calls in tests.
* constraint the test template to help in more accurate testing which
  otherwise results in iterating through other unnecessary items.

partial blueprint heat-python34-support

Change-Id: I7480555330161cf6f197c83f3245685511284307
2015-08-13 13:59:19 +05:30
Rabi Mishra bb15fcc3be Fix validation error for parameter group
Fix bug introduced by change
I2b1b096b314b52b2a51d09c47c46b902e6647c59

Change-Id: I9cda70f60c3c7651e433f9464ea5926d64659721
Closes-Bug: #1465110
2015-06-30 11:38:33 +05:30
Angus Salkeld caa1bd8602 Produce more meaningful exception messages in nested stacks
This produces a nested exception like:
 'ValueError: resources.nested.resources.my_server: it is broken, sorry'

This re-uses the path mechanism that StackValidationFailed exception
uses.

Change-Id: Id5204c15ee96784e04522ab3c5a8e66900f9a1d3
Closes-bug: 1459837
2015-06-24 08:52:08 +10:00
Peter Razumovsky 5791d52c95 Improve StackValidationFailed resp param_groups
Use StackValidationFailed exception's parameters
error and path in parameter groups validation for
fuller error message: which error has been raised,
where this error has been raised and why.
This patch relies on topic [1].

[1]: https://review.openstack.org/#/q/status:merged+project:openstack/heat+branch:master+topic:improve-validation-error,n,z
Change-Id: I2b1b096b314b52b2a51d09c47c46b902e6647c59
2015-06-03 14:05:54 +00:00
huangtianhua 0ba3206b40 Use EntityNotFound instead of ImageNotFound
There are too many **NotFound exception classes defined
in exception.py. This patch will define a general notfound
exception: EntityNotFound. And will use it instead of
ImageNotFound exception.

Change-Id: I2a09637be97905548b29673273e8579c69d42d52
Closes-Bug: #1461343
2015-06-03 15:40:12 +08:00
Rabi Mishra d3d4f62395 Add validation for parameter_groups
This fix adds validation to raise error when the parameter_groups
is not a list and parameters in a parameter_group is not a list.

Change-Id: I35cdca9ce07522b5dbcfcf02d2c30393a4c15d1f
Partial-Bug: #1454559
2015-05-18 18:46:56 +05:30
Angus Salkeld b87ecfc614 Catch warnings consistently during tests
Use unittest2.assertWarnsRegex() for making sure we generate the
required warnings.
Use WarningsCapture to not print the other repeated warns to the
console.

Change-Id: I7223f8956394208eaf2eb8a1d651ba1425128bc9
2015-05-14 17:35:08 +10:00
Jenkins bc60a1543b Merge "Improve StackValidationFailed response in properties" 2015-04-20 03:54:09 +00:00
Ethan Lynn 65a77a3105 Show deprecated warnings when import parser
Classes in parser.py are already split to stack.py and
template.py, it's not recommended to import it.

Closes-Bug: #1442011
Change-Id: Ia8cc0ca07e7926fe8b7d9f5c89b39053a799e689
2015-04-14 17:49:15 +08:00
Peter Razumovsky 3ad4614276 Improve StackValidationFailed response in properties
In some cases, there is no information about resource and
section, where Property error raised. This patch improves
StackValidationFailed msg, so this msg look like "Property
error : resource_name.section_name.key_name: error_msg", where
section_name is section, where Property error raised, e.g.
'update_policy'.

Change-Id: Iab2a6acdec254b39983de420ab03f994cff48d89
Closes-bug: #1358512
2015-04-10 15:42:21 +03:00
Michal Rostecki d9d68cf52b Novaclient v2 instead of v1_1
Nova API v1.1 is now deprecated and causes warnings.

Change-Id: Ib4b57a308b7637f4015a07b4e888ccd6347cb947
Closes-Bug: #1437158
2015-04-03 11:24:59 +02:00
Angus Salkeld e3950a4e72 Pass the environment into the Template class and not Stack
When we moved parameters from the stack table to the template table
we did it in a "flexible" way, but this led to some difficult to
diagnose problems. So this patch totally removes "env" as an argument
to the Stack class to make absolutely sure that we are doing this
correctly. I came across these bugs when rebasing decouple-nested.

- env was not getting passed into instance group make_templates()
- template deepcopy was not working correctly (causing some problems
  when deleting backup stacks)

Change-Id: I85cfb01c68ff5a44e2ff4f6e446870efb7205933
2015-03-18 10:29:55 +10:00
Peter Razumovsky 3f16eebd06 Add resource_definitions check to Stack.validate
We should check resource attributes' types in validate method
for preventing stack creation with incorrect resources. So
this patch splits resource_definitions() methods on
validate_resource_definitions() and resource_definitions()
methods and add validate_resource_definitions() to Stack.validate
method.

Change-Id: I16b35fc105154d9391f64285e90c396537fe3bb7
Partial-bug: #1397969
2015-02-27 15:46:27 +00:00
Jenkins dca4d59498 Merge "Enhanced error handling in outputs block" 2015-02-01 22:17:21 +00:00
Tetiana Lashchova 15ba5c1eb0 Enhanced error handling in outputs block
Change-Id: I20cab805d7a423efc008049c447bde0747530970
Closes-bug: #1411721
2015-01-28 13:40:23 +02:00
Pavlo Shchelokovskyy cb8712281d Enable H904 style checking rule
Use parentheses instead of backslash for line continuation.

Plus, usage of escaped quotes in long strings is avoided where it makes
sense.

Change-Id: If2e78012b85a4430c6f03f65784cac2d032cf116
2015-01-20 11:14:06 +02:00
Jenkins 967b1a3779 Merge "Always use fakes_v1_1 for heat.tests.v1_1 fakes import" 2014-12-12 02:29:36 +00:00