1.As mentioned in [1], we should avoid using six.iteritems to achieve
iterators. We can use dict.items instead, as it will return iterators
in PY3 as well. And dict.items/keys will more readable.
2.In py2, the performance about list should be negligible, see the
link [2].
[1] https://wiki.openstack.org/wiki/Python3
[2] http://lists.openstack.org/pipermail/openstack-dev/2015-June/066391.html
Change-Id: I45fa65427318e1c35bb521de46e81ea12ca7b770
Hot Package does shutil.copy before checking that the file
exists, which causes an IOError to be thrown.
Change-Id: I11bbb468f53253107ff84e5c8e0d340d6102c4bf
Closes-Bug: #1628688
`Scope` keyword is introduced to declare method visibility.
For that package format version is incremented to 1.4.
`Scope` attribute can have two possible values:
* `Session` - regular method that is accessible from anywhere in
the current execution session. This is the default if the
attribute is omitted;
* `Public` - accessible anywhere, both within the session and from
outside through the API call (i.e. it is an action in current
terminology).
It allows to declare static actions with the following syntax:
methodName:
Usage: Static
Scope: Public
For backward compatibility `Usage: Action` remains valid and
acceptable but it is deprecated for format versions > 1.3. An
exception is raised if both `Usage: Action` and `Scope: Session`
are specified.
Change-Id: I5a6d664d017d5ea99a5159d7eb2e111143c7b0a3
Partially-implements: blueprint static-actions
* Replace len(MultiDict.keys()) with len(MultiDict). On Python 3,
MultiDict.keys() now returns a generator which has no length.
* Replace json with jsonutils to fix Unicode vs bytes issues.
* Use a bytes strings for the HTTP body, not Unicode strings.
* Replace LOG.exception() with LOG.error() when the argument is a
string, not an exception object.
* Only declare PackageException.__str__() on Python 2. The default
implementation of __str__() is fine on Python 3. Moreover, the
'message' attribute has been removed from Exception on Python 3.
* tox.ini: run the following API v1 unit tests on Python 3.4
- test_actions
- test_catalog
- test_env_templates
- test_environments
- test_sessions
Blueprint: murano-python-3-support
Change-Id: I4c8bedef019495477ae668769595ac4abc1ac0f4
Generated UIs of the non-muranoPL packages (HOT, cloudify_tosca and
CSAR) reference the type of object model's object by class FQNs.
However, when these packages are uploaded to glare, their class
contents are not indexed since the client which uploads them has no
knowledge of the generated classes.
To workaround this issue it is proposed to reference objects not just
by class FQNs but to include the package FQN and the version, so the
engine does not have to rely on Glare's indexes.
Change-Id: I6175e89b68bcdfc29d33ae3616d6ecba662f2509
Partial-bug: #1565805
HeatStack class expects to be owned by some other and uses it to find
OpenStack region name it belongs to. If there is no owner provided then
the default region is used which is random for multi-region setups.
MuranoPL code that was generated from HOT and TOSCA provided that owner
on "deploy" phase but not on "destroy". So the stack that was
successfully created earlier could never be deleted in multi-region
OpenStack installations for non-default region.
Change-Id: I19ca0c7129b073fdd852b22a6a7f84f7ee377ee7
Closes-Bug: #1565777
Added tests for new logo validation behaviour which
restricts file type of the logo image uploaded.
Also fix i18 import in package_base.py
Closes-Bug: #1497257
Related-Bug: #1560547
Change-Id: I772e4b1fc56de86171938a536197b253a131645c
Used PIL to allow support for png, jpeg and gif file formats
for upload. An exception is raised if the image size exceeds
500 KiB.
Change-Id: Ifd2b8bf689214022664b7bccf846c0de63cc0bfe
Closes-Bug:1497257
The Environment class got two additional properties:
region - name of the region where environment should be deployed. If
null then home region is used.
regionConfigs - config property (taken from class config) that specifies
region parameters. It is a dictionary of a form regionName -> regionSettings.
Currently the only setting available is a agentRabbitMq that is itself
a dictionary containing the same options as [rabbitmq] section of murano.conf
Thus the configuration file io.murano.Environment.yaml may look like:
regionConfigs:
RegionOne:
agentRabbitMq:
host: rabbitMqHostName
port: 5672
login: murano
password: murano
virtual_host: /
ssl: false
ca_certs: null
Implements Blueprint: assign-environment-to-region
Change-Id: I3c84c96f8eeae738f83ed7b3691be38100b30e1d
With this commit most of the Python 3 compatibility
issues in murano-engine are resolved.
If run on yaql with https://review.openstack.org/#/c/286110/
fix all of the unit tests except for one success.
The only failing test is base64 encoding/decoding test
which require rethink of resource management to get away
from string types for binary content
Change-Id: Iee87d27fe4f04118202de07f376d41fbf2c90f54
Now it is possible to
1) Define metadata classes that describe arbitrary metadata.
Meta-classes has all the capabilities of regular classes and
in addition has 3 new attributes: Cardinality, Applies and Inherited
2) It is possible to attach meta-class instances to packages,
classes (including other meta-classes), properties, methods and
method arguments. Each of them got new "Meta" key containing
list (or single scalar) of meta-class instances
Implements: blueprint metadata-in-muranopl
Change-Id: Ieac999a877221a9ecd7d7379b39557036e882aa7
* Remove all code that now available as part of yaql 1.1
* New format version MuranoPL/1.3 that is identical to 1.2
but protects apps that uses yaql 1.1 new features from being
imported to Murano Liberty
Change-Id: I2d8fc7050403be082e9389ebe5e682816ec6aca1
Changes APIPackageLoader class, to enable it to persist downloaded
packages on disc. Adds synchronization between eventlet threads and
different murano-engine workers, since it's now possible for
several engines to attempt to download the same package
The functionality is controlled with 'enable_package_cache' conf
variable.
Targets blueprint: murano-engine-package-cache
Change-Id: Ia7bc60ea44463724758c9313683c28b5d736bbd5
The builtin basestring abstract type was removed in python3,
use str instead, we need to use six.string_types to replace
the basestring for py3 compatibility.
This patch was generated by the following tool using 'basestring'
option.
https://github.com/haypo/sixer
Command:
python sixer.py -w basestring murano/
This also adds a check to murano/hacking/checks.py that should
catch this error in the future.
Blueprint murano-python-3-support
Change-Id: I26c236ce6dd2fffd6a4ca50e55ad62deb01bd9dd
This also adds a check to murano/hacking/checks.py that should
catch this error in the future.
Blueprint murano-python-3-support
Change-Id: I172e257d0b8a89eff89e35a2f87bb42d769cad62
This also adds a check to murano/hacking/checks.py that
should catch this error in the future.
Blueprint murano-python-3-support
Change-Id: Ib8b2b91a40577d18739f0c1bf2c37b7cf4060560
The Unicode type is 'unicode' in Python 2 and 'str' on Python 3.
This patch replaces unicode with six.text_type to make Murano
more compatible with Python 3. Also it replaces the tuple
'(str, unicode)' with six.string_types for instance comparision.
Blueprint murano-python-3-support
Change-Id: I4db27afdb556ec30dd36d7d4e170055274edfa8b
This commit does dot HOT template outputs (for Heat.HOT/1.0 format)
the same that was previously done for the inputs. Instead of generating
MuranoPL property per each output now all outputs are put into a
single 'templateOutputs' property. This allows to eliminate the constraints
on output names, not to have collisions with input names and remain
consistent with previous changes.
Change-Id: Ibbd0a2f410c55385cd3cb9e85ea135a36460afe7
Closes-Bug: #1520954
Since python 2.2, use of types module is not prefered way to
access the type hence it is replaced by using names of built-in
factory functions.
Change-Id: Ibdcc1ae75204f67076edab5edcffe94a9f1f1d8e
Implements: blueprint replace-types-module-with-new-style-classes
Adds support for package type classes in plugins.
As a result additional package types may be introduced
without changes to Murano source code
Implements-Blueprint: pluggable-package-types
Change-Id: I61c400b7be4c8836825c00ce7ca7d1b8501d0bd4
1.1 is the same as 1.0 but packages with format other than
MuranoPL/1.0 (or just 1.0) will be rejected by Murano Kilo.
As a result it will be possible to specify that application
require yaql 1.0 which was introduce in Liberty.
MuranoPL/1.2 differs in that it has yaql legacy mode turned
off and several functions are now cannot be called as
MuranoObject methods. It is recommended to start development
of new applications using 1.2 format
Change-Id: Ie68cc0c37f198c10eeac79ed349f9d9d420e038f
Implements-Blueprint: muranopl-format-version-upgrade
Before test_hot_package file used lists for testing which files were
generated. This could potentially lead to false negatives, as os.walk
returns file names in arbitrary order.
This commit replaces lists with sets to mitigate the problem.
Renames _build_hot_resources_dict as it does not return a dict.
Change-Id: I348d5c5bbf0db0d4d734d07a5449e6f3d171fcbf
Closes-Bug: #1494722
* Refactoring of "packages" Python package
* Now MuranoPL YAML classes can be parsed by different engines
depending on Format string in package manifest
With this change MuranoPL classes will no longer
be read and parsed (including YAQL expressions parsing)
upon package load but only on first class access and
with ability to peak different parsers depending on
format version specified in manifest. As a consequence
it is now possible to use different YAQL engines for
different package format versions. Also startup
time was greatly improved as unneeded classes are no more
parsed and logos and UI forms are not loaded.
Partially implements: blueprint murano-versioning
Change-Id: I23fc5da1a43b405d526438329dc04aed589dee13
With this change MuranoPackage becomes first-class DSL citizen.
Packages have version, runtime_version (that is specified
in Format attribute of the manifest file) and a list of classes.
Previously engine used to have package loader which had most
of "load" functionality and class loader that mostly acted as an
adapter from package loader to interface that DSL used to
get classes. Now class loader is gone and is replaced with
package loader at the DSL level. Package loader is responsible
for loading packages by either package or class name (as it was
before) plus semantic_version spec (for example ">=1.2,<2.0").
Package loader can now keep track of several versions of the same
package.
Also packages now have requirements with version specs.
All class names that are encountered in application code are
looked up within requirements only. As a consequence
packages that use other packages without referencing
them explicitly will become broken. An exception from this rule
is core library which is referenced automatically.
Partially implements: blueprint murano-versioning
Change-Id: I8789ba45b6210e71bf4977a766f82b66d2a2d270
* adds version and runtime_version properties
* package/runtime version in package format string doesn't
have to be fixed anymore. Now the version part is extracted,
validated and will be available via runtime_version property
* package initialization code was moved to the package
initializer removing the need for hot_v1/mpl_v1 scripts
* added requirement on semantic_version library
Partially implements: blueprint murano-versioning
Change-Id: I985846c2ad35be039e7025ff020ccbf22a870fb4
Enable the user to choose one environment from all environment files
located in the package under /Resources/HotEnvironments to be deployed
with the heat template as part of a Murano environment.
Partially implements: blueprint add-support-for-heat-environments-and-files
Change-Id: Id14fea94854221ba92b3eb09189c211c3d10d82f
Sends all files under /Resources/HotFiles to Heat when deploying
a Murano environment, while keeping the format Heat expects
Partially implements: blueprint add-support-for-heat-environments-and-files
Change-Id: Id7574642b8e09a7f659e4d9576a95f9f6efd8792
translate_parameter method of hot_package converter may raise
ValueError exception. It should be handled and raise PackageLoadError.
Change-Id: Icd66da398dc7e2fc91802b3420e3f708f801a111
Closes-bug: #1450541
* Unused module was removed
* Existing modules were updated
* Readme file was added
* Exceptions classes, used from unsupported exceptions from
oslo-incubator are moved to murano.common
* Orginize imports to the correct order in files, where the order were modified
Next patch will remove all locations of openstack.common.log usage
Change-Id: I4382215df1bcf81aea60e29039de548bcfe5a356
The HOT-to-MuranoPL converter did not support 'boolean' type as Heat
Template input parameter. This lead to a
400 'Unsupported parameter type boolean', during package uploads.
This patch adds support for boolean type, adds .bool() to boolean field
contracts and adds 'boolean' UI type during UI translation.
Change-Id: I960473916e70f9e17e5b5b1c228b7efd1deb98b5
Closes-Bug: #1450536
This commit makes UI generator respect parameter_groups
section of HOT template (if present). Application
name input was moved to a separate step before HOT
parameters.
Also respect label attribute of HOT parameters
Implements blueprint hot-parameters-ui-improvements
Change-Id: I43720c541d2b50660514a343f0dbe6f496ceedfd
The HOT-to-MuranoPL converter did not support some types for Heat
Template's input parameters.
Type 'comma_delimited_list' was not supported at all which was leading
to unhandled exceptions during package import operation.
Type 'json' was suported, but the generated UI definition did not
contain 'type' attribute for appropriate field, which was leading to
unhadled exceptions when the UI form was about to be displayed.
This patch fixes both these issues.
Closes-Bug: #1450536
Change-Id: I539514cc7cd884f98278cf5c990e8833e1b09259
Using python-muranoclient you can get an exception that doesn't
give needed information. Add this information to it.
Change-Id: Ic94a131e7cf0d25f10eb827df233a1c2f59ba9c3
Closes-bug: #1416021
* Provide exceptions details in the response body
* Decode package exception message to keep correct formatting
Partly-Closes-Bug: #1328662
Change-Id: I28000ec42db644d5db75f92d968955f7dd620091
This commit adds the ability to include
additional information about a package's supplier
to the manifest.yaml file.
The format of the optional fields are as follows:
Supplier:
Name: <name of vendor>
CompanyUrl:
Link: <url>
Text: <anchor text>
Summary: <plain text summary of vendor>
Description: <html summary for dashboard>
Logo: <png filename of vendor logo in same dir>
Targets: blueprint additional-author-information
Change-Id: I9bb67089ad1bc554524ee828b9bfda38dc8251f6
ljhuang