Commit Graph

433 Commits

Author SHA1 Message Date
Yasufumi Ogawa 65dfa754e2 Fix unittest for auto scaling
This update is to fix failures in unittests below.

* `custom_def` in Policy class was changed from str to dict in
  tosca-parser. This arg isn't interested in the test, so it's replaced
  with empty dict.

* Fix unsafe yaml loader with `yamf.safe_load`.

It also removes nouse pip packages which cause failures while
resovlving dependencies, and some tiny refactoring.

Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
Change-Id: I8ae60a4ae9f139073f05ec0f99fa82c8bb24ee50
2022-03-16 01:01:23 +09:00
Nobuhiro Uematsu c6e04928fb ETSI-NFV SOL 001 translation: AffinityRule
The current heat-translator supports several conversions of
resources defined in ETSI NFV-SOL 001[1].
This patch supports the conversion of new resource types.
Specifically, the following types related to
'PlacementConstraints'.
- tosca.groups.nfv.PlacementGroup
- tosca.policies.nfv.AffinityRule
- tosca.policies.nfv.AntiAffinityRule

[1] https://www.etsi.org/deliver/etsi_gs/NFV-SOL/001_099/001/02.06.01_60/gs_NFV-SOL001v020601p.pdf

Co-Authored-By: Hiroo Kitamura hiroo.kitamura@ntt-at.co.jp
Implements: blueprint support-placement-constraints
Change-Id: Ic22a6183ee0d7c8357063bdafbb1a11c1a6efa3c
2021-03-10 11:57:14 +09:00
Hiroo Kitamura 2c94431c0d Fix processing related to required properties
In the current UT testcase and yaml, there are several cases where
the required property of policy is insufficient or the definition
part is incorrect.
In addition, there is a bug in the code that has the translation
logic under 'translator/hot/tosca/'.
This patch fixes these translation logic codes, testcase and yaml.

Among the fixed yaml files, the following has a line feed code of CRLF,
so it has been modified to LF.

* translator/tests/data/autoscaling/tosca_cluster_autoscaling.yaml
* translator/tests/data/custom_types/senlin_cluster_policies.yaml
* translator/tests/data/nfv/test_tosca_nfv_autoscaling_with_params.yaml

In addition, NOTE description is added for
'translator/hot/tosca/tosca_policies_monitoring.py',
which is one of the translation logic codes.

Closes-Bug: #1918360
Change-Id: Ib9dec61007db5d354eaca86000ab3bcf6744a838
2021-03-10 11:52:49 +09:00
Manpreet Kaur e41b382afd Dropping explicit unicode literal
In python 3, all strings are considered as unicode string.

This patch drops the explicit unicode literal (u'...')
or (u"..") appearances from the unicode strings.

Change-Id: Iaa040f10028935782d8315efc81d344e72b2a152
2021-01-30 13:57:04 +05:30
Manpreet Kaur 6687b35b46 Move heat-translator to new hacking 4.0.0
Hacking 4.0.0 has introduce new optimal assert checks.
Please refer [1] for more details.

Additionally Hacking has enabled some new checks by default,
and the next update to the new release will introduce failures
in heat-translator pep8 jobs.
The main impact is from I122d250cab90964c346e9d53046a97c25054bc00.

This patch adopts new optimal assert check.
  [H211]:  Change assertTrue(isinstance(A, B)) to
           assertIsInstance(A, B).

On bumping up the hacking version, below mention pep8 jobs failed.
This patch address the issues now, when the new hacking release starts
being used it will not cause a disruption.

pep8 run-test: commands[0] | flake8
./translator/tests/test_utils.py:51:13: H211: Use assert{Is,IsNot}instance
         self.assertTrue(

[1] https://docs.openstack.org/releasenotes/hacking/unreleased.html#relnotes-4-0-0

Change-Id: Iae79b8c5c6873d6bf5f6ef1b427dc8c60bade03f
2021-01-26 14:00:21 +05:30
Yasufumi Ogawa 4809045ed5 Remove six support
This update is to remove six support for dropping python2 from
heat-translator. Only the example of dockerfile is remained in which
it's expected libs are installed under `/usr/local/lib/python2.7`, so
it should also be considered to be updated later.

Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
Change-Id: Iafbd1588674c95d6b6ec6b5429f2d34fb23081d0
2020-12-11 17:07:14 +00:00
Sean McGinnis 6229b16442
Use unittest.mock instead of third party mock
Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib.

Change-Id: I7a7b0090139a72a06ec98d22e27e015307c3aec3
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-04-18 16:09:09 -05:00
Andreas Jaeger 98e848606f Update hacking for Python3
The repo is Python 3 now, so update hacking to version 3.0 which
supports Python 3.

Fix problems found.

Remove hacking and friends from lower-constraints, they are not needed
for installation.

Change-Id: I36cc496d9374169b0eafdf8d66747b07f2b2cabb
2020-04-07 17:35:28 +02:00
Zuul 2a57298dce Merge "Add support for list data type" 2020-04-01 00:54:25 +00:00
Niraj 7be4cd79e8 Add support for list data type
In ETSI tosca definition, node type “tosca.nodes.nfv.VNF” has
property “vnfm_info” that is defined as of type list.

Definition given in etsi_nfv_sol001_vnfd_types.yaml:
tosca.nodes.nfv.VNF:
    properties:
      vnfm_info:
        type: list
        required: true

Example:
tosca.nodes.nfv.VNF:
    properties:
      vnfm_info: [vnfm1, vnfm2]

Added translation support for list data type.

Change-Id: I02a4ba46da588ef7c1ac36a15d7cd537187e3ecc
2020-03-31 03:13:24 +00:00
Zuul 6ddb2a7114 Merge "Replace assertItemsEqual with assertCountEqual" 2020-03-30 04:01:33 +00:00
JO Hiroyuki 0d0b1a1821 Replace assertItemsEqual with assertCountEqual
assertItemsEqual was removed from Python's unittest.TestCase in
Python 3.3 [1][2]. We have been able to use them since then, because
testtools required unittest2, which still included it. With testtools
removing Python 2.7 support [3][4], we will lose support for
assertItemsEqual, so we should switch to use assertCountEqual.

[1] - https://bugs.python.org/issue17866
[2] - https://hg.python.org/cpython/rev/d9921cb6e3cd
[3] - https://github.com/testing-cabal/testtools/issues/286
[4] - https://github.com/testing-cabal/testtools/pull/277

Change-Id: I53f758e7e8870b43b675f4c63d986f07d199c259
2020-03-30 00:02:49 +09:00
Hiroo Kitamura 292248be2c ETSI-NFV SOL 001 translation: ScalingPolicy
Currently heat-translator supports translation of TOSCA Simple Profile
for YAML[1] and TOSCA Simple Profile for NFV[2] only.
This commit enables to translation of the follwoing type defined in
ETSI NFV-SOL 001[3].
- tosca.policies.nfv.InstantiationLevels
- tosca.policies.nfv.ScalingAspects
- tosca.policies.nfv.VduInitialDelta
- tosca.policies.nfv.VduInstantiationLevels
- tosca.policies.nfv.VduScalingAspectDeltas
- tosca.policies.nfv.VirtualLinkInstantiationLevels

[1] http://docs.oasis-open.org/tosca/tosca-nfv/v1.0/tosca-nfv-v1.0.html
[2] http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-YAML/v1.0/TOSCA-Simple-Profile-YAML-v1.0.html
[3] https://www.etsi.org/deliver/etsi_gs/NFV-SOL/001_099/001/02.06.01_60/gs_NFV-SOL001v020601p.pdf

Change-Id: Ifdb287c84dfb19d25b519a2e23f1b82ccc3f4088
Story: 2006372
Task: 37622
2020-01-24 04:52:54 +00:00
Hiroo Kitamura 59c61d71eb ETSI-NFV SOL 001 translation: BlockStorage
Currently heat-translator supports translation of TOSCA Simple Profile
for YAML[1] and TOSCA Simple Profile for NFV[2] only.
This commit enables to translation of the follwoing type defined in
ETSI NFV-SOL 001[3].
- tosca.nodes.nfv.Vdu.VirtualBlockStorage

[1] http://docs.oasis-open.org/tosca/tosca-nfv/v1.0/tosca-nfv-v1.0.html
[2] http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-YAML/v1.0/TOSCA-Simple-Profile-YAML-v1.0.html
[3] https://www.etsi.org/deliver/etsi_gs/NFV-SOL/001_099/001/02.06.01_60/gs_NFV-SOL001v020601p.pdf

Change-Id: I026a06de41ed390967e9ef137fa633ecd4fe747f
Story: 2006372
Task: 37621
2020-01-24 04:50:43 +00:00
Hiroo Kitamura 6f94614c71 ETSI-NFV SOL 001 translation: CP,VL
Currently heat-translator supports translation of TOSCA Simple Profile
for YAML[1] and TOSCA Simple Profile for NFV[2] only.
This commit enables to translation of the follwoing type defined in
ETSI NFV-SOL 001[3].
- tosca.nodes.nfv.VduCp
- tosca.nodes.nfv.VnfVirtualLink

[1] http://docs.oasis-open.org/tosca/tosca-nfv/v1.0/tosca-nfv-v1.0.html
[2] http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-YAML/v1.0/TOSCA-Simple-Profile-YAML-v1.0.html
[3] https://www.etsi.org/deliver/etsi_gs/NFV-SOL/001_099/001/02.06.01_60/gs_NFV-SOL001v020601p.pdf

Change-Id: I28b4ff67b74e7d5ad4264acb93cd8c292b91a0d4
Story: 2006372
Task: 37620
2020-01-22 03:35:50 +00:00
Hiroyuki JO 4833eb5e92 ETSI-NFV SOL 001 translation: VNF, Compute
Currently heat-translator supports translation of TOSCA Simple Profile
for YAML[1] and TOSCA Simple Profile for NFV[2] only.
This commit enables to translation of the follwoing type defined in
ETSI NFV-SOL 001[3].
- tosca.nodes.nfv.VNF
- tosca.nodes.nfv.Vdu.Compute

[1] http://docs.oasis-open.org/tosca/tosca-nfv/v1.0/tosca-nfv-v1.0.html
[2] http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-YAML/v1.0/TOSCA-Simple-Profile-YAML-v1.0.html
[3] https://www.etsi.org/deliver/etsi_gs/NFV-SOL/001_099/001/02.06.01_60/gs_NFV-SOL001v020601p.pdf

Story: 2006372
Task: 36161

Change-Id: I69263dce88bab5413a2d6f74f8d0c920dd454bf3
2020-01-22 03:35:40 +00:00
Hiroyuki JO 9d5056a9d7 nit: fix loading tosca classes including ., p, y
When loading tosca clsses, '.', 'p' and 'y' are splited so files
including these charactors in its basename are failed to load. This
patch fixes it by removing only file extension.

Change-Id: I6ad8a32982aec67f8d8aecaafcbaec803d4c7a02
2019-11-27 17:52:38 +09:00
Hiroyuki JO d21c0ffffa Fix ToscaPolicies overwrites scheduler_hints
ToscaPlicies.handle_properties overwrites if scheduler_hints exists.
This patch changes it to use 'update' method to add new key and value.

Change-Id: I6062980eaa3c392a48c878518d2d61425eceb86f
Story: #2004963
Task: #29403
2019-07-10 11:46:20 +09:00
shubham potale 68e06e2435 Create VNF with parameter file using scaling policy
If user provides parameter file while creating VNF using scaling
policy, earlier it was failing with error 'The Parameter was not
provided" This patch allows creation of VNF using scaling policy
with parameter file.

APIImpact
Return 200 instead of 400 error when VNF is created using scaling
policy with parameter file

Change-Id: If102519127a3ef63449ab59f849e8cfaaa3ae62b
Closes-Bug: #1799683
2019-02-26 06:50:06 +00:00
Zuul 66ce5cf926 Merge "Update hacking version" 2019-02-11 15:14:16 +00:00
nirajsingh 30f4d2ff83 Support reservation policy in heat-translator
Implementation of reservation policy in heat-translator

Tacker Blueprint: https://blueprints.launchpad.net/tacker/+spec/reservation-vnfm
Tacker Spec: https://review.openstack.org/#/c/561840/
Depends-On: Ic5d790df938b40d75bc50252e1e688e9c09eb568
Change-Id: I2b989a49ac3447995a82ddb7193bf478bb847b73
2019-01-09 06:28:27 +00:00
zhulingjie de9d98cfc6 Update hacking version
Use latest release 1.1.0 and compatible changes w.r.t pep8

Change-Id: Ib3a7568750ca5bf14b445e3de3aa3b020c6dbb2a
2018-12-10 05:25:03 -05:00
Alvaro Lopez Garcia ef32ac6991 Use oslo.log for osc plugin
If we do not use oslo.log for logging in the openstackclient plugin, all
the errors and messages that are generated by other parts of the client
(cliff, openstackclient) may be lost.

Change-Id: Ie9aec900e477f3f319ee34873018c15e10dafb4a
2018-10-26 11:43:16 +02:00
dineshbhor b8c2c86333 Trivial: Fix unnecessary redundant code
While translating node_template, heat-translator first checks whether
the policy is derived from the policies 'tosca.policies.Scaling' or
'tosca.policies.Monitoring', If yes, then it adds them into
TOSCA_TO_HOT_TYPE map. In the subsequent code, it again checks for
those policies and if they are not present in TOSCA_TO_HOT_TYPE map
then it raises UnsupportedTypeError exception.

This patch removes this unnecessary redundant code of checking
policies again and ensures that the affected code is getting
covered through test case.

TrivialFix
Change-Id: Ibe435e23e7ac11834e662745d327e6f99280f9f5
2018-07-21 04:31:01 +00:00
dineshbhor fd75f0c5de Fix 'Placement' policies not translated
Some policies derived from standard policies are translated
by base policy translator. For example, a policy which has
`tosca.policies.tacker.Alarming` type derived from
`tosca.policies.Monitoring` type is translated by a translator to
`tosca.policies.Monitoring` type.

A policy which has a type derived from tosca.policies.Placement
doesn't seems to be translated by a translator to
`tosca.policies.Placement`.

This patch fixes this issue by adding the `tosca.policies.Placement`
specific check while translating nodetemplates.

This policy translation support is required to implement below
tacker blueprint:
https://blueprints.launchpad.net/tacker/+spec/vdu-affinity-policy

Closes-Bug: #1755433
Change-Id: Ib265ac3add18027c91a601140a4f03a661603615
2018-05-30 01:52:36 -07:00
hoangphuocbk 9e3a6157f0 Move usage from deprecated Ceilometer API to Gnocchi API
This removes usage of deprecated Ceilometer API

1. Changing trigger type from "OS::Ceilometer::Alarm"
to "OS::Aodh::GnocchiAggregationByResourcesAlarm"
2. Add "resource_type" fixed with value "instance"
3. Change some paramters (meter_name-> metric,
statistic ->aggregation_method, period-> granularity)
4. Change value from "average" to "mean" in method to
compare to the threshold

Change-Id: I66f8e4c76a918494df26b4af277453e24421b917
2018-05-16 01:26:51 +09:00
wangqi 5fc4173b6d fix a spelling error
Change-Id: Ie16fd93699d979076c9031a54c6e126946cc7c82
2018-03-13 16:40:16 +00:00
Jenkins 6ebb3de3d2 Merge "Use id to identify images from Glance" 2017-10-09 17:03:42 +00:00
Jenkins d289c1b8b9 Merge "Use image attribute if specified in os properties" 2017-10-09 17:01:09 +00:00
Jenkins fe14b82600 Merge "Test against None" 2017-10-09 16:59:15 +00:00
Mathieu Velten e40b4ccfae Use id to identify images from Glance
Name can be empty or match several images

Change-Id: I5c5358d0b5fb08d91339acd7c7720db127386f42
2017-09-20 11:41:43 +02:00
Mathieu Velten 420a41d60d Use image attribute if specified in os properties
Change-Id: I1240e19a2724ed5c1a0f1ab6859539e876ae9dae
2017-09-20 11:41:05 +02:00
Mathieu Velten 3d027de030 Test against None
The current code doesn't handle correctly booleans for example

Change-Id: I06393af31305e1eea1ea8e9bd05995363101af88
2017-09-19 15:11:30 +02:00
Mathieu Velten 54cf66ae6a Ignore error if the log config file is not present
Change-Id: I3db7bb29fa04c242ff3c4437aed8ddaec45611a8
2017-09-19 14:38:16 +02:00
Jenkins 68ea2742cc Merge "When removing an empty res also remove refs to it in depends_on" 2017-09-19 06:41:17 +00:00
Mathieu Velten eb6354ab6b When removing an empty res also remove refs to it in depends_on
Change-Id: I4cad4f29d0f9153eb966370089663b419a53dfaf
2017-08-21 11:47:17 +02:00
Mathieu Velten b154640433 Test if property is a get_input before treating it as such
Change-Id: I353b738e728073fd197faca33b5702b95d0ab3e4
2017-08-21 11:20:19 +02:00
Mathieu Velten 59a65caaf5 Revert "Fix: nested resources cannot be given out"
This reverts commit 3c53dc729f.

Change-Id: I0230daa7b860f7f35ef5cd7be6e1d1a0d4547912
2017-08-11 12:52:54 +00:00
doantungbk 3c53dc729f Fix: nested resources cannot be given out
In Tacker, we now are fixing Tacker scaling to fully leverage heat-translator.
To make a best fit, Tacker need to have nested resources from heat-translator.
Heat-translator already provided this by using translate_to_yaml_dict().
Unfortunately, this method always return to the main heat yaml file not to
nested yaml file.

Change-Id: I7144d9926141dc4fc70f3a3c73c7fd41e2ff288c
2017-06-06 09:38:43 -07:00
Bob.Haddleton 6904aab001 Support os_version, os_type, and os_distro image properties
Modified default images to use os_type, os_version and os_distro
properties.  Added logic to look for distribution/os_distro,
type/os_type and version/os_version image properties when
matching image properties.

Cleaned up minor python warnings in tosca_compute.py

Removed DOS-style newline characters from
tosca_cluster_policies_scaling.py

Change-Id: I72043cf4a4358cfdbc8f98238276d85dc2f5bcc0
Closes-Bug: 1689673
2017-05-23 06:59:58 -05:00
doantungbk 3ce7de1d64 Add 'cooldown' attribute to scaling policy
Currently, Tacker cannot leverage fully Heat Translator because
the lack of attribute.

Change-Id: I14fb9d9b6a0addf4529e6d21a2af7b3c7947926f
2017-05-17 01:59:25 -07:00
Jenkins 59eda86b88 Merge "Implement monitoring policy in HT" 2017-04-27 13:54:52 +00:00
doantungbk cb4883a551 Implement monitoring policy in HT
Currently, monitoring policy have not been added to Heat
Translator yet. The current implementation of scaling policy
in Heat Translator showed the couple between scaling and
monitoring policies. But actually monitoring policy shows
that it could have other very good use cases so that it
can stand alone.
This patch will implement monitoring policy in HT.

Implements bp: #monitoring-policy

Change-Id: Ibc412c409846919a8edb581c7cf56e6dcd0cc01f
Depends-On: Idd5268b7ff0a19b57ea927f23c76db24220c462d
2017-04-13 07:56:32 -07:00
Jenkins 22d6209167 Merge "Assign floating IP to the vdu" 2017-04-12 03:10:54 +00:00
futangw d7d6aee6fc Fix bug that only 1 volume will be attached to VDU automatically
With Tacker when more than 1 volume to be created and attached,
but after heat translation, only 1 volume will be attached automatically.
The code change is to allow all volumes to be created and attached as
what specified in TOSCA template.

Change-Id: Iba9e089347d6dfd31e7d7c40b46706b75145c696
Closes-Bug: #1675715
2017-03-28 01:50:23 -04:00
Saju Madhavan 7ccbc878af Assign floating IP to the vdu
Assign floating IP to the vdu using TOSCA template

Change-Id: Iee4ab0c4f6e47f64ac43e8a57d6775ea7939e5e1
Partial-Bug: 1537636
2017-03-17 15:23:16 +05:30
Sahdev Zala 640bbdb7e4 Add sample dockerfile to build translator container
The file content can be copied as is to Dockerfile and use to build a heat
translator docker container.

Change-Id: I32fa7bd9def0842a053c4c61e7f6652db4c16967
Co-Authored-By: Eduardo Patrocinio <epatro@gmail.com>
2017-03-09 11:51:02 -08:00
Jenkins 8547078499 Merge "Allow boot vdu from volume" 2017-02-10 00:52:27 +00:00
Luong Anh Tuan 3cf5688141 Replace yaml.load() with yaml.safe_load()
Avoid dangerous file parsing and object serialization libraries.
yaml.load is the obvious function to use but it is dangerous[1]
Because yaml.load return Python object may be dangerous if you
receive a YAML document from an untrusted source such as the
Internet. The function yaml.safe_load limits this ability to
simple Python objects like integers or lists.

In addition, Bandit flags yaml.load() as security risk so replace
all occurrences with yaml.safe_load(). Thus I replace yaml.load()
with yaml.safe_load()

[1]https://security.openstack.org/guidelines/dg_avoid-dangerous-input-parsing-libraries.html

Change-Id: Ifaecff145e91f72911ae05ea274a4977c56212c7
Closes-Bug: #1634265
2017-01-23 05:53:27 +00:00
gong yong sheng 221dec4958 Allow boot vdu from volume
In this case, there will be no image property for server
resource.

Change-Id: I3e3d62f4ad60fdb4ceddd46b70386d4623367c51
Partial-bug: #1649459
2017-01-19 09:26:43 +08:00