Various components defines different options for the client sessions.
Standardize them with the help of keystonauth1 lib.
Change-Id: I2f791caaf230a58b8426d1c1d6e1eb4316a85a28
In 'io.murano.system.NetworkExplorer', there is a
'get_external_network_id_for_network' method calling
'getExternalNetworkIdForRouter' to get external network id for specific
router interface, which should be 'get_external_network_id_for_router'
Change-Id: Id2046106e274b8f6682703659ab8e27233d49209
Closes-Bug: #1638923
We are replacing all usages of the 'retrying' package with
'tenacity' as the author of retrying is not actively maintaining
the project. Tenacity is a fork of retrying, but has improved the
interface and extensibility (see [1] for more details). Our end
goal here is removing the retrying package from our requirements.
Tenacity provides the same functionality as retrying, but has the
following major differences to account for:
- tenacity uses seconds rather than ms as retrying did.
- tenacity has different kwargs for the decorator and
Retrying class itself.
- tenacity has a different approach for retrying args by
using classes for its stop/wait/retry kwargs.
- By default tenacity raises a RetryError if a retried callable
times out; retrying raises the last exception from the callable.
Tenacity provides backwards compatibility here by offering
the 'reraise' kwarg.
- tenacity defines 'time.sleep' as a default value for a kwarg.
That said consumers who need to mock patch time.sleep
need to account for this via mocking of time.sleep before
tenacity is imported.
This patch updates all usages of retrying with tenacity.
Unit tests will be added where applicable.
Note: This change is not newton critical so projects are welcome
to hold off on committing until post-newton. Ideally this change
will merge by the first part of Ocata so dependant functionality
can land and have time to solidify for Ocata.
[1] https://github.com/jd/tenacity
Change-Id: I18a0075e9a7c376b6881555ce67267a8944caafa
Because of several bugs/code design issues in the
DSL a cyclic references between objects were created.
Thus the object model objects were not automatically
deleted upon deployment finish even if there were no
cross-links between the objects in object model.
This commit both breaks the links and increases engine
performance due to
1) For LHS expressions there is no more need to parse
yaql function definitions upon each variable modification
because now the base LHS context is fixed and static
2) In most cases the objects now are reclaimed immediately
after deployment finish thus python GC doesn't have to
traverse large graphs
Targets-blueprint: dependency-driven-resource-deallocation
Change-Id: I4b1e0038bf7c08ced357fa20c4b1e3d612c93ae9
Clients for various OpenStack services
were updated so that it would be
possible to specify region name explicitly.
If it is not provided then for clients that are
used internally by the core library it is taken
from the current region. Otherwise the default
environment region is used.
Change-Id: I289c7f8dc93a30ba1d5aa06d9da29a6af1308b9a
Now all OpenStack resource classes inherit CloudResource
that provides getRegion method and regionName property.
This allows to assign resources to different regions.
getRegion() returns CloudRegion instance that resource or it
parent belong to. CloudRegion has the similar interface to
Environment class and is the correct way to get HeatStack
instance associated with the regoin, default network
configuration, security group manager and agent listener
instances. Environment acts as the default region so backward
compatibility is not broken. However new applications
should not use environment to set security group rules but
rather a region(s) of their instance(s) in order to work correctly
when their instances were configured to use region other then
the default.
Change-Id: I4dbf40c65042e9a354f3bfadfcd63a63e6e3e418
Existing implementation of Neutron-based networking assumed that the
neutron's security groups are used to manage VM accessibility.
However there may exist environments with disabled security-group
extension in Neutron and thus relying on something else to restrict
the traffic. Murano could not operate in such environments since it
always was attempting to create resources of type
OS::Neutron::SecurityGroup and attach VMs' ports to this resource.
This is addressed by introducing a new subclass of
SecurityGroupManager - DummySecurityGroupManager, which actually does
nothing but silently ignores the calls to create security rules. This
new security manager is instantiated instead of
NeutronSecurityGroupManager for Neutron-based networks in cases if the
'security-group' extension is not present in Neutron's configuration.
If it is instantiated a warning message is reported to the end-user to
notify them that security requirements of the application were
ignored.
Change-Id: Ia3bc6c17f9ca0a4b8bf8c272481760a8c81b27b7
Closes-bug: #1593253
The Environment class got two additional properties:
region - name of the region where environment should be deployed. If
null then home region is used.
regionConfigs - config property (taken from class config) that specifies
region parameters. It is a dictionary of a form regionName -> regionSettings.
Currently the only setting available is a agentRabbitMq that is itself
a dictionary containing the same options as [rabbitmq] section of murano.conf
Thus the configuration file io.murano.Environment.yaml may look like:
regionConfigs:
RegionOne:
agentRabbitMq:
host: rabbitMqHostName
port: 5672
login: murano
password: murano
virtual_host: /
ssl: false
ca_certs: null
Implements Blueprint: assign-environment-to-region
Change-Id: I3c84c96f8eeae738f83ed7b3691be38100b30e1d
With this commit most of the Python 3 compatibility
issues in murano-engine are resolved.
If run on yaql with https://review.openstack.org/#/c/286110/
fix all of the unit tests except for one success.
The only failing test is base64 encoding/decoding test
which require rethink of resource management to get away
from string types for binary content
Change-Id: Iee87d27fe4f04118202de07f376d41fbf2c90f54
* Single universal ClientManager class was dropped in favor of
of individual in-context methods to create OS clients without
ClientManager restrictions.
* Environment class was renamed to ExecutionSession to avoid
common confusion with io.murano.Environment
* execution_session_local module was introduced to simplify
keep of per-execution session (per-deployment) data. This
is similar to thread-locals with the difference that there can
be many threads in single session.
* All OS-clients related code was migrated to keystone client
sessions and API v3 (except for GLARE and Mistral that doesn't
support sessions). This increases performance and solves
authentication problems that could be caused by token expiration
even with trusts enabled.
* [DEFAULT]/home_region setting was introduced instead of
[murano]/region_for_services to configure what region
should be used by the clients by default (where Murano API
resides). All client factories respect this setting.
Change-Id: If02c7e5d7d39574d0621e0e8dc27d1f501a31984
It is possible that there are several public networks exist.
ExistingNeutronNetwork used to pick first one of them.
Now when internal network name or ID supplied it tries
to find such public network that the internal network has
a common router with.
Change-Id: I252bb5545891903355cf7f8480a2cd76febcfebe
Closes-Bug: #1524322
Fix using of malformed argument in retry decorator. It should use
check which will return True\False instead of exception types.
Related-Bug: #1493883
Change-Id: I10bc8db1db1fd0791012de85def7ed92e9f05076
Previously when murano deployed a bunch of environments from scratch
it was possible to have an error during router creation. Router was
created by env_1 and before it was fully configure it was accessed by
env_2 and then the error occurred.
Change-Id: I6dcd281a24825bb9c2ae09f45009faa6eca5a451
Closes-Bug: #1493883
* Code migrated to yaql 1.0.0
* New MuranoPL object initialization
* Lots of refactoring
See referenced specs for more information
Implements: blueprint migrate-to-yaql-vnext
Implements: blueprint object-construction
Depends-on: I7f314634ab5f08a521e51082d5c84dffca4b0b5c
Closes-Bug: #1454264
Change-Id: I740a4f83c76d8b56a1da585a739d770ef823a524
To follow latest community practices we need to use oslo_log in
murano instead of handcrafted libs.
Unsupported log level 'audit' was changed to 'info'.
Change-Id: I40c0f3790e34bf80d0b63554e86b3cbc019eefca
In the several places the CONF variable is used from the murano.config.
This patch declares the local CONF variable and uses the oslo_config library.
Change-Id: I1f9a42f7f2cf7fef2cf0d63b185428dfddd9e78e
The Oslo libraries have moved all of their code out of the 'oslo'
namespace package into per-library packages. The namespace package was
retained during kilo for backwards compatibility, but will be removed by
the liberty-2 milestone. This change removes the use of the namespace
package, replacing it with the new package names.
The patches in the libraries will be put on hold until application
patches have landed, or L2, whichever comes first. At that point, new
versions of the libraries without namespace packages will be released as
a major version update.
Please merge this patch, or an equivalent, before L2 to avoid problems
with those library releases.
Blueprint: remove-namespace-packages
https://blueprints.launchpad.net/oslo-incubator/+spec/remove-namespace-packages
Change-Id: I975592f3694be42d52685ebf606f6d3012caf1a8
Also refactores networking code to move networking code from
Instance so that different types of network can be represented by
single simple interface
Implements: blueprint join-existing-neutron-networks
Change-Id: I90afcea6b1c12d7f6f564d8524f5c0d30e1b4686
Instead of using user's auth token (which can expire) for interactions with
other services engine creates Keystone trust that impersonate user and
create new tokens on demand.
Heat stack is created on deployment start using token rather than trust so that
Heat could establish trust of its own (trusts cannot be chained).
New behavior is disabled by default and can be enabled using [engine]/use_trusts = True in murano.conf.
With trusts enabled engine will not work with Heat prior to Juno.
For Heat stacks with deferred actions or long deployment time to work it is also required to turn on trusts in Heat itself.
This can be done via [DEFAULT]/deferred_auth_method=trusts in heat.conf and ensuring that current user
has heat_stack_owner role (or any other that is in [DEFAULT]/trusts_delegated_roles=trusts in heat.conf)
Change-Id: Ic9f3f956ddb6ff2a300a08056ee841cf3c0db870
Implements: blueprint auth-for-long-running-requests
If it doesn't find a router by the name specified in router_name
in murano.conf, create a router with that name. Uses
external_network_id or external_network_name as the
external_gateway_info ID. Requires:
https://review.openstack.org/#/c/119800
Change-Id: If8b966a7d43eb2af485113de2a0708e554605725
Murano should not use first available router it may create
different networking issues cause router may not have access to
external network.
Closes-bug: #1366124
Change-Id: I223a4862c3b20cfb20281be6bd9376be3644ad92
* H202 assertRaises Exception too broad
* H402 one line docstring needs punctuation
* H404 multi line docstring should start without a leading new line
Change-Id: I2f662b8b97d14daa501620c8237bf93bd2251243
zhurong