Legacy tasks adapter checks if there are pre- and post- deployment
stages for role-based emulation using in predicate. Unfortunately,
it is broken due to legacy orchestrator graph constructor
implementation and retunrs false positive results.
Instead we should use safe get that works properly.
Change-Id: I22d216af9933250c473c1ccfaa931d0b41cfb1f2
Closes-bug: #1635051
This commit switching tasks resolution approach to the tags based one.
Tag - minimal unit what's necessary only for task resolution and can be
mapped to the node through the role interface only. Each role provides set
of tags in its 'tags' field and may be modified via role API. Tag may be
created separately via tag API, but, this tag can not be used unless it's
stuck to the role.
Change-Id: Icd78fd124997c8aafb07964eeb8e0f7dbb1b1cd2
Implements: blueprint role-decomposition
Following OpenStack Style Guidelines[1]:
[H203] Use assertIs(Not)None to check for None (off by default)
Unit test assertions tend to give better messages for more specific
assertions. As a result, assertIsNone(...) is preferred over
assertEqual(None, ...) and assertIs(None, ...), and assertIsNotNone(...)
is preferred over assertNotEqual(None, ...) and assertIsNot(None, ...).
[1] http://docs.openstack.org/developer/hacking/#unit-tests-and-assertraises
Change-Id: Ie72bbf01a4c516ab4fd83757879ad1a04df600b7
A 'tags' attribute has been added to each role in 'roles_metadata'.
Initially all non-controller roles will only have a tag of their own
role name. This will allow existing tasks which do not have tags
associated with them to work correctly. In the abscence of tags a
task's roles will be used to determine which nodes it will run on.
Implements: blueprint role-decomposition
Change-Id: I390580146048b6e00ec5c42d0adf995a4cff9167
DeployChanges will use the sequence 'deploy-changes' if it is available.
DeployNodes and ProvisionNodes will be use apropriate graphs if it is available.
Change-Id: Icb66826ce96c41f77d85d97990b4cc600aa5fe04
Closes-Bug: 1620620
In order to enable building a history of deployment transactions
for a custer it is necessary to keep track of start and end time
for them. This patch adds timestamp_start and timestamp_end attributes
to transactions, makes changes to the API serializers and introduces
appropriate database fields to store that information
I've added times only for 'deployment' task, which is the only
one, needed by new UI feature. At this point there is no single
point, where task switches to error or ready state and I'd have
to set in in many places.
Whem everything-is-a-graph approach will be fully implemented
it would be very easy. So let's not add unnecessary code now.
Co-authored by: Roman Prykhodchenko <me@romcheg.me>
Co-authored by: Dmitry Guryanov <dguryanov@mirantis.com>
DocImpact
Closes-bug: #1593753
Change-Id: Ib206d75a8d2215f8fd6b3f89e209c6e86ed20d0f
The size of deployment_info grows as n^2 depending on
nodes number. That's because common_attrs, which is
merged into each node's contains info about all nodes.
For example for 600 nodes we store about 1Gb of data in
the database. So as first step let's store common_attrs
separately in deployment_info structure inside python
code and in the database.
Also removed old test for migrations, which are not related
to actual database state.
Change-Id: I431062b3f9c8dedd407570729166072b780dc59a
Partial-Bug: #1596987
Simultaneous run of 2 or tasks may cause
side-effects and the simplest way is prevent this action.
Also fixed all places where rpc.cast called without commit
Change-Id: I029768900d345540c3b501f1fa3649b063d3a55d
Partial-Bug: 1615557
Deleting empty environment leads to an inaccurate notification
message "Environment 'xxx' and all its nodes are deleted".
Now notification message looks like as follows:
"Environment 'xxx' is deleted"
Change-Id: I827686e15120e8548aba0e9fb430d38f08285650
Closes-Bug: 1589864
This change introduces a new callback on_nodegroup_delete
which is called when a nodegroup is deleted.
It also adds a decorator that can be used to generate
before and after callbacks for any method.
Change-Id: Ia1c4ef3956175af6c223af854c9543cd781e8dbf
Blueprint: network-manager-extension
Deployment and provisioning preparations could last a long when Fuel is
working with a hundreds of nodes. But operator should be able to
observe cluster state and operate other clusters.
Change-Id: I73802c91f93a46855b006cb05a8b5722109e9e6a
Partial-Bug: #1569859
Deployment tasks snapshot is now saving during DeploymentTask message creation
in Transaction.tasks_snapshot field.
Methods:
* attach_tasks_snapshot
* get_tasks_snapshot
are added to Transaction model.
Handler /transactions/:transaction_id/deployment_history/
now showing tasks parameters in output and supporting
request parameter:
/transactions/:transaction_id/deployment_history/?tasks_names=task1,task2
Change-Id: I59446ef456f7d3bd249c686b7a8e1a93a364daf2
Partial-Bug: #1563317
DocImpact
Now we get last success transaction not only for every task but for
every node in the cluster. The previous version of state collecting
function may lead to false-positive choice of previous deployment
transaction and issues with smart YAQL-based redeployment.
Change-Id: If9e22924acb465487a2fa97534fe50f03a1590a1
Closes-Bug: #1581015
Closes-Bug: #1582269
This fix does proper generation of dependencies for pre-task-based
deployment tasks. It checks if a deployment task is in pre-/post-
or main deployment graph and assigns dependencies properly without
creating dependencies loops.
Example:
taskA: requires: post-deployment-start, groups:B
w/o this fix it would create the following loop:
taskA->B_end->post-deployment-start->taskA
Change-Id: Iaa5d87946c1a9957d99a329009d578c210b6f888
Closes-bug: #1572276
We shouldn't skip tasks, if node is in provisioned state,
so clear state in this case.
Partial-Bug: #1573105
Change-Id: If8536cbd2559e745ad69dffc0ca8847015e1180e
When we remove the controller from cluster we need a redeployment of all
left controllers. Before this patch we always use tasks.DeploymentTask,
now if lcm supported we use tasks.ClusterTransaction
Change-Id: Ia5a226221eae4f1d2ade3633bc32d22cdf24a3f2
Closes-Bug: #1578571
Now serialized deployment info for environments >= 9.0 does not contain
'nodes' field; it is removed from the output of the serializer.
Unit and integration tests are updated. Deployment tasks fixture
changed: 'nodes' references in yaql expressions changed to
'network_metadata.nodes'
Change-Id: Ibdb2cf85732c0293314298e6a342b071ece18cc3
Closes-Bug: #1531128
Now we get deployment state from DeploymentHistory model. For every task
we get last success transaction and its state.
Change-Id: I2288bc2bc34023c2ca705f1d3cc6ff48347bf549
Closes-bug: #1572226
Since self.env.create always return db object now,
we can use this returned value instead of
self.env.clusters list.
It's a refactoring, so no bug or blueprint.
Change-Id: If7c84cb7124bcf08ef5ff110542012564190fae1
The new option 'propagate_task_deploy' was added to cluster.
this option allows to use legacy task adapatation algorithm
to make tasks from granular deployment LCM ready on the flight.
Also the same aproach is used for adaptation legacy plugin tasks.
Change-Id: Ib212bd906acc0e6915e3c14e4741b306bdedaa98
Closes-Bug: 1572276
We should select all nodes those were provisioned
for deploy in LCM. The method nodes_to_deploy has overridden for
clusters that are LCM ready.
Also changed selecting default nodes to get deployment info.
now by default will be selected all nodes except nodes, those are marked
for deletion.
Closes-Bug: 1566384
Change-Id: I264bece7da8aebaef78c15bef43c800ca565e583
Now when you run only one task from CLI or API this not starts the whole
deployment
Change-Id: Ibf339eaacee4f5596d3c5338323f79605604ffc1
Related-Bug: #1567112
nailgun.errors have a huge set of exceptions but without hierarchy. This
patch remove exception generation from dict and make it explicitly with
python classes and add some exceptions hierarchy. Now all network errors
inherit from NetworkException and same for other exceptions.
Change-Id: I9a2c6b358ea02a16711da74562308664ad7aed97
Closes-bug: #1566195
The following legacy tasks were reworked to use ClusterTransaction:
- OpenstackConfigTaskManager
- SpawnVMsTaskManager
Change-Id: I4a6f5f37161e4290050ec4926cf029cd7af566e4
Closes-Bug: 1565885
Closes-Bug: 1561994
Closes-Bug: 1565760
From this moment Nailgun aslo do not delete, but mark as deleted
such tasks:
- reset
- network
- stop
- dump
- update_dnsmasq
- capacity_log
It will delete from DB only after 2 actions:
- cluster reset;
- cluster deletion;
Change-Id: I7f1e4c15fba97c5de031e0728f21841eb75861f8
Closes-Bug: #1564966
UI needs to be able to check task status after cluster is removed.
We cannot delete deletion task with rest of the cluster.
Change-Id: Idc87f75123f3bff0745efd4a65f843036c1c4e50
Closes-Bug: 1519446
Now the deployment info is saved only if LCM engine is used,
It should be saved also if legacy serializers are used.
Change-Id: I00d05342cb53f0bad655d3a81995dfb9f6b7252c
Closes-Bug: 1563434
The primary-roles is applied in deployment serializers,
also the role resolver should be created after deployment
serializers have completed.
Change-Id: Idd344739bc65292a88122de0491b2e9f04e1d5d0
Closes-Bug: 1563308
We need separate task serializers for LCM,
because LCM uses context per node for serialize tasks.
It also allows to isolate LCM related code for backward
compatibility with existing environments, that is not ready for LCM.
Change-Id: Ie95a58c8cf86eac1a5c3dbd956fafc401e40fed6
Implements: blueprint computable-task-fields-yaql
Also implemented handlers to get last deployed settings
and networks for the cluster
Change-Id: I1dffada651b71d6e9dac4f02020eb4f900a5e2f4
Implements: blueprint unlock-settings-tab
Implements: blueprint save-deployment-info-in-database
Nailgun integration tests use fake tasks, which run in threads.
This has been causing random failures. Although those failures could
of been fixed by different sorts of improvements, increasing timeouts,
etc. - this is still not an ideal solution.
Proposed patch removes use of threads in tests. All fake tasks code
which used to be running in threads only is ran synchronously for tests.
Threads are still used for fakeUI as before.
More details about random test failures can be found in mailing list:
http://lists.openstack.org/pipermail/openstack-dev/2016-March/089514.html
Change-Id: Iaa5b245680e7257ff46b5ddc1b7aa9400284e705
Andreas Jaeger