There is the possibility to change OpenStack config after deployment.
Changes could be applied per role. And in the case of multiple roles
on a node, many changes could pretend to be applied. I.e. if we have
a config for the role 'compute', a config for the role 'cinder' and
a node with role 'cinder+compute' we have to choose one of them.
Previous decision was 'sort it in the lexicographical order', so it
applies 'cinder' then --- 'compute'. It is contra intuitive. The best
option is applying to a node the last related config, to new config
overlaps old ones.
Change-Id: I7db388ca3baeb351adc9fdb70c55b0be50fafe48
Closes-bug: #1671521
This commit switching tasks resolution approach to the tags based one.
Tag - minimal unit what's necessary only for task resolution and can be
mapped to the node through the role interface only. Each role provides set
of tags in its 'tags' field and may be modified via role API. Tag may be
created separately via tag API, but, this tag can not be used unless it's
stuck to the role.
Change-Id: Icd78fd124997c8aafb07964eeb8e0f7dbb1b1cd2
Implements: blueprint role-decomposition
A 'tags' attribute has been added to each role in 'roles_metadata'.
Initially all non-controller roles will only have a tag of their own
role name. This will allow existing tasks which do not have tags
associated with them to work correctly. In the abscence of tags a
task's roles will be used to determine which nodes it will run on.
Implements: blueprint role-decomposition
Change-Id: I390580146048b6e00ec5c42d0adf995a4cff9167
nailgun.errors have a huge set of exceptions but without hierarchy. This
patch remove exception generation from dict and make it explicitly with
python classes and add some exceptions hierarchy. Now all network errors
inherit from NetworkException and same for other exceptions.
Change-Id: I9a2c6b358ea02a16711da74562308664ad7aed97
Closes-bug: #1566195
Now it is possible to update deployment graph name
and tasks keeping the graph database ID intact.
DeploymentGraph.update and DeploymentGraph.create_for_model
methods are inroduced that allow to update or create graph.
Change-Id: I8170569d9283d226f53b5fb2879a5ff9ee7bb984
Implements: blueprint custom-graph-execution
To avoid confusion with objest.DeploymentGraph and
orchestrator.DeploymentGraph second one was renamed
to the GraphSolver because this entity is used to
find subgraphs, loops, and solve graph for the
old orchestrators.
Secondary changes:
DeploymentGraphValidator -> GraphSolverValidator
DeploymentGraphTasksValidator -> GraphSolverTasksValidator
GraphVisualizationValidator -> GraphSolverVisualizationValidator
validators/graph.py -> validators/orchestrator_graph.py
orchestrator/deployment_graph.py -> orchestrator/orchestrator_graph.py
Change-Id: Ic42a17115723b8145b9d94cc034bf2217d1d634b
Implements: blueprint custom-graph-execution
This models store deployment tasks sets that is grouped under
DeploymentGraph record with unique id.
DeploymentGraph is linked via m2m to the:
- releases
- plugins
- clusters
This transitive tables containing 'type' field, that should be used as
graphs merge key in business logic.
DocImpact
Implements: blueprint custom-graph-execution
Change-Id: If53cb58e8ae4cf8fe4caa470bf207ff1683b1cda
Serializer used incorrect logic to merge configuration
dictionaries. It did merging on the first nesting level
while it is required to do it on the second.
Change-Id: I1ef13bc80098d6e9b1f450ea35d4ec8b32d6171d
Closes-Bug: #1524284
ironic_upload_images should execute once on primary controller
and should not execute on new nodes added to cluster.
Change-Id: Ic19f2ee13ec022a708002c0508ac3596033786d3
Closes-bug: #1522792
Multiple configurations for node roles are merged
into one single configuration during serialization process.
It is required to keep the same order of configurations
to have a predictable behavior.
Change-Id: I649cccc73ae406a1f6953ad8139910bb18acc95f
Closes-Bug: #1524284
Now in case when type of task is not known to Nailgun proper error is
thrown.
Change-Id: I33923c04e3eba0ed8028efac51e36f347296d0ae
Closes-Bug: #1506915
Support {CLUSTER_ID} interpolation for 2 new tasks:
- copy_keys_ceph
- generate_keys_ceph
Change-Id: I4f14d19afd522ee2db51fdba34ef9ee662c28af7
Closes-Bug: #1478076
In Fuel 7.0 we want to have ability to extend node roles on fly by using
plugins. In order to achieve this, we want to have declarative
schemaless approach, so we won't require database object manipulations
(CRUD) each time we somehow change node role.
This commit drops such tables like:
* roles
* pending_roles
* node_roles
and use postgresql arrays in nodes tables instead:
* roles
* pending_roles
* primary_roles
Implements blueprint role-as-a-plugin
Change-Id: I2d3fccb7187fa4e82595c9ca6b27ef8ce90ee4b2
Signed-off-by: Igor Kalnitsky <igor@kalnitsky.org>
We need to pass CLUSTER_ID to two new astute prehook tasks:
- generate_haproxy_keys
- copy_haproxy_keys
Implements: blueprint ssl-endpoints
Change-Id: I2261dcfb55743b0fb31f4cbe42e7793176c8e1c0
nailgun.orchestrator.deployment_serializers has more than 1800 lines of code.
This makes it hard to read and update.
Change-Id: I1d1fa22899009eaef811b1de9ad897d065a47cd4
Implements: blueprint granular-network-functions
If nodes are not ready or selected for deployemnt post_deployemnt
task for uploading nodes info and updating hosts should not be
executed on them.
Also fixed PEP8 warnings in files I've touched.
Change-Id: Icf2c51ec15f4028ff4da74a633481ca154e42d0a
Closes-Bug: #1450051
* Create /etc/apt/apt.conf.d/05disable-ipv6 and set Acquire::ForceIPv4 "true";
to disable IPv6 as Ubuntu DNS returns IPv6 sometimes.
* Delete the same part from fuel_agent_ci cloud-init template as it should be
configured by Orchestrator
Change-Id: I90fe58157c3eac04842102b1cb956bbeb324413a
Closes-Bug: 1450004
Signed-off-by: Sergii Golovatiuk <sgolovatiuk@mirantis.com>
Current approach is an error prone, because Debian repos may contain
metadata that doesn't fit our pinning template. Yes, in some cases
our template will work (and it does with upstream Ubuntu and Fuel),
but we want to be flexible and cover all cases.
In order to set corrent pinning, we have to use repo's metadata and
that means download repo's Release file, parse it and create pinning
based on it. In that case we'll use all available pinning rules, so
all possible issues will be up to repo maintainer and not because
we're using error prone approach.
Related-Bug: #1435892
Related-Bug: #1438783
Closes-Bug: #1446686
Change-Id: I5279b7f414f831503fd993109dd5c91bef51a9ef
Signed-off-by: Igor Kalnitsky <igor@kalnitsky.org>
We used to remove /etc/apt/sources.list, but the 'command-not-found'
tool requires this file and if it's absent prints the warning. In
order to avoid this irritating message, we're going to clear this
file, so everyone will be happy.
Change-Id: I452a0350f3f98794a9ccdc7a963cb4a0878b87c2
Closes-Bug: #1439670
According to bp/separate-mos-from-linux we shouldn't set priority for
the upstream Ubuntu repos and should let them to be resolved by APT
defaults.
Taking into account this fact we should introduce a new state for the
'priority' attribute - the 'null' state, which means do not set any
priority at all.
Implements: blueprint consume-external-ubuntu
Implements: blueprint separate-mos-from-linux
Change-Id: Ib5c59b743effa9256be7e4a0f1ccd071343ec1c6
Since we want to consume external Ubuntu, we should point to official
repos by default. Obviously, user will be able to change it on Settings
tab.
Implements: blueprint consume-external-ubuntu
Change-Id: I1666803b91d86247ac13066fc42ae12fa7377dbb
We need it to be possible to set priorities (pinning)
for deb and rpm repositories. This patch moves repo_metadata
to cluster attributes and changes its format.
Plugins' repos should have higher priority than default ones,
so we're setting them according to settings.yaml.
Implements: blueprint consume-external-ubuntu
Change-Id: I5a98469b9547014abec9406a7b36f1cf22ba8086
This patch introduces changes that necesarry to join all stages
into one graph.
<stage>_start is root for stage entity
<stage>_end is end task in stage entity
That is done to ensure that each stage is self contained,
and for example in case of adding more stages there will be no need to
look for end tasks and connect them
The only two harcdoded parameters are deploy_start and deploy_end,
stuff that we are including in pre_deployment is everything that comes before
deploy_start, and for post_deployment after deploy_end
related to blueprint granular-deployment-based-on-tasks
Change-Id: Ida5880525f9c3428ad8bb590f7ef78a0b20b1049
* added serializer that adds nodes info that will be uploaded
* replaced upload_repo_task with upload_file_task
* removed 'create_serializer' function and replaced with
'get_serializer_for_cluster'
Change-Id: Ia6fdcbd57fa35f0fe7655ab7e3b96dab0849d191
Implements: blueprint granular-deployment-based-on-tasks
In 6.0 we had plugins that are using role: [controller]
to execute tasks on all controllers.
We need to change this behaviour because we need to be able
to differentiate between primary controller and other controllers
on orchestration layer
Also refactored tests to use real objects instead of Mock
because mocking wasnt flexible enough
DocImpact
related to blueprint granular-deployment-based-on-tasks
Change-Id: I3087ea70fd5dc9d3486fad8aae337cdcd7c8cfef
Added several examples of conditional tasks based on exsiting
expression parser. Currently only 2 models supported:
- cluster - cluster db model
- settings - cluster.attributes.editable field
implements blueprint granular-deployment-based-on-tasks
Change-Id: Idff1644c9dcb7f217d9e8dbcf2a4ba4b9f0cb2d4
Restart radowsgw task requires condition - that this task
should be executed only on controllers in case there was
nodes with ceph-osd role in current deployment
implements blueprint granular-deployment-based-on-tasks
Change-Id: Ifbf22b684978bbf1c5b211166f8dc6d04a21fbbf
Implemented backbone for pre/post task serializers,
with few examples for pre task serializers
- Rsync puppet
uses mask in arguments, that is processed in serializer
- Upload repos
For given task in config - serializes number of tasks,
one task for each repo in repo_metadata, and one task for
regenerating metadata after repo was added
implements blueprint granular-deployment-based-on-tasks
Change-Id: I3baf1b06b7c0fa9d712edff5283c9854a7b26431
Andreas Jaeger