Merge "blueprint: puppet-noop-run"

This commit is contained in:
Jenkins 2016-08-24 17:01:45 +00:00 committed by Gerrit Code Review
commit f6392801da
1 changed files with 286 additions and 0 deletions

View File

@ -0,0 +1,286 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
================================================
Puppet noop run for Fuel puppet deployment tasks
================================================
https://blueprints.launchpad.net/fuel/+spec/puppet-noop-run
--------------------
Problem description
--------------------
Currently, Fuel Environment re-deployment re-runs all Fuel tasks without any
check of customizations which could be applied for different OpenStack and
Fuel components aka files and config values changes, running and stopped
services and etc. If such changes weren't applied to Fuel deployment tasks
(manifests, scripts) as well (that the most frequent case for the users)
new tasks run (in case of re-deployment or update) could lead to losing of
applied customization.
----------------
Proposed changes
----------------
Before re-deployment or update run of successfully deployed cluster it should
be possible to get a report about those customizations which were applied to
the Fuel cluster or particular node which differs from performed previously
deployment and which could be possible overridden by new Fuel tasks run. Those
customizations should be stored in the report file or database in readable
(or parsable format) and it should be possible to get it using REST API.
Noop run for Fuel tasks could be used as a mechanism for detecting of any set
of customizations applied to the services, configuration files and etc in the
cluster. Tasks noop run is able to show changes in the metaparameters for
files (e.g. owner, mode, content), services (e.g. status, service provider),
OpenStack configs (e.g. missed options, incorrect values for options) and
other (even custom) resources. This approach could be easily implemented
for all types of tasks: Puppet tasks could be executed with '--noop' option,
other types could be just skipped.
Specifically Puppet Noop run will help to detect required types of
customizations in Fuel environment. Puppet store report of each run (even noop
run) in /var/lib/puppet/reports/<node-fqdn>/ folder in YAML format. Each puppet
operation is tracked here and it has detailed description. The most important
information is: was resource changed or not? This is shown by 'changed'
parameter. So every puppet operation could be easily checked by status. Another
aprroach here is to generate Puppet report in JSON format. For enabling of
this feature is required to add '--logdest /path/to/file.json' to the end of
puppet apply command. In that case it's possible to store a report for all
Fuel puppet tasks in one file or separate a report per running task.
The implementation of this approach requires changes in the Fuel:
* Puppet tasks: all Fuel puppet tasks should support noop action. Some tasks
may error with '--noop' option. Such failures will be stored in report but
they won't stop Tasks noop run. They also won't affect cluster/node status.
* Astute: will be created additional task types for noop run; those types
will adapt all existing tasks to run them in noop mode: in case of puppet
and shell types tasks will be executed in noop mode run, in case of other
types those tasks will be skipped. Logging output for noop tasks run will
be reduced: debug and verbose options for Puppet noop run will not be used.
* Task history: Noop run report should be stored in deployment tasks history.
* Nailgun: Noop run report should be available through nailgun API for each
particular node in environment.
* Fuel CLI: it should be possible to run any custom graph for particular
environment or node with Noop option.
This Noop run for the any cluster or set of nodes shouldn't change their
statuses. Noop run is not a part of deployment. It should work similar
to addional checks (like OSTF is working).
Web UI
======
None
Nailgun
=======
* Nailgun API should handle noop_run parameter from request to deploy,
re-deploy or execute graph of tasks for the cluster. If 'noop_run' is set,
nailgun should execute requested actions as Noop run for the cluster.
* Nailgun shouldn't change cluster state (e.g. deployed -> deploying) during
and after Noop run even if it has failed.
Data model
----------
None
REST API
--------
Described in Nailgun section.
Orchestration
=============
RPC Protocol
------------
'Noop_run' flag will be added to RPC protocol. The end message sent by Nailgun
should look like:
.. code-block:: json
{
"api_version": "1",
"method": "task_deploy",
"respond_to": "transaction_resp",
"args": {
"task_uuid": "10",
"tasks_graph": "cluster_graph",
...,
"dry_run": false,
"noop_run": true,
}
}
Fuel Client
===========
Fuel client should support following Noop actions:
* Run any graph with a 'noop' option which would ask nailgun to format
a message to Astute properly, so that Astute runs only 'noop' tasks.
* Start Noop run for particular environment, node, task or
set of tasks (custom graph).
* Get report from each Noop run.
Plugins
=======
Fuel Puppet tasks in plugins should also support Puppet noop run with new
log destination.
Fuel Library
============
None
------------
Alternatives
------------
Manual detect of customizations applied to the clusrer.
--------------
Upgrade impact
--------------
None
---------------
Security impact
---------------
None
--------------------
Notifications impact
--------------------
None
---------------
End user impact
---------------
End users will be able to check their environment for customizations before
cluster re-deployment, update or upgrade. They will be notified about the
differences between current cluster/nodes state and original (after last
deployment). It will help to reduce the risk of missing important
customizations applied to cluster/nodes.
------------------
Performance impact
------------------
None
-----------------
Deployment impact
-----------------
None
----------------
Developer impact
----------------
None
---------------------
Infrastructure impact
---------------------
None
--------------------
Documentation impact
--------------------
Documentation will have to be updated to reflect changes.
--------------
Implementation
--------------
Assignee(s)
===========
Primary assignee:
Denis Egorenko
Other contributors:
Ivan Berezovskiy
Mandatory design review:
Vladimir Kuklin
Vladimir Sharshov
QA engineer:
Timur Nurlygayanov
Work Items
==========
* Update Fuel Astute to support Noop run for all type of tasks.
* Add support for keeping Puppet Noop run report in parsable format
(YAML or JSON) and make it available to download through API call or using
Fuel client.
* Update Fuel client to be able to apply custom graph on particular environment
or set of nodes with Noop option.
* Update Nailgun to ignore Noop run errors. They shouldn't affect cluster or node
state/status.
Dependencies
============
None
------------
Testing, QA
------------
* Nailgun's unit and integration tests will be extended to test new feature.
* Astute's unit and integration tests will be extended to test new feature.
* Fuel Client's unit and integration tests will be extended to test new feature.
Acceptance criteria
===================
* Noop run should be possible to execute on only successfully deployed
environment.
* It should be possible check custom changes in services, files, OpenStack
components configuration and other puppet resources applied to cluster or
particular node using simple command of Fuel client.
* It should be possible to get report of Noop run using REST API.
* Noop run shouldn't affect cluster deployment status.
----------
References
----------
1. LP Blueprint https://blueprints.launchpad.net/fuel/+spec/puppet-noop-run