Support Vitrage resources in Heat

The purpose is to automate the auto-healing process that
involves external monitoring, Vitrage alarm deduction and
Mistral workflow execution.

Story: 2002684
Task: 22527

Depends-On: Ie28ba2087c6d87aec57198afe9c328542a4c25ca
Change-Id: If66248e07a662a225799a2bd3fc88a31d1539021
This commit is contained in:
Ifat Afek 2018-06-28 12:37:46 +00:00
parent 8e865c99e9
commit 7888aecf10
1 changed files with 141 additions and 0 deletions

View File

@ -0,0 +1,141 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
=========================
Support Vitrage Resources
=========================
https://storyboard.openstack.org/#!/story/2002684
This Blueprint proposes to add support for Vitrage resources in Heat.
The purpose is to automate the auto-healing process that involves external
monitoring, Vitrage alarm deduction and Mistral workflow execution.
Problem description
===================
Auto-healing a Heat stack when an instance is down is extremely important.
This use case is already handled when Nova sends a notification about the
instance state, Aodh raises an event alarm and as a result a Mistral healing
workflow is executed.
However, there are cases where Nova is not aware about the real state of the
instance. One example is a network failure: a NIC that is down can result in no
network connectivity to certain instances, while their state in Nova remains
'Active'. We would like to support auto-healing in such cases as well.
Proposed change
===============
An OS::Vitrage::Template resource will be added in Heat, under
heat/engine/resources/openstack/vitrage.
Its role will be to create, based on the properties given in HOT template,
a Vitrage template with a condition->action scenario that will handle the
healing.
The VitrageTemplate resource will support the following use case:
#. An external monitor detects a network failure
#. Vitrage is notified, and based on its topology-graph it identifies all
affected resources
#. If an instance that belongs to a Heat stack is affected, Vitrage executes
a Mistral healing workflow
The VitrageTemplate definition:
.. code-block:: yaml
resources:
name:
type: OS::Vitrage::Template
properties:
description: String
alarm_name: String
stack_id: String
workflow_id: String
workflow_input: {...}
Properties:
- description - Description of the Vitrage template
- alarm_name - The name of the alarm that should trigger the workflow
execution. This can be an alarm from an external monitor (like Zabbix,
Nagios, Collectd, or Prometheus), or a deduced alarm that was raised by
Vitrage.
- workflow_id - Id of the Mistral workflow to be executed
- workflow_input - values to be passed as inputs to the workflow
Example:
.. code-block:: yaml
resources:
execute_healing:
type: OS::Vitrage::Template
properties:
description: Execute Mistral healing workflow if the instance is down
alarm_name: Instance down
stack_id: {get_param: "OS::stack_id"}
workflow_id: auto_heal
workflow_input:
heat_stack_id: {get_param: "OS::stack_id"}
The implementation will be done in two phases.
**Phase 1:** A simple Vitrage template will be created with and will support
the following scenario:
If an alarm named input_alarm_name is raised **on an instance that is part of
the given Heat stack** -> execute the Mistral workflow
**Phase 2:** We should enable creation of more complex Vitrage templates, like
the one in the network failure use case. This will require additional
development in Vitrage, so it will provide "template skeletons" for different
scenarios.
Alternatives
------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
ifat_afek
Milestones
----------
Target Milestone for completion:
rocky-3
Work Items
----------
Phase 1:
- Implement the VitrageTemplate resource
- Add unit tests and tempest tests
- Add a HOT template example to heat-templates
Phase 2 (future):
- Create different types of Vitrage templates
Dependencies
============
No dependencies for phase 1.
Phase 2 depends on future Vitrage development.