Support Vitrage resources in Heat
The purpose is to automate the auto-healing process that involves external monitoring, Vitrage alarm deduction and Mistral workflow execution. Story: 2002684 Task: 22527 Depends-On: Ie28ba2087c6d87aec57198afe9c328542a4c25ca Change-Id: If66248e07a662a225799a2bd3fc88a31d1539021
This commit is contained in:
parent
8e865c99e9
commit
7888aecf10
|
@ -0,0 +1,141 @@
|
|||
..
|
||||
This work is licensed under a Creative Commons Attribution 3.0 Unported
|
||||
License.
|
||||
|
||||
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||
|
||||
=========================
|
||||
Support Vitrage Resources
|
||||
=========================
|
||||
|
||||
https://storyboard.openstack.org/#!/story/2002684
|
||||
|
||||
This Blueprint proposes to add support for Vitrage resources in Heat.
|
||||
The purpose is to automate the auto-healing process that involves external
|
||||
monitoring, Vitrage alarm deduction and Mistral workflow execution.
|
||||
|
||||
Problem description
|
||||
===================
|
||||
|
||||
Auto-healing a Heat stack when an instance is down is extremely important.
|
||||
This use case is already handled when Nova sends a notification about the
|
||||
instance state, Aodh raises an event alarm and as a result a Mistral healing
|
||||
workflow is executed.
|
||||
|
||||
However, there are cases where Nova is not aware about the real state of the
|
||||
instance. One example is a network failure: a NIC that is down can result in no
|
||||
network connectivity to certain instances, while their state in Nova remains
|
||||
'Active'. We would like to support auto-healing in such cases as well.
|
||||
|
||||
Proposed change
|
||||
===============
|
||||
|
||||
An OS::Vitrage::Template resource will be added in Heat, under
|
||||
heat/engine/resources/openstack/vitrage.
|
||||
|
||||
Its role will be to create, based on the properties given in HOT template,
|
||||
a Vitrage template with a condition->action scenario that will handle the
|
||||
healing.
|
||||
|
||||
The VitrageTemplate resource will support the following use case:
|
||||
|
||||
#. An external monitor detects a network failure
|
||||
#. Vitrage is notified, and based on its topology-graph it identifies all
|
||||
affected resources
|
||||
#. If an instance that belongs to a Heat stack is affected, Vitrage executes
|
||||
a Mistral healing workflow
|
||||
|
||||
The VitrageTemplate definition:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
resources:
|
||||
name:
|
||||
type: OS::Vitrage::Template
|
||||
properties:
|
||||
description: String
|
||||
alarm_name: String
|
||||
stack_id: String
|
||||
workflow_id: String
|
||||
workflow_input: {...}
|
||||
|
||||
|
||||
Properties:
|
||||
|
||||
- description - Description of the Vitrage template
|
||||
- alarm_name - The name of the alarm that should trigger the workflow
|
||||
execution. This can be an alarm from an external monitor (like Zabbix,
|
||||
Nagios, Collectd, or Prometheus), or a deduced alarm that was raised by
|
||||
Vitrage.
|
||||
- workflow_id - Id of the Mistral workflow to be executed
|
||||
- workflow_input - values to be passed as inputs to the workflow
|
||||
|
||||
|
||||
Example:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
resources:
|
||||
execute_healing:
|
||||
type: OS::Vitrage::Template
|
||||
properties:
|
||||
description: Execute Mistral healing workflow if the instance is down
|
||||
alarm_name: Instance down
|
||||
stack_id: {get_param: "OS::stack_id"}
|
||||
workflow_id: auto_heal
|
||||
workflow_input:
|
||||
heat_stack_id: {get_param: "OS::stack_id"}
|
||||
|
||||
|
||||
The implementation will be done in two phases.
|
||||
|
||||
**Phase 1:** A simple Vitrage template will be created with and will support
|
||||
the following scenario:
|
||||
|
||||
If an alarm named input_alarm_name is raised **on an instance that is part of
|
||||
the given Heat stack** -> execute the Mistral workflow
|
||||
|
||||
**Phase 2:** We should enable creation of more complex Vitrage templates, like
|
||||
the one in the network failure use case. This will require additional
|
||||
development in Vitrage, so it will provide "template skeletons" for different
|
||||
scenarios.
|
||||
|
||||
|
||||
Alternatives
|
||||
------------
|
||||
|
||||
None
|
||||
|
||||
Implementation
|
||||
==============
|
||||
|
||||
Assignee(s)
|
||||
-----------
|
||||
|
||||
Primary assignee:
|
||||
ifat_afek
|
||||
|
||||
Milestones
|
||||
----------
|
||||
|
||||
Target Milestone for completion:
|
||||
rocky-3
|
||||
|
||||
Work Items
|
||||
----------
|
||||
|
||||
Phase 1:
|
||||
|
||||
- Implement the VitrageTemplate resource
|
||||
- Add unit tests and tempest tests
|
||||
- Add a HOT template example to heat-templates
|
||||
|
||||
Phase 2 (future):
|
||||
|
||||
- Create different types of Vitrage templates
|
||||
|
||||
Dependencies
|
||||
============
|
||||
|
||||
No dependencies for phase 1.
|
||||
Phase 2 depends on future Vitrage development.
|
Loading…
Reference in New Issue