Documentation update

Here is a new architecture diagram with some updates on the
glossary and on descriptions of architecture elements.
I also fix some warning logs.

Closes-Bug: #1657405
Change-Id: I442082d702fc8667e9397c090da51ca1ead5d86e

doc/source/architecture.rst

@@ -21,7 +21,7 @@ Overview
 Below you will find a diagram, showing the main components of Watcher:
 
 .. image:: ./images/architecture.svg
-   :width: 100%
+   :width: 110%
 
 
 .. _components_definition:
@@ -37,13 +37,12 @@ AMQP Bus
 The AMQP message bus handles internal asynchronous communications between the
 different Watcher components.
 
-.. _cluster_history_db_definition:
+.. _cluster_datasource_definition:
 
-Cluster History Database
-------------------------
+Datasource
+----------
 
-This component stores the data related to the
-:ref:`Cluster History <cluster_history_definition>`.
+This component stores the metrics related to the cluster.
 
 It can potentially rely on any appropriate storage system (InfluxDB, OpenTSDB,
 MongoDB,...) but will probably be more performant when using
@@ -51,14 +50,6 @@ MongoDB,...) but will probably be more performant when using
 which are optimized for handling time series data, which are arrays of numbers
 indexed by time (a datetime or a datetime range).
 
-.. _cluster_model_db_definition:
-
-Cluster Model Database
-------------------------
-
-This component stores the data related to the
-:ref:`Cluster Data Model <cluster_data_model_definition>`.
-
 .. _archi_watcher_api_definition:
 
 Watcher API
@@ -193,8 +184,8 @@ data:
   :ref:`Managed resources <managed_resource_definition>` (e.g., the data stored
   in the Nova database). These models gives a strategy the ability to reason on
   the current state of a given :ref:`cluster <cluster_definition>`.
-- The data stored in the :ref:`Cluster History Database
-  <cluster_history_db_definition>` which provides information about the past of
+- The data stored in the :ref:`Cluster Datasource
+  <cluster_datasource_definition>` which provides information about the past of
   the :ref:`Cluster <cluster_definition>`.
 
 Here below is a sequence diagram showing how the Decision Engine builds and
@@ -452,6 +443,10 @@ state may be one of the following:
 -  **CANCELLED** : the :ref:`Action Plan <action_plan_definition>` was in
    **RECOMMENDED**, **PENDING** or **ONGOING** state and was cancelled by the
    :ref:`Administrator <administrator_definition>`
+-  **SUPERSEDED** : the :ref:`Action Plan <action_plan_definition>` was in
+   RECOMMENDED state and was automatically superseded by Watcher, due to an
+   expiration delay or an update of the
+   :ref:`Cluster data model <cluster_data_model_definition>`
 
 
 The following diagram shows the different possible states of an

doc/source/dev/plugin/strategy-plugin.rst

@@ -271,57 +271,44 @@ requires new metrics not covered by Ceilometer, you can add them through a
 .. _`Ceilometer plugin`: http://docs.openstack.org/developer/ceilometer/plugins.html
 .. _`Ceilosca`: https://github.com/openstack/monasca-ceilometer/blob/master/ceilosca/ceilometer/storage/impl_monasca.py
 
+Read usage metrics using the Watcher Datasource Helper
+------------------------------------------------------
 
-Read usage metrics using the Python binding
--------------------------------------------
-
-You can find the information about the Ceilometer Python binding on the
-OpenStack `ceilometer client python API documentation
-<http://docs.openstack.org/developer/python-ceilometerclient/api.html>`_
-
-To facilitate the process, Watcher provides the ``osc`` attribute to every
-strategy which includes clients to major OpenStack services, including
-Ceilometer. So to access it within your strategy, you can do the following:
+The following code snippet shows how to invoke a Datasource Helper class:
 
 .. code-block:: py
 
-    # Within your strategy "execute()"
-    cclient = self.osc.ceilometer
-    # TODO: Do something here
+    from watcher.datasource import ceilometer as ceil
+    from watcher.datasource import monasca as mon
+
+    @property
+    def ceilometer(self):
+        if self._ceilometer is None:
+            self._ceilometer = ceil.CeilometerHelper(osc=self.osc)
+        return self._ceilometer
+
+    @property
+    def monasca(self):
+        if self._monasca is None:
+            self._monasca = mon.MonascaHelper(osc=self.osc)
+        return self._monasca
 
 Using that you can now query the values for that specific metric:
 
 .. code-block:: py
 
-    query = None  # e.g. [{'field': 'foo', 'op': 'le', 'value': 34},]
-    value_cpu = cclient.samples.list(
-        meter_name='cpu_util',
-        limit=10, q=query)
-
-
-Read usage metrics using the Watcher Cluster History Helper
------------------------------------------------------------
-
-Here below is the abstract ``BaseClusterHistory`` class of the Helper.
-
-.. autoclass:: watcher.decision_engine.cluster.history.base.BaseClusterHistory
-    :members:
-    :noindex:
-
-The following code snippet shows how to create a Cluster History class:
-
-.. code-block:: py
-
-    from watcher.decision_engine.cluster.history import ceilometer as ceil
-
-    query_history  = ceil.CeilometerClusterHistory()
-
-Using that you can now query the values for that specific metric:
-
-.. code-block:: py
-
-    query_history.statistic_aggregation(resource_id=compute_node.uuid,
-                                  meter_name='compute.node.cpu.percent',
-                                  period="7200",
-                                  aggregate='avg'
-                                  )
+        if self.config.datasource == "ceilometer":
+            resource_id = "%s_%s" % (node.uuid, node.hostname)
+            return self.ceilometer.statistic_aggregation(
+                resource_id=resource_id,
+                meter_name='compute.node.cpu.percent',
+                period="7200",
+                aggregate='avg',
+            )
+        elif self.config.datasource == "monasca":
+            statistics = self.monasca.statistic_aggregation(
+                meter_name='compute.node.cpu.percent',
+                dimensions=dict(hostname=node.uuid),
+                period=7200,
+                aggregate='avg'
+            )

doc/source/glossary.rst

@@ -101,12 +101,6 @@ Cluster Data Model (CDM)
 
 .. watcher-term:: watcher.decision_engine.model.collector.base
 
-.. _cluster_history_definition:
-
-Cluster History
-===============
-
-.. watcher-term:: watcher.decision_engine.cluster.history.base
 
 .. _controller_node_definition:
 

doc/source/image_src/plantuml/action_plan_state_machine.txt

@@ -9,8 +9,10 @@ FAILED --> DELETED : Administrator removes\nAction Plan
 SUCCEEDED --> DELETED : Administrator removes\nAction Plan
 ONGOING --> CANCELLED : Administrator cancels\nAction Plan
 RECOMMENDED --> CANCELLED : Administrator cancels\nAction Plan
+RECOMMENDED --> SUPERSEDED : The Watcher Decision Engine supersedes\nAction Plan
 PENDING --> CANCELLED : Administrator cancels\nAction Plan
 CANCELLED --> DELETED
+SUPERSEDED --> DELETED
 DELETED --> [*]
 
 @enduml

doc/source/image_src/plantuml/watcher_db_schema_diagram.txt

@@ -60,6 +60,7 @@ table(audits) {
     interval : Integer, nullable
     parameters : JSONEncodedDict, nullable
     scope : JSONEncodedList, nullable
+    auto_trigger: Boolean
 
     created_at : DateTime
     updated_at : DateTime
@@ -73,7 +74,6 @@ table(action_plans) {
     foreign_key("audit_id : Integer, nullable")
     foreign_key("strategy_id : Integer")
     uuid : String[36]
-    first_action_id : Integer
     state : String[20], nullable
     global_efficacy : JSONEncodedDict, nullable
 
@@ -91,7 +91,7 @@ table(actions) {
     action_type : String[255]
     input_parameters : JSONEncodedDict, nullable
     state : String[20], nullable
-    next : String[36], nullable
+    parents : JSONEncodedList, nullable
 
     created_at : DateTime
     updated_at : DateTime

watcher/decision_engine/planner/weight.py

@@ -41,7 +41,7 @@ class WeightPlanner(base.BasePlanner):
     *Limitations*
 
     - This planner requires to have action_weights and parallelization configs
-    tuned well.
+      tuned well.
     """
 
     def __init__(self, config):